CephNotes

Some notes about Ceph
Laurent Barbe @CCM Benchmark

Change log level on the fly to Ceph daemons

Aaahhh full disk this morning. Sometimes the logs can go crazy, and the files can quickly reach several gigabytes.

Show debug option (on host) :

# Look at log file
tail -n 1000 /var/log/ceph/ceph-osd.33.log

# Check debug levels
ceph daemon osd.33 config show | grep '"debug_'
    "debug_none …

Main new features in the latest versions of ceph

It's always pleasant to see how fast new features appear in Ceph. :)

Here is a non-exhaustive list of some of theme on the latest releases :

Kraken (October 2016)

  • BlueStore declared as stable
  • AsyncMessenger
  • RGW : metadata indexing via Elasticseasrch, index resharding, compression
  • S3 bucket lifecycle API, RGW Export NFS version 3 …

Check OSD version

Occasionally it may be useful to check the version of the OSD on the entire cluster :

ceph tell osd.* version

osd.0: {
    "version": "ceph version 0.94.5-224-g4051bc2 (4051bc2a5e4313ac0f6236d7a34ed5fb4a1d9ea2)"
}
osd.1: {
    "version": "ceph version 0.94.5-224-g4051bc2 (4051bc2a5e4313ac0f6236d7a34ed5fb4a1d9ea2)"
}
osd.2: {
    "version": "ceph version 0.94.5-224-g4051bc2 (4051bc2a5e4313ac0f6236d7a34ed5fb4a1d9ea2)"
}
osd.3: {
    "version …

Find the OSD location

Of course, the simplest way is using the command ceph osd tree.

Note that, if an osd is down, you can see "last address" in ceph health detail :

$ ceph health detail
...
osd.37 is down since epoch 16952, last address 172.16.4.68:6804/628

Also, you can use …

LXC 2.0.0 First support for Ceph RBD

FYI, the first RBD support has been added to LXC commands.

Example :

# Install LXC 2.0.0 (ubuntu) :
$ add-apt-repository ppa:ubuntu-lxc/lxc-stable
$ apt-get update
$ apt-get install lxc

# Add a ceph pool for lxc bloc devices :
$ ceph osd pool create lxc 64 64

# To create the container, you only need to …