Update 2021: Since Pacific version, there is a specific operating mode for the monitors in case of an stretched cluster. See : https://docs.ceph.com/en/latest/rados/operations/stretch-mode/ For more informations, check out Greg's talk on Fostem

An example of crushmap for 2 Datacenter replication with 2 or …

Aaahhh full disk this morning. Sometimes the logs can go crazy, and the files can quickly reach several gigabytes.

Show debug option (on host) :

# Look at log file
tail -n 1000 /var/log/ceph/ceph-osd.33.log

# Check debug levels
ceph daemon osd.33 config show | grep '"debug_'
    "debug_none …

It's always pleasant to see how fast new features appear in Ceph. :)

Here is a non-exhaustive list of some of theme on the latest releases :

Kraken (October 2016)

• BlueStore declared as stable
• AsyncMessenger
• RGW : metadata indexing via Elasticseasrch, index resharding, compression
• S3 bucket lifecycle API, RGW Export NFS version 3 …

Occasionally it may be useful to check the version of the OSD on the entire cluster :

ceph tell osd.* version

osd.0: {
    "version": "ceph version 0.94.5-224-g4051bc2 (4051bc2a5e4313ac0f6236d7a34ed5fb4a1d9ea2)"
}
osd.1: {
    "version": "ceph version 0.94.5-224-g4051bc2 (4051bc2a5e4313ac0f6236d7a34ed5fb4a1d9ea2)"
}
osd.2: {
    "version": "ceph version 0.94.5-224-g4051bc2 (4051bc2a5e4313ac0f6236d7a34ed5fb4a1d9ea2)"
}
osd.3: {
    "version …

Of course, the simplest way is using the command ceph osd tree.

Note that, if an osd is down, you can see "last address" in ceph health detail :

$ ceph health detail
...
osd.37 is down since epoch 16952, last address 172.16.4.68:6804/628

Also, you can use …