Lets take a look how rbd object are store on rados and the difference between format 1 and format 2.

Format 1

$ rbd create myrbd --size=10 
$ rados ls -p rbd      
myrbd.rbd
rbd_directory

$ rbd map myrbd
$ dd if=/dev/zero of=/dev/rbd/rbd/myrbd
$ rados ls -p rbd …

Some common problems when adding a monitor to an existing cluster, for example if config is not found :

 $ service ceph start mon.ceph-03
 /etc/init.d/ceph: mon.ceph-03 not found (/etc/ceph/ceph.conf defines osd.2 , /var/lib/ceph defines osd.2)

If you do not …

To get the real size used by a rbd image :

rbd diff $POOL/$IMAGE | awk '{ SUM += $2 } END { print SUM/1024/1024 " MB" }'

For exemple :

$rbd info myrbd
rbd image 'myrbd':
    size 2048 MB in 512 objects
    order 22 (4096 KB objects)
    block_name_prefix: rb.0.2c6a.238e1f29
    format …

To verify the integrity of data, Ceph uses a mechanism called deep scrubbing which browse all your data once per week for each placement group. This can be the cause of overload when all osd running deep scrubbing at the same time.

You can easly see if a deep scrub …

The purpose is to verify where my data is stored on the Ceph cluster.

For this, I have just create a minimal cluster with 3 osd :

$ ceph-deploy osd create ceph-01:/dev/sdb ceph-02:/dev/sdb ceph-03:/dev/sdb

Where is my osd directory on ceph-01 ?

$ mount | grep ceph
/dev/sdb1 …