Howto add support of rbd device on debian wheezy.

libvirt

Since wheezy, libvirt supports rbd device.

qemu-kvm

If you do not add the support in qemu-kvm you may have an error like this:

error: Failed to start domain ubuntu
error: internal error process exited while connecting to monitor: char device …

Lets take a look how rbd object are store on rados and the difference between format 1 and format 2.

Format 1

$ rbd create myrbd --size=10 
$ rados ls -p rbd      
myrbd.rbd
rbd_directory

$ rbd map myrbd
$ dd if=/dev/zero of=/dev/rbd/rbd/myrbd
$ rados ls -p rbd …

Some common problems when adding a monitor to an existing cluster, for example if config is not found :

 $ service ceph start mon.ceph-03
 /etc/init.d/ceph: mon.ceph-03 not found (/etc/ceph/ceph.conf defines osd.2 , /var/lib/ceph defines osd.2)

If you do not …

[update] Since Ceph octopus (v15.2)

Use command "rbd du"

$ rbd du rbd/myrbd
NAME                   PROVISIONED  USED
myrbd                        4 TiB  1023 GiB

For previous version

To get the real size used by a rbd image :

rbd diff $POOL/$IMAGE | awk '{ SUM += $2 } END { print SUM/1024/1024 " MB" }'

For exemple …

To verify the integrity of data, Ceph uses a mechanism called deep scrubbing which browse all your data once per week for each placement group. This can be the cause of overload when all osd running deep scrubbing at the same time.

You can easly see if a deep scrub …