CephNotes

Some notes about Ceph
Laurent Barbe @Adelius / INRAE

Ceph osd reweight

ceph health
HEALTH_WARN 1 near full osd(s)

Arrhh, Trying to optimize a little weight given to the OSD. Rebalancing load between osd seems to be easy, but do not always go as we would like...

Increase osd weight

Before operation get the map of Placement Groups.

$ ceph pg dump …

Using ceph rbd with libvirt on Debian Wheezy

Howto add support of rbd device on debian wheezy.

libvirt

Since wheezy, libvirt supports rbd device.

qemu-kvm

If you do not add the support in qemu-kvm you may have an error like this:

error: Failed to start domain ubuntu
error: internal error process exited while connecting to monitor: char device …

Object difference between RBD format1 and format2

Lets take a look how rbd object are store on rados and the difference between format 1 and format 2.

Format 1

$ rbd create myrbd --size=10 
$ rados ls -p rbd      
myrbd.rbd
rbd_directory

$ rbd map myrbd
$ dd if=/dev/zero of=/dev/rbd/rbd/myrbd
$ rados ls -p rbd …

Mon failed to start

Some common problems when adding a monitor to an existing cluster, for example if config is not found :

 $ service ceph start mon.ceph-03
 /etc/init.d/ceph: mon.ceph-03 not found (/etc/ceph/ceph.conf defines osd.2 , /var/lib/ceph defines osd.2)

If you do not …

RBD Image Real Size

[update] Since Ceph octopus (v15.2)

Use command "rbd du"

$ rbd du rbd/myrbd
NAME                   PROVISIONED  USED
myrbd                        4 TiB  1023 GiB

For previous version

To get the real size used by a rbd image :

rbd diff $POOL/$IMAGE | awk '{ SUM += $2 } END { print SUM/1024/1024 " MB" }'

For exemple …