Howto add support of rbd device on debian wheezy.
libvirt
Since wheezy, libvirt supports rbd device.
qemu-kvm
If you do not add the support in qemu-kvm you may have an error like this:
error: Failed to start domain ubuntu
error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/2
kvm: -drive file=rbd:vmimages/ubuntu:id=vmimages:key=AQAfwTFScNKmLxAAo9abfSLct78rfIQRzQnS5A==:auth_supported=cephx\;none:mon_host=192.168.0.100\:6789\;192.168.0.101\:6789\;192.168.0.102\:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writeback: could not open disk image rbd:vmimages/ubuntu:id=vmimages:key=AQAfwTFScNKmLxAAo9abfSLct78rfIQRzQnS5A==:auth_supported=cephx\;none:mon_host=192.168.0.100\:6789\;192.168.0.101\:6789\;192.168.0.102\:6789: No such file or directory
install ceph dependencies
apt-get install lsb-release
wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add -
echo deb http://ceph.com/debian-firefly/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list
apt-get update
apt-get install librbd-dev librados-dev build-essential
build kvm
mkdir -p sources/qemu-kvm
cd sources/qemu-kvm
apt-get source qemu-kvm
apt-get build-dep qemu-kvm
cd qemu-kvm-1.1.2+dfsg/
vim debian/optional-features
Uncomment line # --enable-rbd
and compile :
dpkg-buildpackage -rfakeroot -b
cd ..
Installation :
apt-get install librados2 librbd1
dpkg -i kvm_1.1.2+dfsg-6_amd64.deb qemu-kvm_1.1.2+dfsg-6_amd64.deb
Becareful on upgrade ! You can also hold package version :
echo "kvm hold" | dpkg --set-selections
echo "qemu-kvm hold" | dpkg --set-selections
qemu (for util like qemu-img)
(Only if needed) You also need to add support in debian rules : "--enable-rbd"
mkdir -p sources/qemu
cd sources/qemu
apt-get source qemu
apt-get build-dep qemu
cd qemu-1.1.2+dfsg/
vim debian/rules +82
dpkg-buildpackage -rfakeroot -b
cd ..
Use
Create a pool and associated key on ceph admin host :
ceph osd pool create vmimages 100 100
ceph auth get-or-create client.vmimages mon 'allow r' osd 'allow rwx pool=vmimages'
keep the key returned.
Create a secret file for libvirt for this user :
echo "
<secret ephemeral='no' private='no'>
<usage type='ceph'>
<name>client.vmimages secret</name>
</usage>
</secret>" > secret.xml
virsh secret-define --file secret.xml
Define the key value :
first uuid is the value return by the previous command
follow by the key return by ceph auth get-or-create
virsh secret-set-value 76e3a541-b997-58ac-f7bd-77dd7d4347cb AQAREH1QkNDNCBaac03ZICi/CePnRDS+vGyrqQ==
Create a device:
(on ceph admin host if you do not want to specify the user, the key, and monitors)
with rbd : rbd create vmimages/ubuntu-newdrive --size=2048
with qemu (you need to add support for that) : qemu-img create -f rbd rbd:vmimages/ubuntu-newdrive 2G
Attach on existing host (even if running) :
(192.168.0.100, 192.168.0.101, 192.168.0.102 is the mon host.)
echo "
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<auth username='vmimages'>
<secret type='ceph' uuid='76e3a541-b997-58ac-f7bd-77dd7d4347cb'/>
</auth>
<source protocol='rbd' name='vmimages/ubuntu-newdrive'>
<host name='192.168.0.100' port='6789'/>
<host name='192.168.0.101' port='6789'/>
<host name='192.168.0.102' port='6789'/>
</source>
<target dev='vdz' bus='virtio'/>
</disk>
" > device.xml
virsh attach-device ubuntu device.xml --persistent