Create a disk with an insane size can be fun, but a little hard to remove.
Here's a little trick (use with caution!) to remove a too big image for rm rbd (if image is not initialized, or not fully).
Image format 1 :
$ rbd info rbdbigsize
rbd image 'rbdbigsize':
size 1862 PB in 500000000000 objects
order 22 (4096 kB objects)
block_name_prefix: rb.0.47aa.74b0dc51
format: 1
Remove the header file :
$ rados -p rbd rm rbdbigsize.rbd
Remove all rbd data : (This may take some time depending on the space already used on the blockdevice.)
$ rados -p rbd ls | grep '^rb.0.47aa.74b0dc51' | xargs -n 200 rados -p rbd rm
Finally, remove from rbd list :
$ rbd rm rbdbigsize
2014-06-12 16:27:20.838596 7fd86dff3760 -1 librbd::ImageCtx: error finding header: (2) No such file or directory
Removing image: 100% complete...done.
Image format 2 :
$ rbd info rbdbigsize
rbd image 'rbdbigsize':
size 1024 TB in 268435456 objects
order 22 (4096 kB objects)
block_name_prefix: rbd_data.81d12ae8944a
format: 2
features: layering
Remove header
$ rados -p rbd rm rbd_id.rbdbigsize
$ rados -p rbd rm rbd_header.81d12ae8944a
Remove all rbd data :
$ rados -p rbd ls | grep '^rbd_data.81d12ae8944a.' | xargs -n 200 rados -p rbd rm
Remove from rbd list :
$ rbd rm rbdbigsize
2014-06-12 17:37:24.483587 7f413506c760 -1 librbd::ImageCtx: error finding header: (2) No such file or directory
Removing image: 100% complete...done.
Time comparison :
Example of delete time for blockdevice of 1PB just formatted.
$ rbd create rbdbigsize --size=1073741824 --image-format=2
$ rbd map rbdbigsize
$ mkfs.xfs /dev/rbd0
$ rbd unmap /dev/rbd0
$ rados -p rbd ls | grep '^rbd_data.81d12ae8944a.' | wc -l
1536
The block device is composed solely of 1536 block and can be quickly removed via rados rm :
$ time rados -p rbd ls | grep '^rbd_data.81d12ae8944a.' | xargs -n 200 rados -p rbd rm
real 0m10.073s
user 0m1.244s
sys 0m0.228s
To compare:
$ time rbd rm rbdbigsize
Removing image: 100% complete...done.
real 1123m5.409s
user 686m19.924s
sys 229m56.596s
Fully initialized image :
Of course, deleting a fully initialized image is faster with simple "rbd rm". :)
$ rbd create rbdbigsize --size 1048576
$ rbd map rbdbigsize; dd if=/dev/zero of=/dev/rbd0; rbd unmap /dev/rbd0;
$ time (rados -p rbd ls | grep `rbd info rbdbigsize | grep block_name_prefix | grep -o 'rb.*$'` | xargs -n 200 rados -p rbd rm; rados -p rbd rm rbdbigsize.rbd; rbd rm rbdbigsize)
real 143m6.373s
user 2m49.100s
sys 0m22.080s
$ rbd create rbdbigsize --size 1048576
$ rbd map rbdbigsize; dd if=/dev/zero of=/dev/rbd0; rbd unmap /dev/rbd0;
$ time rbd rm rbdbigsize
real 88m44.121s
user 1m5.464s
sys 0m25.244s
