CephNotes

Some notes about Ceph
Laurent Barbe @SIB

Ceph Primary Affinity

This option allows you to answer a fairly constant worry in the case of heterogeneous cluster. Indeed, all the discs do not have the same performance or not the same ratio performance / size. With this option it is possible to reduce the load on a disk without reducing the amount of data it contains. Furthermore, the option is easy to modify because it does not result in data migration. Only preference between primary / secondary will be modified and propagated to clients.

Before playing with cluster options and tune crushmap, remember to verify that your client is compatible with those options. {% blockquote %} You must enable 'mon osd allow primary affinity = true' on the mons before you can adjust primary-affinity. note that older clients will no longer be able to communicate with the cluster. {% endblockquote %} ( For client kernel module you can have a look to http://cephnotes.ksperis.com/blog/2014/01/21/feature-set-mismatch-error-on-ceph-kernel-client. )

Look if the monitor has the primary affinity option:

# ceph --admin-daemon /var/run/ceph/ceph-mon.*.asok config show | grep 'primary_affinity'
  "mon_osd_allow_primary_affinity": "false",

Edit ceph.conf dans add in section [mon]:

mon osd allow primary affinity = true

Reload mon and test. We look how many pg is primary on osd.0, and how many is secondary :

# ceph pg dump | grep active+clean | egrep "\[0," | wc -l
100
# ceph pg dump | grep active+clean | egrep ",0\]" | wc -l
80

Try to change primary affinity :

# ceph osd primary-affinity osd.0 0.5
set osd.0 primary-affinity to 0.5 (8327682)

# ceph pg dump | grep active+clean | egrep "\[0," | wc -l
48
# ceph pg dump | grep active+clean | egrep ",0\]" | wc -l
132

# ceph osd primary-affinity osd.0 0
set osd.0 primary-affinity to 0 (802)

# ceph pg dump | grep active+clean | egrep "\[0," | wc -l
0
# ceph pg dump | grep active+clean | egrep ",0\]" | wc -l
180

Now there will be no more reading on this OSD.

misc

Comments