CephNotes

Some notes about Ceph
Laurent Barbe @SIB

Feature set mismatch error on Ceph kernel client

Depending on the kernel version you may be missing some features required by the cluster (or vice versa, required by client but not available on the cluster). Hence the errors "feature set mismatch".

Some examples of errors that can be encountered :

mon0 192.168.0.1:6789 feature set mismatch, my XXXXXX < server's XXXXXX, missing 2040000

--> Upgrade kernel client up to 3.9 or set tunables to legacy : ceph osd crush tunables legacy

mon0 192.168.0.1:6789 feature set mismatch, my XXXXXX < server's XXXXXX, missing 40000000

--> Upgrade kernel client up to 3.9 or unset hashpspool : ceph osd pool set rbd hashpspool false

mon0 192.168.0.1:6789 feature set mismatch, my XXXXXX < server's XXXXXX, missing 800000000

--> Remove cache pool and reload monitors or upgrade kernel client up to 3.14

mon0 192.168.0.1:6789 feature set mismatch, my XXXXXX < server's XXXXXX, missing 1000000000

--> Upgrade kernel client up to 3.14

mon0 192.168.0.1:6789 feature set mismatch, my 4a042a42 < server's 2004a042a42, missing 20000000000

--> Upgrade kernel client up to 3.15 or disable tunable 3 features

CEPH_FEATURE Table and Kernel Version

You can find the feature missing in that table :

For exemple, missing 2040000 means that CEPH_FEATURE_CRUSH_TUNABLES (40000) and CEPH_FEATURE_CRUSH_TUNABLES2 (2000000) is missing on kernel client.

'R':required, 'S':support, '-X-' feature is new since this version

Feature BIT OCT 3.8 3.9 3.10 3.14 3.15 3.18 4.1 4.5 4.6
CEPH_FEATURE_NOSRCADDR 1 2 R R R R R R R R R
CEPH_FEATURE_SUBSCRIBE2 4 10 -R-
CEPH_FEATURE_RECONNECT_SEQ 6 40 -R- R R R R R R
CEPH_FEATURE_PGID64 9 200 R R R R R R R R
CEPH_FEATURE_PGPOOL3 11 800 R R R R R R R R
CEPH_FEATURE_OSDENC 13 2000 R R R R R R R R
CEPH_FEATURE_CRUSH_TUNABLES 18 40000 S S S S S S S S S
CEPH_FEATURE_MSG_AUTH 23 800000 -S- S S S
CEPH_FEATURE_CRUSH_TUNABLES2 25 2000000 S S S S S S S S
CEPH_FEATURE_REPLY_CREATE_INODE 27 8000000 S S S S S S S S
CEPH_FEATURE_OSDHASHPSPOOL 30 40000000 S S S S S S S S
CEPH_FEATURE_OSD_CACHEPOOL 35 800000000 -S- S S S S S
CEPH_FEATURE_CRUSH_V2 36 1000000000 -S- S S S S S
CEPH_FEATURE_EXPORT_PEER 37 2000000000 -S- S S S S S
CEPH_FEATURE_OSD_ERASURE_CODES*** 38 4000000000
CEPH_FEATURE_OSDMAP_ENC 39 8000000000 -S- S S S S
CEPH_FEATURE_CRUSH_TUNABLES3 41 20000000000 -S- S S S S
CEPH_FEATURE_OSD_PRIMARY_AFFINITY 41* 20000000000 -S- S S S S
CEPH_FEATURE_CRUSH_V4 **** 48 1000000000000 -S- S S
CEPH_FEATURE_CRUSH_TUNABLES5 58 200000000000000 -S- S
CEPH_FEATURE_NEW_OSDOPREPLY_ENCODING 58* 400000000000000 -S- S

(*** Note that CEPH_FEATURE_OSD_ERASURE_CODES is no more required for client since firefly)

(**** staw2 : http://tracker.ceph.com/issues/11364)

Comments