The purpose of this test is to map a RadosGw Bucket to a specific Ceph pool. For exemple, if using a fast pool with ssd and a low pool for archive...
standard_bucket datas --> .rgw.buckets (default pool)
specific_bucket datas --> .rgw.buckets.custom
First, we create a pool .rgw.buckets.custom, with, for example, some specific parameters (different size and different ruleset in crushmap) :
$ ceph osd pool create .rgw.buckets.custom 64 64
pool '.rgw.buckets.custom' created
$ ceph osd pool set .rgw.buckets.custom size 2
set pool 59 size to 2
$ ceph osd pool set .rgw.buckets.custom crush_ruleset 6
set pool 59 crush_ruleset to 6
Then, we need to configure a specific placement_targets in region map and zone. For next step, you need to have a running config of rados-gw...
$ radosgw-admin region get > region.conf.json
$ vim region.conf.json # Add an entry in placement_targets
{ "name": "default",
"api_name": "",
"is_master": "true",
"endpoints": [],
"master_zone": "",
"zones": [
{ "name": "default",
"endpoints": [],
"log_meta": "false",
"log_data": "false"}],
"placement_targets": [
{ "name": "default-placement",
"tags": []},
{ "name": "custom-placement",
"tags": []}],
"default_placement": "default-placement"}
$ radosgw-admin region set < region.conf.json
....
$ radosgw-admin zone get > zone.conf.json
$ vim zone.conf.json # Add an entry in placement_pools with key "custom-placement"
{ "domain_root": ".rgw",
"control_pool": ".rgw.control",
"gc_pool": ".rgw.gc",
"log_pool": ".log",
"intent_log_pool": ".intent-log",
"usage_log_pool": ".usage",
"user_keys_pool": ".users",
"user_email_pool": ".users.email",
"user_swift_pool": ".users.swift",
"user_uid_pool": ".users.uid",
"system_key": { "access_key": "",
"secret_key": ""},
"placement_pools": [
{ "key": "default-placement",
"val": { "index_pool": ".rgw.buckets.index",
"data_pool": ".rgw.buckets",
"data_extra_pool": ".rgw.buckets.extra"}},
{ "key": "custom-placement",
"val": { "index_pool": ".rgw.buckets.index",
"data_pool": ".rgw.buckets.custom",
"data_extra_pool": ".rgw.buckets.extra"}}]}
$ radosgw-admin zone set <zone.conf.json
2014-11-25 18:03:23.894153 7f728c0f2780 0 couldn't find old data placement pools config, setting up new ones for the zone
.....
$ radosgw-admin regionmap update
{ "regions": [
{ "key": "default",
"val": { "name": "default",
"api_name": "",
"is_master": "true",
"endpoints": [],
"master_zone": "",
"zones": [
{ "name": "default",
"endpoints": [],
"log_meta": "false",
"log_data": "false"}],
"placement_targets": [
{ "name": "custom-placement",
"tags": []},
{ "name": "default-placement",
"tags": []}],
"default_placement": "default-placement"}}],
"master_region": "default",
"bucket_quota": { "enabled": false,
"max_size_kb": -1,
"max_objects": -1},
"user_quota": { "enabled": false,
"max_size_kb": -1,
"max_objects": -1}}
$ /etc/init.d/radosgw reload
Reloading ...
To configure s3cmd for RadosGW you can have a look here : http://lollyrock.com/articles/s3cmd-with-radosgw/
Now we can test bucket creation :
$ s3cmd mb s3://custombucket --bucket-location=custom-placement
Bucket 'custombucket' created
$ touch "file_on_custom_pool"
$ s3cmd put file_on_custom_pool s3://custombucket
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
file_on_custom_pool -> s3://custombucket/file_on_custom_pool [1 of 1]
0 of 0 0% in 0s 0.00 B/s done
Verify :
$ radosgw-admin bucket stats --bucket=custombucket
{ "bucket": "custombucket",
"pool": ".rgw.buckets.custom",
"index_pool": ".rgw.buckets.index",
"id": "default.240909.1",
"marker": "default.240909.1",
"owner": "testuser",
"ver": 1,
"master_ver": 0,
"mtime": 1417016078,
"max_marker": "",
"usage": {},
"bucket_quota": { "enabled": false,
"max_size_kb": -1,
"max_objects": -1}}
Pool var is set on ".rgw.buckets.custom".
$ rados -p .rgw.buckets.custom ls
default.241071.1_file_on_custom_pool
It's here !
Data placement pool is define in this order :
- from the request ("bucket location")
- from user ("default_placement" : see with
radosgw-admin metadata get user:<uid>) - from region map ("default_placement")