The Ceph RGW storage driver goes upstream in Libcloud

Datetime:2016-08-23 04:07:17          Topic: Ceph           Share

The Ceph RGW storage driver was upstream in Apache Libcloud today. It is available in the Libcloud trunk repository and it will ship with the next release of Libcloud.

This post will introduce the new RGW driver together with the proper configuration parameters to run some examples uploading/downloading objects in Ceph Jewel.

The Ceph RGW storage driver

The Ceph RGW storage driver requires Ceph Jewel or above. As of this writing, the last Ceph Jewel version is 10.2.1 . This version is available in the downloads section .

The driver extends the Libcloud S3 storage driver to provide a compatible S3 API with Ceph RGW.

The driver also contains support for AWS signature versions 2 (AWS2) and 4 (AWS4). It leverages the Libcloud common auth support on the client side. On the Ceph RGW side it required a little patch to handle unsigned paylods in the AWS4 auth header .

Developers and apps can use the Ceph RGW driver via the S3_RGW provider easily. A simple snippet follows...

from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver
import libcloud

api_key = 'api_key'
secret_key = 'secret_key'

cls = get_driver(Provider.S3_RGW)

driver = cls(api_key,
             secret_key,
	     signature_version='4',
	     region='my-region',
	     host='my-host',
	     port=8000)

container = driver.get_container(...)

If the region has not an explicit value, the driver will use the default region 'default'.

The valid signature versions are '2' (AWS2) and '4' (AWS4). AWS2 is the default signature version.

One host name is always required. No default value here.

The following two examples contain the minimal code to upload/download objects with the new provider:

Running the upload example...

$ ./test-upload-ceph-rgw-driver.py
<Object: name=my-name-abcdabcd-123,
size=110080,
hash=0a5cfeb3bb10e0971895f8899a64e816,
provider=Ceph RGW S3 (my-region) ...>

Running the download example...

$ ./test-download-ceph-rgw-driver.py
<Object: name=my-name-abcdabcd-123,
size=110080,
hash=0a5cfeb3bb10e0971895f8899a64e816,
provider=Ceph RGW S3 (my-region) ...>

Enjoy!

Acknowledgments

My work in Apache Libcloud is sponsored by Outscale and has been made possible by Igalia and the invaluable help of the Libcloud community . Thank you all!





About List