Creating an RBD storage

You can connect a Ceph storage when creating a VMmanager cluster or in the cluster card. Read more in Creating a cluster and Managing cluster storages.

The article describes the actions of the platform when connecting Ceph storage.

How it works

VMmanager creates a pool for RBD storage operation. A pool is a logical container for storing data. Physically the pool can be located on one or more disks in the cluster. 

Pools consist of placement groups (PG) in which objects with data are stored. The required number of PGs is calculated by the formula:

PG = OSD * 100 / number of replicas

where OSD is the number of storage devices. The result should be rounded to the nearest power of 2.

For example, if the Ceph cluster has 10 OSD and the number of replicas is 4, then the total number of PG = 10 * 100 / 4 = 250. After rounding to the nearest power of 2, the total number of PGs is 256.

VMmanager uses the ceph software and libvirt library tools in the connection process. Read more in libvirt documentation and ceph documentation for working with libvirt.

Adding the storage

To add the storage, VMmanager takes the following steps:

1. VMmanager connects with specified settings to the Ceph cluster monitor server.

2. If required, it creates a pool with specified settings:

   ceph osd pool create <pool-name> <pg-size> <pgp-size>

   Comments to the command

   <pool-name> — pool name.

   <pg-size> — quantity of placement groups (PG).

   <pgp-size> — quantity of PGPs — placement groups for placement purpose. It must be equal to PG.

3. VMmanager initializes the pool for RBD operation:

   rbd pool init <pool-name>

4. If a user for working with the pool has already been created, VMmanager checks that user's permissions. If the permissions are insufficient, VMmanager supplements them.

5. If required, VMmanager creates a new pool user and generates an authentication key:

   ceph auth get-or-create client.libvirt mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=<pool-name>'

6. Creates the secret.xml file with the resulting key and adds it as the libvirt's "secret":

   virsh secret-define --file secret.xml

7. Gets the user key:

   ceph auth get-key client.for_ceph

8. Sets the user key for libvirt's "secret":

   virsh secret-set-value --secret {uuid of secret} --base64 $(cat client.for_ceph.key)

9. Creates the pool with libvirt's "secret".