OSD BlueStore RocksDB metadata and WAL placement

Starting with ODF 4.7 you can customize the deployment of your OSDs when it comes to what device to use for what part of BlueStore.

This is useful if you want to offload the OSD’s metadata onto a different disk for performance reasons.

Let’s consider a standard StorageCluster Custom Resource below:

---
apiVersion: ocs.openshift.io/v1
kind: StorageCluster
metadata:
  name: ocs-storagecluster
  namespace: openshift-storage
spec:
  manageNodes: false
  monDataDirHostPath: /var/lib/rook
  storageDeviceSets:
  - count: 1
    dataPVCTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: {size}
        storageClassName: {storageclass}
        volumeMode: Block
    name: {deviceset-prefix}
    portable: false
    replica: 3

This will deploy a cluster in a very standard way where every piece of the OSD is collocated on the PVC that will be provisioned using the storage class {storageclass}. The size of the PersistentVolume that will be provisioned is specified as {size}.

1. RocksDB metadata placement

This feature is Development Preview.

You can now add the following section to your storageDeviceSets parameter to customize the placement of RocksDB at deployment time.

    metadataPVCTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: {size}
        storageClassName: {storageclass}
        volumeMode: Block

When added to your storage cluster the following will happen:

Each OSD data will claim a PVC as {deviceset-prefix-x-data-y}
Each OSD RockDB metadata will claim a PVC as {deviceset-prefix-x-metadata-y}

The LSO discovery and subsequent provisioning will create new PVs using the entire disk and you cannot create partitions after this process. Make sure to configure your RocksDB metadata disk partitions before LSO discovery to create the data and metadata PVs to be used to deploy your storage cluster.

2. RocksDB WAL placement

This feature is Development Preview.

You can now add the following section to your storageDeviceSets parameter to customize the placement of the RocksDB Write Ahead Log (WAL) at deployment time.

    walPVCTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: {size}
        storageClassName: {storageclass}
        volumeMode: Block

When added to your storage cluster the following will happen:

Each OSD data will claim a PVC as {deviceset-prefix-x-data-y}
Each OSD RockDB WAL will claim a PVC as {deviceset-prefix-x-wal-y}

The WAL placement can be combined with the RocksDB metadata placement.