Architecture

Block storage on any CSI driver

Cinder volumes, Glance images and Nova VM disks live on Kubernetes block volumes: OpenKeel works with any block CSI driver, one Cinder backend per StorageClass. Rook-Ceph, Longhorn, NetApp, Huawei Dorado and our own Pure Storage driver are supported.

Supported block CSI drivers

Software-defined storage on the cluster's own disks or enterprise arrays: the choice is a StorageClass, not a rewrite.

Software-defined

Rook-Ceph

Ceph RBD block volumes managed by Rook on the cluster's disks; the same Ceph also serves S3 and Swift through RGW.

Default

Longhorn

The default backend of okadm in production and the reference of the tests: replicated volumes on node disks, snapshots, backups, RWX.

Enterprise array

NetApp

NetApp arrays through NetApp's Trident CSI driver; snapshots and clones are done by the array.

Enterprise array

Huawei Dorado

OceanStor Dorado all-flash arrays through Huawei's CSI plugin; snapshots and clones are done by the array.

OpenKeel driver

Pure Storage

Our own Go CSI driver for FlashArray over iSCSI: multipath, snapshots and group snapshots, clones, online expansion, block RWX for KubeVirt live migration.

Our driver

From a Cinder volume to the array

01
Cinder API
Same volume API and volume types as OpenStack; each type points to a backend
02
StorageClass
One csi-block backend per StorageClass: Longhorn, Ceph RBD, NetApp, Dorado or Pure
03
PVC in Block mode
Volume = PVC in the project's namespace, snapshot = VolumeSnapshot, clone = PVC dataSource
04
KubeVirt
The PVC is hot-plugged into the VM; Nova attachments are kept as in OpenStack

What OpenKeel asks of a CSI driver

CSI capability	Used by OpenKeel for
Block volumes (volumeMode: Block)	Cinder volumes, VM disks, Glance images
VolumeSnapshot	Cinder snapshots, backups, revert to snapshot
Clone (PVC dataSource)	Volume from image, from snapshot or from another volume
Volume expansion	Cinder extend, online when the driver allows it
Block ReadWriteMany	Multi-attach volumes and KubeVirt live migration
VolumeGroupSnapshot	Consistent snapshots of several volumes (Cinder groups are not implemented yet)

QoS specs map to StorageClass parameters when the driver supports them; otherwise they are a known gap. Source: generate-go-from-spec/references/backends/cinder-csi-block.md, openkeel-cinder/config/config.example.yaml.

Several arrays, one Cinder

Each backend names a StorageClass and its snapshot class; volume types let tenants choose. The volume host reads openkeel@<backend>#<storage class>.

backends:
  - name: longhorn
    driver: csi-block
    storage_class: longhorn
    snapshot_class: longhorn-snapshot
    volume_mode: Block
  - name: pure
    driver: csi-block
    storage_class: pure-iscsi
    snapshot_class: pure-snapshot
    volume_mode: Block
    multiattach: true
volume_types:
  - {name: standard, backend: longhorn}
  - {name: performance, backend: pure}

Validation status

Driver	Status in OpenKeel
Longhorn	Default backend; the Cinder, Glance and Nova tests run on it
Pure Storage (OpenKeel driver)	Prototype: csi-sanity passes in memory (79 specs); not yet run on a real FlashArray
Rook-Ceph (RBD)	Supported through the csi-block backend; not yet tested in the OpenKeel lab
NetApp (Trident)	Supported through the csi-block backend; not yet tested in the OpenKeel lab
Huawei Dorado	Supported through the csi-block backend; not yet tested in the OpenKeel lab