cinder — Rewrite pack

Rewrite pack — cinder — branch stable/2026.2

Branch stable/2026.2 @ c117f9cf08b4 (29.0.0-1-gc117f9cf0) — repository https://opendev.org/openstack/cinder — analysed on 2026-10-28 Client references: Nova stable/2026.2 @ 8df1922926dd (nova:)

This file is the entry point for re-implementing Cinder in Go (OpenKeel) as a drop-in replacement of the Block Storage API v3:

• Nova, Glance (cinder store), Horizon/Skyline, python-cinderclient/openstacksdk and Tempest must keep working unchanged;
• the deployment and operation conventions (paste-free, but same configuration names where they matter, same policies, same database) should stay recognisable.

OpenKeel replaces the storage drivers by Kubernetes block CSI volumes (Longhorn…). This pack therefore specifies the API, state machine, scheduling, quota and messaging contracts; the hundreds of vendor drivers are out of scope, and the driver interface is described only where its results are visible to clients (connection_info, capabilities).

1. Contents of _rewrite/

Library versions used for the extraction: Python 3.12.7, oslo.policy 6.0.1, oslo.config 10.7.0, SQLAlchemy 2.0.52, Routes 2.5.1, WebOb 1.8.11, all from the requirements stable/2026.2 upper-constraints.

2. HTTP behaviour that the contracts do not show

2.1 Pipeline and which error wins

Outermost first (etc/cinder/api-paste.ini:15): request ID → CORS → proxy headers → fault wrapper → body size limit → profiler → keystonemiddleware (401) → context (401 without body if no user) → router. Then, inside Resource.__call__ (cinder/api/openstack/wsgi.py:814-966), in this order:

1. microversion header: malformed → 400, out of range → 406 — checked before routing to the method, so an unsupported version on an unknown resource gives 406;
2. Content-Type not JSON → 415;
3. method lookup: unknown action key → 400, controller without the method → 404;
4. body JSON parsing → 400 "Malformed request body";
5. project_id in the URL different from the token's → 400 "Malformed request url";
6. handler: schema validation (400), policy (403), not found (404), state (400/409), quota (413);
7. a versioned method outside its range → 404 with no message.

2.2 Error body

{"<faultName>": {"code": <int>, "message": "<text>"}}, faultName by status (badRequest, unauthorized, forbidden, itemNotFound, badMethod, conflictingRequest, overLimit, badMediaType, notImplemented, serviceUnavailable, else computeFault); 413 adds retryAfter and Retry-After: 0 (cinder/api/openstack/wsgi.py:1310-1363). Clients (cinderclient, hence Nova) map the HTTP status to exception classes; the message text is displayed but not parsed by Nova (nova:nova/volume/cinder.py:371-420). Masking: for non-Invalid/NotFound errors the message is replaced by webob's generic text unless the exception is safe, and safe messages are prefixed with the class name ("VolumeSizeExceedsAvailableQuota: …", cinder/api/middleware/fault.py:55-79).

2.3 Other cross-cutting rules

• Default microversion is 3.0 when the header is absent; responses always echo OpenStack-API-Version: volume X.Y and Vary: OpenStack-API-Version.
• Project-less URLs (/v3/volumes) are accepted at every microversion, in addition to /v3/{project_id}/volumes.
• Empty path segments are tolerated: /v3//volumes and /v3/<project>//volumes/<id> are served (200) — Cinder's own functional client depends on it (158 such requests in exchanges_stable-2026.2.json). A Go router must collapse //.
• Body actions: a single top-level key selects the handler; more than one key → 400.
• Request ID headers: both x-openstack-request-id and x-compute-request-id.
• Version discovery: GET / → 300 with {"versions": [{"id": "v3.0", "status": "CURRENT", "version": "3.71", "min_version": "3.0", "media-types": […], "links": […]}]} (cinder/api/versions.py:32-46); Nova reads version and min_version from the entry whose version contains "3." (nova:nova/volume/cinder.py:84-134).
• Lists: limit ≤ osapi_max_limit (1000), marker, offset, sort=key:dir,…, with_count (3.45 → count in the body), *_links with rel: next. Filters not in resource_filters.json → 400 from 3.34 [inferred: strictness tied to mv.LIKE_FILTER/RESOURCE_FILTER in cinder/volume/api.py:2342-2474].
• QoS associate/disassociate are GET requests with side effects — keep them.
• DELETE returns 202 with empty body for volumes, snapshots, backups; 204 for metadata, attachments and types [see the Success column of 01].

3. Algorithms and contracts a rewrite must reproduce

3.1 Resource state machine (compare-and-swap)

Every transition started by the API is one SQL UPDATE … WHERE <expected> (conditional_update, cinder/db/api.py:551); 0 rows updated means a 400 (or 409) with an explanatory message. A rewrite must keep the same expected sets, because clients and Tempest assert on them:

The statuses are strings in the database and in the API (cinder/objects/fields.py:25-210); os-reset_status writes any of them.

3.2 Scheduling (where a volume is placed)

Inputs: the volume size, the volume type extra specs (capabilities, RESKEY:availability_zones, provisioning:type), the AZ, scheduler hints (same_host, different_host, local_to_instance), and the per-pool stats reported every minute by each backend (§ 3.5 of async).

1. Candidates: pools of cinder-volume services that are up (updated_at within service_down_time), not disabled, not frozen (cinder/scheduler/host_manager.py:680-700).
2. Filters (default AZ, Capacity, Capabilities; configurable): capacity rule of async § 2.2; CapabilitiesFilter matches every extra spec of the type (operators <is> True, <in>, >=, <or>…) against the pool's capabilities, ignoring scoped keys other than capabilities: [inferred: cinder/scheduler/filters/capabilities_filter.py].
3. Weighers (default CapacityWeigher, multiplier 1.0), normalised by OrderedHostWeightHandler; highest weight wins; ties broken by order [inferred].
4. Retry: the attempted backends travel in filter_properties.retry; at most scheduler_max_attempts (3).

With CSI, OpenKeel can keep this algorithm with a single "pool" per storage class, or delegate placement to the CSI driver. Either way, the observable contract is: host = host@backend#pool on the volume (admin attribute os-vol-host-attr:host), GET /scheduler-stats/get_pools shape, and error + VOLUME_VOLUME_001_003 when nothing fits.

3.3 Quotas

• Resources: volumes, snapshots, gigabytes, backups, backup_gigabytes, groups, and per volume type volumes_<type>, snapshots_<type>, gigabytes_<type>; per_volume_gigabytes is a limit check, not a reservation. Defaults quota_* options; -1 = unlimited (cinder/quota.py:36-80).
• Two-phase: reserve (locks quota_usages rows of the project FOR UPDATE, refreshes usage when until_refresh/max_age say so, raises OverQuota if in_use + reserved + delta > limit, inserts reservations expiring after reservation_expire), then commit or rollback; expired reservations are rolled back by the scheduler (cinder/quota.py:319-440, cinder/db/api.py:1585-1760).
• Translation to HTTP 413 messages per resource (cinder/quota_utils.py:90-140).
• Transfers move quota from donor to recipient (cinder/transfer/api.py:235-330).

3.4 Transfer authentication key

salt = first 8 hex characters of sha224(urandom(255)) concatenations, auth_key = 16 such characters (volume_transfer_salt_length, volume_transfer_key_length); stored crypt_hash = HMAC-SHA1(key=salt, msg=auth_key).hexdigest() (cinder/transfer/api.py:106-129). Existing transfers created by Python must be acceptable by the rewrite: reproduce the HMAC exactly.

3.5 User messages

event_id = "VOLUME_" + resource_type + "_" + action_id + "_" + detail_id (cinder/message/api.py:97-105), with the action and detail tables of cinder/message/message_field.py, message_level ERROR, expires_at = now + message_ttl. Clients (Horizon, Skyline) display user_message = "<action text>: <detail text>" [inferred: cinder/api/v3/views/messages.py].

3.6 Microversion-dependent views

Volume detail fields by version are listed in 01 § 2.1; the per-field x-microversions of response-schemas_stable-2026.2.json show what the tests observed. View builders: cinder/api/v3/views/volumes.py, cinder/api/views/*.py.

4. What the clients expect

5. Database strategy

• Reuse the schema to run Python and Go side by side during the migration: use schema-sqlite-migrated_stable-2026.2.sql (or the MySQL result of cinder-manage db sync) as the contract, including the two unused columns of schema-diff_stable-2026.2.md.
• Soft delete everywhere (deleted, deleted_at); volumes.deleted is a boolean, ids are UUID strings; quota_*, reservations, *_metadata have integer ids.
• Concurrency: compare-and-swap updates (§ 3.1), SELECT … FOR UPDATE for quotas, tooz locks per resource for long operations ([coordination]/backend_url, default local files — use a distributed backend with more than one replica).
• services rows are the liveness and version registry; the Go services must write heartbeats in the same table if Python services coexist, and set rpc_current_version/object_current_version so that Python clients pin correctly.
• The workers table implements crash recovery; keep its semantics or the equivalent.

6. Authentication, policies, configuration and CLI

• Tokens validated like keystonemiddleware (cache, service token for Nova, X-Service-Token); context from the validated token: roles, project, domain, is_admin = policy context_is_admin (role:admin).
• 167 oslo.policy rules with the xena_system_admin_or_project_* and admin_api defaults (§ 1.7 of 01); operators' policy files must keep working, so the rule names and the check language (role:, project_id:%(project_id)s, rule:, is_admin:True, is_admin_project:True) are part of the contract.
• Configuration: the options that change behaviour visible to clients are listed in 01–05 (osapi_max_limit, default_volume_type, allow_availability_zone_fallback, snapshot_same_host, enable_force_upload, message_ttl, quotas, scheduler filters/weighers, service_down_time, report_interval…); config_stable-2026.2.conf has them all.
• CLI parity worth keeping: cinder-manage db sync|purge, service list|remove, volume update_host (cinder/cmd/manage.py).

7. Conformance strategy

1. **Differential testing on exchanges_stable-2026.2.json: 967 requests from 196 passing tests, at microversions 3.0 to 3.71. The database is built by the tests through the API itself, so the exchanges replay **in order against an empty Go Cinder configured like the test fixture (noauth user:project tokens, default_volume_type set, a fake backend that always succeeds, cinder/tests/fake_driver.py); compare statuses and bodies with IDs, timestamps and URLs normalised. Test IDs are not recorded (§ index).
2. Response shapes: validate Go responses against response-schemas_stable-2026.2.json; for the 116 operations without an observed schema, use the api-ref fields of the OpenAPI document.
3. Tempest volume API tests (tempest/api/volume) and the cinder-tempest-plugin (both cloned under OpenKeel) are the black-box suite to run against the rewrite, with the RBAC tests under enforce_scope/enforce_new_defaults.
4. Nova: the Nova functional tests that use CinderFixture do not hit a real Cinder; the end-to-end check is Tempest tempest/scenario volume attach/extend tests.
5. Original fixtures for reference: cinder/tests/functional/functional_helpers.py:117-170 (services started, paste pipeline), cinder/tests/functional/api/client.py (client), cinder/tests/fake_driver.py (backend).

8. Gaps found while building the pack

• No gabbi suite and no recorded test IDs for exchanges; replay of the exchanges against the Python service was not run (the recording itself is the run: 196/196 tests passed).
• Only 102 of 218 operations have observed response schemas (the functional suite covers volumes, snapshots, backups, types, QoS, quotas, services, transfers, groups; it does not cover attachments, messages, workers, manageable resources, clusters in depth).
• os-extend_volume_completion microversion 3.71 is not enforced (01 § spec/code mismatches); decide whether the rewrite enforces it (Nova stable/2026.2 does not call the action at all).
• 13 policy operations entries point to non-existent routes; 58 operations have no documented policy operations.
• Config sample generated with googleapiclient stubbed (GCS backup driver options are present, their defaults are literal values and unaffected).

9. Non-functional requirements

• Request path never blocks on storage except the synchronous RPC calls of § 1.3 of 03 (attachment_update, initialize_connection…), bounded by rpc_response_timeout (60 s) — Nova retries them on 500/504.
• Capabilities flow: one stats message per backend per minute to every scheduler; liveness with report_interval 10 s / service_down_time 60 s.
• Message retention 30 days; reservations expire after 1 day.

10. Rewrite checklist

• [ ] Router with project-less and // tolerant paths, body-key action dispatch, microversion negotiation (default 3.0, max 3.71, 406/400/404 rules).
• [ ] Fault body, masking rules, Retry-After on 413, request ID headers.
• [ ] The 112 request schemas with their microversion ranges.
• [ ] Views per microversion (volume, snapshot, backup, attachment, type…).
• [ ] State machine with the expected sets of § 3.1 and the same error messages for the common cases (Tempest checks statuses, not texts).
• [ ] Quotas with reservations, per-type resources, 413 messages, transfer quota moves.
• [ ] Scheduler (or CSI delegation) with host@backend#pool and get_pools.
• [ ] Attachments API compatible with Nova 3.44 flow, idempotent update/delete, service-token check on delete.
• [ ] Nova external events after extend and reimage.
• [ ] User messages (event_id format, TTL, purge).
• [ ] Notifications volume.*, snapshot.*, backup.* if Ceilometer is used.
• [ ] Policies with the same rule names and defaults.
• [ ] Database: migrated schema, soft delete, services heartbeats, workers recovery.
• [ ] Transfer HMAC-SHA1 compatibility.
• [ ] Replay exchanges_stable-2026.2.json and run Tempest volume tests.

Source: reverse-spec-cinder/07-rewrite-pack_stable-2026.2.md. The _rewrite/ and _inventory/ files are not published on the site.