keystone-tempest-plugin — Rewrite pack

Rewrite pack: conformance suite — keystone-tempest-plugin — branch master

Branch master @ 41e16f7f319c — run against Keystone stable/2026.2 @ e6dc5ae079e6 with Tempest 47.0.0 — analysed on 2026-10-28 keystone: = Keystone repository (stable/2026.2)

This file turns keystone-tempest-plugin into a conformance suite for OpenKeel Keystone:

• what it checks;
• how to run it against any implementation;
• the reference results obtained with the Python Keystone;
• what it does not cover.

1. Reference results

A rewrite is conformant when it gives the same outcome for all 1,980 tests, with the same 14 skips. A stricter check also compares the 4,372 assertions one by one with _rewrite/rbac-assertions_master.json.

2. How to run it

Use the keystone_tempest_plugin.py adapter of the generate-reverse-spec skill. Install Keystone, Tempest and the plugin in the same venv:

conformance_capture.py scenarios <keystone-tempest-plugin> --branch master --adapter adapters/keystone_tempest_plugin.py \
    --routes-json <reverse-spec-keystone>/_rewrite/routes_stable-2026.2.json --workdir <tmp> --out <tmp>/cc

• Restrict the run with KTP_MODULES=keystone_tempest_plugin.tests.rbac.v3.test_project,…. One module takes under a minute.
• Keystone settings that the suite needs:
  • [oslo_policy] enforce_scope = true and enforce_new_defaults = true, otherwise the RBAC tests are skipped or test the old rules;
  • [identity-feature-enabled] in tempest.conf: access_rules, application_credentials, project_tags, trust = true; federation, external_idp = false.
• Two process constraints:
  • Tempest and Keystone both use the global oslo.config object, so Tempest runs in a subprocess.
  • Keystone is served on 127.0.0.1:<free port>. A runner adds X-Test-Id to every request, so that each exchange is tagged with its test.
• Speed of the harness, which does not change behaviour:
  • oslo.db forces NullPool on SQLite files, so every session reconnects and re-reads the schema. The adapter switches to QueuePool; without it, the full run takes hours.
  • The adapter sets [identity] password_hash_rounds = 4, the bcrypt minimum.
• Against OpenKeel Keystone: keep the generated tempest.conf, point [identity] uri_v3 at the Go service, create the bootstrap admin (admin, project admin, Default domain, system role admin), then run the runner alone, or the adapter with the service started outside.

3. What the RBAC matrix requires

_rewrite/rbac-matrix_master.md gives the expected result for each operation and each of the 11 personas: system_*, domain_* and project_*, with the roles admin, manager, member and reader. It is the executable form of Keystone's default policies. Key points for a rewrite:

1. **An admin role on a project is a cloud administrator.**
   • project_admin has the same result as system_admin on 179 of the 181 operations where both are tested: it can create domains, limits, roles, endpoints, identity providers, endpoint groups…
   • The cause is the rule admin_required = role:admin or is_admin:1 (keystone:keystone/common/policies/base.py:84). Many policies accept it with scope_types=['system', 'project'], for example identity:create_domain (keystone:keystone/common/policies/domain.py:83-89).
   • The tests encode it: ProjectAdminTests inherits from SystemAdminTests (keystone_tempest_plugin/tests/rbac/v3/test_project.py:580-582, keystone_tempest_plugin/tests/rbac/v3/test_user.py:492).
   • The 2 differences are trusts (POST /v3/OS-TRUST/trusts, which needs a project-scoped token) and the access rules of another user.
2. **domain_admin is also a global administrator for users and projects** (DomainAdminTests inherits from SystemAdminTests, keystone_tempest_plugin/tests/rbac/v3/test_user.py:251). However, it is refused the operations whose scope_types exclude domain, such as POST /v3/domains.
3. **domain_manager and project_manager** (the manager role) give ✓/403 cells: allowed in their own domain or project, refused elsewhere. Examples: create projects and users, and grant roles in their own domain. A rewrite must evaluate the target (target.domain.id, target.project.id), not just the role.
4. Readers and members get 403 on every write. Reads of other domains or projects give 403 too.
5. Owner-filtered lists: GET /v3/credentials returns 200 to all 11 personas, so the matrix shows ✓ everywhere. The filtering is checked by the test itself, with assertIn/assertNotIn on the body:
   • domain_* and project_* see their own credentials and not those of other users (keystone_tempest_plugin/tests/rbac/v3/test_credential.py:381-397, inherited by every class from DomainManagerTests down);
   • the system personas, domain_admin and project_admin see all of them (keystone_tempest_plugin/tests/rbac/v3/test_credential.py:211-221).
6. 403 or 404 for a target that does not exist: see 02 § 2.

4. A behaviour the suite no longer checks

Commit d5dcf00f4f69 of the plugin removed the tests of cross-domain grants of domain-specific roles (06):

• Keystone stable/2026.2 still accepts PUT /v3/domains/{d}/users/{u}/roles/{r} with 204 when r belongs to another domain;
• Keystone master refuses it since keystone:fa43c4b3d.

Since the plugin tests neither behaviour, a rewrite must decide on its own. Recommendation: refuse the grant (403), like master, and add that test to the OpenKeel suite.

5. Coverage gaps

63 Keystone routes are not called by any test (_rewrite/keystone-operations-not-covered_master.json). They need their own tests in OpenKeel:

Keystone's own unit tests cover these routes (reverse-spec-keystone 07). Together, the two suites cover the API.

6. Files in _rewrite/

Source: reverse-spec-keystone-tempest-plugin/07-rewrite-pack_master.md. The _rewrite/ and _inventory/ files are not published on the site.