keystone-tempest-plugin — Summary

Reverse-specification of keystone-tempest-plugin — branch master

Branch master @ 41e16f7f319c (0.22.0 + 3 commits) — repository https://opendev.org/openstack/keystone-tempest-plugin — analysed on 2026-10-28 Run against Keystone stable/2026.2 @ e6dc5ae079e6 with Tempest 47.0.0

Scope: the whole plugin (63 files): registration and options, clients, and the RBAC, API and scenario tests.
References are file:line relative to the repository root. keystone: points to the Keystone repository, tempest: to Tempest 47.0.0. [inferred] marks a statement that was not followed to the end in the code.

Files of this specification

File	Content
01-test-suite_master.md	plugin loading and options, the 3 test families, the 11 personas, `do_request`
02-errors_master.md	errors the tests expect (2,685 × 403, 337 × 404), 403 versus 404, the 14 skipped tests
03-interactions_master.md	Keystone, external IdP, K2K; shape of the traffic
04-scenarios_master.md	3 diagrams: persona class lifecycle, one RBAC assertion, the conformance run
05-async-processing_master.md	none
06-change-history_master.md	`0.22.0..master`: removal of cross-domain grant tests (`d5dcf00`), stable/2026.2 job
07-rewrite-pack_master.md	conformance suite for OpenKeel Keystone: how to run it, reference results (1,966/1,980, 4,372 RBAC assertions without difference), RBAC matrix requirements, coverage gaps

Overview

keystone-tempest-plugin is Keystone's black-box test suite: it only drives Keystone through its public API.

Its RBAC tests run the same scenario under 11 personas (system, domain or project scope; admin, manager, member or reader role). For each one, they assert the status Keystone must return. Together they form the executable matrix of Keystone's default policies.
Its API tests cover the federation and OAuth1 APIs.
Its scenarios cover federated login (Keycloak, K2K).

Key figures

1,980 tests (1,966 passed, 14 skipped by design, 0 failure); 17,873 exchanges; 4,372 RBAC assertions; 187 operations × 11 personas; 196 of Keystone's 259 routes exercised.

Points of attention

**An admin role on a project, or on a domain, is a cloud administrator**: project_admin matches system_admin on 179 of 181 operations (role:admin without a scope condition). A rewrite must reproduce it to pass, even though it is surprising from a security point of view (07 § 3).
**The manager role depends on the target**: the same call is allowed in one's own domain or project and refused elsewhere. A rewrite must evaluate the target, not only the role.
403 or 404 for a missing target depends on whether the policy looks at the target (02 § 2).
Cross-domain grants of domain-specific roles are no longer tested (d5dcf00). Keystone stable/2026.2 accepts them and master refuses them (keystone:fa43c4b3d). OpenKeel should refuse them and add its own test (06).
63 Keystone routes are not covered: OS-INHERIT, federated login, OAuth1 delegation, password change, /v3/auth/* introspection… (07 § 5).
**Only statuses are checked by do_request**; bodies are checked by Tempest's response schemas and by a few explicit assertIn checks.

Source: reverse-spec-keystone-tempest-plugin/00-summary_master.md. The _rewrite/ and _inventory/ files are not published on the site.