keystone-tempest-plugin — Test suite

Test suite — keystone-tempest-plugin — branch master

Branch master @ 41e16f7f319c (0.22.0 + 3 commits) — repository https://opendev.org/openstack/keystone-tempest-plugin — analysed on 2026-10-28 The plugin is "branchless": the same master tests every supported Keystone release. Here it was run against Keystone stable/2026.2 @ e6dc5ae079e6.

keystone-tempest-plugin exposes no service. It is a Tempest plugin: a set of black-box functional tests that drive Keystone through its public API only. Its "interface" is therefore its tests, and its contract is the behaviour it expects from Keystone. The 07 file turns this into a conformance suite for OpenKeel Keystone.

1. Loading and configuration

• Registration: the tempest.test_plugins entry point (keystone_tests, setup.cfg:28-29) points to KeystoneTempestPlugin (keystone_tempest_plugin/plugin.py:25-44). It adds keystone_tempest_plugin/tests to Tempest's discovery, and registers two option groups:
  • [identity-feature-enabled]: federation (default false), external_idp (default true), and enforce_scope (deprecated in favour of [enforce_scope] keystone) (keystone_tempest_plugin/config.py:17-38);
  • [fed_scenario]: the external identity provider (idp_id, idp_ecp_url, idp_oidc_url, client and user credentials), the mapping (mapping_remote_type, mapping_user_name, mapping_group_name…) and protocol_id (keystone_tempest_plugin/config.py:40-94).
• Clients: Tempest's identity v3 clients, plus those of the plugin for federation (identity_providers, mapping_rules, service_providers, saml2, auth) and limits (keystone_tempest_plugin/services/identity/v3/, keystone_tempest_plugin/clients.py:28).
• Credentials: Tempest creates dynamic credentials (projects, domains, users and role assignments) for each test class, as requested by its credentials attribute.

2. The three families of tests

RBAC tests: the default policy matrix

Each module (keystone_tempest_plugin/tests/rbac/v3/test_project.py, keystone_tempest_plugin/tests/rbac/v3/test_user.py, keystone_tempest_plugin/tests/rbac/v3/test_grant.py, …) defines an abstract class IdentityV3Rbac…Tests, then one class per persona. These classes change the expected statuses (keystone_tempest_plugin/tests/rbac/v3/test_project.py:24-667).

The 11 personas are:

• system_admin, system_member, system_reader;
• domain_admin, domain_manager, domain_member, domain_reader;
• project_admin, project_manager, project_member, project_reader.

Every check goes through do_request(method, expected_status) (keystone_tempest_plugin/tests/rbac/v3/base.py:34-44):

• an int means the status of a successful response is compared;
• an exception class (Forbidden, NotFound) means the call must fail with that error.

The tests thus encode, persona by persona, what the default policies of Keystone allow: system-scoped tokens, domain managers, project managers (manager role).

3. What runs where

• In the OpenStack gate: devstack plus Keystone configured with enforce_scope and enforce_new_defaults; jobs per release (the stable/2026.2 job was added on this branch, see 06).
• Here: Tempest in a subprocess, against a real Keystone stable/2026.2 served locally (adapter keystone_tempest_plugin.py of the generate-reverse-spec skill). Full run:
  • 1,980 tests: 1,936 RBAC (the 674 functions, inherited by the persona classes), 38 API and 6 scenario;
  • 1,966 passed, 14 skipped by design (8 application-credential creations for personas without a project, 6 federation scenarios), 0 failure;
  • 17,873 exchanges and 4,372 RBAC assertions. Details in 07.

Source: reverse-spec-keystone-tempest-plugin/01-test-suite_master.md. The _rewrite/ and _inventory/ files are not published on the site.