Reverse-specification of heat — branch stable/2026.2
- Repository: https://opendev.org/openstack/heat — specified branch:
stable/2026.2@5872396e548c(27.0.1) — analysed on: 2026-10-29 - Scope: the whole service — heat-api (native Orchestration API v1), heat-api-cfn (CloudFormation query API and signal endpoints), heat-engine (RPC API, convergence engine, legacy engine, timers), heat-manage; template formats and intrinsic functions; the resource-plugin registry; software config, signals, wait conditions; Keystone trusts and stack domain users. Rewrite pack (step 8) for
openkeel-heat, with client expectations for python-heatclient/OSC, Skyline, Terraform (gophercloud), Magnum and Octavia. Change history (step 7): 3 commits since the branch point, one security fix.
| File | Content |
|---|---|
| 01-rest-api_stable-2026.2.md | native and CFN APIs, functional and technical sheets, stack and resource lifecycles |
| 02-errors_stable-2026.2.md | exception → HTTP translation (native, CFN), catalogue of 73 exceptions, asynchronous errors, logs |
| 03-interactions_stable-2026.2.md | RabbitMQ topics and every RPC method, notifications, frames, tables and CAS statements, encryption, Keystone, other services |
| 04-scenarios_stable-2026.2.md | create, concurrent update, rollback, delete during create, wait condition, software deployment |
| 05-async-processing_stable-2026.2.md | convergence engine (graph, sync points, locks, cancellation, failure), legacy engine, timers, signals |
| 06-change-history_stable-2026.2.md | branch history: one security backport (software config access on deployments), carry-over plan |
| 07-rewrite-pack_stable-2026.2.md | rewrite pack: template parser, convergence invariants, resuming Python stacks, resource types to port, client expectations, conformance |
| _verification_stable-2026.2.md | output of check_refs.py --context |
| _inventory/ | raw inventory |
| _rewrite/ | machine-readable contracts and conformance data |
Overview
Heat turns declarative templates (HOT YAML, or AWS CloudFormation JSON/YAML) into running OpenStack resources and keeps them in line with the template over updates. Two stateless API processes translate HTTP into RPC calls to heat-engine, which owns all the logic and the database. Since Newton the default engine is convergence: each operation becomes a traversal of a dependency graph stored in the database, whose nodes (create/update or clean up one resource) are processed by any engine through check_resource casts and joined through sync points; locks are optimistic (compare-and-swap on traversal ids and per-row atomic_key). Resources call the other services with the user's token, and later with a Keystone trust that Heat creates for each stack; instances call back with credentials of throw-away users in a dedicated Keystone domain.
Architecture
flowchart LR
user[Clients: heatclient/OSC, Skyline, Terraform] -->|HTTPS REST /v1| api[heat-api]
agent[In-instance agents: cfn-signal, os-collect-config] -->|HTTPS query + EC2 signature| cfn[heat-api-cfn]
agent -->|HTTPS REST| api
api -->|AMQP call, topic engine| mq{{RabbitMQ}}
cfn -->|AMQP call, topic engine| mq
mq --> eng[heat-engine xN]
eng -->|AMQP cast check_resource, topic engine_worker| mq
eng -->|AMQP call listening, topic heat-engine-listener| mq
eng -->|SQL| db[(MariaDB)]
api -->|REST token validation| ks[Keystone]
cfn -->|REST ec2tokens| ks
eng -->|REST trusts, domain users, ec2 creds| ks
eng -->|REST via client plugins| svc[Nova, Neutron, Cinder, Glance, Swift, Barbican, Manila, Designate, Ironic, Octavia…]
eng -->|notifications| mqProcesses
| Process | Entry point | Role | Listens to / consumes |
|---|---|---|---|
| heat-api | heat.api.openstack.wsgi:init_application (setup.cfg wsgi_scripts) | native REST API | HTTP 8004; publishes RPC engine |
| heat-api-cfn | heat.api.cfn.wsgi:init_application | CFN query API, wait-condition and signal URLs | HTTP 8000; publishes RPC engine |
| heat-engine | heat.cmd.engine:main | all logic, one EngineService per worker process | RPC engine (server = host), engine_worker and heat-engine-listener (server = engine id); timer 60 s |
| heat-manage | heat.cmd.manage:main | database sync and maintenance, stack reset, convergence migration, service list | — |
| heat-status | heat.cmd.status:main | upgrade checks (oslo.upgradecheck) | — |
External dependencies
| System | Protocol | Library | Configuration | Usage |
|---|---|---|---|---|
| RabbitMQ | AMQP 0-9-1 | oslo.messaging | transport_url | API→engine calls, engine↔engine casts and liveness, notifications |
| MariaDB | SQL | oslo.db / SQLAlchemy | [database] | 16 tables; engine and heat-manage only |
| Keystone | REST | keystonemiddleware, keystoneauth1, keystoneclient | [keystone_authtoken], [trustee], stack_user_domain_*, [ec2authtoken] | tokens, trusts, stack domain users and projects, EC2 credentials |
| OpenStack services | REST | 20 client plugins | [clients_*] | resource plugins |
| Swift | REST, TempURL | swiftclient | catalog | files_container, signal and metadata TempURLs |
| HTTP servers | HTTP GET | requests | template_fetch_timeout | template_url, child templates |
| Memcached | memcache | oslo.cache | [cache] | lookup caches (optional) |
Key figures
- Native API: 40 paths, 61 operations (+ 405/
OPTIONSroutes); CFN API: 13 actions, 2 signal routes, 16 undocumented REST-style routes; 58 operations documented in api-ref. - RPC: 54 API→engine methods (engine API 1.36), 3 worker casts (1.9), 1 liveness call; 6 public server methods called by no client.
- Non-HTTP triggers: 3 RPC servers, 1 periodic timer, 2 startup tasks, signal threads, deployment polling.
- 75 exception classes (
heat/common/exception.py), 100 default policy rules, 45 configuration sections. - 16 tables, Alembic head
97b2a986f922, models = migrations. - 201 resource types: 95 to port to OpenKeel services, 33 Heat-internal, 12 Octavia (conditional), 22 of absent services, 38 already hidden, 1 test-only; 13 template classes (CFN 2010-09-09, Heat CFN 2012-12-12, HOT 2013-05-23 → 2021-04-16), 33 intrinsic function names, 70 custom constraints.
- Conformance (07 § 7): 322 functional and API tests run against the real heat-api/heat-api-cfn and engine — 220 ok, 76 skipped (services absent from the test bed), 26 failed or errored (test-bed limits) — 6681 HTTP exchanges recorded; convergence tables captured in 33 states over 7 scenarios.
Points of attention
- Credentials on the bus: every RPC message carries the caller's token and full token body; with
deferred_auth_method = password, the clear-text password (03 § 1.4). - No automatic resume after an engine crash in convergence: the stack stays
IN_PROGRESSuntil timeout or a new user action (05 § 2.6). - Sync-point keys are Python tuple reprs (
"tuple:(42, True, 'resource')") and resource locks rely on a 2-second RPC liveness probe: both bind any other implementation sharing the database (07 § 3.3). - Status by exception class name: several runtime errors surface as 500 (
ActionNotComplete,InvalidTemplateAttribute,ResourceActionRestricted…); unknown resource types give 404 onGET /resource_types/Xbecause of a mis-targetedexcept(02). resource:signalneeds onlyreader;stacks:global_indexis denied by default, which breaks Skyline's admin "all projects" stack list without a policy override.- Security fix in 27.0.1: before it, a software deployment could reference another project's software config; now 404 (06).
[DEFAULT]/debug = Truereturns engine tracebacks in API error bodies (heat/api/middleware/fault.py:119-147).template_urlis fetched by heat-api from any http(s) URL (SSRF surface), bounded bymax_template_sizeandtemplate_fetch_timeout.- Public engine RPC methods meant for internal use (
reset_stack_status,stop_traversal…) are reachable by any bus client (DefaultRPCAccessPolicy). - CFN error mapping uses substring tests for two categories (
heat/api/aws/exception.py:315-317). - The default
auth_encryption_keyis a well-known string (heat/common/crypt.py): deployments must set it, and keep it — changing it makes stored trusts unreadable (InvalidEncryptionKey).
Blind spots / [inferred]
- Resource plugins are described as a registry (schemas dumped from the venv), not plugin by plugin: the
handle_*code of the 95 types to port was not read; per-plugin behaviour (update-in-place vs replace beyondupdate_allowed, polling, error states) must be taken from their source when porting. - The legacy (non-convergence) engine is described at the level of locks and threads only.
- HOT/CFN function semantics are summarised per version; their edge cases are in Heat's unit tests, not converted to vectors.
- YAML loader limits, OSC event polling and the exact 2- vs 3-element sync-point key mismatch across upgrades are marked [inferred].
- contrib/ plugins (Docker, Rackspace) out of scope.
Source: reverse-spec-heat/00-summary_stable-2026.2.md. The _rewrite/ and _inventory/ files are not published on the site.