masakari — Interactions

Interactions — masakari — branch stable/2026.2

Branch stable/2026.2 @ ae41528e1ebc — repository https://opendev.org/openstack/masakari — analysed on 2026-10-28

Outbound REST calls

Authentication

novaclient() (masakari/compute/nova.py:86) builds a new keystoneauth session for every Nova call, with password auth for os_privileged_user_name/os_privileged_user_password in project os_privileged_user_tenant (or system_scope). Each call therefore costs a POST /v3/auth/tokens [inferred: no session or token cache is kept between calls]. The Nova endpoint is the catalogue entry compute/nova/publicURL of os_region_name, unless the request context already carries a catalogue. The X-OpenStack-Request-ID of the incoming request is passed as global_request_id. No timeout is set (timeout=None, masakari/compute/nova.py:131), so a hung Nova blocks a green thread until TCP gives up.

Calls to Nova

All calls carry X-Auth-Token, X-OpenStack-Nova-API-Version: 2.53 and OpenStack-API-Version: compute 2.53. Recorded with the real novaclient in [_rewrite/nova-calls_stable-2026.2.json](_rewrite/nova-calls_stable-2026.2.json).

Error translation (translate_nova_exception, masakari/compute/nova.py:52):

• Timeout, CommandError, keystoneauth ConnectionError, DiscoveryFailure → MasakariException;
• 400 → InvalidInput;
• 403 → Forbidden;
• 404 → NotFound;
• 409 → Conflict;
• **a 5xx (ClientException) is not translated** and escapes as a novaclient exception (observed: API 500 "…<class 'novaclient.exceptions.ClientException'>").

Sample frame (host failure, auto method), as recorded:

PUT /v2.1/os-services/9811decb-80f0-5552-99d1-1d2f6f5bea9a HTTP/1.1
X-Auth-Token: <service token>
X-OpenStack-Nova-API-Version: 2.53
Content-Type: application/json

{"status": "disabled", "disabled_reason": "Masakari detected host failed."}

POST /v2.1/servers/aa7ba2b1-b24c-5227-90dc-49925583e0f3/action HTTP/1.1
X-OpenStack-Nova-API-Version: 2.53

{"evacuate": {}}

RabbitMQ

Topology

RPC methods

Objects travel as oslo.versionedobjects primitives through MasakariObjectSerializer. The request context is serialised with every message, including auth_token, service_catalog, roles, user_id, project_id, request_id and global_request_id (context_keys in [_rewrite/rpc-frames_stable-2026.2.json](_rewrite/rpc-frames_stable-2026.2.json)).

Sample frame (process_notification, oslo.messaging message body, as recorded, timestamps shortened):

{"method": "process_notification", "version": "1.0",
 "args": {"notification": {
   "masakari_object.name": "Notification", "masakari_object.namespace": "masakari",
   "masakari_object.version": "1.3",
   "masakari_object.data": {"id": 1, "notification_uuid": "7882ccb9-ebb1-43b3-848b-f2333ae4b79c",
     "generated_time": "2026-10-28T10:00:00Z", "type": "PROCESS", "status": "new",
     "payload": {"event": "STARTED", "process_name": "nova-compute"},
     "source_host_uuid": "e3fe016c-f32c-4971-8e7f-8ca518e30f72",
     "failover_segment_uuid": "44f3f05c-8e15-4fac-9e0b-994d3c1f4991", "message": null,
     "created_at": "2026-10-28T22:43:57Z", "updated_at": null, "deleted": false, "deleted_at": null}}},
 "_context_auth_token": "…", "_context_user_id": "…", "_context_project_id": "…",
 "_context_request_id": "req-…", "_context_roles": ["admin"]}

Versioned notifications

They are emitted only when [oslo_messaging_notifications]/driver is set (the oslo default is none, which makes them no-ops). They use topic versioned_notifications (masakari/rpc.py:57), publisher_id masakari-api:<hostname> or masakari-engine:<hostname>, and priority INFO (or ERROR with a fault payload). Samples are in doc/notification_samples/.

Host updates made by the engine (on_maintenance, reserved) go through Host.save() and therefore also emit host.update.start/end, with publisher masakari-api (the default binary of masakari/api/utils.py:63) even though the engine sends them.

Database

Tables (main database)

The schema is identical between the models and the alembic migrations ([_rewrite/schema-diff_stable-2026.2.md](_rewrite/schema-diff_stable-2026.2.md)). All tables are soft-deleted (deleted = id when deleted, oslo.db SoftDeleteMixin).

Taskflow persistence database

Tables logbooks, flowdetails, atomdetails (and their own alembic_version) are created by TaskFlowDriver.upgrade_backend (masakari/engine/drivers/taskflow/driver.py:250) ([_rewrite/schema-sqlite-taskflow_stable-2026.2.sql](_rewrite/schema-sqlite-taskflow_stable-2026.2.sql)). There is one logbook per notification, named after the notification UUID (masakari/engine/drivers/taskflow/base.py:132), one flow detail per run (retries add flow details), and one atom detail per task, whose meta holds progress and progress_details (written by update_details, masakari/engine/drivers/taskflow/base.py:58). Nothing ever purges this database.

Notable transactions

• Every DB API function is its own transaction (@context_manager.reader/writer), retried 5 times on deadlock (masakari/db/sqlalchemy/api.py:171). There is **no SELECT … FOR UPDATE** and no row versioning.
• The duplicate check (SELECT then INSERT) is not atomic. It relies on the tooz lock (COMPUTE_HOST only) or on the random sleep.
• The "in use" check of segment/host update and delete is a separate SELECT COUNT(*) before the write (masakari/db/sqlalchemy/api.py:304): racy, but harmless.
• Segment delete soft-deletes the segment and its hosts in one transaction (masakari/db/sqlalchemy/api.py:289).

Representative queries (reconstructed):

-- duplicate detection (masakari/ha/api.py:277)
SELECT * FROM notifications WHERE deleted = 0 AND type = :type AND source_host_uuid = :host
  AND generated_time >= :generated_time - INTERVAL 180 SECOND ORDER BY created_at DESC, id DESC;
-- segment under recovery (masakari/db/sqlalchemy/api.py:304)
SELECT count(id) FROM notifications WHERE deleted = 0 AND status IN ('new','running','error')
  AND source_host_uuid IN (SELECT uuid FROM hosts WHERE deleted = 0 AND failover_segment_id = :seg);
-- reserved hosts (masakari/engine/manager.py:223)
SELECT * FROM hosts WHERE deleted = 0 AND failover_segment_id = :seg AND reserved = 1 AND on_maintenance = 0
  ORDER BY created_at DESC, id DESC;

Other

• No subprocess, rootwrap or privsep. The engine needs no access to the compute nodes: everything goes through Nova.
• GET /healthcheck (oslo.middleware, disable_by_file backend, /etc/masakari/healthcheck_disable) is served by the API paste composite.

Source: reverse-spec-masakari/03-interactions_stable-2026.2.md. The _rewrite/ and _inventory/ files are not published on the site.