Instance high availability; node health from Node Healthcheck and Self Node Remediation.
Go Medik8sopenkeel-masakari
The module
OpenKeel implementation in Go of OpenStack Masakari, the instance high-availability service (masakari-api, masakari-engine, masakari-manage), generated from the reverse-specification of branch stable/2026.2 by the generate-go-from-spec skill. It replaces the Python service without its clients noticing: same routes, status codes, error bodies, microversions and policies. What is reproduced and what differs is in docs/CONFORMANCE.md; the security checklist is in docs/SECURITY.md.
Go module: github.com/openkeel/openkeel-masakari
Binaries (one image):
masakari-api: the instance-ha API v1, microversions 1.0–1.3 (port 15868; health and metrics on 9090);
masakari-engine: claims new notifications and runs the recovery workflows (host evacuation by segment recovery method, instance restart, compute-service disable) through Nova (openkeel-nova), runs the periodic tasks on the Lease holder and, in a cluster, the Medik8s/KubeVirt integration below;
masakari-manage: db sync, db version, db purge.
Kubernetes-native HA (internal/kube, [references/backends/masakari-kubevirt.md] of the skill and openkeel-docs/NODE-PLACEMENT.md § 5.6 of the workspace): one Medik8s NodeHealthCheck per segment detects and fences failed nodes (Self Node Remediation, OutOfServiceTaint); the engine reports the host failure only once the node is fenced and evacuates its VMs per segment policy; it live-migrates the VMs of degraded or drained nodes before they fail, and mirrors segments and reserved hosts on node labels and taints.
Configuration: YAML, see config/config.example.yaml
Setting up a development environment, the database and the tests: docs/GETTING-STARTED.md
Database schema, migrations and upgrades: docs/DATABASE.md
Excerpt from openkeel-masakari/README.md (in English).
REST API
21 operations of the original REST API (routes and body actions), and their state in OpenKeel.
Python Masakari's recorded scenarios are replayed against the Go service, the recovery engine is compared with the Nova calls Python made, and concurrency is tested on Galera.
Replay of recorded exchanges
E2E replay of recorded scenarios
The 201 exchanges of 23 ordered scenarios, with the same fake Nova and periodic tasks, give the same answers as Python.
How it runs
make e2e, on go-mysql-server and on MariaDB 11.4 (E2E_MYSQL_DSN); make e2e-strict passes.
Latest result 195 / 201 on both engines; 6 deliberate differences listed in test/e2e/known-differences.txt.
Source: openkeel-masakari/docs/CONFORMANCE.md
Client contract
Engine against recorded Nova calls
Host and instance recovery flows send Nova the same mutations, per server, service and aggregate, as the Python engine.
How it runs
internal/engine/engine_test.go on nova-calls_stable-2026.2.json (143 requests).
Latest result Identical mutations per server, service and aggregate; GET polling counts are not compared.
Source: openkeel-masakari/docs/CONFORMANCE.md
Unit and golden tests
Unit and policy tests
API routing, microversions, validation messages and the 24 policy rules for two personas behave as in Python.
How it runs
make test, with client-go fakes for the Kubernetes integration.
Latest result Pass.
Source: openkeel-masakari/docs/CONFORMANCE.md
Concurrency on Galera
Concurrency on MariaDB Galera
Unique names and duplicate notifications hold through three nodes, and engines on three nodes evacuate a host only once.
Masakari functional tests (openstacksdk) and Tempest-like tests against a deployed service.
How it runs
Need a deployed cluster.
Latest result Not run.
Source: openkeel-masakari/docs/CONFORMANCE.md
Left out of the rewrite
Masakari keeps its API and recovery workflows but detects host failures with Medik8s instead of masakari-monitors and replaces RPC, taskflow and tooz with database claims.
Done by the Cloud Native platform
masakari-monitors host monitor
Taken over by Medik8s
Why it was left out
Kubernetes already detects unhealthy nodes: one NodeHealthCheck per segment, with Self Node Remediation or Fence Agents Remediation, fences the node.
Why the solution is not affected
The engine raises the same COMPUTE_HOST notification once the node is fenced, so evacuation works as before. The instance and process monitors are not replaced yet.
Source: openkeel-masakari/docs/CONFORMANCE.md
Internal machinery simplified
oslo.messaging RPC and tooz coordination
Taken over by Kubernetes Leases
Why it was left out
The API inserts the notification row and engine replicas claim it with a compare-and-swap; duplicates and per-host work are serialised by database rows that work across Galera nodes.
Why the solution is not affected
No RabbitMQ is needed and no cast can be lost; the Galera test gives exactly one 202 for 24 identical concurrent reports.
Source: openkeel-masakari/docs/CONFORMANCE.md
Internal machinery simplified
Taskflow logbook and custom recovery flows
Why it was left out
Progress is a table of the single database instead of a second taskflow database; Go has no plugin loading, so the default flow lists are built in.
Why the solution is not affected
recovery_workflow_details returns the same progress list. Custom task lists and third-party stevedore tasks are not supported.
Source: openkeel-masakari/docs/CONFORMANCE.md
Done by the Cloud Native platform
WSGI server options, healthcheck, osapi_v1 filters
Taken over by Kubernetes probes
Why it was left out
A Go HTTP server replaces the WSGI stack, Kubernetes manages processes and probes, and the osapi_v1 options are deprecated upstream.
Why the solution is not affected
The API and its microversions 1.0 to 1.3 are unchanged.