skyline-apiserver — Summary

Reverse-specification of skyline-apiserver — branch stable/2026.2

Branch stable/2026.2 @ 4af72bfd4577 (tag 9.0.0) — repository https://opendev.org/openstack/skyline-apiserver — analysed on 2026-10-28

• Repository: https://opendev.org/openstack/skyline-apiserver — specified branch: stable/2026.2 @ 4af72bfd4577d115af29e428cd0affa116e22e70 — analysed on: 2026-10-28
• Scope: the whole project: the FastAPI API server (26 endpoints), its database, its outbound REST calls (Keystone, Nova, Cinder, Glance, Neutron, Prometheus), the policy engine it embeds, the CLI tools (skyline-nginx-generator, skyline-policy-manager, skyline-swagger-generator, skyline-sample-config-generator) and the deployment glue (nginx template, gunicorn, container start script). Rewrite pack (step 8) included; change history (step 7) not requested.
• Client repository cited: skyline-console stable/2026.2 @ 806d2003 (prefix skyline-console:), and cinder stable/2026.2 @ c117f9cf0 (prefix cinder:) for one wire detail.

Overview

skyline-apiserver is the back end of the Skyline dashboard (skyline-console). It is not the data path for most of the UI: the console talks to Nova, Neutron, Cinder, Glance… directly, through an nginx reverse proxy generated from the Keystone catalog (/api/openstack/<region>/<service>/, skyline_apiserver/templates/nginx.conf.j2:111), with the raw Keystone token it obtained from skyline-apiserver. The API server itself does four things:

1. Session management: password, TOTP (two-step with a Keystone receipt) and WebSSO login against Keystone; it returns the project-scoped Keystone token in the body and a signed HS256 JWT in the session cookie, switches project or region by re-scoping the token, revokes it at logout (skyline_apiserver/api/v1/login.py:395).
2. Aggregated list views (/extension/*) that join Nova, Cinder, Glance, Neutron and Keystone data into one paginated list, partly with the privileged system session of the skyline service user (skyline_apiserver/api/v1/extension.py:62).
3. Policy evaluation for the UI: it embeds the default policy rules of 16 OpenStack services (1 860 rules, overridable by files) and answers "which actions is this user allowed?" so that the console can hide buttons (skyline_apiserver/api/v1/policy.py:96).
4. A small settings store and a Prometheus proxy filtered by project (skyline_apiserver/api/v1/setting.py:52, skyline_apiserver/api/v1/prometheus.py:112).

It has no message bus, no periodic task and no worker. Its own database has two tables: revoked session IDs and setting overrides (skyline_apiserver/db/models.py:22).

Architecture

flowchart LR
  browser[Browser: skyline-console SPA] -->|HTTPS| nginx[nginx :9999]
  nginx -->|static files| static[(skyline_console/static)]
  nginx -->|/api/openstack/skyline/ → unix socket| api[skyline-apiserver<br/>gunicorn + uvicorn workers]
  nginx -->|/api/openstack/&lt;region&gt;/&lt;service&gt;/<br/>X-Auth-Token from the browser| os[OpenStack APIs<br/>Nova, Neutron, Cinder, Glance…]
  api -->|REST: auth, token validation, catalog| ks[Keystone]
  api -->|REST: list servers, services| nova[Nova]
  api -->|REST: volumes, snapshots| cinder[Cinder]
  api -->|REST: images| glance[Glance]
  api -->|REST: networks, ports, extensions| neutron[Neutron]
  api -->|HTTP GET /api/v1/query*| prom[Prometheus]
  api -->|SQL| db[(SQLite or MariaDB:<br/>revoked_token, settings)]
  api -->|reads| files[/etc/skyline/skyline.yaml<br/>/etc/skyline/policy/*_policy.yaml/]

Processes

External dependencies

Key figures

• 26 HTTP operations under /api/v1 (+ 4 FastAPI documentation routes: /docs, /docs/oauth2-redirect, /redoc, /api/v1/openapi.json).
• 0 RPC methods, 0 notifications, 0 periodic tasks, 0 bus consumers.
• 2 tables (revoked_token, settings), 2 Alembic revisions (000, 001).
• 0 exception classes of its own used by the API: every error is a FastAPI HTTPException with {"detail": …} (02).
• 1 860 embedded policy rules for 16 services (barbican, cinder, designate, glance, heat, ironic, keystone, magnum, manila, masakari, neutron, nova, octavia, placement, trove, zun).
• 45 recorded scenarios issuing 222 outbound requests (_rewrite/outbound-calls_stable-2026.2.md); 115/115 unit tests pass.

Points of attention

1. **GET /extension/recycle_servers returns 500 as soon as one soft-deleted server exists** [verified by capture]: each server is validated into RecycleServersResponseBase at skyline_apiserver/api/v1/extension.py:405, whose reclaim_timestamp is required (skyline_apiserver/schemas/extension.py:292), but that field is only computed later at skyline_apiserver/api/v1/extension.py:457. The unit test mocks the model and does not see it.
2. Response models require UUID v4 identifiers for servers, images and volumes (skyline_apiserver/schemas/extension.py:206, skyline_apiserver/schemas/extension.py:211, skyline_apiserver/schemas/extension.py:338): a resource with a non-v4 UUID makes the whole list fail with 500 [verified by capture with non-UUID data].
3. CORS cannot be enabled: the middleware is added inside the lifespan hook (skyline_apiserver/main.py:56), which Starlette 1.x refuses with RuntimeError: Cannot add middleware after an application has started [verified with the stable/2026.2 upper-constraints]. Setting cors_allow_origins therefore prevents start-up.
4. Every authenticated call costs at least two Keystone round trips (version discovery + GET /v3/auth/tokens validation, skyline_apiserver/core/security.py:66), plus one DELETE on the database (skyline_apiserver/main.py:117), plus a re-scoping POST /v3/auth/tokens for every endpoint that builds a user session (skyline_apiserver/client/utils.py:54). Login costs 16 outbound requests (03).
5. The middleware turns every failure into 401 (skyline_apiserver/main.py:172): a Keystone outage or a database error is reported as Unauthorized: …, which makes the console log the user out (skyline-console:src/client/client/request.js:42).
6. **Hard-coded default secret_key** (skyline_apiserver/config/default.py:62, repeated in etc/skyline.yaml.sample:18). The JWT only wraps a Keystone token that is re-validated on each request, so a forged JWT gives no access by itself, but it defeats the session revocation list (a new uuid can be minted) and the cookie expiry.
7. The raw Keystone token is returned in every profile (keystone_token, skyline_apiserver/schemas/login.py:69) and kept by the console in localStorage (skyline-console:src/stores/root.js:119); the session cookie is set without HttpOnly or Secure (skyline_apiserver/api/v1/login.py:172, Starlette defaults). Logout deletes session but not time_expired (skyline_apiserver/api/v1/login.py:684).
8. Tenant isolation done by Skyline, not by the service, wherever the system session is used: recycle-bin listing (skyline_apiserver/api/v1/extension.py:374), volume and snapshot lookups by ID (skyline_apiserver/api/v1/extension.py:586, skyline_apiserver/api/v1/extension.py:743), image and root-volume enrichment (skyline_apiserver/api/v1/extension.py:203). The skyline user must be admin in Nova, Cinder and Glance for those views to work.
9. Database "transactions" are not transactions: Transaction opens a session that the queries do not use, each query opens its own session (skyline_apiserver/db/base.py:36, skyline_apiserver/db/base.py:40), so the SELECT … FOR UPDATE of update_setting does not hold any lock (skyline_apiserver/db/api.py:95) [inferred: two concurrent first writes of a key can collide on the primary key].
10. **PUT /setting stores any JSON value**: the value is not validated against the option's schema (skyline_apiserver/api/v1/setting.py:90).
11. Prometheus: TLS verification is disabled (skyline_apiserver/utils/httpclient.py:28); any PromQL is accepted; non-admin users only see series that carry a project_id label equal to their project (skyline_apiserver/api/v1/prometheus.py:50).
12. **Cinder id/snapshot_id filters are sent as Python list literals** (id=['<uuid>']), which Cinder accepts only because it runs ast.literal_eval on filter values (cinder:cinder/volume/api.py:2369) [verified by capture]. A re-implemented Cinder must keep that behaviour, or Skyline's batching breaks.
13. **switch_project access check is dead code**: the profile built by the middleware never carries projects (skyline_apiserver/core/security.py:73), so the check at skyline_apiserver/api/v1/login.py:712 never applies; Keystone's own refusal to re-scope is what protects (→ 401).
14. **Log messages with %s**: the code logs with %-style arguments through loguru, which does not interpolate them (skyline_apiserver/api/v1/login.py:113, skyline_apiserver/api/v1/login.py:136); the log shows TOTP authentication failed for user %s in domain %s: %s [verified in the capture log].

Blind spots / [inferred]

• The behaviour against a real OpenStack was not run: outbound calls were recorded against a fake (_rewrite/tools/outbound_capture.py) that answers with minimal bodies; error bodies of real services, pagination over several pages and Keystone policies (who may list projects, rescope, read system scope) are therefore [inferred] from the client libraries.
• The 16 policy modules under skyline_apiserver/policy/manager/ are generated copies of the services' defaults; they were dumped (_rewrite/policies_stable-2026.2.json) but not compared rule by rule with each service's stable/2026.2.
• skyline-policy-manager generate-rule and validate need the services' Python packages installed; not run.
• The websso path was exercised with an already-issued unscoped token; the federation round trip through the IdP (Keystone /auth/OS-FEDERATION/.../websso) was not.
• skyline_apiserver/context.py:28 (RequestContext) is built for every request but its authorize method is never called by any endpoint: dead code on this branch.

Source: reverse-spec-skyline-apiserver/00-summary_stable-2026.2.md. The _rewrite/ and _inventory/ files are not published on the site.