cloudkitty — Summary

Reverse-specification of CloudKitty — branch stable/2026.2

Branch stable/2026.2 @ 0685cc330e83 (25.0.0-2-g0685cc3) — repository https://opendev.org/openstack/cloudkitty (package cloudkitty) — analysed on 2026-11-09

• Scope: all non-test code (~19,000 lines): the WSGI API (v1 Pecan/WSME, v2 Flask), the cloudkitty-processor (orchestrator, collectors, fetchers, rating modules, reprocessing), the storage back ends (v2: InfluxDB 1/2, Elasticsearch, OpenSearch, Loki; v1: SQLAlchemy, hybrid Gnocchi), the state tables and the CLIs. The change history covers the 2026.2 cycle (24.0.0..stable/2026.2, file 06); the rewrite pack (step 8) was added on 2026-11-09: 07-rewrite-pack_stable-2026.2.md and _rewrite/.
• References are file:line relative to the repository root. [inferred] marks a statement that was not followed through to the end in the code (most often: behaviour of a third-party library such as WSME, flask-restful or a storage server).
• Reading angle: OpenKeel is weighing CloudKitty against its own openkeel-pricing service (cost and carbon footprint per project). Section Points of attention and file 03 call out what matters for that comparison.

Files of this specification

Overview

CloudKitty is OpenStack's rating service: it turns usage into prices. It does not meter anything itself and it does not invoice:

1. A fetcher lists the scopes to rate — by default the Keystone projects on which the CloudKitty user holds the rating role (cloudkitty/fetcher/keystone.py:84-105).
2. For each scope and each period ([collect]/period, 3600 s by default, cloudkitty/collector/__init__.py:43-45), a collector queries a metrics back end — Gnocchi, Prometheus or Aetos — for every metric declared in the metric file ([collect]/metrics_conf, sample etc/cloudkitty/metrics.yml) (cloudkitty/orchestrator.py:418-443).
3. The collected DataFrame (one list of DataPoint per metric) goes through the enabled rating modules in decreasing priority order (cloudkitty/orchestrator.py:257-265, cloudkitty/orchestrator.py:509-521): hashmap (declarative price table), pyscripts (arbitrary Python executed in the processor), noop.
4. The rated frame is pushed to a storage back end (InfluxDB by default, cloudkitty/storage/__init__.py:29-36) and the scope's last_processed_timestamp advances in SQL (cloudkitty/orchestrator.py:523-524).
5. The API exposes the rated data (/v2/summary, /v2/dataframes, /v1/report/*, /v1/storage/dataframes), the rating rules (/v1/rating/module_config/hashmap|pyscripts/...), instant quotes (POST /v1/rating/quote, an RPC call to a processor) and operational controls (scopes, reprocessing).

Everything is computed per period, after the fact, with a lag of wait_periods periods (2 by default, cloudkitty/utils/__init__.py:200-223). A new scope starts being rated from the first day of the current month (cloudkitty/utils/__init__.py:215-216).

Architecture

flowchart LR
  subgraph API["cloudkitty-api (WSGI)"]
    v1[v1 Pecan/WSME\nrating, report, storage, info, collector\n+ hashmap & pyscripts config]
    v2[v2 Flask\nsummary, dataframes, scope, task, rating]
  end
  user[Project user / Skyline / Horizon] -->|REST| API
  admin[Administrator / cloudkitty CLI] -->|REST| API
  API -.->|keystonemiddleware| ks[Keystone]
  API -->|SQL: rules, module state, scope state, reprocessing schedules| db[(MariaDB: cloudkitty DB)]
  API -->|InfluxQL / Flux / ES DSL / LogQL| ts[(Storage back end\nInfluxDB, ES, OpenSearch, Loki)]
  API -->|AMQP RPC call quote / cast reset_state, reload| mq[[RabbitMQ topic cloudkitty]]
  mq --> proc
  subgraph proc["cloudkitty-processor (cotyledon)"]
    w[CloudKittyProcessor x max_workers]
    r[CloudKittyReprocessor x max_workers_reprocessing]
  end
  proc -->|list scopes| fetch[Keystone / Gnocchi / Prometheus / static list]
  proc -->|collect usage| coll[Gnocchi / Prometheus / Aetos]
  proc -->|SQL| db
  proc -->|push rated frames, delete for reprocessing| ts
  proc -->|tooz locks| coord[(Coordination back end\nfile:// by default)]

Processes

External dependencies

Key figures

• 66 method + path pairs: GET /, 20 v1 core, 27 hashmap, 5 pyscripts, 13 v2 (see 01), plus /healthcheck served by oslo.middleware.
• 38 policy rules (cloudkitty/common/policies/), of which 7 base rules and 31 operation rules; one (v2_rating:list_modules) is registered but never enforced.
• 6 RPC methods on the processor; no notification is emitted or consumed.
• 3 collectors, 4 fetchers, 3 rating modules, 4 v2 + 2 v1 storage back ends (entry points pyproject.toml:52-79).

Points of attention

For OpenKeel, the items marked ★ bear directly on the comparison with openkeel-pricing.

1. ★ Quote ignores flat descriptions. POST /v1/rating/quote forwards each resource's desc as is, but the processor reads only desc.groupby and desc.metadata (cloudkitty/orchestrator.py:158-167). A client sending the documented flat desc ({"flavor_id": "…"}, cloudkitty/api/v1/datamodels/rating.py:46-61) gets a point with no attributes, so hashmap field mappings never match and only service-level rules apply. A quote also prices one period of the given volume (cloudkitty/orchestrator.py:149-150).
2. ★ Elasticsearch/OpenSearch delete ignores the scope filter. delete_by_query builds its query from _build_must only, which uses filters['type'] but not the scope key (cloudkitty/storage/v2/elasticsearch/client.py:369-383, cloudkitty/storage/v2/elasticsearch/client.py:70-86; same in cloudkitty/storage/v2/opensearch/client.py:372-374). A reprocessing or a PUT /v2/scope reset of one project deletes the rated data of every project over the window (cloudkitty/orchestrator.py:604-607, cloudkitty/orchestrator.py:233-235).
3. ★ **Loki totals are computed over at most limit log lines** (≤ 5000) in Python (cloudkitty/storage/v2/loki/client.py:389-406, cloudkitty/storage/v2/loki/__init__.py:218-228): /v2/summary silently under-reports on large windows.
4. ★ No carbon or non-monetary dimension. A DataPoint has qty, unit and one price (cloudkitty/dataframe.py:45-62); storage sums qty and price only (cloudkitty/storage/v2/influx.py:838-840). A footprint would need either a second "price" encoded by a pyscript or a separate pipeline.
5. ★ Rules are global, optionally per tenant, with validity windows. Hashmap mappings carry tenant_id, start, end and are soft-deleted (cloudkitty/rating/hash/db/sqlalchemy/models.py:193-259, cloudkitty/rating/common/db/models.py:19-60); thresholds have no validity window and are hard-deleted (cloudkitty/rating/hash/db/sqlalchemy/models.py:272-338, cloudkitty/rating/hash/db/sqlalchemy/api.py:642-650).
6. PyScripts executes arbitrary Python with exec in the processor, unsandboxed (cloudkitty/rating/pyscripts/__init__.py:98-101); likewise skip_datapoints_expression is evaled for every data point (cloudkitty/orchestrator.py:501). Both are admin-controlled but give code execution on processor hosts.
7. Keystone TLS verification disabled for the audit-user lookup done on every hashmap mapping and pyscript write ('verify': False, cloudkitty/common/custom_session.py:42-43).
8. RPC module reload/enable/disable messages have no effect. The processor only stores them (cloudkitty/orchestrator.py:182-207); the code that would read them is commented out (cloudkitty/orchestrator.py:683-688). Processors pick changes up anyway because each scope cycle re-reads module state and rules from SQL (cloudkitty/orchestrator.py:257-265, cloudkitty/orchestrator.py:329-339). reload_modules is sent with version 1.1 while the endpoint declares 1.0 (cloudkitty/api/v1/controllers/rating.py:51-53, cloudkitty/orchestrator.py:122-123) **[inferred: rejected with UnsupportedVersion on the server side]**.
9. v1 collector mappings and collector states are dead configuration: they are stored (cloudkitty/api/v1/controllers/collector.py:29-133) but nothing in the processor reads them.
10. **GET /v2/rating/modules has no policy check** although v2_rating:list_modules (admin) is registered (cloudkitty/api/v2/rating/modules.py:102-118, cloudkitty/common/policies/v2/rating.py:22-26).
11. InfluxQL built by string formatting from user-controlled filters and custom_fields (cloudkitty/storage/v2/influx.py:197-238); custom_fields is checked only against a list of whole forbidden words (cloudkitty/storage/v2/influx.py:260-268) [inferred: injection-prone].
12. **Default start of a rule is the API process start time**: the column default is datetime.datetime.now() evaluated at import (cloudkitty/rating/common/db/models.py:21-32); the DB layer usually overrides it with the request time (cloudkitty/rating/hash/db/sqlalchemy/api.py:324-326).
13. Reprocessing window is inclusive at the end — the period starting at end_reprocess_time is re-rated (cloudkitty/orchestrator.py:583-584) while the purge covers [start, end) (cloudkitty/orchestrator.py:604-607) [inferred: the last period may be stored twice].
14. Local time zone everywhere: periods, month boundaries and stored state are converted with the host's local time zone (cloudkitty/utils/tz.py:28-30); hosts must share one TZ (UTC in practice).

Blind spots / [inferred]

• Third-party behaviour is [inferred]: WSME error bodies and status selection, flask-restful error bodies, keystonemiddleware, tooz drivers, the InfluxDB/ES/Loki servers.
• The v1 storage back ends (sqlalchemy, hybrid+Gnocchi) are described briefly: they are deprecated (cloudkitty/cli/status.py:31-38) and disable the v2 API.
• The Flux query builder of InfluxDB v2 (cloudkitty/storage/v2/influx.py:358-745) is summarised, not specified line by line.
• cloudkitty/hacking/, Alembic migration bodies, devstack/ and contrib/ were not analysed.
• No dynamic capture was made (no running environment).

Source: reverse-spec-cloudkitty/00-summary_stable-2026.2.md. The _rewrite/ and _inventory/ files are not published on the site.