Processing outside the API — ec2-api — branch 17.0.0
Branch
17.0.0@32740781f605— repository https://opendev.org/openstack/ec2-api (retired; local branchpre-retirement) — analysed on 2026-10-29
ec2-api has no bus consumer, no RPC server, no periodic task and no event listener: the inventory finds 0 triggers (_inventory/inventory_17.0.0.json), there is no oslo.messaging dependency in requirements.txt, and cert_topic is a deprecated leftover of nova-cert (ec2api/api/image.py:66). Everything that happens outside a request happens in one of the four mechanisms below. A rewrite must reproduce them or replace them explicitly, because clients observe their effects.
1. Green threads spawned by a request
| Trigger | Code | Functional purpose | Effects | Concurrency, errors |
|---|---|---|---|---|
CreateImage without NoReboot on a running server | ec2api/api/image.py:135, spawned at ec2api/api/image.py:183 | stop the server so that the snapshot is consistent, snapshot it, restart it | Nova os-stop, polling GET /servers/{id} every second (≤ 1 h), createImage with metadata ec2_id, DB UPDATE of the ami item's os_id, Nova os-start | runs in the API process with the request's context and token [inferred: a token expiring during a long stop/snapshot breaks the sequence]; any exception sets state=failed and is logged; lost on process restart (no persistence of the step reached) |
RegisterImage of an S3 bundle | ec2api/api/image.py:887, spawned at ec2api/api/image.py:954 | import an EC2 AMI bundle (manifest + encrypted parts) into Glance | S3 GetObject of each part, concatenation, RSA-decrypt the AES key and IV, openssl enc -d -aes-128-cbc, untar (with a path-traversal check, ec2api/api/image.py:1059), Glance upload; progress in the Glance property image_state (downloading, decrypting, untarring, uploading, available, or failed_*) | same process and token; failures leave the temporary directory; the image stays pending if the process dies |
Both rely on eventlet; under a threaded or multi-pod Go service they need a persistent job (for example a work queue or a Kubernetes Job) and a state machine visible through imageState.
2. Reconciliation inside Describe operations
There is no periodic synchronisation between ec2-api's database and OpenStack. Every Describe* of an OpenStack-backed kind reconciles lazily (ec2api/api/common.py:389):
- creates items for OpenStack objects that have none (instances, volumes, snapshots, images, security groups; with the kind-specific extensions of
ec2api/api/instance.py:1470andec2api/api/image.py:820), so objects created through the OpenStack APIs get EC2 IDs the first time a project describes them; - deletes items whose object disappeared, with their tags; for instances it also detaches their ENIs and deletes the ENIs flagged
delete_on_termination(ec2api/api/instance.py:844); - repairs: the default security group of each VPC (
ec2api/api/security_group.py:182), the association state of elastic IPs (ec2api/api/address.py:124), pending images whose Glance image appeared (ec2api/api/image.py:453),is_publicof images (ec2api/api/image.py:429).
Cost: each Describe lists the project's whole set of OpenStack objects of that kind (and more: DescribeInstances also lists volumes, flavors, ENIs, addresses, ports, security groups and images), whatever the filters. Filters are applied afterwards in Python (ec2api/api/common.py:428). With N instances, DescribeInstances is O(N) in Nova, Cinder and Neutron listings plus O(N) DB reads, on every call; the metadata service runs it for each cache miss.
Concurrency: two concurrent describes of the same project can both auto-create an item for the same OpenStack object; the unique os_id index makes the second insert fail, and add_item then re-reads and reuses the first row (ec2api/db/sqlalchemy/api.py:107), so the EC2 ID stays stable.
3. Default VPC creation on first use
With disable_ec2_classic = True, ec2api.api.vpc._check_and_create_default_vpc (ec2api/api/vpc.py:169) runs from DescribeVpcs, DescribeSubnets, DescribeRouteTables, DescribeInternetGateways, DescribeSecurityGroups (and therefore from most instance and network operations, which call DescribeSecurityGroups internally), DescribeAccountAttributes default-vpc, ReleaseAddress and RunInstances without subnet. It creates the default VPC, internet gateway, subnet and default route once per project (04 § 3). Protection: an oslo.concurrency external lock per project in lock_path — per host only. Failure: logged, the caller proceeds as if there were no default VPC (VPCIdNotSpecified where one is required).
4. Metadata cache
ec2-api-metadata caches the whole metadata tree of an instance for [metadata] cache_expiration seconds (default 15) under ec2api-metadata-<instance uuid> in the oslo.cache region (ec2api/metadata/api.py:102). There is no invalidation: a change of tags, IPs or security groups is visible to the instance only after expiry. The owner check is done again on every hit (ec2api/metadata/api.py:106).
Source: reverse-spec-ec2-api/05-async-processing_17.0.0.md. The _rewrite/ and _inventory/ files are not published on the site.