ec2-api — Summary

Reverse-specification of ec2-api — branch 17.0.0

Branch 17.0.0 @ 32740781f605 — repository https://opendev.org/openstack/ec2-api (retired; local branch pre-retirement) — analysed on 2026-10-29

• Repository: https://opendev.org/openstack/ec2-api (local clone OpenKeel/ec2-api) — specified branch: tag 17.0.0 = local branch pre-retirement @ 32740781f6050cbb08847066fc2fef996d93f2a1 (2023-06-23, last release before the project was retired; 2023.2-eol adds only two CI commits) — analysed on: 2026-10-29.
• Scope: the whole service: ec2-api (EC2 Query API), ec2-api-metadata (instance metadata), ec2-api-s3 (S3 object store), ec2-api-manage (DB commands); the rewrite pack (step 8) for openkeel-ec2-api. The change history (step 7) covers the last two cycles, 15.0.0.0rc1..17.0.0 (12 commits).
• Clients whose expectations are in the pack: AWS CLI / boto3 (botocore 1.29.118 EC2 model, checked mechanically), Terraform's AWS provider (aws-sdk-go, same model; provider behaviour [inferred]), euca2ools / boto2 (SigV2), instance metadata consumers (cloud-init).
• Other repositories cited: keystone: openstack/keystone stable/2026.2 @ e6dc5ae07, keystoneclient: python-keystoneclient stable/2026.2 @ b76de6a3 (signature verification), novaclient:/neutronclient: local clones (exception attributes only).

Overview

ec2-api is a translation layer: it serves the AWS EC2 Query API (Action=RunInstances&Version=…, XML responses, AWS error codes) and turns each action into calls to Nova, Neutron, Cinder and Glance made with the caller's own Keystone token. Requests are signed with AWS signatures (v2 or v4) using EC2 credentials stored in Keystone; ec2-api does not verify them itself but forwards them to Keystone's POST /v3/ec2tokens, which returns a token for the credential's user and project.

Its only state is a two-table database: items, which maps every EC2 identifier (i-, ami-, vol-, snap-, vpc-, subnet-, eni-, sg-, eipalloc-, rtb-, igw-, dopt-, vgw-, cgw-, vpn- followed by 8 hex digits) to an OpenStack UUID and keeps the attributes OpenStack has no place for, and tags. Several EC2 resources exist only there (route tables, internet and VPN gateways, DHCP options, customer gateways, VPN connections) and are realised in Neutron as router gateways, subnet host routes, port DHCP options and VPNaaS objects. A VPC is a Neutron router, an EC2 subnet a Neutron network with one subnet, an ENI a port, an elastic IP a floating IP.

Two side services complete it: ec2-api-metadata builds the EC2 instance-metadata tree from a DescribeInstances behind Neutron's metadata proxy, and ec2-api-s3 is a minimal, unauthenticated S3 server kept as a source of AMI bundles for RegisterImage.

Architecture

flowchart LR
  cli[AWS CLI / boto3 / Terraform / euca2ools] -->|HTTPS POST / Action=…<br/>AWS SigV4 or SigV2| api[ec2-api :8788<br/>paste: faultwrap, log, EC2KeystoneAuth, Requestify, Executor]
  api -->|POST /v3/ec2tokens<br/>signature check| ks[Keystone]
  api -->|SQL items, tags| db[(MariaDB ec2api)]
  api -->|REST, user token| nova[Nova]
  api -->|REST, admin session<br/>all_tenants listings| nova
  api -->|REST, user token| neu[Neutron incl. VPNaaS]
  api -->|REST, user token| cin[Cinder]
  api -->|REST, user token| gl[Glance]
  api -->|S3 GetObject, user's EC2 credential| s3[S3 endpoint<br/>ec2-api-s3 or Swift s3api]
  vm[Instance] -->|169.254.169.254| nmp[Neutron metadata proxy]
  nmp -->|X-Instance-ID, signed| md[ec2-api-metadata :8789]
  md -->|admin session| nova
  md -->|admin session| neu
  md -->|SQL| db
  md -->|oslo.cache| cache[(memcached)]
  md -->|/openstack/* proxied| nmeta[Nova metadata :8775]
  s3srv[ec2-api-s3 :3334] -->|files| fs[(buckets_path)]

Processes

No process consumes a message bus or runs periodic tasks (05).

External dependencies

Key figures

• 97 EC2 actions (48 in CloudController, 49 VPC actions in VpcCloudController), all present in the AWS SDK model; the api-ref "supported features" list matches the code exactly (api-ref/source/supported_features.inc:7). 21 describers with filter maps.
• 3 HTTP services; 0 RPC calls, 0 bus consumers, 0 periodic tasks; 2 green-thread background jobs (CreateImage, S3 import).
• 2 tables (items, tags), 1 migration; 17 item kinds plus 5 derived ID forms (rtbassoc-, eni-attach-, eipassoc-, default sg-, reservations r-).
• 87 exception classes, 78 of them EC2 client errors (HTTP 400) with 78 distinct EC2 codes.
• 15 543 lines of Python outside tests; 34 unit-test modules, 443 tests (all pass on Python 3.10 except test_hacking, not run).
• Conformance data: 1009 + 33 + 12 + 7 recorded exchanges, 955 with the OpenStack/DB calls of the action; 726 responses parsed by botocore.

Points of attention

1. **Any parameter the code does not declare causes 500 TypeError** — DryRun, TagSpecification (Terraform tags on create), the AWS CLI's --protocol/--port/--cidr form of security group rules, newer RunInstances options, and the SigV4 query-string parameters (presigned requests) (ec2api/api/cloud.py:76, ec2api/api/__init__.py:252; _rewrite/botocore-compat_17.0.0.md).
2. Every parameter value is type-coerced (ec2api/api/ec2utils.py:52): a numeric tag value (Value=123) gives 500 TypeError, a description True 500 UnboundLocalError, 0755 becomes 493 (probes in _rewrite/ec2-actions_17.0.0.json).
3. OpenStack exceptions leak with their status and Python class name as EC2 code (NotFound, Forbidden…); programming errors leak as TypeError, KeyError (ec2api/api/__init__.py:304).
4. Describe operations write: they create EC2 IDs for OpenStack objects and delete items whose object vanished (ec2api/api/common.py:389); terminated instances disappear from the DB only on a later describe.
5. ID mapping is global: os_id is unique across kinds and projects, foreign objects get project-less ID rows, and public images are found with a LIKE '%"is_public": True%' on JSON that only works with case-insensitive matching and Python's JSON separators (ec2api/db/sqlalchemy/api.py:228).
6. **One Keystone ec2tokens call per request**, without timeout or cache (ec2api/api/__init__.py:211); Keystone must enable the ec2credential method.
7. SigV4 only works for POST form requests (ec2-api sends empty params to Keystone for SigV4, ec2api/api/__init__.py:182).
8. Errors before the Executor are not EC2 XML (missing Action/Version, timestamps: webob plain text, 403 for timestamps) (ec2api/api/__init__.py:262).
9. No transactions, no locks except a per-host file lock for default-VPC creation (ec2api/api/vpc.py:175); partial failures are compensated by in-memory clean-ups that can themselves fail and orphan Neutron/Nova objects (ec2api/api/common.py:63).
10. The S3 server has no authentication and a prefix-based path check (ec2api/s3/s3server.py:256); the VPN pre-shared keys are stored in clear (ec2api/api/vpn_connection.py:93); the AES key of S3 bundles is passed on openssl's command line (ec2api/api/image.py:1037).
11. DhcpOptions association updates the ports of every VPC of the project (ec2api/api/dhcp_options.py:147); CreateSecurityGroup always applies the VPC character set (ec2api/api/cloud.py:69); StartInstances/StopInstances return true instead of state changes.
12. EC2-Classic floating IPs use Nova addFloatingIp, removed in Nova microversion 2.44 (ec2api/api/address.py:321); ec2-api pins 2.32 (ec2api/clients.py:59).

Blind spots / [inferred]

• No black-box run: the upstream functional suite (ec2api-tempest-plugin) and a devstack were not available; behaviour against real OpenStack services is read from the code and the unit tests (whose OpenStack and DB layers are mocks).
• Terraform AWS provider requirements (§ 4.1 of 07) are inferred from the SDK model and the provider's known behaviour, not from a run.
• The MySQL branch of the migration was not executed (SQLite only).
• Keystone and keystoneclient are cited at stable/2026.2 (the OpenKeel target), not at the 2023 versions ec2-api 17.0.0 was released with; signature rules may have changed since (the ec2credential method requirement is recent).
• ec2api/wsgi.py (eventlet server, SSL) and ec2api/hacking were not specified: framework code.

Source: reverse-spec-ec2-api/00-summary_17.0.0.md. The _rewrite/ and _inventory/ files are not published on the site.