Errors — swift — branch stable/2026.2
Branch
stable/2026.2@a1bdaeba21bd— repository https://opendev.org/openstack/swift — analysed on 2026-10-29
Swift returns HTTP statuses, not error codes: there is no JSON error body and no business code. What a client sees is decided in three places: the validation helpers and middlewares (4xx before any backend call), the storage servers (statuses of each replica) and the proxy's quorum logic, which turns N backend statuses into one client status. Exceptions are internal: they are converted to statuses inside each server and never cross the wire as such. The S3 API (s3api) is the exception: it returns S3 XML errors with codes.
1. Error body
| Layer | Body | Source |
|---|---|---|
| Swift API (proxy and storage servers) | default: <html><h1>{title}</h1><p>{explanation}</p></html> (Content-Type: text/html; charset=UTF-8), from RESPONSE_REASONS; many validation errors set a short text/plain body instead (Invalid Content-Type, Metadata name too long: …) | swift/common/swob.py:1469, swift/common/swob.py:59 |
| HEAD responses | no body, same status | swob |
| Unhandled exception in the proxy | 500 Internal Error, logged ERROR Unhandled exception in request | swift/proxy/server.py:631 |
| Exception escaping the proxy WSGI entry point | 500 Server Error, body Internal server error.\n | swift/proxy/server.py:517 |
| Unhandled exception in a storage server | 500 with the Python traceback as body (internal only) | swift/obj/server.py:1452 |
catch_errors (outermost) | catches anything raised below and answers 500; also enforces that the body length matches Content-Length (logs Client disconnected) | swift/common/middleware/catch_errors.py:150 |
| S3 API | <?xml …?><Error><Code>NoSuchKey</Code><Message>…</Message><Resource>…</Resource><RequestId>tx…</RequestId></Error>; code is the class name | swift/common/middleware/s3api/s3response.py |
Clients parse statuses, not texts. python-swiftclient raises ClientException(http_status, http_reason, http_response_content) and retries 5xx, 408, 499 and (by default) 498/429; it re-authenticates once on 401 (python-swiftclient:swiftclient/client.py:1773). Glance's store maps 404 → NotFound, 409 → Duplicate, 413 → StorageFull (glance_store:glance_store/_drivers/swift/store.py:1129). See 07 § 4.
2. How the proxy chooses the status: quorum
For writes and for HEAD/GET of accounts and containers the proxy collects one status per backend node and calls best_response (swift/proxy/controllers/base.py:2225):
- Statuses are grouped by class 2xx, 3xx, 4xx (
swift/proxy/controllers/base.py:2272). The first class, in that order, that reaches the quorum wins; within it the highest status is returned with its reason and body. - Quorum:
floor(n/2)+1for account/container rings and replicated policies (swift/common/utils/__init__.py:2550,swift/common/storage_policy.py:438);ec_ndata + min_parity_fragments_needed()for EC PUTs (swift/common/storage_policy.py:637). - If no class reaches the quorum, the proxy answers 503 and logs
<type> returning 503 for <statuses>(swift/proxy/controllers/base.py:2267). overrideslets some statuses count as others: for DELETE, a 404 is turned into 204 when a quorum is otherwise impossible; for container PUT, 202 and 201 mix.
Consequences that clients observe (and a rewrite must reproduce):
- 5xx never wins a quorum by itself: 3 nodes answering 507, 507, 201 give 503, not 507.
- A 404 on GET object is returned only when every primary (and the handoffs tried) answered 404; one node holding a tombstone newer than another node's data makes the proxy return 404 because the newest timestamp wins (
X-Backend-Timestamp). - A POST with mixed 404 and 202 where a replica proved the object exists gives 503 (
swift/proxy/controllers/obj.py:961). - A node that keeps failing is error-limited: after
error_suppression_limit(10) errors withinerror_suppression_interval(60 s) it is skipped (swift/proxy/server.py:229,swift/common/error_limiter.py:33). A 507 marks a node error-limited at once (swift/proxy/controllers/obj.py:783).
3. Client-facing statuses
| Status | Situation | Raised at |
|---|---|---|
| 400 | path version other than v1/v1.0 | swift/proxy/server.py:563 |
| 400 | negative Content-Length (Invalid Content-Length) | swift/proxy/server.py:548 |
| 400 | metadata name empty / too long, value too long, too many, total too large | swift/common/constraints.py:144–swift/common/constraints.py:169 |
| 400 | object name longer than 1024 bytes | swift/common/constraints.py:207 |
| 400 | Content-Type with forged swift_bytes or invalid (Invalid Content-Type) | swift/common/constraints.py:222 |
| 400 | X-Delete-After / X-Delete-At non-integer or in the past | swift/common/constraints.py:330, swift/common/constraints.py:351 |
| 400 | unknown or deprecated X-Storage-Policy on container PUT | swift/proxy/controllers/container.py:69 |
| 400 | invalid ACL (X-Container-Read/Write, X-Account-Access-Control) | swift/proxy/controllers/container.py:87 |
| 400 | SLO manifest invalid (per-segment error list in the body) | swift/common/middleware/slo.py:395 |
| 400 | name rejected by name_check | swift/common/middleware/name_check.py:117 |
| 401 | no or invalid token, invalid temp URL / form signature | swift/common/middleware/keystoneauth.py:588, swift/common/middleware/tempurl.py:787 |
| 403 | authenticated but not allowed (ACL, role, read-only account); max_containers_per_account reached; Invalid host header | swift/common/middleware/keystoneauth.py:401, swift/proxy/controllers/container.py:664, swift/proxy/server.py:574 |
| 404 | account / container / object absent, object expired, path not splittable | swift/proxy/server.py:568, swift/obj/server.py:1202 |
| 405 | method not allowed on this resource (Allow header); account PUT/DELETE without allow_account_management; writes under read_only | swift/proxy/server.py:595, swift/proxy/controllers/account.py:113 |
| 406 | listing Accept not satisfiable | swift/common/middleware/listing_formats.py:40 |
| 408 | client too slow sending the body (client_timeout, default 60 s) | swift/proxy/controllers/obj.py:1168 |
| 409 | container DELETE on a non-empty container; container PUT changing the policy; object PUT with a timestamp not newer than the stored one (internal X-Timestamp reuse) | swift/container/server.py:355, swift/proxy/controllers/obj.py:726 |
| 411 | object PUT with neither Content-Length nor chunked encoding | swift/common/constraints.py:175 |
| 412 | If-None-Match: * on an existing object; If-Match mismatch; invalid UTF-8 path; Bad URL; listing limit > 10000; bad Destination header | swift/proxy/controllers/obj.py:724, swift/proxy/server.py:551, swift/common/middleware/copy.py:148 |
| 413 | object larger than max_file_size; container or account quota exceeded; SLO manifest too large | swift/common/constraints.py:197, swift/common/middleware/container_quotas.py:65, swift/common/middleware/slo.py:1362 |
| 416 | Range not satisfiable | swob |
| 422 | ETag header does not match the body MD5 (object server), or replicas returned mismatched ETags | swift/obj/server.py:1019, swift/proxy/controllers/obj.py:1226 |
| 498 | rate limited (ratelimit, Slow down) | swift/common/middleware/ratelimit.py:232 |
| 499 | client disconnected before sending the whole body | swift/proxy/controllers/obj.py:1153 |
| 500 | unhandled exception | swift/proxy/server.py:631 |
| 503 | no quorum; not enough connectable nodes for the PUT (Object PUT returning 503, n/m required connections); container info lookup failed; unknown policy index; a shard of a sharded container failed during listing | swift/proxy/controllers/obj.py:819, swift/proxy/server.py:480, swift/proxy/controllers/container.py:572 |
4. Storage-server statuses (internal wire)
These are what a Go storage node must return to Python proxies and daemons (and what a Go proxy must interpret):
| Status | Situation | Source |
|---|---|---|
| 201 / 202 / 204 | create / accepted (POST, DB updates) / delete | each server |
| 400 | invalid path, missing or invalid X-Timestamp (Missing X-Timestamp header), bad X-Delete-At, MIME errors (no MIME boundary, couldn't find PUT commit MIME doc, no Content-MD5 in footer, invalid JSON for footer doc) | swift/common/constraints.py:293, swift/obj/server.py:916, swift/obj/server.py:568, swift/obj/server.py:597 |
| 404 | object/DB absent; for objects the response carries X-Backend-Timestamp (tombstone time) and X-Backend-Durable-Timestamp so that the proxy can compare | swift/obj/server.py:1202 |
| 409 | PUT/DELETE/POST older than or equal to what is stored, with X-Backend-Timestamp; container DELETE when not empty; container PUT changing the policy | swift/obj/server.py:874, swift/container/server.py:355 |
| 412 | If-None-Match hit on PUT; X-If-Delete-At mismatch on expirer DELETE | swift/obj/server.py:867, swift/obj/server.py:1354 |
| 403 | DiskFileCollision: the name in the metadata of the data file is not the requested name (hash collision or corruption) | swift/obj/server.py:1444 |
| 408 / 499 | body read timeout / client disconnect | swift/obj/server.py:1115, swift/obj/server.py:1048 |
| 422 | footer MD5 or ETag mismatch | swift/obj/server.py:599, swift/obj/server.py:1019 |
| 500 | unhandled exception, traceback in the body; multiphase commit missing (HTTPServerError) | swift/obj/server.py:1452, swift/obj/server.py:1033 |
| 503 | DiskFileStateChanged during GET; replication concurrency limit reached (replication_concurrency) | swift/obj/server.py:1209 |
| 507 | device not mounted (mount_check), no xattr support, no space (ENOSPC, EDQUOT) | swift/obj/server.py:852, swift/obj/server.py:857, swift/account/server.py:134 |
| 529 | backend_ratelimit per device | swift/common/middleware/backend_ratelimit.py:198 |
5. Exception catalogue (internal)
Exceptions are defined in swift/common/exceptions.py and converted by the server that catches them. The ones that change a status or a behaviour:
| Exception | Hierarchy | Situation | What the caller sees |
|---|---|---|---|
DiskFileNotExist | DiskFileError (swift/common/exceptions.py:80) | no .data/.ts for the hash | 404 |
DiskFileDeleted | DiskFileNotExist (swift/common/exceptions.py:92) | newest file is a tombstone; carries its timestamp | 404 with X-Backend-Timestamp (swift/obj/server.py:1307) |
DiskFileExpired | DiskFileDeleted (swift/common/exceptions.py:100) | X-Delete-At passed | 404 (swift/obj/server.py:1303) |
DiskFileQuarantined | DiskFileError (swift/common/exceptions.py:72) | metadata unreadable, size or MD5 mismatch while reading: the hash directory is moved to <dev>/quarantined/objects[-N]/ | 404; replication will restore it (swift/obj/diskfile.py:350) |
DiskFileCollision | (swift/common/exceptions.py:76) | name in metadata ≠ requested path | 403 (swift/obj/server.py:1444) |
DiskFileNoSpace | (swift/common/exceptions.py:104) | ENOSPC/EDQUOT writing data or xattrs | 507 (swift/obj/diskfile.py:297) |
DiskFileXattrNotSupported | (swift/common/exceptions.py:112) | filesystem without user xattrs (ENOTSUP) | 507 (swift/obj/diskfile.py:209) |
DiskFileBadMetadataChecksum | (swift/common/exceptions.py:116) | user.swift.metadata_checksum ≠ MD5 of the pickle | quarantine (swift/obj/diskfile.py:238) |
DiskFileDeviceUnavailable | (swift/common/exceptions.py:108) | device not mounted / not a directory | 507 |
DiskFileStateChanged | (swift/common/exceptions.py:84) | file vanished between listdir and open | 503 on GET, 404 elsewhere (swift/obj/diskfile.py:214) |
PathNotDir | OSError (swift/common/exceptions.py:132) | suffix directory missing while hashing: the suffix is dropped from hashes.pkl | — (swift/obj/diskfile.py:1387) |
LockTimeout, ReplicationLockTimeout, PartitionLockTimeout | MessageTimeout (swift/common/exceptions.py:168) | .lock file of a partition or DB directory not obtained in time | DB servers: 500/HTTPConflict; SSYNC: error line |
DatabaseConnectionError | sqlite3.DatabaseError subclass (swift/common/db.py) | DB missing or skip_commits | 404 / 500 (swift/account/server.py:116) |
ChunkReadTimeout, ChunkWriteTimeout, ConnectionTimeout | Timeout (swift/common/exceptions.py:148) | client_timeout, node_timeout, conn_timeout expired | 408, or the node is skipped (error-limited) |
ChunkReadError | (swift/common/exceptions.py:140) | client socket closed | 499 |
InsufficientStorage | (swift/common/exceptions.py:44) | a node answered 507 during PUT connect | node error-limited, next handoff tried (swift/proxy/controllers/obj.py:782) |
ListingIterError, SegmentError | (swift/common/exceptions.py:204, swift/common/exceptions.py:218) | a DLO/SLO segment is missing or changed after headers were sent | the response is truncated and the connection closed (the status was already 200) |
RingLoadError, RingValidationError | (swift/common/exceptions.py:172) | invalid ring at startup | process does not start; at reload the old ring is kept (swift/common/ring/ring.py:344) |
ReplicationException | (swift/common/exceptions.py:226) | SSYNC protocol violation, unexpected status | logged by the replicator/reconstructor, job retried next pass |
APIVersionError | (swift/common/exceptions.py:242) | invalid version prefix | 400 |
EncryptionException, UnknownSecretIdError | (swift/common/exceptions.py:246) | crypto metadata unreadable, unknown key id | 500 on GET (logged) |
The inventory (_inventory/inventory_stable-2026.2.json) lists all 196 exception classes found, including the 60 s3api error classes and the CLI ones.
6. S3 API errors
s3api converts Swift statuses into S3 errors (swift/common/middleware/s3api/s3response.py): 404 on a container → NoSuchBucket, on an object → NoSuchKey; 409 on container DELETE → BucketNotEmpty; 401/403 → AccessDenied, SignatureDoesNotMatch, InvalidAccessKeyId, RequestTimeTooSkewed; 413 → EntityTooLarge; 400 from SLO for small parts → EntityTooSmall; unknown → InternalError. Each class sets its _status and _msg; extra elements (BucketName, Key…) come from constructor kwargs. The functional run recorded the combinations actually returned: _rewrite/observed-errors_stable-2026.2.md.
7. Errors outside the request path
| Where | Situation | Effect, log |
|---|---|---|
| object server, container update | container server unreachable or non-2xx | ERROR Container update failed (saving for async update later); async pending written (swift/obj/server.py:339) |
| object updater | async pending cannot be sent | kept, retried next pass; after a 404 of the root container and redirect → follows container_path (swift/obj/updater.py:407) |
| replicator / reconstructor | remote 507 | node skipped; Remote drive not mounted / handoff used |
| replicator | hashes.pkl corrupt | Unable to read %r warning, full rehash (swift/obj/diskfile.py:1347) |
| auditor (object) | MD5/size/metadata mismatch | ERROR Object … failed audit and was quarantined; quarantined, quarantines stat in recon |
| DB broker, DB auditor | SQLite reports a corrupt or malformed database (database disk image is malformed, no such table…) | DB moved to <dev>/quarantined/<type>s/<hash> and the error re-raised (swift/common/db.py:502, swift/common/db.py:479) |
| DB broker | .pending entry not unpicklable | Invalid pending entry %(file)s: %(entry)s, entry dropped (swift/common/db.py:849) |
| sharder | cleaving or shard-range replication incomplete | retries next cycle; container stays in sharding state; warnings every periodic_warnings_interval |
| expirer | DELETE answered 412 (object rewritten without the same X-Delete-At) | queue entry removed, object kept |
| ring reload | new ring invalid | old ring kept silently (swift/common/ring/ring.py:344) |
Source: reverse-spec-swift/02-errors_stable-2026.2.md. The _rewrite/ and _inventory/ files are not published on the site.