glance — Async processing

Processing outside the API — glance — branch stable/2026.2

Branch stable/2026.2 @ a3e5f490b7a2 — repository https://opendev.org/openstack/glance — analysed on 2026-10-28

Glance has no RPC server, no bus consumer and no notification listener: it only emits notifications (see 03). The three "notification listener" triggers found by the inventory (glance/notifier.py:107, :110, :113) are false positives: they are the warn/info/error methods of Glance's own notification emitter.

Everything that happens after a 202 Accepted runs inside the glance-api worker that received the request, in a thread pool, as a taskflow flow. The other non-HTTP processing comes from timers in separate commands (the scrubber, the deprecated cache commands) and from one-shot startup work.

Summary

Task execution model (all three task types)

• Trigger: the API handler creates the task row (status = pending), then common.get_thread_pool("tasks_pool").spawn(task.run, executor) (glance/api/v2/images.py:496-497, glance/api/v2/tasks.py:91-92). The pool size is [wsgi]/task_pool_threads (default 16, glance/common/config.py:652), set at startup (glance/common/wsgi_app.py:135).
• Emitter: the glance-api worker that accepted the request. There is no queue between workers: if that worker dies, the task stays in processing (or pending) until a later request steals its lock (see Import lock expiry).
• Executor: TaskExecutor._run (glance/async_/taskflow_executor.py:174) moves the task to processing, loads the flow by name from the glance.flows stevedore namespace (glance/async_/taskflow_executor.py:134; pyproject.toml:49-52 maps api_image_import, import, location_import), and runs it with a taskflow engine (glance/async_/taskflow_executor.py:201) in [taskflow_executor]/engine_mode (parallel by default) with max_workers threads.
• End: the last task of the flow calls task.succeed() or task.fail(). Both set expires_at = now + [task]/task_time_to_live hours (default 48, glance/domain/__init__.py:437-446, glance/common/config.py:148). If the flow raises, _fail_task (glance/async_/taskflow_executor.py:149) records the message.
• Shutdown: at process exit, drain_workers waits for the running tasks of tasks_pool and stops the cache worker (glance/common/wsgi_app.py:86).

api_image_import (interoperable image import)

• Trigger: POST /v2/images/{image_id}/import (202), see 01.
• Purpose: move the image data from wherever the import method says into the target stores, then activate the image.
• Flow (glance/async_/flows/api_image_import.py:930-1069). The flow is linear, with retry.AlwaysRevert(): any failure reverts the tasks already run, in reverse order.
  1. *(before the flow starts, still synchronous in get_flow)* Unless the method is copy-image, the image goes to importing. The target stores are written to os_glance_importing_to_stores and removed from os_glance_failed_import, and os_glance_stage_host is dropped (glance/async_/flows/api_image_import.py:1041-1047). Then quotas are checked: image_size_total for glance-direct; image_stage_total and image_count_uploading for the other methods (glance/async_/flows/api_image_import.py:1049-1067, glance/async_/flows/api_image_import.py:891). A quota failure fails the task and reverts the image.
  2. _ImageLock checks that the image still carries this task's lock (glance/async_/flows/api_image_import.py:487).
  3. glance-download only: _ImportMetadata fetches the remote image (GET {remote}/v2/images/{id}), refuses it unless it is active, and copies disk_format, container_format and the properties listed in [glance_download_properties]/extra_properties (glance/async_/flows/api_image_import.py:804).
  4. web-download, copy-image, glance-download: an internal plugin writes the data to the staging store. web-download downloads the URL (glance/async_/flows/_internal_plugins/web_download.py:39); copy-image reads an existing location (glance/async_/flows/_internal_plugins/copy_image.py:48); glance-download downloads {remote}/v2/images/{id}/file and checks that the size matches (glance/async_/flows/_internal_plugins/glance_download.py:46).
  5. _VerifyStaging checks that the staging file exists (glance/async_/flows/api_image_import.py:522).
  6. Except for copy-image, the configured processing plugins run in order ([image_import_opts]/image_import_plugins, default none): image_decompression (zip, gzip, lha; glance/async_/flows/plugins/image_decompression.py:144), image_conversion (qemu-img info and qemu-img convert to [image_conversion]/output_format, default raw, after format inspection and safety checks; glance/async_/flows/plugins/image_conversion.py:114), inject_image_metadata (glance/async_/flows/plugins/inject_image_metadata.py:66), no_op.
  7. One _ImportToStore per store, in order (glance/async_/flows/api_image_import.py:596). With [image_import_opts]/max_parallel_stores > 1 and several stores, the parallel variant writes them concurrently (glance/async_/flows/parallel_api_image_import.py:60, glance/async_/flows/parallel_api_image_import.py:76). Each write updates the task message every chunk (Copied N MiB) and aborts if the task is no longer processing (glance/async_/flows/api_image_import.py:687-707). The image becomes active after the last store, or after the first one when all_stores_must_succeed is false; never for copy-image.
  8. _DeleteFromFS deletes the staging file (glance/async_/flows/api_image_import.py:444).
  9. _VerifyImageState requires active, otherwise raises _NoStoresSucceeded ("None of the uploads finished!"); its revert puts the image back to queued (except copy-image) (glance/async_/flows/api_image_import.py:723).
  10. _CompleteTask marks the task success with {"image_id": ...} and drops the lock (glance/async_/flows/api_image_import.py:748).
• Error behaviour: with all_stores_must_succeed = true (the default), a store failure reverts everything: the locations already added are removed, the stores are moved to os_glance_failed_import, the image goes back to queued and the task to failure. With false, a failed store is only recorded in os_glance_failed_import (glance/async_/flows/api_image_import.py:673-682). The client sees the outcome only through GET /v2/images/{id} (status and os_glance_* properties) or GET /v2/images/{id}/tasks.
• Concurrency: one import per image, enforced by the atomic property os_glance_import_task (insert that fails on duplicate; glance/db/sqlalchemy/api.py:891). Two workers can run imports of different images in parallel.
• Cost: the whole image is read once per store, plus once by each processing plugin. copy-image and the download methods first copy the image into staging (local disk of the worker: node_staging_uri, or the reserved os_glance_staging_store).

location_import (asynchronous add-location)

• Trigger: POST /v2/images/{image_id}/locations (202) on a queued image.
• Flow (glance/async_/flows/location_import.py:299):
  1. _ImageLock, then _UpdateLocationTask adds the location after validating it against the stores (glance/async_/flows/location_import.py:224).
  2. With do_secure_hash = true (default), _CalculateHash reads the data through the store and computes os_hash_value with http_retries attempts. It runs before activation when validation data was given, after activation otherwise (glance/async_/flows/location_import.py:54). It publishes os_glance_hash_op_host so that other workers proxy a delete to this one, and it registers with the cancellation tracker so that DELETE /v2/images/{id} can stop it (glance/task_cancellation_tracker.py:53, glance/task_cancellation_tracker.py:73).
  3. _VerifyValidationData compares the computed hash with the one given by the client; a mismatch fails the task and puts the image back to queued (glance/async_/flows/location_import.py:152).
  4. _SetImageToActiveTask activates the image (glance/async_/flows/location_import.py:261), and _CompleteTask ends the task.
• Error behaviour: an invalid location raises _InvalidLocation; a hash failure after http_retries raises _HashCalculationFailed and reverts to queued only if the image is not yet active (glance/async_/flows/location_import.py:85-103).

Legacy import task (POST /v2/tasks, type import)

• Trigger: POST /v2/tasks with {"type": "import", "input": {"import_from": "http://…", "image_properties": {…}}}. Access needs tasks_api_access (admin by default, glance/policies/tasks.py:105).
• Flow (glance/async_/flows/base_import.py:431): _CreateImage creates the image from image_properties (glance/async_/flows/base_import.py:44); _ImportToFS downloads import_from into [task]/work_dir (glance/async_/flows/base_import.py:84); the optional convert/introspect/ovf_process sub-flows run (pyproject.toml:54-57); _ImportToStore writes to the default store (glance/async_/flows/base_import.py:227); _DeleteFromFS cleans up.
• The tasks API is deprecated (glance/api/v2/tasks.py:106) but still routed.

Import lock expiry

• Trigger: a new import or add-location request finds os_glance_import_task already set (glance/api/v2/images.py:389-392, glance/api/v2/images.py:1203-1206).
• Rule (glance/api/v2/images.py:171-220): the lock is stolen if the task row no longer exists, or if its status is pending, processing, success or failure and it was last updated more than 60 minutes ago (120 minutes for pending). The old task is marked failure ("Expired lock preempted"), its in-progress store markers are cleaned, and the new request proceeds. Otherwise the request gets 409 "Image has active task".
• Concurrency: the property is removed with a compare-and-delete (glance/db/sqlalchemy/api.py:946); losing the race gives 409.

Task expiry

• _task_soft_delete marks as deleted every task whose expires_at is past, each time tasks are listed or read by image (glance/db/sqlalchemy/api.py:1884, called at glance/db/sqlalchemy/api.py:1842 and glance/db/sqlalchemy/api.py:1930). There is no timer; glance-manage db purge removes the soft-deleted rows later.

Staging cleanup at startup

• Trigger: init_app of each glance-api worker starts a daemon thread (glance/common/wsgi_app.py:102, glance/common/wsgi_app.py:164).
• Purpose: delete leftovers of staged uploads whose image was deleted while this worker was down.
• Data read: the file names of the staging directory (file:// only, glance/housekeeping.py:31), each parsed as an image UUID, then image_get per file.
• Effects: files of missing or deleted images are removed; others are kept (glance/housekeeping.py:85-126). Errors are logged and counted, never raised.

Image cache worker

• Trigger: PUT /v2/cache/{image_id} queues the image in the cache metadata and submits it to the in-process CacheWorker queue (glance/api/v2/cached_images.py:276). The worker is started by the first CacheController instance when image_cache_dir is set (glance/api/v2/cached_images.py:67-71).
• Flow: Prefetcher.fetch_image_into_cache reads the image as an admin, refuses non-active images, and tees the first readable location into the cache, verifying the checksum (glance/image_cache/prefetcher.py:43-73).
• Concurrency: one worker thread per glance-api process; cache files are local to the node (image_cache_dir). With the default centralized_db driver, the cache metadata of all nodes lives in the node_reference and cached_images tables, keyed by worker_self_reference_url (glance/image_cache/drivers/centralized_db.py:65-69).

glance-scrubber (delayed delete)

• Trigger: glance-scrubber --daemon loops every wakeup_time seconds (default 300, glance/scrubber.py:105, glance/scrubber.py:309); without --daemon it runs once (cron).
• Purpose: with delayed_delete = true (default false, glance/scrubber.py:74), DELETE /v2/images/{id} only moves the image to pending_delete (glance/domain/__init__.py:249-255); the scrubber deletes the data later.
• Data read: pages of images with status = pending_delete (glance/scrubber.py:200-220), keeping those whose deleted_at + scrub_time is past (glance/scrubber.py:222-262).
• Effects: delete_from_backend per location, image_locations.status = deleted, then images.status = deleted (glance/scrubber.py:358-414), with scrub_pool_size threads (default 1).
• Error behaviour: a store error is logged and the location is retried on the next run. glance-scrubber --restore <id> puts a pending_delete image back to active (glance/scrubber.py:416); it refuses to run together with --daemon.

Source: reverse-spec-glance/05-async-processing_stable-2026.2.md. The _rewrite/ and _inventory/ files are not published on the site.