manila — Async processing

Asynchronous processing — manila — branch stable/2026.2

Branch stable/2026.2 @ 4fbc4b4f3688 — repository https://opendev.org/openstack/manila — analysed on 2026-10-29

Almost every write of the API only records an intent: the work is done by manila-scheduler and manila-share (and manila-data for copies and some backups) when they consume RPC casts, and by periodic tasks. This file lists those triggers. Periodic tasks run through oslo_service.periodic_task: each service calls manager.periodic_tasks() every [DEFAULT]/periodic_interval (60 s, manila/service.py:52-54), starting after a random delay up to [DEFAULT]/periodic_fuzzy_delay (60 s, manila/service.py:55-57); each task runs when its own spacing has elapsed, so **no task runs more often than periodic_interval** (manila/service.py:277-281). There is no leader election: a task runs in every process that has it, and only some take a tooz lock.

Summary

Process	Trigger	Type	Purpose	Reads	Writes / emits	Defined at
every service	`report_state`	timer `[DEFAULT]/report_interval` (10 s)	heartbeat	`services` row	`services.report_count`, `state`	`manila/service.py:282-325`
every service	`cleanup_services`	timer `cleanup_interval`	delete `stopped` services of the same topic that stopped reporting	`services`	`DELETE services`	`manila/service.py:326-345`
manila-scheduler	`create_share_instance` and 10 other methods	RPC cast/call `manila-scheduler`	place shares, replicas, groups; check extend/migration targets	host states (memory)	`share_instances.host`, cast to share	`manila/scheduler/manager.py:106-345`
manila-scheduler	`update_service_capabilities`	RPC fanout cast	cache back-end capabilities and pools	—	in-memory `HostManager.service_states`	`manila/scheduler/manager.py:106-118`
manila-scheduler	`_expire_reservations`	periodic, 600 s, run at start	expire quota reservations older than `reservation_expire` (86400 s)	`reservations`	`quota_usages`, `reservations`	`manila/scheduler/manager.py:260`
manila-scheduler	`_clean_expired_messages`	periodic, `message_reap_interval` (86400 s), run at start, tooz lock `locked-clean-expired-messages`	purge user messages older than `message_ttl`	`messages`	`DELETE messages`	`manila/scheduler/manager.py:324-325`
manila-scheduler	`_mark_services_as_down`	periodic, `service_down_time` (60 s), run at start, tooz lock `locked-mark-services-as-down`	mark services whose heartbeat is too old as `down`	`services`	`services.state`	`manila/scheduler/manager.py:330-334`
manila-share	`create_share_instance`, `delete_share_instance`, `update_access`, `extend_share`, `shrink_share`, `create_snapshot`, `delete_snapshot`, `revert_to_snapshot`, `manage_*`, replicas, groups, migration, backups, transfers…	RPC cast/call `manila-share.<host>@<backend>` (52 methods)	drive the storage back end	DB rows of the resource	driver calls, statuses, export locations, notifications	`manila/share/manager.py` (03 § RPC methods)
manila-share	`_report_driver_status`	periodic, `periodic_interval`	refresh `driver.get_share_stats()`	driver	`self.last_capabilities`	`manila/share/manager.py:4674-4688`
manila-share	`_publish_service_capabilities`	periodic (every run)	send capabilities to the schedulers	—	fanout cast `update_service_capabilities`	`manila/manager.py:170-178`
manila-share	`periodic_share_status_update`	periodic, `periodic_interval`	finish shares that the driver creates asynchronously	`share_instances` in `creating_from_snapshot` or `reverting_to_snapshot` on this host	`driver.get_share_status`, status, export locations	`manila/share/manager.py:5875-5887`
manila-share	`periodic_share_replica_update`	periodic, `replica_state_update_interval` (300 s)	refresh `replica_state` of non-active replicas	`share_instances` (replicas) on this host	`driver.update_replica_state`, `replica_state`	`manila/share/manager.py:2933`
manila-share	`periodic_share_replica_snapshot_update`	periodic, `replica_state_update_interval`	refresh snapshot instances of replicas	`share_snapshot_instances`	statuses	`manila/share/manager.py:4515`
manila-share	`do_deferred_share_deletion` / `do_deferred_snapshot_deletion`	periodic, `periodic_deferred_delete_interval` (300 s)	retry the driver deletion of instances in `error_deferred_deleting`	`share_instances`, `share_snapshot_instances`	driver delete, rows, quota	`manila/share/manager.py:3935-4000`, `manila/share/manager.py:4281`
manila-share	`delete_free_share_servers`	periodic, 600 s	delete unused share servers older than `unused_share_server_cleanup_interval` (10 min), DHSS=True only	`share_servers`	driver teardown, Neutron ports	`manila/share/manager.py:4002-4022`
manila-share	`delete_expired_share`	periodic, `check_for_expired_shares_in_recycle_bin_interval` (3600 s)	purge soft-deleted shares older than `soft_deleted_share_retention_time` (604800 s)	`shares`	`share_api.delete`	`manila/share/manager.py:4025-4040`
manila-share	`delete_expired_transfers`	periodic, `check_for_expired_transfers` (300 s)	drop transfers older than `transfer_retention_time` (300 s)	`transfers`	`transfer_api.delete`, share back to `available`	`manila/share/manager.py:4043-4050`
manila-share	`migration_driver_continue`	periodic, `migration_driver_continue_update_interval` (60 s)	drive driver-assisted migrations	`share_instances` migrating	`driver.migration_continue`	`manila/share/manager.py:1532`
manila-share	`share_server_migration_driver_continue`	periodic, `server_migration_driver_continue_update_interval` (900 s)	drive share server migrations	`share_servers`	driver	`manila/share/manager.py:6409`
manila-share	`create_backup_continue` / `restore_backup_continue`	periodic, `driver_backup_continue_update_interval` / `driver_restore_continue_update_interval` (60 s)	poll driver backups	`share_backups`	statuses	`manila/share/manager.py:5671`, `manila/share/manager.py:5795`
manila-share	`update_share_usage_size`	periodic, `share_usage_size_update_interval` (300 s), only with `enable_gathering_share_usage_size` (False)	gather used size	driver	notification `share.consumed.size`	`manila/share/manager.py:5852`
manila-share	`_execute_periodic_hook`	periodic, `periodic_hooks_interval`	share hooks (`manila/share/hook.py`)	share instances	hook drivers	`manila/share/manager.py:4690`
manila-share	`init_host` → `ensure_driver_resources`	process start	set up the driver, re-export shares, re-apply access rules	`share_instances` of the host, `backend_info`	`driver.ensure_shares`, statuses, export locations	`manila/share/manager.py:376-412`, `manila/share/manager.py:427-630`
manila-data	`migration_start`, `data_copy_*`, `create_backup`, `delete_backup`, `restore_backup`	RPC `manila-data`	copy data between exports	DB	mounts, file copy, statuses	`manila/data/manager.py:136-673`
manila-data	`create_backup_continue` / `restore_backup_continue`	periodic, `backup_continue_update_interval` / `restore_continue_update_interval` (10 s)	progress of data-manager backups	`share_backups`	statuses	`manila/data/manager.py:429`, `manila/data/manager.py:673`

manila-scheduler

create_share_instance (RPC cast, topic `manila-scheduler`, version 1.2)

Emitter: manila-api (share_api.create_instance when the host is not known yet, manila/scheduler/rpcapi.py:61-66).
Purpose: choose the pool that will host a new share.
Flow:
1. FilterScheduler.schedule_create_share (manila/scheduler/drivers/filter.py:90-111): build the filter properties from the request spec and the share type (manila/scheduler/drivers/filter.py:134-226), get all host states of live, enabled manila-share services with their pools (manila/scheduler/host_manager.py:656-720), filter, weigh, take the best (07 § 3).
2. Write share_instances.host = host@backend#pool and scheduled_at (manila/scheduler/drivers/base.py:44-55).
3. Cast create_share_instance to manila-share.<host@backend> with the request spec and filter properties (retry information included).
Error behaviour: NoValidHost → share instance and share error, user message (action 001, detail from the last filter) and notification scheduler.create_share_instance (manila/scheduler/manager.py:120-135). Any other exception does the same and is re-raised (logged by the RPC server).
Concurrency: several schedulers may run; each has its own in-memory host states, refreshed by the fanout capability updates. consume_from_share lowers the chosen pool's free space in memory only (manila/scheduler/host_manager.py:429-450), so two schedulers can pick the same pool for two simultaneous requests; the share manager does not re-check capacity.
Cost: one service_get_all_by_topic per request, filters and weighers in memory.

update_service_capabilities (fanout cast, version 1.10)

Emitter: every manila-share back end at each periodic_interval (manila/manager.py:170-178 → manila/scheduler/rpcapi.py:67-76), and at start (publish_service_capabilities in init_host).
Purpose: keep the scheduler's view of back ends and pools current. Stored in memory with the timestamp (manila/scheduler/host_manager.py:629-655). A back end that stops reporting stays in memory but is skipped once its services row is not up (manila/scheduler/host_manager.py:670).
Restart behaviour: a freshly started scheduler knows no back end until the next fanout; it asks for one with request_service_capabilities → publish_service_capabilities fanout to all share services (manila/scheduler/manager.py:238-239). Requests arriving before that fail with NoValidHost / "SHARE_BACKEND_NOT_READY_YET".

extend_share (RPC cast, version 1.11) and other scheduler methods

extend_share checks that the current pool can grow with the extend filters (scheduler_default_extend_filters) before casting extend_share to the share host; on failure the share is set back to available and an error notification is sent (manila/scheduler/manager.py:336-370). create_share_replica, create_share_group, manage_share, migrate_share_to_host follow the same pattern; get_pools (call) serves GET /scheduler-stats/pools.

create_share_instance (RPC cast, version 1.4)

Emitter: scheduler (or the API when the host is known: share from snapshot, share group).
Purpose: create the share on the back end.
Flow (manila/share/manager.py:2184-2440):
1. Notification share.create.start; read the instance; for DHSS=True, find or create a share server (Neutron ports, driver.setup_server); for DHSS=False with a share network → error + user message UNEXPECTED_NETWORK.
2. driver.create_share (or create_share_from_snapshot, which may return a model update with status available or creating_from_snapshot).
3. driver.get_optional_share_creation_data → share metadata; export_locations_update; instance available with launched_at; notification share.create.end.
Error behaviour: any exception → instance error, export locations from detail_data if the driver gave some, user message (action 002), re-raised and logged (manila/share/manager.py:2385-2420).
Idempotency: none: a replayed cast calls the driver again. Drivers usually fail on an existing name.

update_access (RPC cast, version 1.14)

Emitter: API (allow_access, deny_access, rule update), share manager itself (migration, replica promotion, transfers).
Purpose: make the back end's export rules match the database.
Flow (manila/share/manager.py:4648-4672 → manila/share/access.py:253-470), under the per-instance file lock (manila/share/access.py:28-55):
1. Move queued_to_apply → applying, queued_to_deny → denying, queued_to_update → updating.
2. Call driver.update_access(context, share_instance, access_rules=<all rules that must be on the share>, add_rules, delete_rules, update_rules, share_server); for readable replicas the rules are forced to ro (manila/share/access.py:494-505); IPv6 rules are dropped when the back end has no ipv6_support (manila/share/access.py:507-516).
3. Apply the driver's per-rule updates (state, access_key for cephx) (manila/share/access.py:470-492); delete denying rows; set applying / updating rules active.
4. If new rules were queued meanwhile, loop (manila/share/access.py:434-469); otherwise set the instance access_rules_status to active (or error if any rule is in error).
Error behaviour: a driver exception sets every applying / denying rule to error and access_rules_status to error (manila/share/driver.py:817-911 documents the contract).

delete_share_instance (RPC cast, version 1.27)

Deny all rules on the back end, driver.delete_share, delete export locations and the instance row; when the last instance is gone, delete the share row and commit the quota release (manila/share/manager.py:3777-3930). With deferred_delete the instance is marked deferred_deleting, its quota released at once, and a failure becomes error_deferred_deleting, retried by do_deferred_share_deletion. Driver failure (non-deferred) → error_deleting.

extend_share / shrink_share (RPC casts)

driver.extend_share(share, new_size) → size updated, quota reservation committed, available; failure → extending_error, reservation rolled back, user message DRIVER_FAILED_EXTEND (manila/share/manager.py:5002-5060). shrink_share: ShareShrinkingPossibleDataLoss or NotImplementedError from driver.shrink_share put the share back to available with a user message (DRIVER_REFUSED_SHRINK / not supported), any other exception gives shrinking_error; the reservation is rolled back (manila/share/manager.py:5062-5170). shrinking_possibly_data_loss_error is only a legacy status the API still accepts as a starting point (manila/share/api.py:3050-3055).

Snapshots

create_snapshot → driver.create_snapshot → model update (export locations, provider_location) → available, or error (manila/share/manager.py:4116-4190). delete_snapshot → driver.delete_snapshot, quota release (manila/share/manager.py:4192-4280). revert_to_snapshot → driver.revert_to_snapshot(context, snapshot, share_access_rules, snapshot_access_rules) (manila/share/manager.py:3627-3700).

init_host / ensure_driver_resources (process start)

driver.do_setup and check_for_setup_error, retried for ever with exponential back-off up to 600 s (manila/share/manager.py:383-398): while it fails, report_state skips the heartbeat (manila/service.py:284-290), so the back end appears down and is not scheduled.
ensure_driver_resources (manila/share/manager.py:427-630): if driver.get_backend_info() returns the same hash as the one stored in backend_info, nothing is re-exported; otherwise every instance of the host is passed to driver.ensure_shares(context, shares), whose answer can update export locations, set status, and ask to re-apply access rules (manila/share/driver.py:2886-2917). With [DEFAULT]/update_shares_status_on_ensure the instances are moved to ensuring during the operation (API 2.86 POST /services/ensure-shares triggers the same on demand).

Concurrency and cost notes

Periodic tasks of manila-share run per back end; two share services configured for the same back end on different hosts would both run them — use the same host value (active/passive) or a tooz backend.
periodic_share_status_update and periodic_share_replica_update issue one driver call per matching instance per cycle.
The scheduler's capability cache is refreshed by N back ends × 1 fanout message every periodic_interval.

Source: reverse-spec-manila/05-async-processing_stable-2026.2.md. The _rewrite/ and _inventory/ files are not published on the site.