API Backend Rewrite¶
Current tracker for the completed api-backend rewrite from Flask/Python to Go. This page records final cutover state, retained non-api-backend Python worker runtimes, and validation history.
Current Runtime¶
| Field | State |
|---|---|
| Branch | feature/rewrite-api-backend-in-golang |
| PR | #232 |
| Active milestone | Done |
| Last updated | 2026-06-17 |
| Current architecture | Go binary owns public loopback 127.0.0.1:5000; no Python compatibility backend is supervised; no unported route proxy remains. |
| Latest verified slice | Final cutover removes Python api-backend process supervision, proxy fallback, Flask app factory, Python API route tree, internal Aegis bridge, AssemblyCache bridge, retained workflow helpers, retained scheduled-job API helpers, obsolete Python Flask route tests, and Python remnants from the job-scheduler container. Go workflow runtime now dispatches Ansible through active site-worker /automation/ansible/run and executes subworkflows natively. |
| Rewrite posture | Complete. api-backend public and internal route behavior is Go-owned; retained Python code is limited to non-api-backend worker/helper runtimes. |
Status Legend¶
| Status | Meaning |
|---|---|
Done |
api-backend public route behavior is Go-owned, matching Flask route code is removed, and validation/smoke is recorded. Retained site-worker or Agent runtime is noted when it is not a Flask API fallback. |
Hybrid |
Historical status only. No inventory rows remain Hybrid. |
Python |
Historical status only. No inventory rows remain Python. |
Removed |
Route surface intentionally deleted from both WebUI and API. |
Rewrite Inventory¶
| Domain | Status | Go-owned now | Python dependency | Cleanup note |
|---|---|---|---|---|
| Gateway, health, Go status | Done |
Public listener, health/status responses, Go-only status payloads, no Python process supervision, no reverse proxy. | None. | Python compatibility backend wiring removed. |
| Auth session, profile, logout | Done |
Borealis token verification, /api/auth/me, logout, session invalidation checks. |
No Flask API fallback remains for this row. | Duplicate Python profile/logout route handlers removed in current cleanup slice. |
| Bootstrap, Aegis lifecycle, local login, local MFA | Done |
Go-native bootstrap state/auth gate, Aegis bootstrap setup/unlock, public Aegis status/setup/unlock/rotate/force-reset, admin bootstrap setup/recovery with signed pending-MFA token, local password login, directory password login, signed pending-MFA token, local/directory TOTP verification, and Aegis protected-secret migrate/rotate/reset. | None for api-backend. | Internal Python Aegis bridge removed; Python auth route/runtime code removed. |
| Passkeys | Done |
Current-user passkey list, label update, delete, WebAuthn registration/authentication options, signed ceremony state, registration verification, passwordless login verification, Aegis-encrypted passkey bundle storage, lookup-HMAC matching, and sign-count update. | None for public api-backend route surface. Python passkey helper module remains only for retained Python Aegis migration/helper code, not as Flask API fallback. | Python passkey list/update/delete handlers and WebAuthn ceremony handlers removed; Go routes no longer proxy this surface. |
| Users, RBAC, MFA admin, site assignments | Done |
User list/create/delete, role update, admin password reset, current-user password reset, MFA enable/disable/reset, current-user MFA reset, assignment selection/assign. | Directory cached-user enable/disable remains part of Directory Services, not this user/RBAC route surface. | Python access_management/users.py deleted in current cleanup slice. |
| Directory services | Done |
Provider/site reads, provider create/update/delete, LDAPS certificate metadata fetch, provider connectivity test, cached-user sync/disable, lookup diagnostics, effective access preview, LDAP/LDAPS directory login, AD-compatible simple bind, JIT cached-user upsert, directory MFA pending-token flow, group role mapping, and group-to-site assignment refresh. | None for public api-backend route surface. Kerberos/GSSAPI is intentionally not implemented in the Go port; AD providers use LDAP/LDAPS bind. | Python directory_services.py deleted and Python auth no longer registers Directory Services routes. |
| Credentials and GitHub token | Done |
Credential list/detail/create/update/delete, Aegis-backed credential secret encryption/clear/reset metadata handling, GitHub token read/write, token verification, and repo-hash refresh after token update. | None for api-backend. | Python credential/GitHub route modules and internal Aegis bridge deleted. |
| Server admin, logs, settings | Done |
Server time read, overview, runtime settings reads/mutations, worker list, logs, log retention/deletion, Agent release-channel read/mutation/refresh, container service actions, non-container systemd restart, and WireGuard recover queueing. | No Flask API fallback remains. Python Agent release-channel helper is retained read-only for not-yet-cutover device/watchdog code. | Python server/info.py, empty server API namespace, obsolete Python server-info/release-channel unit tests, and Python release-channel mutation/download/poller paths removed/disabled in current cleanup slice. |
| Timezone management | Done |
No WebUI timezone changer and no Go route for timezone POST/PUT. | None. Operators manage host timezone from host CLI. | Keep API absent. Do not reintroduce WebUI timezone mutation. |
| Agent enrollment, token, update, script, metadata, hash, repo manifest | Done |
Enrollment request/poll, token refresh, script request, Agent metadata read, software-management override read, update manifest/download, hash update/list, repo hash read. | Agent heartbeat/status/detail ingest remains a separate callback row. | Python token/enrollment packages, duplicate Agent read/update/script handlers, and hash/repo handlers removed in current cleanup slice. |
| Agent heartbeat, status, and detail ingest callbacks | Done |
POST /api/agent/heartbeat, POST /api/agent/status, and POST /api/agent/details, including bearer/DPoP auth, heartbeat metrics updates, metadata-field sync acks, role-health normalization, inventory/service/software/session/process normalization, software audit sync, Agent update reconciliation, duplicate suppression, and Go operator realtime fanout for agent_status_changed, device_services_changed, and device_inventory_changed. |
No Flask API fallback or Python internal fanout bridge remains. | Python callback and internal fanout bridges deleted; WebUI receives Go-originated events over Go SSE. |
| Device inventory, search, details, description, release channel, purge | Done |
Device list/search/detail, per-device detail, description update, Agent release-channel assignment, agents list, device purge barrier creation, scheduled-job target pruning, trust/runtime row cleanup, VPN/VNC runtime revocation. | Agent detail ingest moved to Go under Agent callback domain; no Flask fallback remains for public WebUI device-management routes. | Python device-management API file deleted; Go owns destructive purge path with focused tests. |
| Activity history | Done |
Device activity list/delete and per-activity detail reads. | No Flask API fallback remains. | Python assembly execution API module deleted; activity-history Flask handlers removed. |
| Metadata fields, device metadata, list views, filters | Done |
Metadata field reads/updates, per-device metadata, saved device-list views, filter CRUD/search/preview/usage/archive/clone. | No required Python runtime dependency known. | Python API route registration and duplicate route modules removed in current cleanup slice; keep shared metadata/filter helpers used by other Python domains. |
| Sites, enrollment codes, device approvals | Done |
Site list/create/delete/assign/rename/auto-approval, site device map, enrollment-code list/deprecated responses/delete, device approval approve/deny. | No required Python runtime dependency known. | Python API route registration, mixed site handlers, and isolated admin approval route module removed in current cleanup slice. |
| Software, process, service, Agent maintenance | Done |
Software audit, icon, refresh, override/block actions, bulk software actions, single-device and bulk software uninstall dispatch, process list/terminate, service list/action, bulk Agent maintenance queueing, single-device Agent update dispatch. | Agent and site-worker still execute service/process/software quick-job events; no Flask API fallback remains for this row. | Python process route module and Python device service/software/Agent-maintenance route module are deleted; Go signs uninstall quick jobs and emits targeted site-worker quick_job_run events. |
| Remote-operation session broker | Done |
Signed scoped operation-token broker for shell, desktop, and file capabilities. | No required Python runtime dependency known. | Python route registration and isolated broker module removed in current cleanup slice. |
| Remote shell | Done |
Public /api/shell/establish and /api/shell/disconnect, site access/device resolution through Go remote-op broker, remote-op token assembly, and Go VPN tunnel session preparation. |
Site-worker Python still owns shell Socket.IO runtime; this is worker runtime, not Flask API fallback. | Python devices/shell.py deleted; shell now uses Go VPN runtime. |
| Remote file management | Done |
Browse roots/children, upload conflict check, text read/write, mkdir, rename, move, delete, paste, upload/download transfer start, transfer status/cancel/content proxy. | Site-worker Python still owns transfer session storage, Agent transfer callbacks, and artifact streaming runtime; no Flask API fallback remains. | Python devices/file_management.py deleted; shared transfer store moved to services/remote_files/transfers.py for site-worker use. |
| Remote desktop, tunnel, VPN runtime | Done |
Tunnel connect/status/active, Agent VPN ensure/ready, Agent VNC ensure, VNC viewer capability, VNC establish/session/disconnect/handoff/session listing, internal VNC session events, WireGuard listener/key/IP lease management, and scheduler VPN session helpers. | Site-worker/Agent still perform live VNC, shell, and WireGuard service events; no Flask API fallback remains. | Python devices/tunnel.py, devices/vnc.py, VPN/* runtime modules, and RemoteDesktop/vnc_sessions.py deleted; Python scheduler/workflow callers now use Go internal VPN endpoints. |
| Assemblies, scripts, quick execution | Done |
Go owns default assembly list/detail, create/update/delete, clone, import/export, Dev Mode write controls, Aurora official catalog refresh, single official update, update-all, and /api/scripts/quick_run signed dispatch through site-worker host-service events. |
Site-worker still executes signed host-service events; this is worker runtime, not api-backend fallback. | Python AssemblyCache bridge and API assembly route tree removed. |
| Workflows | Done |
Workflow run reads, node reads, webhook CRUD, manual workflow start, editor access, run resolution, webhook token trigger, internal job-scheduler workflow-start bridge, scheduled workflow work-item execution from the Go job-scheduler manager, persisted workflow run/node lifecycle, script workflow execution through signed Go quick-run dispatch, Ansible workflow-node dispatch through site-worker /automation/ansible/run, Execute Subworkflow node execution, and Go realtime workflow/activity fanout. |
None for api-backend. | Python WorkflowRuntimeService helpers and workflow API route tree removed. |
| Watchdogs | Done |
Watchdog metadata, list/detail, create/update, preview, delete, incident list/count reads, incident acknowledgement/state changes, device-specific watchdog read view, device-specific override mutation, background due evaluation loop, immediate post-save evaluation, incident state persistence, offline-only resolved-incident purge, notification/service-control/script/workflow/Ansible remediation dispatch, and Go realtime watchdog/device fanout. | Site-worker/Agent still execute service-control, script, and Ansible remediation events; this is worker runtime, not Flask API fallback. | Python watchdog runtime module and legacy Python watchdog API tests are removed; Python watchdog management is a no-op compatibility shim. |
| Scheduled and onboarding jobs | Done |
Job list/detail, create, update, rerun, delete, run history clear, toggle, device targets, onboarding target read, onboarding redeploy queue reset, internal credential decrypt helper, internal public-base-url, host-service-event helpers, internal VPN session prepare/snapshot helpers, internal work-item enqueue helper, scheduler/workflow runtime credential/service-account/online-host/online-site fetch through Go internal helpers, Go job-scheduler manager role for due ticks, queue lease cleanup, service-action queueing, orchestrator-backed site-worker lifecycle, stale-image reconcile, route file reconcile/cleanup, service snapshots, global scheduled workflow dispatch, scheduled workflow work-item execution, scheduled script/Ansible work-item dispatch, Agent maintenance work-item execution through site-worker host-service calls, onboarding work-item retirement, and Go-native job-scheduler/orchestrator container healthchecks. | Site-worker Python keeps remote-ops sockets and /automation/ansible/run; it no longer claims api-backend scheduler work items. |
Python scheduled-job API helpers, Flask route tree, Python job-scheduler container healthcheck, and job-scheduler Python image/dependency copy are removed. Automatic remote onboarding execution remains retired; operators use generated Agent install commands until Go-native remote onboarding execution is implemented. |
| Notifications | Done |
Public authenticated notification payload validation/dispatch route and Go operator realtime fanout for borealis_notification. |
No Flask API fallback or Python internal notification bridge remains for public notification dispatch. | Python public notification duplicate and internal notification broadcast endpoint removed; WebUI receives Go-originated toasts over Go SSE. |
Cleanup Queue¶
| Priority | Target | Why |
|---|---|---|
| Done | Metadata fields, device metadata, list views, filters | Python route registration removed, isolated Python route files deleted, mixed device-list-view handlers removed, Go handlers no longer fallback for this domain. |
| Done | Sites, enrollment codes, device approvals | Python route registration removed, isolated Python admin approval module deleted, mixed site handlers removed, Go handlers no longer fallback for this domain. |
| Done | Remote-operation session broker | Python route registration removed and isolated Python broker module deleted. |
| Done | Agent enrollment/token/update/script/hash/repo routes | Python token/enrollment packages deleted, duplicate Agent read/update/script handlers removed, mixed hash/repo handlers removed. |
| Done | Auth profile/logout and simple user/RBAC duplicates | Python profile/logout, passkey list/update/delete, user list/delete/role, MFA admin/reset, and site-assignment route handlers removed; ceremony/password-reset flows retained. |
| Done | Server admin duplicate reads and settings | Python server time, overview, worker read, ansible/site-worker settings, Agent release-channel GET, and container service-action route handlers removed; Go fallbacks tightened for fully cutover paths. |
| Done | Directory and credential read duplicates | Python directory provider/site GET handlers and credential list/detail GET handlers removed; mutation fallbacks retained. |
| Done | Low-risk duplicate/public cleanup | Python authentication.py, server/log_management.py, public notification dispatch route, public bootstrap state/setup/unlock duplicates, and public Aegis status duplicate removed; internal bridges and retained Python lifecycle routes left intact. |
| Done | User create and password reset | Go owns local user create, admin password reset, and current-user password reset; Python access_management/users.py deleted. |
| Done | Remote Shell and Remote File Management public API | Go owns shell establish/disconnect and full public remote-file API surface; Python shell/file Flask route modules deleted; site-worker runtime helpers retained outside compatibility API. |
| Done | Agent release-channel mutation/refresh and WireGuard recovery | Go owns final Server Info mutations and server/info.py route surface is deleted. |
| Done | Timezone API | Mutation route remains removed; Server Info keeps read-only clock display only. Operators manage host timezone from host CLI. |
| Done | Remote Desktop, tunnel, VPN runtime | Go owns VNC public routes, tunnel public routes, Agent VPN/VNC callbacks, WireGuard session tracking, and scheduler VPN helpers; Python Flask route/runtime modules deleted. |
| Done | Device inventory/search/details/description/release-channel/purge | Go owns public WebUI device-management surface including destructive purge; Python device-management API route tree is deleted. |
| Done | Aegis lifecycle and admin bootstrap public API | Go owns public Aegis setup/unlock/rotate/force-reset and admin bootstrap setup/recovery/MFA verification; Python public/internal lifecycle/bootstrap route handlers and internal Aegis bridge are deleted. |
| Done | Credential and GitHub token mutations | Go owns credential create/update/delete and GitHub token get/update; Python credential/GitHub token route modules and internal Aegis bridge are deleted. |
| Done | WebAuthn passkey ceremonies | Go owns passkey registration/authentication options and verify routes with signed ceremony state; Python WebAuthn route handlers removed. |
| Done | Directory Services and directory login | Go owns provider mutations, LDAP/AD login, diagnostics, sync, certificate fetch, and cached-user disable; Python Directory Services route/manager module deleted. |
| Done | Process Management | Go owns process list and terminate routes; Python devices/processes.py route module deleted. |
| Done | Internal job-scheduler credential helper | Go owns /api/internal/job-scheduler/credential/<id> and decrypts credential secrets after closing DB handles; Python workflow runtime consumes this Go helper and the Python internal credential route is removed. |
| Done | Scheduled job create/update | Go owns /api/scheduled_jobs POST and /api/scheduled_jobs/<id> PUT with site-scoped target persistence, component/domain validation, onboarding save validation, and credential-reset warning behavior; matching Python Flask route handlers removed. |
| Done | Scheduled job rerun | Go owns /api/scheduled_jobs/<id>/rerun and records workflow, script/device, shared Ansible, individual Ansible, and onboarding pending/skipped occurrence snapshots; matching Python Flask route handler removed. |
| Done | Onboarding redeploy route | Go owns /api/onboarding/jobs/<id>/redeploy, clears prior run history, creates a fresh pending onboarding occurrence, and returns queued run IDs; matching Python Flask route handler removed. |
| Done | Software/process/service/Agent-maintenance route module | Go owns service/process/software/Agent-maintenance public API, including /api/device/software/<hostname>/uninstall and /api/software/uninstall; Go writes signed software-management quick-job activity rows, emits targeted site-worker events, and applies Go-side uninstall override/blocklist capability resolution. Python devices/services.py is deleted. |
| Done | Scheduled job public/helper fallback cleanup | Go owns public scheduled-job reads/mutations, onboarding target read/redeploy, and internal scheduler helper endpoints; Python public scheduled-job/onboarding route handlers and duplicate internal helper route handlers are removed. |
| Done | Agent heartbeat/status callback routes | Go owns POST /api/agent/heartbeat and POST /api/agent/status; Python callback routes and internal fanout bridge are deleted. |
| Done | Activity history route fallback cleanup | Go owns /api/device/activity/<hostname> and /api/device/activity/job/<id>; duplicate Python handlers removed from assembly execution module. |
| Done | Workflow read and webhook CRUD fallback cleanup | Go owns workflow run list/detail, node-run detail, and webhook list/create/delete routes; Python workflow API route tree is deleted. |
| Done | Agent details ingest route fallback cleanup | Go owns POST /api/agent/details with inventory normalization, service pending-action preservation, software audit sync, icon asset upsert, update-status reconciliation, and duplicate suppression; Python device-management API route tree is deleted. |
| Done | Watchdog read fallback cleanup | Go owns watchdog list/detail/metadata and incident list/count reads; Python watchdog API route tree is deleted. |
| Done | Watchdog incident state fallback cleanup | Go owns incident acknowledge/state mutation routes and emits watchdog_incidents_changed through Go realtime; Python watchdog API route tree is deleted. |
| Done | Watchdog delete fallback cleanup | Go owns watchdog delete, dependent watchdog row cleanup, and watchdog/device refresh fanout; Python watchdog API route tree is deleted. |
| Done | Device watchdog read fallback cleanup | Go owns /api/devices/<device>/watchdogs assignment/incident/override reads; Python watchdog API route tree is deleted. |
| Done | Watchdog preview fallback cleanup | Go owns /api/watchdogs/preview target resolution and rule preview evaluation; later watchdog cleanup rows remove the remaining public Flask watchdog handlers. |
| Done | Watchdog device override fallback cleanup | Go owns POST /api/devices/<device>/watchdogs/overrides, persists per-device overrides, immediately suppresses active watchdog incident/state on override set, and emits Go realtime refresh events. |
| Done | Watchdog create/update fallback cleanup | Go owns POST /api/watchdogs and PUT /api/watchdogs/<id> policy persistence, validation, target/action normalization, and realtime refresh fanout; Python watchdog API/runtime code is deleted. |
| Done | Agent/device operator fanout bridge | Go owns /api/realtime/events SSE for Go-originated Agent status and device inventory/service events; Python devices/management.py internal bridge is deleted. |
| Done | Notification internal broadcast bridge | Go public notification route emits borealis_notification through Go realtime; Python notification internal broadcast route is deleted. |
| Done | Scheduler API credential fetch cleanup | API-attached Python scheduler runtime now fetches decrypted credentials through Go /api/internal/job-scheduler/credential/<id> instead of querying and decrypting credentials locally. |
| Done | Scheduler credential fallback cleanup | Shared Python scheduler now fails closed when no credential fetcher is configured instead of reading credential secret columns directly. |
| Done | Scheduler service-account helper cleanup | Python scheduler runtimes now fetch WinRM service-account payloads through Go /api/internal/job-scheduler/service-account/<agent_id> instead of reading agent_service_account locally. |
| Done | Scheduler online-host helper cleanup | Python scheduler API/manager runtimes now fetch online host snapshots through Go /api/internal/job-scheduler/online-hosts instead of reading devices.last_seen locally. |
| Done | Scheduler online-site helper cleanup | Python scheduler manager/site-worker runtimes now fetch online site snapshots through Go /api/internal/job-scheduler/online-sites instead of joining device_sites and devices.last_seen locally. |
| Done | Scheduler API work-item enqueue cleanup | API-attached Python scheduler runtime now posts onboarding/scheduled/workflow work-item enqueue requests to Go /api/internal/job-scheduler/work-items; Go inserts queue rows and marks onboarding runs running. |
| Done | Scheduler API direct execution fallback cleanup | API-attached Python scheduler now requires queue dispatchers and fails closed instead of directly running scheduled scripts, workflows, shared Ansible, or onboarding threads inside the API process. |
| Done | Scheduler compatibility group cleanup | Python compatibility default startup no longer registers the scheduled_jobs group or initializes API-attached scheduler runtime; workflow-start internal bridge moved under the retained workflow execution group. |
| Done | Assembly management Go port | Go owns default assembly CRUD/import/export/clone/dev-mode routes, Aurora catalog refresh/update, and quick-run dispatch; Python API assembly route tree is deleted. |
| Done | Assembly management Python fallback cleanup | Broad Python assembly management blueprint and internal cache bridge are deleted. |
| Done | Aurora official catalog route cleanup | Go owns refresh_catalog, single official update, and update-all with Git checkout, catalog crawl, official assembly import, catalog-state persistence, and deleted-official cleanup. Python catalog Flask route files are deleted. |
| Done | Quick-run dispatch route cleanup | Go owns /api/scripts/quick_run, resolves assemblies from Go tables, signs script payloads, writes activity rows, emits targeted site-worker quick_job_run events, and broadcasts activity changes through Go realtime. Python assembly execution API module is deleted. |
| Done | Workflow execution route cleanup | Go owns workflow start, editor access, run resolution, webhook trigger, internal job-scheduler workflow-start bridge routes, Ansible nodes, and subworkflow nodes; Python workflow helpers are deleted. |
| Done | Watchdog evaluation/remediation runtime cleanup | Go owns watchdog due evaluation, immediate post-save evaluation, incident lifecycle, offline-only resolved cleanup, remediation dispatch, and realtime fanout; Python watchdog runtime and obsolete Python watchdog API tests are deleted. |
| Done | Job-scheduler manager runtime cleanup | Go binary now runs the job-scheduler container manager role, owns scheduled due ticks, queue lease cleanup, service-action queueing, orchestrator-backed site-worker lifecycle, stale-image reconcile, worker route reconciliation/cleanup, service snapshots, and global scheduled workflow dispatch. Python job_scheduler/manager.py is deleted. |
| Done | Agent maintenance work-item execution cleanup | Go job-scheduler manager claims agent_maintenance_run work items, waits for Agent system socket acknowledgement through the active site-worker host-service call endpoint, updates scheduled run/activity status, and excludes that kind from Python site-worker claims. Python site-worker Agent maintenance executor/tests are removed. |
| Done | Scheduled workflow work-item execution cleanup | Go job-scheduler manager claims scheduled_workflow_run work items by kind and starts them through the Go internal workflow-start bridge. Python site-workers no longer claim or dispatch scheduled workflow work items. |
| Done | Scheduled script/Ansible work-item execution cleanup | Go job-scheduler manager claims scheduled_run work items by kind, signs scheduled script payloads, records activity links, prepares credential/VPN-backed Ansible target specs, queues Ansible runner work through active site-workers, and excludes scheduled-run claims from Python site-workers. |
| Done | Onboarding work-item execution retirement | Go job-scheduler manager claims onboarding_run work items and marks their run/target history failed with explicit retirement guidance. Python site-workers keep remote-ops sockets and /automation/ansible/run but no longer import/build the Python scheduler or claim work items. |
| Done | Job-scheduler container Python cleanup | Job-scheduler image uses the compiled Go api-backend binary in scheduler mode on a slim Debian runtime image, with Go healthcheck mode replacing the Python healthcheck and Ansible preflight removed from the scheduler entrypoint. Site-worker containers remain Python/Ansible-capable. |
Hard Python Dependencies¶
| Dependency | Blocks |
|---|---|
| Done | No Python dependency blocks api-backend rewrite or route ownership. Retained Python is worker/helper runtime only: site-worker remote-ops sockets, file-transfer streaming, VNC proxy helpers, Ansible runner, crypto migration helpers, and assembly catalog/cache helpers. |
Final Codebase Troubleshooting Issues¶
| Priority | Severity | Issue | Reported evidence | Initial focus | Status |
|---|---|---|---|---|---|
| 1 | Critical | Site-worker restart/panic during quick and scheduled jobs leaves runs orphaned as Running; Site List assigned-task count does not show active job. |
Quickjob assembly job 140 installed 7-Zip on Windows but stayed Running; Site List Assigned Tasks column never changed; site-worker restarted twice. Scheduled job 141 behaved similarly. |
job_scheduler site-worker lifecycle, Go scheduler completion handling, host-service event acknowledgement, activity/run terminal-state updates, and Site_List.jsx assigned-task data source. |
Resolved by operator testing; keep regressions. |
| 2 | Critical | Windows Remote Shell fails with remote_shell_backend_unreachable; Linux Agent shell works. |
Repeated Windows shell attempts fail after retries; Linux remote shell path succeeds; attached Windows Agent log showed PowerShell sessions returning output, then TCP reset mid-command. Engine correlation first showed the LAB-OPERATOR-01 Agent Socket.IO websocket disconnecting at the same timestamp as shell closure after the Go Agent blocked its socket read loop on remote-op handlers and missed Engine.IO pings. Follow-up heartbeat testing showed the rebuilt Agent kept the shell TCP stream alive longer, but the site-worker was still running Socket.IO in threading mode on Werkzeug and logged write() before start_response at Agent socket disconnect timestamps. Later eventlet testing removed the Werkzeug symptom and exposed site-worker exits from heartbeat DB deadlocks during active remote shell sessions. Latest heartbeat testing shows workers stay up, while active shell TCP closes when Agent Socket.IO transport disconnects or after fast remote-shell recovery restarts. Follow-up Engine correlation showed LAB-OPERATOR-01 Agent socket reconnecting on roughly watchdog heartbeat-stale cadence even outside shell sessions. |
Windows Agent shell host/socket registration, Agent Socket.IO handler dispatch, site-worker remote-ops socket bridge, Go /api/shell/establish target routing, WireGuard peer readiness. |
Resolved by operator testing; Agent Socket.IO traffic now refreshes socket liveness, and watchdog no longer restarts a locally healthy Agent solely for stale HTTP heartbeat when live socket traffic proves the control channel is active. |
| 3 | Critical | Remote Desktop cannot connect to target device. | Device is current on same branch as Engine; connection fails; WebUI reaches Authenticating Session and never reaches Desktop Ready; guacd reported Unable to connect to VNC server after Go /api/vnc/establish returned success; operator validation connected twice after an earlier fix, then later testing showed repeated credential/start requests and UI stuck at Authenticating Session. Engine correlation showed VNC sessions registered successfully, followed by guacd/browser closes before display instructions and one Guacamole proxy rejected session for a token issued seconds earlier. Go VNC establish also force-restarted the Agent tunnel after a failed 750ms fast probe even when the later recovery probe succeeded. June 24 testing showed site-worker still performed an RFB VNCAuth diagnostic probe before Guacamole, producing This server does not have a valid password enabled and too many attempts UltraVNC lockout responses; later same-day multi-device testing showed Agent UltraVNC could remain STOP_PENDING while port 5900 still accepted TCP, causing Engine/guacd false-ready behavior. Follow-up Agent logs showed rapid reconnects overlapped operator_disconnect/component_unmount, vnc_establish, startup, recovery, and always-on ensure paths; LAB-DC-01 hit Agent VNC firewall ensure failed and Engine vnc_backend_unreachable at 2026-06-24 19:33:23, while LAB-DC-03 logged duplicate firewall ensure failures during overlapping disconnect/reconnect work. |
VNC readiness callbacks, Guacamole/session handoff, remote desktop site-worker route, Windows Agent desktop role. | Completed by operator validation; keep Remote Desktop readiness, Guacamole retry, first-frame, stop-debounce, and lifecycle serialization regressions. |
| 4 | High | Ansible SSH quickjob fails to load staged private key with error in libcrypto. |
Ansible playbook quickjob 143 failed loading /opt/Borealis/Engine/Services/api-backend/cache/Ansible/Generated/Runtime/d200f86ceb7d426ca9601ec5b1c43a8b/runtime/auth/id_borealis_ssh; SSH then returned public-key permission denied; operator validation confirmed SSH fixed after private-key staging normalization. |
Go credential decrypt/normalization, scheduler_execution.go runtime file staging, site-worker Ansible runner key file write mode/content, CRLF or escaped-newline handling. |
Resolved by operator testing; keep regressions. |
| 5 | High | Agent socket instability causes transient service-management agent_unavailable. |
Service action failed once with agent_unavailable; Engine socket role showed disconnected/degraded, then self-resolved and service control worked; Agent Socket later degraded then reconnected. |
Agent socket lifecycle, role health ingestion, site-worker socket registry, Go service-management dispatch retry/error mapping. | Fix implemented; service-control dispatch now retries transient SYSTEM socket gaps before surfacing agent_unavailable, and persistent unavailability maps to HTTP 503. Pending operator validation. |
| 6 | Medium | Agent redeploy over existing install preserves stale branch/channel metadata in informational update path. | Redeploy still references old branch feature/convert-agent-to-windows-service-with-watchdog; repo hash API returns GitHub 422 and fallback commit API returns 422 for missing SHA/branch. |
Agent redeploy script generation, release-channel/branch overwrite semantics, repo hash cache invalidation, Agent update target resolution. | Open |
| 7 | Medium | Server Info reports Let's Encrypt SSL Certificate as unavailable and Active Operator Sessions as unknown. |
Server_Info.jsx shows unavailable certificate status and unknown active sessions. |
Go server overview/status payload parity, Traefik certificate state read path, operator session counter source. | Fix implemented; Server Overview now reads Traefik ACME certificate state and counts live operator SSE sessions. Pending operator validation. |
Validation Snapshot¶
| Slice | Last reported validation |
|---|---|
| Final Go-only api-backend and scheduler-container cutover | Focused Go workflow/auth/Aegis/assembly/health/scheduler tests; Python syntax compile for retained worker/helper modules; focused retained Python pytest through repo test venv; route/residue scan confirms no Python API route tree, no Flask app factory, no compatibility backend supervision/proxy, no internal Aegis bridge, no workflow runtime helper, no scheduled-job API helper, no AssemblyCache bridge, and no Python job-scheduler container runtime remain; full go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health, Go status, job-scheduler healthcheck, and auth-gate smokes; git diff --check. |
| Agent maintenance/update Go port | go test ./..., build-api-backend.sh, sudo bash Engine.sh --network-mode public|local deploy prod, safe live route smoke. |
| Notification Go dispatch | Go tests, build, deploy, safe notification smoke through Python broadcast bridge. |
| Bootstrap/Aegis/local login/MFA Go port | Go tests, build, deploy, live /health, /api/bootstrap/state, /api/aegis/status, local login/MFA-safe checks. |
| Metadata/filter/view Python route cleanup | go test ./..., build-api-backend.sh, Python syntax compile for touched Flask modules, sudo bash Engine.sh --network-mode public|local deploy prod, public Go 401 smoke, internal Python 404 route-removal smoke. |
| Site/admin/remote-op broker Python route cleanup | go test ./..., build-api-backend.sh, Python syntax compile for touched Flask modules, sudo bash Engine.sh --network-mode public|local deploy prod, public Go 401 smoke, internal Python route-file absence and route-removal smoke. |
| Agent enrollment/token/update/script/hash/repo Python route cleanup | Python syntax compile for touched Flask modules, no Python duplicate route residue, go test ./..., build-api-backend.sh, sudo bash Engine.sh --network-mode public|local deploy prod, public Go route smoke, exact POST validation smoke, internal Python 404 route-removal smoke. |
| Auth/profile/user/RBAC Python route cleanup | Python syntax compile for touched Flask modules, no Python duplicate route residue, go test ./..., build-api-backend.sh, sudo bash Engine.sh --network-mode public|local deploy prod, public Go smoke, runtime Python source scan, internal Python route-removal smoke. |
| Server-admin Python route cleanup | Python syntax compile for touched Flask module, no Python duplicate route residue, go test ./..., build-api-backend.sh, sudo bash Engine.sh --network-mode public|local deploy prod, public Go smoke, retained Python mutation smoke. |
| Directory/credential read cleanup | Python syntax compile for touched Flask modules, no Python duplicate route residue, go test ./..., build-api-backend.sh, sudo bash Engine.sh --network-mode public|local deploy prod, public Go smoke, retained Python mutation smoke. |
| Low-risk duplicate/public cleanup | Python syntax compile for touched Flask modules, no Python duplicate route residue, go test ./..., build-api-backend.sh, sudo bash Engine.sh --network-mode public|local deploy prod, public Go smoke, internal Python route-removal smoke, operator WebUI smoke for login/logout, Server Info, Engine Log Access, device inventory, filters/views, sites/approvals, Access Management, and credential reads, git diff --check. |
| User create/password reset Go cutover | Go unit tests for create/admin reset/current-user reset dispatch, Python syntax compile for retained auth module, no Python user route residue, go test ./..., build-api-backend.sh, sudo bash Engine.sh --network-mode public|local deploy prod, safe route smoke, git diff --check. |
| Remote Shell/File Management cutover | Go unit tests for shell establish token assembly, upload transfer start, and transfer content proxy; Python syntax compile for API package, site-worker, and moved transfer helpers; Python route residue scan; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; public unauth Go route smoke; direct Python route-removal smoke; operator WebUI smoke confirmed Remote Shell and File Management working. |
| Server-admin final Go cutover | Go unit tests for Agent release-channel refresh/cache packaging, systemd restart queueing, and WireGuard recover queueing; Python syntax compile for retained API package and read-only Agent release helper; Python route residue scan; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; public Go health/auth-gate smoke; direct Python route-removal smoke. |
| Remote Desktop/tunnel/VPN runtime cutover | Go unit tests for VNC/tunnel route ownership, VNC credential worker-call unwrapping, and internal scheduler route wiring; Python syntax compile for retained API package, agent routes, scheduled jobs, workflows, server, RemoteDesktop, VPN, and WebSocket package; route residue scan; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke; direct Python route-removal smoke; operator WebUI smoke confirmed Remote Desktop working on lab-operator-01. |
| Device management/purge cutover | Go unit tests for purge handler dispatch, admin gate, scheduled-target prune parity, and VNC session revocation; Python syntax compile for retained Agent detail ingest module; route residue scan confirms no Flask registration for public device-management routes; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public unauth Go auth-gate smoke for /api/devices and purge; direct Python POST purge smoke confirms compatibility backend no longer accepts purge mutation. |
| Aegis lifecycle/admin bootstrap cutover | Go unit tests for Aegis rotate handler and admin bootstrap signed pending-token flow; Python syntax compile for retained auth/passkey/directory login module and internal Aegis bridge; route residue scan confirms Python public Aegis/admin-bootstrap handlers removed; follow-up cursor-close fix for live unlock pq protocol error; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go bootstrap/Aegis auth-gate smoke; direct Python route-removal smoke. |
| Credential/GitHub token mutation cutover | Go unit tests for credential create/update/delete route ownership and GitHub token lock/store paths; Python syntax compile for retained login module; Python credential/GitHub route residue scan; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for /api/credentials and /api/github/token; direct Python route-removal smoke confirms GitHub route 404 and credential mutation route no longer accepts POST. |
| Passkey WebAuthn ceremony cutover | Go unit tests for register/authenticate options and invalid signed-session rejection; Python syntax compile for retained login module; Python passkey ceremony route residue scan; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go passkey route bootstrap-gate smoke; direct Python GET route-removal smoke; direct Python POST method-rejection smoke; git diff --check. |
| Directory Services cutover | Go unit tests for route ownership and AD binary LDAP attribute formatting plus full go test ./...; Python syntax compile for retained login/tests; route residue scan confirms Python Directory Services registration removed; api-backend build passes; engine deploy healthy; safe local smoke returned health 200, unauthenticated directory routes 401, Aegis-locked login 423, and direct Python Directory Services POST non-success 405. |
| Process Management fallback cleanup | Python syntax compile for retained API package; route residue scan confirms Python process route registration removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke; direct Python route-removal smoke; git diff --check. |
| Internal scheduler credential helper cutover | Go unit tests for internal credential route, reset-required mapping, and decrypted scheduler payload; Python syntax compile for workflow/scheduler management modules; route residue scan confirms Python internal credential endpoint removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go internal credential route auth-gate smoke; direct Python internal credential route-removal smoke; git diff --check. |
| Scheduled job create/update cutover | Go unit tests for create/update route ownership and full scheduler test lane; Python syntax compile for retained scheduler module; route residue scan confirms Python POST/PUT scheduled-job handlers removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke; direct Python route-removal smoke; git diff --check. |
| Scheduled job rerun cutover | Go unit tests for rerun route ownership and full scheduler test lane; Python syntax compile for retained scheduler module; route residue scan confirms Python rerun handler removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke; direct Python route-removal smoke; git diff --check. |
| Onboarding redeploy route cutover | Go unit tests for redeploy route ownership and scheduler queue-reset behavior; Python syntax compile for retained onboarding scheduler runtime; route residue scan confirms Python redeploy handler removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke; direct Python route-removal smoke; git diff --check. |
| Software/process/service/Agent-maintenance cutover | Go unit tests for signed uninstall quick-job dispatch and Go capability resolver paths; Python syntax compile for retained API package; route residue scan confirms Python device service/software/Agent-maintenance route module removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for single and bulk uninstall routes; direct Python route-removal smoke; git diff --check. |
| Scheduled job public/helper fallback cleanup | Python syntax compile for retained scheduler modules; route residue scan confirms Python public scheduled-job/onboarding route handlers and duplicate internal helper endpoint handlers are removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for scheduled-job/onboarding/internal helper routes; direct Python route-removal smoke; workflow-start bridge auth-gate smoke; git diff --check. |
| Agent heartbeat/status callback cutover | Go unit tests for authenticated heartbeat/status handlers, metadata ack response, status broadcast dispatch, and duplicate suppression; Python syntax compile for retained device management bridge; route residue scan confirms old Python heartbeat/status callback module removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for heartbeat/status/details; direct Python heartbeat/status POST non-success smoke; Python internal status bridge auth-gate smoke; git diff --check. |
| Activity history fallback cleanup | Python syntax compile for retained assembly execution module; route residue scan confirms Python activity-history handlers removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for activity-history routes; direct Python activity-route POST/GET route-removal smoke; git diff --check. |
| Workflow read/webhook CRUD fallback cleanup | Python syntax compile for retained workflow management module; route residue scan confirms Python workflow run-read and webhook CRUD handlers removed while execution/editor/resolve/trigger routes remain; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for workflow read/webhook routes; direct Python route-removal smoke for removed routes; retained Python route auth-gate smoke; git diff --check. |
| Agent details ingest cutover | Go unit tests for authenticated details ingest, duplicate suppression, inventory event fanout, software dedupe, and service pending-action preservation; Python syntax compile for retained internal bridge; route residue scan confirms Python /api/agent/details handler removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for /api/agent/details; direct Python route-removal smoke; internal bridge auth-gate smoke; git diff --check. |
| Watchdog read fallback cleanup | Python syntax compile for retained watchdog management module; route residue scan confirms Python watchdog list/detail/metadata/incident-list handlers removed while preview/mutation/ack/state/device routes remain; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for watchdog reads; direct Python route-removal smoke for removed reads; retained Python mutation auth-gate smoke; git diff --check. |
| Watchdog incident state fallback cleanup | Go unit tests for incident acknowledge/state handlers and realtime fanout; Python syntax compile for retained watchdog management module; route residue scan confirms Python incident acknowledge/state handlers removed while preview/create/update/delete/device routes remain; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for incident acknowledge/state routes; direct Python route-removal smoke for removed incident mutation routes; retained Python mutation auth-gate smoke; git diff --check. |
| Watchdog delete fallback cleanup | Go unit tests for delete route ownership, dependent-host refresh fanout, and incident/device realtime events; Python syntax compile for retained watchdog management module; route residue scan confirms Python DELETE handler removed while preview/create/update/device routes remain; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for watchdog delete; direct Python DELETE route-removal smoke; retained Python create/update auth-gate smoke; git diff --check. |
| Device watchdog read fallback cleanup | Go unit tests for /api/devices/<device>/watchdogs route ownership and assignment payload dispatch; Python syntax compile for retained watchdog management module; route residue scan confirms Python device watchdog GET handler removed while override POST remains; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go auth-gate smoke for device watchdog reads; direct Python GET route-removal smoke; retained Python override auth-gate smoke; git diff --check. |
| Watchdog preview fallback cleanup | Go unit tests for /api/watchdogs/preview route ownership, validation mapping, and offline-rule evaluation; Python syntax compile for retained watchdog management module; route residue scan confirms Python preview handler removed while create/update/override routes remain; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go preview auth-gate smoke; direct Python preview route-removal smoke returned non-success (GET 404, POST 405); retained Python create/update auth-gate smoke; git diff --check. |
| Watchdog device override fallback cleanup | Go unit tests for /api/devices/<device>/watchdogs/overrides route ownership, validation mapping, and realtime fanout; Python syntax compile for retained watchdog management module; route residue scan confirms Python override Flask handler removed while create/update routes and runtime helper remain; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go override auth-gate smoke; direct Python override route-removal smoke; retained Python create/update auth-gate smoke; git diff --check. |
| Watchdog create/update fallback cleanup | Go unit tests for /api/watchdogs create and /api/watchdogs/<id> update route ownership, validation mapping, and realtime refresh fanout; Python syntax compile for retained watchdog runtime bootstrap module; route residue scan confirms Python create/update handlers removed while runtime save/evaluator helper remains until later runtime cleanup; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go create/update auth-gate smoke; direct Python create/update route-removal smoke; git diff --check. |
| Agent/device realtime fanout and notification bridge cleanup | Go unit tests for operator realtime hub, SSE event encoding/auth gate, Agent status/detail fanout, and notification dispatch; Python syntax compile for retained API package; route residue scan confirms Python internal Agent/device and notification bridge endpoints removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go /api/realtime/events auth-gate smoke; direct Python bridge route-removal smoke; public notification auth-gate smoke; git diff --check. |
| Scheduler API credential fetch cleanup | Python syntax compile for scheduler management; residue scan confirms API-attached scheduler no longer queries/decrypts credential secrets locally; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go internal credential helper auth-gate smoke; git diff --check. |
| Scheduler credential fallback cleanup | Focused Python scheduler regression confirms missing credential fetcher does not read credential tables; Python syntax compile for scheduler runtime/tests; residue scan confirms scheduler credential secret-column fallback removal; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go internal credential helper auth-gate smoke; git diff --check. |
| Scheduler service-account helper cleanup | Go unit tests for internal service-account helper route; focused Python scheduler regression confirms missing service-account fetcher does not read local tables; Python syntax compile for scheduler API/manager/worker/runtime/tests; residue scan confirms scheduler service-account direct table read removal; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go internal service-account helper auth-gate smoke; git diff --check. |
| Scheduler online-host helper cleanup | Go unit tests for internal online-host helper route; Python syntax compile for scheduler API/manager modules; residue scan confirms scheduler API/manager no longer read devices.last_seen directly for online host snapshots; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go internal online-host helper auth-gate smoke; git diff --check. |
| Scheduler online-site helper cleanup | Go unit tests for internal online-site helper route; Python syntax compile for scheduler manager/site-worker modules; residue scan confirms scheduler manager/site-worker no longer join device_sites and devices.last_seen for site online snapshots; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go internal online-site helper auth-gate smoke; git diff --check. |
| Scheduler API work-item enqueue cleanup | Go unit tests for internal work-item route auth gate, store dispatch, and Python queue payload parity; Python syntax compile for scheduler management; residue scan confirms API-attached scheduler no longer imports direct queue enqueue helpers; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; Go internal work-item helper auth-gate smoke; git diff --check. |
| Scheduler API direct execution fallback cleanup | Focused Python regressions confirm missing queue dispatchers do not directly execute scheduled runs, workflows, or onboarding threads inside api-backend; Python syntax compile for scheduler runtime modules; residue scan confirms API-attached scheduler direct execution fallback state is removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; scheduler/internal helper auth-gate smoke; git diff --check. |
| Scheduler compatibility group cleanup | Python syntax compile for scheduler/workflow/config/bootstrap modules; residue scan confirms default API group lists no longer include scheduled_jobs, and workflow-start bridge moved to workflows; focused retained Python pytest later covered scheduler queue/site-worker/Ansible host-service runtime through repo test venv; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public scheduled-job auth-gate smoke; direct Python scheduled-job route-removal smoke; workflow-start bridge auth-gate smoke; git diff --check. |
| Assembly management Go port | Go unit tests for assembly list routing, catalog-refresh fallback, and Dev Mode-gated non-user import; Python syntax compile for internal cache bridge; route scan confirms Go owns default assembly management route prefix while Python still owns Aurora catalog fallback and quick-run dispatch; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public assembly auth-gate smoke; direct Python internal cache bridge auth-gate smoke; git diff --check. |
| Assembly management Python fallback cleanup | Python syntax compile for assembly helper/catalog/cache modules; route residue scan confirms Python assembly API registration only keeps Aurora catalog refresh/update, quick-run dispatch, and internal cache bridge; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go assembly auth-gate smoke; direct Python default assembly route-removal smoke; direct Python catalog fallback auth-gate smoke; git diff --check. |
| Aurora official catalog route cutover | Go unit tests for refresh fallback annotation and update-all import/catalog-state persistence; Python syntax compile for retained assembly execution/cache modules and API package; route residue scan confirms Python Aurora catalog routes removed while quick-run/cache bridge remain; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go catalog auth-gate smoke; direct Python catalog route-removal smoke; git diff --check. |
| Quick-run dispatch route cutover | Go unit tests for validation, out-of-scope target rejection, signed worker event payload, PowerShell variable rewrite, activity insert, and Go realtime fanout; Python syntax compile for retained assembly execution helpers, workflow runtime, watchdog runtime before later runtime cleanup, cache bridge, and API package; route residue scan confirms Python quick-run Flask route removed while Go owns /api/scripts/quick_run; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go quick-run auth-gate smoke; direct Python quick-run route-removal smoke; git diff --check. |
| Workflow execution route cleanup | Focused Go workflow/internal-route tests for start/editor/resolve/webhook/internal bridge; Python syntax compile for retained workflow/scheduler helper modules; route residue scan confirms Python workflow Flask routes removed while scheduler workers call the Go internal workflow-start bridge; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go workflow auth-gate/token smoke; direct Python workflow route-removal smoke; git diff --check. |
| Watchdog evaluation/remediation runtime cleanup | Go unit tests for watchdog route ownership plus runtime cooldown/offline-only purge helpers; Python syntax compile for retained watchdog compatibility shim and API bootstrap; route/residue scan confirms Python watchdog runtime and Flask handlers removed; docs updated from Python runtime to Go runtime source map; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live /health; public Go watchdog auth-gate smoke; direct Python watchdog route-removal smoke; git diff --check. |
| Job-scheduler manager runtime cleanup | Go unit tests for scheduler manager cadence helpers, route reconciliation helpers, and existing work-item payload parity; Python syntax compile for retained scheduler queue/site-worker/healthcheck modules and scheduler queue tests after manager-test removal; route/residue scan confirms runtime entrypoint no longer imports Python manager and manager.py is deleted; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; sudo bash Engine.sh --network-mode public|local --service job-scheduler rebuild prod after job-scheduler image hash inputs were corrected; live health and job-scheduler container smoke; site-worker stale-image reconcile smoke; public scheduled-job auth-gate smoke; git diff --check. |
| Agent maintenance work-item execution cleanup | Go unit tests for manager-to-site-worker host-service call and Agent maintenance error mapping; Python queue focused test confirms site-worker claims exclude Go-owned agent_maintenance_run; Python syntax compile for retained scheduler queue/site-worker and tests; residue scan confirms Python site-worker Agent maintenance executor symbols removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live health and job-scheduler smoke; public Agent maintenance auth-gate smoke; git diff --check. |
| Scheduled workflow work-item execution cleanup | Go unit test confirms manager dispatches scheduled_workflow_run through the internal workflow-start bridge; Python queue focused test confirms site-worker claims exclude Go-owned scheduled workflow work items; Python syntax compile for retained scheduler queue/site-worker and tests; residue scan confirms Python site-worker workflow executor symbols removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live health and job-scheduler smoke; public scheduled-job auth-gate smoke; git diff --check. |
| Scheduled script/Ansible work-item execution cleanup | Go focused tests for scheduler execution helpers plus full go test ./...; Python queue focused test confirms site-worker claims only onboarding work and leaves Go-owned scheduled_run, scheduled_workflow_run, and agent_maintenance_run queued; Python syntax compile for retained scheduler queue/site-worker and tests; residue scan confirms Python site-worker scheduled-run executor symbols removed; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live health and job-scheduler smoke; public scheduled-job auth-gate smoke; git diff --check. |
| Onboarding work-item execution retirement | Go focused tests for onboarding scope parsing; focused retained Python pytest covers scheduler queue/site-worker socket/Ansible host-service runtime through repo test venv; Python syntax compile for retained site-worker and queue tests; residue scan confirms Python site-worker onboarding executor symbols removed; go test ./...; build-api-backend.sh; sudo bash Engine.sh --network-mode public|local deploy prod; live health and job-scheduler smoke; public scheduled-job auth-gate smoke; git diff --check. |
| Tracker cleanup | Run git diff --check after doc edit. |
Next Work¶
Done. No remaining api-backend rewrite work.
Detailed Codex Breakdown
Source map¶
- Go gateway and native handlers:
Data/Engine/Containers/api-backend/cmd/api-backend/. - Retained Python worker/helper modules:
Data/Engine/Containers/api-backend/data/services/. - Public Go listener:
127.0.0.1:5000. - Compatibility backend listener: removed.
- Route ownership rule: Go registrations in
main.go; unregistered API paths return Go 404.
Done rule¶
Porting a handler to Go is not enough for Done. Done requires:
- Public route registered in Go.
- Behavior parity verified by focused test or live smoke.
- No required Python compatibility route, challenge/session manager, broadcast path, or api-backend-owned runtime remains.
- Duplicate Python route deleted or intentionally retained with reason.
git diff --checkpasses.
Removal guidance¶
Python process supervision and compatibility proxy are removed. Retained Python worker/helper runtimes are not api-backend fallback paths.