Stack: Control Plane for Multi-Agent AI Orchestration

system_name: DocumentIntelligence Pipeline, agent_roles: 3 extractor agents (PDF/HTML/image), 2 classifier agents, 1 summarizer agent, 1 quality-check agent, primary_workload: Process 50,000 documents/day with structured data extraction and summarization, scale_requirements: P99 latency < 30s per document, burst to 500 concurrent tasks, infrastructure_provider: AWS (ECS Fargate + SQS + DynamoDB)

## 1. ARCHITECTURE OVERVIEW Recommended topology: **Hierarchical with a stateless control plane tier**. A central scheduler service (2 replicas, active-active behind ALB) coordinates all agents via SQS. Agents are stateless ECS tasks — the control plane holds all state in DynamoDB. Rationale: At 50k docs/day (~0.58 docs/sec average, burst to 500 concurrent), a centralized scheduler is sufficient and dramatically simplifies debugging vs. peer-to-peer. Stateless agents enable zero-downtime scaling. SPOF mitigations: Scheduler runs multi-AZ; DynamoDB is inherently HA; SQS message visibility timeout prevents task loss on scheduler restart. ## 2. AGENT LIFECYCLE STATE MACHINE ``` PROVISIONING → REGISTERING → IDLE → ASSIGNED → EXECUTING → REPORTING → IDLE ↘ FAILED → DRAINING → TERMINATED ``` **Health checks:** HTTP `/health` every 10s; 3 consecutive failures → DRAINING state; ECS replaces task automatically. **Retry policy:** Exponential backoff: 2s, 4s, 8s, max 3 retries → DLQ with SNS alert. ## 3. TASK ROUTING Capability registry stored in DynamoDB `agents` table: ```json { "agent_id": "extractor-pdf-001", "capabilities": ["extract", "pdf"], "current_load": 3, "max_load": 10, "status": "IDLE" } ``` Dispatch algorithm: Capability-filtered, then least-loaded agent wins. SQS message attributes carry `required_capability` for client-side filtering. ## 4. OBSERVABILITY Key metrics pushed to CloudWatch every 30s: - `agent.task.duration_ms` (p50/p95/p99 per agent class) - `queue.depth` per SQS queue - `agent.error_rate` (5-min rolling window) Alert: queue depth > 1000 for > 5 min → scale-out trigger. Error rate > 5% on any agent class → PagerDuty. ## 5. GOVERNANCE Cost cap: Each document task tagged with `budget_usd: 0.05`. Summarizer agent checks running cost via token counter middleware; exceeds budget → truncate input and flag in output metadata. Audit log entry example: ```json { "ts": "2026-04-16T10:23:01Z", "decision": "DISPATCH", "task_id": "t-8821", "agent_id": "extractor-pdf-003", "reason": "least_loaded_capable_agent" } ```