failed to get captured area - error: 503 Service Unavailable
Service
cupixvista-api-migration-worker
Occurrences
164
Status
Reported
Confidence
85%
Actionability
32%
Cluster ID
3a8d30cc-330a-4d70-8ac8-5ce4883bc982
First Seen
2026-07-15 12:33 UTC
Last Seen
2026-07-30 08:46 UTC
Datadog
SlackActions
failed to get captured area - error: 503 Service Unavailable
What Happened#
2026-07-15 21:33~21:44 KST 사이 CUPIXVISTA 마이그레이션 워커에서 facility 8건에 대한 calculate_captured_size! 처리가 실패했다. 실패 원인은 downstream voxel service Lambda(captured_area)가 Athena GetQueryExecution API 호출에서 ThrottlingException을 받아 500을 반환했고, 일부는 API Gateway 단에서 503으로 반환된 데 있다. 마이그레이션 성격상 다수 facility를 병렬 처리하면서 Athena polling loop가 rate limit을 초과했다.
Quick Facts#
| Field | Value |
|---|---|
| exception.class | Cupix::Errors::System (code SYS20000) |
| exception.message | failed to get captured area - error: 503 Service Unavailable |
| top_frame | app/services/cupix/voxel_service.rb:97 |
| downstream | AWS Lambda voxel/captured_area (data-pipeline-functions) |
| downstream_error | ThrottlingException on GetQueryExecution (max retries: 3) |
| env | production, us-west-2, tenant cupix (CUPIXVISTA) |
Affected Teams#
| Team / Domain | Error Count | Impact |
|---|---|---|
| cupixvista-api-migration-worker | 3 (본 cluster) | Facility 594, 597, 599, 604, 607, 609, 610, 612 등 8건의 captured_size migration 실패, 재시도 대상 |
같은 시간대에 svc-scope incident로 묶인 관련 cluster: a558bd06-2374-433b-968a-5e17dfd21a2f, fe12d217-509c-4320-b68e-d1ec5c4460b2.
Timeline#
- 2026-07-15 21:33 KST — Facility 594 처리 중 첫 503 발생 (
captured_area!) - 2026-07-15 21:36~21:42 KST — Facility 597, 599, 604, 607, 609, 610 처리 중 500 (
ThrottlingException) 및 503 반복 - 2026-07-15 21:44 KST — Facility 612 처리 중 마지막 503 발생 (cluster last_seen)
- 2026-07-15 21:49 KST — voxel Lambda가 다시 정상 응답 시작 (Athena throttle 해소)
- 2026-07-15 21:58 KST — 별도의 재시도 파형에서 다시
ThrottlingException발생 (동일 원인 재현)
Scoring#
Confidence 85% · Actionability 32%
Confidence#
| Dimension | Score | Detail |
|---|---|---|
| Evidence completeness | 5/5 | 13 code refs, 3 log queries |
| Code path explicitness | 3/3 | 13 file:line refs |
| Log evidence | 3/3 | section, query, pattern |
| RCA structure | 3/3 | 3/3 sections |
| Revision history | 0/3 | 0 revision(s) |
| Agent execution | 3/3 | success |
Actionability#
| Dimension | Score | Detail |
|---|---|---|
| Fix complexity | 5/5 | trivial |
| Affected files | 1/5 | 15 file(s) |
| Single repo | 1/3 | 2 service(s) |
| Immediate actions | 0/3 | absent |
| Testability | 0/3 | no test info |
| External dependency | 1/3 | external deps found |
| Data migration | 0/3 | migration needed |