batch_pull! error on batch arn:aws:ecs:us-west-2:002596530511:task/cupix-tesla-ece/f08cf04288c144f38
Service
cupixworks-migration-worker
Occurrences
83,343
Status
Reported
Confidence
85%
Actionability
16%
Cluster ID
cac55d3a-4a07-413a-ac78-9818fbaeb1e5
First Seen
2026-07-13 10:02 UTC
Last Seen
2026-07-30 09:40 UTC
Datadog
SlackActions
batch_pull! error on batch arn:aws:ecs:us-west-2:002596530511:task/cupix-tesla-ece/f08cf04288c144f38
What Happened#
2026-07-13 19:02 KST 부터 cupixworks-migration-worker (tesla EB migrationworker role) 의 AwsTask.batch_pull! cron job 이 ecs:DescribeTasks API 호출에서 AWS AccessDenied 를 반환하면서 74분간 988건의 에러를 발생시켰다. tesla-api-production-5nod IAM user 의 identity-based policy 가 새 ECS long ARN 포맷(task/<cluster>/<taskId>)의 task 리소스와 매칭되지 않아 ecs:DescribeTasks 권한이 거부되었다.
Quick Facts#
| Field | Value |
|---|---|
| exception.class | AwsTask (message-only, exception class not captured) |
| exception.message | User: arn:aws:iam::002596530511:user/system/tesla-api-production-5nod is not authorized to perform: ecs:DescribeTasks on resource: * because no identity-based policy allows the ecs:DescribeTasks action |
| top_frame | app/models/aws_task.rb:87 |
| env | production, region tag ap-southeast-1, ECS cluster in us-west-2 |
| iam_user | arn:aws:iam::002596530511:user/system/tesla-api-production-5nod |
| ecs_cluster | cupix-tesla-ece (account 002596530511, region us-west-2) |
Affected Teams#
| Team / Domain | Error Count | Impact |
|---|---|---|
| cupixworks-migration-worker (tesla EB migrationworker) | 988 | ECS task status polling 중단 — AwsTask 레코드의 last_status/desired_status 가 갱신되지 않아 downstream job callback 및 stopped 처리 지연 |
Timeline#
- 2026-07-13 19:02 KST —
AwsTask.batch_pull!첫AccessDenied발생 (first_seen) - 2026-07-13 19:02 ~ 20:16 KST — 988건 반복 (cron 이 배치별로 계속 재시도)
- 2026-07-13 20:16 KST — 클러스터 마지막 관측 (
last_seen)
Scoring#
Confidence 85% · Actionability 16%
Confidence#
| Dimension | Score | Detail |
|---|---|---|
| Evidence completeness | 5/5 | 13 code refs, 4 log queries |
| Code path explicitness | 3/3 | 13 file:line refs |
| Log evidence | 3/3 | section, query, pattern |
| RCA structure | 3/3 | 3/3 sections |
| Revision history | 0/3 | 0 revision(s) |
| Agent execution | 3/3 | success |
Actionability#
| Dimension | Score | Detail |
|---|---|---|
| Fix complexity | 1/5 | critical |
| Affected files | 1/5 | 11 file(s) |
| Single repo | 1/3 | 2 service(s) |
| Immediate actions | 0/3 | absent |
| Testability | 0/3 | no test info |
| External dependency | 1/3 | external deps found |
| Data migration | 0/3 | migration needed |