S3 list_objects_v2 pagination delay — large object listing
Service
cupixworks-api
Occurrences
318
Status
Dismissed
Confidence
85%
Actionability
28%
Cluster ID
0510c02c-5976-4e37-811e-0b9e6768e4f6
First Seen
2026-05-26 04:30 UTC
Last Seen
2026-06-24 03:15 UTC
Datadog
SlackActions
현재 상태 dismissed 에서는 실행할 액션이 없습니다.
S3 list_objects_v2 pagination delay — large object listing
What Happened#
2026-05-26 04:30~06:08 UTC 사이에 cupixworks-api 서비스의 Api::V1::PanosController#check_tile_uploading 엔드포인트에서 평균 8.9초, 최대 32.4초의 응답 지연이 발생했다. us-west-2 리전에서 team "gad" (id: 590)의 대량 pano 업로드 처리 중 S3 list_objects_v2 API 호출이 병목이 되었다.
Quick Facts#
| Field | Value |
|---|---|
| resource_name | Api::V1::PanosController#check_tile_uploading |
| top_frame | app/models/concerns/tile/s3.rb:10 |
| env | production, us-west-2 / ap-southeast-1 |
| deploy | production-us-west-2-20260526t0452z0-dd7bd097-cupixworks |
Affected Teams#
| Team / Domain | Error Count | Impact |
|---|---|---|
| gad (team_id: 590) | 50+ slow requests | Pano tile upload 확인 요청이 16~32초 소요, 클라이언트 타임아웃 가능성 |
Timeline#
- 2026-05-26T04:30:46Z — 최초 고지연 요청 감지 (APM threshold 초과)
- 2026-05-26T04:51:24Z — team gad의 대량 pano 처리 시작 (pano IDs 85878xxx 범위)
- 2026-05-26T06:08:23Z — 마지막 고지연 요청 기록
- 2026-05-26T06:40:37Z — 동일 엔드포인트 정상 응답 확인 (249ms, 작은 pano)
Scoring#
Confidence 85% · Actionability 28%
Confidence#
| Dimension | Score | Detail |
|---|---|---|
| Evidence completeness | 5/5 | 13 code refs, 2 log queries |
| Code path explicitness | 3/3 | 13 file:line refs |
| Log evidence | 3/3 | section, query, pattern |
| RCA structure | 3/3 | 3/3 sections |
| Revision history | 0/3 | 0 revision(s) |
| Agent execution | 3/3 | success |
Actionability#
| Dimension | Score | Detail |
|---|---|---|
| Fix complexity | 1/5 | critical |
| Affected files | 1/5 | 19 file(s) |
| Single repo | 1/3 | 2 service(s) |
| Immediate actions | 0/3 | absent |
| Testability | 0/3 | no test info |
| External dependency | 1/3 | external deps found |
| Data migration | 3/3 | no migration |