PanosController#create DB contention under concurrent bulk upload
Service
cupixworks-api
Occurrences
11,598
Status
Reported
Confidence
85%
Actionability
28%
Cluster ID
314c7ba2-9b90-4bee-bf14-41b1e2070690
First Seen
2026-05-26 03:50 UTC
Last Seen
2026-06-23 22:55 UTC
Datadog
SlackActions
PanosController#create DB contention under concurrent bulk upload
What Happened#
2026-05-26 03:50~06:16 UTC 사이에 ap-southeast-2 리전의 cupixworks-api 서비스에서 PanosController#create 엔드포인트가 평균 1321ms, 최대 1878ms의 응답 지연을 보였다. 총 12건의 slow request가 발생했으며, 모든 요청은 HTTP 200을 반환했지만 SLA 기준(500ms)을 초과했다.
Quick Facts#
| Field | Value |
|---|---|
| resource_name | Api::V1::PanosController#create |
| avg_duration_ms | 1321 |
| max_duration_ms | 1878 |
| env | production, ap-southeast-2 |
| user_agent | cupix-agent |
| hosts | ip-10-1-145-251, ip-10-1-145-106 |
Affected Teams#
| Team / Domain | Error Count | Impact |
|---|---|---|
| aialdp001 (team 134) | 5 | Lucy Walker - capture 73275 bulk upload 지연 |
| successful (team 149) | 2 | YongFeng Xian - capture 73124/73225 upload 지연 |
| built (team 16) | 3 | Jonah Nelson, Joseph Wassef - capture 73306/73247 지연 |
| endeavourgroup (team 180) | 1 | Mark Hickey - capture 73263 upload 지연 |
Timeline#
- 2026-05-26T03:50:58Z — 최초 slow request 감지 (capture 73247, team 16)
- 2026-05-26T03:51:37Z — 동시 다발 burst: team 180 + team 149 동시 upload
- 2026-05-26T05:19:00Z — 두 번째 burst: capture 73275 (Lucy Walker, 5건 집중)
- 2026-05-26T06:16:00Z — 세 번째 burst: capture 73306 (Jonah Nelson)
- 2026-05-26T06:43:00Z — 정상 응답 복구 (~215ms)
Scoring#
Confidence 85% · Actionability 28%
Confidence#
| Dimension | Score | Detail |
|---|---|---|
| Evidence completeness | 5/5 | 11 code refs, 3 log queries |
| Code path explicitness | 3/3 | 11 file:line refs |
| Log evidence | 3/3 | section, query, pattern |
| RCA structure | 3/3 | 3/3 sections |
| Revision history | 0/3 | 0 revision(s) |
| Agent execution | 3/3 | success |
Actionability#
| Dimension | Score | Detail |
|---|---|---|
| Fix complexity | 1/5 | critical |
| Affected files | 1/5 | 17 file(s) |
| Single repo | 1/3 | 3 service(s) |
| Immediate actions | 0/3 | absent |
| Testability | 0/3 | no test info |
| External dependency | 1/3 | external deps found |
| Data migration | 3/3 | no migration |