ES /docs

Api::V1::PanosController#index (avg 1364ms, max 2937ms)

Service

cupixworks-api

Occurrences

2,560

Status

Reported

Confidence

85%

Actionability

44%

Cluster ID

7a428cc7-40a5-4183-9d7c-40fb35e9e66b

First Seen

2026-05-26 03:41 UTC

Last Seen

2026-06-24 00:05 UTC

Datadog↗

Slack↗

Actions

Api::V1::PanosController#index (avg 1364ms, max 2937ms)

What Happened#

2026-05-26 03:41~06:12 UTC 동안 cupixworks-api의 PanosController#index 엔드포인트에서 평균 1364ms, 최대 2937ms의 응답 지연이 4개 리전(ap-southeast-2, us-west-2, ap-southeast-1, eu-central-1)에서 39건 발생했다. 주요 원인은 serializer에서 resources와 masks 연관 객체를 preload 없이 접근하는 N+1 쿼리 패턴과, 클라이언트의 병렬 pagination 요청에 의한 thundering herd 현상이다.

Quick Facts#

Field	Value
resource_name	`Api::V1::PanosController#index`
top_frame	`app/serializers/pano_serializer.rb:101-102`
runtime	Ruby on Rails (cupixworks-api)
env	production (ap-southeast-2, us-west-2, ap-southeast-1, eu-central-1)

Affected Teams#

Team / Domain	Error Count	Impact
walmart (us-west-2)	7	대량 pano 조회 시 페이지당 922-1186ms 지연, 사용자 체감 느린 로딩
cupix (ap-southeast-2)	17	병렬 pagination으로 첫 페이지 826-911ms 지연

Timeline#

2026-05-26T03:41:53Z — 최초 slow trace 감지 (ap-southeast-2)
2026-05-26T04:59:44Z — us-west-2에서 최대 지연 1186ms 기록 (capture_id=702401, 634 panos)
2026-05-26T06:12:40Z — ap-southeast-2에서 thundering herd 발생 (17 페이지 동시 요청)
2026-05-26T06:12:56Z — 마지막 slow trace 기록

Scoring#

Confidence 85% · Actionability 44%

Confidence#

Dimension	Score	Detail
Evidence completeness	5/5	12 code refs, 5 log queries
Code path explicitness	3/3	12 file:line refs
Log evidence	3/3	section, query, pattern
RCA structure	3/3	3/3 sections
Revision history	0/3	0 revision(s)
Agent execution	3/3	success

Actionability#

Dimension	Score	Detail
Fix complexity	5/5	trivial
Affected files	1/5	23 file(s)
Single repo	1/3	3 service(s)
Immediate actions	0/3	absent
Testability	0/3	no test info
External dependency	1/3	external deps found
Data migration	3/3	no migration