ES /docs

Cupix::Errors::System: Mysql2::Error::TimeoutError: Lock wait timeout exceeded; try restarting transaction

Service
cupixworks-api
Occurrences
40
Status
Dismissed
Confidence
85%
Actionability
36%
Cluster ID
f4349f55-c62e-496a-909d-e71f6c79e575
First Seen
2025-04-13 00:17 UTC
Last Seen
2026-08-05 18:28 UTC
Datadog↗ Slack↗
Actions
현재 상태 dismissed 에서는 실행할 액션이 없습니다.

Cupix::Errors::System: Mysql2::Error::TimeoutError: Lock wait timeout exceeded; try restarting transaction

What Happened#

cupixworks-api (tesla) 의 다수 write 엔드포인트(주로 PUT /api/v1/element_traces bulk)에서 MySQL InnoDB Lock wait timeout exceeded 가 발생한다. 요청은 약 50초(InnoDB innodb_lock_wait_timeout 기본값)동안 row lock 을 기다린 뒤 ActiveRecord::LockWaitTimeout 을 던지고 HTTP 502 (BG10000) 로 매핑된다. Sitetrack post-processing 이 같은 facility 의 ElementTrace row 를 동시에 대량 갱신하면서 자기 자신과 lock 경합을 일으키는 것이 주 원인이다. 16개월간 38건으로 저빈도이며 대부분 qa/regression 테스트 트래픽이다.

Quick Facts#

Field Value
exception.class ActiveRecord::LockWaitTimeout (representative 는 Cupix::Errors::System)
exception.message Mysql2::Error::TimeoutError: Lock wait timeout exceeded; try restarting transaction
top_frame app/controllers/concerns/server_error_controller.rb:18 (502 mapping)
runtime Rails / ActiveRecord 7.2.2, MySQL (InnoDB)
env qa (dominant), us-west-2

Affected Teams#

Team / Domain Error Count Impact
algtest (qa, Sitetrack regression) 대부분 element_traces bulk 갱신이 502 로 실패 → postprocessor agent 재시도
기타 (panos/jobs/captures update 등) 소수 산발적 write 요청 502

Timeline#

  1. 2025-04-13 09:17 KST — 최초 발생 (first_seen).
  2. 2026-07-30 ~ 08-04 — element_traces#bulk 502 burst 반복 (facility qinlxf, Sitetrack 처리 중).
  3. 2026-08-03 15:02 KST — 최근 발생 (last_seen, cluster 기준).
  4. 2026-08-04 13:40~13:46 KST — retention 창 내 최신 burst 관측 (요청당 50s db, 45건 연속).

Scoring#

Confidence 85% · Actionability 36%

Confidence#

Dimension Score Detail
Evidence completeness 5/5 21 code refs, 3 log queries
Code path explicitness 3/3 21 file:line refs
Log evidence 3/3 section, query, pattern
RCA structure 3/3 3/3 sections
Revision history 0/3 0 revision(s)
Agent execution 3/3 success

Actionability#

Dimension Score Detail
Fix complexity 1/5 critical
Affected files 1/5 18 file(s)
Single repo 1/3 4 service(s)
Immediate actions 0/3 absent
Testability 0/3 no test info
External dependency 3/3 no external deps
Data migration 3/3 no migration