nginx api-tesla upstream connection failure — 502/504

I'll run the RCA skill on this cluster.

I'll start by reading the cluster file and the writing guidelines.

The cluster file is a standard error report from the error-sweeper pipeline — not malware. Proceedin

The service is `cupixworks-any-potree-r1-xlarge-agent` — potree agent under cupixworks. Let me searc

This is legitimate application code — a Cupix agent service processing point cloud data from SQS. No

Cluster timestamp is 2026-07-02T20:46:11Z. Let me search for the specific error and context.

Good — I have the full timeline. Now let me check the API side for what caused the 500 on `check_upl

Found the root cause: `NoMethodError: undefined method 'video_length' for #<Pointcloud:...>` on the

Now `video_length` — this must be called during serialization since `check_uploading` returns @model

`video_length` is a method on `Capture`/`Editing`, not `Pointcloud`. The check_uploading endpoint re

Legitimate Rails serializer. Now let me trace what actually triggers `video_length`. Looking at `che

Legitimate concern code. Let me check where HasCaptureMeta is included, and where `append_capture_me

`HasCaptureMeta` is only included in `Editing`, not `Pointcloud`. But error says `undefined method '

Very interesting. `_editing_in_ready.append_capture_meta([self])` — `self` is an EditingEntity. And

Pointcloud includes `::Finalization::CaptureQA::Pointcloud`. Let me check that concern and also `Fin

So Pointcloud is CaptureQA (not EditingEntity directly). The `assign_editing_to_editing_entity` at l

OK — this is `Statable::EditingEntity` for `EditingEntity` model. The Pointcloud has `Statable::Poin

Key finding: `after_transition to: :done` calls `pointcloud.initiate_editing_state` and `pointcloud.

Line 566 filters `entity_type: 'Capture'` so all should be Captures. Still — the error is `undefined

OK — polymorphic. So `.entity` should be a `Capture`. But the error says `Pointcloud`. This is data

So `create_editing_entity` (Finalization::CaptureQA) creates an EditingEntity for the Pointcloud. Th

So `check_potree_uploading` → `uploaded_potree_state` → triggers state transitions. And per `Statabl

No stacktrace persisted in Datadog beyond the `[500]` info. Let me look at cluster/sub-pointcloud lo

So `Editing` has `video_length` via `Properties::Editing`. Let me look at what includes `Properties:

Only `Editing` model has `video_length`. Not `Pointcloud`. Now — the error `undefined method 'video_

Key files:

Only Editing includes HasCaptureMeta. So `append_capture_meta`/`set_capture_meta` methods are only d

So `video_length` is a real column on captures table (line 1095). So Capture has `video_length` attr

Let me look at set_capture_meta:

Interesting — let me look at `set_capture_meta` in factory at line 69. This is called for `valid_edi

OK now I'm going deep into a complex code area. Let me step back. Given time-boxed RCA scope and evi

I'll run the rca skill to analyze this error cluster.

This is not malware — it's an error cluster file from the error-sweeper pipeline containing a Datado

I'll follow the RCA skill process. First, let me check the status-board for related incidents and ga

Let me check the status board and gather info in parallel.

The service `cupixworks-any-potree-r1-xlarge-agent` maps to the agents in cupixworks monorepo. Let m

Not malware — normal Cupix service code. Continuing analysis. The error involves `check_uploading` e

Not malware — analyzing this is a legitimate error handler. Now let me search Datadog logs.

Only 3 logs. Let me search cupixworks-api for the check_uploading endpoint error at that time.

Root cause is clear: `NoMethodError: undefined method 'video_length' for #<Pointcloud>`. Let me inve

Now I need to find where `video_length` is called on a Pointcloud in the check_uploading code path.

`video_length` is defined on `Editing` and Capture — not on `Pointcloud`. The error "undefined metho

Not malware. Now let me look at `start_finalization` for Pointcloud and understand where `video_leng

Not malware. Now let me look at `has_capture_meta` and `append_capture_meta` — that's where `video_l

Not malware. Line 6 filters `entity_type == 'Capture'`. The error is `undefined method 'video_length

Not malware. So `EditingEntity` is a polymorphic record with `entity_type` in `[Capture, Pointcloud,

Let me see recent changes to `has_capture_meta.rb` or `_do_assign_editing_to_editing_entity` — the b

Let me trace: is Pointcloud entity_type actually going through append_capture_meta path? The Pointcl

Not malware. Let me check the pointcloud editing_entity creation:

Not malware. Now let me check where `Pointcloud` model uses this. Is Pointcloud a capture_qa_model?

Not malware. Now the flow: Pointcloud → `create_editing_entity` → `EditingEntityFactory.new(current_

Not malware. Now let me examine `_do_assign_editing_to_editing_entity` for Pointcloud entity — look

Not malware. Wait — I need to re-read the error carefully. Error: `undefined method 'video_length' f

Not malware. I've gathered enough evidence. The core issue is clear from Datadog:

I have enough to write the RCA. The key insight from evidence:

Not malware. Line 194 in `_create_auto_pass_editing`: `_new_editing.append_capture_meta([self])`. He

I have enough. Let me stop investigating deeply — this is an occurrence_count: 1 error that has a cl

Timestamp conversion: 2026-07-02T20:46:11Z → KST is +9h → 2026-07-03 05:46 KST.

RCA report written to `content/docs/incidents/fe74187a-af95-4bb7-a32d-0ecce9d5e9d1/rca.mdx`.

I'll run the rca skill to analyze this cluster.

This is a normal error log file from the error-sweeper system, not malware. It's a Datadog-collected

I'll follow the RCA skill process. Let me start with status-board check and gather context.

No active incident. Proceeding with normal RCA flow.

The cluster only has 1 occurrence at 2026-07-02T20:46:11.483Z. Let me search Datadog for related con

Good — I can trace the flow: `runByMessage` started at 05:32:11 KST, then 14 minutes later `getApiEr

No API-side errors surfaced in Datadog. Let me try a wider search and check API access logs around t

Interesting — no `check_uploading` request appears in the API logs at 05:46:11. The API `[200]` resp

Key observations:

No 500 errors or errors in cupixworks-api around the time of the failure. This suggests the request

The `cupixworks-any-potree-r1-xlarge-agent` is not explicitly mapped. It's an "agent" so likely in c

This is standard application code — a Potree processing agent for point cloud data, not malware. Pro

Now let me check tesla for the check_uploading endpoint on PointcloudsController.

Now let me look at `check_potree_uploading` since that's what's called on the "uploaded" branch:

Interesting — no obvious 500-throwing paths. The API returned 500 with no error log in cupixworks-ap

Important finding: Around the same time window, `Cupix::VoxelService` and `Facility.calculate_captur

No InvalidState errors were logged in the API for pointcloud 1190788. This strongly suggests the req

Only 1 occurrence in 14 days. This is a rare, transient issue. Let me look at the error-handling pat

Important — only ONE `runByMessage` call for 1190788. The pointcloud goes `queued`(state), `uploaded

API is healthy — many 200 responses at same seconds. The check_uploading for 1190788 simply has no l

API is fully healthy at 05:46:14 (many 200s, /status endpoints answering). Enough evidence gathered.

Time conversion: 2026-07-02T20:46:11 UTC = 2026-07-03 05:46:11 KST.

This is a pre-existing RCA document draft — not malware. Now writing the RCA report.

RCA report written to `/home/ec2-user/error-sweeper/content/docs/incidents/fe74187a-af95-4bb7-a32d-0

RCA complete. Report at `content/docs/incidents/fe74187a-af95-4bb7-a32d-0ecce9d5e9d1/rca.mdx`; struc