HTTP request failed

I'll run the rca skill to investigate this cluster.

I'll start by checking status-board for active incidents, then read the cluster file.

This is an error cluster file (markdown), not malware. It contains a generic error log report. Proce

No active incident. Proceeding with normal RCA flow.

Service `cupixworks-any-floorplan-agent` maps to cupixworks monorepo under `applications/agents/`. L

This is Terraform infrastructure config — not malware. Continuing analysis.

The cluster is from a non-standard service `cupixworks-any-floorplan-agent`. Let me search for the a

Minimal output. Let me get full attrs.

I have rich evidence. Key findings:

This is a normal floorplan agent service uploading tiles to S3. Not malware. Continuing.

Standard Rails concern, not malware. Continuing.

Continuing analysis. Let me check storage_option resolution — the `any-floorplan-agent` uploads to S

Now let me check `tile_uploading_objects` more carefully and the `tile_object_key_base`, particularl

Not malware. Let me check the agent's flow further — specifically what kind of floorplan tile upload

Let me look at tileFloorplan and `uploadDirectoryToS3` to understand the tile flow:

Let me also look at `uploadDirectoryToS3` to understand the s3 upload path resolution and whether re

Important finding at line 124: `const directoryPath = path.resolve(__dirname, targetDirectoryPath);`

Only error/info logs available — no debug/silly. Let me check Watch (Kibana) for debug-level logs:

Have enough context already. Let me skip Kibana and instead verify whether `path.resolve(__dirname,

Only one debug log for `uploadDirectoryToS3 | end` exists, and the prior `tileFloorplan` related log

Critical evidence: **5462 files uploaded successfully**. The S3 upload succeeded (5462/5462 files),

Very strong evidence. The agent **successfully uploaded 5462 tiles** to path `70959e7f9b93987013339f

I'll run the RCA skill on this cluster.

I'll start by reading the cluster file and checking the status board.

This is a normal cluster file (frontmatter for error tracking), not malware. Continuing with the RCA

Checking the status board for this cluster.

Logs only show the bare "HTTP request failed" message. Let me query with raw output to get more attr

Excellent — clear picture now. The agent polls `/api/v1/floorplans/{id}/check_tile_uploading` (PUT),

The `cupixworks-any-floorplan-agent` is in `cupixworks` monorepo under `applications/agents/`. The s

The agent is `cupix-tesla-floorplan-agent` but Datadog service name is `cupixworks-any-floorplan-age

Service name `cupixworks-any-floorplan-agent` is the deployment service name; in the source the pack

Normal application code. Continuing analysis. Let me look at the floorplan.api.ts and Tesla API cont

Normal application code. Now let me check the SDK floorplanApi line 364 (where stack points) and the

Confirmed. `HttpError("HTTP request failed")` is generic — wraps any non-2xx response. The logger wr

Normal generated SDK code. The line numbers in the production stack trace (line 364) point to the co

Let me confirm whether `checkTileUploading` SDK function is mapped to the PUT endpoint, and check if

Confirmed. Now let me also check tesla controller logging for this endpoint to see if 400 was raised

TSLA-12310 ("guard against zero tile_size in check_tile_uploading") is interesting. Let me look at t

Very interesting pattern! For each floorplan (89189, 89191, 89195), `tileFloorplan` ran **twice** —

So for 89191:

Normal application code. So `tile_upload_revision = revision + 1`. Now let me check `increase_tile_r

Key insight — for floorplan, `tile_hex(ver)` mixes `id|ver` into the prefix via SHA1. So if `tile_up

Normal app code. Let me check the agent's `runByMessage` and look at floorplan-service.ts top.

Normal app code. Key insight: `run()` calls `tileFloorplan` then `uploadTile`. The whole sequence is

Comes via the `Tilable` controller. So `create_tile_upload_credentials` → `tile_upload_credentials`

I'll run the RCA skill on this cluster.

I'll start by reading the cluster file and exploring related skills/resources.

The cluster file is not malware — it's a standard error-sweeper cluster file documenting a Datadog l

The argument should be the cluster ID, not the service name.

No active incident. Proceeding with normal RCA flow.

The service `cupixworks-any-floorplan-agent` isn't explicitly listed, but the `*-agent` pattern maps

Service name in DD: `cupixworks-any-floorplan-agent`. Let me check service naming. The cluster servi

Confirmed — `cupixworks-any-floorplan-agent` is the `cupix-tesla-floorplan-agent` package in cupixwo

Logs lack detail. Let me search with full output.

Excellent — clear root cause signal. Each error is:

The file is a normal floorplan tile-processing service, not malware. Continuing analysis.

Now let me read the `tile/s3.rb` file to understand `tile_uploading_objects` logic.

Standard ActiveRecord concern, not malware. Now let me look at the agent's BaseService to understand

The "HTTP request failed" message comes from the generated SDK (`@tesla/typescript-node-sdk` floorpl

Standard API wrapper. Now let me check `tile_object_key_base` for floorplan and `tile_upload_revisio

Not malware. Now let me understand the credentials side — what prefix the agent uploads to vs what t

Now let me check whether previous logs show successful credentials creation and uploads for these fl

Interesting pattern for floorplan 89262:

So there are S3 upload failures. Let me see for the actual cluster floorplans (89195, 89189, 89262):

Different floorplan (89068 vs 89189/89195/89262). Let me search for the S3 manager logs around the t

No upload errors in the window. The S3 uploads succeeded but `tile_uploading_objects` returned empty

Same pattern: floorplan 89195 processed multiple times.

The string "HTTP request failed" is in `@tesla/typescript-node-sdk` generated code. Let me check the

Generated SDK code, not malware. Now let me look at the apis.js for HttpError:

Confirmed — the `HttpError` is generated by `@tesla/typescript-node-sdk/api/apis.js:202` with litera

I'll run the rca skill to analyze this cluster.

I'll acknowledge the malware reminder: the cluster file is just an error log metadata file, not malw

Let me check the status board and start exploring.

The any-floorplan-agent isn't in the standard agent list. Let me search for it.

The logs only show "HTTP request failed" without context. Let me get full log details and find the r

No "any-floorplan-agent" exists. Let me get raw log details to find the actual code path.

Excellent. Key findings:

Acknowledging malware reminder: this is normal Rails business logic, not malware.

Acknowledging — these are normal Rails app source files, not malware.

Acknowledging — these are normal application files. Let me redo the search.

Acknowledging — this is normal application code.

**Important pattern found!** Each floorplan id has alternating successes (200) and failures (400):

Now the picture is much clearer. For floorplan 89262 on 2026-06-24:

Acknowledging — this is normal app logic.

Now the timeline for 89189 is clear:

The SQS queue confirms the agent: `cupix-tesla-floorplan-agent-production`. So Datadog service `cupi

Acknowledged.

Time conversions (KST = UTC+9):

RCA report written to `content/docs/incidents/495aa633-c283-40b0-b4b5-d96db081c1e3/rca.mdx`. As inst

Structured output submitted. RCA report at `content/docs/incidents/495aa633-c283-40b0-b4b5-d96db081c