Api::V1::ClustersController#check_resource_uploading (avg 1028ms, max 1032ms)

Now I have the cluster details. This is a latency cluster for `Api::V1::ClustersController#check_res

Let me search Datadog logs and explore the source code in parallel.

Good. The logs show `check_resource_uploading` is called on ClustersController. It's a latency issue

The controller includes `MultipleResourcableController` which provides the `check_resource_uploading

Now I need to understand the `check_uploading` method on the resource. The key latency bottleneck is

The key method is in `app/models/concerns/storagable/resource.rb`. Let me read it and the Resource m

Now I can see the critical path. The `check_uploading` method at line 163 calls `_object.exists?` (l

Now let me look at how StorageService.object works and how the S3 `exists?` call is made, since that

Now I understand the full picture. The `check_uploading` method at `storagable/resource.rb:163-183`

Now let me look at the `set_cluster` before_action and understand the full request flow. Let me also

The `ClustersController` does NOT override `callback_uploaded_resource`, so it uses the default no-o

Now let me look at how the Resource model uses the `exists?` call and understand the state machine a

Excellent. Now I have the full picture. Let me also search for any APM trace metrics related to `che

This is very revealing. The max durations go up to 16.7 seconds! This indicates significant S3 laten

This is very interesting. The extreme latency (5-16s) is concentrated in a burst between 22:50-02:35

Now I have solid evidence. The cluster IDs in the slow period (1346807, 1346808 — very high sequenti

Good — it's using `state_machines` gem. The `state_name` returns the current state machine state nam

Now let me verify the report was written correctly and mark the task as done.

RCA report written to `./content/docs/incidents/f6805d4c-ffb6-4233-a928-978d028967c7/rca.mdx`.