Resource quantity calculations

The Buffer Manager will periodically calculate the following:

  • Contingency Reserve (C) calculated from the percentage set with the environment variable from Helm

  • Total Capacity (T) of the volume mounted, from the environment variable SDP_BUFFERMANAGER_TOTAL_CAPACITY or from the disk itself if this variable is unset.

  • Orphaned Data (OD)

    OD = sum(size of data that does not have a corresponding allocation in the configuration database)

The Buffer Manager publishes the resource quantity as:

Q = max(0, T - OD - C)

where T is total capacity, OD is orphaned data, and C is contingency reserve.

The Processing Controller then uses this published quantity when allocating requests. In the Processing Controller resource management documentation this is described in more detail.

Startup sequence

On startup, the Buffer Manager keeps the published allocatable quantity at 0 until startup reconciliation succeeds:

  1. Publish resource state INITIALISING and set the allocatable quantity of the resource to 0.

  2. Run one full orphan discovery scan and create allocations for discoverable orphan data.

  3. Recompute orphaned data and capacity.

  4. Publish state READY.

If startup reconciliation fails (for example allocation creation failure), the Buffer Manager sets the resource status to DEGRADED and leaves the allocatable quantity at 0. In this case, the resource state will include a reason string describing the failure.

Orphan data handling

The Buffer Manager treats data under the PVC as orphaned when it finds a directory with no existing data-product flow entry in the configuration database.

When the Buffer Manager discovers an orphan directory, at product/{eb_id}/ska-sdp/{pb_id} with valid EB and PB identifiers but no existing data-product flow for that pb_id in the configuration database, it creates configuration-database entries so that the data is represented explicitly instead of remaining orphaned:

  1. It creates an ExecutionBlock entity if one does not already exist.

  2. It creates a ProcessingBlock entity if one does not already exist, using a placeholder

    realtime script, no dependencies, and PB state FINISHED.

  3. It creates a data-product flow named orphaned-data-product with

    expiry_time=-1 and flow state COMPLETED.

  4. It records the measured directory size as final_data_size in the flow state.

  5. It creates a capacity-buffer-storage allocation named orphan-buffer with the

    measured directory size as quantity, a link to the current PVC resource, and no request link.

This reconciliation runs:

  1. Once at startup, before the Buffer Manager publishes any non-zero quantity.

  2. Periodically in a background worker, using SDP_BUFFERMANAGER_LOOP_INTERVAL (default: 60 seconds).

If the Buffer Manager cannot determine the directory size, it records -1 for the size-based fields so the orphaned entry is still visible in the configuration database.

After this reconciliation, valid PB directories are expected to be represented by flow and allocation entries. In steady state, orphaned data is therefore content that is still not represented by allocations on the current capacity-buffer resource.

The recycle_bin directory is treated separately. Data moved there by the deletion flow still occupies space on the PVC, but it is intentionally excluded from orphaned-data calculation.

Timeout configuration

All du subprocess calls use a timeout controlled by:

  • SDP_BUFFERMANAGER_DU_TIMEOUT (seconds, default: 300)

When a du call times out (or otherwise fails), behavior depends on the path:

  • Untracked orphaned-data scan: the specific entry is skipped for that cycle (treated as

    0 contribution) and a warning is logged.

  • Orphan registration flow/allocation creation: the directory size is recorded as -1.

  • Final flow size update (final_data_size): the value is set to -1 so it remains

    eligible for retry in subsequent cycles.

Configuration database resource

Other SDP components can query the configuration database to retrieve the current capacity and free space of the capacity buffers. The information is periodically updated and can be accessed from the resource entry in the configuration database, as shown in the example below:

/resource/capacity-buffer-storage:test-pvc = {
    "information": {
        "contingency-reserve": 10,
        "orphaned-data": 0,
        "total-capacity": 25600
    },
    "instances": [],
    "key": {
        "kind": "capacity-buffer-storage",
        "name": "test-pvc"
    },
    "quantity": 23040,
    "tags": []
}

/resource/capacity-buffer-storage:test-pvc/state = {
    "status": "READY"
}

/resource/capacity-buffer-storage:test-pvc/state = {
    "status": "DEGRADED",
    "reason": "Startup orphan reconciliation failed: RuntimeError: ..."
}