Moving data to recycle bin

On the same frequency as the storage usage checks, the Buffer Manager will also look for any Flow entries that match the following criteria and mark them for “DELETING”.

• All the dependencies on a flow are in FINISHED state.

• The state is in COMPLETED, INCOMPLETE, or FAILED.

• The Flow’s expiry_time entry is set (and is positive).

• And that the last_updated + expiry_time is in the past.

In addition, it also periodically checks data-product flows and moves data associated with the ones in DELETING state to a “recycle bin”, which is a reserved area in the data PVC. Once a data product is in the “recycle bin”, the state of its associated flow is set to DELETED.

Users can then go to the recycle_bin directory and delete the data products manually if they are no longer needed.