The Database Can Roll Back, S3 Cannot: Cleaning Up Files After Failure

After direct upload to S3 and staging before activation, the next important question appears: what happens when a storage operation succeeds but the following database operation fails?

With SQL we are used to DB::transaction(): if something fails inside the transaction, the database changes can roll back.

S3 or MinIO is not a participant in that transaction. If an object has already been copied, created, or deleted, a database rollback does not undo it.

I therefore do not model database work and object storage as one atomic operation. I design the order of steps and compensation for every failure point instead.

For the user this may be one action: "replace the video."

For the system it is at least two separate infrastructure operations: changing database state and changing object storage state.

Those systems can succeed or fail independently. The design has to account for that explicitly.

After the exception, the File record may disappear because of the rollback.

The object at $permanentPath remains in S3 if the copy already succeeded.

This is a classic orphan object: the bytes exist, but application state no longer references them.

Once the stages are explicit, it becomes easier to decide which inconsistent state is more dangerous.

For me, a database row pointing to a missing file is worse than an unused object left in storage.

I therefore order the steps so that partial failure tends to produce a removable orphan object rather than a broken database reference that affects users.

This order does not make the operation atomic.

It does preserve an important invariant: before database state considers the file active, the permanent object already exists.

If the database transaction fails afterward, an extra storage object remains. That requires cleanup, but application state does not point at a missing file.

The cleanup here is not an S3 rollback. It is a separate compensating action.

That distinction matters because compensation can fail too.

A cleanup failure should not disappear silently. It should at least be reported, and preferably retried or handled by a reconciliation process.

If copying the new file or updating the database fails after the first line, the old working resource has already been lost.

This is cleanup happening too early.

I keep the old object available until the new object is prepared and the database commit has confirmed the new active path.

Now database state switches to the new path first and commits successfully.

Only after the commit does deletion of the old object begin.

If deleting the old file fails, the user flow still works because the database already points to the new existing object.

DB::afterCommit() solves ordering, but it does not make an external request infallible.

After commit, the process can terminate, the network can fail, or S3 can temporarily reject the request.

Important cleanup therefore benefits from a retryable job or reconciliation mechanism that can finish the work later.

If cleanup runs again, the desired outcome should remain the same: the obsolete object is gone.

This matters especially for queued jobs where retries are a normal part of execution.

I do not want a repeated deletion attempt to become a new failure condition when the previous attempt already removed the object.

If the temporary object is deleted immediately after the copy and the database commit then fails, a retry may no longer have a source object.

I therefore remove tmp/ after successful activation or leave it under the normal cleanup policy until the staging window expires.

If the business operation failed because of a database exception, that should remain the primary error returned to the caller.

A compensation failure matters too, but it is a second failure that should be recorded separately.

Otherwise diagnostics may show only "delete failed" and hide the reason compensation started in the first place.

There is no guarantee that the process will always survive long enough to reach compensation code.

It can terminate between the storage request and the next line of PHP.

Periodic reconciliation is therefore the final safety layer: it finds old temporary or candidate objects that no longer have a valid application reference.

I want one clear answer to the question "which file is active right now?"

That answer comes from application state in the database.

S3 may contain temporary, candidate, old, and active objects at the same time. Mere object existence does not make an object active.

That is why orphan cleanup should reconcile storage against authoritative application state, not the other way around.

If cleanup is treated as "just the last line," its failures are easy to miss.

It is useful to observe failed compensation, failed old-file deletion, stale temporary objects, and reconciliation deletions separately.

This prevents object storage from accumulating garbage silently for months.

I verify:

• Does the code avoid assuming that DB::transaction() can roll back S3?
• Does the permanent object exist before the database considers it active?
• Does a new object have a compensation path if the database commit fails?
• Is the old active file kept until after commit?
• Can post-commit cleanup be retried?
• Does the temporary object survive long enough for safe retry?
• Does a cleanup error preserve the primary failure?
• Can orphan objects be found later through reconciliation?
• Is database state the source of truth for the active file?
• Do failure-path tests cover the boundary between storage and database operations?

MySQL and S3 can participate in one business operation without becoming one technical transaction.

Reliability therefore comes not from one DB::transaction(), but from ordering, explicit compensation, after-commit cleanup, retries, and reconciliation.

I prepare the new object first, commit authoritative database state second, and only then remove old or temporary objects.

Database rollback protects SQL. External side effects need deliberately designed cleanup.