Infrastructures that forget: from compliance to design

Saving is easy. Erasing is hard. For decades, data systems were built on that silent asymmetry: keep by default, delete by exception. Today, with personal data piling up in every layer of the stack and European regulation tightening the pace, that logic has flipped — or should have. Designing infrastructures that forget by default, and not as a late add-on, is one of the most important architectural shifts data engineering is learning to take on. This piece focuses on the classic layer: databases, analytical lakes, backups. The parallel problem of trained models —where erasing the data is not enough— is a different article.

The starting point: the asymmetry between keeping and deleting

A traditional system accumulates data by sedimentation. Every table, every log, every backup, every index, every cache behaves like a geological layer that settles without needing any active decision to persist. Removing them, on the other hand, requires locating each copy and each derived trace — a job that, in practice, almost never gets fully completed.

The data from the Coordinated Enforcement Action of February 2026, published by the European Data Protection Board, describes it with clinical precision. The review of the implementation of the right to erasure found incomplete responses that confirmed receipt of the request but not that the erasure had been executed, reliance on exceptions without identifying which one applied, and systematic delays beyond the legal deadline. It’s not bad faith. It’s architecture: systems were not designed to forget.

And here is where the shift that names this article appears. The question is no longer “how do we comply with requests when they arrive?” — reactive, late, expensive. The question is: what if forgetting were the default state, and persistence the exception that has to be justified?

From legal principle to design principle

GDPR incorporates, in Article 25, data protection by design and by default: systems must implement technical measures from the design phase, not as a subsequent add-on. Article 5 introduces the storage limitation principle: personal data is retained only as long as necessary.

On paper, this has been written since 2016. In practice, most architectures still operate under the previous logic. The distance between principle and execution is exactly where the problem lives — and where three concrete shifts open up.

TTL as a first-class citizen of the schema

In a traditional system, retention is handled by external processes: cron jobs that delete, scripts scheduled to run at night. It is fragile by nature: the process fails, gets forgotten, becomes obsolete when the schema changes.

The alternative that is consolidating in modern databases is to declare retention inside the schema itself. ClickHouse allows this with the TTL clause at the table or column level, evaluated during merge operations. Google Cloud’s Firestore and Datastore implement it with TTL policies bound to a timestamp field. Delta Lake incorporates the VACUUM operation with a default retention of 30 days —explicitly aligned with the undue delay of the GDPR—.

The difference is not just technical. It is existential for the data: in the traditional model, the datum exists until someone deletes it; in the declarative model, it is born with an expiry date and its persistence beyond that date requires a positive act of renewal. It’s the inversion of the burden of proof applied to data engineering.

Anonymisation as a verifiable transformation

Another frequent confusion is treating anonymisation as an administrative label —“this field is anonymised”— when it should be a verifiable technical transformation. The joint paper by AEPD and EDPS from 2021 on common misunderstandings around anonymisation is blunt: much of what organisations call anonymisation is actually reversible pseudonymisation, and therefore still personal data under the GDPR.

The architectural consequence: real anonymisation requires information loss, not just a hash or encryption. And that loss has to be integrated into the pipeline as an explicit transformation, not as a property declared by the developer who implemented it. A datum “anonymised” that can be re-identified by cross-referencing it with other tables of the same system is not an anonymised datum — it is a promise of anonymisation the system does not honour.

Default retention is inverted

A natural consequence of the two previous points: in an ephemeral-by-design system, the absence of a declared policy should be a schema error, not an implicit policy of retention.

This translates into schema linters that reject tables without a TTL clause, into CI/CD pipelines that verify that each new type of personal data has an assigned time-to-live, and into code reviews that treat “just in case we might need it” as a decision that demands justification, not as the path of least resistance. The absence of forgetting becomes the exceptional state that has to be defended, not the default state to be assumed.

Cases that illuminate the practice

Three current implementations help understand how this is landing in real industry, beyond the abstract principle.

Delta Lake and VACUUM. Databricks built its response to the right to be forgotten around the VACUUM operation, which physically removes files outside the declared retention window. The default value of 30 days is aligned with the interpretation of the undue delay of the GDPR. But Databricks’ own article acknowledges an uncomfortable side effect: VACUUM also removes all older versions of the table, leaving the system in a state they call “data amnesia” — history loss beyond the datum you actually wanted to erase. Coarse forgetting erases what should not have been forgotten.

ClickHouse and declarative TTL. ClickHouse’s approach is more granular: it lets you declare TTL at the row, column or table level, with actions that go beyond deletion —moving to cheaper storage, aggregating before deleting—. It turns forgetting into a part of the lifecycle managed by the engine, not an external process that has to be orchestrated. Probably the most mature model available today in analytical databases.

LangSmith and inference traces. The most recent case is especially relevant for data engineering applied to AI: LangSmith incorporates TTL for the execution traces of agents, extending the period automatically based on certain actions. It’s a signal of where the industry is heading: forgetting is also coming down to the observability systems of AI, where the volume of incidental personal data —conversation fragments, prompts, intermediate outputs— can be massive and sensitive.

What this approach does not solve

Let’s be precise. Designing infrastructures that forget by default does not solve the problem of what gets recorded in the first place. A system with perfect declarative TTLs can still be collecting personal data that should never have entered. Minimisation at collection precedes minimisation at retention.

Nor does it solve the structural tension with immutable layers. Blockchain-based systems, append-only audit logs, immutable event stores — all built on the premise that “what is written is not changed”. That premise conflicts directly with Article 17 of the GDPR, and there is no clean technical solution inside the immutable paradigm. Practical exits pass through not storing personal data in the immutable layer, or through designing cryptographic mechanisms where destroying a key is functionally equivalent to deleting the datum.

And there is a layer this article cannot cover: forgetting in trained models. A datum erased from a table disappears; a datum erased from a model does not. It stays dissolved in the weights, diffuse but recoverable. That is a different story, with its own engineering and its own impossibilities, and it deserves its own article.

The other side: forgetting as a structural right

So far, engineering. But forgetting in data systems touches something that goes beyond architecture: it touches the question of what kind of collective memory we are building, and whom that memory serves.

On protective opacity

There is one idea worth naming. The absence of a datum is not just a legal obligation — it can be an active form of protection. When a system does not keep something, it cannot be leaked, it cannot be hacked, it cannot be demanded by an authority, it cannot be bought by a third party. Retention builds a surface of risk; forgetting reduces it by subtraction.

Opacity, treated as a design principle, stops being a transparency failure and becomes a protective layer for the people whose data could be exploited if the system kept them. It is not opacity towards the legitimate user — it is opacity towards the potential attacker, the future unauthorised use, the derived analysis nobody consented to. The fading of the datum, when designed with intent, is a form of care.

On the asymmetry between whoever collects and whoever is collected

Systems that accumulate personal data are operated by organisations with technical, legal and commercial resources. The people whose data is accumulated are individuals with little visibility into what is kept, where and for how long.

When forgetting depends on the reactive act of the person —requesting erasure, arguing, waiting, verifying—, the minority willing to invest the effort gets forgotten; the majority keeps being remembered by default.

Ephemerality by design inverts that asymmetry. Forgetting stops requiring a positive act from the subject and becomes the automatic behaviour of the system. Persistence, in turn, requires explicit justification from the organisation. This is more than a technical decision: it is a redistribution of effort between the parties.

On the responsibility of the builder

Here the circle closes. The question is not whether a system should forget. The question is what deserves to be remembered, and for how long, and for what. Every retention decision is, implicitly, a decision about the power the system will accumulate over time. Whoever designs accepts that responsibility, or ignores it — but cannot avoid it.

And that responsibility has three dimensions worth naming as a triad: minimum retention (keep only what’s necessary), explicit expiry (declare the end of each datum), verifiable proof (check that the forgetting really happened). All three must be present. None alone is enough.

Open questions

• If saving is easy and deleting is hard, when was it decided that this would be the default behaviour of our systems — and who benefited from that decision?
• Can a system claim to have forgotten if it cannot verify that the erasure has propagated to all its derived layers?
• Are we designing architectures that serve the people who use them today, or permanent archives that will serve the powers that inherit them tomorrow?

The questions have no closed answer. But one final idea is worth keeping: building infrastructures that forget is not a concession to regulatory compliance. It is a conscious design act about what kind of digital memory deserves to be collectively sustained — and what kind of memory it is wise to deliberately let go of. That decision is not just technical. It is civic. And it belongs to whoever sustains it with their code.

References

• European Data Protection Board — Coordinated Enforcement Action on the right to erasure (CEF 2025), February 2026. edpb.europa.eu
• Recording Law — GDPR Right to Be Forgotten: Article 17 Erasure (2026). recordinglaw.com
• AEPD and EDPS — Joint paper on 10 misunderstandings related to anonymisation (2021). edps.europa.eu
• Databricks — Implementing the GDPR ‘Right to be Forgotten’ in Delta Lake (2022). VACUUM and data amnesia. databricks.com/blog
• OneUptime — How to Implement ClickHouse TTL for Data Retention, January 2026. oneuptime.com/blog
• Google Cloud — Manage data retention with TTL policies (Firestore, Datastore), 2026. cloud.google.com
• LangChain Docs — Enable TTL and data retention (LangSmith), July 2026. docs.langchain.com
• arXiv — A Systematic Literature Review of the Tension between GDPR and Public Blockchain Systems (2022). arxiv.org/pdf/2210.04541