AI system mapping is impossible at scale, and what to do instead
Documenting the wrong metric
By Yule Guttenbeil
Most privacy and AI compliance projects I am brought in to document begin with the same assumed starting point. Map the systems. List the tools, data flows, and workflow that touch personal information, then produce a document that describes how the organisation is intended to operate. The instruction is well intentioned. It is also, in any organisation of meaningful size, impossible to comply with in good faith. Systems are always changing, being tweaked by staff and the developers alike.
Prospective system mapping fails as a compliance posture, so we need to determine what to put in its place. It draws on a submission I recently made to the Office of the Australian Information Commissioner on the new automated decision-making transparency obligation in the Privacy Act, but the argument applies far beyond that obligation.
Why mapping fails
The failure is structural, not motivational. Four facts about modern enterprise software defeat the mapping exercise before it begins.
First, systems change continuously. Patches, vendor changes, feature toggles, and reconfigurations are measured in weeks, not years. By the time a workflow map is drafted, reviewed, and approved, the underlying systems have moved.
Second, there is no canonical workflow. Multiple systems process the same personal information for different purposes, configured by different teams operating under different functional KPIs. The composition of effects is emergent from how individual staff configure their tooling, then use those tools, not from any unified design.
Third, institutions do not, and cannot, fully know their own systems. Shadow IT, unsanctioned SaaS, individually configured tooling, and AI features that are introduced, changed and activate by default in standard productivity suites all sit outside the visibility of a central compliance or technology function. Asking a documentation function to map them is asking it to document what it cannot see.
Fourth, vendor opacity is real. Even where a tool is centrally procured, the deploying entity often cannot describe with precision what the tool does internally, particularly where AI components are involved.
The three failure modes of mapping at scale
When a compliance regime demands the unknowable, the result is predictable. Three failure modes appear, sometimes in combination.
- Pro-forma boilerplate, in which the documentation is generic enough to be technically true and operationally useless.
- Immediately stale honest documentation, in which a serious effort produces an accurate snapshot that begins drifting from reality the day it is published.
- Compliance theatre, in which the documentation exists primarily to be shown to auditors, regulators, or boards, and plays no role in how the organisation actually operates.
None of these protects individuals. None of these reduces the entity’s exposure when something goes wrong. They satisfy a process and miss the point.
A different posture: categorical disclosure plus configuration retention
The principled response is to stop trying to document a moving target prospectively, and to build the capability to recover the truth retrospectively when it matters. This has two components.
The first is categorical disclosure. Pitch the description of what your automated systems do at the level the statute itself adopts. The kinds of decisions automated processes shape, and the kinds of personal information they use. This description is stable across system change. It is meaningful to the people you are describing it to. It is tractable for the entity to maintain.
The second is configuration retention and auditability. The operative configuration of modern systems, particularly agentic AI systems, is increasingly held in primitive instruction artefacts that are written in plain text. Skills.md files in current agentic platforms are one example. Equivalent instruction files exist across vendor stacks. These artefacts are inspectable, version controllable, and enumerable. They tell you what the system was actually doing at a point in time. Fortunately, these are increasingly in plain English, requiring no coding knowledge to comprehend.
If you retain those artefacts, you can reconstruct, after a decision is challenged, what filtering and routing was in force at the time of the decision. You convert categorical disclosure (which is all a privacy policy can practically carry) into something with operational teeth, because behind the categorical statement there is a recoverable provenance trail.
Why this is the systems-thinking move
The shift here is the same shift mature organisations make in safety, in financial controls, and in privacy more broadly. You stop privileging the existence of a control and start privileging the operation of a control. You stop producing documents whose purpose is to be shown, and start producing capabilities whose purpose is to work.
The mapping instinct comes from a reactive posture. It treats compliance as a fire to be put out by producing a document. The configuration retention posture treats compliance as a system to be designed, in which the work is front-loaded into how tools are deployed and configured, and the artefacts produced are usable when an issue arises.
It is also the posture that scales. A boutique business and a large enterprise both struggle with prospective mapping. Both can adopt categorical disclosure. Both can put discipline around configuration retention, even if the tools and the volume differ.
Practical first steps
The first moves are not large.
- Audit your current privacy policy disclosures against the categorical test. Do they describe the kinds of decisions and the kinds of information, in terms a layperson could act on?
- Identify where instruction artefacts already exist in your tooling. Skills files, prompt libraries, automation rules, workflow configurations.
- Decide who owns retention. Set a retention period that aligns with your contestability windows.
- Treat AI features that activate by default in productivity suites as in scope. The deploying entity has arranged for them by failing to disable or configure them.
- Stop investing in workflow maps that will be stale before they are signed off.
The result is a compliance function that operates instead of one that documents. That is a better posture in privacy, in AI governance, and in the next decade of regulation that is heading toward Australian businesses.
If you want to talk about how this applies in your environment, you can reach me at yule@attune.legal.
