Told vs. Shown: Moving Assurance from Attested to Observed

by | 2026/10/23 | Observed Governance

Told vs. shown

The October 2023 Okta compromise exposed thousands of Cloudflare's credentials, and Cloudflare set out to rotate all of them. It missed four: one service token and three service accounts. In Cloudflare's own account, they weren't rotated because "mistakenly it was believed they were unused." One was a Moveworks service token with remote access into Cloudflare's Atlassian environment. Another was a Smartsheet account with admin access to Jira.

Unused or not, they still worked. On November 15, an attacker Cloudflare believes was a nation state used the Moveworks token to get into Jira and Confluence. Over the next eight days the attacker viewed 120 code repositories, until Cloudflare detected it on November 23. Cloudflare says the integration vendors weren't at fault.

The word that matters is "believed." The rotation was done. The four exceptions rested on a belief, and a belief is an attestation. The attacker was the one who tested it.

Now compare a company that said it came through the August 2025 Salesloft Drift token theft unaffected. Secure Code Warrior says its use of Drift had been formally decommissioned and all system integrations were "verifiably removed in December 2022." That's their own account, not an audit. But note the word.

Believed versus verifiably. Most organizations "know" their controls work because someone told them: a questionnaire answer, a signed attestation, an audit opinion. Very few know because the control showed them.

My position: attested assurance starts going stale the day it's signed. Assurance should rest on evidence the system produces while it runs.

If you're the one filling out the questionnaires, this is mostly good news. I'll get to why.

Two models

Attested means someone tells you the state of a control and you take their word for it. "Yes, we have an incident response program." "Yes, we have a security person."

Here's a made-up example. Picture a small bank answering that second question. Call its security person Wilma. Wilma is a teller with an iPhone, which makes her the most technical person in the building, so she's the name on the form. The answer is true. It just isn't what the reader thinks it means.

Observed means you see evidence of the control operating, produced by the system rather than written by a person, as close to real time as you can get. It's a stricter version of what quality auditors call objective evidence. The control executes, the evidence lands, and it's checked against defined criteria. Right owner? Inside the SLA? Approved process? If any of those are off, something fires. Out of compliance, go fix it.

So where do screenshots and reports fall? A screenshot is evidence of one moment that someone chose to capture. A SOC 2 report, a HITRUST certification or an ISO 27001 certificate is better, because an independent party tested something. But that report is still an attestation: the auditor attesting to what they saw, within the scope somebody paid for, over a window that closed before you read it.

Some call an annual report continuous assurance. It isn't. It was signed once. Say someone upgraded a platform to 5.0, half the business process broke, and the controls riding on it went with it. You find out when the external auditor pulls a sample. Or the vendor got breached nine months ago and you learn it from next year's report.

I see it as a spectrum: told, told with a screenshot, told by an independent party, shown by the system. Who produces the evidence, and when, moves you along it.

Why attestation stuck, and why it's breaking

Attestation stuck because it's easy. Ask a few questions when a vendor comes on, get a report if you want one, and decide they're good.

It also does something useful: it tells you what normal is supposed to look like. I'd keep that. Even in an observed model you need the full picture up front: what's been approved, what leadership has blessed, and for what purpose. That's your baseline. The mistake is relying on it from then on. After the baseline, you want to see that what was approved is actually happening. If you're sending data to a partner, it should be the data you approved, not your clients' card data riding along with it.

Attestation costs both sides. The person filling out the questionnaire presents the company in the best light possible, even if that means assuming a few things. "Do you have an access management policy?" Yes. It's an email someone wrote once. That's our policy. Nobody means harm by it. That's how questionnaires get filled out.

The person reading it sees something that points toward good and accepts it. In a third-party relationship, that means accepting risk without knowing you did.

The assessments I trust most are the ones that say, "No, we don't do that yet. It's on our roadmap." Those companies get it. An assessment isn't there to punish anyone. It's there to show where you need to improve.

Why it's breaking is pace, and the causes feed each other.

Vendors: anyone with a credit card can sign up for a service. Someone installs a plugin that puts AI on their email and the CRM, and a company you've never assessed can now read your inbox and your client cases.

AI: I don't mean whether a model is secure. I mean speed. The window between a flaw going public and someone exploiting it has already shrunk from months to days, and attackers are using AI to move faster still. That happens whether your SLAs account for it or not.

And the chain behind your vendor keeps getting longer.

An annual answer can't keep up.

"Isn't this just continuous auditing?"

Partly, yes. If all I meant was "look more often," this would be continuous auditing with a new name. Continuous auditing and continuous controls monitoring (CCM) are established disciplines. The IIA publishes guidance on both. I'm not claiming a new category.

But the interval isn't the point. Whether I look at a control every second or every year, if what I'm looking at is someone telling me it worked, it's the same thing. It's told, just more often.

The first difference is who produces the evidence. In an observed model, the system produces it as the control executes, it's checked against criteria, and a mismatch triggers something.

Plenty of what gets sold as continuous monitoring is attestation on a faster schedule. A monthly control-owner sign-off in a GRC tool is still someone saying yes. If part of your system can't report out through an API, someone exports it by hand once a month. Better than once a year, but still attested.

The second difference is the reference point. Continuous auditing is a method, and a good one. It already tests transactions against specific approvals, like a purchase over a cardholder's authorized limit. What I'm arguing about is what gets tested: the baseline decisions leadership already made. This connection was approved for this data. This vendor was approved for this service. This AI use was approved for this purpose. Each was true when someone signed it. Observed assurance keeps asking whether it's still true now.

Go back to Cloudflare. A test of the rotation checks the rotation, and all but four credentials passed. A test of the decision asks a different question: are those four actually unused? Authentication logs can answer that every day.

So observed assurance is a continuous test of past baseline decisions. Continuous auditing is one way to run that test. CCM is another.

That makes attested vs. observed the higher-level question: who produces the evidence, and which decision is it tested against? It also applies where those disciplines don't usually go: a company whose systems you don't own. You can't watch their network, but you can ask for evidence their systems produce instead of answers.

One frame, four domains

Third-party risk. Attested: "Do you encrypt your databases?" Yes. "All your email?" Yes. Often the person answering is in sales, repeating what they've been told. No offense to salespeople, but you might as well read the white paper. Observed: show me the encryption setting on the database as it stands right now, and the change history behind it.

Internal audit. Attested: finance says changes to vendor bank details are verified by a callback before anyone gets paid. Observed: every change to vendor payment details in the ERP needs a matching verification record before the next payment run, and any change without one holds the payment.

Connectivity and data flows. Attested: the connection was approved and there's a contract. Observed: the data actually crossing it matches what the risk owner approved. The leader who signed off is the one accepting the risk, so you want to know they're accepting the risk of what's really flowing, and not a broader set of data that rode along unapproved.

AI governance. Attested: a policy says employees don't put protected data into AI tools, and everyone signs it once a year with their annual training. Observed: watch the traffic to the AI platforms you allow, or host the model yourself, so you can see what's actually going in.

Same question each time: told, or shown?

Who assures the sensor?

This is where observed assurance can get worse than what it replaced. A green dashboard feels like proof, which makes it more dangerous than a questionnaire you already distrust.

Coverage gaps. You can only observe what you know exists. Say the marketing team stood up ten servers outside governance years ago, and nobody patches them. Your dashboard is green because they aren't on it. Something has to check the inventory itself, like broad scanning of the internal network and anything external-facing.

False precision. The numbers can look exact and still mislead you. A VPN that averages 50 connections a day suddenly shows 50,000. That may be an attack, or it may be an update. The number is precise, but on its own it doesn't tell you which. Or patch compliance reads 98%, and the denominator is only devices with the agent installed. The laptops without the agent aren't failing. They aren't counted.

That's how a green dashboard ends up sitting over a blind spot. Leadership sees a green KPI. They don't see that the green covers one part of the control. If a control has four parts, you need four green lights before you call it green. Know where the gaps are and account for them out loud.

What should stay attested on purpose. Some things are hard to observe, or hard to trigger on even when you can see them.

  • Relational and human controls. You can put a camera on a door, but triggering on whether the right person was let in is hard. These need attestation, plus some agreement on what evidence looks like.
  • Risk acceptance. A leader accepting a risk is a human act. When business leadership changes, have the new owner re-attest to what was set up under the old one.

If you're the one being assessed

If you run a 50-person SaaS company, a HealthTech startup or a defense supplier, this changes things for you, and mostly in your favor.

Be realistic first. Very few companies will let a customer put an appliance on their network to watch their traffic, and saying no doesn't mean you're hiding something.

What "shown" means for you is being evidence-ready. That's different from having a polished answer bank. An answer bank goes stale. Some answers are stale the day after you write them, and most are stale within six months. Then you're back chasing every control owner: are you still doing this, where's the updated SOP? Evidence-ready means each control has a short SOP that says where the evidence lives and how to pull it. The evidence comes from the system, covers the whole period, and can be pulled again tomorrow without a rewrite.

Two examples. For encryption: the database's encryption setting plus its change log, showing it's been encrypted since January 1 of last year with no changes. For identity: an export of your identity provider's policy showing MFA enforced for all users, plus the sign-in logs for the period. Better still if the reviewer gets a system-generated report or read-only access instead of a file you assembled and could have edited.

What should you make observable first? Your inventory: information assets, hardware, software, and your own vendors. If I can see you don't control your own hardware, that devices walk out the door and nobody collects certificates of destruction for the drives, I don't need to read the rest of the questionnaire.

The payoff is time, and fewer security reviews that stall a deal. When a customer's questionnaire lands, or a C3PAO shows up for a CMMC Level 2 assessment, you show how your program performs instead of describing it.

Making the move, and the bet

If I had to pick the first two things to move on Monday:

One: your inventory. Make it accurate, and put something in place that watches for changes so it stays that way. That covers third parties, hardware and information assets. It also covers the integrations and service accounts connected to your systems, with the last time each one was used. Start with the ones you believe are unused. You can begin without an inventory, but you'll be building on blind spots. Get it done in the first quarter, or the first month if your size allows.

Two: the riskiest thing you already know about. Pick the one control where being wrong would hurt most and make it observable.

The most common mistake is boiling the ocean. A close second is chasing the flashy thing: AI is the boogeyman right now, so it gets the budget whether or not your risk is there.

The quieter mistake is doing security in a vacuum. If your observed controls protect a process that doesn't line up with the company's strategy, you're building a Ferris wheel in the middle of the forest. A security program exists to reduce the risk of harm to critical data, people and systems, in service of what the business is trying to do. Otherwise it's spending money on a false sense of assurance that will come back on you.

So here's my bet. In five years, the annual questionnaire won't be annual. It will be for onboarding only, tailored to the relationship, the service and the risk. The rest will be a memory of a time when we sent a lot of emails, read a lot of answers, collected a lot of screenshots and waited a year for the next report.

Until then, take every control you rely on and ask which it is: believed, or verified?

Kris Decker founded ClearSec, a security consultancy for regulated small and mid-sized companies, providing fractional security leadership (vCISO), security program design, third-party cyber risk work, and HIPAA, CMMC and GLBA assessments.

Sources