Control testing answers a question that sounds simple and is not: does the control work? Not whether it exists, not whether someone wrote a procedure describing it, but whether it operated, consistently, across the whole period, and whether it would have caught the thing it was designed to catch. Most control failures found in audits and regulatory reviews are not missing controls. They are controls that were present, documented, and not actually working.
Design and operating effectiveness are different questions
A control has to clear two hurdles in order. Design effectiveness asks whether a control, if operated as intended, would prevent or detect the risk it addresses. Operating effectiveness asks whether it was in fact operated that way throughout the period.
The order matters because testing the operation of a badly designed control is wasted effort: a monthly review that never looks at the population where the risk lives will pass a hundred samples and prove nothing. Design is assessed by walking a transaction through the process end to end and asking what would have happened had the error occurred. Only when the answer is satisfactory is there any point sampling.
The second distinction is between preventive controls, which stop something happening — segregation of duties, system-enforced approval limits, three-way matching — and detective controls, which find it afterwards: reconciliations, exception reports, management review. Preventive controls are generally stronger and cheaper to rely on; detective controls only work if someone acts on what they surface, which is why an exception report nobody investigates is not a control at all.
What evidence actually looks like
Testing methods sit on a ladder of persuasiveness. Enquiry alone — asking the control owner what they do — proves almost nothing and should never stand on its own. Observation shows the control operating at a point in time, but only that point. Inspection of documentary evidence is the workhorse: the signed approval, the completed reconciliation, the system log. Re-performance, where the tester independently carries out the control, is the strongest and the most expensive.
The chronic weakness is evidence of review controls. A manager who genuinely scrutinises a reconciliation each month and initials it has performed a control; the initial alone does not demonstrate what was reviewed, against what criteria, or what happened to the items that looked wrong. The fix is unglamorous — a short note of what was checked and what was followed up — and it converts an unprovable control into a testable one.
Sampling without kidding yourself
Sample sizes follow the frequency of the control and how much reliance is being placed on it. Annual and quarterly controls are tested in full or nearly so; monthly controls need a handful of instances; daily and transaction-level controls need substantially more. Automated controls are the exception: where the configuration is fixed and general IT controls over change management and access are effective, a single test of the configuration can support reliance for the whole period — which is why testing IT general controls first is efficient rather than pedantic.
Two sampling errors recur. The first is choosing the sample from a population the tester assembled rather than the complete system-generated population — if the list is incomplete, the sample is meaningless however large. The second is treating a deviation as a one-off. A single failure in a sample of twenty-five is not "an exception"; statistically it implies a failure rate the sample was designed to detect. The response is not a mechanical extension of the sample, though. The standard requires the tester to ask specific questions about what happened and why, and then decide between three answers: the testing still supports reliance, more testing is needed, or the risk has to be addressed substantively instead. What is never right is to note the deviation and move on.
From findings to something worth reading
A finding is only useful if it says what failed, why it matters, and what the consequence was. The structure that survives challenge:
- The condition — what was found, with the population and the number of deviations stated;
- The criterion — what should have happened, by reference to the policy, the standard or the regulation;
- The cause — why it happened, which is what distinguishes a training issue from a design flaw from a resourcing problem;
- The effect — what the failure exposed the organisation to, quantified where possible;
- The recommendation and owner, with a date that someone has agreed to rather than been given.
Findings without causes generate remediation that fixes symptoms. The same reconciliation control fails the following year, in a different department, because nobody asked why it failed the first time.
Where it fits
Control testing is done by different people for different purposes, and conflating them causes friction. Management tests its own controls as part of running the business — the first line. A risk or compliance function monitors that testing — the second. Internal audit provides independent assurance over both — the third. And the external auditor's controls work is narrower than most boards assume, though not as narrow as it is often described: under ISA (UK) 330 the auditor must test controls where they intend to rely on them and where substantive procedures alone cannot provide sufficient appropriate evidence at the assertion level — which is the position for many high-volume, systems-generated populations regardless of whether reliance was planned.
Each of those has a different standard of independence and a different reporting line, and evidence produced for one is not automatically adequate for another. Where an organisation wants the same testing to serve multiple purposes — and it usually should, for cost reasons — that has to be designed in from the start rather than negotiated afterwards.
Acumon runs control testing programmes through internal audit and outsourced internal audit, and prepares finance teams for external scrutiny through audit readiness work — with the governance framing set out in our guide to the three lines model. If your control documentation has not changed in three years but your systems have, that gap is where the next finding comes from.