Sampling Terms: Confidence Level, Tolerable Error and Friends

Why the vocabulary matters

Sampling questions on the exam are rarely calculations. They describe a change, like “controls are strong” or “more errors are expected”, and ask whether the sample gets bigger or smaller. So the goal here is intuition.

Think about how much you’d need to taste before saying a pot of soup is fine. If you need to be very sure (high confidence), you taste more. If you already suspect the cook was careless (high expected errors), you taste more. If a little too much salt is acceptable (high tolerable error), you taste less. If it’s the same reliable cook as always (strong controls), you taste less.

The terms in plain words

Term	Plain meaning
Population	Everything the conclusion covers. Check it’s complete first, for example by matching record counts and totals.
Sampling unit	One item tested: a change ticket, an invoice line, a currency unit.
Confidence level (confidence coefficient, reliability factor)	How sure you want to be that the sample reflects the population. 95% means a 5% chance the sample misleads you.
Level of risk	100% minus the confidence level.
Precision	How close to the true answer the estimate must be. “Give or take 1%” is tighter than “give or take 3%”.
Expected error rate	How many errors you expect, based on last year or a small pilot.
Tolerable error rate	The most error you can accept and still conclude the objective is met.
Sample mean	The average value in the sample (variable sampling).
Standard deviation	How spread out the values are. More spread means a bigger sample.

What makes the sample bigger or smaller

Higher confidence, expected error, precision and variability increase sample size; higher tolerable error and stronger controls decrease it; population size barely matters.

If this goes up…	The sample…	Because
Confidence level	Grows	More certainty needs more evidence
Expected error rate	Grows	More errors make the estimate shakier
Precision demanded (narrower range)	Grows	A tighter estimate needs more items
Variability (standard deviation)	Grows	More spread means less predictable
Tolerable error rate	Shrinks	You can accept more error
Strength of internal controls	Shrinks	A lower confidence level is acceptable
Population size (when already large)	Barely changes	Sample size levels off for large populations

A shortcut: ask “does this make me need more proof?”. If yes, the sample grows.

The two exam favourites

Strong internal controls let the auditor accept a lower confidence level, which shrinks the sample. Weak controls require higher confidence and a bigger sample.

Expected errors close to tolerable errors make the sample very large. If more errors are expected than can be tolerated, sampling isn’t efficient: the auditor should expect to conclude the control doesn’t work, or test the whole population instead.

Example. Last year’s test of 60 user access approvals found no exceptions, and nothing has changed. This year the auditor accepts 90% confidence instead of 95%, with tolerable error still at 5%, so the sample shrinks. If a reorganisation had changed the approval process, the auditor would go back to 95% and expect a bigger sample.

Evaluating the results

In attribute sampling, the sample’s failure rate is projected to the population with an upper limit. If that upper limit is above the tolerable rate, the control isn’t relied on.
In variable sampling, the projected error is compared with the tolerable error.

The nature of errors matters too: one deliberate override can matter more than several honest slips.

How the exam asks about it

If the question asks…	The answer is usually…
Effect of strong internal controls on sampling	A lower confidence level, so a smaller sample
Which change reduces sample size?	Higher tolerable error, lower confidence, wider precision, stratification
Which change increases sample size?	Higher expected error, higher confidence, tighter precision, more variability
Level of risk at 95% confidence	5%
Effect of doubling an already large population	Very little change in sample size

In short

Confidence level (confidence coefficient) is how sure you want to be that the sample reflects the population. Higher confidence needs a bigger sample.
Level of risk is 100% minus the confidence level: 95% confidence means 5% risk.
Precision is how close to the true answer the estimate must be. Tighter precision needs a bigger sample.
Expected error rate is how many errors you expect. More expected errors need a bigger sample.
Tolerable error rate is the most error you can accept. Accepting more error allows a smaller sample.
More variety (standard deviation) in the population needs a bigger sample; stratification reduces it.
Strong internal controls let the auditor use a lower confidence level, so a smaller sample.
For large populations, population size hardly affects sample size.

Exam traps

Strong controls mean a LOWER confidence level and a SMALLER sample, not a higher one.
When expected errors get close to tolerable errors, the sample gets very large. If expected is above tolerable, sampling isn't worth it.
Doubling a large population doesn't double the sample.
Tighter precision means a BIGGER sample.

Slides

Click the slide and use the arrow keys. F toggles full screen.