Calibrating with other judges at the start of a competition flight

Why calibration matters

Competition entries are scored by small panels, usually two or three judges per flight. If one panel scores everything in the 20s and the next scores in the 30s, the medals depend on which table a beer landed on, not on how good it was.

Calibration reduces that. Before judging starts, everyone tastes the same beer, scores it independently and compares notes. The aim is not identical scores. It is a shared sense of what a 30 and a 40 mean on the day, plus which faults the room agrees are present.

How a calibration round usually runs

Practice varies between competitions, but a common pattern is:

1. The organiser pours a calibration beer, often a well-known commercial example of a style being judged that day.
2. Each judge fills in a full scoresheet without talking.
3. The organiser or head judge reads out the scores and asks a few judges to explain theirs.
4. The group discusses where the beer sits against the style guidelines and agrees a rough target range.

It takes about 15 to 20 minutes. It feels slow when 200 entries are waiting, but it saves far more time later, because panels spend less time arguing over each beer.

Calibrating with your table partner

Room-wide calibration is useful, but the real work happens at your table. Before the first entry, talk to your partner for two minutes. Who is more experienced? Is either of you sensitive or blind to a particular compound? Plenty of people struggle to detect diacetyl at low levels. Others are very sensitive to DMS.

After the first two or three beers, compare scores early. If you are consistently 6 to 8 points apart in the same direction, one of you is running hot or cold. Adjust now rather than on beer nine.

Closing the gap on each entry

Many competitions ask a pair of judges to bring their scores within about 5 to 7 points of each other before submitting. When you are further apart, talk it through:

• Start with what you each perceived, not the number.
• Check whether one of you found a fault the other missed. Re-taste for it.
• Look at the style guidelines together.
• Adjust honestly. Do not simply split the difference. Do not cave to the louder judge.

If you still disagree after discussion, say so to the head judge. A well-reasoned disagreement is better than a forced consensus on a sheet that reads oddly.

Palate fatigue and order effects

Calibration drifts during a session. By beer ten, palates are tired and judges often score lower or higher than they did at the start. The order of beers matters too: a delicate helles after a big, bitter one will taste flat.

Competitions manage this by keeping flights to a sensible size, often around 8 to 12 beers. They also order them from lighter to stronger, as our beer flight guide describes. As a judge, drink water, eat plain crackers between beers and spit rather than swallow. Glassware matters a little too. Tall, thin glasses, such as the ones covered in our piece on the stange and the willibecher, show colour and carbonation clearly.

Recalibrate in the middle of a long day

A good head judge will sometimes slip a second calibration beer in after lunch, or ask panels to re-taste their top entries before passing them on. If your table has drifted, it shows up quickly.

You can do this yourself. If the last three beers all scored below 25, pause and ask whether the beers are really that weak or whether your palate is tired. Re-taste the first beer of the flight if a sample is still available. That simple check keeps the flight fair from first to last.

Know your own blind spots

Every taster has thresholds. Some people detect diacetyl at very low levels while others need several times as much before they notice it. The same goes for DMS, acetaldehyde and the skunky thiol from light. A small number of people cannot smell certain compounds at all.

Calibration is partly about learning these limits. If your partner keeps finding butter that you miss, you may have a high threshold for diacetyl. That is not a failure. It is useful information. Write it down. Tell future table partners at the start of a session so they can double-check that note for you.

Spiked samples are the best way to find your thresholds before a competition. A clean light lager dosed with a known amount of a reference compound shows exactly where you start to notice it.

Calibration is a skill you can practise

Judges who calibrate well have usually tasted the same beers alongside other trained tasters many times, compared notes and learned where their own palate runs hot or cold. That takes deliberate practice, not just more competitions.

Your first step: before your next judging session, taste one commercial classic of a style you will judge, fill in a full scoresheet and compare it with a friend or a published description. Note which section you scored furthest from the reference. That is where your calibration needs work.

Common questions

What is a calibration beer?

A beer every judge tastes and scores before the competition, used to bring the panel onto a shared scale.

How close should two judges' scores be?

Many competitions ask judges at a table to settle within about 5 to 7 points before submitting a consensus.

What if judges cannot agree?

Discuss what each perceived, re-taste for disputed faults and consult the head judge if needed. Do not split the difference by default.

Does palate fatigue affect scores?

Yes. Scores drift as palates tire, so flights are kept short and judges use water, crackers and spit cups.