Skip to content

What does Intercoder Agreement measure?

1 min readAnalysis

Overview

The agreement view compares whether two coders applied codes within comparable documents and highlights disagreements for review. It reports percent agreement, Cohen’s kappa and Krippendorff’s alpha for document-level code presence, and lists disagreements so you can discuss and adjudicate them. Unless a measure explicitly compares matching segment boundaries, document-level agreement should not be reported as segment-level intercoder reliability. Find it under Analysis › Rigour › Intercoder Agreement.

What to do

  1. Confirm that coders worked from comparable source copies and codebook versions.
  2. Review disagreements rather than relying only on an overall figure.
  3. Use Adjudication to document the final decision and reasoning.
  4. Report the unit of analysis, the two coders and the measures used.

When to use it

  • After two people have coded the same documents independently.

Terms in this answer