International Journal of Internet Science

A peer reviewed open access journal for empirical findings, methodology, and theory of social and behavioral science concerning the Internet and its implications for individuals, social groups, organizations, and society.

Volume 5, Issue 1 (2010)

ReCal: Intercoder Reliability Calculation as a Web Service
Deen G. Freelon
University of Washington, Seattle, USA

Abstract: Despite the fact that intercoder reliability is an indispensable validity criterion for studies that employ content analysis, currently available options for calculating it are sharply limited both in terms of the number of coefficients they offer and the range of operating systems they support. This paper introduces ReCal, an online intercoder reliability Web service that addresses both of these limitations by offering multiple coefficients for nominal-level data and by functioning through any Web browser regardless of OS. After describing the program's functions, its accuracy is demonstrated through a set of worked examples and a systematic comparison of its results to those of alternative reliability calculators. Finally, ReCal's usage statistics are briefly presented.

Keywords: Intercoder, interrater, interjudge, reliability, Web service, online software, Krippendorff's alpha, nominal data

pdf Download full paper