KAPPA
A
A
B
a
b
B
c
d
Percent observed agreement (O)
=
Percent expected by chance (E)
=
Kappa = (O - E) / (100 - E)
=
Kappa > 0.75 represent excellent agreement.
http://www.gotoknow.org/posts/395368