Mutual information I ( X ; Y ) between two random variables X and Y is what is left over when their mutual conditional entropies H ( Y | X ) and H ( X | Y ) are subtracted from their joint entropy H ( X , Y ) . It can be given by the formula I ( X ; Y ) = − ∑ x ∑ y p X , Y ( x , y ) log b ⁡ p X , Y ( x , y ) p X | Y ( x | y ) p Y | X ( y | x ) . Wiktionary