djtaylor.me

Sending strings

Intro.

section

Hypothesis:

• The matrix $\begin{pmatrix} 1 & 0 \\ 1 & -1 \end{pmatrix}$ is going to perform slightly better than $\begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}$. The reasoning is that for images whose histograms are relatively flat the 1,1 term in the top row will keep it flat, but for histograms that are spiked it will squash and spread the histogram out more. This should increase entropy since, if $p_1 < p_2$ and the original $H = -p_1 \log p_1 - p_2 \log p_2$, then $H' = -(p_1 + \epsilon) \log(p_1 + \epsilon) - (p_2 - \epsilon) \log(p_2 - \epsilon) \approx H + \epsilon (\log{p_2} - \log{p_1}) + O(\epsilon^2)$. In other words spreading out a distribution should increase entropy if $\log{p_2} - \log{p_1} > 0 \iff p_1 < p_2$.