A simple connection between Entropy and Information

This article follows a simple example laid out by Jaynes (1996).
Jaynes’ example is one that shows how one’s computation of the change in entropy in a physical / chemical process depends on the precise variables that one uses to label the macro state. If you use different variables (say you are insensitive to properties that someone else does have the ability to measure) you can invent situations where the entropy change might (to one observer) be contrived to violate the Second Law of Thermodynamics while the other observer sees no such violation.
Let’s consider a vessel whose volume is $V$ , with a diaphragm that separates it into two parts – volumes $V_1$ and $V_2$ with $N_1$ and $N_1$ molecules of each type, respectively. The “1” side is filled with Argon gas of type $A_1$ which is indistinguishable from the type of Argon gas filling side “2”, which we call type $A_2$ , at least for the observer named Babu. However, Alisha, with her access to superior technology is indeed able to perceive the difference between the two types of Argon. The container is in close thermal contact with a heat bath that maintains a constant temperature $T$ . The equilibrium condition (same temperature throughout and equal pressure) implies that $n_1/V_1 =n_2/V_2 =(n_1+n_2)/V$ .
Alisha, in addition to her ability to notice the difference between $A_1$ and $A_2$ also has sole access to a material called Whiffnium, which is permeable to $A_1$ but impervious to $A_2$ . During the course of her research, she has also discovered Whaffnium, a new material which is permeable to $A_2$ , but impervious to $A_1$ . Let’s suppose that Alisha constructs two (infinitesimally thin) pistons that are initially placed very close to each other, one piston made of Whiffnium and the other of Whaffnium, as in the picture below.

Let’s suppose that just enough of $A_1$ permeates through the Whiffnium so that the partial pressure of $A_1$ is the same in the left side of the container as well as the intercalated region (between the Whiffnium and Whaffnium pistons). Similarly, let’s assume that just enough of $A_2$ permeates through the Whaffnium into the intercalated region (between the pistons) so that the partial pressure of $A_2$ is the same in the intercalated region as well as on the right side of the container. Now, due to the unbalanced pressure of $A_2$ impinging upon the Whiffnium piston, it is reversibly moved to the left and the entropy change in $A_2$ is
$= n_2 k_B ln(V/V_2)$
Similarly, the Whaffnium piston is reversibly moved to the right and the entropy change in $A_1$ is
$= n_1 k_B ln(V/V_1)$
The total entropy change is hence
$n_1 k_B ln(V/V_1 )+n_2 k_B ln(V/V_2 )$
All this is pretty logical from Alisha’s point of view, since she does see the two parts of the container as having different materials, $A_1$ and $A_2$ . She understands that the entropy change in the container is a consequence of the heat flowing into the system from the heat bath.
However, Babu sees a conundrum. He sees the argon as one undifferentiated gas and so the initial and final states of the system are identical. However, the system has absorbed an amount of heat and converted all of it into work, in violation of the Second Law of Thermodynamics. In addition, he sees the entropy change as $0$ . This is, however, simply a reflection of the fact that the entropy is a function of the macrostate variables that one uses and if Babu has an insufficiently specified macrostate, then Alisha is simply able to manipulate phenomena to cause Babu to think he has observed a violation of the Second Law.
How much information would Babu need to in order to deduce the correct entropy change? In the initial state, if he knew about the sub-identities $A_1$ and $A_2$ , the macrostate where $A_1$ is on the left and $A_2$ is on the right (side of the container) has the following number of (equally probable) microstates
$=(V_1/v)^(n_1 ) (V_2/v)^(n_2 )$
Where we have assumed that each molecule can be localized to a minimum volume $v$ and we can do this for all the $n_1$ molecules in $V_1$ and the $n_2$ molecules in $V_2$ .
In the final state, all the $n_1+n_2$ molecules are strewn about in the total volume $V$ and the total number of microstates is
$= (V/v)^(n_1+n_2)$
So to specify the microstate, he needs to be communicated extra information (along the lines of traditional information theory)

$I= \log_2((\frac{V}{v})^{n_1+n_2} ) - \log_2 ( (\frac{V_1}{v})^{n_1} (\frac{V_2}{v})^{n_2} )$

$= n_1 \log_2(V/V_1)+n_2 \log_2(V/V_2 )$
Which is exactly (up to a multiplicative factor of $k_B$ and difference between $\ln$ and $\log$ , just the same as the entropy change to separate the molecules into the different varieties.
Note that if Babu didn’t actually meet Alisha and had no idea that there were two varieties of Argon, his calculation for the number of microstates before and after would be identical, equal to $(V/v)^(n_1+n_2 )$ – this is because he doesn’t even think the diaphragm separating the two sides of the container is even necessary – they are the same materials and are in thermodynamic equilibrium with each other.
However, once Babu has this information, in the form of a detailed message, he will have been supplied with enough information to deduce completely (as far as Alisha’s abilities admit) to the situation with the two varieties, where he had zero before. Ergo, the extra information he needs is the entropy difference.