How does the entropy change during the cooling of a hot coffee in a cold cup?

You are correct that the entropy of the coffee will decrease while the entropy of the cup increases. However this will not decrease the total entropy of the system. Rather, heat will continue to flow between the two objects until entropy can no longer increase.

Let $\Delta S$ mean the change in total entropy as the energy of contents of the two components change by $\Delta Q_\text{cup,coffee}$. For infinitesimal energy transfer, we should have $$\Delta S = \beta_\text{cup} \Delta Q_\text{cup} + \beta_\text{coffee} \Delta Q_\text{coffee}$$ for some quantity $\beta$, that is not necessarily the same for both components. Let us see what this quantity is.

Since energy is conserved, we have $\Delta Q_\text{coffee}= - \Delta Q_{cup}$ and we can write the change in entropy as $$\Delta S = \Delta Q(\beta_\text{cup} - \beta_\text{coffee}).$$ Here I have taken $\Delta Q = \Delta Q_\text{cup}$ to be positive -- it's the heat gained by the cup. Thus the entropy change is consistent with thermodynamics if $$\beta_\text{cup} \ge \beta_\text{coffee}.$$ The condition for equilibrium is $$0 = \Delta S \Leftrightarrow \beta_\text{cup} = \beta_\text{coffee}.$$ We see that $\beta$ is a quantity that is equal when two systems that can exchange heat are equilibrium -- it must be a function of the temperature. Since heat flows from objects with smaller $\beta$ to objects with greater $\beta$, we define temperature through the relation $\beta = 1/T$.