Some properties of Rényi entropy over countably infinite alphabets

Abstract

We study certain properties of Rényi entropy functionals H α(P) on the space of probability distributions over ℤ+. Primarily, continuity and convergence issues are addressed. Some properties are shown to be parallel to those known in the finite alphabet case, while others illustrate a quite different behavior of the Rényi entropy in the infinite case. In particular, it is shown that for any distribution P and any r ∈ [0,∞] there exists a sequence of distributions Pn converging to P with respect to the total variation distance and such that limn→∞ lim α→1+ H∞(Pn) = lim α→1+ limn→∞ H ∞(Pn) + r. © 2013 Pleiades Publishing, Ltd.