Entropy and Effective Support Size

Marian Grendar

Department of Mathematics, FPV UMB, Tajovskeho 40, 974 01 Banska Bystrica, Slovakia
Institute of Measurement Science, Bratislava, Slovakia
Institute of Mathematics and Computer Science, Banska Bystrica, Slovakia

Received: 5 May 2006 / Accepted: 10 August 2006 / Published: 21 August 2006

Abstract: Notion of Effective size of support (Ess) of a random variable is introduced. A small
set of natural requirements that a measure of Ess should satisfy is presented. The measure with
prescribed properties is in a direct (exp-) relationship to the family of R´enyi’s -entropies which
includes also Shannon’s entropy H. Considerations of choice of the value of imply that exp(H)
appears to be the most appropriate measure of Ess.
Entropy and Ess can be viewed thanks to their log / exp relationship as two aspects of the same
thing. In Probability and Statistics the Ess aspect could appear more basic than the entropic one.

Keywords: R´enyi’s entropy; Shannon’s entropy; support, interpretation, Probability, Statistics.

Full text in PDF form, 157 K