Mean Codeword Lengths and Their Correspondence with Entropy Measures

The objective of the present communication is to develop new genuine exponentiated mean codeword lengths and to study deeply the problem of correspondence between well known measures of entropy and mean codeword lengths. With the help of some standard measures of entropy, we have illustrated such a correspondence. In literature, we usually come across many inequalities which are frequently used in information theory. Keeping this idea in mind, we have developed such inequalities via coding theory approach.

Authors:



References:
[1] Arimoto, S. (1971): "Information theoretical consideration on estimation
problems", Information and Control, 19,181-194.
[2] Bhattacharya, A. (1943): "On a measure of divergence between two
statistically populations defined by their probability distributions",
Bulletin of the Calcutta Mathematical Society, 35, 99-109.
[3] Behara, M. and Chawla, J. S. (1974): "Generalized -entropy", Selecta
Statistica Canadiana, 2, 15-38.
[4] Burg, J. P. (1972): "The relationship between maximum entropy spectra
and maximum likelihood spectra", Modern Spectral Analysis, Childrers,
D.G. (ed.), pp. 130-131.
[5] Campbell, L. L. (1965): "A coding theorem and Renyi's entropy",
Information and Control, 8, 423-429.
[6] Guiasu, S. and Picard, C. F. (1971): " Borne in ferictur de la longuerur
utile de certains codes", Comptes Rendus Mathematique Academic des
Sciences Paris, 273, 248-251.
[7] Havrada, J. H. and Charvat, F. (1967): "Quantification methods of
classification process: Concept of structural -entropy", Kybernetika, 3,
30-35.
[8] Kapur, J. N. (1986): "Four families of measures of entropy", Indian
Journal of Pure and Applied Mathematics", 17, 429-449.
[9] Kapur, J. N. (1995): "Measures of Information and Their Applications",
Wiley Eastern, New York.
[10] Kraft (1949): "A device for quantizing grouping and coding amplitude
modulated pulses", M.S. Thesis, Electrical Engineering Department,
MIT.
[11] Longo, G. (1972): "Quantitative-Qualitative Measures of Information",
Springer-Verlag, New York.
[12] Renyi, A. (1961): "On measures of entropy and information",
Proceedings 4th Berkeley Symposium on Mathematical Statistics and
Probability, 1, 547-561.
[13] Shannon, C. E. (1948): "A mathematical theory of communication", Bell
System Technical Journal, 27, 379-423, 623-659.
[14] Sharma, B. D. and Mittal, D. P. (1975): "New non-additive measures of
entropy for a discrete probability distributions", Journal of Mathematical
Sciences, 10, 28- 40.
[15] Sharma, B. D. and Mittal, D. P. (1977): "New non-additive measures of
relative information", Jr. Comb. Inf. and Sys. Sci., 2, 122-132.
[16] Varma, R. S. (1966): "Generalization of Renyi-s entropy of order",
Journal Math. Sci., 34-48.