The inference of Shannon entropy out of sample histograms is known to be affected by systematic and random errors that depend on the finite size of the available data set. This dependence was mostly investigated in the multinomial case, in which states are visited in an independent fashion. In this paper the asymptotic behavior of the distribution of the sample Shannon entropy, also referred to as plug-in estimator, is investigated in the case of an underlying finite Markov process characterized by a regular stochastic matrix. As the size of the data set tends to infinity, the plug-in estimator is shown to become asymptotically normal, though in a way that substantially deviates from the known multinomial case. The asymptotic behavior of bias and variance of the plug-in estimator are expressed in terms of the spectrum of the stochastic matrix and of the related covariance matrix. Effects of initial conditions are also considered. By virtue of the formal similarity with Shannon entropy, the results are directly applicable to the evaluation of permutation entropy.

Asymptotic distribution of sample Shannon entropy in the case of an underlying finite, regular Markov chain / Ricci, Leonardo. - In: PHYSICAL REVIEW. E. - ISSN 2470-0045. - 103:2(2021), pp. 022215.1-022215.7. [10.1103/PhysRevE.103.022215]

Asymptotic distribution of sample Shannon entropy in the case of an underlying finite, regular Markov chain

Ricci, Leonardo
2021-01-01

Abstract

The inference of Shannon entropy out of sample histograms is known to be affected by systematic and random errors that depend on the finite size of the available data set. This dependence was mostly investigated in the multinomial case, in which states are visited in an independent fashion. In this paper the asymptotic behavior of the distribution of the sample Shannon entropy, also referred to as plug-in estimator, is investigated in the case of an underlying finite Markov process characterized by a regular stochastic matrix. As the size of the data set tends to infinity, the plug-in estimator is shown to become asymptotically normal, though in a way that substantially deviates from the known multinomial case. The asymptotic behavior of bias and variance of the plug-in estimator are expressed in terms of the spectrum of the stochastic matrix and of the related covariance matrix. Effects of initial conditions are also considered. By virtue of the formal similarity with Shannon entropy, the results are directly applicable to the evaluation of permutation entropy.
2021
2
Ricci, Leonardo
Asymptotic distribution of sample Shannon entropy in the case of an underlying finite, regular Markov chain / Ricci, Leonardo. - In: PHYSICAL REVIEW. E. - ISSN 2470-0045. - 103:2(2021), pp. 022215.1-022215.7. [10.1103/PhysRevE.103.022215]
File in questo prodotto:
File Dimensione Formato  
PhysRevE_2021_103_022215_Ricci.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 274.86 kB
Formato Adobe PDF
274.86 kB Adobe PDF   Visualizza/Apri
PhysRevE_2021_103_022215_Ricci_preprint.pdf

accesso aperto

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 162.83 kB
Formato Adobe PDF
162.83 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/293897
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? 6
social impact