Entropy Rate Bounds of Integer-Valued Processes via Second-Order Statistics
METADATA ONLY
Loading...
Author / Producer
Date
2023
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
This work contains two single-letter upper bounds on the entropy rate of an integer-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large alphabets. The first bound stems from Gaussian maximum-entropy considerations and depends on the power spectral density (PSD) function of the process. While the PSD function cannot always be calculated in a closed-form, we also propose a second bound, which merely relies on some finite collection of auto-covariance values of the process. Both of the bounds consist of a one-dimensional integral, while the second bound also consists of a minimization problem over a bounded region, hence they can be efficiently calculated numerically. In some models that appear naturally in various disciplines and that consist of infinitely large alphabets, assessing the related entropy rates may be a complicated numerical problem. In such cases, we argue that the new proposed bounds can still be efficiently calculated.
Permanent link
Publication status
published
Editor
Book title
2023 IEEE Information Theory Workshop (ITW)
Journal / series
Volume
Pages / Article No.
492 - 496
Publisher
IEEE
Event
IEEE Information Theory Workshop (ITW 2023)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03529 - Lapidoth, Amos / Lapidoth, Amos