Eprints of Rényi Institute

Information inequalities for four variables

Csirmaz, László (2013) Information inequalities for four variables. CEU . (Unpublished)

[img]
Preview
PDF
204Kb

Abstract

The entropy region for a fixed number of random variables
consists of those real vectors where the coordinates take the Shannon entropies of the subsets of the variables. It is known that the closure of this region is a convex cone, and only boundary points might be missing, see F. Matus. The closure is bounded by the so-called Shannon inequalities. For at most three random variables this bound is tight. For four variables a systematic search was initiated to find further bounding inequalities using the technique found by Zhand and Yeung in 1998. This paper contributes to this work.
1. We state and prove a very general form of the copy lemma which is behind the method of generating new entropy inequalities.
2. We show that non-Shannon inequalities can be written using a natural coordinate system with non-negative coordinates. We prove F. Matus' theorem which says that four of these coordinates must necessary be zero, and two other can be assumed to be zero.
3. Given a general copy instance, how all its consequences can be extracted algorithmically? To this end we translate this problem to a multi-objective linear programming. We discuss and improve significantly Benson's algorithm finding all extremal solutions.
4. We show the feasibility of our approach significantly by running it through all copy instances in the extensive list of Dougherty and al.
5. Information theoretical inequalities have strong ties with matroid theory and the question of representability. the notion of adhesivity stems from the technique of getting new non-Shannon type entropy inequalities. F. Matus gave a sufficient and necessary conditions for self-adhesivity on four-element polymatroids. Using our techniques we extend his result for 3-, 4-, and 5- self-adhesivity. Our characterizations also give new information inequalities on four random variables.

Item Type:Article
Keywords and phrases:Entropy; information inequalities; multiobjective linear programming
Subjects:52B12 Special polytopes (linear programming, centrally symmetric, etc.)
90C29 Multi-objective and goal programming
26A12 Rate of growth of functions, orders of infinity, slowly varying functions
94A17 Measures of information, entropy
Divisions:Research Groups > Theoretical cryptography
Last Modified:03 Jan 2013 15:56

Repository Staff Only: item control page