Does a lower index mean everyone has more?

No. All values can fall proportionally while inequality remains unchanged.

Does decomposition identify causes?

No. It partitions the measured inequality according to a chosen grouping; causal claims require further analysis.

Is a zero value always equality?

Zero T indicates equality among the defined, weighted observations, provided the calculation is valid. It does not describe omitted dimensions or groups.

AI Glossary / Mathematics and optimization / Intermediate

Theil index

Definition

The Theil index is an entropy-based measure of inequality in nonnegative quantities such as income or resource allocations. The commonly used Theil T index equals zero when equally weighted observations have identical values and rises as their shares become more unequal. It can be decomposed into between-group and within-group contributions.

What it means

Suppose a total resource is divided among people, locations, or other comparable units. Theil T compares the observed shares with equal shares. It measures relative distribution, so multiplying every value by the same positive amount leaves the index unchanged.

The unit of observation matters. Treating each region equally measures inequality across regional values; weighting regions by population addresses a different question. The index is descriptive rather than a judgment about fairness, need, effort, or the reasons for unequal outcomes. Those interpretations require additional evidence and an explicit decision framework.

How it works

For n equally weighted nonnegative values yi with positive mean μ, Theil T is (1/n)Σ(yi/μ)ln(yi/μ). Equivalently, if si is each observation's share of the total, T=Σsi ln(nsi). Natural logarithms give the usual convention, and zero shares contribute zero by a limiting convention.

Partitioning observations into mutually exclusive groups separates T into a between-group term and an income-share-weighted sum of within-group Theil indices. The between-group term depends on group resource shares relative to population shares.

Theil L, also called mean log deviation, is a related but different measure, commonly written (1/n)Σln(μ/yi) for strictly positive values. Its weighting and zero-value behavior differ. Always state whether a report uses T or L, its weights, logarithm convention, and treatment of zero or invalid observations.

Three practical examples

1. Resource reporting: describe how a nonnegative allocation is distributed among comparable service units, alongside the actual allocation levels.

2. Regional analysis: separate inequality between defined regions from inequality within them, using weights aligned with the intended population question.

3. Model evaluation: examine concentration in a model's nonnegative assigned benefits while treating this summary as one diagnostic rather than a complete fairness assessment.

Think of it this way

Think of comparing a distribution of tokens with equal piles for every participant. The index summarizes how far the observed shares depart from equal shares and can separate differences between tables from differences among people at each table.

Equal piles are only the mathematical reference. The analogy does not establish that equal allocation is appropriate when participants have different needs or responsibilities. It also loses meaning if the quantities can be negative without a justified alternative definition.

Worked example

Consider an original allocation [1,1,3,3] across four equally weighted units. The mean is 2, so the ratios to the mean are [0.5,0.5,1.5,1.5]. Using natural logs gives T=(1/4)[2(0.5ln0.5)+2(1.5ln1.5)]≈0.130812.

Divide the units into group A=[1,1] and group B=[3,3]. Each group contains half the units. Their resource shares are 0.25 and 0.75. Within each group, values are equal, so both within-group indices are zero.

The between-group contribution is 0.25ln(0.25/0.5)+0.75ln(0.75/0.5)=0.130812. It therefore explains the full index for this grouping.

Now regroup as A=[1,3] and B=[1,3]. Both group means equal the overall mean, so the between-group term is zero. Each group has Theil T=0.130812, and their equal resource shares produce the same overall index through the within-group component.

The overall inequality is unchanged, but its decomposition changes with the grouping. Multiplying all four allocations by ten also leaves T unchanged. These calculations illustrate distribution and decomposition, not a causal explanation or measured social outcome.

Illustrative case study

Fictional case: Elm Network compares allocations across four service centers. Managers initially present one regional decomposition as proof that regional policy caused the observed inequality.

An analyst shows how different group definitions change the between-group share without changing any allocation. The team retains the index as a descriptive summary, reports unit-level values, and investigates demand and population separately. Its revised interpretation distinguishes accounting decomposition from causal attribution. The scenario does not claim that a particular allocation policy is fair or unfair.

Common misconceptions and mistakes

Theil T is not ordinarily a percentage and is not automatically bounded by one. For n equally weighted nonnegative observations with positive total, its maximum is ln(n), reached when one unit receives everything.

Do not substitute Theil L without checking definitions. A zero observation has a finite limiting contribution to T but creates a divergence in the usual L formula.

Negative values and an all-zero total are outside the simple T formula's valid setup. Arbitrarily shifting all observations to make them positive changes the distributional question and should not be presented as a neutral fix.

Business use

The main advantage is a consistent inequality summary with a useful additive decomposition. Pair it with averages, totals, group sizes, and distribution plots so users can understand the scale and shape hidden by one number.

For comparisons over time, keep observation definitions and weights consistent. Examine whether changes arise from the underlying distribution or from population and group reclassification. If the metric informs an allocation decision, evaluate the decision's objectives directly rather than treating minimization of the index as an automatic policy prescription.

Check your understanding

Why does regrouping [1,1,3,3] change the between-group contribution but not the overall Theil T?

Answer: The underlying four values stay the same, so their overall distribution is unchanged. Grouping changes which differences are counted as between groups and which as within groups.

Related terms

Entropy
inequality measure
mean log deviation
generalised entropy index
Gini coefficient
resource share
weighted mean
subgroup decomposition

Learn next

Calculate T for equal allocations, a concentrated allocation, and the worked four-unit example. Then repeat the decomposition under different groupings. Study population weighting and Theil L before comparing published indices, and keep descriptive inequality, absolute wellbeing, and causal interpretation separate.

Sources

Primary source: PySAL inequality documentation, “Demonstrating the Theil Index of Inequality”, https://pysal.org/inequality/user-guide/measure/theil.html . The T formula and group-decomposition equations were read. The four-unit examples and regrouping check are independently derived, not copied empirical results. Scope is chiefly Theil T, with L distinguished explicitly.

Previous: The Bitter Lesson Next: Thematic grid