What does an HSMM add to an HMM?

It permits explicit, more flexible duration modeling for hidden states or segments.

Are observations always independent within a segment?

Not in every variant. Conditional independence is a common simplifying assumption that must be checked for the selected model.

Can two duration models have the same mean but behave differently?

Yes. Their probabilities of short, typical and unusually long durations can differ substantially.

AI Glossary / Probabilistic machine learning / Intermediate

Hidden semi-Markov model

Definition

A hidden semi-Markov model, or HSMM, is a probabilistic model for sequences in which unobserved states can have explicitly modeled durations. It extends a hidden Markov model by allowing more flexible state dwell times than the geometric durations implied by ordinary time-homogeneous HMM self-transitions. Observations provide indirect evidence about the current state and its boundaries.

What it means

Many processes involve phases that persist for characteristic lengths of time: an operating mode, an activity or a sound segment. An ordinary HMM can keep a state active through repeated self-transitions, but its probability of leaving that state does not depend on how long it has already persisted.

An HSMM can give duration a separate distribution. A phase might rarely last one interval, commonly last three and almost never last twenty. Hidden means that the phase labels and often their boundaries are not directly observed. Semi-Markov refers to the timing structure of state changes; it does not mean that the model is only partially probabilistic.

How it works

In a common explicit-duration formulation, the model selects a state for a segment, draws a positive duration from that state's duration distribution and generates observations for the segment. At the boundary, it chooses the next state and duration. Segment-level self-transitions are typically excluded so that adjacent segments do not ambiguously represent the same uninterrupted state.

The model specifies initial-state probabilities, transition probabilities, duration distributions and observation distributions. A simple version treats observations as conditionally independent given their state; richer segment models can represent additional within-segment structure. These choices should be stated rather than assumed to be universal HSMM properties.

Inference considers possible state identities and segment lengths. Dynamic-programming procedures can compute probabilities or find a likely segmentation, while fitting estimates the relevant parameters. Duration truncation can make computation manageable but must be appropriate for the process. The beginning and end of an observation window need careful treatment because a segment may be only partially observed.

Three practical examples

1. Equipment monitoring: infer operating phases from noisy sensor readings while representing typical phase durations. Distinguish an unusually long phase from a single unusual measurement.

2. Activity recognition: identify periods of walking, resting or another activity from sensor sequences. The duration model can complement observations, but should not force every person's behavior into the same rigid timing pattern.

3. Speech or audio segmentation: model sound units whose lengths vary according to an explicit distribution. Evaluate boundary accuracy and duration assumptions alongside recognition quality.

Think of it this way

Think of a timetable that describes both which phase can follow another and how long each phase tends to last. An observer sees noisy clues and infers which timetable phase is active. The analogy is limited because an HSMM uses distributions rather than fixed appointments, and its inferred phases need not correspond to directly observable categories.

Worked example

Suppose an operating phase has duration D, measured in whole intervals, with probabilities P(D=1)=0.1, P(D=2)=0.2, P(D=3)=0.5 and P(D=4)=0.2. These sum to one. Its expected duration is 1 × 0.1 + 2 × 0.2 + 3 × 0.5 + 4 × 0.2 = 2.8 intervals.

Conditional on lasting at least two intervals, the probability that it ends exactly at interval two is 0.2/(0.2+0.5+0.2)=2/9, approximately 0.2222. Conditional on lasting at least three, its probability of ending at three is 0.5/(0.5+0.2)=5/7, approximately 0.7143. The chance of ending depends on elapsed duration.

An ordinary HMM state with the same mean dwell time would have constant exit probability 1/2.8, approximately 0.3571, and self-transition probability approximately 0.6429. It would not reproduce these two duration-dependent exit probabilities. Matching average duration is therefore insufficient to match duration shape. This is a calculation for one state's stipulated duration distribution, not a complete hidden-state inference example or a fitted machinery model.

Illustrative case study

Fictional case: Pine Process Lab segments sensor records into operating phases. A basic HMM often inserts brief phases between similar observations. The team investigates whether measured phase durations support an explicit-duration model rather than simply penalizing every transition.

Reviewers also notice that recordings sometimes end while a phase is still active. They distinguish such censored segments from completed durations when specifying fitting and evaluation. This fictional case illustrates modeling and boundary checks, not a guarantee that HSMMs eliminate spurious transitions or improve every segmentation task.

Common misconceptions and mistakes

An HSMM does not require a fixed duration for each state. A duration distribution can be narrow, broad or multimodal if the chosen family allows it. Conversely, using an arbitrary duration table without enough data can overfit.

The usual geometric-duration limitation refers to a single state in a standard time-homogeneous HMM. Expanded-state constructions can represent more complex aggregate durations, so it is misleading to claim that no HMM formulation can encode them. Segment labels can also be hard to identify when states have very similar observation and duration distributions. A most likely segmentation should not be mistaken for certainty about every boundary.

Business use

Use an HSMM when the time spent in a state carries information important to the decision. Compare duration distributions from credible observations with the assumptions of candidate models. Record the sampling interval: a duration of three frames has a different operational meaning at different recording rates.

Validate on complete sequences and examine timing errors, state confusion and uncertainty around transitions. Choose maximum duration limits transparently and assess what happens to longer segments. If operational actions depend on detecting a phase change, measure detection delay and false alarms rather than relying only on an overall likelihood score.

Check your understanding

Why does a geometric duration with mean 2.8 fail to match the example's timing behavior?

Answer: Its exit probability stays at approximately 0.3571, whereas the example's conditional exit probabilities change with elapsed duration.

Related terms

Hidden Markov model
Segmentation
Forward-backward algorithm
Censoring
dwell time
duration distribution
semi-Markov process
Viterbi decoding

Learn next

Compare geometric and explicit duration distributions with the same mean. Then trace a small segment-level forward calculation and study how an observation window ending mid-segment affects inference.

Sources

Yu, Hidden Semi-Markov Models, model definitions and duration discussion: https://www.cs.ubc.ca/~murphyk/Teaching/CS540-Spring10/projects/Yu-hsmm09.pdf

Independent review consulted Yu’s explicit-duration formulation, zero segment self-transition and HMM relationship. Original mean and duration-dependent exit probabilities verified; case fictional.

Previous: Hidden representation Next: Hidden units