AI Glossary / Data science and analytics / Intermediate
Market basket analysis
Definition
Market basket analysis examines transaction data to identify items or item groups that occur together. A common approach discovers frequent itemsets and association rules, such as baskets containing item A also tending to contain item B. These patterns describe observed co-occurrence. They can suggest merchandising or recommendation ideas, but they do not establish that purchasing one item causes another purchase.
What it means
A basket is the unit of analysis: often one transaction, but sometimes another clearly defined group of events. The same dataset can produce different patterns if a basket means a checkout, a customer's week of purchases, or an entire account history. Analysts therefore define the basket before interpreting rules. In a simple presence-based analysis, buying three units of one item still records that item's presence once in that basket; quantity-sensitive analyses require additional choices.
How it works
The process begins by identifying transactions and their items, resolving duplicate or inconsistent item codes, and choosing an analysis period. An algorithm finds combinations meeting a frequency threshold. Association rules divide combinations into an antecedent and a consequent. Support is the fraction of baskets containing the combined items. Confidence is the fraction of antecedent-containing baskets that also contain the consequent. Lift compares that confidence with the consequent's overall frequency. Algorithms such as Apriori and FP-growth help search large collections, but their output still requires interpretation and validation.
Three practical examples
- Retail planning: a shop examines items commonly purchased together to propose display arrangements, then evaluates whether a change improves relevant outcomes.
- Online recommendations: an application uses co-purchase patterns as one signal for suggesting complementary items, alongside availability and other constraints.
- Service analysis: an organization treats service combinations as baskets to understand common bundles, while keeping that unit distinct from individual customer needs.
Think of it this way
Think of counting which ingredients repeatedly appear together on shopping lists. The counts reveal combinations worth examining. They do not explain whether the combination reflects a recipe, a promotion, a shared household preference, or coincidence. The pattern is a starting point for inquiry, not a causal explanation.
Worked example
Suppose 500 baskets include 150 containing tea, 100 containing biscuits, and 60 containing both. The tea-and-biscuits itemset has support 60 divided by 500, or 12%. The rule tea implies biscuits has confidence 60 divided by 150, or 40%. Since biscuits occur in 20% of all baskets, lift is 40% divided by 20%, or 2. Under an independence comparison using these marginal frequencies, the expected joint share would be 30% times 20%, or 6%, rather than the observed 12%.
Reversing the rule gives biscuits implies tea a confidence of 60 divided by 100, or 60%. Its lift remains 2 because tea's baseline frequency is 30%. Thus confidence depends on direction while this pair's lift does not. None of these calculations shows that a tea promotion would double biscuit sales. The observed baskets could reflect an existing bundle, customer preferences, or another influence. A proposed intervention needs its own evaluation.
Illustrative case study
In a fictional grocery analysis, a team discovers a strong association between two products and proposes displaying them together. Before acting, it notices that a holiday package included both products during the analysis period. The team separates package transactions from ordinary purchases and checks later periods. It then treats a new display arrangement as a hypothesis to test. The case illustrates why an association's commercial meaning depends on how the transactions were produced.
Common misconceptions and mistakes
High confidence can result simply because the consequent is common. High lift based on very few baskets can be unstable. Low support does not automatically make a rule useless, but it limits the evidence available. Searching thousands of combinations creates opportunities for chance discoveries. A rule's arrow does not necessarily describe purchase order, and it does not establish causation. Combining transactions across a long period can make unrelated purchases appear as one basket. Returns, substitutions, stockouts, and promotions can also change the interpretation.
Business use
Start with a business question such as improving relevant recommendations or understanding purchase combinations. Report counts alongside support, confidence, and lift. Examine whether patterns persist in later data and across meaningful contexts. Consider availability, margins, customer usefulness, and the cost of acting. Avoid presenting a rare rule as a universal preference. If a recommendation or promotion is deployed, measure its incremental effects through a suitable evaluation rather than assuming historical co-occurrence will translate directly into additional purchases.
People also ask
Does market basket analysis require supermarket data?
No. It can analyze other clearly defined sets of items or events.
Does a rule predict what happens next?
Ordinary basket rules describe co-occurrence within a basket; sequential analysis addresses ordering explicitly.
Is the highest-lift rule always best?
No. Evidence volume, stability, relevance, and the consequences of acting also matter.
Check your understanding
Why does tea implies biscuits have 40% confidence while biscuits implies tea has 60% confidence in the example?
Answer: Both use the same 60 joint baskets, but their denominators differ: 150 tea baskets versus 100 biscuit baskets.
Related terms
- Lift
- Apriori Algorithm
- Sequential pattern mining
- Association rule
- frequent itemset
- support
- confidence
- FP-growth
- recommendation system
Learn next
Create a small transaction table and calculate support, confidence, and lift by hand. Reverse each rule to see what changes. Then investigate frequent-itemset algorithms and design a later-period validation before proposing an operational intervention.
More in Data science and analytics
- Logarithm - A logarithm gives the exponent needed to raise a specified base to a positive number. If b raised to y equals x, then log base b of x equals y.
- Long Short-Term Memory - Long Short-Term Memory, usually abbreviated LSTM, is a type of recurrent neural network designed to learn relationships across sequences.
- Loops - Loops are programming structures that repeat a block of instructions. A loop may visit each item in a collection or continue while a condition remains true.
- Mobile business intelligence - Mobile business intelligence is the delivery and use of business data, reports, and analytical insights through mobile devices such as smartphones and tablets.
- Model-based clustering - Model-based clustering groups observations by fitting an explicit statistical model for how the data could have been generated.
- Monte Carlo simulation - Monte Carlo simulation uses repeated random sampling to explore the behavior of a model or estimate a quantity that is difficult to calculate directly.
Sources
MIT OpenCourseWare, original association-rule lecture notes: https://www.ocw.mit.edu/courses/15-062-data-mining-spring-2003/d286f7eda9814013279284c97f2f2a53_Lecture_16.pdf IBM, association-rule lift definition: https://www.ibm.com/docs/en/db2/11.1.0?topic=SSEPGG_11.1.0%2Fcom.ibm.im.model.doc%2Fc_lift_in_an_association_rule.htm
Previous: Markerless motion capture Next: Markov assumption