AI Glossary / Computer vision and multimodal AI / Advanced
Shi - Tomasi (good features to track)
Definition
The Shi - Tomasi feature-selection criterion scores a local image patch using the smaller eigenvalue of its gradient second-moment matrix. A sufficiently large minimum eigenvalue indicates intensity variation in two directions, making local displacement better constrained than in a flat patch or along a single edge. It is commonly used to select candidate points for tracking, rather than to guarantee that those points remain trackable indefinitely.
What it means
Some image locations provide much clearer motion evidence than others. A uniform patch looks similar after small movements, while an edge can slide along its own direction with little appearance change. A corner-like patch changes more distinctly in multiple directions. Shi - Tomasi selection favors this stronger local evidence. The selected point need not be a literal corner of a physical object, and its usefulness still depends on later motion, visibility, lighting and the assumptions of the tracking method.
How it works
Compute local horizontal and vertical image gradients and accumulate their products in a neighborhood to form a symmetric 2×2 matrix. Its eigenvalues describe directional strength in the local displacement problem. The criterion uses min(λ₁,λ₂), retaining strong candidates under a threshold and often applying spatial suppression or minimum-distance rules. Window size, gradient calculation and score scaling affect the result. The original good-features-to-track work also considers feature monitoring; selecting a strong starting patch is only one part of maintaining a reliable track through changing images.
Three practical examples
- A video-analysis exercise selects candidate points before running a local optical-flow tracker.
- An inspection study compares feature locations on textured labels, smooth packaging and long straight edges.
- A teaching demonstration changes the neighborhood size to examine how a point's score depends on the surrounding image structure.
Think of it this way
Imagine locating yourself on a plain wall, beside one straight painted stripe or at the crossing of two stripes. The plain wall gives few clues, and the single stripe leaves uncertainty along its length. The crossing gives stronger clues in two directions. The analogy captures local motion distinguishability, but it does not guarantee that the mark remains visible or unique elsewhere in the scene. Those are additional challenges a tracker must address.
Worked example
In an original score illustration, suppose patch A has gradient-matrix eigenvalues 18 and 12, while patch B has eigenvalues 40 and 0.5. Their Shi - Tomasi scores are 12 and 0.5. Under a stipulated absolute threshold of 2, A passes and B fails despite B's larger maximum eigenvalue. B has strong evidence in one direction but weak evidence in the other. The numbers and threshold are chosen only for teaching; real implementations may use relative quality thresholds and different scaling. The calculation does not show that A will survive occlusion, large motion or repeated-pattern ambiguity.
Illustrative case study
A fictional parcel-tracking team detects many strong points on printed labels. During testing, several tracks fail when reflections obscure the print or when similar labels pass nearby. The team adds track-quality checks and evaluates how points are distributed across parcels rather than simply maximizing the initial count. It distinguishes successful local tracking from identifying a parcel over time. This scenario illustrates why a good feature score supports initialization but does not solve every association or visibility problem in an operational tracking system.
Common misconceptions and mistakes
The method does not select points solely by the largest eigenvalue. Requiring strength in the weaker direction is central to its criterion. Another mistake is equating a high score with semantic importance or permanent physical identity. Repeated textures can contain many strong but ambiguous candidates. Threshold values cannot be compared blindly across different gradient scales or window settings. Finally, detecting features is not the same as tracking them, and a successful short-term track does not automatically establish a stable correspondence under substantial appearance or viewpoint changes.
Business use
Shi - Tomasi selection can support motion estimation and feature-based imaging workflows when paired with appropriate tracking and validation. Evaluate the full process under expected lighting, motion and occlusion, including low-texture regions. Review feature coverage so a large count in one small area is not mistaken for good evidence across the scene. Record selection settings and monitor failed or inconsistent tracks. The practical objective is a useful set of observable local patterns for the downstream task, rather than the greatest possible number of corner responses.
People also ask
Must the selected point be a geometric object corner?
No. The criterion measures local image-gradient structure.
Does a strong edge always make a good candidate?
No. Its weaker directional eigenvalue can remain small.
Does goodFeaturesToTrack perform tracking itself?
Feature selection identifies starting candidates; tracking is a separate operation in typical workflows.
Check your understanding
Which patch passed the toy threshold?
Answer: Patch A.
Why did patch B fail despite its eigenvalue of 40?
Answer: Its smaller eigenvalue was only 0.5.
Why was passing not a guarantee of later tracking success?
Answer: The score described local initial evidence and did not account for every future visibility or appearance change.
Related terms
- Structure tensor
- Corner detection
- minimum eigenvalue
- Lucas–Kanade tracking
- aperture problem
- nonmaximum suppression
Learn next
Construct flat, edge-like and corner-like patches and compare their gradient-matrix eigenvalues. Apply an explicit score threshold, then test the selected points across a short image sequence. Record selection failures separately from later tracking failures to understand which stage needs improvement.
More in Computer vision and multimodal AI
- Shape from shading - Shape from shading estimates surface orientation or shape from brightness variations interpreted through a model of illumination and surface reflectance.
- Shape grammar - A shape grammar is a formal system of spatial rules that generates or describes designs by matching and transforming shapes.
- Sharpening - Image sharpening is processing that increases the apparent definition of edges or fine detail, often by strengthening selected local intensity differences.
- Shift-invariant - In signal and image processing, a shift-invariant system responds to a translated input by producing the same translation of its original output.
- Shift-Map editing - Shift-Map editing is an image-editing framework that assigns each output pixel a displacement identifying its source location in an input image.
- Shift theorem - The Fourier shift theorem describes how translating a signal changes its Fourier transform.
Sources
Shi and Tomasi, Good Features to Track: https://publications.ri.cmu.edu/good-features-to-track OpenCV, Shi–Tomasi Corner Detector and Good Features to Track: https://docs.opencv.org/4.0.1/d4/d8c/tutorial_py_shi_tomasi.html The two-patch eigenvalue calculation, stripe-crossing analogy and parcel-tracking case are original educational illustrations. The threshold is stipulated and is not a recommended universal library setting.
Previous: Shear matrix Next: Shift-invariant