Barcode Benchmark

Retail barcode benchmark

How reliably do barcode scanners read real product photos once they are rotated, blurred, downscaled, badly lit or crowded together? Every score links to the images and the saved reads behind it.

—Open Food Facts photos
—test images
9challenges
7configurations compared

Overview

Share of images where the scanner returned the expected barcode value (recall), with median scan time. The best recall per challenge is marked. Collages count recovered values out of 48. Perspective, blur and low resolution use different photo subsets, so compare scanners within a row, not difficulty across rows. A dash means the scanner has not been run yet.

Measurement setup

Full methodologyZXing raw readsApple Vision raw reads (blur)Apple Vision raw reads (collages)

Results by challenge

Each photo gets one lighting effect and one of perspective, blur or downsampling. Assignment follows a fixed order and never depends on scanner results. Click a scanner name to inspect its saved reads, or a count to see the affected images.

Explore and download

Browse all photosDownload the test set

Method

Photos and labels

Product photos from Open Food Facts. Every scanner is asked for all four retail formats (EAN-13, EAN-8, UPC-A, UPC-E).

Labels were checked individually where scanners disagreed and by visual review otherwise; barcode-free photos were confirmed on the full image. Labels name the expected values; they do not outline every physical barcode in a photo.

Controlled variations

Every transformation starts from the original photo, turned upright; natural camera tilt remains. Collages keep barcodes at native resolution, and the enlarged inputs test processing cost and size limits rather than added detail.

Reproducible inputs

Only the original photos are hosted. Every test image is rendered from them on the fly by a deterministic generator that produces byte-identical pixels in every browser and in Node, and each render is checked against a SHA-256 fingerprint before scanning. Anyone can rebuild the full set with the download instructions.

Readable is not guaranteed

Blur and resolution strengths are fixed settings, not proven decoding limits, and no scanner was used to pick them. Transformed images inherit the labels of their source photo, so some may no longer be readable at all.

Scoring

Images derived from the same photo are correlated, so there is no combined winner score. Errors stay visible and count as misses; failed calls do not contribute to timing. Ten barcode-free photos are too few to estimate a general false-positive rate.

Wrong predictions and duplicates

Wrong predictions are values absent from the reviewed labels. Duplicates are reads beyond the annotated count of the same normalized value within one scan (three reads with two labeled copies = one duplicate); equivalent UPC-E expansions count as the same value. Without instance boundaries this is a count-based allowance, not verified spatial matching. A repeated wrong value counts in both columns.

Counts use the first measured scan per image. In the detailed tables, click a scanner name to inspect all its saved reads and locations, or a count to see the affected images.