Every calorie app that estimates from a photo makes mistakes, and most don’t say how big. We’d rather show our numbers, how we got them and where they fall short. This page describes the benchmark we ran on October 9, 2026, before choosing the AI model and prompts that Calorix 2.0 uses.
What we tested
We measured how close Calorix’s AI analysis comes to the truth in two situations: a photo of a plate, and a meal described in words, typed or spoken. We also checked the nutrition label reader, Fix with AI corrections and meal ideas.
- Photos: dishes from Nutrition5k, a public dataset from Google Research in which cafeteria plates were built ingredient by ingredient on a scale, so the true calories and macros are known. We used the overhead photo of each dish, filtered out dishes with label errors or food cut off by the frame, and balanced the set across simple and mixed plates and calorie ranges.
- Descriptions: 160 meals written the way people type or dictate, in 14 languages, from home cooking to restaurant dishes and packaged snacks, with and without amounts. True values were computed from USDA FoodData Central and, for French and North African dishes, ANSES-CIQUAL.
- Labels: 8 synthetic nutrition labels in US and EU formats.
No photos, descriptions or data from Calorix users were used.
How we kept it honest
Prompts were tuned on a validation set only. The results on this page come from held-out sets the AI had never been tuned on: 94 test dishes plus a second set of 58 dishes built later, disjoint from the first, for 152 photos, and 98 test descriptions. The main measure is the median absolute percentage error (MdAPE): half the estimates were closer than this, half further. We also report the mean error in kcal and the average bias, because a systematic under-count matters more than random error for anyone trying to lose weight.
Results
On photos, the median error was 30.5% (95% confidence interval 24.7 to 36.6%), the mean error 117 kcal per dish, and the average bias +2 kcal, so estimates weren’t systematically low. 34% of dishes were within 20% of the truth and 48% within 30%. Fat was under-estimated by about 6% on average, mostly on oily dishes. Small dishes were sometimes over-estimated by a large percentage, but by few calories: on dishes under 150 kcal, the mean error was about 50 kcal.
On typed and spoken descriptions, the median error was 11.5% (95% CI 8.2 to 13.9%), the mean error 78 kcal, and 71% of estimates were within 20%. Descriptions with amounts did best; vague ones were judged against USDA’s default portions, which are often smaller than restaurant servings.
Errors partly cancel over a day. In a simulation of days with three logged meals, a photo-only day missed the true total by a median of 18.6% (about 181 kcal on 1,000 kcal; 90th percentile 48.5%), a day with one photo and two descriptions by 11.8%, and a day of descriptions by 8.9%. All 8 test labels were read exactly, including European kilojoules and salt.
How this compares with published studies
Our numbers come from different meals and methods than independent studies, so they can’t be ranked against them. For context: in a preliminary 2026 NIH study, the photo features of four popular apps under-counted weighed meals by about a third on average (abstract), and in a 2025 study, general AI chatbots missed the calories in 52 food photos by about 36% (Fridolfsson et al.). Our photo error is in the same range as other AI systems; what we focused on is removing the systematic under-count and making each estimate easy to check and fix.
Limits
- Cafeteria photos from 2019. Nutrition5k plates come from one California cafeteria, shot from overhead at low resolution. Home and restaurant food photographed at an angle, in mixed light, may give different errors. Phone-like side photos gave similar error levels (33 to 43%) in a smaller test.
- The dataset is public, so AI models may have seen it during training, which would flatter every model alike.
- Sample sizes of 98 to 152 give confidence intervals of about ±5 to 6 points on the median error.
- Vague descriptions depend on standard portion sizes.
- Labels were synthetic and clean; real labels with glare and curves still need a field test.
- AI models change. We re-run this benchmark before changing the model or prompts, and will publish a new version rather than edit these numbers.
How to get closer than any benchmark
The benchmark measures the first estimate. You can do better: scan barcodes and labels for packaged food, add the cooking fat, correct the grams of the biggest items, or describe a meal with amounts. Calorix lists every ingredient with a confidence badge so you know which ones to check. More in how accurate are AI calorie counters?
Data and credits
- Photos and weighed reference values: Nutrition5k by Quin Thames, Arjun Karpur, Wade Norris, Fangting Xia, Liviu Panait, Tobias Weyand and Jack Sim (Google Research), released under the Creative Commons Attribution 4.0 license (CC BY 4.0). Described in “Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food”, CVPR 2021, pages 8903 to 8911. We selected and filtered a subset of its overhead dish photos as described above; no dataset images are reproduced on this page, and the figures here are our own results, not Google’s.
- Description reference values: USDA FoodData Central and ANSES-CIQUAL 2020.
- Benchmark design, code and results: Invikta Technology Ltd. The method, datasets (ids and ground truth) and scoring code are kept with the results so the benchmark can be re-run; we’ll publish a new dated version whenever the model or prompts change.
Updated October 23, 2026
