Pensum
How accurate is AI calorie counting?
I weighed real plates on a kitchen scale to see how far AI photo estimates land from the true grams, and what brings them closer.
An AI calorie counter does two jobs with one photo. It works out which foods are on the plate, and it works out how many grams of each there are. The first job is largely solved: a current vision model will look at a plate and say it holds fish, couscous and spinach, and it will usually be right. Nearly all of the error comes from the second job. Calories follow the grams, so a wrong portion makes a wrong calorie number before anything else can go wrong.
I tested how big that error is. I weighed every item on real plates with a kitchen scale, replayed the exact prompt the Pensum app sends, and scored a range of current vision models, from lightweight tiers up to premium frontier models, on how far their gram estimates landed from the scale. The method and every table are on the photo research page.
What a food photo can and cannot tell an AI
A photo is flat and taken from one angle. It shows the outline of each food well and its height badly, and height is where most of a heaped food's weight sits. Density does not show either, and oil or butter barely shows at all, even though both carry a lot of calories.
That is why, in my tests, the food mattered more than the model. With nothing to go on but the photo, the models read four foods like this:
- Folded turkey cold cuts, 65 g on the scale: read well, with most models within about 10 to 15%.
- A sourdough slice, 65 g: consistently under-read. The models assume bread is lighter than a dense slice really is.
- Heaped cottage cheese, 56 g: consistently over-read, often as 80 to 130 g.
- A lone sweet-potato wedge, 50 g: over-read by 50 to 200%. Small items alone on a plate read far bigger than they are.
On the cottage cheese, a model that confidently reports 130 g when the bowl holds 56 g produces a calorie error of roughly 2.3 times, on a food it recognised without trouble.
How accurate photo calorie counters are, in numbers
From a photo alone, the better models missed the grams by roughly 20 to 30% on average per plate. Across my full set of plates the average was about 29%. Those are plate averages, so a single item, like the wedge above, can land much further off.
In calories, a 20% miss on a 700 kcal dinner is 140 kcal. That is small next to a day of guessing and large next to a deficit of 300 to 500 kcal. Across a day of photo-logged meals, misses of that size can add up to a large part of the deficit you are aiming for.
A second kind of error has nothing to do with grams. A perfect weight matched to the wrong food entry still logs the wrong calories. Ask for couscous and the match might land on the dry entry instead of the cooked one, and dry couscous carries more than three times the calories per 100 g. This mistake is sometimes larger than the gram error, and you can only catch it if the app shows which food each line was matched to.
Why a bigger AI model does not fix it
Premium frontier models, including ones with extended thinking turned up, did not beat the cheap lightweight ones on gram accuracy. On the plates I published, the lighter models averaged around 20% error and the premium tier landed closer to 27 to 29%. The reasoning models drifted worse, talking themselves into confident volume math that was wrong.
The problem is the input. One flat photo from one angle does not carry the depth needed to recover volume, and no model can invent information that is missing from the pixels. I found the same pattern when I tested AI micronutrient estimates, where the larger model also did worse.
The one number that halves the error: total plate weight
The largest single improvement I measured came from telling the model the total weight of the food on the plate. With the total fixed, the model no longer has to guess the volume of everything. It only has to split a known total across the items it can see, and it is much better at that.
Typed context such as ingredients, plate size and cooking method gave a modest improvement. The weighed total brought the average gram error down to about 13 to 17%. Across that full set, pinning each estimate to the true total cut the average from about 29% to 17%, and on the hardest plate from about 50% to 25%. Re-run live on a fresh plate, the total weight took a single-photo estimate from roughly 27% down to about 13%, half the error.
A floor remains even then, around 17% on my plates. It comes from the split: small or lone items keep getting more than their share of the total. On a mixed plate that you cannot weigh item by item, the total is still a single weighing: the plate with the food on it, minus the empty plate.
When to weigh instead of taking a photo
If the food is in front of you and a scale is within reach, weigh it. Weighing and searching is faster and more accurate than any photo, and it gives you a number you can reuse next time. The photo tool is for the meals you cannot weigh, like a restaurant plate or a dish someone else cooked.
Those meals still belong in the diary, as estimates. Even a careful photo log carries error, and an adaptive TDEE absorbs the part that repeats. It is measured from what you logged and how your weight actually moved, so if your photo estimates run low week after week, the measured expenditure reflects that and your target follows it.
How Pensum's photo logging is built around these results
I built the photo tool around one position: a photo gives a starting estimate that you correct. You can add up to five photos of one meal and fill in what you already know: the ingredients, the cooking method, the plate or bowl size and, if you weighed it, the total weight. Any weight you type stays exactly as typed, and the rest of the plate is estimated around it.
What comes back is a list of ingredients with grams and macros. Each line is matched to a food in Pensum's catalogue, so the nutrition per gram comes from the database whenever there is a match, and you can swap the match if it is wrong. Cooking oil, butter and added sugar are added only when the cooking method points to them and the photo shows evidence, and those lines carry a mark so you know they were inferred. Every photo entry is saved as estimated, so a number read from a picture never sits in the diary looking like one you weighed.
Photo logging is free in Pensum. Cal AI keeps its photo scan behind a subscription. Its own website calls the scan about 80% accurate, which fits the 20 to 30% gram error I measured from photos alone. Photo logging in Pensum walks through the feature step by step, and the photo guide covers what to tell the app for a closer estimate.