Pensum
Can AI estimate food weight from a photo?
We weighed real meals, then measured how close the models' gram estimates came.
A meal-photo tracker rests on one number: how many grams of each food are on the plate. Get the grams wrong and the calories are wrong before you touch anything else.
The sample is small and the numbers move as we add plates.
Why the grams are the hard part
Identifying the food in a photo is largely a solved problem. Modern vision models will tell you a plate holds fish, couscous and spinach, and they are usually right. The value, and nearly all of the error, comes one step later: turning that plate into portions. A model that confidently says "130 g of cottage cheese" when it is really 56 g has produced a roughly 2.3x calorie error.
How we measured it
- Real ground truth. Each item on each plate was weighed on a kitchen scale before eating. That weight is the answer key.
- The real engine. We replay the exact prompt the shipping app uses, so the results reflect what the product would actually return.
- Several models. Each plate is run through a range of current vision models, from lightweight "flash" tiers up to premium frontier models.
- Different amounts of context. Photo alone; photo plus the kind of notes a user would type (ingredients, plate size, cooking method); and photo plus a weighed total.
- One metric. For each item, the percentage gap between the estimate and the true weight. We average the absolute gaps across a plate. Lower is better.
Finding 1: the food sets the difficulty
The shape of the food sets the error more than the choice of model does. Flat, legible foods are read well by almost everything. Heaped, soft, or small lone items get over-estimated by almost everything, because a single flat photo does not capture height, which is where most of a heaped food's weight sits.
| Food (weighed truth) | What the models did |
|---|---|
| Folded turkey cold cuts, 65 g | Read well. Most models within about 10 to 15% with no help at all. |
| Sourdough slice, 65 g | Consistently under-read. Models assume bread is lighter than a dense slice really is. |
| Heaped cottage cheese, 56 g | Consistently over-read, often 80 to 130 g. Up to roughly 2.3x the real weight. |
| Lone sweet-potato wedge, 50 g | Over-read by 50 to 200%. Small items alone on a plate read far bigger than they are. |
A heaped soft food and a thin folded one are not the same task.
Finding 2: a bigger, smarter model does not fix it
Premium frontier models, including ones with extended "thinking" turned up, did not beat the cheap lightweight ones on gram accuracy. If anything the reasoning models drifted worse, talking themselves into confident but wrong volume math. On the plates we share below, the lighter models averaged around 20% error and the premium tier landed closer to 27 to 29%.
The big models are not bad. One flat photo from one angle does not carry the depth needed to recover volume, and no model can invent information that is not in the pixels.
Finding 3: the biggest win is a number you already have
The largest single improvement we measured came from telling the engine the total weight of the food, so it only has to guess how that total splits across the items, which it is much better at.
| What the model was given | Average gram error |
|---|---|
| Photo alone (better models) | ~20 to 30% |
| Photo plus typed context (ingredients, plate size, method) | a modest improvement |
| Photo plus the weighed total | ~13 to 17% |
Across our cached set, pinning each estimate to the true total cut the average error from about 29% to 17%, and on the hardest plate from about 50% to 25%. Re-run live on a fresh plate, giving the model the total weight took a single-photo estimate from roughly 27% down to about 13%, cutting it in half. The biggest accuracy win we found is a kitchen scale.
Finding 4: the floor, and a second kind of error
Even with the total handed to it, a floor remains, around 17% on our plates. It is a segmentation problem: small or lone items keep getting more than their share of the total.
There is also a second error that has nothing to do with grams. A perfect weight matched to the wrong database entry still mislogs. Ask for "couscous" and you might land on the dry entry instead of the cooked one, and dry couscous carries more than three times the calories per 100 g. That is a real and sometimes larger source of error.
What this means for how Pensum's photo tool works
We built the tool around one position: photo logging is a starting estimate you correct.
- You lead with what you know. Any weight you type is held fixed and never re-estimated. A total weight, when you have it, is the strongest input you can give.
- It returns an editable breakdown. A list of ingredients and grams you can adjust, because the raw estimate will be off on exactly the foods this research flags.
- Photo entries are flagged as estimated. You always know which numbers were weighed and which were read from a picture.
If the food is in front of you and a scale is within reach, weigh it. The photo tool is for when weighing is not an option.
Caveats
This is a living benchmark on a small number of labelled plates. Absolute numbers will shift as we add cases, and run-to-run noise is real, so the directions here are more reliable than any single figure.
This is one of three pages where we show the arithmetic instead of describing it. The others cover how Pensum measures your energy expenditure and whether AI can fill in a food's missing micronutrients.
Pensum is a fast, private macro tracker built on national food databases (USDA plus German and Swiss sources), with photo logging designed around exactly this research. Get Pensum on Google Play, or read the photo guide for how to get a closer estimate. Back to the homepage.