Receipt OCR with a vision LLM cost 6x the estimate: full-resolution photos

We launched receipt extraction in our expense app: users snap a photo, a vision-capable model returns merchant, date, total, tax and currency as JSON. The finance estimate assumed roughly 1,000 input tokens per receipt. After the first month, the provider invoice for the feature was six times that estimate, and support mentioned that extraction "spins for ages" on some phones.

The usage logs were clear once we looked:

receipt_id=r_8812  image=4032x3024  input_tokens=1621  latency_ms=8930
receipt_id=r_8813  image=4000x3000  input_tokens=1598  latency_ms=9410
receipt_id=r_8814  image=1080x1440  input_tokens=  877  latency_ms=2210

Most images were full-size 12-megapixel photos. Upload time from the phone to our API plus our API to the provider dominated latency, and token counts were higher than the small test images we had used for the estimate. On top of that, a retry path re-sent the image on any schema validation failure.

Reproducing it

We took 200 real receipts and sent each one at original size and at several resized versions, logging tokens, latency and extraction accuracy against hand-labelled values.

What we ruled out

• Prompt length. The text prompt was about 350 tokens and constant.
• Output size. Responses averaged 120 tokens.
• A pricing change. Unit prices had not changed during the month.

Root cause

We sent images exactly as the phone captured them. Providers resize large images internally, but the cost and latency are driven by the image size they process, and the bytes still have to travel. The estimate had been made with screenshots of receipts at around 800 pixels wide. Real usage meant multi-megabyte uploads, more image tokens per request, and slower requests, with occasional re-sends doubling the cost of failing receipts.

What is happening under the hood

Vision models convert an image into a number of tokens that depends on its dimensions. The formulas differ by provider, but the shape is the same: more pixels, more tokens, up to a cap after internal downscaling.

• OpenAI models accept a detail setting. low processes a small fixed-size version for a flat token cost. high scales the image to fit within limits and counts tiles of 512 by 512 pixels, each adding a fixed number of tokens.
• Anthropic documents an approximate cost of width x height / 750 tokens, and resizes images whose long edge exceeds a documented limit, so sending larger images only adds upload time and latency.

In both cases, a receipt does not need 12 megapixels. Text on a receipt is readable at around 1,000 to 1,500 pixels on the long edge, as long as the receipt fills the frame.

Environment

Minimal reproduction

with open("receipt.jpg", "rb") as f:                       # 4032x3024, 3.8 MB
    b64 = base64.b64encode(f.read()).decode()
resp = client.chat.completions.create(model=MODEL, messages=[{
    "role": "user",
    "content": [
        {"type": "text", "text": PROMPT},
        {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}},
    ]}])
print(resp.usage.prompt_tokens)

The fix

1. Resize and compress before sending. We tested long-edge sizes from 768 to 2048 pixels; accuracy plateaued at 1,280.

from io import BytesIO
from PIL import Image, ImageOps

MAX_EDGE = 1280

def prepare(image_bytes: bytes) -> bytes:
    img = Image.open(BytesIO(image_bytes))
    img = ImageOps.exif_transpose(img)                 # respect phone rotation
    img = img.convert("L")                             # grayscale is enough for receipts
    img.thumbnail((MAX_EDGE, MAX_EDGE), Image.Resampling.LANCZOS)
    out = BytesIO()
    img.save(out, format="JPEG", quality=82, optimize=True)
    return out.getvalue()

exif_transpose matters: phones store rotation in EXIF, and an image sent sideways costs the same and reads worse.

2. Crop to the receipt. The mobile apps now use the platform document scanner (VisionKit on iOS, ML Kit document scanner on Android), which detects edges and crops. A receipt that fills the frame needs fewer pixels for the same legibility.

3. Resize on the device. The app uploads the prepared image instead of the original, cutting upload time on mobile networks. The server still runs prepare in case an old client sends a large file.

4. No blind re-sends. If the JSON fails validation, we retry once with a text-only follow-up that includes the validation error, rather than re-sending the image.

Alternatives and trade-offs

• detail: low or the smallest setting. Cheapest, but small text on long receipts became unreadable in our tests; accuracy dropped 9 points.
• Classic OCR plus a text model. Cheaper per receipt and very fast; struggled with crumpled and handwritten receipts. We use it as a first pass for clean receipts in a later iteration.
• Batch processing at lower prices for receipts that do not need instant results.

How to test the fix

The 200-receipt evaluation:

size          avg_input_tokens  p95_latency  field_accuracy
original            1,604          9.1 s         0.941
2048 long edge      1,280          5.2 s         0.943
1280 long edge        790          2.9 s         0.944
1024 long edge        560          2.6 s         0.927

A unit test checks that prepare never returns an image with a long edge above 1,280 and that EXIF rotation is applied.

Confirming in production

• Average input tokens per receipt dropped from about 1,600 to about 760 including the prompt, and the feature's monthly cost fell by 78 percent once the re-send fix was included.
• p95 end-to-end latency went from 9.1 to 3.2 seconds.
• Extraction accuracy on the weekly human-checked sample stayed at 94 percent.

Regression guard

• Per-feature cost per request dashboard with an alert at 1.5 times the expected value.
• The API logs incoming image dimensions; an alert fires if more than 5 percent of uploads exceed 2,048 pixels, which would mean a client regression.

Security implications

Original photos often include more than the receipt: faces, ID cards on the table, other documents. Cropping and downscaling sends less personal data to a third party, which is a privacy improvement. We also stopped storing original photos after extraction; the prepared image is kept for 90 days for disputes.

Prevention checklist

• Estimate costs with real user inputs, not clean samples.
• Resize, rotate and compress images before sending them to a model.
• Crop to the region of interest on the device when possible.
• Do not re-send images on retries unless necessary.
• Track tokens and cost per request per feature from day one.

References

• OpenAI vision guide (image inputs and detail) — https://platform.openai.com/docs/guides/images-vision
• Anthropic vision documentation (image size and cost) — https://docs.anthropic.com/en/docs/build-with-claude/vision
• Pillow ImageOps.exif_transpose — https://pillow.readthedocs.io/en/stable/reference/ImageOps.html