Why every PM needs to understand AI pipelines, not just prompts

There's a dangerous myth in product management right now: that understanding AI means knowing how to write good prompts. It doesn't. Prompting is to AI what writing SQL queries is to data engineering — a useful skill, but nowhere near the full picture.

When I built CartCoach AI, I learned this the hard way. The app scans grocery products and tells you whether they're good for your fitness goal — lose fat, gain muscle, go anti-inflammatory, whatever. Claude API powers the recommendations. But here's the thing: the prompt that generates the recommendation is maybe 15 lines of code. The system around it? That's 95% of the work.

The prompt is the tip of the iceberg

When a user scans a barcode in CartCoach, here's what actually happens before Claude ever sees a prompt:

1. The barcode hits our API, which queries OpenFoodFacts for nutrition data
2. If that fails, we fall back to a curated demo product database
3. The product gets run through a category-aware health scoring algorithm (0-100) that weights protein density, sugar, fibre, processing level, and fat quality differently based on what kind of food it is
4. We pull the user's current fitness goal and macro targets (calculated from their biometrics using the Mifflin-St Jeor equation)
5. We pull their current cart contents so Claude has context on what they've already picked up
6. Only then does the prompt get assembled and sent to Claude
7. Claude's response gets parsed for structured output — the recommendation message and a swap suggestion
8. If Claude fails (rate limit, timeout, no API key), the rule-based fallback engine kicks in and generates a recommendation using pattern matching

That's 8 steps. The prompt is step 6. A product manager who only understands prompting would have built steps 6 and 7 and called it a day. They would have missed the health scoring, the cart context, the graceful degradation, and the fallback engine — which, by the way, delivers about 80% of the AI's value with zero external dependencies.

Graceful degradation is a product decision, not a technical one

The single most important architectural decision in CartCoach wasn't which model to use. It was deciding that the app must work fully without an API key.

This sounds like an engineering constraint. It's not. It's a product strategy decision. If your AI product breaks when the API is down, you don't have a product — you have a wrapper. Users don't care why something failed. They care that it did.

So we built a rule-based recommendation engine that runs entirely locally. It detects high sugar, low protein, excessive processing, and poor fat quality. It has pre-written swap suggestions for common products. It's not as good as Claude, but it's always available, always fast, and always free.

Context injection is where the real intelligence lives

Here's what most people get wrong about AI recommendations: they treat every request as independent. User scans a product, AI evaluates it in isolation, done.

But recommendations aren't independent. If you've already got chicken breast and brown rice in your cart, and you scan a protein shake, the recommendation should be different than if your cart was empty. The protein shake might be great for someone starting their shop, but redundant for someone who's already hit their protein target.

This is context injection — feeding the model not just the product data, but the user's goal, their biometrics-derived targets, and their current cart state. It's the difference between a generic nutrition label reader and a personalised shopping assistant.