Let’s be honest: in the FMCG (Fast-Moving Consumer Goods) space, trade marketing budgets are almost always misallocated. Companies tend to fund invoice history rather than actual potential.
For the DataStorm 7.0 competition, our team, Stack Kings, decided to stop guessing and start modeling latent demand. We built an analytics pipeline and a Next.js field app to optimize a LKR 5M trade spend across 20,000 retail outlets in Sri Lanka. Here is the unvarnished breakdown of how we achieved a +253% lift over a naive budget allocation.
The "Empty Shelf" Problem (Right-Censoring)
The core data science issue here is a concept called right-censoring. The monthly sales volumes we observed in the 2.3M transaction records were just a lower bound. If a small shop shows low sales, it might just be under-stocked or credit-constrained, not lacking in customer demand.
Because of this, standard averages systematically underestimate a shop's true potential. Instead of using standard textbook models like Tobit—which require strict indicators of exactly when a shop ran out of stock (which we didn't have)—we built an ensemble model.
We took a two-pronged approach:
Lookalike Clustering: We clustered similar outlets to see what the top performers in that specific group were achieving, assuming that less-constrained shops reveal the true ceiling.
Upper-Tail Regression: We used a specific type of regression (Quantile Regression) designed to estimate the maximum possible demand based on a shop's features, rather than the average.
Taking the maximum of these two estimates ensured we weren't artificially pulling down a shop's potential.
Mapping the Real World (Spatial Features)
To give the models a real signal, we engineered spatial features using OpenStreetMap Points of Interest (POIs).
Rather than using arbitrary "flat disk" counts—like just counting how many shops are within a 3km radius—we implemented a distance-decay model. In plain English: a bus stop 200 meters away matters a lot more to foot traffic than one 2 kilometers away.
We grouped locations into tiers. Transport and food places have a "fast" drop-off in influence, meaning you need to be right next to them to get the benefit. Meanwhile, schools and temples cast a wider, "slower" net of influence over the entire neighborhood.
Squeezing Every Drop of Budget (The Optimizer)
Ranking outlets by potential isn't enough; you need to maximize the incremental liters gained per rupee spent.
We modeled the volume response to trade spend as a curve with diminishing returns. Simply put, the more you spend on a single shop, the less extra volume you get for your next rupee.
To solve this efficiently across 9,000 Western Province outlets, we broke that curve into straight-line segments. This allowed us to use a Linear Programming solver to allocate the LKR 5M budget mathematically perfectly. The result? A massive 253% lift compared to the standard gut-feel approach of just splitting the money evenly among top shops.
Building Trust with the Field App
A complex optimizer is useless if field sales reps don't trust the numbers. They need to know why a shop is getting a certain budget.
We built a live Outlet Intelligence Web App using Next.js and Postgres. To explain the outputs, we added an Explainable AI (XAI) layer. It uses a local Ollama process (gemma3:1b) running in the browser to generate a structured SWOT summary. If the local AI is offline, it falls back to a Gemini cloud API or a safe deterministic template.
Crucially, the AI doesn't make the predictions. It simply translates our hard, pre-computed data into plain business language so the sales reps can actually use it on the ground.
The Dirty Reality of Data
Data engineering isn't glamorous. We built a strict Medallion architecture. We didn't silently drop messy data. We explicitly quarantined 37,205 records using failure reason codes. We even retained 7,417 "blackout" outlets (shops with zero December transactions), treating them correctly as supply signals rather than zero demand.
Check out the live production app at stackkings.inusha.me
We don't fund history. We fund potential.
