Portfolio project · Reinforcement learning · MLOps on Azure
Learning the right price while selling
A dynamic pricing engine that treats each price as an experiment. Thompson Sampling quotes a price per request, watches whether the customer buys, and shifts traffic toward the price that earns the most margin. Here it is tested on a simulation built from 1M real order lines.
The problem
A retailer usually sets one price per product and keeps it. That price is rarely the one that maximises margin: too high and buyers leave, too low and every sale earns less. The best price cannot be read off a spreadsheet, because the data only shows the prices that were charged, not what would have happened at the others. It has to be learned, and every request spent learning is a request that might have been priced better.
How it works
Real demand
UCI Online Retail II: 1M order lines from a UK retailer, where the same product sold at several prices over two years.
A curve per product
A price–response curve is fitted per product, giving how often a customer buys at each price.
Five candidate prices
Each product gets five prices across the band it really sold at. Each price is an arm of a multi-armed bandit.
Thompson Sampling
For every request it samples a belief about each price, picks the one with the highest expected margin, and updates on the outcome.
Learning curve
Share of the best possible margin earned so far, as requests come in. The best fixed price (the oracle) needs the true curve, which no real policy has. Thompson Sampling starts even below random pricing: with no data, its beliefs are the same for every price, so the dearest prices, with the widest margin, look best and get tried first. Then it learns; the alternatives never improve.
Table
Margin over 20,000 requests per product
Across the top 50 products, as a share of the margin the best fixed price would have earned. Every policy faced the same requests and the same purchase decisions, so the difference comes from the policy, not from luck.
Explore a product
Pick a product to see what each of its five prices earns, and where Thompson Sampling ended up sending the traffic.
Table
What these numbers are, and are not
A known world, not the retailer's books
The prices in the data were set by the retailer, not randomised, so the fitted elasticities are an upper bound. The simulation is a world where the true answer is known, which is what lets the policy be judged.
Two assumptions fill gaps in the data
The retailer already priced well
Within the band it really sold at, the retailer's usual price is the best of the five in most products. Thompson Sampling is not told that: it pays for exploring first, and earns its small lead by also finding the products where the usual price was not the best.
Why the cheapest price often wins here
Most of these products are very price-sensitive in the data (median elasticity around −2.7), so in this world a lower price sells enough more to pay for itself.
What would make it real
A randomised price test on live traffic. The serving API already logs every price with the probability it was chosen, which is exactly what that test needs.
How it runs in production
The same policy is served by an API on Azure Container Apps, scaled to zero when idle. Each product's beliefs live in one Cosmos DB document, and each purchase updates them with a single atomic write. Callers authenticate with an API key, the app reaches the database through its managed identity with no stored credentials, and GitHub Actions deploys through OIDC. On a live run, the server answered a price in under 25 ms at the 95th percentile, and the stack costs nothing while idle.