Forecasting Impact conversation with Nicolas Vandeput
- FutureUP
- 24 hours ago
- 1 min read
When the Evaluation Framework Matters More Than the Model
In the latest episode of the Forecasting Impact podcast (#1 in Predictive Analytics 2025), Mariana Menchero and George Boretos spoke with Nicolas Vandeput, supply chain data scientist, author, educator, founder of SupChains, and organizer of the VN forecasting and inventory competitions.
The conversation covered machine learning, demand forecasting, inventory optimization, and some widely used forecasting practices Nicolas believes deserve to be challenged.
But one idea stood out in particular:
Before building a better model, build a better way of deciding whether the model actually adds value in practice.
A sophisticated model is not automatically an improvement. It needs to be tested against meaningful benchmarks, over the right forecasting horizons and aggregation levels, and with metrics that reflect the business decisions the forecast is ultimately intended to support.
Nicolas's competitions make this particularly tangible. In the VN2 Inventory Planning Competition, only a few participants beat the simple benchmark.
The lesson extends well beyond demand forecasting.
It strongly resonates with how we think about AI-driven pricing at FutureUP. No Predictive AI model should earn its place simply because it is more sophisticated. It needs to prove its value against meaningful real-world benchmarks and business outcomes that decision-makers can understand and trust.
For pricing, forecasting, or any other AI-driven decision, perhaps the first question shouldn't be:
“Which model should we use?”
But rather:
“How will we know if it actually adds value?”
🔗 Watch or listen to the full episode here:
AI-driven pricing insights to unlock hidden revenue & margin
Using your data. Minimal involvement. No implementation needed.
Comments