Who is this research for? Supply chain leaders, demand planning managers, operations executives, and professionals responsible for forecasting, analytics adoption, and AI implementation.
Top Answer
Research suggests that managers do not treat artificial intelligence forecasts the same as traditional model-based forecasts, even when their performance is identical. While performance remains the primary driver for decision-making, managers have heightened reactions to errors from AI systems, making larger adjustments when AI performs poorly. This indicates that perceptions of AI meaningfully influence forecasting behavior and human–algorithm collaboration.
Executive Summary
This study by Dr. Finnegan McKinley (Walton College PhD alumnus; Colorado State University), Dr. Rebekah Brau (Walton College PhD alumna; Brigham Young University), and Drs. John Aloysius and Adriana Rossiter Hofer (Dept. of Supply Chain Management, Sam M. Walton College of Business, University of Arkansas) examines how managers adjust forecasts when using artificial intelligence versus traditional model-based systems in demand planning.
Using two controlled laboratory experiments and a field study with approximately 575,000 observations from a multinational retailer as they transitioned from a traditional forecasting model to an AI-based forecasting system, the researchers analyzed how users responded to those different systems and levels of algorithm performance.
The findings suggest that algorithm performance strongly shapes behavior, but the label “AI” changes how that performance is interpreted. Managers consistently correct forecast errors, reducing overestimates and increasing underestimates, but they react more strongly when those errors come from AI systems. In addition, users reduce their adjustments as algorithms improve over time, indicating that they can detect and respond to performance gains.
Overall, the research highlights that AI adoption introduces behavioral changes, not just technical improvements, suggesting that demand planning processes may need to be redesigned to account for how managers perceive and interact with AI-based systems.
Expert Insights: What should demand planning leaders know about AI forecasting behavior?
Why do managers react more strongly to poor performance from AI-based forecasting systems compared to traditional models?
Dr. John Aloysius notes: “People expect more from AI because they know it to have access to extensive data sources, to use complex algorithms, and also think that it is imbued with the ability to make decisions to achieve goals rather than just follow instructions. With these increased expectations however comes greater disappointment if the AI performs poorly - and consequently the reaction to poor performance is amplified.”
Dr. Finnegan McKinley adds: “We document an expectation violation that managers appear to have for AI-based forecasting systems: since expectations for AI performance are higher, when poor performance is observed, managers react more strongly.”
→ Takeaway: Higher expectations for AI mean forecasting errors often trigger stronger reactions than similar mistakes from traditional systems.
How should demand planning processes change when organizations introduce AI-based forecasting tools?
Dr. John Aloysius notes: “The best way is to redesign the process so as to automatically measure and anticipate the built-in biases of both AI systems and human users, and to automate the process to correct for these biases. Our ongoing research that we hope to publish soon has made some very interesting progress in designing such a process!”
Dr. Finnegan McKinley explains: “Updates to demand planning processes should be guided by the behaviors and attitudes of the forecasters who use these tools. By incorporating what we know about human behavior into the design of these processes, we can improve performance while keeping the human at the center.”
→ Takeaway: Design demand planning processes around both AI capabilities and human decision-making to improve forecasting performance.
How can organizations train planners to collaborate more effectively with AI systems?
Dr. John Aloysius explains: “People need to be trained when they should intervene, and when they need to let the AI system do its thing. With forecasting, the human should only step in when they know something about a special event that the system doesn’t – maybe a supplier strike, a weather event, or a competitor having supply problems.”
Dr. Finnegan McKinley notes: “Organizations can involve planners early in the implementation process. Giving planners an opportunity to express concerns about new AI technology can inform managers where adjustments to AI implementation processes are needed.”
Dr. Adriana Rossiter Hofer adds, “In addition to educating managers on the basics of how AI applications generate results compared with the legacy systems they are accustomed to, training should include two important elements. First, planners need to learn when they should intervene and when they should let the AI system do its thing. Studies show that, in forecasting, human intervention can improve results when managers know something about a special event that the system does not — perhaps a supplier strike, a weather event, or a competitor experiencing supply problems.
Second, it is critical that training helps managers understand human biases toward the ‘black box’ nature of AI, as well as how our own thinking can lead us to overreact or underreact in certain situations — both of which can reduce forecasting accuracy. By understanding these natural human biases, managers can become more effective at recognizing and adjusting behaviors that may compromise the accuracy of their forecasting recommendations, which is ultimately their professional goal.”
→ Takeaway: Train planners to recognize when human expertise adds value—and when trusting the AI will produce better forecasts.
What risks arise when managers overcorrect or under-trust AI-generated forecasts?
Dr. Rebekah Brau notes: “Firms Sales and Operations Planning (S&OP) processes usually calibrate over time so that organizations achieve reasonable performance. Introducing a disruptive technology like AI can mess with that stability, which could create ripple effects that result in increased volatility in inventory levels which usually lead to higher inventory costs and lower on-shelf availability.”
Dr. Finnegan McKinley adds: “We risk wasting manager energy on scrutinizing AI-generated forecasts instead of focusing manager energy on adding additional value from other sources of information. The more we can promote healthy skepticism towards AI-generated forecasts, the more capacity we provide managers for the more complicated challenges of demand planning.”
→ Takeaway: Overreacting to AI forecasts can undermine planning stability, increasing costs and distracting managers from higher-value decisions.
Link to the Original Research
Published in Journal of Operations Management (2026)
Frequently Asked Questions
Do managers trust AI forecasts more than traditional models?
Not necessarily. The research suggests that trust depends heavily on performance. Managers may have strong expectations about AI, but they tend to react more critically when AI performs poorly, leading to larger adjustments.
How do managers respond to inaccurate forecasts?
Managers adjust forecasts in predictable ways, reducing forecasts that are too high and increasing those that are too low. This indicates that users actively correct for bias rather than blindly following algorithm recommendations.
Can managers recognize when forecasting systems improve?
Yes. The findings suggest that users reduce the size of their adjustments as algorithm performance improves, indicating an ability to detect learning and increased accuracy over time.
Why does labeling a system as “AI” matter?
Labeling shapes perception. Even when systems perform similarly, the designation of “AI” can influence how users interpret errors and adjust outputs, especially when expectations are high.
What does this mean for AI adoption in supply chains?
It suggests that successful AI implementation requires more than improving model accuracy. Organizations also need to manage user expectations, training, and decision processes to ensure effective human–AI collaboration.
