The Future of Business Growth Prediction: Bridging econometrics and machine learning approaches

The following blog written by Dr Rita Nana-Cheraa is based on the findings from our recently published report – Predicting Business Growth.

The quest to predict business growth has long been a holy grail for economists and management scholars. Historically, however, this has proven incredibly difficult. Traditional empirical research often struggles to pinpoint consistent growth drivers, and most conventional models demonstrate low predictive power. This inherent unpredictability is driven by limited data scope and firm heterogeneity—the fact that every company is different, and variations across industries, technologies, and countries make broad generalizations nearly impossible.

A recent joint report by the Innovation & Research Caucus (IRC) and the Enterprise Research Centre (ERC) synthesises the latest evidence on business growth prediction. It compares traditional econometric approaches with emerging machine‑learning methods and examines the implications of this shift for research, policy, and practice.

Where econometric models do well — and where they fall short

Research on firm growth has traditionally relied on econometric models grounded in deductive theory. These approaches derive testable hypotheses from theory and assess how factors such as firm size, innovation, and market structure influence growth outcomes like employment, sales, and profitability. Their core strength lies in explanation: econometric models are well-suited to identifying statistically significant relationships and, in some cases, causal effects that help explain why firms grow.

However, this explanatory strength does not translate into strong predictive performance. Evidence from the IRC/ERC review shows that standard econometric techniques, such as Ordinary Least Squares (OLS), typically yield R-squared values below 0.09, meaning included predictors explain less than 10% of the observed growth variation. Even more advanced panel data methods show wide and inconsistent predictive performance, with R-squared values ranging from 0.026 to 0.63 depending on the dataset, time period, and growth measure.

Comparative evidence suggests that panel fixed effects models tend to outperform most econometric techniques: Applied to the same dataset and firm context, they can achieve an R-squared of around 0.34, compared with 0.03 for OLS and 0.02 for quantile regression. Nonetheless, even the best performing econometric models leave a large share of firm growth unexplained.

How machine learning changes the picture

Machine learning (ML) techniques, increasingly popular in finance, economics, and business research, offer a fundamentally different approach to analysing firm growth. Rather than relying on predefined economic theory or fixed functional forms, ML algorithms are data‑driven, identifying complex patterns, non‑linear relationships, and interactions that are difficult to specify ex ante. They are particularly well-suited to high‑dimensional and unstructured data—such as financial reports or web‑scraped text—where conventional econometric methods often struggle.

Applied to firm growth, supervised ML models demonstrate substantially higher predictive accuracy, especially in identifying high‑growth firms. In terms of predictive performance, they consistently outperform traditional econometric models. For example, whereas OLS models typically exhibit very low explanatory power, ML algorithms such as CatBoost have been shown to achieve prediction accuracies of up to 86%.

The trade‑off: accuracy vs understanding

However, the superior predictive performance of machine learning models often comes at the cost of interpretability. Many ML techniques function as “black boxes”: while they can accurately predict which firms are likely to grow, they provide limited insight into how or why those predictions are generated. As a result, the specific mechanisms linking firm characteristics to growth outcomes often remain opaque.

This lack of transparency is not a trivial limitation. Policymakers, investors, and business support organisations typically require more than binary growth predictions; they need to understand the underlying drivers of growth in order to design effective interventions, allocate resources efficiently, and justify decisions to stakeholders. Without interpretability, ML-based predictions are difficult to translate into actionable policy measures or long-term corporate strategies.

A new integrated paradigm

The key takeaway from the report is that econometric and ML approaches are not substitutes. They are complementary.

• Econometrics provides theory, causality and interpretability.
• Machine learning provides flexibility, scale and predictive accuracy.

The most promising path forward lies in combining these strengths — using ML to enhance prediction while retaining econometric rigour to ensure insights remain transparent, credible and policy‑relevant.

Predicting business growth will never be easy. But by blending methods rather than choosing sides, we can do it better.