The Evolution of Predictive Analytics: Machine Learning and the Predictive Step in Empirical Asset Pricing—From Factor Zoo to Factor Refinery

Abstract

In the last fifty years, empirical asset pricing has depended on a consistent knowledge-production pipeline: economic theory first suggests hypotheses, hypotheses are then operationalised through candidate factors, and the validity of those hypotheses is finally determined by in-sample statistical significance. As the number of candidate factors grows into the hundreds, this pipeline is no longer a reliable way of identifying genuine factors among data-mined false positives. Organised around four steps — testing, construction, discovery, and integration — the paper argues that machine learning has established, alongside the conventional pricing tests, an evaluation framework based on out-of-sample predictive performance. This does not mean, however, that the out-of-sample evaluation framework is a substitute for alpha, GRS, or risk-price tests. The main thesis is that machine learning has made genuine instrumental advances in prediction-based tasks, but cannot be regarded as a substitute for causal inference or mechanistic explanation; its benefits are real but restricted, and its drawbacks (loss of interpretability, conflation of correlation with causation, and overfitting) are significant. A more precise statement is thus not that machine learning has changed empirical asset pricing, but that it has changed the mode of execution of prediction-related tasks within the field without changing the basic logic of understanding-related tasks.