In this issue, Robert Arnott, founder and chairman of Research Affiliates, provides insight on machine learning techniques and their applicability to quantitative, model-based approaches. Brandon Kunz, partner and multi-asset strategist at Research Affiliates, reviews the drivers of the current overall and compositional risk tolerance of the All Asset strategies. As always, their insights are in the context of the PIMCO All Asset and All Asset All Authority funds.
Q: What are your thoughts on machine-learning techniques within quantitative model-based strategies such as All Asset?
Arnott: As is true of most quantitative applications in finance, machine learning offers both great promise and potential danger. Unlike the physical and biological sciences, where machine learning has been used quite successfully, financial data is much more limited, particularly for informing longer-horizon asset allocation decisions. While at a glance there appears to be a wealth of individual data points across markets and economies – prices, volumes, etc. – in fact, longer-term data series that are precisely defined and consistent enough to inform machine learning have only appeared in the last few years, and on a still limited scale. This makes their use for machine learning applied to anything other than daily or intra-day trading problematic.
Machine learning, artificial intelligence (AI), and “big data” applications are in their element when there are billions of independent data samples. Millions of samples aren’t enough, and thousands of samples are woefully inadequate. For tick data (i.e., prices, volumes, and other data on specific trades) and high frequency trading (HFT), machine learning is in its element. In time, as more robust data becomes available, machine learning’s applications within broader quantitative models, or within narrow and precisely defined strategies, may expand. Today, however, for the longer one- to three-year tactical investment horizon of the All Asset strategies, we believe the dangers far exceed the potential benefits.
That being said, these quantitative tools can be valuable in identifying shorter-horizon alpha opportunities, and investors in the All Asset strategies will naturally benefit from developments in machine learning and AI by virtue of PIMCO’s work in these areas. PIMCO has committed significant resources to this effort in seeking higher net-of-fee alpha across their funds, including those underlying the All Asset strategies (more to come on this from PIMCO in an upcoming issue of All Asset All Access). This is yet another example of the complementary nature of the partnership between Research Affiliates and PIMCO in seeking to provide the highest possible value add for investors in the All Asset strategies.
How can one seek to harness the potential benefits of machine learning while avoiding the potentially perilous pitfalls? Cam Harvey, Harry Markowitz, and I recently published an article1 in The Journal of Financial Data Science in which we outline a seven-step protocol for empirical research in finance in today’s world of machine learning. This paper warns of the consequences of putting data mining on autopilot and then adding a supercharger from machine learning. But the points we make in the paper can be applied broadly to any form of quantitative research. Naturally, at Research Affiliates this protocol is deeply ingrained in our research and product evolution, notably for the All Asset strategies over the last 16-plus years.
Briefly stated, the problem is that our natural human behavioral tendencies allow the data to seduce us into thinking our models are better than they are. Almost all finance research requires backtesting, but not all backtesting is truly research. We can fall into the trap of testing, modifying, and retesting models with strong results, while quickly discarding tests with poor results. We refer to this as “using the backtest to improve the backtest.” When we express it in these terms, the perils of data mining become self-evident. This leads to two problems: a false negative, when a model is rejected as a result of the data set, and a false positive, when the model works well in our backtest for reasons that are in no way predictive of the future efficacy of the model.
Research Affiliates’ protocol can help avoid false positives, or worse, an exaggerated (distorted) positive – when test results are much stronger than the live results in portfolios. False negatives are far harder to prevent; this is where an element of art creeps into quantitative research.
The protocol has seven steps, many designed to help the quant community avoid unintended performance chasing. (While quants would disdain merely choosing whichever stocks have performed best in the last 20 years, they’re perfectly happy to choose whichever models have worked best in the last 20 years.)
- The empirical research should begin with a strong theoretical foundation. Too often quantitative research mines the data to see what works, then develops the theory after the fact. This is not “scientific method.” Creating an economic story to explain empirical findings after the fact is a common recipe for disaster, made worse with the power of machine learning. Without a prior theoretical foundation, the chances are high that the strategy will fail in live trading. If machine-learning is not governed by reasonable hypotheses, the chance of a false positive is high.
- Keeping track of the number of strategies tried and their correlations as well as the combinations of variables is important. The higher the number of strategies tested, the greater the chance that the results achieve high significance, purely by luck. Similarly, the larger the number of variables, the greater the possible interactions, which also increases the chance that findings appear robust – again, purely by luck.
- The test sample must be defined in advance, and be of high quality. “Garbage in, garbage out” applies in machine learning even more than in traditional quantitative techniques. Data transformations, such as volatility scaling or standardization, should be documented and ideally decided in advance. Outliers should be explained by the model and never excluded after the fact.
- We should also acknowledge that no true out-of-sample data exist. Researchers associate leading variables with past experience. Even when looking at backfilled data that predates the testing sample, researchers should be well aware of historical market movements that can influence their hypotheses and are in-sample in their own experience. Similarly, modelers are overfitting if they adjust the model so that it works both in-sample and out-of-sample; this is not the same as model validation. For success in live trading, a model must account for transaction costs and implementation shortfall. Absent this practical consideration, the robustness of the anomaly essentially vanishes in live applications.
- A model’s dynamics are also important. Machine applications can adapt through time. In economics and finance, we are dealing with human beings and their changing preferences and norms. The dynamics of these relations over time can easily shift. Moreover, investing live assets moves prices and can shrink the efficacy of a model. Market inefficiencies are hardly static, and past cross-validated relations may cease to apply once live trading begins. Tweaking the model to get better results, although a natural response, can lead to overfitting, or even worse, a poor live-trading experience.
- Simplicity rather than complexity is key, in our view. The more dimensions in a model, the greater the need for data, but as I mentioned earlier, data are limited in finance. Without sufficient data, multidimensionality increases the probability that one or more models will work well in sample. Higher levels of complexity lead to greater dependence on non-intuitive relationships, increasing the potential for slippage between simulated returns and live results.
- Lastly, a culture that rewards good science rather than good results is crucial to identifying a robust anomaly. Be aware of the incentives that motivate researchers to maximize backtest results and not future live experience.
Machine-learning applications face many of the same issues that we quantitative finance researchers and practitioners have struggled with for decades. If quantitative methods offer an engine for research, machine learning adds a supercharger. But if the driver is inept, the crash could be that much worse.
Humility is very helpful in research. To quote George Box, “all models are wrong, but some models are useful.” If we are knowledgeable, thoughtful, and deliberate, and if we have the discipline to strictly follow the seven steps above even in the face of inevitable underperformance cycles, then long-term potential would be expected to be much improved for any quantitative, model-based approach. The All Asset strategies are no exception.
Q: What is the current risk tolerance and risk composition of the All Asset strategies, and what are the drivers of these characteristics?
Kunz: Recall that the All Asset funds are optimized to seek maximum real return potential per unit of risk, all while maintaining a structural emphasis on nontraditional asset classes or strategies, including real assets, emerging markets (EM), and high yield bonds. This structural emphasis, what we’ve long referred to as a “Third Pillar” orientation, is a meaningful contributor to the longer-term compositional risk of the strategies, and it alone allows the All Asset funds to deliver natural diversification benefits relative to most equity-centric conventional portfolios. Potentially overlooked, however, are the components of the All Asset strategies’ investment process that lead to shorter-term tactical deviations in both compositional and absolute risk.
I’ll first address absolute risk tolerance. While we expect the All Asset funds to realize longer-term levels of volatility similar to conventional 60/40 portfolio averages, at any given time we allow risk tolerance to rise (or fall) based on the rising/steepening (falling/flattening) of the efficient frontier. Our forecasted efficient frontier will evolve due to changes in a host of portfolio inputs such as asset class yields, valuations, price trends, underlying PIMCO fund alpha estimates, and country-specific business cycle models signaling which asset classes we believe are more likely to experience return tailwinds. All inputs considered, as estimated return compensation per unit of risk rises, the All Asset strategies systematically increase their absolute risk tolerance in seeking to capture the higher expected return potential, and vice versa.
Today, Research Affiliates’ forecasted efficient frontier indicates that increased risk tolerance is merited, and both the All Asset and All Asset All Authority funds are positioned to assume modestly higher (though downward-trending) volatility than their since-inception averages of 8.58% and 8.70%, respectively.2
The predominant driver of this modestly elevated risk tolerance is what we believe to be increased attractiveness of select Third Pillar markets. Year-over-year through 31 March 2019, emerging markets equities, local bonds, and currencies have sold off by −6.2%, −7.1%, and −4.1%, respectively.3 As a result, our five- to seven-year real return forecasts have risen to 6.7% for EM equities, 4.7% for EM local bonds, and 3.7% for EM currencies, and that’s without considering the rising alpha potential from the underlying PIMCO funds.4 For example, PIMCO RAE Emerging Markets Fund, a value-oriented equity strategy available in the All Asset strategies opportunity set, now trades at an all-time-high discount of 46% (using an equally weighted combination of price/sales, price/earnings, price/dividends, price/cash flows, and price/book ratios) to its secondary benchmark, the MSCI EM Index.
We’ve written extensively about the link between starting valuations and subsequent excess return potential for both equity factors and strategies, so these peak discounts naturally lead to peak alpha forecasts. With elevated asset class and fund alpha forecasts, the All Asset and All Asset All Authority funds recently reached all-time-high exposure to emerging market assets, with 31 March 2019 allocations settling in at just below 40% in All Asset and 46% in All Asset All Authority.
On to risk composition. While our tactically elevated emerging markets exposure currently (and intentionally) accounts for our largest single risk contribution within the All Asset strategies, this specific exposure should not be interpreted as increased appetite for general procyclical “risk on” exposure. In fact, today’s ex ante equity beta (one measure of procyclical risk tolerance, here tracked relative to the S&P 500) for both funds is below the since-inception average of 0.4 for All Asset and 0.3 for All Asset All Authority. This below-average beta is accomplished through reduced exposure to developed market equity and credit markets, and increased exposure to defensive fixed income and low-beta alternative strategies. The rationale for the All Asset funds’ increasingly defensive compositional risk posture is largely driven by overvaluation considerations and rising probabilities of economic slowdown. In other words, our tactically elevated EM exposures are counterbalanced by tactically elevated exposures to defensive countercyclical assets.
Speaking to overvaluation considerations, nearly every procyclical asset class within developed markets sports high valuations relative to both history and our equilibrium expectations: As shown in Figure 1, U.S. large and small cap equities trade at valuation multiples well into the top quintile; they are also well above our business-cycle-adjusted equilibrium expectations. Today, only developed ex U.S. equities trade at valuations close to expected equilibrium.
Figure 2 tells a similar story on the credit front. U.S. high yield, investment grade credit, and bank loans all trade at depressed spreads to like-maturity U.S. Treasury securities and are well below our equilibrium expectations.
Over the last two years, the All Asset and All Asset All Authority funds have reduced total exposure to these procyclical asset classes by a meaningful 12.3 and 17.1 percentage points, respectively, with all-in allocations standing at a mere 10.8% in the All Asset Fund, and at −5.9% (yes, that’s a negative number) in the All Asset All Authority Fund as of 31 March 2019. Given these low/negative allocations, the expected contribution to each fund’s overall risk is negligible or even risk reducing.
In our view, a portfolio-level rotation into more defensive assets and alternative strategies is also supported by rising probabilities of a global economic slowdown. We begin by modelling slowdown probabilities for 20 different countries using various economic and monetary policy indicators. Figure 3 combines each country’s slowdown probability since 2000 into a GDP-weighted global aggregate. Over the last 12 months, global slowdown probabilities have risen from a below-neutral 37% to a slightly above-neutral 51%. This 14 percentage point transition into above-neutral territory means that risk assets now face increased valuation headwinds while defensive assets now face valuation tailwinds. That said, our calculations show we’re not too far away from neutral, so these valuation headwinds and tailwinds should remain subtle unless slowdown probabilities rise considerably from here.
In any case, this upward move has contributed to All Asset and All Asset All Authority’s respective year-over-year 6.7 and 2.7 percentage point increase in defensive exposure to short-term, core, long duration, and inflation-linked bonds along with low-beta alternative strategies. Today, all-in allocations to these asset classes and strategies make up 36% of All Asset Fund and 33% of All Asset All Authority Fund, and given powerful diversification characteristics, their estimated contribution to portfolio-level risk comes in at about one-third of those numbers.
Bottom line, the All Asset strategies have incorporated a “barbell approach” to their current portfolio composition. While our willingness to underwrite what we believe to be cheap emerging markets exposure has almost never been higher, this is offset by significantly reduced exposure to developed market risk assets, which we generally view as overvalued, and increased exposure to more defensive, diversifying asset classes and strategies. This approach seeks to achieve multiple concurrent goals: preserving attractive real return potential, increasing U.S. equity diversification without sacrificing inflation hedging, and also allowing the All Asset strategies to maintain a healthy yield as a buffer against potential drawdowns.
Recent editions of All Asset All Access offer in-depth insights from Research Affiliates on these key topics:
- Differences between the All Asset and All Authority strategies, and their underlying allocation infrastructure (April 2019)
- Assessing investment risk in terms of the likelihood of meeting long-term wealth accumulation goals (March 2019)
- The potential impact of a bear market in U.S. stocks on emerging markets (February 2019)
- Research Affiliates’ outlook and asset allocation views for 2019 (January 2019)
- Market impact of the U.S. midterm elections and what differentiates All Asset’s positioning versus peers (December 2018)
- Outlook for achieving All Asset’s long-term real return benchmark and its approach to assessing country risk (November 2018)
The All Asset strategies represent a joint effort between PIMCO and Research Affiliates. PIMCO provides the broad range of underlying strategies – spanning global stocks, global bonds, commodities, real estate and liquid alternative strategies – each actively managed to maximize potential alpha. Research Affiliates, an investment advisory firm founded in 2002 by Rob Arnott and a global leader in asset allocation, serves as the sub-advisor responsible for the asset allocation decisions. Research Affiliates uses their deep research focus to develop a series of value-oriented, contrarian models that determine the appropriate mix of underlying PIMCO strategies in seeking All Asset’s return and risk goals. 1 A new journal, The Journal of Financial Data Science, was launched last fall, because of the growing interest in machine learning, artificial intelligence, and “big data.” Our paper, “A Backtesting Protocol in the Era of Machine Learning,” was published in their inaugural issue, in December 2018.
2 PIMCO All Asset Fund inception date is 31 July 2002, and PIMCO All Asset All Authority Fund inception date is 31 October 2003. Volatility measured as standard deviation of monthly returns.
3 EM equities are represented by the MSCI EM Index, EM local bonds are represented by the J.P. Morgan GBI EM Index, and EM currencies are represented by the J.P. Morgan ELMI+ Index.
4 Forecasts are based on the indexes cited in footnote 3. Return assumptions are for illustrative purposes only and are not a prediction or a projection of return. Return assumption is an estimate of what investments may earn on average over the long term. Actual returns may be higher or lower than those shown and may vary substantially over shorter time periods. Research Affiliates return forecasts may vary from PIMCO’s capital market assumptions.