Monday, January 2, 2017

Testing Momentum’s Robustness




Happy new year!

I have noticed that my quantitative posts get the most readership and discussion. So this year, I’ll be posting a lot more research and will start the year off by exploring momentum’s robustness.

There are two good ways to test the robustness of a rules-based trading strategy:
  1. The test of time - how does the strategy behave in different market regimes?
  2. Parameter sensitivity – how stable is the performance if the strategy’s parameters are varied
Let’s look at each of these separately.


Test of Time

The goal is to see how well a strategy performs over a very long-term, under different market conditions. Gary Antonacci has some good articles on the importance of having a long-backtest period (see Bring Data and Bring More Data).

In 1937, Cowles & Jones did the first paper showing that 12-month momentum works. In 1967, Robert Levy did the first computer backtest on momentum. In 1993, Jegadeesh & Titman wrote their seminal paper showing 3-12 month momentum works. Also in that year, Eugene Fama (father of the efficient market theory) called momentum the "premier anomaly."

In 2013, Geczy and Samonov published a paper showing a momentum backtest that went back over 200 years. The following year, Greyserman and Kaminski did an 800 year backtest. Both papers show that the momentum anomaly is persistent over the long-run.

The table below is how the Global Equities Momentum (GEM) strategy has performed from 1971-2015. 


While these results are nice, looking at cumulative outperformance over a backtest period is not enough – we should also look at individual periods within the data.

I did a post a while back showing what happens when we add Gold to the GEM model (see Should we consider gold?). GEM’s annual returns (since 1970) jumped from 18% to 21% by including gold. At first glance, this sounds exciting. But a closer look showed all this additional gain came from the 1970s when the US went off the gold standard. No longer being suppressed, the price of gold rapidly rose in just a few years. We need to be careful not to be misled by these types of one-time events.

The graph below shows that equities (red and green lines) had a secular bear market during the 1970s and 2000s, and a bull market during the 80s and 90s. In both market types, GEM (blue line) did well.


Let’s dig further with a longer period of data. How does momentum do on a decade-by-decade basis?

Meb Faber looked at this in his paper “Relative Strength Strategies” (read here). The table below shows the outperformance each decade of a US sector momentum strategy (hold the top performing sector and rebalance monthly) compared to the entire US stock market.   


While some decades were better than others, momentum had positive alpha every decade since 1930. This is an indication that the momentum anomaly has persistence.

OK, all this looks nice however…

While momentum (and all trend-following) strategies tend to perform well in trending markets, they typically fail in very erratic, sideways markets. An extreme market-regime test is to see how momentum would fare using Japan’s Nikkei stock index, which has traded sideways for 30 years. This will be the topic of my next post. Let’s continue on.


Parameter Sensitivity

Any quantitative strategy has one or more input variables. GEM has two: the lookback period (length of time over which past performance of assets is measured) and rebalancing period (how often you check for a new trading signal). GEM’s recommended parameter settings are 12 months for the lookback and monthly for the rebalance.

By varying these input parameters, we want to see how the strategy’s performance changes. I want to be clear: this is NOT done to find the optimal parameter settings. That would be data mining. Instead, this test is done to determine the model’s sensitivity. A sign of a robust trading strategy is one whose performance is not affected by small changes in its settings.

Note: The analysis that is presented below was done using daily data and MATLAB software. Source code and data is available upon request. 

We first look at how GEM’s performance (between 1988-2016) changes as we vary the rebalancing period from 1 to 100 trading days while keeping the lookback period constant at 12-months.


We see from above that a rebalancing period between 15-25 trading days results in returns performance that is stable while providing high returns and low portfolio turnover. This confirms the recommended setting of 20 trading days (1 calendar month).

It’s interesting to see that a rebalancing period less than 10 trading days results in very high portfolio turnover and with lower returns (due to whipsaw losses). There is absolutely no reason to be doing weekly rebalancing with GEM.

Next, we look at how GEM’s performance changes as we vary the lookback period from 30 to 500 trading days, while keeping the rebalancing period constant at one calendar month.


We see above that any lookback period less than 200 trading days results in higher whipsaw losses and higher portfolio turnover. A lookback between 250-350 trading days results in stable performance with fairly low portfolio turnover. This confirms the recommended setting of 12 calendar months (250 trading days).  

The 12 month look back period was first discovered by Cowles and Jones in 1937. Various other research since then has shown momentum works best with monthly rebalancing and a 12-month lookback period. The results above confirm this, as well as demonstrate how the prior research has held up since it was first published.


Conclusion:

Momentum has passed several checks for robustness. It has proven itself over a long backtest period and in different market regimes. In addition, the performance of a simple momentum strategy like GEM is not sensitive to small changes in the strategy’s parameters.

I often get asked about using much shorter rebalancing and lookback periods to be able to adapt faster to market changes. Up until now, I have just verbally been telling people that this would create higher # of trades and whipsaw losses. Now I can back up my recommendation with quantitative evidence.

I leave you with Dilbert’s take on model sensitivity.



Results are hypothetical results and are NOT an indicator of future results and do NOT represent returns that any investor actually attained. Additional information regarding the construction of these results is available upon request. Past performance is no assurance of future success. Please see our disclaimer page for more information.


37 comments:

  1. thanks so much for this site ... Gary Antiknock has changed my thinking 180 degrees regarding investing in the financial markets. I avoided the markets my whole life because it seemed too risky. Now I see that was wrong and I have started to invest.. cost averaging in...Am really looking forward to your next post on the worst case scenario Nikkei..

    ReplyDelete
  2. Have you tried dividing the portfolio into multiple tranches each with a different look back period length? I believe it may lower volatility and drawdowns.

    ReplyDelete
    Replies
    1. I believe the guys at Newfound do this (https://blog.thinknewfound.com/2015/04/new-research-paper-minimizing-timing-luck-portfolio-tranching/) but I don't see that much value. I'll try to do a post on this.

      Delete
  3. They tranche the portfolio so you don't rebalance the whole portfolio the same day, but each tranche has same look back period.

    ReplyDelete
    Replies
    1. It's not just the rebalance period that creates timing risk, but also the lookback period. If you really want to reduce this type of risk, you would create tranches with diff lookback and rebalance lengths.

      You'll certainly be trading more. Is it worth it to spend higher transaction costs to reduce a small, unknown loss (maybe even a gain)? I'll add it to my research list.

      Delete
  4. Was just talking to Gary about DMSR and how the new data affected his thinking. I love the rebalancing comparison, great stuff Gogi!

    ReplyDelete
    Replies
    1. That's why having lots of data is important

      Delete
  5. As a further robustness test, what about replacing ACWI-ex and SPX with other indices to create true out-of-sample testing? For example, ACWI-ex + EM; or maybe SPX + Japan; etc.? Or with/without currency hedging on ACWI-ex?

    Nick de Peyster
    http://undervaluedstocks.info/

    ReplyDelete
    Replies
    1. Way ahead of me. This will be subject of next post, stay tuned

      Delete
  6. For the benefit of those interesred, here is a copy of an exchange I recently had with Gogi!

    Hi Gogi,

    First of all, thx for sharing with us your ideas and research.

    Now, being canadian and investing part of my money through tax exempt accounts, I wonder what it would be trading xiu instead of an international etf.
    Have you tested xiu, zsp (us) and xbb (bonds) the way GEM trades US, Intl and US bond markets?

    Correlation might be higher between xiu and zsp and impact the results...

    But One might argue that correlation is now higher anyway between developped markets and that currency variations is the main factor that set US and any developped market apart.

    Thanks,
    Andre

    Hi Andre,

    I have looked at adding Canada (but in addition to US and World ex-US). Same for Emerging Market. I'll look at what happens when World ex-US is replaced with Canada.

    Short answer is: Canada is concentrated in Banks & Resource (these make up >70% of XIU). This adds a lot of volatility to GEM and Sharpe Ratio doesn't improve much. All this will be a separate blog post. Can you post this comment and future comments on the blog so that others can benefit?

    -Gogi

    ReplyDelete
  7. Hi Gary, could I view the source code and data?

    ReplyDelete
  8. Gogi, what if we get a buy or sell signal at end of month, and the market is severely short-term overbought or oversold? Do you know if it pays to wait a few days (or sometimes a few hours) to let the condition naturally resolve? (so that you get a better price?). It would be very interesting to know how that works out.

    ReplyDelete
    Replies
    1. http://www.sharpereturns.ca/2015/07/never-override-your-system.html

      Delete
    2. Oscillators that measure overbought/oversold suffer from the same problems as sentiment and valuation. What is oversold/hated/cheap can get even more oversold/hated/cheap. Never do something without fully testing it.

      Delete
  9. Since weird things happen at the end of the month some people re-balance on 33 day schedule. In the long run I've found it doesn't help to try to out-think the model. Those who don't like Fridays and end of month window dressing can maybe just do re-balances every Wednesday in the first week of each month. Often there will be no trades to make anyhow.

    ReplyDelete
    Replies
    1. I was going to test this (another item on my long to-do list). Checkout this article as well: http://blog.alphaarchitect.com/2015/11/30/momentum-seasonality/#gs.BVG_kkY

      Delete
  10. You mention that you are checking for robustness, not to optimise. But still, your test indicates that a 350 days LB gives the same performance as 250, but with close to half the trades.

    Why don't you check the robustness of RB and LB all the way back to 1971?

    ReplyDelete
    Replies
    1. A lookback of 250 days (ie. one year) is what research papers support and is why I use it. The sensitivity analysis requires using daily data, which I can only find going back to 1988. Otherwise I would've done a longer backtest

      Delete
  11. I'm still pretty skeptical. From this paper it looks as if relative momentum worked at its best precisely during the period for which GEM results are presented
    https://www.jstor.org/stable/2676188?seq=1#page_scan_tab_contents
    There is a possibility that momentum will continue to give better results, but I don't see a fundamental reason why what we loose in whipsaw losses will not be superior to what we gain from bear market protection. The one sure thing is that it involves fees and taxes that you avoid by B&h So the latter seems the more rational option to me

    ReplyDelete
  12. "Unknown", were you invested in 2000? In 2008? I was. It was awful. Twice is enough for me. No more B&H. https://www.linkedin.com/hp/update/6247785929277263872

    ReplyDelete
    Replies
    1. Yes I see your point, though whipsaw losses can be frustrating. Look at howmuch GEm has underperformed since 2009 due to those; read a paper by AQR yesterday. It ended saying 'diversification trumps market timing'. Best, Paul

      Delete
  13. This comment has been removed by the author.

    ReplyDelete
  14. By the way i am the unknown, didn't connect properly before

    ReplyDelete
  15. the other thing is that abs momentum will not necessarily avoid large DD I think the largest since 1930 would have been around 50%, and larger before then. You can check eg on the AA blog.

    ReplyDelete
  16. After the NEXT bear market the 2009 to Present performance will look totally different. Be careful about cherry-picking data and sectors. Yes, in a US bull market period, everything expect US bull market will look bad; that's by defnition. It proves nothing. Eight years is not a full market cycle, it's a 1/2 or 2/3 market cycle.

    ReplyDelete
  17. Wait... that's a terrible time period for compare, 2009 to now. Stocks had the stuffings pulled out of them. It was the low of the low. That's serious cherry picking. Re-do your analysis for 1999-2017. Or 2007-2017. Totally different pictures. portfoliovisualizer.com Do not rely on "experts" who want to sell you something. AQR has lost me lots of money in the past.

    ReplyDelete
  18. Hi Peter, I was not cherry picking. Actually according to some academics I have talked to, the results for GEM are cherry picking...If you read the paper I provided the link to, perhaps you'll see why. Anyway, my point was that just like losing money in a bear market is bad, so is losing money due to whipsaw losses. I don't like AQR and don't buy their products, but I admit they are clever people. As for selling you somehing, isn't Gary Antonacci selling Enhanced GEM?...

    ReplyDelete
  19. Sure he charges for the private GEM, but he GAVE away the other one. Look, just take the OLDEST mutual fund you know of, put it into the Dual Momentum calc at portfoliovisualizer.com, from the 1980s to 2017, see if you like the results. That can't possible be cherry picking. I am right now looking at DM on SNXFX, the first fund I ever invested in going back to the 1990s, and it looks good in the DM calculator, it runs 1994-2017. Is that cherry picking? Suggest an older one, let's look at it. Don't go by hearsay, trust the data, the math, and your eyes.

    ReplyDelete
  20. USAAX... I bought it in 1986 when I was 25 years old. Looks beautiful in the DM calc. God, I wish I'd known about this back then.

    ReplyDelete
  21. Peter, yes I was initially impressed by the back-test on PV. Dual momentum has worked well during the last two huge bear markets. But PV doesn't go back enough. Antonacci himself quotes Kenneth French in Dual momentum saying that 78 years of data might not be enough to separate noise from the signal (page 119 of the book). If you look at Alpha Architect's blog you's see that trend following didn't work during long time periods. To me it's clear that you will incur transaction expenses for sure; but it's dubious whethr you'll out perform. There was a question on Jonathan Clement's blog the other day on this, and Clements pointed out that there are penty of mathods that work in back-tests, but will not continue to do so in future

    ReplyDelete
    Replies
    1. You can see trend following back to 1934 here as used by Antonacci and back to 1223 used similarly by others: http://www.dualmomentum.net/2017/02/factor-zoo-or-unicorn-ranch.html

      Delete
    2. Interesting article by Ben Carson who seems quite knowledgeable on these matters. http://awealthofcommonsense.com/2017/04/my-evolution-on-asset-allocation/
      He writes 'All of my research has shown that these systems tend to give you similar returns to a buy and hold strategy but with a different volatility profile' and 'The goal is not necessarily to “beat the market” but to manage risk and help investors survive severe market downturns'. This way of undertanding trend following makes more sense to me. The main issue I had is with the certainty of the proponents of Dual momentum that this will beat the market.

      Delete
    3. Paul, if 78 years is not enough: In 2013, Geczy and Samonov published a paper showing a momentum backtest that went back over 200 years. The following year, Greyserman and Kaminski did an 800 year backtest. Both papers show that the momentum anomaly is persistent over the long-run.

      Nothing is guaranteed to beat the market. But like Peter said, trust the data and your eyes.

      Delete
  22. you can check the Q and As here. Clements is dfinitely not trying to sell you something, he gives good advice for free in this blog.
    http://www.humbledollar.com/2017/04/no-we-cant/

    ReplyDelete
  23. How can we request the source code? Thanks!

    ReplyDelete
    Replies
    1. Hi, please use the contact form to send me your email

      Delete

Note: Only a member of this blog may post a comment.