length of price history at daily frequency

kc975943

It appears that the longest bar history at daily frequency one can get is 250 days. Is this an intended limitation? Or is there a way to extend it to 750? It would help with building slower-trading strategies that rely on fitting models.

ptunney

Each backtesting date has a data set built with all the data required for that date. That means that if it is the 24th of December 2014 when you access the daily bars for say the SP500 you will be seeing those daily bars exactly as you would have seen them on that date in 2014. Now 2 weeks later some of those symbols may have changed/split/had other events that caused their prices to change. Building a backtesting system you have two options, calculate on the fly using a table of changes or pre-calculate. Now if your backtest was daily with a limited symbol set you could get away with generating the bars on the fly. But you cannot do that if your backtester goes all the way down to tick by tick trading. So we precalculate and build for each day. We analysed data usage and observed that 95% of our users went no further back on data than a couple of week. 99% went back no more than a year. The storage increase for each additional month for each symbol for every day can quickly get out of control not to mention when you have to recalculate them all for some reason (which does happen).

So we built the system with one year of daily bars for all 8000 symbols for every day back to 2011. As time moved on we had more and more people wanting to write multiday longer term backtests and so there was a demand for being able to fetch more daily bars. Still the usage levels were too low to merit extending all the data and all the work/storage that would entail. So we added a command that will look at the fundamental changes and calculate more daily bars for you on a per backtest basis.

If you use this command it WILL slow your backtest down. If you use this command it is recommended that you limit your symbol list as much as possible as the system will have to prepare daily bars for any symbol you qualify. You must call it in on_start.

There... background and caveats completed!

If you go to the main documentation page and search for 'load' you will find the command quite quickly. To save you the effort here it is...

md.bar.load_daily_bars(days=730)

# Used in on_start to expand the number of daily bars from base of 1 year, use multiples of 365 (so 730 gets you 502/503 bars, 1095 gets you 755/756 bars). Caution, this WILL SLOW DOWN YOUR SCRIPT!

Hope this helps!

kc975943

Thanks for these details.