MT4 Backtesting F.A.Q.

[Barry]

mcboogs,
Thanks very much for the trouble you took to write up this FAQ, I do a lot of back-testing in MT4, and I found it very valuable. Good work!

[mcboogs]

Everyone,

Sorry, I haven't posted in here in a while, been spending a lot of time on some other forums. Anyway I'm glad that there has been a positive response and that the FAQ has helped some people. Anyway, I have some responses to various posts and a few more things to mention, so I'll get started in my respones.


Billworld:
I was reading if not the same thread a similar one about the MT4/MT3 data, as well as various other discussions about backtesting in general. Unfortunately this is something we are just going to have to deal with, as many data sources include the spread as increased to the high and decreased to the low of bars. Ultimately this results in somewhat skewed data, PARTICULARLY ON SMALLER TIMEFRAMES.

Unfortunately, there is not much that can be done, and this reinforces my statement in the FAQ about backtesting being less accurate and reliable on smaller timeframe strategies and ideas than on longer ones. On a trading system that has very narrow SL and TP levels, this could results in very misleading results.

-----------------------------------------------------

Maratha:
Your post was from a while ago, but I thought I'd respond anyway. There is no confirmation that the period converter has finished, but it finishes very shorty after it has run. Check your history center to see the number of bars in different timeframes to be sure.

-----------------------------------------------------

Other stuff:

Something that I feel I should mention is to be aware of time differences in different data sources. If you are importing your data from some other source besides Alpari, the time of the bars may be different. There are some expert advisors out there that many of you might download and look at [for example Hans123 is very popular right now]. You need to be doubly sure that the time of day on the data you are using is the same as the time of day such an EA is programmed to run on. Otherwise your tests are obviously going to be dramatically skewed.

Backtesting doesn't have to be just about testing a strategy that you plan on trading. I recommend making use of the optimization features in MetaTrader4 to just explore the behavior of markets so you can learn more while you develop strategies. I have run "backtests" with optimization runs solely to learn about market behavior and see how far away a market is from being truly random. As a simple example, you could quickly and easily write an expert advisor that simply counts the number of times price moved X pips in a Y hour span, then set the optimition features to run with a range of variables, say X=50-100 in a 5 step, and Y is just 1 and 2 hours. Run this through and you can find out how many times price moved substantially in a short time period, then maybe develop a strategy from there. The point is you don't have to necessarily be backtesting a strategy to benefit from the backtester.

MetaTrader4 is an extraordinarily powerful and versatile program for a free one and the optimization feature is not something I discussed at length at all in the FAQ. I imagine someday, I'll take the time to write a FAQ on different uses of MetaTrader's optimization feature, but I don't have the time or patience to get to that just now.

The point of backtesting isn't necessarily to get EXACT results, because that is impossible. What backtesting does specifically is provide you with your average Winner/Loser profit/loss ratio and approximate winning percentage. If your system has a lot of trades under it's belt, these values start to become statistically meaninful. You can take these numbers and determine the approximate mathetmatical expectancy of your system, which is nice to have a feel for before you go running forward with real money on the line. As MetaTrader allows you to program in different money management methods, you can also see firsthand what they can mean towards mathematical expectancy.

Okay, enough for now. Good luck everyone, and happy trading.

[aymensaket2]

Hello There,

Thanks a lot for your F.A.Q. Just a quick question, my EA is set to run on 1H. I imported i minute data from Alpari, and used your method and it works with 90% model quality than you.

Q: should i set the period in the strategy tester to 1M or is 1H Ok, the results would be very different.

Aymen

[Loopy]

Just a quick note about alpari data. I talked to one of their online assistants last night and asked about timezone differences. He said that the alpari databank was GMT+1 in winter and GMT+2 in the summer. Hope this helps!

Jack

[traden4x]

Has anyone been able to confirm or deny the quality of alpari data?

Or does anyone know of a better data solution?

Is there any problems associated with MT4 for doing backtesting?

Thanks,
Carl

[qu2r]

When using the FAQ to put new data into MT4's history, is it necessary to purge the old data?

What would cause a mismatch between H1 data and M30 data (for example, a range where the highest M30 bar is about 23 pips short of the corresponding H1 or H4 bar)? Would re-running the period converter solve that, and if so... would that data need to be cleaned out first?

Thanks!
-Matt

[dr_gaines]

Originally Posted by mcboogs

I decided to write up this little tutorial, because backtesting different systems comes up very often in threads on this forum. There seems to be a lot of confusion about reliability issues and how to go about achieving the most accurate possible results. I am not a programming or trading guru, but I believe I can provide a helpful little FAQ on backtesting using MT4.
Good backtesting is important when considering a system-trading approach, because you want to have some idea of the feasibility of your idea before you go live with it [at least I do]. If you're backtesting with a 50% model quality, eh... you can't really be sure what's going on. If you have a 90% modeling quality, you can have more confidence on how your system actually would have performed.


+=========================+
|MCBoogs' MT4 Backtesting FAQ v1.0 |
+=========================+

Contents:
- Section 1: Is MT4 Backtesting Reliable?
- Section 2: Downloading/Importing/Converting 1M Data
- Section 3: Configuring the Backtester
- Section 4: Other Issues


Section 1: Is MT4 Backtesting Reliable?

This question often gets pretty heated and people even get to the point of flaming each other about it. Backtesting in MT4 can be reliable, but its reliability is contingent upon the data you are backtesting on. Demo account data that is streamed in through a demo account broker has gaps, holes, and is basically not suitable for testing.
When backtesting, you want to use the EVERY TICK MODEL and have accurate 1M data to get the most accurate test possible. The 1M data is important, because the EVERY TICK MODEL uses whatever the smallest available timeframe available is and "fakes" the movement of price within the smallest available bars. Having 1M data allows for the fractal interpolation within bars to occurs only within the very narrow range of 1M bars.
The easiest [and only] solution to this is to use good 1M data. The most complete data you can get [at least for free] is from Alpari's Databank. They have data in MT native format, on the 1M timeframe back through mid 2004. However, setting up the data for use requires some doing.


---------------------------------------------------------------------------


Section 2: Downloading/Importing/Converting 1M Data

(1) You need to modify MT4 to allow for more bars. Go into the Tools Menu, then go to Options [or just hit C+O]. Go into the charts tab and put in 9999999999999 for bars in history. MT4 will default to whatever it's maximum is.

[Note: The reason MT4 has a limited bar count to begin with is because more bars (particularly when used in backtesting models) means MT4 is going to eat up more HD space.]

(2) Download the 1M data from Alpari's Databank in whatever currency[ies] you're going to test on.

(3) Import the data into MT4 using the History Center. Go to Tools => History Center [or push F2]. Make sure you import it in proper currency and in the M1 timeframe. You don't want EURUSD data being important into USDCAD for instance.

(4) Convert the data using the period converter script included in MT4 [you only have 1M bars right now]. You have to open offline charts to do this.
-Go to the File Menu, then Open Offline, select the 1M data of the currency you need to convert. A chart will pop up with that data.
-Then drag & drop the period_converter script onto the offline chart. The ExtPeriodMultiplier int that you can modify is the multiplier you are applying to the chart. So making it 5, will convert 1M data into 5M data.
-For simplicity's sake, you need to run the period converter with the following integers to get all the backtesting timeframes: 5,15,30,60,240, and 1440.

[NOTE: you can also convert 1M data to timeframes not native to MT4 if you want to do some indicator analysis or something on another timeframe.]

Congratulations, you have now imported and converted data into MT4. Now, for the sake of illustrating one of my earlier points, open up a currency you have imported data on. Look at the difference in the bars from the downloaded data as opposed to data streamed in from a Demo broker [So, if you downloaded 1M data from July 04 to August 05, look at the chart at August 05's end and September 05's beginning]. You will notice that the bars (on every time frime if you have converted them properly) from your downloaded time period will be more complete.


------------------------------------------------------------------


Section 3: Configuring the Backtester

Now that you've succesfully imported complete data, there are a few more things you need to do to run a reliable backtest.

(1) Check the recalculate option the next time you run a backtest, because you need the backtester to utilize your shiny new happy data (which it won't do unless you tell it). Anytime you import new data, you need to recalculate (I recalculate every few tests just to feel safe, maybe its a reflection of internal confidence problems, but that's for another FAQ).

(2) Check the use date option and set the date range only over a time period where you have good reliable data. This way you're only backtesting the good stuff. It will be reflected in the modeling quality percentage.

(3) Make sure the model is set to EVERY TICK. If you're not, all this hard work we just did was for nothing. I addressed why we do this earlier in the FAQ.


------------------------------------------------------------------------


Section 4: Other Issues

MT4 is a work in progress, sometimes there are strange bugs that crop up in backtesting. However, usually when you think you have a bug on your hands, there is something wrong with your code. I can't emphasize enough how important debugging is. If you have problems, check your code first because it's probably the problem. If you really think you have a legit bug on your hands, post it to the MT4 forums.
Because you are not actually backtesting on every tick that happened [you are dealing with an interpolation on 1M data], it is still not a perfect reproduction of what actually happened in the markets. Because of this, 1M and 5M scalping EAs that get in and out of trades really quickly will run into some problems just because of this limitation. The longer timeframe you are trading on, the less likely your testing is to be hampered by this.
Well, that's all I can think of now. I read this over, I think I made everything clear and have the steps outlined correctly. If you notive a mistake, let me know, and I'll correct it in my next version of the MT4 Backtesting FAQ.


Acknowledgements:
I learned most of what I know about MT4 and trading in general from these forums and others like it. Thanks to all the people who contribute that have provided me with useful tidbits of information. There are too many names [and some of them are weird, have lots of numbers in them, etc.] to list, but a serious thanks to all the StrategyBuilder contributors out there.
Best of luck in the markets everyone.

Will there ever be a simulator. A simulator would help a lot.

Dr. Gaine$

[eng_zidan]

Originally Posted by mcboogs

I decided to write up this little tutorial, because backtesting different systems comes up very often in threads on this forum. There seems to be a lot of confusion about reliability issues and how to go about achieving the most accurate possible results. I am not a programming or trading guru, but I believe I can provide a helpful little FAQ on backtesting using MT4.
Good backtesting is important when considering a system-trading approach, because you want to have some idea of the feasibility of your idea before you go live with it [at least I do]. If you're backtesting with a 50% model quality, eh... you can't really be sure what's going on. If you have a 90% modeling quality, you can have more confidence on how your system actually would have performed.


+=========================+
|MCBoogs' MT4 Backtesting FAQ v1.0 |
+=========================+

Contents:
- Section 1: Is MT4 Backtesting Reliable?
- Section 2: Downloading/Importing/Converting 1M Data
- Section 3: Configuring the Backtester
- Section 4: Other Issues


Section 1: Is MT4 Backtesting Reliable?

This question often gets pretty heated and people even get to the point of flaming each other about it. Backtesting in MT4 can be reliable, but its reliability is contingent upon the data you are backtesting on. Demo account data that is streamed in through a demo account broker has gaps, holes, and is basically not suitable for testing.
When backtesting, you want to use the EVERY TICK MODEL and have accurate 1M data to get the most accurate test possible. The 1M data is important, because the EVERY TICK MODEL uses whatever the smallest available timeframe available is and "fakes" the movement of price within the smallest available bars. Having 1M data allows for the fractal interpolation within bars to occurs only within the very narrow range of 1M bars.
The easiest [and only] solution to this is to use good 1M data. The most complete data you can get [at least for free] is from Alpari's Databank. They have data in MT native format, on the 1M timeframe back through mid 2004. However, setting up the data for use requires some doing.


---------------------------------------------------------------------------


Section 2: Downloading/Importing/Converting 1M Data

(1) You need to modify MT4 to allow for more bars. Go into the Tools Menu, then go to Options [or just hit C+O]. Go into the charts tab and put in 9999999999999 for bars in history. MT4 will default to whatever it's maximum is.

[Note: The reason MT4 has a limited bar count to begin with is because more bars (particularly when used in backtesting models) means MT4 is going to eat up more HD space.]

(2) Download the 1M data from Alpari's Databank in whatever currency[ies] you're going to test on.

(3) Import the data into MT4 using the History Center. Go to Tools => History Center [or push F2]. Make sure you import it in proper currency and in the M1 timeframe. You don't want EURUSD data being important into USDCAD for instance.

(4) Convert the data using the period converter script included in MT4 [you only have 1M bars right now]. You have to open offline charts to do this.
-Go to the File Menu, then Open Offline, select the 1M data of the currency you need to convert. A chart will pop up with that data.
-Then drag & drop the period_converter script onto the offline chart. The ExtPeriodMultiplier int that you can modify is the multiplier you are applying to the chart. So making it 5, will convert 1M data into 5M data.
-For simplicity's sake, you need to run the period converter with the following integers to get all the backtesting timeframes: 5,15,30,60,240, and 1440.

[NOTE: you can also convert 1M data to timeframes not native to MT4 if you want to do some indicator analysis or something on another timeframe.]

Congratulations, you have now imported and converted data into MT4. Now, for the sake of illustrating one of my earlier points, open up a currency you have imported data on. Look at the difference in the bars from the downloaded data as opposed to data streamed in from a Demo broker [So, if you downloaded 1M data from July 04 to August 05, look at the chart at August 05's end and September 05's beginning]. You will notice that the bars (on every time frime if you have converted them properly) from your downloaded time period will be more complete.


------------------------------------------------------------------


Section 3: Configuring the Backtester

Now that you've succesfully imported complete data, there are a few more things you need to do to run a reliable backtest.

(1) Check the recalculate option the next time you run a backtest, because you need the backtester to utilize your shiny new happy data (which it won't do unless you tell it). Anytime you import new data, you need to recalculate (I recalculate every few tests just to feel safe, maybe its a reflection of internal confidence problems, but that's for another FAQ).

(2) Check the use date option and set the date range only over a time period where you have good reliable data. This way you're only backtesting the good stuff. It will be reflected in the modeling quality percentage.

(3) Make sure the model is set to EVERY TICK. If you're not, all this hard work we just did was for nothing. I addressed why we do this earlier in the FAQ.


------------------------------------------------------------------------


Section 4: Other Issues

MT4 is a work in progress, sometimes there are strange bugs that crop up in backtesting. However, usually when you think you have a bug on your hands, there is something wrong with your code. I can't emphasize enough how important debugging is. If you have problems, check your code first because it's probably the problem. If you really think you have a legit bug on your hands, post it to the MT4 forums.
Because you are not actually backtesting on every tick that happened [you are dealing with an interpolation on 1M data], it is still not a perfect reproduction of what actually happened in the markets. Because of this, 1M and 5M scalping EAs that get in and out of trades really quickly will run into some problems just because of this limitation. The longer timeframe you are trading on, the less likely your testing is to be hampered by this.
Well, that's all I can think of now. I read this over, I think I made everything clear and have the steps outlined correctly. If you notive a mistake, let me know, and I'll correct it in my next version of the MT4 Backtesting FAQ.


Acknowledgements:
I learned most of what I know about MT4 and trading in general from these forums and others like it. Thanks to all the people who contribute that have provided me with useful tidbits of information. There are too many names [and some of them are weird, have lots of numbers in them, etc.] to list, but a serious thanks to all the StrategyBuilder contributors out there.
Best of luck in the markets everyone.

thank you man.....
I mported the 1M data and Converted it to 5,15,30,60,240, and 1440
but the model quality syill 50%

what can i do??????