Script that automatically runs backtesting/optimization?

[enjoysmath]

I have experience with automating things using Python. For example I've written automators for both binary.com and nadex.com for binary options (which we haven't made use of yet as binary options seems harder than plain forex).

The GUI itself is automatable using pywinauto_recorder and python scripting skills in general.

So I was wondering wouldn't it be nice to run optimization steps say for a week-to-a-month long on a loop? It then automatically would choose the best strategy for you, but if that doesn't work very well we could always put the probablistic change-of-strategy information into an HMM or at least a probability matrix.

You can then run this loop once on all of history and have the correct strategy statistics for working with in the present. Or that is one application of this automatic strategy tester script - to gather statistics of strategies that will probably work.

They made no way to programmatically control the strategy tester so some hacking is needed or simply GUI automation. Later we can replace the easily-made GUI automation code with a hack / patch of the MetaTrader executable (but that is more difficult in my oppinion, so we should first see if this automation leads to good EA's, statistically before doing that).

[enjoysmath]

Actually, there is a way of controlling the Strategy Tester through a setup file:

MQL4/5 forums

[kenorb]

I have experience with automating things using Python. For example I've written automators for both binary.com and nadex.com for binary options (which we haven't made use of yet as binary options seems harder than plain forex).

The GUI itself is automatable using pywinauto_recorder and python scripting skills in general.

...

They made no way to programmatically control the strategy tester so some hacking is needed or simply GUI automation. Later we can replace the easily-made GUI automation code with a hack / patch of the MetaTrader executable (but that is more difficult in my opinion, so we should first see if this automation leads to good EA's, statistically before doing that).

You don't need any GUI automation hacks, MetaPlatform support headless configs, but it's not well documented. Also in that link that you posted.

Check Backtesting using Docker page (could be a bit outdated).

You can then run this loop once on all of history and have the correct strategy statistics for working with in the present.

Currently problem is if you optimize in MT4, some strategies may not work as well as in MT5, and other way round. At some point we'll need to compare candle by candle why it's different and fix it.

So I was wondering wouldn't it be nice to run optimization steps say for a week-to-a-month long on a loop? It then automatically would choose the best strategy for you, but if that doesn't work very well we could always put the probablistic change-of-strategy information into an HMM or at least a probability matrix.

It's already done using GitHub Actions for MetaTrader 4, see:

See:

• .github/workflows/optimize-tf.yml (there are some shell and Python scripts involved)
• which is loading sets from sets/optimize/Lite/tf (also for other versions).
• It's using EA31337/EA-Tester project for the headless testing

Notes:

• The test results from it are read from JSON, processed and best optimized settings are shown.

However I don't currently set best results back to EA files, because:

• It runs for short period of time so the results are not relevant much.
• Short tests using GHA (GitHub Actions) are free, but slow.
• GHA tests are limited to time, so you can't run beyond 4h and free runners aren't designed to that kind of tests (GH can block you if you're a free user and abusing their services with optimization tests on free/shared runners), the only way to setup this is to use self-hosted runners (then it's acceptable). But having them it's additional costs of maintaining the servers which are just waiting for next tests for weeks (which are usually make sense only before the next release). People can do it for themself by forking the repo and set their own runners.
• Results from few months are useless, the optimization should be run for at least a year (but it takes time and beyond the time limit of GHA runners).
• Currently EA Tester supports MT4 which is slow, MT5 support can be done, but it's not yet.
• Taking MT4 optimization results can make it worser in MT5, so I prefer to do optimization test in MT5 (which is much quicker and tick results are more reliable), then backport to MT4.
• So it's not worth it yet for few strategies (which timeframe is better).
• For now I prefer to set strategies to average best (I do 2-3 yearly optimization in MT5 manually before the release), and people can further pick their bests according to their market data.
• Optimizing timeframe is only a small set, for Advanced there are some tests for each filters (see: sets/optimize/Advanced).

The process can be improved further more once it's more reliable. If you check the existing scripts and have further ideas, let me know.

Or that is one application of this automatic strategy tester script - to gather statistics of strategies that will probably work.

Picking EA's strategy best is only a small portion of it, as each strategy has their own set of settings to optimize as well (period, filters, and so on) which are hidden by default (but can be enabled by __input__ compiler property). So each strategy repo has their own optimization sets in place ready to use (however I'm not using this process currently, as it's overcomplicating things).

[Harry040]

How to use optimization sets file, when optimize strategy using mt5 plateform

[kenorb]

SET files for MT5 are in different format comparing to MT4. You've to generate them manually, however Docker tester supports only MT4 for now.