Some basic questions

[hrdrq]

Hi, thank you for working on this project and hope you are doing well.
I'm reading the source code and trying to know how this EA works.
There are some questions I'm curious about:

1. Is "optimization" finding the most profitable strategies per timeframe?

I had read Input parameters and Optimization hints, and I think the key parameters are the strategies of M1 ~ H8?
So the backtest and optimization is to find the best "combination" of strategies?
(It looks like many parameters were removed from v2.0.0?)

2. Why there is no option to set multiple strategies for one timeframe?

For example, maybe some strategies do well for M15, we can set all of them to Strategy_M15, but it's not possible in "Inputs".
Is the reason that the "Inputs" is not flexible in MT4/MT5, so we can only choose one?

3. How to change strategy(indicator)'s parameters?

For example, in Stg_MA, there is INPUT int MA_Indi_MA_Period = 40;. How can we change it when run the EA or do backtest?
Is modifying the source code and recompile the only way?
If I want to do optimization by changing strategies's parameters, how can I do it?

Thank you again :)

[kenorb]

> 1. Is "optimization" finding the most profitable strategies per timeframe?

Depends on your optimization setup, but that's the main idea, to optimize by finding the most profitable strategy per given timeframe.
The quickest way (less interactions) is to optimize one strategy/timeframe at a time.
Then test them how they play together, as adding more strategies at once can influence overall results in different ways.
So it's healthy setup to have various strategies on each timeframe (so they won't trade the same way) to spread the risk.

It looks like many parameters were removed from v2.0.0

Since EA v2, the parameters' structure has been simplified for the easier use.
Having 300+ parameters was confusing and difficult to use, and to complex to optimize.

> 2. Why there is no option to set multiple strategies for one timeframe?

It was possible in v1.080 and earlier. For EA v2, for simplicity, only one strategy can be assigned per timeframe.
However, there is a plan to add meta strategies which can run multiple strategies as one.

For example:

• Best strategy - which can dynamically select strategy which is the most profitable for the current week.
• Double strategy - which can run 2 strategies but act as one.
• And so on.

Using this approach, you're not limited to selecting fixed strategies per timeframe, but they're going to be dynamically selected by the algorithm by meta/wrapper strategy which is going to manage them on the given timeframe.

Is the reason that the "Inputs" are not flexible in MT4/MT5, so we can only choose one?

Yes, partially due to MT4/MT5 platform input limitations. There is no multi-choice select.

> 3. How to change strategy(indicator)'s parameters?

Most of the strategies are optimized by default for EURUSD pair and their indicator input param are not exposed for the easier use. In MT5 there is a collapsible input groups, however in MT4 all params are flat. So by exposing 300+ params, people will have problems finding the right params and it'll make the EA almost impossible to optimize.

To customize indicator params, here are few suggestions:

• Find the strategy's repository of interest (it comes with executable EA file) and optimize it individually, then propose the better optimized parameters (which are shared across all timeframes).
• Compile EA31337 by enabling __input__ in src/include/common/mode.h, all parameters are going to be visible.

Further more I'm looking forward to find some easy to use solution, such as:

• Loading params from the config file.
• Loading values from the param's JSON string (e.g. {"MA_Indi_MA_Period": 40, ...}) (cons: not possible to optimize).
• Combining them into few parameters (instead of hundreds), such as one select list which param to override, and another one with which value (pros: possible to optimize).

If above is not sufficient, then you can propose a feature to include all indicator params for EA's Advanced by default. Unless you've another ideas.

Hope that helps.

[hrdrq]

I see. Thank you. Optimization is more complicated than I thought.
My idea is to make a document so we can know what params they are and what ranges they. Program will convert double to integer if needed, and we choose the most important 5 params to optimize.

I had a config/ approach for each strategy
How does it work while optimizing?

[kenorb]

I had a config/ approach for each strategy.

How does it work while optimizing?

This was optimized by set of automatic action scripts. So each strategy had 2 compilation modes (modified in config/define.h): with config (load overridden input values from config/ for the given timeframe) and without (load default input params).
For the optimizing process, default values are loaded (without config mode), then overridden by prepared SET file (loaded from sets/optimize/) which was generated from the Jinja template (copy of config values per timeframe, e.g. Indi_RSI_Period.set.j2). Once better values has been found, tester script would modify params back to config/ dir via another template file (see: config/template.j2).

[hrdrq]

Thank you! It's so cool!!
Is my understanding correct?

1. Use Github Action to run indicator optimization in each strategy repo's optimize-indi branch
2. The core docker image is ea31337/ea-tester
3. The config and result is stored in Github Action artifacts
4. Updated code of config/**.h will be pushed to dev-optimize-${{ env.PF }}pf-${{ env.DD }}dd-${{ env.TT }}tt branch
5. You'll check the result and merge the branch to optimize-indi branch if the result is better
6. If we want to use it in EA, we have to uncomment out #define __config__ in src/include/common/mode.h and recompile

[kenorb]

Yes, correct. To optimize specific part, a code can be pushed into specific branch (e.g. git push origin HEAD:optimize-foo) then the optimization starts, and proposed results are sent back to another branch for manual review. Test are split per different components, so it won't take too much time due to time limits of the builds.
However I don't recommend using it as it is, especially in mass runs, because GitHub can block your account for too many prolonged tests (it happened for another account). The main org seems to be on some kind of whitelist, however I don't want to risk it and run it periodically. Secondly, the tests and results are slow (especially for MT4), so local run in MT5 with 32 cores (supports multithreading) runs much quicker (currently EA Tester doesn't support MT5, but this can be added, it's partially done).
The recommended setup is to setup your own/self-hosted runners/workers (it's still free) and can be setup at /settings/actions/runners in the repo settings. However EA-Tester Docker container doesn't play well with Actions on self-hosted runners. Due to Docker permission limitations for volumes, it errors when UID of the user is different from the host's user UID (it requires some amount of effort to find the workaround). So I've started refactor of tester scripts using different (easier) approach using Ansible playbooks (repo: EA31337/ansible-role-mt-runner plus other). So metatrader role configures platform on the host, mt-runner runs it. Pros, easier to understand (instead of complex shell scripts), it can support MT5, run outside of Docker, even on Windows, and it can provision Docker container ready-to-use. Even it worked on Colab setup (you need to uncomment the code), but it also needs some work. All this can work fine, however my focus is somewhere else for now, and I'll come back to this new tester at some point.
Based on above, using __config__ approach, I'm planning to discontinue using it (and disintegrate from the code) due to complexity, limitations, and basically nobody-using-and-understanding-it and find another way.

[hrdrq]

Thank you for the detailed explanation again! It helps to understand the whole system.
I really respect your developing skills. I wish I had ability to help something