Outdated for Iceborne

[bull55]

Hello, please update this wonderful tool for iceborne.

[TanukiSharp]

Thank you very much for your consideration, but according to the complexity Iceborn added, the lack of time on my side, and the lack of interest for Armory from the community (I know a few like it but...) I'm not considering to update it, at least not at the moment.

I'm sorry for the ones who enjoyed using it. Pull requests are welcome however.

[ChaosSaber]

I may be interested in doing it. At least as long as i can convince myself not to play too much MH.
I would appreciate some tips and tricks on where to start and a short description of your project structure.

[TanukiSharp]

Hello @ChaosSaber and thank you for your interest and proposal for help.

Armory has been designed to lose couple UI (shell), data, and data processing.

So roughly what will be of interest for you will mainly be the solver, represented by the ISolver interface in the MHArmory.Search.Contracts project, and the SolverBase class in the MHArmory.Search project (Default folder). The class SolverBase has been designed to share as much as possible between World and Iceborne.
You can have a look at the Solver class, which is the default MH World implementation, to help you understand what you can and can't do.

The Documentation folder of the same project has a README.md (https://github.com/TanukiSharp/MHArmory/tree/master/MHArmory.Search/Documentation) that explain quite extensively how the solver and solver data integrate in Armory. I would recommend you to read this first.

The Cutoff folder of the same project is another implementation of the ISolver interface, using ISolverData based implementation located in the Incremental folder.

The interface ISolverData is an abstraction of data for the solver, so they (solvers and solver data) are interchangeable. You can inherit from ISolverData to generate fake data, especially fake jewels, 4-slots sized with 2 skills. Or you can just use the default SolverData implementation and add fake jewels and fake equipment (armor pieces with size 4 slots) after the SolverData has populated lists from files. The reason is, there is currently no data for Iceborne in Armory.

Eventually, to test your solver, it has to be registered for Armory to be able to use it. Have a look at the Extensions class in the MHArmory project. (https://github.com/TanukiSharp/MHArmory/blob/master/MHArmory/Extensions.cs)
Then, when you run Armory, click the Extensions menu and you should see the extensions window, displaying your solver and/or solver data.

The design of classes may sound strange sometimes, but keep in mind the ISolver is integrated in a heavily parallelized process, and abstracts that complexity away from the solver implementation, as much as possible.

One of the biggest complexities is to implement this algorithm in a allocation-free way. Keep in mind that there are millions of combination to check, and that number growing very quickly when elected equipment grows, so please try to keep the implementation with the lowest memory pressure possible.
Mostly, I use a lot of object pooling and structures when possible. You can use the ObjectPool<T> class in the MHArmory.Core project so you don't have to reinvent the wheel. This class is massively used so you will not have hard time to find out how it works.

You can have a look at this folder: https://github.com/TanukiSharp/MHArmory/tree/master/docs/misc/netcore
It contains images that compare perfs between .NET Framework and .NET Core (I'm almost sure there were a page to give context to those images, can't find it :/). You can see that on .NET Core there is only one GC hit. .NET Framework has a lot because the underlying concurrent collection used have been implemented long ago without memory allocation consideration in mind, unfortunately.

Let me know if you have more questions, let's use this thread to discuss.

[ChaosSaber]

Thanks for the long explanation. I read through the links and everything is well understandable.
I already began yesterday "fixing" the MhwDbDataSource. Seemed to me the best way to learn the internal data structures and was a good reintroduction into C#. And as apositive side effect i will have (hopefully) correct iceborne data.

[TanukiSharp]

My policy now about data is extraction from game's master data. No more website scrapping or conversion from fans' DB here and there. You may have seen MHArmory.MhwDbDataSource or MHArmory.AthenaAssDataSource projects but they are not used anymore.

So just to make sure you will not waste time and efforts on something I may reject. If you really want to contribute to Iceborne master data extraction, I recommend you have a look at this project: https://github.com/TanukiSharp/MHWMasterDataUtils

Thank you very much for your help anyway :)

[ChaosSaber]

I support that policy. But i have seen this as a completely different project and therefore put it at the end for now. I would have probably used this project: https://github.com/Synthlight/MHW-Editor as a base. It seems to be well maintained and is already updated for Iceborne. But it currently does not yet have an API. But this can be changed, as long as the developper wants it.
I don't think the time i invested in the MhwDBbDataSource was wasted even though i knew in the end i won't use it anymore. A hands-on example is usually better to understand then pure theory. At least for me.

[TanukiSharp]

I had a look at this project, for the moment they only extract raw data (basically transforming binary to key/value JSON) without sanitation, and it is only translation data. Well at least that's what I can see from the JSON data they pushed to the repository.

[ChaosSaber]

Small Progressupdate:
I have a somewhat working solution. It can find sets but will also miss some. This is due to it prefering the new level 4 decos. This can lead to cases where only level 1 slots remain but no deco which can fit in there. I have not yet found a reliable solution. I have not yet tested the opposite to prefer the old decos.
I added some tests for the current PC meta builds to quickly verify my solutions. Should be added more in the future.

Things i noticed during testing:

• SolverData needs to filter out more parts. On average it has 500 million combinations.
• When searching for high rank sets a lot of skills are marked as unavailable.
• When a deco has the same name as the skill, then searching for that skill will also show all the skills from multi skill decos.
• the maximum useful deco count for multilevel skills is incorrect in the deco override.
• it seems i broke the default search somehow (ups)
• i disabled all other Solver and Solverdata for now until verified they are still working. (I know one is crashing because of the assumption a skill can only be on one deco)

Things left todo:

• Importing Decos from a save file
• Importing the gamedata as data source
• A proper solution for generic decos. (multi skill decos where only one skill is searched by the user)
• performance improvements (currently only around 1.8-1.9 million searches per second)

I will not have much time the next two weeks, so I will probably not continue on this before then.

[ChaosSaber]

I uploaded an Alpha version of Armory to my fork:https://github.com/ChaosSaber/MHArmory/releases/tag/v0.1-alpha.1
This version has a fully functional search. It is a bit slower than before (only 1.5 million sets per second), but can find more sets. This is currently only tested when you have enough decorations of everything. But I'm confident it will also work with limited decos. Besides that I fixed the default search for LR and HR, everything stands as in my previous post.
If you want to test it out, feel free to provide feedback if something is not working as expected.

My next step will be to improve the pre selection so that less sets have to be searched through. After that trying to improve the performance of the search.

[ChaosSaber]

Hello, sorry for the long silence. Sadly i have to admit, that i wont update it any further for now. Due do the current situation i work from home and i don't feel like doing anything work related in my free time. I opened Pull request #23 with my changes up to now. It is in a working state and can find sets. The major problem is, that it will take a really long time, if you search for a set with a lot of different skills. And I'm talking here about 10 minutes and upwards for most of the iceborne searches. You either have to wait that long or manually limit the armor for the search.
So sorry again. I will probably pick it up again once the current situation gets better or I'm accustomed to home office.

[TanukiSharp]

Hello @ChaosSaber

Thank you for your hard work, I will check your PR as soon as I can.

Quick question though, in a previous comment (that I didn't answer, sorry about that), you mentioned to have broken the other solvers, does that mean you modified code in Armory core ?

I think you were not supposed to and the solver and solver data APIs should provides everything needed. Anyway, I will check your PR, but I can see 100 files modified, I'm a bit afraid.

[ChaosSaber]

The new iceborne data broke them. For example at least one assumed a skill can only be at a single decoration or a decoration can only have one skill and similar stuff. At the moment I wrote this, I probably didn't dig deep enough yet to find the cause.
Such an assumption was also made in the skill selection screen for the availability, which was the reason most skills where marked unavailable in HR and LR.
I "fixed" it for the default search by giving it the expected input data for LR and HR (i just removed the MR stuff). Probably works for the other two too, but didn't test it. Only the default search didn't crash when given iceborne data.
The ComparerSolverData was my try at rewriting/updating the ElectionModel class for iceborne. Currently it's only an empty shell and a bunch of unittests.

hui 100 files changed. I didn't really realised i changed that much. There are roughly 15 data files i changed. Additionally i copied them over to the unittest project, so that makes 45 (inclusive the weapon data i didn't change). I should have done that properly from the beginning and moved them to the datasource project and the datasource project should have put them at the appropriate places during installation. Yes i know, unittests should create their own data and don't rely on external data, like I have done it for the ComparerSolverData unittests.
But that still leaves with around 50 files, which is .... a lot.