Algorithms for ill-conditioned problems

[lthurner]

In meshed networks, multiple solution might exist, and different solutions might be found depending on the starting values (see #110). Aside from smart choice of initial solution (see #113), other algorithms than Newton-Raphson might be better suited to deal with ill-conditioned problems, such as Levenberg-Marquart or Holomorphic embedding.

GridCal already has implementations of LM and HELM. Do these implementations work and can we adopt them for pandapower (also due to the license)?

@ascheidl you have run some tests, can you report your findings?

[lthurner]

@WinfriedL: Having a simple test case would be helpful for testing, are there simple test cases for this problem, where different restults are found for different starting solutions? Or does this problem only occurr in larger, more complex grids?

[WinfriedL]

In principle this problem exists in all grids, but usually the voltage magnitudes of the 'wrong' solution should be far enough away from an initial flat start of 1 pu. I am not sure what part of my testgrid leads to the proximity of both solutions.
However, it should be possible to replicate the behaviour with a two-bus grid and a very low initial value for the voltage.

[FlorianShepherd]

I implemented a version of the Iwamoto Newton Rapshon. This may help to improve convergence. The algorithm can be used by choosing
runpp(net, algorithm="iwamoto_nr")

[WinfriedL]

Great addition!

Unfortunatly it currently does not help in my testcase. And I believe it cannot work for any network with PV buses.
Because dV = dVm * exp(1j * dVa) includes PV buses, and for those dVm is zero, there is always at least some dV==0 in
https://github.com/lthurner/pandapower/blob/463311d24012ed7270312e27ba612fd267a965b5/pandapower/pf/iwamoto_multiplier.py#L15-L19
so one gets iwa_multiplier=1.0 all the time.
Or am I missing something?

[WinfriedL]

I have looked again, and either I am missing something completly (which is perfectly possible), or this can never work as expected. Since the voltage of the slack bus is in the matrix there should always be an element with dV==0.

[WinfriedL]

I have looked at the original Iwamoto papers and I think the Implementation was wrong. Or at least I did not understand how the implementation was related to Iwamoto paper. It somehow looked like a strange implementation of the formulas given by Milano (which also differ from the Iwamoto formulas)

I tried to implement the original Iwamoto:
https://github.com/WinfriedL/pandapower/blob/10c7bca902a95dcdfd25aa7243c7c8563adf9c34/pandapower/pf/iwamoto_multiplier.py#L22-L41

It seems to work, and in most cases improves the convergence behaviour. I have not yet done speed comparisons.

It would be great if someone can look at the implementation for some obvious errors and maybe test it. It is not yet ready for a pull request, there is at least some cleanup necessary.

[WinfriedL]

I have checked all formulas again, and I think they are correct. I have alos done some speed and convergence comparison. The iwamoto step is slower than a pure NR step, as expected. However, in most of my tests it converges faster and needs less iterations. So depending on the net and the tolerance, it is either a bit slower (if it needs the same amount of iterations) or faster (if it needs less iterations).

And at least in one testcase, the standard Newton-Raphson converged in an unstable solution, while the Iwamoto converged in the stable solution. But that might be coincidence and is probably not a general rule.

[lthurner]

That sounds great, thanks a lot!

[SteffenMeinecke]

@WinfriedL: Having a simple test case would be helpful for testing, are there simple test cases for this problem, where different restults are found for different starting solutions? Or does this problem only occurr in larger, more complex grids?

If you still need small test cases:

• Data for the widespread 3 Ill-conditioned power systems (11 bus and 43 bus systems from iwamoto and 13 bus system from customs head) can be obtained at https://ieeexplore.ieee.org/document/4111178
• In addition, I found another paper that uses these 3 networks, as well as another 25 node network: https://www.sciencedirect.com/science/article/pii/0142061590900327 Maybe that's helpful too? (I suspect there will be some more and recent papers and test cases)

[WinfriedL]

Thanks, we will look at the 11 bus system. However, it will take some time because the data format is not very "user friendly". Do you have any tools to convert a Ybus matrix into a pandapower network?

[WinfriedL]

I tried to create the 11 bus testcase. Some observations:

• I am not sure if I got all signs right, please have a look if it makes sense to you
• It does not converge with standard NR or Iwamoto NR. So not a big improvement so far...
• After reading some more I stumbled over the problem that the Iwamoto algorithm is designed for power flow calculations in rectangular coordinates, and not polar coordinates like pypower/pandapower does. There is a discussion at the end of the paper on this. But so far I have not really understood why it makes a fundamental difference for the algorithm, and how the proposed modification helps.
• All calculations by Iwamoto seem to be on the edge on the possible precision at the time, so I am not sure how valid the results really are... Maybe a newer testcase would have been better...

[lthurner]

After GridCal has released an updated version of the HELM algorithm (see https://gridcal.readthedocs.io/en/latest/theory/power_flow/holomorphic_embedding.html), I looked into this once more. Unfortunately, the iwamoto 11 bus testcase still doesn't converge with the HELM. But I looked into the source once more, and I have doubts the iwamoto network as it is now in pandapower is correct:

• The admittance values have no unit given in the paper, but I am assuming they are in per unit?
• The base power is not explicetly given in the paper (it is given for the 13bus network, but not for the 11bus). If the admittances are in per unit, the values depend strongly on the base power. The network as it is in pandapower has a base power of 1 MVA, but that does not seem to be defined in the paper?
• The bus powers are given in pu in the paper and translated to pandapower with the assumpation of 1 MVA base power. Once again, the base power is not given, so it is not clear if 0.128pu power means 0.128, 12.8MW or 128MW. From the response of the reviewers (or discussers), it seems that it is supposed to be 12.8MW, which would mean the base power is 100 MVA, but that is a bit speculative.
• The signing system for bus powers is not given in the paper, so it is not clear if its load or generation.
• Results for the voltage are also not given to allow validation

With that much information missing, I don't see how a network can be compiled with certainty... I would vote to remove the iwamoto network from pandapower as long as we are not sure it is correct

[wangzhenassd]

Hi all,
are there any advances in this thread regarding the implementation of HELM algorithm? Does it still make sense to pandapower since iwamoto is already available?

[AnkurArohi]

I tried to follow this thread, I believe its extremely intresting.
Do we have iwamoto impl also in pandapower?

[vogt31337]

Hi all, are there any advances in this thread regarding the implementation of HELM algorithm? Does it still make sense to pandapower since iwamoto is already available?

I have built a prototype, but it doesnot produce correct results, I think I made something wrong while converting the datastructures: #2067