Introduce PTES and TES constraints #1546

TomKae00 · 2025-02-18T08:07:57Z

Changes proposed in this Pull Request

This pull request introduces the following adjustments to the prepare_sector_network and solve_network scripts, replacing PR #1444 (which did not work with the HiGHS solver):

Added PTES.
Introduced a constraint to enforce the energy-to-power ratio (etpr).
Implemented a constraint to ensure that the charger and discharger of the thermal energy stores are sized equally.

The following config data has been used:

scenario:
  clusters:
    - 6

planning_horizons:
  - 2030

foresight: overnight
countries: ['DE', 'DK']

clustering:
  temporal:
    resolution_sector: 12h

Constraints

The etpr ratio constraint and the constraint enforcing equal sizing of the charger and discharger are working as intended, as shown in the figure below. In particular, the size of the store divided by 150 equals the size of both the charger and the discharger.

The annual charging and discharging process for an exemplary PTES store is illustrated in the figure below:

System Costs

System costs have increased slightly with the newly introduced constraints. This is logical given the larger capacities required when the charger and discharger can no longer be chosen independently.

Checklist

[ x] I tested my contribution locally and it works as intended.
[ x] Code and workflow changes are sufficiently documented.
[ x] Changed dependencies are added to envs/environment.yaml.
[ x] Changes in configuration options are added in config/config.default.yaml.
[ x] Changes in configuration options are documented in doc/configtables/*.csv.
[ x] Sources of newly added data are documented in doc/data_sources.rst.
[ x] A release note doc/release_notes.rst is added.

for more information, see https://pre-commit.ci

…nto ltes_stores # Bitte geben Sie eine Commit-Beschreibung ein, um zu erklären, warum dieser # Merge erforderlich ist, insbesondere wenn es einen aktualisierten # Upstream-Branch mit einem Thema-Branch zusammenführt. # # Zeilen, die mit '#' beginnen, werden ignoriert, # und eine leere Beschreibung bricht den Commit ab.

amos-schledorn

Looks great! Just a few small comments. When the CI runs through, I'll test this locally.

scripts/solve_network.py

for more information, see https://pre-commit.ci

…nto ltes_stores

TomKae00 · 2025-02-25T10:42:55Z

@amos-schledorn thanks for the review! I've implemented your suggestions. Moreover, i added error handling to ensure that the TES constraints are only applied when intended.

TomKae00 · 2025-02-27T10:02:12Z

Hi @fneum and @amos-schledorn,

Currently, the CI is failing when running:

snakemake make_summary_perfect --configfile config/test/config.perfect.yaml

Reason for CI Failure:
In the master, the water tank charger and discharger are included in costs.csv with an efficiency of 0.9. Because of this, they can be used as heat vents, allowing energy to be released from the system.

With this PR, the efficiency of both the central/decentral chargers and dischargers has been set to 1.0 according to the DEA catalogue. As a result, they can no longer act as heat vents, which prevents energy from being released. Additionally, chargers are now directly tied to the store size via the energy-to-power ratio, meaning they cannot be used independently.

Possible Solution:
Since energy can no longer be released through the chargers/dischargers, the system costs are now higher. A potential solution could be to allow heat vents to be built for all heat systems, as it is done for urban central heating systems in prepare_sector_network.py line 2437.

Is there a specific reason why heat vents are currently only allowed for urban central heating? Or are there any concerns against extending this option to all systems?

Copilot

PR Overview

This PR implements new PTES and TES constraints to ensure a fixed energy-to-power ratio and equal sizing for TES charger/discharger pairs in the network, replacing the previous implementation incompatible with the HiGHS solver. Key changes include:

Updating the prepare_sector_network.py to dynamically reference cost parameters for water tank and water pit components.
Adding two new TES constraint functions (energy-to-power ratio and charger ratio) in solve_network.py.
Updating the configuration file to include new color mappings for water pits in visualizations.

Reviewed Changes

File	Description
scripts/prepare_sector_network.py	Updates to water tanks/pits component cost/efficiency and related linking.
scripts/solve_network.py	New TES constraint functions added and integrated into network setup.
config/config.default.yaml	New plotting color options added for water pits.

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

scripts/solve_network.py:886

The TES charger ratio constraint assumes a one-to-one ordering between charger and discharger indices. Ensure that these indices are explicitly paired to enforce that each TES unit’s charger and discharger are correctly matched.

lhs = ( n.model["Link-p_nom"].loc[indices_charger_p_nom_extendable] - n.model["Link-p_nom"].loc[indices_discharger_p_nom_extendable] * eff_discharger )

scripts/solve_network.py

…storage

for more information, see https://pre-commit.ci

…price-water-tank-chargers

…er ratio constraints and energy power constraints

…nto ltes_stores

…o ltes_stores

…ilt with correct baseyear

for more information, see https://pre-commit.ci

scripts/prepare_sector_network.py

scripts/solve_network.py

for more information, see https://pre-commit.ci

Copilot

PR Overview

This pull request introduces PTES and TES constraints to the network optimization scripts by enforcing an energy‐to‐power ratio for thermal energy stores and equal sizing between the charger and discharger linked to TES units. Key changes include:

Adding new constraints and variables in prepare_sector_network.py to support water tanks and water pits.
Implementing TES energy-to-power ratio and TES charger ratio constraints in solve_network.py with linopy for expression merging.
Updating configuration settings in config.default.yaml to include cost and plotting parameters for water pits.

Reviewed Changes

File	Description
scripts/prepare_sector_network.py	Adds PTES-related constraints and extends the modeling of water tanks and water pits.
scripts/solve_network.py	Introduces TES energy-to-power ratio and charger ratio constraints and a new linopy import.
config/config.default.yaml	Updates cost and plotting settings to support new water pits parameters and associated components.

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

scripts/prepare_sector_network.py:2732

[nitpick] There is an inconsistency in how storage keys are referenced between water tanks and water pits. Consider constructing the key for water pit storage using heat_system.central_or_decentral to maintain consistency, if applicable.

lifetime=costs.at["central water pit storage", "lifetime"],

Copilot · 2025-03-07T13:26:44Z

scripts/solve_network.py

+        if not tes.replace("-", " ").split(" ")[:5] == charger.split(" ")[:5]:
+            # e.g. "DE0 0 urban central water tanks charger" -> ["DE0", "0", "urban", "central", "water"]


[nitpick] The string-based matching of charger and TES names using split and slice may be brittle. Consider refactoring this logic to use a more robust pairing mechanism (e.g. a helper function that parses and compares structured components) and explicitly sort the indices to ensure correct pairing.

Suggested change

if not tes.replace("-", " ").split(" ")[:5] == charger.split(" ")[:5]:

# e.g. "DE0 0 urban central water tanks charger" -> ["DE0", "0", "urban", "central", "water"]

if parse_name_components(tes) != parse_name_components(charger):

# e.g. "DE0 0 urban central water tanks charger" -> ("DE0", "0", "urban", "central", "water")

Valid comment, but possible issues are addressed by throwing an error when respective string matching fails.

Copilot · 2025-03-07T13:26:44Z

scripts/solve_network.py

+    for charger, discharger in zip(
+        indices_charger_p_nom_extendable, indices_discharger_p_nom_extendable
+    ):


[nitpick] The reliance on splitting strings to match charger and discharger links may lead to unexpected mismatches. It is advisable to standardize and explicitly sort both indices or use a dedicated matching function to ensure consistent pairing.

Suggested change

for charger, discharger in zip(

indices_charger_p_nom_extendable, indices_discharger_p_nom_extendable

):

def standardize_and_sort_indices(indices):

return sorted(indices, key=lambda x: " ".join(x.split(" ")[:5]))

sorted_charger_indices = standardize_and_sort_indices(indices_charger_p_nom_extendable)

sorted_discharger_indices = standardize_and_sort_indices(indices_discharger_p_nom_extendable)

for charger, discharger in zip(sorted_charger_indices, sorted_discharger_indices):

Valid comment, but possible issues are addressed by throwing an error when respective string matching fails.

cpschau

Hey @TomKae00,
only had a couple of questions, where I am not sure if they need to be addressed with new changes in the code. Did you discuss the addition of the "energy to power" column with @fneum?

cpschau · 2025-03-07T13:42:00Z

scripts/prepare_sector_network.py

            )

+            n.links.loc[


Was not aware of the practice to store custom attributes in network files. Is this OK @fneum? Alternatively, I would suggest to read out energy-to-power ratio from costs file in solve_network.

cpschau · 2025-03-07T13:42:54Z

scripts/prepare_sector_network.py

+                    lifetime=costs.at["central water pit storage", "lifetime"],
+                )
+
+                n.links.loc[


cpschau · 2025-03-07T13:43:57Z

scripts/solve_network.py

+            "No valid extendable charger links or stores found for TES energy to power constraints."
+        )
+
+    energy_to_power_ratio_values = n.links.loc[


Here the parameter would have to be accessed using the costs file as an additional data input to the rule.

cpschau · 2025-03-07T13:56:01Z

scripts/solve_network.py

+            )
+
+    eff_discharger = n.links.efficiency[indices_discharger_p_nom_extendable].values
+    lhs = (


Why only the discharger efficiency is considered here?

cpschau · 2025-03-07T14:01:16Z

scripts/solve_network.py

+            n.links.index.str.lower()
+            .str.contains("pits charger|tanks charger", case=False, na=False)


Why lower() and contains(case=False) have to be used?

cpschau · 2025-03-07T14:01:32Z

scripts/solve_network.py

+            n.buses.index.str.lower()
+            .str.contains(
+                r"urban central heat|urban decentral heat|rural heat",
+                case=False,


Why lower() and contains(case=False) have to be used?

TomKae00 and others added 15 commits February 12, 2025 16:52

introduce etpr for TES using stores

55f4d0e

Master into branch

6f684d3

add constraints

2f67385

copleted implementation of constraints for TES stores

8dedc43

Merge branch 'master' into ltes_stores

2a44a18

updated release notes

ca2d24c

adjusted mock snakemake default

3576274

library change

e40fcf6

merge master into branch

a16db94

update release notes

b46ec98

[pre-commit.ci] auto fixes from pre-commit.com hooks

ee3b5f9

for more information, see https://pre-commit.ci

Delete .DS_Store

84ebe30

change retrieval of etpr values in etpr_constraint

df3fc81

changed TES constraints check based on the configs

599867d

amos-schledorn requested changes Feb 19, 2025

View reviewed changes

scripts/solve_network.py Show resolved Hide resolved

scripts/solve_network.py Show resolved Hide resolved

scripts/solve_network.py Outdated Show resolved Hide resolved

scripts/solve_network.py Outdated Show resolved Hide resolved

TomKae00 and others added 7 commits February 19, 2025 19:03

constraint checks

69e0809

enhance error handling for the tes constraints

68740c6

Merge branch 'master' into ltes_stores

cf9c4e4

small changes

b538ee7

[pre-commit.ci] auto fixes from pre-commit.com hooks

ef05a0e

for more information, see https://pre-commit.ci

changed mock snakemake back to default settings

ec0f962

Merge branch 'ltes_stores' of https://github.com/TomKae00/pypsa-eur i…

7933ec2

…nto ltes_stores

amos-schledorn self-requested a review February 25, 2025 10:48

TomKae00 mentioned this pull request Feb 26, 2025

Change heat_vent config.default setting #1556

Merged

7 tasks

Merge branch 'master' into ltes_stores

8ab2e84

fneum requested a review from Copilot March 3, 2025 09:26

Copilot AI reviewed Mar 3, 2025

View reviewed changes

scripts/solve_network.py Show resolved Hide resolved

amos-schledorn and others added 15 commits March 3, 2025 18:41

feat: remove unnecessary central_heat_vent config

665e407

feat: pass heat vent marginal cost to prepare_sector_network

0af1d58

refactor: rename marginal_cost_storage to marginal_cost_home_battery_…

b0536c2

…storage

docs: update configtables

677c555

[pre-commit.ci] auto fixes from pre-commit.com hooks

a041def

for more information, see https://pre-commit.ci

docs: update release notes

b3f1e80

Merge remote-tracking branch 'origin/price-water-tank-chargers' into …

a6524c4

…price-water-tank-chargers

Merge branch 'master' into price-water-tank-chargers

5ba0af2

add suggestions to ensure that charger and discharger align for charg…

e3721e7

…er ratio constraints and energy power constraints

Merge branch 'ltes_stores' of https://github.com/TomKae00/pypsa-eur i…

ad71120

…nto ltes_stores

Merge remote-tracking branch 'upstream/price-water-tank-chargers' int…

96f8b5d

…o ltes_stores

add lifetime to PTES and TTES charger/discharger, so that they are bu…

ca85eff

…ilt with correct baseyear

[pre-commit.ci] auto fixes from pre-commit.com hooks

ba87ae5

for more information, see https://pre-commit.ci

Merge branch 'master' into ltes_stores

631b648

feat: add small penalty on ptes charging

6bc6123

amos-schledorn requested changes Mar 7, 2025

View reviewed changes

amos-schledorn added 5 commits March 7, 2025 12:13

feat: update fixed cost to capital cost str

f02155a

feat: remove unnecessary passing of obsolete config

1c1624f

feat: remove unneeded config passing

02495b5

feat: remove unneeded config passing

5c46a65

feat: remove unneeded config passing

e2eedb7

amos-schledorn reviewed Mar 7, 2025

View reviewed changes

scripts/solve_network.py Outdated Show resolved Hide resolved

amos-schledorn and others added 4 commits March 7, 2025 12:15

Update scripts/solve_network.py

5b32b8e

Update solve_network.py

ec51d01

feat: re-add TES necessary parameters

6c5128c

[pre-commit.ci] auto fixes from pre-commit.com hooks

bcc0334

for more information, see https://pre-commit.ci

amos-schledorn requested review from cpschau and Copilot March 7, 2025 13:25

Copilot AI reviewed Mar 7, 2025

View reviewed changes

cpschau reviewed Mar 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce PTES and TES constraints #1546

Introduce PTES and TES constraints #1546

TomKae00 commented Feb 18, 2025

amos-schledorn left a comment

TomKae00 commented Feb 25, 2025

TomKae00 commented Feb 27, 2025

Copilot AI left a comment

Copilot AI left a comment

Copilot AI Mar 7, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

amos-schledorn Mar 7, 2025

Copilot AI Mar 7, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

amos-schledorn Mar 7, 2025

cpschau left a comment

cpschau Mar 7, 2025

cpschau Mar 7, 2025

cpschau Mar 7, 2025

cpschau Mar 7, 2025

cpschau Mar 7, 2025

cpschau Mar 7, 2025

		if not tes.replace("-", " ").split(" ")[:5] == charger.split(" ")[:5]:
		# e.g. "DE0 0 urban central water tanks charger" -> ["DE0", "0", "urban", "central", "water"]

-    for charger, discharger in zip(
-        indices_charger_p_nom_extendable, indices_discharger_p_nom_extendable
-    ):
+    def standardize_and_sort_indices(indices):
+        return sorted(indices, key=lambda x: " ".join(x.split(" ")[:5]))
+    sorted_charger_indices = standardize_and_sort_indices(indices_charger_p_nom_extendable)
+    sorted_discharger_indices = standardize_and_sort_indices(indices_discharger_p_nom_extendable)
+    for charger, discharger in zip(sorted_charger_indices, sorted_discharger_indices):

		n.links.index.str.lower()
		.str.contains("pits charger\|tanks charger", case=False, na=False)

Introduce PTES and TES constraints #1546

Are you sure you want to change the base?

Introduce PTES and TES constraints #1546

Conversation

TomKae00 commented Feb 18, 2025

Changes proposed in this Pull Request

Constraints

System Costs

Checklist

amos-schledorn left a comment

Choose a reason for hiding this comment

TomKae00 commented Feb 25, 2025

TomKae00 commented Feb 27, 2025

Copilot AI left a comment

Choose a reason for hiding this comment

PR Overview

Reviewed Changes

Copilot AI left a comment

Choose a reason for hiding this comment

PR Overview

Reviewed Changes

Copilot AI Mar 7, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Copilot AI Mar 7, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cpschau left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment