0.5.0 Carbontracker integration

👨‍💻 Compute and display detailed carbon emission and energy consumption data for dry-runs.

📊 Report on data pipeline carbon footprint

♻️ Analyze and identify carbon and energy intensive workloads

We have integrated the enRichMyData greenR tool Carbontracker into SIM-PIPE by containerizing Carbontracker and exposing it as an API service in the local Kubernetes cluster that SIM-PIPE utilizes. SIM-PIPE calls this service to compute detailed carbon emissions and energy consumption of the individual steps of a data pipeline dry-run.

What's Changed

• Chart version update by @goranbs in #184
• Carbontracker by @goranbs in #187
• Carbontracker by @goranbs in #188

Full Changelog: 0.4.8...0.5.0

0.4.8: Merge pull request #183 from DataCloud-project/bugfix-181

🐛 bugfix

• Avoids a problem in the frontend where large artifacts (>2GB) displayed a signed int error in the artifact browser. The issue was simply avoided by not returning the filesize when querying minio.

0.4.7: Merge pull request #169 from DataCloud-project/frontend-hotfix

fix: conrollerUrl == graphqlUrl running on port 8087

0.4.6

Fixes a problem where the frontend does not connect to the backend due to the port of the controller not being correctly set.

In addition introduces a wip -page to display resource consumption and predicted scaling of resource consumption

projects/analytics3

release-0.4.5

Visualize resource consumption and prediction of resource consumption. This is work in progress that builds on the recent additions to the backend API endpoints for computing resource consumption and perform regression on this data to enable extrapolation to provide predictions of resource consumption.

release-0.4.4

bug fix: registry-hotfix

• There returned graphql data for a workflowtemplate differed from the typescript interface definition and created an error when loading the workflowTemplate when adding a registry secret to a project workflowTemplate.

release-0.4.3

Bug fix: update avg memory consumption when selecting a step
Beauty fix: max-height for logs box and graphs.

release-0.4.2

Minor updates to frontend how overall metrics are displayed for a dry-run.

release-0.4.1

• Display the current cpu and network usage in stead of the cumulative
• Error message from workflow displayed in the top of the logs
• Max height for logs before they become scrollable in y-direction

0.4.0: Merge pull request #153 from DataCloud-project/fr-141

• Upload and manage input artifacts in the artifacts browser. These artifacts can be used as inputs for dry runs.
• Upgraded to skeletonlabs v2
• Upgraded to svelte-kit v2
• Bug fixes