Skip to content

Releases: zenml-io/zenml

0.45.2

16 Oct 20:16
@strickvl strickvl
0.45.2
c66f06f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.45.2

UPDATE: a breaking update to two of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1' 'sqlmodel==0.0.8'

UPDATE NOTE: 0.45.2 introduced a minor database migration bug. To avoid this, please install 0.45.3 directly. If you already installed 0.45.2 and are running into database migration issues (especially if you're using ExternalArtifact objects), please contact us directly on Slack or (alternatively) use this migration script to fix your broken database (script should be executed directly on the server if you use remote installation).

This release replaces 0.45.0 and 0.45.1, and fixes the major migration bugs that were in
that yanked release. Please upgrade directly to 0.45.2 and avoid upgrading to
0.45.0 to avoid unexpected migration issues.

Note that 0.45.0 and 0.45.1 were removed from PyPI due to an issue with the
alembic versions + migration which could affect the database state. This release
fixes that issue.

If you have already upgraded to 0.45.0 please let us know in Slack and we'll happy to assist in rollback and recovery.

This release introduces a major upgrade to ZenML, featuring a new authentication mechanism, performance improvements, the introduction of the model control plane, and internal enhancements.

New Authentication Mechanism (#4303)

Our improved authentication mechanism offers a more secure way of connecting to the ZenML server. It initiates a device flow that prompts you to log in via the browser dashboard:

zenml connect --url <YOUR_SERVER_URL>

This eliminates the need for explicit credential input. The previous method (zenml connect --url <URL> --username <USERNAME> --password <PASSWORD>) remains operational but is less recommended due to security concerns.

Critical This change disrupts existing pipeline schedules. After upgrading, manually cancel and reschedule pipelines using the updated version of ZenML.

For more information, read about the device flow in our documentation.

Performance enhancements (#3207)

Internal API adjustments have reduced the footprint of ZenML API objects by up to 35%. This will particularly benefit users with large step and pipeline configurations. Further reductions will be implemented in our next release.

Model Control Plane debut (#5648)

ZenML now includes a preliminary version of the model control plane, a feature for registering models and their metadata on a single ZenML dashboard view. Future releases will provide more details. To test this early version, follow this example.

Breaking Changes

  • Environment variables ZENML_AUTH_TYPE and ZENML_JWT_SECRET_KEY have been renamed to ZENML_SERVER_AUTH_SCHEME and ZENML_SERVER_JWT_SECRET_KEY, respectively.
  • All ZenML server-issued JWT tokens now include an issuer and an audience. After the server update, current scheduled pipelines become invalidated. Reset your schedules and reconnect all clients to the server to obtain new tokens.
  • UnmaterializedArtifact has been relocated to zenml.artifacts. Change your import statement from from zenml.materializers import UnmaterializedArtifact to from zenml.artifacts.unmaterialized_artifact import UnmaterializedArtifact.

Deprecations

  • zenml.steps.external_artifact.ExternalArtifact has moved to zenml.artifacts.external_artifact.ExternalArtifact.

And the rest:

New Contributors

Full Changelog: 0.44.3...0.45.0

Contributors

strickvl, stefannica, and 11 other contributors
Assets 2
Loading

0.45.1 [YANKED]

15 Oct 13:52
@htahir1 htahir1
0.45.1
d1e5fb6
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.45.1 [YANKED]

0.45.1

Update: It's one of those days! This release is also yanked due to alembic migration issues. We are working hard to solve it. In the meanwhile please use 0.44.3

This release replaces 0.45.0, and fixes the major migration bugs that were in that yanked release. Please upgrade directly to 0.45.1 and avoid upgrading to 0.45.0 to avoid unexpected migration issues.

What's changed

This release introduces a major upgrade to ZenML, featuring a new authentication mechanism, performance improvements, the introduction of the model control plane, and internal enhancements.

New Authentication Mechanism (#4303)

Our improved authentication mechanism offers a more secure way of connecting to the ZenML server. It initiates a device flow that prompts you to log in via the browser dashboard:

zenml connect --url <YOUR_SERVER_URL>

This eliminates the need for explicit credential input. The previous method (zenml connect --url <URL> --username <USERNAME> --password <PASSWORD>) remains operational but is less recommended due to security concerns.

Critical This change disrupts existing pipeline schedules. After upgrading, manually cancel and reschedule pipelines using the updated version of ZenML.

For more information, read about the device flow in our documentation.

Performance enhancements (#3207)

Internal API adjustments have reduced the footprint of ZenML API objects by up to 35%. This will particularly benefit users with large step and pipeline configurations. Further reductions will be implemented in our next release.

Model Control Plane debut (#5648)

ZenML now includes a preliminary version of the model control plane, a feature for registering models and their metadata on a single ZenML dashboard view. Future releases will provide more details. To test this early version, follow this example.

Breaking Changes

  • Environment variables ZENML_AUTH_TYPE and ZENML_JWT_SECRET_KEY have been renamed to ZENML_SERVER_AUTH_SCHEME and ZENML_SERVER_JWT_SECRET_KEY, respectively.
  • All ZenML server-issued JWT tokens now include an issuer and an audience. After the server update, current scheduled pipelines become invalidated. Reset your schedules and reconnect all clients to the server to obtain new tokens.
  • UnmaterializedArtifact has been relocated to zenml.artifacts. Change your import statement from from zenml.materializers import UnmaterializedArtifact to from zenml.artifacts.unmaterialized_artifact import UnmaterializedArtifact.

Deprecations

  • zenml.steps.external_artifact.ExternalArtifact has moved to zenml.artifacts.external_artifact.ExternalArtifact.

And the rest:

New Contributors

Full Changelog: 0.44.3...0.45.0

Contributors

strickvl, stefannica, and 11 other contributors
Assets 2
Loading

0.45.0 [YANKED]

12 Oct 11:46
@safoinme safoinme
0.45.0
82da7ef
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.45.0 [YANKED]

IMPORTANT THIS RELEASE HAS BEEN YANKED

  • Due to an unforseen alembic migration issue, this release has been yanked.0.45.1 is on its way to being released to fix the bugs that have been uncovered
  • If you have already upgraded to 0.45.0 please let us know in Slack and we'll happy to assist in rollback and recovery
Assets 2
Loading

0.44.3

26 Sep 22:57
@safoinme safoinme
0.44.3
18824ef
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.44.3

UPDATE: a breaking update to two of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1' 'sqlmodel==0.0.8'

This release, introduces SkyPilot, a new VM orchestrator for ZenML that lets users run pipelines on their choice of cloud provider VMs, offering a GPU option without Kubernetes or serverless orchestrators. This release also brings bug fixes and improvements, including a streamlined 'connect' command, interactive configuration for 'zenml stack deploy,' and enhanced documentation covering SageMaker, GCP, and service connectors with MFA.

New Orchestrator: SkyPilot (#1765)

This release introduces a new orchestrator called SkyPilot. SkyPilot is a VM orchestrator
that can be used to run ZenML pipelines on a VM of choice in one of the three supported
cloud providers. It is an excellent choice for users who want to run ZenML pipelines on a GPU
instance, but don't want to use Kubernetes or serverless orchestrators like SageMaker.

Fixes and Improvements

This release fixes several bugs and improves the user experience of the CLI and the
documentation. The most notable changes are:

  • The new connect command that allows connecting all stack components within a stack to a
    service connector with a single command.
  • Adding an interactive flow to the zenml stack deploy command that allows users to
    configure their stack in a guided manner.
  • Add documentation on how to debug the SageMaker orchestrator, how to get started with
    a quick cloud stack on GCP, and documentation on using service connectors with
    enabled MFA.

What's Changed

  • Add support for empty API token in Kubernetes service connector. by @stefannica in #1808
  • Use the container registry credentials to build images with the local image builder by @stefannica in #1804
  • Fix CI by @fa9r in #1809
  • Add documentation on how to debug the SageMaker orchestrator by @fa9r in #1810
  • Bump rich and uvicorn by @jlopezpena in #1750
  • SageMaker: Enable configuring authentication credentials explicitly by @fa9r in #1805
  • Fix: ZenML DB migrations don't run if zenml is installed in path with spaces by @stefannica in #1815
  • Fix mlflow 'run_name' variable overwriting by @iraadit in #1821
  • Add SECURITY.md file for vulnerability disclosures. by @strickvl in #1824
  • Add MFA limitation to service-connectors docs by @safoinme in #1827
  • Improve zenml stack describe to show mlstacks outputs by @strickvl in #1826
  • Documentation to get started with a quick cloud stack on GCP by @AlexejPenner in #1807
  • Fix missing text in git repo docs by @strickvl in #1831
  • Handle irregular plural of code_repository for error message by @strickvl in #1832
  • Connect stack to a service account by @safoinme in #1828
  • SkyPilot Integration with VM Orchestrators by @htahir1 in #1765
  • Add interactive CLI flow for zenml stack deploy by @strickvl in #1829
  • Add README file for helm chart by @strickvl in #1830
  • Fix slack environment variable in in generative_chat example README by @bhatt-priyadutt in #1836

New Contributors

Full Changelog: 0.44.2...tes

Contributors

iraadit, strickvl, and 7 other contributors
Assets 2
Loading

0.44.2

11 Sep 15:21
@fa9r fa9r
0.44.2
df95a26
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.44.2

This release contains updates for some of the most popular integrations, as well as several bug fixes and documentation improvements.

Minor Default Behavior Changes

  • The default page size for zenml list commands was reduced to 20 (from 50) to speed up the runtime of such commands.
  • Simultaneous connection to local and remote ZenML servers is no longer possible since this caused several unexpected behaviors in the past.

Integration Updates

  • The mlflow integration now supports the newest MLflow version 2.6.0.
  • The evidently integration now supports the latest Evidently version 0.4.4.
  • The SageMaker orchestrator of the aws integration now supports authentication via service connectors.

What's Changed

Full Changelog: 0.44.1...0.44.2

Contributors

strickvl, stefannica, and 7 other contributors
Assets 2
Loading

0.44.1

31 Aug 20:42
@safoinme safoinme
0.44.1
2ca04b9
Compare
Choose a tag to compare
0.44.1

This release brings various improvements over the previous version, mainly focusing on using the newly refactored mlstacks package, ZenML's logging module and the changes in our analytics.

IMPORTANT NOTICE

Note: 0.44.0 was removed from Pypi due to an issue with the alembic versions which could affect the database state. A branch occurred in the versions: 0.42.1 -> [0.43.0, e1d66d91a099] -> 0.44.0. This release fixes the issue.

The primary issue arises when deploying version 0.44.0 using a MySQL backend. Although the alembic migration executes all tasks up to 0.44.0, the alembic version represented in the database remains at 0.43.0. This issue persists irrespective of the measures taken, including trying various versions after 0.43.0.

This imbalance leads to failure when running a second replica migration because the database's state is at 0.44.0 while the alembic version remains at 0.43.0. Similarly, attempts to run a second replica or restart the pod fail as the alembic tries to migrate from 0.43.0 to 0.44.0, which is not possible because these changes already exist in the database.

Please note: If you encounter this problem, we recommend you roll back to previous versions and upgrade to 0.43.0. If you still experience difficulties, please join our Slack community at https://zenml.io/slack. We're ready to help you work through this issue.

What's Changed

Full Changelog: 0.43.0...0.44.1

Contributors

strickvl, stefannica, and 5 other contributors
Assets 2
Loading

0.44.0 (YANKED [SEE RELEASE NOTES])

29 Aug 15:19
@bcdurak bcdurak
0.44.0
0c792d8
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.44.0 (YANKED [SEE RELEASE NOTES])

IMPORTANT NOTICE

This release has been removed from pypi due to an issue with the alembic versions which could affect the database state. A branch occurred in the versions: 0.42.1 -> [0.43.0, e1d66d91a099] -> 0.44.0.

The primary issue arises when deploying version 0.44.0 using a MySQL backend. Although the alembic migration executes all tasks up to 0.44.0, the alembic version represented in the database remains at 0.43.0. This issue persists irrespective of the measures taken, including trying various versions after 0.43.0.

This imbalance leads to failure when running a second replica migration because the database's state is at 0.44.0 while the alembic version remains at 0.43.0. Similarly, attempts to run a second replica or restart the pod fail as the alembic tries to migrate from 0.43.0 to 0.44.0, which is not possible because these changes already exist in the database.

Please note: If you encounter this problem, we recommend that you rollback to previous versions and then upgrade to 0.43.0. If you still experience difficulties, please join our Slack community at https://zenml.io/slack. We're ready to help you work through this issue.

Older Notes

This release brings various improvements over the previous version, mainly
focusing on the usage of newly refactored mlstacks package, ZenML's logging
module and the changes in our analytics.

What's Changed

Full Changelog: 0.43.0...0.44.0

Contributors

strickvl, stefannica, and 4 other contributors
Assets 2
Loading

0.43.0

15 Aug 13:26
@avishniakov avishniakov
0.43.0
97c0667
Compare
Choose a tag to compare
0.43.0

This release brings limited support for Python 3.11, improves quickstart experience with the fully reworked flow, enhances the user experience while dealing with ZenML docs, offers new extended templates for projects and fixes GCP connector creation issue.

Limited support for Python 3.11

This release adds limited support for Python 3.11.
The following integrations are currently not tested and may behave unexpectedly with Python 3.11:

  • gcp
  • kubeflow
  • tekton

This is because:

  • GCP packages that support Python 3.11 are not compatible with KFP 1
  • Upgrade to KFP 2 is blocked by the fact that Tekton doesn't have any release compatible with KFP 2 yet (#1697)

Breaking Changes

A minor breaking change in CLI for zenml init:

  • previously supported flag --starter
  • new flag --template-with-defaults
  • behavior remains the same - flag is responsible for usage of default settings in the template

What's Changed

New Contributors

Full Changelog: 0.42.0...0.43.0

Contributors

strickvl, stefannica, and 9 other contributors
Assets 2
Loading

0.42.1

21 Jul 19:04
@stefannica stefannica
0.42.1
dc7e716
Compare
Choose a tag to compare
0.42.1

0.42.1

This is a minor release that fixes a couple of minor issues and improves the
quickstart example.

Breaking Changes

Disable Implicit Auth Methods for Service Connectors by Default

The implicit authentication methods supported by cloud Service Connectors method
may constitute a security risk, because they can give users access to the same
cloud resources and services that the ZenML Server itself is allowed to access.

For this reason, the default behavior of ZenML Service Connectors has been
changed to disable implicit authentication methods by default. If you try to
configure any of the AWS, GCP or Azure Service Connectors using the implicit
authentication method, you will now receive an error message.

To enable implicit authentication methods, you have to set the
ZENML_ENABLE_IMPLICIT_AUTH_METHODS environment variable or the ZenML helm
chart enableImplicitAuthMethods configuration option to true.

What's Changed

  • Disable implicit auth methods for service connectors by default by @stefannica in #1704
  • New quickstart by @strickvl in #1692
  • Set MLflow configuration as environment variables before deployment subprocess by @safoinme in #1705
  • Fix Migration Guide Links by @fa9r in #1706

Full Changelog: 0.42.0...0.42.1

Contributors

strickvl, stefannica, and 2 other contributors
Assets 2
Loading

0.42.0

20 Jul 10:59
@fa9r fa9r
0.42.0
2213b18
Compare
Choose a tag to compare
0.42.0

This release brings major user experience improvements to how ZenML logs are managed and displayed, removes Python 3.7 support, and fixes the Python 3.10 PyYAML issues caused by the Cython 3.0 release.

Improved Logging UX

The log messages written by ZenML when running pipelines or executing ZenML CLI commands are now more concise and easier to digest and the log message colors were adjusted to be more intuitive. Additionally, all log messages, including custom prints to stdout, now show up as step logs in the dashboard.

Breaking Changes

Python 3.7 Support Dropped

Python 3.7 reached its end of life on on June 27th, 2023. Since then, several MLOps tools have stopped supporting Python 3.7. To prevent dependency issues with our integrations and other open-source packages, ZenML will also no longer support Python 3.7 starting from this release.

Dependency and Integration Version Updates

ZenML now requires PyYAML 6 since older versions are broken under Python 3.10. Subsequently, the following integrations now require a higher package version:

  • Kubeflow now requires kfp==1.8.22
  • Tekton now requires kfk-tekton==1.7.1
  • Evidently now requires evidently==0.2.7 or evidently==0.2.8

What's Changed

New Contributors

Full Changelog: 0.41.0...0.42.0

Contributors

strickvl, schustmi, and 7 other contributors
Assets 2
Loading