Update evidently integration to support NumPy 2.0 #4243

htahir1 · 2025-11-24T09:19:38Z

Updated the evidently integration to version >=0.5.0, which supports NumPy 2.0. This resolves the compatibility issue where users could not install packages requiring NumPy 2.0+ alongside ZenML's evidently integration.

Changes:

Updated evidently version constraint from ">=0.4.16,<=0.4.22" to ">=0.5.0"
Removed the "numpy<2.0.0" constraint that was blocking NumPy 2.0
Updated all imports to use evidently.legacy.* API for backward compatibility
Modified imports in init.py, data_validators, metrics.py, and tests.py
Updated docstring examples to reflect the new import paths

The evidently 0.7+ versions introduced breaking API changes, but by using the evidently.legacy.* compatibility layer, we maintain full backward compatibility with existing ZenML code while gaining NumPy 2.0 support.

Fixes #4242

Describe changes

I implemented/fixed _ to achieve _.

Pre-requisites

Please ensure you have done the following:

I have read the CONTRIBUTING.md document.
I have added tests to cover my changes.
I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
IMPORTANT: I made sure that my changes are reflected properly in the following resources:
- ZenML Docs
- Dashboard: Needs to be communicated to the frontend team.
- Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
- Projects: Depending on the version dependencies, different projects might get affected.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Other (add details above)

Updated the evidently integration to version >=0.5.0, which supports NumPy 2.0. This resolves the compatibility issue where users could not install packages requiring NumPy 2.0+ alongside ZenML's evidently integration. Changes: - Updated evidently version constraint from ">=0.4.16,<=0.4.22" to ">=0.5.0" - Removed the "numpy<2.0.0" constraint that was blocking NumPy 2.0 - Updated all imports to use evidently.legacy.* API for backward compatibility - Modified imports in __init__.py, data_validators, metrics.py, and tests.py - Updated docstring examples to reflect the new import paths The evidently 0.7+ versions introduced breaking API changes, but by using the evidently.legacy.* compatibility layer, we maintain full backward compatibility with existing ZenML code while gaining NumPy 2.0 support. Fixes #4242

github-actions · 2025-11-24T09:40:10Z

Documentation Link Check Results

❌ Absolute links check failed
There are broken absolute links in the documentation. See workflow logs for details
✅ Relative links check passed
_{Last checked: 2025-11-26 14:15:16 UTC}

htahir1 · 2025-11-24T13:18:10Z

@claude fix the linting problems https://github.com/zenml-io/zenml/actions/runs/19629219988/job/56204745370?pr=4243

claude · 2025-11-24T13:25:20Z

Claude finished @htahir1's task —— View job

Fixing Linting Issues ✅

I've successfully fixed the linting problem in the evidently integration.

Tasks:

Fetch and analyze linting error logs
Read modified files to understand current state
Fix identified linting issues
Commit and push fixes

What was fixed:
The issue was in src/zenml/integrations/evidently/__init__.py at lines 77-78. There was a line continuation using a backslash (\) which is discouraged by Python style guides. I refactored it to use implicit line continuation within parentheses instead:

# Before (with backslash)
return cls.REQUIREMENTS + \
    PandasIntegration.get_requirements(target_os=target_os, python_version=python_version)

# After (with implicit continuation)
return cls.REQUIREMENTS + PandasIntegration.get_requirements(
    target_os=target_os, python_version=python_version
)

The fix has been committed and pushed to the branch. The CI should pass now when it re-runs the linting checks.

--- | Branch: claude/zenml-issue-4221-01XfiHWRp7mJ7B5suAJstuGT | Commit: b8f3694

Co-authored-by: Hamza Tahir <[email protected]>

…AJstuGT

strickvl · 2025-11-25T13:39:10Z

@claude /full-review

claude · 2025-11-25T13:45:30Z

Claude finished @strickvl's task —— View job

Full Review of Evidently NumPy 2.0 Update

Performing comprehensive review of PR #4243...

Tasks:

Fetch PR diff and understand all changes
Read and analyze all 6 changed files
Review code correctness and API migration
Check documentation updates
Verify test coverage and potential regressions
Provide detailed review feedback

Summary

This PR successfully updates the Evidently integration from version >=0.4.16,<=0.4.22 to >=0.5.0, enabling NumPy 2.0 support. The changes systematically migrate all Evidently imports to use the evidently.legacy.* API, which provides backward compatibility while supporting newer NumPy versions. The documentation has been appropriately updated to reflect these changes.

✅ Strengths

Clean migration strategy: Using the evidently.legacy.* API is the right approach for maintaining backward compatibility
Comprehensive coverage: All necessary files have been updated consistently
Documentation updated: The documentation examples correctly reflect the new import paths
Linting issues fixed: Previous linting issues with line continuation have been resolved

🔴 Critical Issues

None identified.

🟡 Issues & Suggestions

1. Missing categorical_features mapping in column_mapping.py (src/zenml/integrations/evidently/column_mapping.py:95)

The to_evidently_column_mapping() method is missing the assignment for categorical_features:

def to_evidently_column_mapping(self) -> ColumnMapping:
    column_mapping = ColumnMapping()
    
    # ... other assignments ...
    column_mapping.numerical_features = (
        self.numerical_features or column_mapping.numerical_features
    )
    column_mapping.datetime_features = (
        self.datetime_features or column_mapping.datetime_features
    )
    # Missing: column_mapping.categorical_features assignment!

Recommendation: Add the missing assignment:

column_mapping.categorical_features = (
    self.categorical_features or column_mapping.categorical_features
)

Note: This appears to be a pre-existing bug not introduced by this PR, but it should be fixed.

2. Consider adding version constraint upper bound (src/zenml/integrations/evidently/init.py:57)

The current constraint is "evidently>=0.5.0" with no upper bound. While Evidently provides a legacy API, future major versions might introduce additional breaking changes.

Recommendation: Consider constraining to "evidently>=0.5.0,<1.0" or similar to avoid unexpected breakage, or document why no upper bound is acceptable.

3. Documentation example inconsistency (docs/book/component-guide/data-validators/evidently.md:603)

Line 603 has a subtle bug in the example code - it creates a TestSuite but instantiates Report:

test_suite = TestSuite(metrics=[test_preset.DataQualityTestPreset()])
report.run(  # Should be test_suite.run()

Recommendation: Fix to test_suite.run(...) for consistency.

Note: This is a pre-existing documentation bug not introduced by this PR.

4. Import style inconsistency in documentation (docs/book/component-guide/data-validators/evidently.md)

Some documentation examples show imports like:

from evidently.legacy.report import Report
from evidently.legacy import metric_preset

While others show:

from evidently.legacy.metrics import ColumnRegExpMetric
from evidently.legacy.metric_preset import DataQualityPreset

Recommendation: Consider standardizing on one import style throughout the documentation for better readability. The second style (explicit class imports) is generally clearer.

5. Missing type ignore comments justification (multiple files)

Several imports have # type: ignore[import-untyped] comments. While necessary for untyped Evidently modules, it would be helpful to document if this is expected to be permanent or if Evidently plans to add type stubs.

Recommendation: Consider adding a comment explaining why type ignores are necessary and whether this is expected to change in future Evidently versions.

🟢 Minor Observations

Good practice: The PR correctly updates all docstring examples to use the new API paths (lines 88, 103-107, 161, 177, 184, 268, etc. in documentation)
Consistent pattern: All files follow the same migration pattern, which makes the changes easy to review and maintain
Test preservation: The test imports were correctly updated without changing test logic

Verification Checklist

Based on the CLAUDE.md guidelines:

✅ Changes follow Python style and type hint requirements
✅ Documentation updated to reflect code changes
✅ Imports are consistent across all files
✅ No obvious security issues introduced
⚠️ Tests exist but weren't modified - should verify they still pass with new Evidently version
⚠️ Pre-existing bugs identified (categorical_features mapping, doc example bug)

Recommendations

Before merging: Test that existing pipelines using Evidently continue to work with the new version
Consider fixing: The pre-existing categorical_features bug in column_mapping.py:95
Consider fixing: The documentation bug at line 603
Future work: Consider adding version upper bound to prevent unexpected breakage
CI consideration: Ensure integration tests run with the new Evidently version

Overall Assessment

This is a solid, well-executed migration that achieves its goal of enabling NumPy 2.0 support. The use of the legacy API is appropriate and maintains backward compatibility. The main concerns are minor issues and pre-existing bugs that should be addressed in follow-up work. The PR is ready to merge after verification that integration tests pass.

Recommendation: ✅ APPROVE (after confirming tests pass)

htahir1 · 2025-11-25T15:04:45Z

@claude implement Consider adding version constraint upper bound (src/zenml/integrations/evidently/init.py:57), Documentation example inconsistency (docs/book/component-guide/data-validators/evidently.md:603), and Import style inconsistency in documentation (docs/book/component-guide/data-validators/evidently.md)

claude · 2025-11-25T15:05:18Z

Claude finished @htahir1's task —— View job

Implementing Review Suggestions ✅

Successfully implemented all requested changes from the review.

Tasks:

Read relevant files to understand current state
Add version constraint upper bound for evidently (>=0.5.0,<1.0)
Fix documentation example inconsistency at line 603
Standardize import style in documentation
Run formatter before committing (skipped - dev dependencies not available, but changes don't require formatting)
Commit and push changes

Changes made:

Version constraint (src/zenml/integrations/evidently/init.py:57): Updated evidently version constraint from "evidently>=0.5.0" to "evidently>=0.5.0,<1.0" to prevent unexpected breakage from future major versions while maintaining NumPy 2.0 support.
Documentation bug fixes (docs/book/component-guide/data-validators/evidently.md:603-604):
- Fixed variable name bug where test_suite was created but report.run() was called instead of test_suite.run()
- Fixed parameter name bug where TestSuite was using metrics= instead of the correct tests= parameter
Import style standardization (docs/book/component-guide/data-validators/evidently.md:547-551, 574, 603):
- Standardized all imports to use explicit class imports (e.g., from evidently.legacy.metric_preset import DataQualityPreset) instead of module imports (e.g., from evidently.legacy import metric_preset)
- Updated usage in code examples to match the explicit import style
- This improves code clarity and consistency throughout the documentation

All changes have been committed and pushed to the branch.

View job | Branch: claude/zenml-issue-4221-01XfiHWRp7mJ7B5suAJstuGT | Commit: 187d7af

- Add upper bound to evidently version constraint (>=0.5.0,<1.0) to prevent unexpected breakage from future major versions - Fix documentation bug where test_suite.run() was incorrectly called as report.run() - Fix TestSuite using incorrect parameter name (metrics instead of tests) - Standardize import style to use explicit class imports for better clarity 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Hamza Tahir <[email protected]>

htahir1 · 2025-11-26T10:25:10Z

@strickvl and @bcdurak i now manually tested this locally and it works:

https://cloud.zenml.io/workspaces/zenml-projects/projects/default/runs/37403d64-191d-4acc-a05f-4df6935a491a?tab=overview

github-actions · 2025-11-26T10:27:58Z

ZenML CLI Performance Comparison (Threshold: 1.0s, Timeout: 60s, Slow: 5s)

❌ Failed Commands on Current Branch (claude/zenml-issue-4221-01XfiHWRp7mJ7B5suAJstuGT)

zenml stack list: Command failed on run 1 (exit code: 1)
zenml pipeline list: Command failed on run 1 (exit code: 1)
zenml model list: Command failed on run 1 (exit code: 1)

🚨 New Failures Introduced

The following commands fail on your branch but worked on the target branch:

zenml stack list
zenml pipeline list
zenml model list

Performance Comparison

Command	develop Time (s)	claude/zenml-issue-4221-01XfiHWRp7mJ7B5suAJstuGT Time (s)	Difference	Status
`zenml --help`	1.414554 ± 0.015716	1.448249 ± 0.044967	+0.034s	✓ No significant change
`zenml model list`	Not tested	Failed	N/A	❌ Broken in current branch
`zenml pipeline list`	Not tested	Failed	N/A	❌ Broken in current branch
`zenml stack --help`	1.408841 ± 0.011916	1.404766 ± 0.014107	-0.004s	✓ No significant change
`zenml stack list`	Not tested	Failed	N/A	❌ Broken in current branch

Summary

Total commands analyzed: 5
Commands compared for timing: 2
Commands improved: 0 (0.0% of compared)
Commands degraded: 0 (0.0% of compared)
Commands unchanged: 2 (100.0% of compared)
Failed commands: 3 (NEW FAILURES INTRODUCED)
Timed out commands: 0
Slow commands: 0

Environment Info

Target branch: Linux 6.11.0-1018-azure
Current branch: Linux 6.11.0-1018-azure
Test timestamp: 2025-11-26T12:57:19Z
Timeout: 60 seconds
Slow threshold: 5 seconds

strickvl · 2025-11-26T10:43:29Z

@strickvl and @bcdurak i now manually tested this locally and it works:

Reviewing now

strickvl · 2025-11-26T11:27:29Z

docs/book/component-guide/data-validators/evidently.md

    parameters=dict(
        report_options = [
            (
                "evidently.options.ColorOptions", {


Possibly set this to the .legacy import otherwise maybe it's a bit confusing?

strickvl

Some small nits, mostly about standardising how we refer to things, though it should be backwards compatible.

strickvl · 2025-11-26T11:27:49Z

docs/book/component-guide/data-validators/evidently.md

    parameters=dict(
        test_options = [
            (
                "evidently.options.ColorOptions", {


Same here: possibly set this to the .legacy import otherwise maybe it's a bit confusing?

strickvl · 2025-11-26T11:29:44Z

src/zenml/integrations/evidently/metrics.py

        # Configure an Evidently Metric generator using a full Metric class
        # path
        config = EvidentlyMetric.metric_generator(
            "evidently.metrics.ColumnSummaryMetric", columns=["age", "name"]


Suggested change

"evidently.legacy.metrics.ColumnSummaryMetric", columns=["age", "name"]

I think?

strickvl · 2025-11-26T11:30:07Z

src/zenml/integrations/evidently/metrics.py


        # Configure an Evidently MetricPreset using its full class path
        config = EvidentlyMetric.metric(
            "evidently.metric_preset.DataDriftPreset"


Suggested change

"evidently.legacy.metric_preset.DataDriftPreset"

strickvl · 2025-11-26T11:31:23Z

src/zenml/integrations/evidently/tests.py

        # Configure an Evidently Test generator using a full Test class
        # path
        config = EvidentlyTest.test_generator(
            "evidently.tests.TestColumnShareOfMissingValues", columns=["age", "name"]


Suggested change

"evidently.legacy.tests.TestColumnShareOfMissingValues", columns=["age", "name"]

I think? Feels less confusing this way, even though both should be handled?

strickvl · 2025-11-26T11:32:05Z

docs/book/component-guide/data-validators/evidently.md

 There are several ways you can reference the Evidently tests when configuring `EvidentlyTestConfig` items, similar to how you reference them in an `EvidentlyMetricConfig` object:

 * by class name: this is the easiest way to reference an Evidently test. You can use the name of a test or test preset class as it appears in the Evidently documentation (e.g.`"DataQualityTestPreset"`, `"TestColumnRegExp"`).
 * by full class path: you can also use the full Python class path of the test or test preset class ( e.g. `"evidently.test_preset.DataQualityTestPreset"`, `"evidently.tests.TestColumnRegExp"`). This is useful if you want to use tests or test presets that are not included in Evidently library.


Suggested change

* by full class path: you can also use the full Python class path of the test or test preset class ( e.g. `"evidently.test_preset.DataQualityTestPreset"`, `"evidently.legacy.tests.TestColumnRegExp"`). This is useful if you want to use tests or test presets that are not included in Evidently library.

Same here I think?

htahir1 requested review from bcdurak and strickvl November 24, 2025 09:19

htahir1 mentioned this pull request Nov 24, 2025

Installing package that requires numpy>=2 is incompatible with Evidently Integration. #4242

Open

1 task

htahir1 linked an issue Nov 24, 2025 that may be closed by this pull request

Installing package that requires numpy>=2 is incompatible with Evidently Integration. #4242

Open

1 task

github-actions bot added the internal To filter out internal PRs and issues label Nov 24, 2025

github-actions bot and others added 2 commits November 24, 2025 13:26

Fix line continuation formatting in evidently integration

b8f3694

Co-authored-by: Hamza Tahir <[email protected]>

Merge branch 'develop' into claude/zenml-issue-4221-01XfiHWRp7mJ7B5su…

60e9a1f

…AJstuGT

zenml-io deleted a comment from htahir1 Nov 25, 2025

zenml-io deleted a comment from claude bot Nov 25, 2025

github-actions bot and others added 2 commits November 25, 2025 15:07

Refactor ColumnMapping imports in evidently modules

3f2b99e

strickvl reviewed Nov 26, 2025

View reviewed changes

strickvl requested changes Nov 26, 2025

View reviewed changes

Update evidently legacy options import paths

84a6ec8

htahir1 requested a review from strickvl November 26, 2025 11:33

strickvl approved these changes Nov 26, 2025

View reviewed changes

htahir1 added 3 commits November 26, 2025 12:35

Update Evidently paths to legacy versions

fb5d21b

Update type hints to use string literals for Evidently classes

0b9b952

Add import statements for EvidentlyMetricConfig and EvidentlyTestConfig

9d08e98

htahir1 merged commit ce6c862 into develop Nov 26, 2025
50 of 54 checks passed

htahir1 deleted the claude/zenml-issue-4221-01XfiHWRp7mJ7B5suAJstuGT branch November 26, 2025 16:00


	"evidently.legacy.metrics.ColumnSummaryMetric", columns=["age", "name"]


	"evidently.legacy.tests.TestColumnShareOfMissingValues", columns=["age", "name"]


	* by full class path: you can also use the full Python class path of the test or test preset class ( e.g. `"evidently.test_preset.DataQualityTestPreset"`, `"evidently.legacy.tests.TestColumnRegExp"`). This is useful if you want to use tests or test presets that are not included in Evidently library.

Update evidently integration to support NumPy 2.0 #4243

Update evidently integration to support NumPy 2.0 #4243

Uh oh!

Conversation

htahir1 commented Nov 24, 2025

Describe changes

Pre-requisites

Types of changes

Uh oh!

github-actions bot commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Link Check Results

Uh oh!

htahir1 commented Nov 24, 2025

Uh oh!

claude bot commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixing Linting Issues ✅

Uh oh!

strickvl commented Nov 25, 2025

Uh oh!

claude bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Full Review of Evidently NumPy 2.0 Update

Summary

✅ Strengths

🔴 Critical Issues

🟡 Issues & Suggestions

1. Missing categorical_features mapping in column_mapping.py (src/zenml/integrations/evidently/column_mapping.py:95)

2. Consider adding version constraint upper bound (src/zenml/integrations/evidently/init.py:57)

3. Documentation example inconsistency (docs/book/component-guide/data-validators/evidently.md:603)

4. Import style inconsistency in documentation (docs/book/component-guide/data-validators/evidently.md)

5. Missing type ignore comments justification (multiple files)

🟢 Minor Observations

Verification Checklist

Recommendations

Overall Assessment

Uh oh!

htahir1 commented Nov 25, 2025

Uh oh!

claude bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implementing Review Suggestions ✅

Uh oh!

htahir1 commented Nov 26, 2025

Uh oh!

github-actions bot commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ZenML CLI Performance Comparison (Threshold: 1.0s, Timeout: 60s, Slow: 5s)

❌ Failed Commands on Current Branch (claude/zenml-issue-4221-01XfiHWRp7mJ7B5suAJstuGT)

🚨 New Failures Introduced

Performance Comparison

Summary

Environment Info

Uh oh!

strickvl commented Nov 26, 2025

Uh oh!

strickvl Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

strickvl left a comment

Choose a reason for hiding this comment

Uh oh!

strickvl Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

strickvl Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

strickvl Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

strickvl Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

strickvl Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

github-actions bot commented Nov 24, 2025 •

edited

Loading

claude bot commented Nov 24, 2025 •

edited

Loading

claude bot commented Nov 25, 2025 •

edited

Loading

claude bot commented Nov 25, 2025 •

edited

Loading

github-actions bot commented Nov 26, 2025 •

edited

Loading