Skip to content

Conversation

@strickvl
Copy link
Contributor

Summary

  • Fixes weave import triggering wandb_init_hook and remote project creation even when enable_weave=False
  • Removes unnecessary else branch that imported weave and called weave.init(..., settings={"disabled": True})
  • Prevents PERMISSION_ERROR for users not logged into W&B who have Weave disabled

Problem

With wandb 0.23.0 + weave 0.52.17+, runs fail even when Weave is explicitly disabled. The root cause:

  1. The old code imported weave in both branches (enabled and disabled)
  2. With newer weave versions, import weave alone loads ~548 modules including weave.trace.autopatch
  3. This triggers wandb_init_hook which attempts remote project creation
  4. Users without credentials get PERMISSION_ERROR despite having enable_weave=False

Solution

Simply remove the else branch. When enable_weave=False, we don't import or interact with weave at all. Not importing a library is the ultimate way to disable it.

         if settings.enable_weave:
             import weave
             if self.config.project_name:
                 weave.init(project_name=self.config.project_name)
-        else:
-            import weave
-            if self.config.project_name:
-                weave.init(project_name=..., settings={"disabled": True})

Test plan

  • Verified fix with wandb 0.23.0 + weave 0.52.20 in isolated venv
  • Confirmed import weave loads 548 modules with potential side-effects
  • Confirmed the fix prevents any weave-related code when disabled
  • Manual testing with ZenML pipeline using wandb tracker with enable_weave=False

Previously, the wandb experiment tracker imported weave and called
weave.init(..., settings={"disabled": True}) even when enable_weave
was False. With weave >= 0.52.17, simply importing weave triggers
wandb_init_hook and remote project creation, causing PERMISSION_ERROR
for users not logged in.

The fix removes the unnecessary else branch entirely. When
enable_weave=False, we now skip all weave-related code, preventing
any import side-effects.
@strickvl strickvl added bug Something isn't working internal To filter out internal PRs and issues labels Nov 28, 2025
@strickvl strickvl requested a review from bcdurak November 28, 2025 13:04
@strickvl strickvl moved this to In Review in ZenML Roadmap Nov 28, 2025
@strickvl strickvl changed the title Fix weave import side-effects when enable_weave=False Fix weave import side-effects when enable_weave=False Nov 28, 2025
Copy link
Contributor

@htahir1 htahir1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this tested? i remember i added this else statement for a reason but i forgot why

@strickvl
Copy link
Contributor Author

is this tested? i remember i added this else statement for a reason but i forgot why

Not yet, but I'd be interested in the reason if you remember it. I couldn't understand why that was there.

@htahir1
Copy link
Contributor

htahir1 commented Nov 28, 2025

@strickvl sadly i cant remember... maybe because at some point weave was enabled by default or soemthing? or i wanted to make sure if false was passed to disable it for good? i just cant remember it rn

@strickvl
Copy link
Contributor Author

strickvl commented Dec 1, 2025

@AlexejPenner @htahir1 I tested this just now:

Manually tested with:

  • wandb 0.23.0 + weave 0.52.20
  • Fresh venv with ZenML from this branch
  • Pipeline with wandb experiment tracker and enable_weave=False

Verified:

  1. Main weave package is NOT imported when disabled (only harmless wandb.integration.weave.* stubs load)
  2. Pipeline runs successfully with no permission errors
  3. Metrics logged correctly to W&B

@htahir1
Copy link
Contributor

htahir1 commented Dec 1, 2025

@strickvl Then lets merge!


if self.config.project_name:
logger.info("Disabling weave")
weave.init(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what this implies, if the issue is to import it, perhaps we can cover it in a try-except. Basically, you can still try to disable it if weave is installed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah Hamza also couldn't remember what it implied @bcdurak but I tested it 🤷

Copy link
Contributor

@bcdurak bcdurak Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I saw the discussion above. He mentioned that he implemented this for a reason. That's exactly why I still think that it would be safer to just wrap this in a try-except, but I will leave it up to you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its fine lets merge it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@htahir1 it is already merged.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's just a late comment here.

@strickvl strickvl merged commit 3a3324d into develop Dec 1, 2025
59 of 60 checks passed
@strickvl strickvl deleted the bug/weave-disable-failures branch December 1, 2025 14:03
@strickvl strickvl added the snack label Dec 1, 2025
@strickvl strickvl linked an issue Dec 1, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working internal To filter out internal PRs and issues snack

Projects

Status: In Review

Development

Successfully merging this pull request may close these issues.

Fix weave import side-effects when enable_weave=False

4 participants