-
Notifications
You must be signed in to change notification settings - Fork 559
Fix weave import side-effects when enable_weave=False
#4265
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Previously, the wandb experiment tracker imported weave and called
weave.init(..., settings={"disabled": True}) even when enable_weave
was False. With weave >= 0.52.17, simply importing weave triggers
wandb_init_hook and remote project creation, causing PERMISSION_ERROR
for users not logged in.
The fix removes the unnecessary else branch entirely. When
enable_weave=False, we now skip all weave-related code, preventing
any import side-effects.
enable_weave=False
htahir1
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this tested? i remember i added this else statement for a reason but i forgot why
Not yet, but I'd be interested in the reason if you remember it. I couldn't understand why that was there. |
|
@strickvl sadly i cant remember... maybe because at some point weave was enabled by default or soemthing? or i wanted to make sure if false was passed to disable it for good? i just cant remember it rn |
|
@AlexejPenner @htahir1 I tested this just now: Manually tested with:
Verified:
|
|
@strickvl Then lets merge! |
|
|
||
| if self.config.project_name: | ||
| logger.info("Disabling weave") | ||
| weave.init( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure what this implies, if the issue is to import it, perhaps we can cover it in a try-except. Basically, you can still try to disable it if weave is installed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah Hamza also couldn't remember what it implied @bcdurak but I tested it 🤷
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I saw the discussion above. He mentioned that he implemented this for a reason. That's exactly why I still think that it would be safer to just wrap this in a try-except, but I will leave it up to you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
its fine lets merge it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@htahir1 it is already merged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's just a late comment here.
Summary
wandb_init_hookand remote project creation even whenenable_weave=Falseelsebranch that imported weave and calledweave.init(..., settings={"disabled": True})PERMISSION_ERRORfor users not logged into W&B who have Weave disabledProblem
With wandb 0.23.0 + weave 0.52.17+, runs fail even when Weave is explicitly disabled. The root cause:
weavein both branches (enabled and disabled)import weavealone loads ~548 modules includingweave.trace.autopatchwandb_init_hookwhich attempts remote project creationPERMISSION_ERRORdespite havingenable_weave=FalseSolution
Simply remove the
elsebranch. Whenenable_weave=False, we don't import or interact with weave at all. Not importing a library is the ultimate way to disable it.if settings.enable_weave: import weave if self.config.project_name: weave.init(project_name=self.config.project_name) - else: - import weave - if self.config.project_name: - weave.init(project_name=..., settings={"disabled": True})Test plan
import weaveloads 548 modules with potential side-effectsenable_weave=False