Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

W0000 00:00:1714315704.778368 2252 graph_launch.cc:671] Fallback to op-by-op mode because memset node breaks graph update #66589

Closed
sondosaabed opened this issue Apr 28, 2024 · 3 comments
Assignees
Labels
comp:keras Keras related issues stat:awaiting response Status - Awaiting response from author TF 2.15 For issues related to 2.15.x type:bug Bug

Comments

@sondosaabed
Copy link

Issue type

Others

Have you reproduced the bug with TensorFlow Nightly?

No

Source

source

TensorFlow version

2.15

Custom code

Yes

OS platform and distribution

Kaggle notebook

Mobile device

No response

Python version

3.12

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

What does this warning indicate and how does it affect the training

image

Standalone code to reproduce the issue

I have included an image in the upper cell. I see a warning when training

Relevant log output

No response

@google-ml-butler google-ml-butler bot added the type:others issues not falling in bug, perfromance, support, build and install or feature label Apr 28, 2024
@tilakrayal tilakrayal added comp:keras Keras related issues TF 2.15 For issues related to 2.15.x labels Apr 29, 2024
@tilakrayal
Copy link
Contributor

@sondosaabed,
I suspect that this is due to Keras 3 which uses jit compile by default, and Keras 2 doesn't (user needs to explicitly enable jit), and the warning message is probably from jit/xla. I request you to take a look at this issue where a similar issue/warning has been proposed and it is still open. Also I request to follow the similar issue which has been proposed to have the updates on the similar issue. Thank you!

@tilakrayal tilakrayal added stat:awaiting response Status - Awaiting response from author type:bug Bug and removed type:others issues not falling in bug, perfromance, support, build and install or feature labels Apr 29, 2024
@sondosaabed
Copy link
Author

@sondosaabed, I suspect that this is due to Keras 3 which uses jit compile by default, and Keras 2 doesn't (user needs to explicitly enable jit), and the warning message is probably from jit/xla. I request you to take a look at this issue where a similar issue/warning has been proposed and it is still open. Also I request to follow the similar issue which has been proposed to have the updates on the similar issue. Thank you!

Thank you for your reply! I suggest they make a better warning message so as a user I could easily understand what this is and how to fix it. I will check out the jit/xla !
Thank you for indicating that issue I'll follow up and see if it's affecting my training. I'm gonna close this issue with this...

Copy link

Are you satisfied with the resolution of your issue?
Yes
No

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:keras Keras related issues stat:awaiting response Status - Awaiting response from author TF 2.15 For issues related to 2.15.x type:bug Bug
Projects
None yet
Development

No branches or pull requests

2 participants