-
Notifications
You must be signed in to change notification settings - Fork 691
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
with_platform_config ignored for ImportExampleGen for KubeflowV2DagRunner #6146
Comments
@xivarri, Can you try using below code on Thank you!
|
Yeah that's what I did - but this does not get reflected in the output json, I had to do this hack from that issue to manually modify the json:
(I also could not limit this to only the importexamplegen's executor - if I did that vertex would still use e2-standard-4 for it - so set it for all of them) |
@xivarri, Thank you for sharing this insights. We will work on this internally and update the thread. |
Does kubeflow supports .json file .? from when .? How are you able to run pipeline json files in kubeflow? |
This commit added support for using
with_platform_config
withKubeflowV2DagRunner
for running on Vertex, but because the logic only runs in_build_container_spec
instep_builder.py
, it is not run forImporter
/FileBasedExampleGen
nodes, so addingwith_platform_config
to e.g.ImportExampleGen
is not reflected in the output json.The text was updated successfully, but these errors were encountered: