You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Experimental Feature**: Dynamic pipelines are currently an experimental feature. There are known issues and limitations, and the interface is subject to change. This feature is only supported by the `local`and `kubernetes` orchestrators. If you encounter any issues or have feedback, please let us know at [https://github.com/zenml-io/zenml/issues](https://github.com/zenml-io/zenml/issues).
8
+
**Experimental Feature**: Dynamic pipelines are currently an experimental feature. There are known issues and limitations, and the interface is subject to change. This feature is only supported by the `local`, `kubernetes`, `sagemaker`and `vertex` orchestrators. If you encounter any issues or have feedback, please let us know at [https://github.com/zenml-io/zenml/issues](https://github.com/zenml-io/zenml/issues).
9
9
{% endhint %}
10
10
11
11
{% hint style="info" %}
@@ -265,26 +265,11 @@ When running multiple steps concurrently using `step.submit()`, a failure in one
265
265
Dynamic pipelines are currently only supported by:
266
266
- `local`orchestrator
267
267
- `kubernetes`orchestrator
268
+
- `sagemaker`orchestrator
269
+
- `vertex`orchestrator
268
270
269
271
Other orchestrators will raise an error if you try to run a dynamic pipeline with them.
270
272
271
-
### Remote Execution Requirement
272
-
273
-
When running dynamic pipelines remotely (e.g., with the `kubernetes` orchestrator), you **must** include `depends_on` for at least one step in your pipeline definition. This is currently required due to a bug in remote execution.
274
-
275
-
{% hint style="warning" %}
276
-
**Required for Remote Execution**: Without `depends_on`, remote execution will fail. This requirement does not apply when running locally with the `local` orchestrator.
277
-
{% endhint %}
278
-
279
-
For example:
280
-
281
-
```python
282
-
@pipeline(dynamic=True, depends_on=[some_step])
283
-
def dynamic_pipeline():
284
-
some_step()
285
-
# ... rest of your pipeline
286
-
```
287
-
288
273
### Artifact Loading
289
274
290
275
When you call `.load()` on an artifact in a dynamic pipeline, it synchronously loads the data. For large artifacts or when you want to maintain parallelism, consider passing the step outputs (future or artifact) directly to downstream steps instead of loading them.
0 commit comments