Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port over executor_config for various executors using task sdk #46892

Open
1 task done
amoghrajesh opened this issue Feb 19, 2025 · 4 comments
Open
1 task done

Port over executor_config for various executors using task sdk #46892

amoghrajesh opened this issue Feb 19, 2025 · 4 comments
Labels
area:Executors-core LocalExecutor & SequentialExecutor area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK kind:meta High-level information important to the community

Comments

@amoghrajesh
Copy link
Contributor

Body

We can today specify the executor_config while running a task in KubernetesExecutor, SequentialExecutor, AWSExecutor, EdgeExecutor atleast. Example: https://github.com/apache/airflow/blob/main/airflow/example_dags/example_kubernetes_executor.py#L102-L139

This task will track that effort, feel free to break it into 1 task per executor too.

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.
@amoghrajesh amoghrajesh added the kind:meta High-level information important to the community label Feb 19, 2025
@amoghrajesh
Copy link
Contributor Author

cc: @ashb @jscheffl @o-nikolas

@dosubot dosubot bot added the area:Executors-core LocalExecutor & SequentialExecutor label Feb 19, 2025
@ashb ashb added the area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK label Feb 19, 2025
@o-nikolas
Copy link
Contributor

Thanks @amoghrajesh! Can you add a bit more to the body of the Issue describing what is missing and what needs to be done? It's not clear to me what the blocker is, or what needs to be implemented.

@amoghrajesh
Copy link
Contributor Author

There's absolutely no blocker here! I will add some more details tomorrow. And if i get to K8s executor, we will have an example too

@o-nikolas
Copy link
Contributor

Got it thanks! Just missing implementation in the task API interface then I'm assuming? Looking forward to seeing the changes :)

@amoghrajesh amoghrajesh changed the title Port over executor_config for various executors to task sdk Port over executor_config for various executors using task sdk Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:Executors-core LocalExecutor & SequentialExecutor area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK kind:meta High-level information important to the community
Projects
None yet
Development

No branches or pull requests

3 participants