Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add airflow error-guide command #46932

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

omkar-foss
Copy link
Collaborator

closes: #43171
related: #44616

This PR adds the airflow error-guide command as per the plan in this comment.

@omkar-foss
Copy link
Collaborator Author

Guide for specific error code:

$ airflow error-guide --error-code AERR100

The error you're facing would be with this message: "Invalid DAG structure".

As per our observations, a possible cause could be as follows:
Triggered when a DAG's dependencies or attributes are incorrectly defined.

To resolve this, as first step, you can try the following:
Review the DAG structure for errors. Ensure that all dependencies are correctly defined and that no circular dependencies 
exist.

If this doesn't resolve your problem, you can check out the docs for more info:
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html

You may also ask your questions on the Airflow Slack #user-troubleshooting channel:
https://apache-airflow.slack.com/messages/user-troubleshooting

Happy Debugging! 🐞

Listing all exceptions covered in guide:

airflow error-guide --list-exceptions   
Exception Type           
=========================
AirflowApiException      
AirflowCliException      
AirflowConfigException   
AirflowDagCycleException 
AirflowDagImportException
AirflowDagNotFound       
AirflowDagPausedException
AirflowDatabaseException 
AirflowException         
AirflowExecutorException 
AirflowParseException    
AirflowSchedulerException
AirflowTaskException     
AirflowTaskTimeout       
AirflowTemplateException 
AirflowTriggerException  
AirflowWebServerException
AirflowWorkerException   
AirflowXComException     
AttributeError           
FileNotFoundError        
ImportError              
KeyError                 
ModuleNotFoundError      
PermissionError          
UnpicklingError          
ValueError               

Dump entire guide as yaml:

airflow error-guide --list-guide
AERR001:
  description: Happens when dynamically mapped tasks exceed the maximum number of
    tasks allowed.
  documentation: https://airflow.apache.org/docs/apache-airflow/stable/concepts/dynamic-task-mapping.html
  error_message: Dynamic task mapping exceeded limit
  exception_type: AirflowException
  first_steps: Check the task count limit in the configuration. Consider increasing
    the task limit or optimizing task mapping logic.
AERR002:
  description: Happens when the scheduler or webserver cannot locate a task instance
    in the database.
  documentation: https://airflow.apache.org/docs/apache-airflow/stable/administration/task-instances.html
  error_message: Task instance not found
  exception_type: AirflowException
  first_steps: Verify that the database connection is stable. Check if the task instances
    exist in the metadata database and consider re-running the DAG.
...truncated

@omkar-foss omkar-foss self-assigned this Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make Airflow error messages more specific, clear and actionable
1 participant