Create gremlin provider

[ahmadtfarhan]

New Provider for Gremlin (Apache TinkerPop)

Summary

This PR introduces a new provider for Gremlin—the graph traversal language of Apache TinkerPop. The provider is designed to facilitate connections to graph databases supporting Gremlin, such as Azure Cosmos DB and Amazon Neptune.

Background

Previously, I had developed a custom hook to query data from Azure Cosmos DB using Apache Gremlin. In this PR, I’ve consolidated that functionality into a full provider. While I managed to run it locally on the main branch, I encountered the BaseHook issue (#45233), so I performed all testing on the v2-10-test branch.

Changes

Provider Implementation:

• The new GremlinHook establishes a connection using the Gremlin Python Client.
• It constructs the connection URI based on the Airflow Connection’s properties and extra parameters.
• The hook exposes a run(query: str) method that submits raw Gremlin query strings and returns the results.

Operator:

• The GremlinOperator instantiates the GremlinHook with a given connection ID and query, then calls run(query) to execute the query.

Testing:

• Unit tests have been added to validate the URI construction, client instantiation, and query execution.
• Manual tests confirmed that raw queries using Client.submit() successfully return results from Cosmos DB using Gremlin.

Testing & Validation

• I ran unit tests on the v2-10-test branch after encountering BaseHook issues on the main branch.
• The provider has been validated locally by submitting queries such as g.V() and verifying that the expected results are returned from Azure Cosmos DB.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: [email protected]
  Slack: https://s.apache.org/airflow-slack

[rawwar]

Wondering if gremlin should be a subdirectory in the Apache folder, similar to Hive, iceberg

[ahmadtfarhan]

Wondering if gremlin should be a subdirectory in the Apache folder, similar to Hive, iceberg

I think it's doable, I was thinking to implement that but to keep it simpler to develop and test I left it that way till the Discussion on devlist is done.

[potiuk]

Wondering if gremlin should be a subdirectory in the Apache folder, similar to Hive, iceberg

I think it's doable, I was thinking to implement that but to keep it simpler to develop and test I left it that way till the Discussion on devlist is done.

Yeah. There are two possible paths I see:

• apache/gremlin - the disadvantage of it is that if you do not know what gremlin is, you might not even realise that this is about graph databases
• common/graphdb or similar -> which is quite a bit more discoverable by our users, but it requires Airlfow community to make an opinionated choice that "gremlin" is our recommended common abstraction layer for graph databases.

Both are possible, and I am slighlty in favour of "common", but let's see what others in the community think.