feat(azure_logs_ingestion sink): Initial azure_logs_ingestion sink

[jlaundry]

Summary

The current azure_monitor_logs sink uses the Data Collector API, which has been deprecated and will be removed in September 2026.

This sink uses the replacement Logs Ingestion API.

While I did consider making this a drop-in replacement for the existing sink, users need to make numerous breaking infrastructure changes, including:

• Creating new Data Collection Endpoint and Data Collection Rule resources
• Moving from a workspace-based secret key to an OAuth credential (App Registration, Managed Identity, etc.)
• (optionally) Re-configuring logs to use the built-in tables, instead of _CL custom tables.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

How did you test this PR?

1. Following the Tutorial steps, create a Log Analytics workspace, App Registration, Data Collection Endpoint, and Data Collection Rule
2. Set the AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_CLIENT_SECRET environment variables from the App Registration
3. Use the following vector.yaml:

sources:
  stdin:
    type: stdin

sinks:
  azure:
    type: azure_logs_ingestion
    inputs:
      - stdin
    endpoint: https://dce-e42z.westus2-1.ingest.monitor.azure.com
    dcr_immutable_id: dcr-00000000000000000000000000000000
    stream_name: Custom-vector_CL

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the "no-changelog" label to this PR.

References

• Closes: Azure DCR-based custom logs -> Logs ingestion API #20978
• Mentioned in: Enhance Azure Identity Integration in Vector: Ability to Utilize Passwordless Connections #20625 - while this PR doesn't resolve this issue for azure_blob, by using the Azure Identity crate, this sink supports passwordless credentials.

[bits-bot]

All committers have signed the CLA.

[pront]

Hi @jlaundry, thanks for this contribution!

It looks good. Two things:

• We will need some documentation files. See an example here (all files under website). Note that base/ is generated by make generate-component-docs.
• Is the intention to complete replace the azure_monitor_logs sink? If that's the case maybe we can mark the existing one as deprecated in favor of this new sink.

[rtrieu]

Thank you for your contribution! Approving with one very minor suggestion.

[jlaundry]

We will need some documentation files. See an example here (all files under website). Note that base/ is generated by make generate-component-docs.

Apologies, I thought this page was auto-generated as well - added now.

Is the intention to complete replace the azure_monitor_logs sink? If that's the case maybe we can mark the existing one as deprecated in favor of this new sink.

Good point, added 🙂

[pront]

What extra resources does the new sink require? Worth explaining this a bit better for users of the old sink.

[jlaundry]

Done (bb71403)

[pront]

Hi @jlaundry, we received this report #23036 and we will be reverted to older azure_* crate versions. Does this affect your PR?

[jlaundry]

Hi @jlaundry, we received this report #23036 and we will be reverted to older azure_* crate versions. Does this affect your PR?

From memory, there is a minor refactor required in https://github.com/jlaundry/vector/blob/02562be6447af36404d8b5668434e317c87a45b2/src/sinks/azure_logs_ingestion/config.rs#L139 to change it back to azure_identity::create_default_credential()?;, and possibly subsequent type changes... but reverting to 0.17 or 0.19 won't fundamentally block this PR, as thankfully I was using the raw REST API 🙂

Probably easiest if you rollback the package first, and then I'll rebase, retest, and push.

[pront]

FYI - Reverted deps #23039

[jlaundry]

FYI, I started preparing the rebase, but I've seen that the Azure Rust team have recently decided to change how authentication works with the azure_identity SDK: Azure/azure-sdk-for-rust#2283

Depending on what they decide, we may need to explicitly configure credentials, either via the vector config file or environment variables. The azure_blob sink will need a similar change (unless using connection_string config).

So instead of releasing an initial sink, and then requiring another config/environment change, I'll wait until there's stability in the SDK before moving forward with this PR.

(I'm not giving up!!!)

[yoelk]

Hi @jlaundry,
Thanks a lot for your initiative! In the company I work in we need this sink exactly.
I went over your PR and I haven't seen explicit auth fields (like connection_string for AzureBlobStorage or shared_key for AzureMonitorLogs). Are they implicit in some way? Or are client secrets not supported for authentication?
I'm asking because our use case is running vector on an AWS machine and connecting to multiple Azure sinks, so if only AAD or ENV_VAR based authentication is currently supported, we wouldn't be able to use it.
Do you consider adding the explicit possibility to use client secrets per sink? I'd be happy to contribute to that.

By the way, I see that the discussion here is closed. Does it mean you will go forward with merging your PR?

Thanks a lot!
Joel