Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OTel mode] Provide otelcol executable as a shortcut to elastic-agent otel #4661

Open
2 tasks
ycombinator opened this issue May 2, 2024 · 15 comments · May be fixed by #4816
Open
2 tasks

[OTel mode] Provide otelcol executable as a shortcut to elastic-agent otel #4661

ycombinator opened this issue May 2, 2024 · 15 comments · May be fixed by #4816
Assignees
Labels
enhancement New feature or request Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Comments

@ycombinator
Copy link
Contributor

ycombinator commented May 2, 2024

Describe the enhancement:

Currently, to run Elastic Agent in OTel mode, users must run ./elastic-agent otel ... (or the Windows equivalent). It would be convenient if users could simply run something like ./otelcol ... instead.

Describe a specific use case for the enhancement or feature:

For users who are familiar with OpenTelemetry but not with Elastic Agent, the less we expose Elastic Agent concepts to such users the better.

What is the definition of done?

  • Users should be able to run ./otelcol ... in addition to ./elastic-agent otel ....
  • These two commands should be functionally identical.
@ycombinator ycombinator added enhancement New feature or request Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team labels May 2, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@nchaulet
Copy link
Member

nchaulet commented May 8, 2024

@belimawr Starting to look on how to implement that, I think one way to do it will to create a simlink elastic-otel in the magefile package command does it make sense?

@belimawr
Copy link
Contributor

belimawr commented May 8, 2024

@belimawr Starting to look on how to implement that, I think one way to do it will to create a simlink elastic-otel in the magefile package command does it make sense?

Because it needs to run the elastic-agent command with arguments, I believe a simlink will not works and we also need to support this on Windows.

Because we use cobra for the CLI commands and arguments, it's easy to just create a new binary with the otel command as the entrypoint. Then we can add a simlink to the root folder in the same way we do with elastic-agent.

@cmacknz
Copy link
Member

cmacknz commented May 8, 2024

We need to measure how much we increase the on disk footprint of the agent with this addition.

One quirk of the current Windows packaging is that symlinks in the zip are replaced with copies of the binary and then converted back to symlinks at install time (zip doesn't/didn't have support for symlinks when this was done).

@belimawr
Copy link
Contributor

belimawr commented May 8, 2024

We need to measure how much we increase the on disk footprint of the agent with this addition.

I completely forgot about it 🤦‍♂️. I was thinking only about the implementation side of things.

Another option would be to use a shell script + windows equivalent that that just calls ./elastic-agent otel <ARGS>.

@belimawr
Copy link
Contributor

belimawr commented May 8, 2024

I just don't know if there is any Windows specific quirk about doing it.

@nchaulet
Copy link
Member

nchaulet commented May 9, 2024

@cmacknz @belimawr just get a discussion with @ycombinator and one possible solution here maybe more portable will be to create a new golang binary that call ./elastic-agent otel <ARGS> what do you think of that solution? it seems more portable than a shell script

@cmacknz
Copy link
Member

cmacknz commented May 9, 2024

A separate binary is more portable than a shell script, but shell scripts don't need to be compiled for every architecture, don't need to be signed on Windows+Mac, should take up less space on disk, and more importantly won't show up as another source of CVEs in security scans.

I think the best path is actually just a shell script that invokes ./elastic-agent otel <ARGS> for us. You'll have to write it twice, once for MacOS+Linux and once for Windows, but it should be simple in both cases. This isn't the first platform dependent thing we've had to do so this isn't a con.

@strawgate
Copy link

Two fun notes on this:

  1. Every currently supported version of Windows 10 and Windows 11 and Windows server (except maybe some really old ltsb versions) include tar and can natively open .tar.gz files
  2. Zip doesn't compress across files so every symlink of agent adds 21mb to the zip and 71mb to the pre-install on disk footprint.

@strawgate
Copy link

Perhaps we just do otelcol?

@ycombinator @nimarezainia can we get this included in a sprint asap?

@ycombinator
Copy link
Contributor Author

@ycombinator @nimarezainia can we get this included in a sprint asap?

It is already in a sprint.

@nchaulet nchaulet linked a pull request May 24, 2024 that will close this issue
@ycombinator
Copy link
Contributor Author

Chatting with @strawgate off-issue, we were debating the name of the new shortcut executable. We think otelcol (instead of elastic-otel) might be a better name as it matches up with the executable provided by the upstream OTel Collector release (e.g if you download and extract one of the collector artifacts from https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.101.0, you'll get an otelcol executable). Unless others have objections to this proposal, I'll update this issue's title and description within the next day or so.

@nimarezainia
Copy link
Contributor

Should we not differentiate between our distribution and that of the upstream - mainly because the Elastic distribution will be a superset, for example it will include features/bugfixes that haven't yet been up-streamed (or features that may never be up-streamed).

Only valid for cases where the user would have multiple collectors downloaded and only could differentiate when they are extracted. Not sure if this is common.

@strawgate
Copy link

otelcol makes it much easier for us to be a drop-in replacement and doesn't really label it as a specific distribution (contrib is otelcol-contrib).

Users can still tell ours apart because there is elastic-agent sitting next to it and otelcol is just a shortcut :)

@ycombinator ycombinator changed the title [OTel mode] Provide elastic-otel executable as a shortcut to elastic-agent otel [OTel mode] Provide otelcol executable as a shortcut to elastic-agent otel May 30, 2024
@ycombinator
Copy link
Contributor Author

Updated issue title and description to use otelcol.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
7 participants