New Blog Post: Collecting file-based Java logs with OpenTelemetry #5600

cyrille-leclerc · 2024-11-13T10:02:54Z

This post presents how to use the new experimental-otlp/stdout export format introduced by OpenTelemetry Java Instrumentation (open-telemetry/opentelemetry-java#6675) to export application logs to files with full OpenTelemetry contextualization and ingest them with the OTel Collector File Log receiver and OTLP/JSON connector.

In this post, we explain how to configure the OTel Java instrumentation agent and the OTel Collector, including when deploying on K8s, to implement the end to end file based logs ingestion pipeline.

Blog Post Issue: #5606
SIG: java and specifications
Sponsor:

Preview: https://deploy-preview-5600--opentelemetry.netlify.app/blog/2024/collecting-file-based-java-logs-with-opentelemetry/

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry.md

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

zeitlinger · 2024-11-13T10:58:03Z

@cyrille-leclerc https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/tag/v2.10.0 was just released - I'll update the executable example in grafana/docker-otel-lgtm#145 now

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

svrnm · 2024-11-13T12:56:24Z

@cyrille-leclerc can you raise an issue for this PR following our guidelines? Thanks

cyrille-leclerc · 2024-11-13T14:23:05Z

@cyrille-leclerc can you raise an issue for this PR following our guidelines? Thanks

Thanks @svrnm. Please find the blog post issue:

New Blog Post: Collecting file-based Java logs with OpenTelemetry #5606

content-modules/opentelemetry-specification

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

svrnm · 2024-11-15T08:21:37Z

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

+
+![OTLP/JSON Architecture](otlpjson-architecture.png)
+
+## Configure Java Application to Output OTLP/JSON Logs


the sample application you are using, is this available somewhere? If so, can you point to it? It's always good if people can reproduce your tutorial 1:1 without the need for them to find an application first

@zeitlinger do we have a simplified demo or we use https://github.com/cyrille-leclerc/my-shopping-cart/? Note that I reverted my-shopping-cart to the otlp exporter so I would have to spend a few cycles to setup a demo.

svrnm · 2024-11-15T08:22:53Z

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

+![OpenTelemetry Collector OTLP/JSON pipeline](otel-collector-otlpjson-pipeline.png)
+
+Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)


I like that you provide the link to the otelbin source of that image, but let's figure out a way to visualize this better.

Suggested change

![OpenTelemetry Collector OTLP/JSON pipeline](otel-collector-otlpjson-pipeline.png)

Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)

[View OTel Collector pipeline with <img src="https://www.otelbin.io/favicon.ico" height="20px"/> OTelBin](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)

@svrnm please review proposal

Suggested change

![OpenTelemetry Collector OTLP/JSON pipeline](otel-collector-otlpjson-pipeline.png)

Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)

{{< figure class="figure" src="otel-collector-otlpjson-pipeline.png" attr="View OTel Collector pipeline with OTelBin" attrlink="https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a" >}}

What I meant is to have something that makes it clearer that the Source: (or View ...) belongs to the image, since the spacing is not right, similar to what bootstrap provides for figures. We have a shortcode for that via docsy, it's not exactly what I want, but we can take a look into that separately, cc @chalin

Has this been addressed to your satisfaction @svrnm? See, https://deploy-preview-5600--opentelemetry.netlify.app/blog/2024/collecting-file-based-java-logs-with-opentelemetry/#configure-the-opentelemetry-collector-to-ingest-the-otlpjson-logs.

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

cyrille-leclerc · 2024-11-15T11:05:12Z

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

+
+![OTLP/JSON Architecture](otlpjson-architecture.png)
+
+## Configure Java Application to Output OTLP/JSON Logs


@zeitlinger do we have a simplified demo or we use https://github.com/cyrille-leclerc/my-shopping-cart/? Note that I reverted my-shopping-cart to the otlp exporter so I would have to spend a few cycles to setup a demo.

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

cyrille-leclerc · 2024-11-15T11:31:58Z

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

+![OpenTelemetry Collector OTLP/JSON pipeline](otel-collector-otlpjson-pipeline.png)
+
+Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)


Suggested change

![OpenTelemetry Collector OTLP/JSON pipeline](otel-collector-otlpjson-pipeline.png)

Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)

[View OTel Collector pipeline with <img src="https://www.otelbin.io/favicon.ico" height="20px"/> OTelBin](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)

@svrnm please review proposal

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

tiffany76

Just a few copyedit suggestions. Thanks!

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

chalin · 2024-11-20T10:39:31Z

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

+## Reference architecture
+
+The deployment architecture looks like the following:


I don't think that we need a section heading here, you're only introducing the diagram.

Also, this isn't a "reference architecture", nor even a "deployment architecture". Prefer the use of the simpler term "deployment diagram". Actually, if you just present the diagram, you don't really need to name it, but if you do (e.g., via the "alt" text in the image element below, then avoid the use of "architecture".

Suggested change

## Reference architecture

The deployment architecture looks like the following:

chalin · 2024-11-20T10:47:46Z

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

+
+The deployment architecture looks like the following:
+
+![OTLP/JSON Architecture](otlpjson-architecture.png)


This diagram is really busy, and the content too small. I'd suggest the following improvements:

Remove the following text from the diagram and add it instead as prose to this section (before or after the image):

"Logs contextualized with OTel metadata including: ..."

"experimental-otlp/stout" exporter Available in Java Agent v2.10"

Move the "Observability Platform" block to be below the rest so that the image becomes less wide -- that is, favor vertical image real estate rather than horizontal. Or, if you prefer, put the app at the top of the diagram with the OTel components below. Something (very roughly) like what I show below.

What do you think of this style?

chalin · 2024-11-20T10:48:55Z

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

+<!-- prettier-ignore-start -->
+
+```json
+{"resourceLogs":[{"resource":{"attributes":[{"key":"deployment.environment.name","value":{"stringValue":"staging"}},{"key":"service.instance.id","value":{"stringValue":"6ad88e10-238c-4fb7-bf97-38df19053366"}},{"key":"service.name","value":{"stringValue":"checkout"}},{"key":"service.namespace","value":{"stringValue":"shop"}},{"key":"service.version","value":{"stringValue":"1.1"}}]},"scopeLogs":[{"scope":{"name":"com.mycompany.checkout.CheckoutServiceServer$CheckoutServiceImpl","attributes":[]},"logRecords":[{"timeUnixNano":"1730435085776869000","observedTimeUnixNano":"1730435085776944000","severityNumber":9,"severityText":"INFO","body":{"stringValue":"Order order-12035 successfully placed"}, "attributes":[{"key":"customerId","value":{"stringValue":"customer-49"}},{"key":"thread.id","value":{"intValue":"44"}},{"key":"thread.name","value":{"stringValue":"grpc-default-executor-1"}}],"flags":1,"traceId":"42de1f0dd124e27619a9f3c10bccac1c","spanId":"270984d03e94bb8b"}]}],"schemaUrl":"https://opentelemetry.io/schemas/1.24.0"}]}
+```
+
+<!-- prettier-ignore-end -->


This JSON output is unreadable. Let Prettier format it, or format it manually. If it is too big, then wrap it in a <details> element.

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

chalin · 2024-11-20T10:53:49Z

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

+traces and metrics.
+
+Any feedback or questions? Reach out on
+[GitHub](https://github.com/open-telemetry/opentelemetry-specification/pull/4183).


Why is this linking to a spec PR?
Is this PR really the best place for folks to comment?
If so, that's ok, but then the link text should explain more clearly what the link target is.

chalin · 2024-11-20T10:54:52Z

/fix:refcache

opentelemetrybot · 2024-11-20T10:55:12Z

You triggered fix:refcache action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/11931927870

opentelemetrybot · 2024-11-20T10:59:44Z

fix:refcache was successful.

IMPORTANT: (RE-)RUN `/fix:all` to ensure that there are no remaining check issues.

tiffany76

LGTM, subject to @chalin's comments.

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

…telemetry/index.md Co-authored-by: Severin Neumann <[email protected]>

Co-authored-by: Severin Neumann <[email protected]>

…telemetry/index.md

Co-authored-by: Severin Neumann <[email protected]> Co-authored-by: Tiffany Hrabusa <[email protected]>

…telemetry/index.md Co-authored-by: Severin Neumann <[email protected]>

Co-authored-by: Patrice Chalin <[email protected]>

Co-authored-by: Tiffany Hrabusa <[email protected]>

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md

chalin · 2024-11-21T15:53:46Z

/fix:format

opentelemetrybot · 2024-11-21T15:54:00Z

You triggered fix:format action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/11956865359

opentelemetrybot · 2024-11-21T15:54:21Z

fix:format was successful.

IMPORTANT: (RE-)RUN `/fix:all` to ensure that there are no remaining check issues.

Co-authored-by: Patrice Chalin <[email protected]>

cyrille-leclerc requested a review from a team as a code owner November 13, 2024 10:02

github-actions bot added the blog label Nov 13, 2024

opentelemetrybot requested a review from a team November 13, 2024 10:03

zeitlinger reviewed Nov 13, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry.md Outdated Show resolved Hide resolved

cyrille-leclerc commented Nov 13, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Show resolved Hide resolved

cyrille-leclerc commented Nov 13, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

svrnm requested changes Nov 15, 2024

View reviewed changes

cyrille-leclerc commented Nov 15, 2024

View reviewed changes

opentelemetrybot requested a review from a team November 15, 2024 11:09

cyrille-leclerc commented Nov 15, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

svrnm reviewed Nov 18, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

opentelemetrybot requested a review from a team November 18, 2024 10:35

tiffany76 reviewed Nov 19, 2024

View reviewed changes

svrnm reviewed Nov 20, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

opentelemetrybot requested a review from a team November 20, 2024 07:35

svrnm reviewed Nov 20, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

opentelemetrybot requested a review from a team November 20, 2024 08:50

cyrille-leclerc commented Nov 20, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Show resolved Hide resolved

chalin force-pushed the collect-java-otlpjson-logs branch from 72614b9 to e764e3b Compare November 20, 2024 10:18

chalin requested changes Nov 20, 2024

View reviewed changes

opentelemetrybot requested a review from a team November 20, 2024 12:55

tiffany76 approved these changes Nov 21, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

BlogPost: Collecting file based Java logs with OpenTelemetry

2c4bda8

cyrille-leclerc and others added 21 commits November 21, 2024 10:45

Update content/en/blog/2024/collecting-file-based-java-logs-with-open…

69d254d

…telemetry/index.md Co-authored-by: Severin Neumann <[email protected]>

Apply suggestions from code review

74900fc

Apply suggestions from code review

2520f47

Co-authored-by: Severin Neumann <[email protected]>

Apply suggestions from code review

a483250

Co-authored-by: Severin Neumann <[email protected]>

fix submodules

23be0a1

Apply suggestions from code review

54f6d09

Update content/en/blog/2024/collecting-file-based-java-logs-with-open…

67c9361

…telemetry/index.md

Update content/en/blog/2024/collecting-file-based-java-logs-with-open…

1d7ed71

…telemetry/index.md

wip

7d2bc11

wip

fbe16d6

Update content/en/blog/2024/collecting-file-based-java-logs-with-open…

1ef7c05

…telemetry/index.md

Apply suggestions from code review

35e448c

Co-authored-by: Severin Neumann <[email protected]> Co-authored-by: Tiffany Hrabusa <[email protected]>

Update content/en/blog/2024/collecting-file-based-java-logs-with-open…

b8f989e

…telemetry/index.md Co-authored-by: Severin Neumann <[email protected]>

Fix formatting

4175d74

Fix formatting

e5aee18

Better hyperlink

ee7bfba

Fix formatting

15bbe6f

Add missing alt on img

0bc6699

Results from /fix:refcache

8801600

Apply suggestions from code review

c50d9b0

Co-authored-by: Patrice Chalin <[email protected]>

Apply suggestions from code review

0f04576

Co-authored-by: Tiffany Hrabusa <[email protected]>

chalin force-pushed the collect-java-otlpjson-logs branch from bd35fc6 to 0f04576 Compare November 21, 2024 15:45

chalin reviewed Nov 21, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

chalin reviewed Nov 21, 2024

View reviewed changes

content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md Outdated Show resolved Hide resolved

Results from /fix:format

75068dd

Apply suggestions from code review

3c0354f

Co-authored-by: Patrice Chalin <[email protected]>

opentelemetrybot requested a review from a team November 21, 2024 17:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Blog Post: Collecting file-based Java logs with OpenTelemetry #5600

New Blog Post: Collecting file-based Java logs with OpenTelemetry #5600

cyrille-leclerc commented Nov 13, 2024 •

edited by chalin

Loading

zeitlinger commented Nov 13, 2024 •

edited

Loading

svrnm commented Nov 13, 2024

cyrille-leclerc commented Nov 13, 2024

svrnm Nov 15, 2024

cyrille-leclerc Nov 15, 2024

svrnm Nov 15, 2024

cyrille-leclerc Nov 15, 2024 •

edited

Loading

svrnm Nov 20, 2024

chalin Nov 21, 2024

cyrille-leclerc Nov 15, 2024

cyrille-leclerc Nov 15, 2024 •

edited

Loading

tiffany76 left a comment

chalin Nov 20, 2024

chalin Nov 20, 2024

cyrille-leclerc Nov 21, 2024

chalin Nov 20, 2024

chalin Nov 20, 2024

chalin commented Nov 20, 2024

opentelemetrybot commented Nov 20, 2024

opentelemetrybot commented Nov 20, 2024

tiffany76 left a comment

chalin commented Nov 21, 2024

opentelemetrybot commented Nov 21, 2024

opentelemetrybot commented Nov 21, 2024


		![OTLP/JSON Architecture](otlpjson-architecture.png)

		## Configure Java Application to Output OTLP/JSON Logs

		![OpenTelemetry Collector OTLP/JSON pipeline](otel-collector-otlpjson-pipeline.png)

		Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)

		## Reference architecture

		The deployment architecture looks like the following:

New Blog Post: Collecting file-based Java logs with OpenTelemetry #5600

Are you sure you want to change the base?

New Blog Post: Collecting file-based Java logs with OpenTelemetry #5600

Conversation

cyrille-leclerc commented Nov 13, 2024 • edited by chalin Loading

zeitlinger commented Nov 13, 2024 • edited Loading

svrnm commented Nov 13, 2024

cyrille-leclerc commented Nov 13, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyrille-leclerc Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyrille-leclerc Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

tiffany76 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chalin commented Nov 20, 2024

opentelemetrybot commented Nov 20, 2024

opentelemetrybot commented Nov 20, 2024

IMPORTANT: (RE-)RUN /fix:all to ensure that there are no remaining check issues.

tiffany76 left a comment

Choose a reason for hiding this comment

chalin commented Nov 21, 2024

opentelemetrybot commented Nov 21, 2024

opentelemetrybot commented Nov 21, 2024

IMPORTANT: (RE-)RUN /fix:all to ensure that there are no remaining check issues.

cyrille-leclerc commented Nov 13, 2024 •

edited by chalin

Loading

zeitlinger commented Nov 13, 2024 •

edited

Loading

cyrille-leclerc Nov 15, 2024 •

edited

Loading

cyrille-leclerc Nov 15, 2024 •

edited

Loading

IMPORTANT: (RE-)RUN `/fix:all` to ensure that there are no remaining check issues.

IMPORTANT: (RE-)RUN `/fix:all` to ensure that there are no remaining check issues.