trace-observability-stack

"Who monitor the monitors?"

Here we create a simple example of how to enable tracing across Grafana -> Thanos -> Prometheus and query it using Jaeger.

Below we describe the architeture built by our docker-compose files.

Running

First, let's build our local images for index management. These services will provide auto create index (by age for instance) and delete the old ones (older than some days). Run the command bellow at the root path of this repository:

sh build-local.sh

Now we can start to start up our containers. We create a script to boot them in a specific order. Run the command bellow at the root path of this repository:

sh up.sh

After this, you can open Grafana, Thanos Query Frontend, Prometheus or Jaeger, to verify the traces working.

To uninstall the environment you can run the command:

sh down.sh

Testing

Apply some queries, open some dashboards and then look to Jaeger Query to find the traces of these requests.