Learn how eBPF transforms your infrastructure. Register for Peter Zaitsev's webinar šŸ“£

Implementing RED Method with Distributed Tracing

TheĀ REDĀ (Rate, Errors, Duration) method is a popular approach to instrument and troubleshoot microservices applications. In Coroot – Open Source observability platform – we have designed an effective interface allowing to use RED method with Distributed Tracing and OpenTelemetry (OTEL) generated data.

Let's start from the system's overview

This summary gives us everything you need to apply RED method to the application – we can see the system is handling roughly 70 requests per second with p99 latency below 250ms and no errors.

Errors

What if there are errors? You can clearly see errors in the overview interface – about 5 errors/sec in this case. What is causing them? Select the relevant range on the graph to find the causes – Coroot automatically analyzes all affected requests and pinpoints spans where errors are originated.

Want to double check if this is indeed the case? You can click on the ā€œSample Traceā€:

Compare and Contrast

Another thing which is very helpful to understand Why Errors or high latency is happening is to be able to compare and contrast the difference between the normal baseline and when errors are happening – Coroot allows you to do just that with its ā€œCompare Attributesā€ feature:

Latency Outliers

Another kind of issue we often diagnose with RED method is slow requests or in other words latency outliers. With ā€œLatency Explorerā€ Coroot analyzes all the traces to highlight where specifically elevated latency comes from:

Coroot visualizes latency as Flamegraph where wider frames means higher contribution to the latency and operations highlighted in red in comparison mode are taking more time than before. This makes it very easy to pinpoint where specifically the increased latency is coming from.

Conclusion

Coroot makes distributed tracing easy, allowing you to diagnose the most common issues, enabled by OpenTelemetry Traces in just a couple of clicks. It also can do much more. Check out theĀ Coroot OverviewĀ for details.

TryĀ Coroot CloudĀ now (14-day free trial is available) or follow the instructions on ourĀ Getting startedĀ page to install Coroot Community Edition.

If you like Coroot, give us a ā­ onĀ GitHubļø ā€“ it helps us reach more engineers and improve our product for you.

Any questions or feedback? Reach out to us onĀ Slack.

Share this post

Related posts