Coroot is an open source observability tool that uses eBPF to collect telemetry directly from applications and infrastructure. One of the things it does is capture L7 traffic from TLS connections without any code changes, by hooking into TLS libraries and syscalls.
Works great for OpenSSL. Works for Go.
Then rustls enters the picture and everything stops being obvious. With OpenSSL, everything is nicely wrapped:
SSL_write(ssl, plaintext)
└─ write(fd, encrypted)
SSL_read(ssl, plaintext)
└─ read(fd, encrypted)From eBPF’s point of view this is perfect:
- hook SSL_write, stash plaintext
- write() fires immediately → same thread → you know the FD
- same idea for reads
Everything happens inside one call. Correlation is trivial.
Rustls does things differently
Rustls doesn’t own the socket and never calls read or write itself. It works on buffers, and the application (or runtime) is responsible for actually moving bytes over the network.
The API reflects that separation pretty clearly:
// application writes plaintext into rustls
writer.write(plaintext);
// rustls produces encrypted bytes and writes them via io::Write
conn.write_tls(&mut socket);
// application reads encrypted bytes and feeds them into rustls
conn.read_tls(&mut socket);
// rustls decrypts and updates internal state
conn.process_new_packets();
// application reads decrypted data
reader.read(plaintext_buf);So instead of one call doing everything, you get a pipeline:
- plaintext is buffered first
- encryption happens later
- syscalls happen outside of rustls
- decryption happens before the app reads
The key difference for eBPF:
- writes: syscall happens after plaintext
- reads: syscall happens before plaintext
So the OpenSSL-style correlation only works in one direction.
Writes work as usual
On the write side, nothing fundamentally new is needed. You hook Writer::write, stash the plaintext, and correlate it with the following sendto. The ordering is preserved, so the same approach as OpenSSL still applies here.
Reads are inverted
The read path is where things really break.
recvfrom(fd, encrypted_buf, ...); // happens first conn.read_tls(&mut socket); conn.process_new_packets(); reader.read(plaintext_buf); // plaintext appears here
By the time we see plaintext, the syscall is already gone.
So the logic has to be reversed. Instead of:
- “see plaintext → wait for syscall”
we do:
- “see syscall → remember it → use it later”
Concretely:
- on recvfrom → stash FD per thread
- on reader.read → pick up that FD and attach it to plaintext
It’s basically reverse correlation. Not pretty, but it matches how rustls works.
When “ret=1” doesn’t mean 1 byte
This one took longer than expected. We reused the OpenSSL-style exit probe:
ret = PT_REGS_RC(ctx)The probe fired, but results were weird:
ret=1
ret=0Which made no sense for a read. Turns out Rust returns Result<usize> like this:
- rax → success or error flag
- rdx → actual number of bytes
So we were reading rax and treating it as a size. Meaning:
- ret=1 → actually an error
- ret=0 → success, but size is somewhere else
Fix was straightforward once understood:
if (PT_REGS_RC(ctx) == 0) { // success
size = ctx->dx; // actual byte count
}Classic case of “everything works, but the numbers are garbage”.
Finding rustls in binaries
Rust symbols are heavily mangled:
_ZN55_$LT$rustls..conn..Writer$u20$as$u20$std..io..Write$GT$5write17h0ee1e61402b1a37cE
It looks messy, but it encodes the full path: rustls::stream::Writer<T> implementing std::io::Write::write.
The tricky part is that mangling isn’t stable:
- different compiler versions use different schemes (legacy vs v0)
- optimizations and stripping can change what’s left in the binary
So matching exact names is fragile.
Instead, we:
- check ELF .comment for rustc to detect that the binary was built with Rust
- then scan symbols for patterns like “rustls”, “Writer”+”write”, “Reader”+”read”
Not perfect, but reliable enough in practice.
Results
Because we instrument rustls at the library level, not the frameworks, this works across most Rust clients that use rustls under the hood.
That includes HTTP stacks like hyper when paired with rustls (hyper-rustls, and frameworks like axum or warp when configured with rustls), database clients like sqlx when using its rustls TLS feature, and any async Rust service using tokio-rustls.
No code changes, no SDKs, no wrappers.
For Rust apps using OpenSSL via native-tls or openssl, the existing OpenSSL instrumentation already works. rustls was the missing piece.
Below is an example of a service talking to MySQL over TLS. Coroot shows the actual queries even though everything on the wire is encrypted.
At Coroot we work on eliminating blind spots and making applications observable without code changes, even in places where it usually feels impossible like TLS.
Install Coroot and get full visibility into your apps in minutes. And if you like what we’re building, give us a star on GitHub.
