What 13 Years of Building Observability Has Taught Us: The Problems Stay, the Technology Changes

Technology moves quickly. As a matter of fact, it’s evolving exponentially. New frameworks appear. New architectures replace old ones. Entire categories of tools emerge, mature, consolidate, and occasionally disappear. A technology that sounds futuristic one year can feel almost boring a few years later. Does anyone still think about NoSQL, for example?

After more than a decade of building Sematext, one thing stands out to me:

Many of the problems we are solving today are remarkably similar to the problems we were trying to solve 10 or 15 years ago.

We still want to know:

• What is happening in our systems?
• Why is this application slow?
• Where did this request spend its time?
• Which services depend on which other services?
• What changed?
• Where is the bottleneck?
• What is consuming the CPU?
• What is actually happening on the network?
• And, increasingly: what are our AI agents doing?

The questions have stayed with us. Ask any developers or sysadmins (that was the term back then) from the end of the last millenium and you’ll see they were asking the same questions 30+ years ago.

What has changed, sometimes dramatically, is the technology available to answer them. And new use cases have emerged (see towards the end of this post).

Looking back at Sematext’s own history is an interesting way to see this technology cycle play out.

Platform gravity

One of the things that has changed significantly over the last decade is the expectation that monitoring and observability capabilities should work together.

Today, every major monitoring vendor is a platform.

Metrics are not enough. Logs are not enough. Traces are not enough. Customers expect to be able to move between signals, correlate them, share context, and investigate problems without constantly switching tools and losing information along the way. That is now the default expectation.

But it wasn’t always like this. This was not the case as recently as 10-15 years ago.

Sematext was built around this idea early. Back in 2013 and 2014, we were already combining monitoring and log management in what would eventually evolve into Sematext Cloud. At the time, these were treated as separate product categories: New Relic was the leader in monitoring and Splunk in the log monitoring category.  For us, however, they were already parts of the same operational problem. For several years I was scratching my head and wondering why these big vendors attached only these individual observability dimensions and didn’t see what looked obvious to me, that these signals belong together:

• If a metric suddenly spikes, you want to see the logs.
• If an error appears in the logs, you want to understand what was happening with the infrastructure and application.

The value was not simply in collecting multiple kinds of data. The value was in connecting them. That basic idea has only become stronger over time.

As far as I am aware, Sematext was the first solution that combined metrics, logs, and later traces into a single observability platform, and later synthetic monitoring, user experience monitoring, infrastructure visibility, maps, profiling-related capabilities, and increasingly AI-related observability in one environment.

Things that have changed are architectures, data volumes, and terminology, but the underlying idea has remained remarkably stable:

Operational data becomes much more valuable when it can be understood in context.

Observability verticals

The rise of platforms does not mean the end of focused, single-purpose products. In fact, depending on the goal, there may be more opportunities than ever for specialists.

Platforms are powerful, but they have a natural disadvantage: they are large. Adding a completely new capability to an existing platform involves product strategy, engineering, storage architecture, data models, pricing, user experience, support, documentation, integrations, and the inevitable question of how the new capability fits with everything that already exists.

A specialist can simply focus on solving one problem exceptionally well. That creates an interesting dynamic. A startup can identify an emerging problem, move quickly, build deep expertise, and create technology that would take a larger platform years to reproduce.

Eventually, all platforms face build-versus-buy decisions and oftentimes buying is the obvious answer.

A very recent example is Dash0’s acquisition of Polar Signals. Polar Signals focused deeply on continuous profiling, including CPU, memory, and GPU/CUDA profiling. Dash0, meanwhile, is building a broader observability platform. The acquisition brings Polar Signals’ profiling technology and engineering expertise into that larger platform.

That is a model I expect we will continue to see:

1. A focused company identifies a new problem.
2. It goes deep.
3. It builds expertise and technology.
4. A larger platform recognizes that reproducing all of that internally would be expensive and slow.
5. The specialist becomes part of the platform.

We are seeing similar dynamics around AI. There is a great deal of innovation happening in narrowly focused companies building capabilities around AI agents, AI infrastructure, LLM observability, governance, security, coding agents, and autonomous operations.

Some of the specialist companies will become platforms themselves. Others may become highly valuable pieces of much larger platforms. Some will not make it.

The existence of platform gravity does not eliminate the opportunity for specialists. In many cases, it creates it.

Some recent examples of the AI acquisitions in the observability space:

• Dynatrace bought Arize AI (August 2026)
• Elastic bought Deductive AI (August 2026)
• Datadog bought Adaptive ML (June 2026)

Distributed tracing before “the three pillars of observability”

Distributed tracing is a good example of how terminology can arrive (and also evaporate!) after the technology.

Sematext added distributed transaction tracing in 2015. It was very different from what most people think of when they hear “distributed tracing” today. Our initial implementation focused on JVM applications and used Java bytecode instrumentation. The goal, however, was already familiar: follow transactions through an application and understand where time was being spent.

Today, we have OpenTelemetry and with that we have  standardized APIs and SDKs.

We have auto-instrumentation across many languages, sometimes even multiple choices, like in case of Go (see Zero-Code OpenTelemetry for Go: Runtime Instrumentation with OBI vs. Compile-Time Instrumentation with Otelc).

We have collectors, semantic conventions, OTLP, and increasingly broad ecosystem support.

But the fundamental problem has not changed.

We still want the answer to the same question:

Where did the request go, and where did it spend its time?

In 2015, the now-familiar “three pillars of observability” framing, referring to metrics, logs, and traces, had not yet become a part of the industry vocabulary. Today, the trifecta of these observability signals is so widely accepted that it is easy to forget how much of the ecosystem around these signals was still forming only a decade ago. And the name, “three pillars of observability” came and went. Nobody uses it any more. The “pillars” keep getting added as observability platforms widen their scope in an endless race to increase customer lifetime value, among other things.

OpenTelemetry itself provides an interesting perspective on how long technology maturation can take.

OpenTelemetry now feels as though it is reaching a much more mature stage in 2026. Its core signals are widely adopted, the tooling is substantially better, and the ecosystem around it is far more developed than it was just a couple of years ago.

But this did not happen overnight. The industry has spent well over a decade converging toward more standardized approaches to collecting and exchanging telemetry.

From AppMap to Service Map

In August 2015, Sematext built what we called AppMap. I dug out the original AppMap announcements from archive.org. The UI looks super retro 🙂

But behind that old interface was a problem that is still very much with us today:

How do you understand the relationships and dependencies inside a distributed system?

The technology we had available at the time was different from what we have at our disposal today, of course. There was no OpenTelemetry. There was no broadly adopted, vendor-neutral tracing standard. So we used the signals we could obtain: network packet analysis.

Fast-forward 11 years.

Today, Sematext has a slick Service Dependency Map, built on top of OpenTelemetry and modern distributed tracing. A bunch of things are different:

• The implementation is fundamentally different
• The instrumentation model is different
• The data model is different
• The backend is different
• Most obviously, the UI 😂

But at the core, both products are trying to answer the same set of questions. We still want and need to know the same:

2015                                     2026
AppMap                                   Service Map
-----------------------------------------------------------------
What talks to what?                      What talks to what?
What depends on what?                    What depends on what?
Where are dependencies?                  Where are dependencies?
Where might the bottleneck be?           Where is the bottleneck?

How we go about getting answers to these questions has changed, but the core problem remains.

Network Map: major technology change

Here is another example. We also built an early Network Map. We called it NetMap.

It was built so long ago that the original announcement is no longer available on our blog. Somewhere along the way, probably during one of those SEO and content-cleanup projects that every company eventually goes through, the original post disappeared.

The Wayback Machine still has an archived version of the old Network Monitoring product page from 2019, although the original NetMap itself was built several years before that.

Now, roughly a decade later, Sematext has a new Network Map.

The technology behind it is entirely different. The old and new versions are not incremental versions of the same implementation. The agent, the data collection technology, the backend, the frontend, they are all very different and, because of that, the capabilities and usability are substantially better.

The new Network Map is built using eBPF.

And this is where the technology cycle becomes particularly interesting (more towards the end).

eBPF gives us a fundamentally different way to observe what is happening inside modern systems. It allows us to create visibility that would have been much more difficult, or in some cases simply impractical, to build with the technologies available to us a decade ago.

And what are we doing with this, what problems are we solving, what questions are we answering? Well, we are once again answering questions such as:

• Which processes are communicating?
• Which services depend on each other?
• What is the network topology?
• Where are unexpected connections?
• What are the communication paths through the infrastructure?

In other words:

The implementation is radically different. The core operational question is not.

Profiling in 2016 vs profiling in 2026: OpenTelemetry

Another example is application profiling.

In March 2016, Sematext announced on-demand Java profiling. It was very different from what we think of as modern continuous profiling. It was limited to JVM applications. It was on-demand. You would run it for a few minutes to capture and investigate what was happening inside an application.

The goal? Understand where the application was spending its time and resources.

Fast-forward more than 10 years to 2026. In March 2026, OpenTelemetry Profiles officially entered public Alpha.

The goal is not just to define another data format. It is to make continuous, low-overhead production profiling a first-class, standardized observability signal that can be correlated with traces, metrics, and logs, and that can be used to profile all kinds for runtimes.

2016
│
├── On-demand profiling
├── JVM-focused
└── Run it when you need to investigate
 │
 ▼
2026
│
├── Continuous profiling
├── Multiple runtimes and environments
├── Low-overhead production collection
├── eBPF-based approaches
└── Standardization through OpenTelemetry

The need for finding bottlenecks inside applications running in production, but without having to attach computationally expensive and impractical profiles, has been around for decades. We finally have the means to do that. The open-source standards and technology simply needed time to catch up.

Incremental additions to observability

Looking back over all of this, I think there is a broader lesson.

Observability is often presented as a sequence of new categories.

• First metrics
• Then logs
• Then traces
• Then profiling
• Then eBPF (that provides a substantially better tech for these 4 telemetry signals)
• Then OpenTelemetry
• Then AI observability

And, yes, new technologies genuinely do create new capabilities. But they also repeatedly improve our ability to solve old problems.

Network visibility did not begin with eBPF. Distributed tracing did not begin with OpenTelemetry. Profiling did not begin with continuous profiling platforms. Service dependency visualization did not begin with today’s Service Maps.

The problems often existed first! The technology caught up later. And when new technology arrives, it often does two things at once:

1. It makes existing use cases dramatically better.
2. It makes entirely new use cases possible.

eBPF is a good example. It can be used to solve a whole set of diverse problems:

• It can help build a new generation of Network Maps
• It can also be used for profiling
• It can contribute to tracing and application visibility
• It can be used for security, network routing, filtering, etc.

And the same fundamental ability to observe what software is actually doing can be extended into completely new areas……. including AI.

The next frontier: observing AI agents

The AI era is upon us and AI agents introduce a new category of operational questions. It is no longer enough to ask the health of applications, or how much CPU they use, or what are the main bottlenecks, and so on. We now increasingly need to ask questions such as:

• Which AI agents are running in our environment?
• What are they doing?
• Which systems are they accessing?
• What tools are they using?
• What actions are they taking?
• Are they behaving according to policy?
• Are they creating security or operational risks?
• What sensitive data are they sending out to third party services?
• ….

I think we can all relate to this. That is why we recently introduced Sematext’s AI Agent Watch capabilities. This sort of functionality is desperately needed right around now, whether or not every one of us recognizes this yet or not.

And so the underlying technologies and techniques behind modern observability continue to find new uses. The same industry that spent years learning how to observe servers, processes, services, requests, and code is now applying similar thinking to AI agents.

And this is only the beginning.

 

Start Free Trial

---