Log Metrics and Log Data

There is a role for both in a cloud native world.

While you can send log data directly to Wavefront, the primary use case there is to convert logs into metrics. VMware Log Insight can do some of this as well, but is a better place to data mine the log data.

Metrics and log monitoring are complementary. If you embrace the power of tracking everything that moves in your environment, then you’ve added instrumentation to likely thousands of places across your codebase.

Metrics give you an aggregated view over this instrumentation, and Wavefront can derive metrics from the log stream. Logs give you information about every single request or event, and the log data can be archived in log data mining tools for root cause analysis, debugging, and troubleshooting.

Metrics are the best place to start dealing with a problem. They’re where to look for trends. They’re also the basis for performance-related alerts.

Combined with well-designed dashboards, they help you to quickly isolate to which subsystem of which application is behaving oddly. From there you can deep dive with profiling tools, data mine your log archive and cross-check against the source code itself.

Deploying scalable and reliable monitoring platforms has been the goal of SREs for quite some time. With the ever increasing volumes of both human and machine generated data, the need for such platforms is greater.

Wavefront is optimized for time-series metrics. Now it derives metrics directly from log data as well.