Mastering Telemetry Pipelines: Driving Compliance and Data Optimization

I had the opportunity to present with Michael Fratto, Senior Research Analyst at S&P Global Market Intelligence, at a virtual event hosted by Redmond. We discussed how telemetry pipelines are critical in controlling telemetry data (logs, metrics, events, and traces). 

Mike shared excellent insights from his recent research survey that discussed the proliferation of observability tools in enterprises and the challenges organizations face in managing those tools.

‍

He also discussed how organizations are centralizing the collection and distribution of telemetry data and the benefits that organizations are enjoying from the deployment of telemetry pipelines. 

‍

Key Takeaways:

• Telemetry pipelines’ usefulness grows with more sources and destinations
• Pipelines improve governance, management, and enforcement
• Filtering and routing data to destinations to save capacity and processing costs
• Using features such as redacting, masking, and encrypting act on data in motion to help with compliance
• Monitoring and reporting of the telemetry pipeline is critical
• Automatic detection of new data streams and types reduces leakage

‍

The benefits of pipeline go well beyond log volume and cost reductions. In addition to using pipelines as pre-processors of data going to observability and SIEM systems, they are also used to support your compliance initiatives. 

‍

I covered some practical examples of how enterprises deploy pipelines to understand and optimize their data for log reduction and use it to meet their compliance needs.

‍

‍

We are happy to share an accompanying Business Impact Brief by S&P Global Market Intelligence titled “ Observability pipelines offer both cost-saving and compliance-related operational benefits.”

‍

The complete webinar recording is available here. As always, if you have any questions, please contact us.