Introducing Responsive Pipelines from Mezmo

Introduction

The ability to swiftly resolve incidents is central to SREs responsible for a service's reliability and its users' satisfaction. Mezmo has recognized this need and, at Kubecon, unveiled an innovative solution: Mezmo Responsive Pipelines. Responsive Pipelines enable users to pre-configure a Pipeline to respond automatically in the case of an incident. This solution makes it easier to balance the need for high-fidelity data required during incident response with the need to reduce data load for cost reduction.

Why Responsive Pipelines are Essential

The core idea behind Responsive Pipelines is simple: Enable Pipelines to adjust their behavior in response to specific triggers, such as incidents. This capability is crucial for Site Reliability Engineers, who need high-fidelity data during critical moments while managing the usual trade-off between cost savings and data volume. Most organizations want to retain all their telemetry data, just in case - even though surveys show that only 2% of the data is ever used. Responsive Pipelines empower organizations to optimize their telemetry data use, leading to cost efficiency and enhanced incident response effectiveness.

‍

Understanding Responsive Telemetry Pipelines

Responsive Pipelines enable users to define and name different pipeline states and offer visual indicators for processor status corresponding to these states. These states can be toggled through API calls from external incident workflow systems, such as PagerDuty, ServiceNow, or Opsgeinie. Furthermore, the system visually represents deactivated pipeline processors and allows users to revert the pipeline to a normal state manually or via API calls. This balance between cost-saving data reduction and the necessity for full-fidelity log data during incidents is a core feature of Responsive Pipelines.

Features and Benefits

• Customizable Pipeline States: Users can establish and name states, adapting the pipeline to their specific needs.
• Visual Status Indicators: Easy identification of processor status, enhancing user experience and response efficiency.
• API Integration: Seamless toggling between states using API calls for collaboration and incident management.
• Flexibility and Control: The ability to manually or automatically return the pipeline to its standard state.
• Cost Efficiency: Reduces the conflict between cost savings and the need for high-fidelity data during incidents.

Responsive Pipelines in Action: A Use Case

Imagine an incident where SREs require immediate access to detailed log data. Responsive Pipelines immediately switch to an 'incident' state, either through an automated trigger or an API call. Processors that usually aggregate or condense logs are disabled, providing SREs with the comprehensive data they need. After incident resolution, the pipeline will return to its 'normal' state, resuming its cost-efficient operation.

To see a more detailed example of Responsive Pipelines in action, check out this Mezmo Product Demonstration from Chelsea Wright.

Conclusion: The Importance of Responsive Pipelines

Responsive Pipelines from Mezmo introduce new flexibility in how organizations handle telemetry data in real-time scenarios. They offer a balance between data fidelity and cost efficiency, ensuring that businesses are equipped to handle incidents with agility and precision.