Monitoring and Logging Requirements for Compliance

Learning Objectives

  • Introduction to regulatory compliance
  • Learn about SOX, ISO 27001, NERC-CIP, HIPAA, and GDPR

As developers, site reliability engineers, and DevOps engineers, part of our job is to ensure that all our software applications and systems follow the compliance regulations for our specific industry. Laws and stipulations in compliance clauses are there for a reason. Accidental non-compliance can be costly in the form of fines and loss of customer trust.

Meeting regulations is a shared responsibility, and we need to own the compliance aspects of logging and monitoring. Compliance isn’t something to put off. By the time you realize you’re not compliant, it’s too late to go back and regenerate data that’s missing. Let’s walk through the basics of regulations and how they apply to developers in practice.

Regulatory Compliance

Regulatory compliance identifies the regulations, laws, policies, and rules an organization should follow to protect the data of its employees, partners, and customers. Regulations apply across the application environment, guiding everything from who accesses the application to where the workload runs.

While most regulations have similar goals, best practices for compliance vary by industry. For example, an e-commerce application stores payment card details differently from how a hospital stores patients’ healthcare information.

Regulations also vary between countries and geographies, even when handling the same kind of data. For example, US hospitals may store patient information differently from European hospitals. However, both aim to be compliant, taking patient privacy seriously.

Besides storing data, regulations also identify policies about what information organizations should log and monitor from a systems and application perspective. This further complicates an already complex situation, as it entails the full spectrum of running an application, both from a user perspective and from a developer or operations perspective.

Types of Regulations

While it is not possible to provide a complete overview of all compliance regulations in different industries and countries, there are two main categories you should be aware of. First, there are laws and regulations requiring organizations to store additional information. The following list provides some common examples.

Sarbanes Oxley (SOX)

  • Protects: Accounting data and systems.
  • Impact for developers: SOX calls for strict auditing, logging, and monitoring of all internal systems’ activity. In particular, it requires timeline tracking to log changes to financial records. Of course, this includes when changes were made and who made them. Entities should also track and monitor user access to all financial applications to prevent unauthorized access to confidential data.

Information Security Management System (ISO 27001)

  • Protects: All data stored in systems and applications.
  • Impact for developers: Developers should log and monitor all authentications and authorizations to applications and systems. For instance, developers, DevOps engineers, and testers must have a company-wide solution in place that traces systems’ and applications’ identity access.

North American Electric Reliability Corporation / Critical Infrastructure Protection (NERC-CIP)

  • Protects: Broad incident and security protection for energy suppliers.
  • Impact for developers: Developers should log cyber-attack threats, especially login attempts, both failed and successful. For instance, the site reliability engineer (SRE) and DevOps teams should provide a cybersecurity incident logging system of unauthorized access to a critical system or data components, even if it is not possible to access the data itself.

Then, there are laws and regulations explicitly prohibiting organizations from storing information. The following list provides some common examples.

Health Insurance Portability and Accountability Act of 1996 (HIPAA)

  • Protects: Dictates what patient information to store and not to store in a healthcare-related organization.
  • Impact for developers: Developers should store audit logs for patient information, recording when this information is retrieved and from where. They should also identify what data is needed to treat the patient and what should remain private (for example, confidential information like age, gender, weight, health conditions, insurance coverage, and so on should only be accessible when necessary).

General Data Protection Regulation (GDPR)

  • Protects: Protects personal information from any citizen in Europe but applicable globally.
  • Impact for developers: The focus is on protecting the privacy of users. Developers can still build apps, but they need to guarantee the necessary guardrails are in place to prevent data collection without a proper purpose. As developers, we can use the user ID, session key, and tokens instead of auditing and logging the actual user's first name and last name or email address.

It’s time to let data charge