OWASP Insufficient Logging and Monitoring

INTRODUCTION

Logging and monitoring have been a necessity for system administrators and developers since the early days of software. Historically, most logging and monitoring efforts have been focused on the collection of operational metrics within an isolated computer system or complex network. Routine operational logs such as CPU usage, network throughput, latency, and critical system events have been used by administrators to debug problems and maintain system health. 

Increased threats towards web-facing systems coupled with advances in application logging and monitoring technology have spawned an entirely new category in the 2017 OWASP Top Ten, focused on security-specific logging and monitoring. Administrators and security teams are now tasked with building logging and monitoring programs that not only collect traditional operational metrics, but are also capable of storing, analyzing, and even mitigating a variety of attacks. 

This lesson will outline best practices as well as common pitfalls involved when building and maintaining a security-focused logging and monitoring program.

INSUFFICIENT LOGGING AND MONITORING

When it comes to application security, adequate logging and monitoring can make or break an organization during a confirmed or attempted breach. While the implementation details of an organization’s logging and monitoring program may vary greatly from application to application, there are several critical deficiencies that security teams and system administrators should be on the lookout for that may increase the risk of a breach going undetected. Below are a few examples of insufficient logging and detection as defined by A10 of the OWASP Top Ten:

  • Sensitive actions such as successful logins, failed logins, or password changes are not logged.
  • Logs are stored in only one location with no redundancy.
  • Server errors or warnings such as “500 Bad Request” error are not logged.
  • Traffic from vulnerability scanning tools goes undetected.
  • Application logs do not trigger alerts in an adequate amount of time.
  • Logs are not adequately protected and may be modified or deleted by an adversary.

It is important that we drive home the concept of security logging and monitoring as an imperative part of any application architecture. It is not optional. Without strong logging mechanisms, an organization is truly left in the dark before, during, and after any incident.

Attacks on sophisticated systems are often carried out for months or even years. Would your organization be able to detect and block a probe like this? If a motivated adversary can slowly pick apart an application for that length of time and go undetected, there is a high chance that an actual exploit will occur.  A study conducted in 2016 revealed that it took average of 191 days for organizations to identify a breach.

We can do better.

LOGGING AND MONITORING BEST PRACTICES

Building a comprehensive security logging and monitoring system is no easy task. It takes tremendous effort and cross-team collaboration to deploy a system that strikes a balance between actionable alerts, reduced false positives (and false negatives), as well as comprehensive coverage.  Whether you are building a logging and monitoring program from scratch or you have a full-blown security operations center, it is important to keep a few best practices in mind:

  • Work with security teams to enforce company-wide policies and procedures that define logging requirements in detail for all systems.
  • Check with legal and compliance teams to ensure your logging and monitoring program meets any applicable regulations such as Sarbanes-Oxley, HIPAA, or PCI.
  • Identify areas of applications that perform sensitive or high-value transactions and ensure appropriate logging and alerting is in place for these transactions.
  • Establish an incident response plan and rehearse it at regular intervals
  • Ensure that logs and alerts are generated in a standardized format so that they can be easily consumed by multiple log aggregation technologies.
  • Establish monitoring and alerting so that suspicious activities are detected and responded to in a timely manner.
  • Build logging systems with security and redundancy in mind. This includes forcing log files and database tables to be append-only and applying cryptographic signatures to validate the integrity of the logs themselves.

LOGGING TECHNOLOGIES

Choosing the right technologies for a logging and monitoring architecture can be an overwhelming endeavor. There is a dizzying amount of commercial and open source log management frameworks available to choose from. The goal should be to automate as much of the monitoring process as possible, while constantly tuning your alerts and log sources as threats evolve and your codebase changes. Over time, you should be able to have eyes on a situation before it’s too late.

The following projects are available as free and open source tools through OWASP and can help organizations get a head start when building a logging and monitoring platform.

OWASP AppSensor

AppSensor is an open-source framework that aims to provide real-time event detection and response. It defines over 50 different detection points along with a variety of response methods to any particular malicious action. These responses include logging the user out, locking the account, sending an alert, or just logging the event. AppSensor is a well-established tool that supports a variety of deployment methods and can help mitigate attacks through logging and alerting at the application layer. 

OWASP ModSecurity Core Rule Set

ModSecurity is an open source web application firewall that arms developers and security staff with insight into real-time web attacks on their applications. It provides rules to log just about any event within an HTTP or HTTPS stream and alert on custom event triggers. The OWASP ModSecurity Rule Set (CRS) is a set of generic, open source, ModSecurity rules that are built with web application defense in mind. CRS comes with vetted rules to protect against a variety of attacks, including those found in the OWASP Top Ten. Many tutorials are available for each of these systems to help developers get started with security logging efforts quickly.

CRS rules use the .data extension and are configured by modifying the ModSecurity modsec_includes.conf file to include the new CRS rules. Complete

SECURITY LOGGING INTERFACES

This component is an accordion comprised of collapsible content panels containing display text. Select the item titles to toggle the visibility of these content panels.

Application logging has long been a staple of traditional software development and can come in many different flavors. Typical logging interfaces give developers the ability to log events ranging from expected parameters being passed into a function (INFO) to application-halting exceptions (CRITICAL). 

While traditional logging interfaces can help security teams glean some insight into suspicious behavior, the logging mechanisms often fall short when it comes to building a features security event monitoring system. This is where security logging interfaces come into the picture. 

When using security-specific event logging patterns, teams can benefit from understanding what attacks may or may not actually be going on in real time under the hood of an application. A few benefits of using security logging interfaces include the following:

Diagnositics and Forensics

When investigating a breach or suspicious behavior, having access to relevant and high-integrity logs. Building security-specific logging interfaces can help carry out rapid responses in the event of a forensic investigation.

Security Alerting

Triggering actionable, security alerts can make the difference between quickly blocking an attempted attack and ending up in the news due to a breach. Alerting on suspicious behavior collected from logs allows security teams to gain insight into the actions of adversaries and act quickly during early stages of an attack.

Compliance

Many regulatory and compliance controls require that organizations collect and analyze logs and alert on potential security concerns. A well-established security logging program can help meet and exceed these requirements.

Integration with Existing Security Tools

Logging and alerting mechanisms often work best when integrated with other tools already in use. Chat applications such as Slack and operations management platforms such as PagerDuty are great places to feed security logs and alerts. 

Below are a few examples of events that are worthwhile to log and alert on when appropriate:

  • Changed password
  • Authorization failures
  • Failed login attempts
  • Successful logins
  • Missing CSRF token
  • Password reset attempts
  • Missing HTTP header
  • User ID or email change

In the same way logs can help prevent a breach, they can also be a juicy target for attackers due to their sensitive nature. Below are some best practices for implementing a security logging interface in your application:

  • Secure and back up your logging architecture.
  • Always omit or filter sensitive information from logs.
  • Classify logs when possible per your organization’s security and compliance requirements.
  • When possible, associate logs with session identifiers or a user identifier.

OWASP Security Logging Project

Open source projects exist to help build a security logging architecture for your applications. The OWASP Security Logging Project is one of these projects that aims to provide developers with APIs for logging security-related events, using logging tools and frameworks that they are already familiar with, such as Log4J.

KNOWLEDGE CHECK

This component is a multiple choice question. Once you have selected an option select the submit button below

  • Which metric is most indicative of an active attack on a web application?Two 500 errors over the course of 24 hours
  • 50 failed login attempts using the same username in less than 1 minute
  • CPU threshold reached on 2 of 12 load-balanced servers
  • Excessive 404 errors after a new feature was released

A high number of failed login attempts in an extremely short period of time is a sign of a brute force attack. This is a simple metric that should be logged and alerted on immediately.