Home > Can Shift Left Go Too Far? Why Testing in DevOps May Never Be the Same

Can Shift Left Go Too Far? Why Testing in DevOps May Never Be the Same

Shift Left blog post image
Testing is commonly understood to be an essential and fundamental part of software development, but when and how to test is open to a wider variety of opinions. DevOps practitioners often advocate for performance and quality testing early in the development and deployment process. This is known as a “shift left” approach. In this post, I’ll explain what shift left is, how it can be used in DevOps, and provide my take on whether shift left testing is really the latest and greatest methodology or a potential obstacle to achieving true agility in the development lifecycle.

What Is Shift Left in DevOps?

The term “shift left” comes from how we typically visualize the agile software development process, with requirements, design, and coding to the left of a linear representation while user acceptance testing and deploying to production fall toward the right end of the spectrum. The basic idea behind shift left in DevOps is to consider performance and quality testing earlier in the software development lifecycle than in a traditional waterfall methodology. It has been common to perform unit tests early to ensure basic units of code function as expected and integration tests to verify modules work together. Other software testing, such as for performance, was left to later stages, like immediately before deploying to production—or, worse, in production. Figure 1 is a good depiction of shift left. Shift Left Figure 1

Figure 1: Shift left in a nutshell

Shift Left Testing Example

Let’s look at an example of shift left testing. Consider a team of software engineers developing the back end of an e-commerce application. The system is designed to receive requests to check available stock, allocate items to orders, verify payments, and create shipping orders. Software engineers have long used automation to streamline testing, as manual testers can introduce human error. For example, development teams can employ unit and integration tests in their build pipelines by combining components and creating executable images to check code validity within development environments. Code passing these automated test cases is considered ready for deployment. While we may feel confident the unit and integration tests have sufficient coverage to find some errors, we don’t have any information on how well the code will function under normal production loads. We haven’t incorporated tests for simulating a production environment, so we don’t have any data on how the code responds under expected conditions. For this, we’ll need additional tests. Additional tests can be added to the build pipeline to execute the code under simulated production loads. During those tests, we can collect data, such as logs, metrics, and traces. This data will help us understand how well the code functions under real-world conditions.

Shift Left vs. Shift Right

Shift left is used to perform more comprehensive tests early in the software development lifecycle, but this doesn’t mean shift right testing isn’t required. The two types of testing complement each other. When we talk about “shift right” testing, we mean testing in production-like environments. This kind of testing is important for users to validate functional requirements are met along with non-functional requirements, such as scalability and reliability. Shift right testing should enable users to test code before it’s deployed to production. The focus at this point is on ensuring the service or application meets the system’s requirements. At this point, we should have already collected data on performance and reliability. If errors or deficiencies are found during shift left testing, they can be corrected before users begin their acceptance testing. While shift right testing has been common, those of us who’ve practiced it have seen its limitations. For example, I’ve had long-running data pipelines executing on costly cloud infrastructure with bugs early in the pipeline, which would have been found sooner and at a lower cost with a ‘test early and test everything’ approach.

Why Does DevOps Recommend Shift Left Testing Principles?

DevOps advocate for shift level testing principles for several reasons:
  • Cost is a primary driver to adopting shift left. The cost of correcting an error early in the software development process is significantly less than fixing it when code is in user acceptance testing or has been deployed to production.
  • Reliability is another essential feature of systems, but not one easily tested with unit or integration tests. Distributed systems, such as those built on , have components that can interact in complicated ways. Basic integration tests don’t have sufficient coverage to test the various execution paths through a distributed system. Integration tests don’t necessarily include a combination of input parameters and operating conditions, so more comprehensive test automation is used when testing for reliability.
  • Speed of delivery is another reason DevOps practitioners favor shift left testing. Since bugs and other problems can be caught early in the development process, they can be fixed faster. This helps enable a higher velocity of feature delivery since there’s no wait for user acceptance testing to find problems that could have been detected and addressed earlier in development workflows.
  • To help with security by detecting bugs and other kinds of errors early on to help reduce the vulnerable attack surface of applications. By starting with unit tests and building to comprehensive, integrated tests, you’re essentially probing the attack surface of an application, which is what attackers will often do, too. Shift left testing can help detect vulnerabilities earlier and prevent them from slipping through to production, where they can be exploited by malicious actors.

Can Shift Left Go Too Far?

So, back to my initial question: can shift left lead to too much of a good thing? I doubt it—the whole point of shift left testing is to catch problems and errors earlier instead of later because it can help save time and money while streamlining the software development lifecycle. If we expect too much testing too early, we could, in theory, impose unnecessary friction on developers. However, in practice, this is highly unlikely, given the value of finding errors early on.

How Observability Can Help DevOps With Shift Left Testing

Similar to how shift left has evolved the way we test during software development, observability can be understood as an evolution of the single, timed-based output traditional monitoring tools are built to provide. By synthesizing and analyzing multiple outputs from individual components, observability can not only help you understand what has changed over time but also the health across your systems. Since today’s applications are highly modularized, gaining this single pane-of-glass view can be key to understanding the dependencies between application performance, user experience, and the overall environment required for modern application architecture. SolarWinds® Observability provides comprehensive and integrated data built to go beyond logs, metrics, and traces to include insights from the network, database, cloud, and other crucial systems. With the ability to provide a fuller picture of performance, SolarWinds Observability can help organizations better understand how services are functioning and solve problems faster with automation features designed to help deliver optimal user experience.
Dan Sullivan
Dan Sullivan is a principal engineer and architect specializing in cloud architecture, data engineering, and analytics. He has designed and implemented solutions for a wide…
Read more

Tweets

SolarWinds's Twitter avatar
SolarWinds
@solarwinds

SolarWinds has been named a finalist for Best Observability Solution in the 2022 DevOps Dozen Awards! Vote at… t.co/w4lbMkVKcm

SolarWinds's Twitter avatar
SolarWinds
@solarwinds

Looking for the right subscription service to handle the business’s IT management needs? Look no further than the f… t.co/7SnQgo7HfG

SolarWinds's Twitter avatar
SolarWinds
@solarwinds

Learning how to implement your HADR solution is an important architecture decision that affects the continuous avai… t.co/vARkDbJyh8