Shift Right Testing (Do I need it? How Is It Done?)

Curious about shift right testing? We explain what shift right is, how it’s done, how it differs from a shift left approach, and whether it is important.

What is shift right testing?

Shift right testing refers to continuously performing tests after the software is in a production environment. Techniques include:

• Fault injection testing
• A/B testing
• Canary deployment
• Chaos testing

The easiest way to think about shift right testing is to look at it as testing software in a live environment. Shift right testing happens in the pre-release and post-release phases of the application lifecycle, with a wide variety of tests aimed at measuring how the software performs in a live environment. 

Doing so can help you spot issues that teams may overlook during the development process and helps widen the net of errors teams are looking for. While there might be some debate on whether to prioritize shift left vs. shift right, they are both important in their own ways for creating a better customer experience.

Shift right testing happens later in the development process through continuous monitoring, logging, analysis, and incremental fixes. SRE teams work closely together with development teams to ensure that the software performs the way it should. As a result of this comprehensiveness, the concept of shift right testing has gained popularity in DevOps. 

Shift right vs. shift left isn’t really an either/or scenario; in this case, it’s a way to ensure testing occurs at each part of the application development lifecycle. This process is referred to as continuous testing.

Why does shift right testing matter? 

Employing both shift left and shift right testing ensures that you have a comprehensive strategy in place for testing, which leads to a far better development experience for teams and a better product for customers. Shift right, in particular, is crucial because it tests the software’s performance by modeling the way customers use it. This gives you confidence that if the software is acceptable now, it will be acceptable to users.

Deploying software without shift right testing means that the user experience might be compromised if the deployment is not adequately monitored. In addition, it’s hard to predict how the software will respond in production, which has many external factors, such as third party dependencies or fluctuating resource usage. Shift right testing helps alleviate many of these concerns by allowing you to test in an environment that simulates these factors.

Shift right testing includes using methods that mimic the way users interact with the product combined with automated testing tools. These tests help uncover issues with performance, functionality, and user experience that teams can fix in small increments rather than putting out major fires. 

As data comes through during production, teams can use it to identify and fix problems while continuing right-shifted production tests to proactively find upcoming issues.

Different types of shift right testing

As the software finishes development and goes into the hands of users, shift right testing comes into play. The main aims of the tests are to understand usability, stability, and emerging production issues. 

• A/B testing: Two versions of the same features are released to test responses. Used to understand software usability and user preferences.
• Blue/Green deployment: Two identical production environments are maintained, with one in use and one held as a backup. If new code causes an issue when deployed, the system can quickly switch to the other environment.
• Canary testing and feature flagging: Used to test code releases by releasing to a small sample to understand stability.
• Fault injection: Purposely introducing errors to the software to see how it performs and if it can recover.
• Production testing: Using tools like Sentry or Bugsnag to notice abnormal behavior in production as soon as it occurs.
• Load tests, usage scenarios, and other tests are added based on user data and any production issues that have been flagged early on to investigate scenarios that seem like they could cause issues.

How do I get started with shift right testing?

The first step to getting started with shift right testing is to ensure you’re collecting the right data. Your system produces tons of data, and you need to understand which ones reflect service health and the results of tests. Monitoring tools can help you sort through to what matters.Once those measures are in place, some processes need to be implemented for shift right testing to be successful. 

Teams need to start considering application monitoring, which is collecting log data around usage, availability, bugs, and other measures. Other monitoring can also include API monitoring and collecting API information to identify issues specifically related to APIs and third party components. Once core shift right tests are identified and performed, teams can start thinking about the scope for automation and how to speed up the shift right process. Tests that are simple or frequently used are good targets for automated testing.

What are the benefits of shift right testing?

There are several benefits that come with shift right testing for customers and teams. Firstly, shift right tests are often more consistent between different projects. As the testing occurs later in the lifecycle, the environment for testing and requirements are standardized. This allows automation and other optimizations to provide greater benefits, as they’ll apply to more frequently used tests. 

Secondly, continuous testing leads to a better customer experience. Any feedback coming in helps guide teams on what to fix and test, creating a more laser-focused strategy to ensure reliability and usability. As data is collected, teams can start to make more accurate real-world scenarios to test in the production environment, leading to better processes that ensure customer issues are solved. 

When it comes to shift left vs. shift right, there are benefits to both. With shift left testing, there’s more focus on getting code right and getting it to customers quickly. Shift right complements this by improvising usability once the code is released to improve the customer experience. 

How Blameless helps

With shift right, there is a lot of opportunity for both SRE and development teams. SRE teams can focus on reliability and creating a resilient system while still keeping the customer experience central. For development teams, shift right testing offers more comprehensive coverage of testing that is modeled against real-life scenarios. Teams can gain more value from their data by monitoring and logging customer usage, thereby creating a better and more robust process that benefits teams and customers. 

Part of the process of adopting shift right testing is creating a blameless culture and having the right tools in place to help teams reduce workload and spot errors. Blameless offers a broad suite of tools to optimize each stage of incident response, enabling teams to act faster and better. 

With Blameless, teams can structure service level objectives and monitor reliability insights to help speed up development velocity and create a better product. To learn more about how Blameless fits into shift right testing! For more insights into DevOps, SRE, and more, make sure to check out a demo and sign up for our newsletter.

‍