Don’t let testing stop your agility

AlexQin

Utilize beta groups. One practice testers can use to ensure their product is working without sending it into production is to have it tested internally to a limited amount of users. “What that allows you to do is get field testing and real-world testing back so the developers can react quickly and then fix defects, bugs or any kind of confusion that occurred from that testing,” said John Basso, CIO and cofounder of Amadeus.

Additionally, instead of rolling out to 100% of your users, teams can roll out to only 10% so that if anything happens it only impacts a subset of the population and can be fixed and re-deployed quickly, Basso explained.

AlexQin

Test less, but fix faster. Since testers no longer have the luxury of waiting until the end of a development cycle to test, Rogue Wave’s McLeod believes it is sometimes easier to test less, and then live test. Of course, he notes that it all depends on if your software is mission-critical. “If your software keeps airplanes in the air, it is probably not the best approach,” he said. “But if your software is hosted, fast, easy to update, and maybe isn’t too mission-critical, I think to some degree you should take a little more risk in your release. And since you are working in smaller increments you can fix it much more quickly.” The idea is to deliver value to your customer and to be in a position where you can respond if something doesn’t work, McLeod explained.

This is taking a shift-right approach rather than a shift-left approach, according to Micro-soft’s Kamat. “If we can’t do enough testing before you release, you have to complement the risk exposure by monitoring the application in production, so that if there are any issues, you are the first one to detect them, and you can rectify them quickly,” he said.

It is important to note that this does not eliminate the need to shift left, Kamat explains. “The more testing you can do early in the life cycle with more automation, the better the results,” he said.

AlexQin

Agile doesn’t mean fail
There is a popular saying in agile and general software development initiatives: Fail fast and fail often. The point of this saying is that organizations will fail, but they will learn from their failures. While this saying might work for some companies, this is not a good mindset to have.

According to RBCS’s Black, falling into a fail fast, fail often mentality is dangerous because it makes it sound like getting things right the first time doesn’t matter. “I suppose this attitude is appropriate or at least tolerable if you are building the Kim Kardashian game to run on someone’s mobile phone,” he said. “It is not an attitude that is in any sense tolerable if you are building self-driving cars.”

Black explained organizations often adopt the philosophy of just throwing stuff out there because if it is broken they can just release another update, but that can make the life of testers and developers difficult. “As software becomes more and more a critical part of the infrastructure of our lives, the importance of quality in software goes up, not down. Therefore sloppy and ‘ready, aim, fire’ approaches become less appropriate rather than more,” he said.

Instead of rushing to get stuff out there quickly, developers and testers should ask themselves if they can rush their product out because it doesn’t matter if it’s broken, or is it something that is being implemented in self-driving cars, elevators, pacemakers, and other critical solutions, Black explained.

“Agile isn’t just stuff that gets out fast. The compromise that is going to be made is almost inevitably quality because it is certainly less visible than budget and schedule, and people are very good at counting days and counting dollars. But quantifying what is an acceptable level of quality, organizations aren’t always good at that, and that can lead to some bad decision-making,” he said.

AlexQin

Updating your skillsets to stay relevant
Understanding that addressing test is one of the most essential parts of the transformation is only the first step. Businesses need to figure out in this new agile world who is responsible for testing and what constitutes testing, according to Gartner’s Murphy. “In an agile organization, everyone plays a role in quality and testing. Developers test, testers test, product owners test, and, yes, the users do certain types of testing,” he said.

So when developers and testers are stuck in the middle of this transition, they need to understand their skillsets need to change immediately.

According to Parasoft’s Ariola, testers need to have a clear understanding of the business and how software is going to impact the business. “This means your blinders associated with just writing tests need to be removed, and you are going to have to understand a much more global perspective of what you are doing as a tester,” he said.

Instead of just testing parts of the application, testers also need to understand its breadth. “You have to understand how your application runs, but you are also going to have to understand the dependencies associated with the application,” said Ariola. “Meaning, what does it connect to? Are there third-party connections from web services? Are there other APIs that this application is dependent on in order to do its job? Does it connect into a mainframe? You are going to have to get to know the architecture associated with that application quite well.”

Having a thorough understanding will also help testers monitor and react better. “They need to be able to really understand how an application works so when it fails they can better make sure the quality is there because they understand how it works, and understand how it could break,” said cPrime’s Irani.

The expectation is that testers are going to be able to participate in not only the discussion about requirements raised as user stories, but also discussions about automation of tests associated with behavior-driven development and acceptance-test-driven development, according to RBCS’s Black.

The job of a tester is no longer to figure out how to manually do things. Their job is to ensure the quality of a product, according to Amadeus’ Basso. “It becomes less about the testing and more about how you can make this whole ecosystem of development, deployment and monitoring better in such a way you can make the quality higher,” he said.

This new role is being seen as a quality engineer, according to AppDynamics’ Stoner. “Quality engineers have to have some coding skills so they can sit with development and build test cases side by side,” he said. “Quality engineering is increasing the quality of builds and developments so that less time has to be used finding these issues. To do this, testers need to get brushed up on their automation skills, Continuous Integration skills and Continuous Delivery platforms”

In order to brush up on those automation skills, testers need to become more technical. This means being able to write code and scripts in order to automate things. “Testers need a growing amount of ability to write code because most testing tools and the most effective way with most systems to automate is you must write code, you must understand the code you are testing, and you must feel comfortable treating your testing assets as code elements,” said Gartner’s Murphy.

According to Trailblazers’ Taylor, it doesn’t necessarily have to be any one language in particular. “What I look for is whether or not they have the ability to pick up some form of automated test scripting, which is really a lighter form of programming,” he said.

For organizations that have hundreds of manual testers, they can’t just hire someone to automate all their manual scripts, according to cPrime’s Irani. “You have to spend the time and money to invest in architecting out how you should be automating testing,” he explained. “You have to think about who you have that can learn those skills, and then acquire the gaps of skills you don’t have.”

In addition, you can’t just train all your manual testers to become automation test engineers. “You have to architect a solution, figure out which of your people can make the transition, and then provide them with the training and skills they need,” said Irani.

On the other side, as roles and responsibilities between testers and developers become more blended, developers need to be more familiar with tests. They have to understand the value of a good unit test. “Writing good unit tests around your code gives you a safety net to make changes down the road,” said Taylor. “If you want to continue to go fast, you need to have that safety net so you can make changes without fear or without having to second-guess yourself, or without having to introduce a lot of defects into production.”

And as developers become more “test infected,” the importance of testers becomes greater. “Developers are typically more familiar with code and writing the unit test, whereas testers have an eye for details, they have expertise in terms of connecting with real-world customers, and they have a better understanding of application topologies in the real world.”

In order to work in this new world seamlessly, developers and testers need to make sure they have good communication skills in place because they need to be able to work together to find and execute the right suite of tests. “The more testers understand, the more respect they are going to get from developers,” said Irani. “The more respect they get from developers, the more they are going to work with testers, collaborate and teach them as well to help them get better.”

AlexQin

Stakeholders need to change their mindset
Transitioning to agile is not only an initiative that happens at the team level. Stakeholders and executives need to be able to change their way of thinking to accommodate this way of working, according to Microsoft’s Kamat.

According to him, stakeholders have grown up with a mindset that if they feel confident about releasing a particular solution, they should release it. “Stakeholders need to understand that just by throwing more people and more hardware doesn’t mean they will be successful,” he said. “The agile principles don’t say that you are still going to do everything you have done before, but in a compressed timeframe.”

The intent is to focus on what has changed, what is being used in production, use the data points to test, and have confidence in the testers that they understand what the business risks are.

“Testers in an agile world need to be supported by the stakeholders to say yes, we are going to understand the risk, and we are going to mitigate the risk by monitoring the application in production.”