Automation testing is a reliable approach that can make the software testing process easy, fast, and convenient. But many specialists in the QA field have claimed that automation testing failed or did not meet their expectations. One of the main mistakes is that development teams rely too much on automated testing or use it improperly. Automation testing is a robust tool, but not a panacea. And like every tool, it requires an experienced master who knows how and when to use it.

Problem 1: What to Automate

Every company has its requirements for a product. Scope for automation testing depends on the type of project and its end goals.

Sometimes, companies want to cover as much as they can by automation, even when it isn't necessary. Some development teams believe that they can automate approximately 100% of test cases, or that automated testing will do all the work for them, or that manual is less useful than automation. But all these myths lead to failures of automation. Moreover, the thoughtless usage of automation brings more material and physical expenses. 

According to Michael Cohen’s test automation pyramid, every automated testing process should have a robust base from unit tests. Such an approach gives instant feedback and prevents failures. Manual and exploratory testing are at the top of the pyramid on the level of UI testing that we can't automate. Manual testing allows us to examine new functions as it is cheaper than automation. But as the number of test cases increases, the cost of manual testing grows, too. In this case, automation can reduce expenses. At the beginning of a project, automated testing is expensive, but with the project growing, the price of automation decreases. 

Some cases that we can automate:

• Regression testing. This is a type of testing with the primary goal to ensure that code changes don’t have unwanted side effects. To test it manually after each launch can be time-consuming. In a big project with millions of code lines, manual regression testing is an impossible task. Automation helps save time so the development team can focus on the primary tasks of a project.  

• High-risk test. Certain functions are crucial for the business model, and their crashes can bring a lot of risks for product owners. Automation testing after each release can quickly identify problems and eliminate them.

• Repetitive/big data test cases. Manual testing of test cases with big data can lead to a large number of errors related to the human factor. Engineers can't cope with checking thousands of code lines, which makes it easy to miss failures. Automation easily copes with complex monotonous work and allows teams to focus on core goals.

Developers teams should not reject manual testing. The right strategy with balancing manual and automated testing only enhance the development process. Use these two powerful tools in collaboration according to project goals.

Problem 2: Unrealistic Expectations

As we said earlier, automated testing is a useful method, but not a panacea. We can't achieve 100% automation and transfer all work to the algorithms. 

People who make management decisions are rarely into software testing. It leads to unrealistic expectations. Paul Grizzaffi, a principal automation architect at Magenic, said that he frequently faces questions like, “How long does it take to automate 500 test cases?”

Managers don't understand the principles and scope of work, so they expect that automation will do all work quickly and effectively. But large parts of testing such as risk analysis, creating strategy, communication and cooperation with developers, and evaluating results can only be done manually, which takes time. Furthermore, writing algorithms for automation requires deep analysis and knowledge of product specifications.  

Also, sometimes, teams have to learn new knowledge or master a new automation program, which also takes effort and time. 

Problem 3: Lack of Collaboration 

Testing is an essential part of the development lifecycle and continuous integration process. The development delivery approach Agile prescribes that testing should be a simultaneous activity along with writing code. Such an approach prevents accumulating bugs and identifies failures early. 

But one of the reasons for automation testing failure is that testers and developers work separately. 

Also, late implementing automation testing in the development process brings even more problems. Implementing testing activity at the beginning helps to identify critical failures earlier and won't delay product release. Identifying bugs is much cheaper than fixing them. Testers have a system point of view and focus on users, which can help the development team ensure a better program decision. Collaboration between engineers and developers facilitate creating tests by considering automation. This has more effect than automation test cases for manual purposes.

Problem 4: Wrong Tool

Choosing a robust and flexible tool is important for successful automation testing. First, you should consider the project requirements, e.g. type, scope, and level of team experience. 

Our team faced the problem of an inappropriate tool. Our company tested a web application for a software development company. A tool that we used maintains only web and desktop platforms. But when the company grew and expanded, product owners decided to design a mobile app. Our team was forced to choose and learn a new automation tool, which took some time. 

Another case happened with a team where I took an internship and had useful experience and lessons. The company developed an app where GUI was automated. After the UI was improved, automated tools couldn't recognize the objects, and testing was inconsistent. Updating the app was delayed due to this situation.

Here are some recommendations for choosing an automated testing tool:

• Easily integrate with other test management tools to keep process clarity.
• Tool reports should be understandable for all team members. 
• Consider technical skills and experience that tool is required.
• Suitability for project expansion.
• Support program languages required for your system.

If you already have a tool, you can compare it with another to evaluate the pros and cons.

Sum Up

Automation testing is a robust and helpful tool to facilitate the development process. You should be knowledgeable and be able to deeply analyze your options to avoid failures during automation. We assumed the most common automation failures that you can face, but every project and its problems are unique.