The Current State of Open Source Testing Tools
The Current State of Open Source Testing Tools. Tricentis and three survey project collaborators recently published findings that provide new insights on global trends in open-source testing.
The study focused on the open-source tool testing industry, not how or if companies and software developers test their code. This is an issue that confronts both open-source and commercial or proprietary software. The Current State of Open Source Testing Tools.
The results revealed that a lack of technical skills is the major roadblock to open-source tool adoption, with 30 percent of organizations surveyed saying they lacked the skills to adopt open-source tools, according to Kevin Dunne, senior vice president for strategic initiatives at Tricentis. The Current State of Open Source Testing Tools.
Cost is the single, largest driver of open-source tool adoption, with 39 percent of those surveyed stating it as the number one benefit. Other factors, such as community support (16 percent) and ease of customization (14 percent) also rank highly.
The published report is gleaned from 2,000 global survey respondents that work in functional testing, load/performance testing, or behavior-driven development. The Current State of Open Source Testing Tools.
The Current State of Open Source Testing Tools. The results sho.ws open-source testing tools gaining support as viable options for even the largest and most security-focused enterprises. But some respondents still face opposition to open-source security and support within their organizations.
The Current State of Open Source Testing Tools. “With the current economic climate impacting tool budgets at many organizations, we believed it was critical to further examine the developments related to open source and free testing tools,” Dunne told LinuxInsider.
Open-source testing tools are most often used by those identified as testers or quality assurance workers. However, what was surprising was the low level of engagement from developers and engineers within this survey, noted Dunne. The Current State of Open Source Testing Tools.
Respondents averaged almost nine years of experience in the engineering field. The majority (52 percent) are in quality assurance roles. Software development engineers comprised 14 percent of the survey respondents. The Current State of Open Source Testing Tools. Development engineers (8 percent), and other roles (26 percent) comprised the rest of the survey participants.
Tricentis collaborated with TestProject, SpecFlow, and Flood to capture a snapshot of where the industry stands today and track how open-source testing trends evolve in the years to come.Tricentis claims one of the largest open-source testing communities in the world.
The survey was conducted in April 2020 and was distributed online via SurveyMonkey. Sponsor companies promoted the survey via mailing lists, social media, and partnerships with testing media outlets, Dunne said. The Current State of Open Source Testing Tools.
Results Across the Board
Tricentis assembled a panel of experts across each testing category or domain to take a deep dive into the results. The Current State of Open Source Testing Tools. The panels focused on Behavior-driven development (BDD), functional testing and load/performance testing.
Here are the key findings in each area, according to Dunne:
- Functional Testing: 84 percent of organizations still employ dedicated testers to perform functional test automation, despite an alleged trend to have developers replace dedicated testers to build functional test automation.
- BDD: Only 21 percent of organizations consider themselves experts when it comes to BDD. Fifty-one percent of organizations are developing their test scenarios in a “Given-when-then” format, but there is still room to grow when it comes to automating these tests completely and organizing that code for reuse.
- Load/Performance Testing: Though performance is critical to customer onboarding and retention, only 56 percent of companies are performing load/performance testing on a regular basis. Time, training, technical skills and proper testing environments remain the largest impediments to more frequent load/performance testing.
These are the most significant high-level takeaways the survey reveals, Dunne maintained:
The Current State of Open Source Testing Tools. Offshore testing still seems to be popular and thriving, with 61 percent of respondents coming from Asia. Most are located in India and employed with large system integrators.
“With COVID-19 allowing everyone to work remotely, the previous notion that agile
Software Testing Not Universal
Open-source platforms vary wildly when it comes to testing. The Current State of Open Source Testing Tools.Many of them are not tested on a regular basis, or they are not tested completely, according to Thomas Hatch, CTO and co-founder at SaltStack.
“The Tricentis survey gives us a good insight into the trends impacting open-source testing and testing tools. I think it is a step in a positive direction, particularly with regard to awareness of the issues,” he told LinuxInsider. The Current State of Open Source Testing Tools.
The roadblocks highlighted in the results can be boiled down to an unwillingness to change and a general lack of expertise. The expertise issue highlights a broader problem with software engineering today, he offered. Our tools are more customizable than they are easy to use.
“This speaks to a fundamental issue in open source. It creates significant amounts of innovation, but not very much refinement,” explained Hatch.
The trends regarding industry practices for open-source testing swing like pendulums. He sees a greater shift toward the usage of open-source testing tools.
“But as it becomes clear that we need expertise and the overhead associated with using these open-source tools is more widely known, I think we will see a shift back toward the adoption of enterprise-grade tools,” Hatch countered.
Testing’s Key Role
The survey shows a trend toward automation. More companies reported using automated functional testing than manual testing (50 percent) approaches. One surprising result, according to the report, is the low number (31 percent) of respondents performing unit testing of their code.
Other results in this category include:
- Coded tests (67 percent)
- Exploratory Testing (32 percent)
- Load Testing (23 percent)
- Security Testing (14 percent)
development would shift focus away from offshoring seems to not be the case,” he said.