Open Source Code Quality On Par with Proprietary Code in 2011 Coverity Scan Report

In 2011, open source projects in Coverity Scan were upgraded to the Coverity 5 development testing platform analysis engine to accommodate significant advances of the maturity in static analysis technology over the past five years – in particular, the ability to find more new and existing types of defects in software code. The 2011 Scan report details the analysis of Scan’s most active open source projects, totaling over 37 million lines of open source software code. In addition, the report details the results of over 300 million lines of proprietary software code from a sample of anonymous Coverity users.

Key findings from the 2011 Scan report include:

Over 37 million lines of code from forty-five of the most active open source projects in Scan were analyzed. The average open source project in Scan has 832,000 lines of code. The average defect density, or the number of defects per thousand lines of code, across open source projects in Scan is .45.

Over 300 million lines of code from forty-one proprietary codebases of anonymous Coverity users were analyzed. The average proprietary codebase has 7.5 million lines of code. The average defect density for proprietary codebases of Coverity users is .64.

Both open source code quality and proprietary code quality, as measured by defect density, is better than the average for the software industry, which is a defect density of 1.0.

Linux 2.6, PHP 5.3, and PostgreSQL 9.1 are recognized as open source projects with superior code quality and can be used as industry benchmarks, achieving defect densities of .62, .20, and .21 respectively.

Open source code quality is on par with proprietary code quality, particularly in cases where codebases are of similar size. For instance, Linux 2.6, a project with nearly 7 million lines of code, has a defect density of .62 which is roughly identical to that of its proprietary codebase counterparts.

Organizations that make a commitment to software quality by adopting development testing as a part of their development workflow, as illustrated by the open source and proprietary codebases analyzed, reap the benefits of high code quality and continue to see quality improvements over time.

In addition to the 2011 Scan report, Coverity also announced the appointment of Zack Samocha as the new Coverity Scan Project Director. Samocha spent nearly a decade at Mercury Interactive, now Hewlett-Packard, where he was instrumental in the development of its quality assurance testing products and building quality best practices within enterprise application development. His enterprise development and testing experience will be instrumental in bringing development testing best practices to the open source community.

"The quality of our code is critical to the ongoing success and adoption of PHP, which includes some of the world's most popular web sites," said Rasmus Lerdorf, creator of PHP. "As our code grows and becomes more complex, Scan will become even more important for us as a way to help improve our code quality."

“The line between open source and proprietary software will continue to blur over time as open source is further cemented in the modern software supply chain,” Zack Samocha, Coverity Scan Project Director. “Our goal with Scan is to enable more open source projects to adopt development testing as part of their workflow for ongoing quality improvement, as well as further the adoption of open source by providing broader visibility into its quality.”

To obtain a copy of the 2011 Scan report, register here.

About Coverity
Coverity, Inc., (www.coverity.com), the development testing leader, is the trusted standard for companies that need to protect their brands and bottom lines from software failures. More than 1,100 Coverity customers use Coverity's development testing suite of products to automatically test source code for software defects that could lead to product crashes, unexpected behavior, security breaches, or catastrophic failure. Coverity is a privately held company headquartered in San Francisco. Coverity is funded by Foundation Capital and Benchmark Capital. Follow us on Twitter or check out our blog.