Open source software security in an ICT context – benefits, risks, and safeguards
Director of Security, Network Product Solutions, North America
Master Security Specialist
Master Security Specialist - PSIRT (Product Security and Incident Response Team), Ericsson Network Security
Director of Security, Network Product Solutions, North America
Master Security Specialist
Master Security Specialist - PSIRT (Product Security and Incident Response Team), Ericsson Network Security
Director of Security, Network Product Solutions, North America
Master Security Specialist
Master Security Specialist - PSIRT (Product Security and Incident Response Team), Ericsson Network Security
Many information and communications technology (ICT) vendors and communication service providers (CSPs) leverage open source software (OSS) for their software projects and products with the purpose to enable communications service providers to build open, interoperable networks at a lower cost. Examples of industry collaborations promoting the use of open source code are the Open Network Automation Platform (ONAP) and O-RAN Software Community (OSC) hosted by the Linux Foundation (LF), and Openstack hosted by the OpenInfra Foundation. OSS has inherent benefits that can provide secure code, but also has inherent security risks that require a higher level of due diligence. It is the responsibility of the software product vendor to ensure proper safeguards are in place for secure use of shipped product with OSS and proprietary software components. Ericsson is an advocate for proper use of open source software and has active leadership roles in the OpenChain Project and O-RAN Alliance.
OSS has many benefits
The National Institute of Standards and Technology (NIST) defines OSS in S 6106.01 as “Software that can be accessed, used, modified, and shared by anyone. OSS is often distributed under licenses that comply with the definition of "Open Source" provided by the Open Source Initiative and/or that meet the definition of "Free Software" provided by the Free Software Foundation”.
OSS is a powerful tool that can be used by organizations to accelerate innovation while reducing the development timeline, product time to market, and overall cost. OSS also reduces fragmentation and increases interoperability among different products by producing components and protocols that become the de facto standard. OSS provides a platform for talented coders to openly collaborate and build software. Open source works optimally when developers behave as “good citizens” in which consumers also contribute, provide useful feedback, and share fixes. The transparency of code reviewed by many expert eyeballs reduces software complexity and the number of bugs. This crowdsourcing approach to software development has effectively produced quality software at low cost.
Software developers use open source development platforms, such as Microsoft’s GitHub to host code and manage software projects. These development platforms maintain source code repositories, open source libraries, and open source communities. Open source projects benefit from the support of foundations such as The Linux Foundation, Eclipse Foundation, OpenInfra Foundation, Apache Software Foundation, and GNU Project. OSS is available for the Linux kernel, Openstack, Docker, Kubernetes, GNU Compiler Collection (GCC), and others.
OSS comes with risk
Like so many things in life, OSS’s advantages can be exploited as disadvantages, and its strengths can be exploited as weaknesses. The 2020 Open Source Security and Risk Analysis Report from Synopsys cited 99 percent of codebases contained open source components and 49 percent contained high-risk vulnerabilities. While the community approach benefits OSS, it also provides an attack surface. OSS has many attack vectors, including intentional backdoors made by malicious developers, propagation of vulnerabilities through reuse, exploitation of publicly disclosed vulnerabilities, and human error.
Vulnerabilities often propagate as developers reuse OSS, enabling attacks on software that reuse the vulnerable software component. As open source vulnerabilities can propagate, it becomes more difficult to ensure patches are implemented. “Using Components with Known Vulnerabilities” made the Open Web Application Security Project (OWASP) Top 10 Vulnerabilities for 2020. In addition, vulnerabilities are publicly disclosed in knowledge bases, such as the National Vulnerability Database (NVD), intended for developers and security researchers to disclose vulnerabilities, but can also be used as a resource to develop OSS exploits.
GitHub reported in its 2020 “State of the Octoverse” report that vulnerabilities often go undetected for more than four years before being disclosed. We have seen live bugs in high profile open source codebase for decades, as demonstrated by the Shellshock vulnerability in Bash that enabled an attacker to gain unauthorized access. This was discovered in 2014 and was isolated to code added in 1989. A recent example is the Dirty Cow privilege escalation vulnerability in Linux kernel versions before 2018 (CVE-2016-5195), which took nine years and one month to fix.
It only takes a single third party component from an upstream developer to unintentionally or maliciously slip in a vulnerability that has a cascade effect, introducing vulnerabilities that propagate and persist throughout the ecosystem, potentially for years. Alerts about known vulnerabilities help to get code patched, but the reuse of OSS projects and libraries over many years leads to complex ‘trees of dependencies’ that make it difficult to ensure all uses of the code are patched. In the meantime, malicious actors have the opportunity to exploit the vulnerability.
The OSS process could facilitate security, but it is difficult for a crowdsourcing community to sustain uniform best of breed security by design principles. In general, it has been challenging for OSS to reach the level of security often required. At a minimum, the OSS crowdsourcing approach needs to have a rigorous security management process established so that the openness and many eyeballs can address bug fixing and vulnerability removal.
Crowdsourcing is just one tool of many in a security-by-design approach. The report on the 2020 FOSS Contributor Survey, from The Linux Foundation and The Laboratory for Innovation Science at Harvard, notes that contributors to free and open source software (FOSS) spend only 2.27 percent of their contribution time on security and recommends that a new model is created to incentivize developers to make the effort to follow secure software development best practices. According to White Source Software, more than 4,000 security vulnerabilities are discovered in open source projects every year. OSS can be weaponized by malicious actors. Crowdsourcing can make OSS secure, but sufficient attention must be paid to the security of the code during the development process.
OSS vulnerabilities happen
One of the first OSS vulnerabilities to grab major attention was the Heartbleed vulnerability with OpenSSL, the cryptographic software that is tasked to handle a large stake of the internet’s web traffic. Heartbleed exposed private keys and other sensitive information due to a critical bug which had existed from 2012 to its discovery in 2014. Despite OpenSSL’s wide use, the project was mostly run by a single developer. The OpenSSL foundation's president, Steve Marquess, said: "The mystery is not that a few overworked volunteers missed this bug; the mystery is why it hasn't happened more often". After Heartbleed, the Linux Foundation raised money for OpenSSL to be audited and to pay for-hire permanent employees to work with the project.
Mozilla established the SOS (Secure Open Source) fund with an initial investment of USD 500,000 to help projects get their code audited. The majority of the open source projects do not have this luxury. There are more recent examples, such as the 2018 “Colourama” crypto-stealing based upon a slight name change from the legitimate “Colorama” and backdooring of the NPM JavaScript package manager "event-stream" library.
In 2020 we have also seen the Boot Hole vulnerability for GRUB2 bootloader (CVE-2020-10713), seven other GRUB2 vulnerabilities, and the critical HAProxy vulnerability with Malformed HTTP/2 Requests (CVE-2020-11100). Since June 2017, the Linux kernel has had 12 CVEs with a CVSS score of 10.0 (critical). Prevasio’s recently completed Operation Red Kangaroo scanned the entire Docker Hub and found 51 percent of all containers had “critical” vulnerabilities. As new OSS vulnerabilities continue to be introduced into development projects, it has become clear that ICT software vendors and CSPs need to implement secure software development best practices and cannot rely exclusively upon the open source community to build secure software.
Safeguards can be put in place
Secure use of OSS requires the implementation of a Secure Software Development Life Cycle (SDLC) process that can be structured and governed in many ways. A general overview of common SDLC models can be found in the US-CERT archive. Organizations that promote secure coding practices, perform rigorous testing, code audits and invest in developer education are more likely to ship more secure software. Consistent with the Shift-Left development philosophy, the software development life cycle should include supply chain management, asset inventory, a current software bill of materials (SBOM), license compliance, and patch management.
The scope of the open source ecosystem is so vast that it is a challenging task to review and audit everything that is consumed in a modern, complex ICT system. Simply having the product’s source code available does not guarantee the software will be secure. Code audits performed by experts who are trained to identify insecure code is paramount. Testing techniques such as fuzzing, a powerful tool which does not always mandate access to the original source code to identify security vulnerabilities, should be utilized. Both proprietary software and open source software can equally adhere to rigorous processes and techniques for security assurance.
Use of OSS requires a higher level of due diligence which organizations can implement by applying industry best practices for supply chain management, secure software development, and secure software maintenance. There are government and industry organizations available to help, including the US Department of Commerce’s National Institute of Standards (NIST), The Linux Foundation (LF), and OWASP. The LF Core Infrastructure Initiative (CII) has a Best Practices Badge for open source projects to self-attest. OWASP has made available many automated vulnerability detection tools that are available for free to open source projects. NIST offers valuable guidance for OSS security. NIST’s recent white paper “Mitigating the Risk of Software Vulnerabilities by Adopting a Secure Software Development Framework (SSDF)” recommends that open source software consumers ensure that software modules are vetted for vulnerabilities and actively maintained for remediation. NIST SP 800-161 “Supply Chain Risk Management Practices for Federal Information Systems and Organizations” provides guidance for OSS to be continuously evaluated, monitored, whitelisted, inventoried, supported, and remediated.
Security is a process
Is open source software better than proprietary software when it comes to security vulnerabilities? Elias Levy, the person behind the infamous (vulnerability) full disclosure mailing list, Bugtraq, said two decades ago: “No. Open Source Software certainly does have the potential to be more secure than its closed source counterpart. But make no mistake, simply being open source is no guarantee of security”.
Building and delivering complex system software without security vulnerabilities requires investment and due diligence, regardless if the code is open sourced or proprietary. As the Mozilla Foundation states: “Security is a process. To have substantial and lasting benefit, we need to invest in education, best practices, and a host of other areas”.
Tools and resources are available. With safeguards in place, OSS can be used effectively at low risk to realize its intended benefits. ICT products relying on OSS must be developed using methodologies and safeguards that ensure the expected level of security is met. OSS can accelerate innovation, reduce the development timeline, speed time to market, realize cost savings, and be secure. ICT vendors must take responsibility and practice a higher level of due diligence when using OSS components.
Development organizations must practice a higher level of due diligence
- Attack surface - 99% of codebases contained open source components and 49% contained high-risk vulnerabilities (Synopsys 2020 Open Source Security and Risk Analysis Report)
- Non-secure development - Contributors to FOSS spend only 2.27% of their contribution time on security (Report on the 2020 FOSS Contributor Survey, The Linux Foundation and The Laboratory for Innovation Science at Harvard University)
- Vulnerabilities - 51 percent of all containers in Docker Hub had "critical" vulnerabilities (Prevasio's Operation Red Kangaroo)
- Propagation - Open-source projects are usually built on open-source projects and libraries leading to trees of dependencies
OWASP Top 10 Vulnerabilities #9 = "Using Components with Known Vulnerabilities" - Persistence - Vulnerabilities often go undetected for more than four years before being disclosed (Github State of the Octoverse Report 2020)
Protecting the end user of mobile network services
The security posture of deployed mobile networks ultimately defines the end-user security experience. A comprehensive approach to security is required to address the four key levels (see figure 1, below) of vulnerability mitigation: standards, products and related development processes, network deployments, and network operations. Collectively, mitigation at each of these four levels defines the security level of deployed networks and the end-user security experience. In this contribution, we have focused the debate on one of the key levels, products and development process for Radio Access Network products such as integrated RAN, virtual RAN, Cloud RAN, O-RAN or Core network products , and the appropriate due diligence to ensure security and mitigate vulnerabilities in OSS and by extension in proprietary software.
Ensuring security in deployed networks requires mitigation strategy on four levels
Figure 1. Security posture of a deployed networks – four levels.
Key conclusions
- OSS provides many benefits including accelerating innovation, reducing development timelines, and reducing software complexity and bugs, but there are risks of security vulnerabilities that may be added to the code unintentionally or maliciously.
- Key reasons to exercise due diligence when using open source software are:
- Attack surface
- Non-secure development
- Vulnerabilities
- Propagation
- Persistence
- ICT software vendors and service providers need to implement secure software development best practices and cannot rely exclusively upon the open source community to build secure software.
- Safeguards can be put in place, but they require investment and due diligence.
- Use of OSS requires a higher level of due diligence which organizations can implement by applying industry best practices for supply chain management, secure software development, and secure software maintenance.
- When security is properly addressed, OSS can be an important contributor to the development of virtual, cloud-native 5G RAN, including O-RAN and Core functions.
- The deployment and operations processes, including configuration and interoperability of multi-vendor software products, are needed to deploy and operate a secure network that protects end users and services.
Useful references:
The Linux Foundation Projects,
O-RAN Software Community (OSC),
Core Infrastructure Initiative (CII), The Linux Foundation Projects,
“Open Source Software Supply Chain Security”, 2020, The Linux Foundation,
“Report on the 2020 FOSS Contributor Survey”, 2020, The Linux Foundation & The Laboratory for Innovation Science at Harvard,
“State of the Octoverse”, 2020, Github,
“Open Source Security and Risk Analysis Report”, 2020, Synopsys,
ENISA Advancing Software Security in the EU,
National Vulnerability Database (NVD),
Secure software development life cycle processes,
“Top 10 Vulnerabilities”, OWASP,
Open Networking Foundation (ONF),
The Open Network Automation Platform (ONAP),
