Select from this list or scroll down
- ISO/IEC 25000
- CWE/SANS Top 25 and OWASP Top 10 Security Weaknesses
- NIST Cybersecurity Framework
The ISO/IEC 25000 series of standards, also known as SQuaRE (System and Software Quality Requirements and Evaluation), contains a framework for evaluating software product quality. ISO/IEC 25010 defines a set of eight software quality characteristics, including Security, Reliability, Maintainability, etc. ISO/IEC 25023 describes how to measure software product quality using these characteristics, and largely describes quality at the behavioral level. For example, ISO/IEC 25023 measures Reliability as availability rather than as source code weaknesses that cause reliability problems. In general, ISO/IEC 25023 does not address the underlying issues in source code affecting the reliability or other quality characteristics of software-intensive systems.
To supplement ISO/IEC 25023, CISQ has enumerated specific vulnerabilities in system source code and architecture that underlie the quality characteristics of Security, Reliability, Performance Efficiency and Maintainability and cause unwanted behavior. For example, the Reliability standard developed by CISQ is true to ISO/IEC 25010 definition and is composed from 29 critical violations of architectural and coding practice that affect the availability, fault tolerance, recoverability, and data integrity of an application. ISO has defined product quality and measured aspects of system behavior, while CISQ measures quality at the source code level by defining and measuring structural weaknesses in software underlying each characteristic. This level of standardization advances the ability to progress automated tools for measuring software quality. Dr. Bill Curtis, Executive Director of CISQ, is participating as a co-editor in the revision of ISO 25010. For more on CISQ and ISO, click here.
Two standard sources for security vulnerabilities are the CWE/Sans Institute Top 25 Most Dangerous Software Errors and the Open Web Application Security Project (OWASP) Top 10 Project. The OWASP Top 10 lists the most critical web application security risks. The CWE/SANS Top 25 applies to a broader range of software and usually covers the vulnerabilities on OWASP’s Top 10 list. Both the Sans Institute and OWASP lists were drawn from the Common Weakness Enumeration (CWE) Repository maintained by MITRE Corporation on behalf of the U.S. Government. This repository contains a list of over 800 known weaknesses in software that hackers have exploited to gain unauthorized entry or cause malicious behavior. Each CWE is labelled with a unique identifier and description.
The CISQ Security standard is composed from weaknesses on the CWE/SANS Top 25 list. Of the Top 25 CWEs, 22 can be detected through static code analysis. These 22 provided the content for the CISQ measure. Working with MITRE, CISQ published an OMG standard for measuring the Security of a software application that is constructed from these 22 CWEs.
NIST is an agency of the U.S. Department of the Commerce. The Information Technology Laboratory (ITL) at NIST develops and deploys standards, tests, and metrics to make U.S. information systems more secure, usable, interoperable, and reliable. The NIST Cybersecurity Framework is used by government and industry to map their cybersecurity, software assurance, and risk management practices to various levels of capability. Standards from CISQ are actionable metrics for software risk assessment that can be used within an organization’s implementation of NIST’s Cybersecurity Framework. For more on CISQ and the NIST Cybersecurity Framework, click here.
CMMI is a process-based improvement framework developed at the Software Engineering Institute (SEI) at Carnegie Mellon University with support from the U.S. Department of Defense. CMMI is an expansion to product systems of the original Capability Maturity Model for Software (CMM) developed by SEI in 1991. CMMI is currently administered by the CMMI Institute, a subsidiary of ISACA. CMMI is a framework for measuring the maturity of an organization’s software and system development processes. The framework consists of five maturity levels, each representing a plateau in an organization’s transition from inconsistent, undisciplined development activities to standardized, quantitatively managed, and innovative processes. CMMI measures software processes, but not the products built through them.
CISQ supplements CMMI by measuring the quality of a software product. Both practices can be used together to improve an organization’s ability to develop and maintain high quality software. Dr. Bill Curtis, Executive Director of CISQ, led development of CMMI in the 1990’s. CMMI is referenced in U.S. Department of Defense and government contracts. CISQ measures are starting to be cited in these contracts as well.
SPICE is an ISO/IEC 15504 standard for process assessment. The SPICE framework is currently being transitioned to the ISO/IEC 33000 family of standards. The SPICE/ISO 33000 standard is structured around a reference model divided into two dimensions: a process dimension and a capability dimension. SPICE is similar to CMMI in that it’s used to measure the maturity of an organization’s individual software development processes. CISQ supplements SPICE/ISO 33000 by providing standards to measure software products developed through these processes.
COBIT is an ISACA framework for the governance and management of enterprise IT. The framework includes a set of IT controls mapped to processes and business risks. COBIT is commonly referenced in audits and regulatory compliance reports. COBIT 5 was released in 2012. COBIT is a process standard that does not provide product measures. Thus, CISQ supplements COBIT in the same way it supplements CMMI and SPICE by providing product-level measures for software applications.
ITIL is a collection of IT Service Management practices that focus on aligning IT services with the needs of business. The British standard 15000 and the subsequent ISO/IEC 20000 Service Management standard are based on the ITIL framework. ITIL was developed by the British government’s Central Computer and Telecommunications Agency (CCTA) in the 1980s. Since 2013, ITIL is owned by Axelos which licenses businesses to use the ITIL framework in their consulting practice, while managing updates and process changes. However, ITIL is free for internal use. CISQ supplements ITIL in the same way it supplements CMMI, SPICE, and COBIT by providing product-level measures for software applications.
Function Points are used for software sizing. IFPUG is an international organization of function point analysts. IFPUG develops and manages a standard set of counting rules for Function Point Analysis. ISO/IEC 20296:2009 specifies the definitions, rules, and steps for applying the IFPUG’s functional size measurement method.
Function Points are traditionally measured manually by IFPUG-certified function point counters. CISQ developed an OMG standard for Automated Function Points (AFP) that can be computed by software tools. The AFP standard stayed as close to IFPUG counting guidelines as possible while eliminating the ambiguity to support automation. The benefits of an automated measure include speed, cost-efficiency, and repeatable, consistent counts that eliminate the ± 10% variance among counters. OMG submitted the required documentation as ISO 19515 through OMG’s fast-track to be considered for adoption by ISO.
Nesma is a software industry organization in the Netherlands focused on software metrics and measurement. Similar in scope to IFPUG, Nesma promotes Function Point Analysis for measuring the functional size of a piece of software. Nesma manages the COSMIC method which is an alternative to the IFPUG method for functional size measurement. The COSMIC method has been approved by ISO as ISO/IEC 19761:2011.
MISRA develops secure coding guidelines for programming languages used in embedded control systems and software in motor vehicles and other industries. MISRA publications include MISRA C, MISRA C++, MISRA SA, and MISRA Autocode. MISRA C is a set of software development guidelines and coding rules for the C programming language. Its aims to facilitate code safety, security, portability and reliability in the context of embedded systems in critical environments. While MISRA is focused on embedded systems, CISQ supplements MISRA with rules primarily designed for non-embedded IT business systems.