It has been quite a long time since I’ve worked directly with software development teams to help them create more security code. It was before the rise of agile development principles that are very popular today. Now that I’m back in the saddle journeying sprint to sprint and expanding a software security assurance group, I need a good measuring stick for software development security activities. My hope is that by taking a maturity snapshot now it will be easier to show value to the business, the performance of our teams, and status of the program over time. Since I didn’t have a good understanding of existing maturity models for software security I asked my research assistant, Google, who quickly turned me on to both the Building Security In Maturity Model (BSIMM) and the Software Assurance Maturity Model (OpenSAMM).
They are both open source maturity models so it will be easy to compare them and pick the one you like most, right? That’s what I thought but I soon discovered that although they are similar in structure, they take different approaches to providing guidance. BSIMM is a descriptive model that was born out of a study conducted and maintained by Cigital. In assessing organizations that pay to participate in the BSIMM community, Cigital can correlate security activities that are used by each organization and provides statistical analysis based on the assessment data in each study. OpenSAMM is a prescriptive model that is maintained by the Open Web Application Security Project (OWASP). The guidance in OpenSAMM has been put together by volunteer experts in the OWASP community and is not yet influenced by measurement data resulting from its maturity assessments.
Before I go into other differences in the models, I’ll describe some of the similarities. Maturity is measured on a scale of 1 to 3 with 3 being the most mature. Each lower level is a dependency of higher levels. Both frameworks could do a better job of making these dependency crystal clear in their documentation. The security activities for level 1 should be verified as in place even if a subsequent level 2 or 3 activity already is. At the highest level both of the frameworks are divided into four categories. Those categories are then broken down into twelve security practices as detailed for each model below:
Beyond the overall approach these frameworks take, there are some other more minor differences. The latest BSIMM study as of this blog post is version 6 and its 12 security practices are made up of 112 activity descriptions that organizations can implement. OpenSAMM on the other hand currently prescribes 72 activities for its 12 practices; two security activities are needed for each maturity level within a practice. This set number of 2 activities isn’t present in BSIMM. Some maturity levels have 2 activities per level while others have as many as 6.
With the high level structural comparison out of the way I want to get into the biggest difference in these two maturity models – data analysis. BSIMM is the only model I’ve found so far that delivers data about what organizations are actually doing to make software more secure. The 1st version had 9 firms participate and the latest version had 78. Cigital’s CTO Gary McGraw mentioned in a keynote late last year that there are now over 85 firms participating so it is growing steadily. Some of the companies involved are very well known such as Bank of America, Cisco, EMC, Paypal, Salesforce, and many others. If you are doing strategic planning, pitching a software security program to management, or want to know how your program compares to other organizations, this kind of data is extremely helpful. Especially as more industries get involved and the data resolution increases:
One of the most striking numbers that jumped out at me is the ratio of security group personnel to developers. It looks as though most security groups continue to be spread really thin.
OpenSAMM may not have descriptive assessment data, but since it is prescriptive it can provide information the BSIMM cannot. For each of the maturity levels, OpenSAMM provides a stated objective. If you have not already defined objectives in your software security group these can be helpful examples to draw from in creating your own. I really like that objectives have been included.
You can sets the purpose of a security group if you clearly describe:
- Risks
- Security activities needed
- How the activities support security objectives
- Security capabilities as measured in maturity
- How security capabilities and objectives help initiatives important to the company
That purpose can then be clearly communicated outside of the security team and to senior management.
Of the two models, I’m finding the BSIMM to be more helpful for what I’m trying to accomplish currently. Luckily I do not have to pick one over the other. Both have been released under the Creative Commons so I can easily pull from both to create a hybrid in-house maturity model of my own. I greatly appreciate that both Cigital and OWASP maintain these as open source projects and contribute to the community. Software security is hard and organizations do not have time to re-invent the wheel.
Are you measuring your software security program? What is working for you and what isn’t? Please let me know in the comments or contact me directly. I would greatly appreciate any experiences you can share.
Feature Image Attribution: Andy Magee (CC BY-NC 2.0) with text added
