Hello Security folks and marketers,
I'm collaborating with Jojo Toth (mogdesign) on a marketing piece about security in Drupal. It will mostly be about the process of handling an issue. We're trying to brainstorm what statistics we might want to use, but most of them end up seeming negative when you first look at them. For example, if we brag that we handled ~60 issues in 2011 then that looks like Drupal is insecure ("wow, 60 issues is a lot!") until you dig into the facts that this was across Drupal core and ~5,000 contributed projects.
So, we're looking for some statistics that probably fall into two buckets:
* Numbers that immediately feel "good" - the fact that there are 40 active people on the security team seems to me like it would be seen as a positive thing to most people.
* Numbers that may not initially feel good, but we can give context to them within this "1 page" document so that they do feel good. It's possible that we can explain "60 issues across ~5,000 projects" and make people see it as a positive thing
Please feel free to leave a comment with your ideas or contact Jojo or myself with your thoughts.
Thanks for any ideas!
Comments
Maybe instead of just saying
Maybe instead of just saying "60 issues", you break it down more and say "X number in core" and "X in community contributed modules". I would think maybe people would sort of expect lesser security in community modules but expect better security in the core. So if you responded to 2 issues in core and 58 in community modules, that sounds to me like core is very secure and that you're also helping developers to make their code more secure.
Another thing you could do is use percentages up front. 58 out of 5000 is roughly 1% or so? You can make a claim that out of the 5000+ community contributed modules, less than 1% of them required a response from the security team.
Maybe another area you can use is response time. You can also use either absolutes or percentages here again. If 90% have patches issued in less than 30 days from the initial report, things like that will also sound great.
I would think other areas that you can "brag" about are usage of the Security Review module and how well that is being accepted. Use any opportunity you can to further advertise that.
Do you know what stats other
Do you know what stats other CMSs use, for direct comparison? I would think that would be most helpful. 40 active members on the security team may not be impressive, if I don't have a point of reference. It might also help to differentiate between issues for core versus contrib.
Almost all the bug tracking metrics I know of are based on time, but would something like issues per line of code work? (thinking out loud) If there are any visible trends, like the rate of security issues is decreasing versus the rate at which Drupal code is changing, that would be interesting.
I agree with plaverty about the response time. It is probably for more impressive to show how quickly problems are dealt with, instead of just how many.
I agree with the response
I agree with the response time idea. What also might be interesting is to show that Drupal users care about security - eg not just how quickly the security team responds, but how quickly are Drupal installations updated? How small is the percentage of Drupal sites that are running vulnerable code?
The result would be a stat that Drupal users would be happy to brag about, something that would make people want to move to Drupal to show they care about security, etc.
Protected Industries
"information security" vulnerability metrics
I will take a look at industry, government and academic best practices for documenting infosec measurement and response. I can't really work on this much until next week, so mention your time frame.
My belief is that a marketing piece should give confidence to Drupal users, but also address the broad security concerns of IT decision-makers.
I'm a marketer (engineer by education) that has done a lot in the information security space and monitors the infosec mail lists. Last year, an infosec client Gideon Technologies was acquired by Symantec. Contact me anytime if I can help.
Jim Caruso
MediaFirst
Jim@MediaFirst.net
@jimcaruso
(M) +1.404.788.0188
http://MediaFirst.net
I think "next few weeks" is
I think "next few weeks" is the time frame, but Jozef Toth would know better.
I agree on your statement of purpose.
Thanks!
knaddison blog | Morris Animal Foundation
Some thoughts
The actual number of affected sites: Some of the contrib modules that had security fixes were used only by 20-30 sites. Somehow categorising contrib modules based on the number of users could improve the impression?
Education: The security team has been making an effort in educating developers on security. For example, there is always a presentation at every Drupalcon on Drupal security by the security team members. I also recall Jakub Suchy giving presentations on security at several Drupal events in London as well.
There is also an application review process for full projects. Could we somehow enumerate such educational opportunities and attempts?
Great ideas. I'm hesitant to
Great ideas.
I'm hesitant to talk about our role int he project review process unless we, as a team, are willing to be committed to doing that as a full-time part of our responsibilities. I don't think we can expand to include that right now.
knaddison blog | Morris Animal Foundation