Developer knowledge/education - comments on draft outline? about wg-best-practices-os-developers HOT 12 CLOSED

ALL: If you're interested in seeing or reviewing the draft contents of this course, please send me an email with a subject line like "Request early access to secure software development course". My email address:

dwheeler AT linuxfoundation DOT org.

I would love to have your feedback/suggestions/etc.

The draft course material is a Google docs document. My plan is to provide "Suggestion" access to people, so that you can directly suggest specific changes (my preference!). If you have broader suggestions that aren't easy to capture that way, create comments.

Thanks for all the comments above! Here are a few notes.

• I've added a unit on privacy. I agree that this is important! I think it's important to briefly counter the "why worry if you have nothing to hide" thinking, so I think that this unit in particular needs to explain that.
• I added a little about vulnerability reporting early, and then added several units in part 3 specifically to go into more depth about receiving & reporting vulnerabilities. We'll have to be careful; this is a basic course, so it's really about the basics of sending & receiving vulnerability reports. Someone who is a full-time security researcher will need more information, but this course isn't intended by itself to give people a "full-time security researcher" level of expertise.

Our educational folks have asked me to divide up the course. I've divided it into 3 parts:

• Secure Software Development - Part I, Fundamentals [Covers basics, requirements, design] - est. 2-4 hours
• Secure Software Development - Part II, Implementation - est. 4-6 hours
• Secure Software Development - Part III, Verification and Advanced Topics [Covers verification, threat models, cryptography, & other advanced topics] - est. 3-5 hours

Don't take the estimates too seriously, I'm sure things will change anyway.

Thank you!

from wg-best-practices-os-developers.

This is a simple thing to put together. I've got internal materials as well as some from FIRST (Forum of Incident Response and Product Security Teams) based off of the Coordinated Vulnerability Disclosure framework that are available to refer to.

from wg-best-practices-os-developers.

100% agree that the course needs to discuss vulnerability disclosure, I will definitely add it. If there are URLs people recommend referring to, please reply here!

Yes! Something like "Life of a CVE", that covers the research, disclosure, reservation of the CVE, publication, embargoes, patching, resolution, etc. process would be useful probably by itself.

from wg-best-practices-os-developers.

Thanks for the thumbs-up!

Oh, a quick clarification: The intended audience is software developers for both closed & open source software. How to develop secure software is mostly the same for both, so I didn't think it made sense to split it like that. However, I definitely intend to include specific information on how to use OSS in a more secure manner. That definitely affects developers of both closed & open source software! For example, a brief discussion about how to avoid typosquatting won't take a lot of space, but it might prevent a lot of problems.

from wg-best-practices-os-developers.

This not intended to be a graduate course, but it is intended to cover the basics for many different kinds of software.

Where are you thinking of hosting/ publishing/ offering this?

Some thoughts:

• This looks great!
• Under Design, possibly under "Other Design Principles", I would suggest adding something about privacy as well - e.g., don't collect data you don't need.
• I don't see anything here about compliance, and how that relates to security. That may be on purpose, just calling out.
• Somewhere (under "Vulnerabilities"?) it would be worth explaining how vulnerability disclosure 'typically' works.

I'd be interested in particular to see what content you have for "Selecting" software in the supply chain section!

from wg-best-practices-os-developers.

Where are you thinking of hosting/ publishing/ offering this?

I'm planning to use EdX, for a variety of reasons:

• It can easily handle a large number of course-takers (potentially millions!)
• Mobile-ready & should work on any device/OS
• Strong focus on accessibility
• It will be easy to make the course itself free of charge, maximizing the number of people who take it. If you want to prove that you took & passed the course there's a fee, but in that case it's way cheaper than a college class.
• Can easily support little "quizzes as you go" - I think those really help online learning, as people can otherwise "zone out"
• The Linux Foundation has a long-standing relationship with EdX, & it's what our educational folks recommended for this case, so it's an "easy path".

The intent is to make the instructional material available under the Creative Commons Attribution (CC-BY) license, so it will be open content.

Some thoughts:
This looks great!

Thanks so much!

Under Design, possibly under "Other Design Principles", I would suggest adding something about privacy as well - e.g., don't collect data you don't need.

I agree that privacy is vital! I would put it even earlier than design. I think privacy is really a requirement that flows down to design & implementation. That said, I'll definitely make sure to cover it.

I don't see anything here about compliance, and how that relates to security. That may be on purpose, just calling out.

If it's required, again, I would call those requirements. But yes, that should get a mention.

That may be a little tricky, because there are so many specific compliance requirements for different industries. Also, if you have a compliance requirement, you often already know about it & don't need a course to tell you that :-). But I agree that compliance should be discussed somewhere.

One challenge: Sometimes people confuse compliance with security. You can comply with some checklist and be hideously insecure. If someone's goal is to make a secure system, and they view compliance as a tool to help make it secure (e.g., by not forgetting something important), that can work well. If their goal is to maliciously comply, then they can easily make a system comply while also being hideously secure. I want the course to focus on actual & practical security, not security theater.

Somewhere (under "Vulnerabilities"?) it would be worth explaining how vulnerability disclosure 'typically' works.

I agree!

I'd be interested in particular to see what content you have for "Selecting" software in the supply chain section!

I'm very concerned about that topic, I hope we agree it's important!

I have no special magic. Instead, I think we can offer a set of reasonable tips/recommendations on how to manage risk. Here's kind of tips I had in mind, additional suggestions welcome!:

• Is it easy to use securely? If something is hard to use securely the result is far more likely to be insecure.
  • Look at the defaults of its interface and configuration. Is its API secure by default, or are “simple examples” using the defaults also insecure?
  • If it has a discussion about how to use it securely, that’s generally a good sign, especially if it’s clear that its warnings recommend that you keep its defaults.
  • This is a big reason to avoid using C and C++ to implement new software when there’s no strong reason to use them; C and C++ have many insecure defaults (as we’ll discuss later).
• Is there evidence that its developers work to make it secure?
  • If it’s OSS, has the project earned a CII Best Practices badge (or at least are they well on their way to that)?
  • Is there evidence that the developers use tools to detect defects and vulnerabilities as early as possible?
  • Is there documentation explaining why its developers believe it’s secure (aka an “assurance case”)?
  • Is there evidence of a security audit, and that any problems found were fixed?
• Is it maintained? In theory, software can be “completed” and not need future changes. But in most situations, unmaintained software is a risk. If it’s not maintained, it’s more likely to have unaddressed security vulnerabilities, and it’s more likely it will be slow to fix vulnerabilities when they are reported.
  • If it’s OSS, you can generally look at its repository and see its commit history. If it continues to have active commits, especially by multiple people, that’s a good sign. An OSS component with no changes in the last year is generally much riskier.
  • Are there recent releases or announcements from its developer?
• Does it have significant use?
• What is its license? Licenses are technically not security, but licenses can have a big impact on security.
• If it’s important, what is your own evaluation of it? If the software is important to you, and especially if it’s OSS, you can download and examine it yourself.
  • When you review the more detailed artifacts (e.g., the source code), Is there evidence that the developers were trying to develop secure software (such as rigorous input validation of untrusted input and the use of prepared statements)?
  • Is there evidence of insecure or woefully incomplete software (such as a forest of TODO statements)?
  • What are the “top” problems reported when running it through static analysis tools (that examine the code to look for problems)?
  • Is there evidence that the software is malicious? [Ohm2020] notes traits that are especially common in malicious packages: most malicious packages start their routines on installation (so check the installation routines), most aim at data exfiltration (so check for extraction and sending of data like ~/.ssh or environment variables), and about half use some sort of obfuscation (so look for encoded values that end up being executed). You could also run the software in a sandbox with an environment intended to trigger likely issues, and see if the software attempts to do something malicious.

from wg-best-practices-os-developers.

Somewhere (under "Vulnerabilities"?) it would be worth explaining how vulnerability disclosure 'typically' works.

I think it would be great to explain this both to OSS maintainers and to OSS consumers.

from wg-best-practices-os-developers.

100% agree that the course needs to discuss vulnerability disclosure, I will definitely add it. If there are URLs people recommend referring to, please reply here!

There's also an OpenSSF working group forming on vulnerability disclosures; once I have something I intend to post an issue there for their comments. Their WG is here:
https://github.com/ossf/wg-vulnerability-disclosures

from wg-best-practices-os-developers.

I am a leader of that WG and I'll keep an eye on areas where we could collaborate.

from wg-best-practices-os-developers.

This is a simple thing to put together. I've got internal materials as well as some from FIRST (Forum of Incident Response and Product Security Teams) based off of the Coordinated Vulnerability Disclosure framework that are available to refer to.

https://www.first.org/global/sigs/vulnerability-coordination/multiparty/guidelines-v1.1

from wg-best-practices-os-developers.

Yes! Something like "Life of a CVE", that covers the research, disclosure, reservation of the CVE, publication, embargoes, patching, resolution, etc. process would be useful probably by itself.

Ironically enough I'm in the middle of writing something like this up for our internal engineering teams. The beginning piece upstream would marry very well with that effort to tell 2/3 of the story (with downstream/end-users being the last leg)

from wg-best-practices-os-developers.

Thank you everyone for your feedback! We got that feedback & incorporated it, so I'm closing this issue.

from wg-best-practices-os-developers.