Presentation from Open Source Leadership Summit 2019. This talk will highlight some best practices that your Open Source Program Office (OSPO) can use to manage security vulnerabilities for open source projects using GitHub’s security alerts at scale. We’ll discuss the mechanics and governance around the process we’ve set up at Verizon Media to notify internal employees about CVEs on their projects.
3. 3
Verizon Media Open Source Program Office
7K
All engineering employees benefit
from OSPO services
330
Support tickets quarterly
440
Active Open Source Projects
published by Verizon Media
25
GitHub organizations that we
manage
200+
Mobile and TV Applications that rely
upon our services for compliance
4. 4
What does an OSPO do?
Program
Management
Community
development
License inbound
review
New project
publication
Reviewing
publication steps
completed prior to
publication
Reviewing the use of
open source in our
products and platforms
Promoting projects
via blogs, podcasts,
and speaking
events
Supporting internal
engineering groups
with open source
issues
Contributions to
projects
Issue support and
resolution
Compliance
Management
Security Alerts
GitHub alerting us
about vulnerable
dependencies
Responsible for mobile
and TV app compliance
engineering and
automation
Ensuring issues are
addressed on our
external repos
Reviewing
contribution policies
and CLAs
8. 8
OSPOs need to care about
security issues in their
published code.
9. 9
GitHub can help
It’s limited and not designed for OSPOs,
only for project owners.
Good News, Bad News
10. 10
● What GitHub does to help your companies’ open source
security issues
● Where the alerts and APIs fall short
● A call for you to help develop a better solution
Agenda
15. 15
Some of the problems that OSPOs will have
● Opt-in only for private repos
● Vulnerability Alerts API cannot turn on notifications
● Email give you only 10 repos in daily digest
● Not all project languages supported
● No dashboard of alerts including notification dismissal
reasons
● Not automated!
20. 20
Automate Security Workflow
GraphQL
API v4
Security Alerts
Depency Graph
GitHub Raw DB of
GitHub Alerts
with CVE info
JIRA Tickets
Email
JIRA API
Slack
POCs on GitHub
Projects and
Related Info
Screwdriver Cron Job
22. 22
If you are in the audience or you work
for GitHub, help us automate OSPOs
workflows.
23. 23
We’d love your help
● Add automation for different solutions
○ JIRA
○ Email
○ Slack
● Contribute GitHub security alerts to GHCrawler
Project: https://github.com/yahoo/GitHub-Security-Alerts-Workflow
25. 25
But that’s only if you take advantage
of the information available in the
open source community and patch
vulnerable dependencies.
And contribute back.
26. 26
Thank You
● Gil Yehuda, Verizon Media
● Justin Hutchings, GitHub
● Jamie Jones, GitHub
● Jeff McAffer, Microsoft
● James Siri, Amazon
● Manikandan Subramaniam, Verizon Media
● Henri Yandell, Amazon
● Simon Maple, Snyk
27. Thank You
Ashley Wolf
Open Source Program Manager
Verizon Media
awolf@verizonmedia.com
Twitter: @Meta_Ashley
Hi everyone, welcome to my talk Don’t Ignore GitHub security alerts, automate them into your workflow.
My name is Ashley Wolf. I am an open source program Manager at Verizon Media.
Verizon Media is now part of Verizon. It’s effectively Yahoo + AOL, we used to be called Oath now we are Verizon Media. I’ve worked on the open source team for 5 years and I did a stint in between at a cyber security start up before rejoining Verizon Media.
I want to share a little bit about our open source program office.
We run a fairly large open source program. We support about 7,000 developers.
We have a few hundred open source projects... Two dozen orgs... and Dozens of mobile apps.
There are some traditional OSPO functions that you know about: inbound, outbound, license compliance.
In our team, we found that we spend about a quarter of our time on security related things.
That’s something you might not have associated with an OSPO.
Our company already deals with production security alerts through our internal security team, SEs, and code scanning.
Remember Martin at the keynote? This is his company and we are his customers. We run the largest global bug bounty program which is an excellent method for inviting external folks to identify production problems.
But We’re not the information security team, they deal with infosec. We’re an OSPO. let’s talk about that gap.
As OSPOS we care about vulnerabilities and dependencies with vulnerabilities.
Specifically, if it’s in a published piece of code. We have to be concerned that we’ve published something that the community uses and a few months later some dependency on it makes it vulnerable and potentially causing you to introduce a vulnerability into your code. That’s a problem.
We want people to trust our code. But if our code calls in someone else’s code we need to know. We want to make sure when you come to our open source projects you are getting good code. Because errors caught early cost less in the long run.
As an OSPO we have a vested interest to make sure our dependencies are free of vulnerabilities.
Infosec people care about security in production code.
It’s not typical for infosec team to care about non-production published code, but it’s important for an OSPO to care, because we want people to trust the quality of the code.
As an OSPO, you have this unique need to care about an information security issue that your infosec team doesn’t have to. We have a reputation to uphold.
There’s Good news, GitHub can help us.
Bad news, not as helpful as we were hoping.
What I want to do today is share with you what GH does and where I think we need to take it to be good news.
Heres the agenda for today and I know it’s slide 10.
There’s a category of security issues that you have to care about, because no one else does.
I’m going to share with you what GitHub does
and where it falls short, and
I’m going to end with a CTA to help us make this better.
First, I want to say thank you to GitHub for providing a security alerts feature.
For public repositories which are using a supported language/package manager,
GH SEC alerts tells you about your project’s dependencies and vulnerabilities right on the repo.
You can double click into the issue on a package and find out what the appropriate remediation steps are.
BTW - I want to point out these are pretty simple fixes. It most of the cases, it’s upgrading to the new version.
So It’s not that we are dealing with bad code, we’re just dealing with code that hasn’t been patched yet.
By default, you should be receiving weekly emails summarizing your security alerts. If you have not already, you can configure these alerts and switch from weekly to individually or daily.
This all sounds good, but where this falls short is the OSPO specific use cases.
As OSPOs, we care about a lot of projects.
The GH solution was focused on the project manager, but as an OSPO we need a better approach.
Here’s some examples of what you’ll encounter with GH Security alerts
This information is all very useful to OSPOs for audting and accountability.
We want to know that teams looked at alerts
We want to know if they dismiss it and why
Sometimes, project owner ignores the issues.
As some point we have to ask ourselves is this project dead? Or do we need a new project owner?
This page told us we need to make a decision. Elected leader, archive the project, or a better system to deal with alerts.
We decided to publish an open source project to automate the security workflow designed for OSPOs.
The Amazon OSPO published a little code, and we published a little code, and now we have a proof of concept to get our security alerts in real time.
We’re not limited by 10 repos.
We can get more information.
And can create tickets in JIRA.
We’re beginning to chip away at some of those limitations.
What we’d like to see is a fully automated workflow.
With a DB and an endpoint that is pluggable so if your OSPO wants to trigger an email, or create a ticket in any system you can do that.
In these events, we can really get into the information that OSPOs need
As you see, we have a need, we are working toward a solution. And in the spirit of open source, we’d like to invite other OSPOs to consider this problem and to work with us to solve it.
We’d love your help and contributions.
Some of the near term development tasks we have
We’ve been talking about security vulnerabilities and I feel the need to remind you sometimes Open Source gets a bad rap from security professionals.
They say open source is less secure. We think OSS has -- more potential -- to be more secure.
We’re here to do something about it so that we can all agree open source is not less secure, it’s more securable.
And what we want to do is make that really happen.