International collaborative efforts to share threat data in a vetted member community by Mike D'Ambrogia, Andrew Breksa
•Als PPTX, PDF herunterladen•
1 gefällt mir•103 views
The APWG has been sharing threat data for over 12 years to help protect organizations and the all internet users against cyber threats. Initially founded to focus on the phishing, as the threat landscape on the internet has grown so has APWG. Today our vetted member community shares information to fight cybercrime and fraud not only on phishing but numerous other types of threat data including malicious IP addresses and ransomware information. This session will look at the history of sharing these types of data, how sharing has changed over the years and the necessity to automate these process.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie International collaborative efforts to share threat data in a vetted member community by Mike D'Ambrogia, Andrew Breksa
Ähnlich wie International collaborative efforts to share threat data in a vetted member community by Mike D'Ambrogia, Andrew Breksa (20)
Mehr von CODE BLUE
Mehr von CODE BLUE (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
International collaborative efforts to share threat data in a vetted member community by Mike D'Ambrogia, Andrew Breksa
- 1. International Collaborative Efforts to Share Threat Data in a Vetted Member Community Mike D’Ambrogia Project Lead Andrew Breksa Lead Developer
- 2. Who Are We? • Founded in 2003 to focus on the emerging new threat called “Phishing” • Began collecting statistics and data to produce reports and track activity around Phishing • Gathered together a collection of experts and concerned industry leaders and researchers • Membership grew to include a collection of cyber-crime fighters: Financial institutions ISPs Technology companies Law enforcement agencies Government agencies Treaty organizations E-commerce sites and solutions providers Research partners: - Country CERTs, universities, industrial laboratories, volunteer responder organizations
- 3. Since 2003 Cyber-Crime and Fraud have evolved, APWG has evolved too • Several areas of influence • Cyber Policy • Education / User Awareness • Tracking Trends and Malicious Activities • Research • Sharing Threat Data – the eCrime Exchange
- 4. eCrime Research Program - US • Annual Symposium on Electronic Crime Research • Peer reviewed research paper program • Two publication tracks • Theoretical cybercrime academic research • Applied, industrial cybercrime research • Accepted papers published via the IEEE.org • Accepted papers are presented to members and attendees at the annual meeting • For the academic research track • Travel stipends for one author of each paper • Cash award presented to the two top papers
- 5. eCrime Research Projects – EU Foundation • Mostly focused on User Awareness • Three ongoing Horizon 2020 projects • VIVET: creation of videos and educational materials on cybersecurity for Vocational Training (For students and educational institutions, but also for refugees and unemployed people) Organize 4 seminars • TRUESSEC.EU (Trust-Enhancing Certified Solutions for Security and Protection of Citizens' rights in digital Europe) will develop requirements to achieve the RETEL (Recommendations for a European Trust-Enhancing Label). • Cyber-volunteer Networking Tool: A simple platform to support the establishment and organization of networks of cyber-volunteers for training and education
- 6. APWG’s eCrime Exchange • A warehouse of threat intelligence data • Free service for APWG members • eCX Mission: • Limit access to a trusted and diverse community of companies and users • Handle sharing of any type of threat data • Make data available nearly instantly • Drive data to users • Don’t force business rules on others, instead integrate cleanly into existing business processes
- 7. How We Share Data - The 2 Sides of eCX • Side 1 - Web User Interface • Side 2 - High Performance REST API • Built on a Lambda style LAMP stack • PHP 5.6 and 7 • AMQP Messaging using RabbitMQ • MySQL • ElasticSearch ELK stack (ES, Logstash, Kibana) with Filebeats and ElastAlert • SphinxSearch • Memcache • Distributed
- 8. eCX – Side 1, Web User Interface • Threat intelligence data is held in Modules or Workgroups • Pick the data you are interested in and request access • Consistent searching/filtering in all modules • Data driven to users with real time Alerts • All API documentation in Swagger/OpenAPI • While eCX is complex behind the scenes, the bootstrap interface keeps things simple and quick for the web user
- 9. eCX – Side 2, High Performance REST API • GET, POST, PATCH • A single interface for the entire platform with individual endpoints per module • The user is validated via a unique API token • 38 query filtering and ‘ranging’ options on GET • Fast. Handling the query from start to finish averages < 200ms • A testing “sandbox” is available for new script development • Data output samples at https://ecrimex.net/samples.zip (3.9Mb)
- 10. eCX - High Performance REST API • Submitting data, 4 fields • Date first seen, an epoch data type • The brand the phish is attacking • The URL • A confidence factor • 100, 90, or 50 • Same 4 fields since 2003 • Validates user, receives data, normalizes, inserts in < 75ms • Robust result codes and messaging to know what happened instantly
- 22. Workgroups • Securely store and share any kind of threat intelligence data • Files, images, pdfs, notes of analysis, IP’s, etc – anything • Share this data with other users • Full membership control • Approve, deny, revoke • Workgroup can be visible to others, or completely invisible • Access the workgroup using the eCX API • Live demonstration of setting up and using an eCX workgroup
- 23. Get Involved with the APWG • Memberships for various access levels • eCrime Exchange 10 day trial • Share your threat intelligence data mike@ecrimex.net andrew@ecrimex.net support@ecrimex.net
Hinweis der Redaktion
- History from 2003, to 2014
- Modules and their associated API endpoint names, showing phish but I’ll go through screenshots of the other 2 modules, malicious IP and report phishing, shown here as well. The output for each module is in a datatable layout with a consistent interface – column headings, pagination, ability to edit on the R side.
- If I scroll down to the bottom of the screen the datatables I see a count of records shown, of total records. I also have a column filter available for all the columns that Ive circled in red – if I add a value here, and I’ll use the maslog.club domain, then eCX will search and find every entry that has that domain name in the phish data.
- eCX found only 1 entry in the 2.9 million records of data currently stored. I can use multiple filters if needed, perhaps this domain but with a specific brand to help narrow down the results when there are too many matches to analyze visually.
- Now I’ve switched over to the Malicious IP data module. The same datatable layout with the same paging and column filtering and result counts are used here as well. This data contains traffic from bad actors using scanners, bots, and login attacks. It is fed to eCX
- And this is the report phishing module. We receive phishing emails sent to us by the general public at reportphishing@apwg.org that get fed into this data set. We also feed spam that we receive on our personal APWG email accounts. Same datatable display widget is in use, but in addition we’ve added 2 “show” columns here on the right side, one for body and one for headers. Clicking these opens up a modal popup and allows you to view the body/headers data
- I’ve clicked the Show button for headers for the first record shown and this is the pop up that allows me to visually review the data in the header. I can use the column footer filtering in the datatable to search for data in all fields but the attachment column. A checkbox in attachments indicates that the parsing logic found an attachment. While I won’t get too detailed when we get into the API side of things I will say that there is an API option to only return results that have attachments.
- I’ve clicked the Show button for headers for the first record shown and this is the pop up that allows me to visually review the data in the header. I can use the column footer filtering in the datatable to search for data in all fields but the attachment column. A checkbox in attachments indicates that the parsing logic found an attachment. While I won’t get too detailed when we get into the API side of things I will say that there is an API option to only return results that have attachments.
- And finally the alerts section which meets the mission of driving data to the user. As data is ingested into eCX a monitoring process checks the incoming data against matches that the user has setup – for example a country CERT sets up an alert to track any new insert of data where the TLD of the reported domain matches their country TLD. A brand manager might look for matching brands, or a list of brands matching their clients brands, or brand name derivatives within a URL string. The system checks the data in near real time, usually within 2-3 seconds, and if a match is found eCX emails the user letting them know a match was found. We can also alert the user over a Slack channel.