A presentation about using Open Source Intelligence for proactive defense delivered at Rootconf 2019 Bangalore, India. RedHunt Labs https://redhuntlabs.com/

1. OSINT for Proactive Defense
Rootconf 2019

2. # whoami
● Shubham Mittal
○ Director at RedHunt Labs
○ CFP Review Board Member - BlackHat Asia & InSEC World HongKong
○ Co-Founder - Recon Village (DEFCON Hacking Conference)
○ Project Lead - DataSploit (OSINT Framework)
○ 8+ Years Experienced Security and OSINT Enthusiast
○ Expertise in Offensive Security, Perimeter Security, OSINT
○ Speaker/Trainer/Presenter - BlackHat, DEFCON, Nullcon, c0c0n, IETF
○ Bike Rider, Beat Boxer
○ Twitter: @upgoingstar

3. Agenda
● Overview of OSINT
● Why Security Teams should use OSINT
● Continuous Discovery and Monitoring of Assets
● Use OSINT Data for Periodic Attack Simulation
● Discovering Sensitive Information Leakage
● Monitoring Breached Passwords
● Proactively Identifying Security Incidents using SOCMint
● OSINT Countermeasures

4. What I mean when I use these..
- Brute Force
- Trial-and-error method used to obtain information such as a user password,
bucket names, subdomains, PIN, OTP Codes, etc.
- Black Box / White Box / Gray Box
- No access to the information / Complete access to the information / Hybrid mode.
- Patch
- Fixing security vulnerabilities and other bugs.

5. What is OSINT?
● Open Source INTelligence (OSINT) is the collection and
analysis of information gathered from publicly available
sources.
Intelligence
Analysis
Correlation
Raw Information
Actionable Intelligence
https://en.wikipedia.org/wiki/Open-source_intelligence

6. Why OSINT?

7. Why Security Teams should do OSINT?
Because, Hackers do.

8. Why Security Teams should worry about OSINT?
● Sensitive Info Leakage on Code Aggregators
● Untracked Assets running Easy Targets
● Frequent Release Cycles, Dynamic Cloud Environments
● Targeted attacks, less noisy. Sometimes, can’t be caught.
● Employees use personal breached passwords in Corporate Accounts.
● Full Patched Systems? How about credentials leak?
Find it before Hackers do.

9. Why Security Teams should worry about OSINT?
http://hackerone.com/hacktivity

10. How?
● Continuous Discovery and Monitoring of Assets
● Periodic Attack Simulation
● Discovering Sensitive Information Leakage
● Monitoring Breached Passwords
● Proactively Identifying Security Incidents using SOCMint

11. What is an Asset?
● Any resource of monetary value.
● Owned by individuals, companies, or governments.
● Example?
○ Servers, HDD, Network Devices, Laptops, Domains, Patents, etc.
● How about..
○ Social Media Accounts, Source Code Repositories, Relevant Dumped Passwords,
Cloud Storage objects (Buckets, Blobs, Spaces, etc.), Elastic IP Addresses, API Keys
and Credentials and a lot more.
○ No monetary value, but can cause huge reputational and financial loss.
https://redhuntlabs.com/blog/redifining-assets-a-modern-perspective.html

12. Continuous Discovery and Monitoring of Assets
- IP Addresses (Dynamic and Elastic)
- Domains
- Subdomains
- Cloud Storage Objects
- Leaked Credentials / API Keys / etc.
- Social Media Accounts
- Third Party API Keys
- Analytics Tags
- Supply Chain (Vendors, Acquisitions, Mergers, etc.)

13. - IP Addresses
● Cloud API (WhiteBox)
● DC Administrators (WhiteBox)
● Internet Wide Scans (Project Sonar, Shodan, etc.)
● Using ASN ID (Autonomous Synchronization Number)
● Whois Reverse Search
● Reverse PTR Records

15. DEMO - WhoIs / ASN ID

16. Project Sonar
● By Rapid7 and MIT
● Periodic DNS Queries
● FDNS
○ A, AAAA, CNAME, TXT, SOA
● RDNS
○ PTR
● https://scans.io
https://opendata.rapid7.com/about/

17. - Domains
● Reverse Whois on Email and Phone Numbers

18. - Subdomains
● Search Engines (Google/Yahoo/Bing/Yandex)
● Internet Wide Scans - Project Sonar
● Certificate Transparency Reports
● Brute Forcing Subdomains
● Reverse IP Lookup, etc.
● Tools
○ Sublist3r / Amass (Well maintained and good number of sources)
○ aio-dns-brute (Very quick) ~ Threat to Network Bandwidth
https://github.com/aboul3la/Sublist3r
https://github.com/blark/aiodnsbrute

19. DEMO - Open Data Querying
(FDNS/RDNS DataSet)

20. - Cloud Storage Objects
● Buckets / Blobs / Spaces
● Stores Sensitive Data (Intentionally and Unintentionally)
● Misconfigured ACLs (Access Control Lists)
● How?
○ Spider, Fetch, Extract, Check for Permissions.
○ Create Possible bucket names (Common patterns) and try each one.

21. Custom Python Script
https://digi.ninja/projects/bucket_finder.php

22. - Leaked Creds
● Identify leaked sensitive information.
● Passwords, API Keys, Third Party Access Tokens, DB Creds, Internal domains, etc.
● GitHub, BitBucket, Pastebin, .Onion Websites, etc.
● Identify Organization Repos / Identify Employees and their personal Repos.
● Google CSE (Custom Search Engine)
● Manual Search
○ GitHub Advanced Search
● Automated tools
○ Gitrob, TruffleHog, etc.
https://github.com/search/advanced
https://github.com/michenriksen/gitrob
https://github.com/dxa4481/truffleHog

23. Manual Search
Example

24. DEMO - TruffleHog

25. - Social Media Monitoring
- Security Incidents
- Organization Reputation
- Keyword Based Monitoring
- Streaming APIs / Scrapers
- Google Alerts / Page Change Detection
- Tweet-Monitor
- Someone tweets, Alert on Email, Dump to ElasticSearch.
- Dashboards (Users, Frequency, Relationships, Geolocations, etc.)
- https://www.youtube.com/watch?v=OjLP5k5NIMY
https://github.com/upgoingstar/TweetMonitor

26. - Identifying Relationships between Domains
● Third Party Tags for Analytics
● Admin holds one account.
● Same Tag used across
different assets owned.
● Reverse Lookup can be done.
https://builtwith.com/relationships/

27. Periodic Attack Simulation
● Create a list of assets.
● Classify the assets (IPs, Subdomains, Domains, Buckets, etc.)
● Run custom scans.
● Pass these assets to Vulnerability Scanners, Review Reports.
● New Release? New Acquisition? New Merger?
○ Check for new assets.
○ Check for vulnerability resurfacing.
○ Run a complete cycle.

28. OSINT Countermeasures
- Do it yourself before someone else use it against you
- OSINT Awareness Campaigns
- MetaData Stripping
- Data Loss Prevention
- SIEM Integration with CIF
- HoneyCreds
- Identify the root cause, instead of fixing the issue.

29. Implement OSINT
Countermeasures
Identify Asset
Data Sources
Implement Asset
Discovery Process
Periodic Attack
Simulation / Vulnerability
Resurfacing Checks
Security Team
● IP Addresses (Dynamic and Elastic)
● Domains
● Subdomains
● Cloud Storage Objects
● Leaked Credentials / API Keys / etc.
● Social Media Accounts
● Third Party API Keys
● Analytics Tags
● Supply Chain (Vendors, Acquisitions, Mergers, etc.)

30. What next?
● Awesome Asset Discovery List
○ https://github.com/redhuntlabs/Awesome-Asset-Discovery
● Awesome OSINT Resources
○ https://github.com/jivoi/awesome-osint
● DataSploit - OSINT Framework
○ https://github.com/DataSploit/datasploit
● Handpicked Weekly OSINT News
○ https://medium.com/week-in-osint
● Open Data - Internet Wide Scans
○ https://opendata.rapid7.com/about/

31. Q & A
- Email: shubham@redhuntlabs.com
- Twitter: @upgoingstar
- Website: www.redhuntlabs.com
- Would like to talk? Fix a meeting:
https://calendly.com/shubham_mittal/short_meeting