Spencer Pearlman, Security Researcher at Detectify, presented A Hacker’s Approach to Finding Security Bugs in Open Source Software in a partnered webinar with friends at Debricked.

Securing modern web applications takes new approaches, and this includes looking at it from a hacker’s perspective. Here are highlights from the presentation on how tech teams can apply the same hacker mindset to discover vulnerabilities in open-source software in their tech stack.

Why should devs and engineers care about a hacker mindset?

Many, including hackers, love open-source because it’s a great place to look for information that will get them further in executing an attack. It doesn’t mean that organizations should think twice about using this, in fact it is the opposite. This can make using open-source software a safer option because many eyes are looking at the libraries to help find bugs, exposures and will report it before it falls into the wrong hands. This is a heated topic in the security community itself, and you can have some reddit commentary here. 

Having a hacker’s approach to security testing the software will help avoid pitfalls commonly exploited by hackers. Let’s get to it:

Hacker’s approach to bug hunting in open-source software:

Right away, you might be thinking, “this requires code analysis, right?” yes, but…  For some individuals, static code analysis is overwhelming, but the process can be a lot more efficient with the proper methodology. It starts with this three-step approach to conducting open-source research:

1. Recon using Dynamic Analysis
2. Recon using OSINT
3. Conduct Static Analysis

Run Dynamic Analysis to understand how the end product works

Before you can break something, you need to know how it is supposed to work. Running dynamic analysis as part of recon gives you this insight into the application’s intended functionality. This can give you a visual understanding and advantage when you start to get deep into the code.

Spencer says: “I think there’s no need to dive deep into the forest because eventually, you’ll be in thousands of lines of code. I like to warm up before doing that to see how the application works and truly understand it. I always throw up a Burp, log everything and observe keywords, variables, URL parameters and try to cross-reference that with the code once I dive into it. This process or this phase for me doesn’t take too long, but gives me a healthy feel for the code and the product.”

Recon with OSINT – where the juicy stuff comes in

There’s a lot, A LOT of helpful information out there on the interwebs, and you gotta scrape it. Using open-source intelligence (OSINT), you can find a lot of information about your target without actively engaging them.

One can look into Github and Gitlab commits or public JIRA and Trello boards to find exposed data for a web app or organization. Using Google BigQuery, you can create wordlists or analyze sensitive files on a large scale.

Repos – where the juicy stuff happens:

Looking at the history and seeing all the different commits on all the different branches, and seeing what developers have pushed to the repo gives a hacker a lot of helpful information. Some tool tips:

TruffleHog, you can enhance the process by searching for secrets within all commits and branches of a repo. Gitrob searches for sensitive files in those repositories. Using these for general analysis of all the Git history will likely serve up some juicy information, like old endpoints. Even if the endpoints may not be active anymore, they may give you a hint for potential naming of new endpoints to look for in the application.

Pro tip from Spencer: “I like to seek out fixed issues because if there’s one bug in one part of the application, there could be a bug in another part. If I see a commit for fixed RCE or fixed SSF, I will look for those kinds of vulnerabilities in other parts of the code.”

Stackoverflow is also where many developers tend to ignore security and want the end result as fast as possible, making the perfect bug hunting ground. One can often find sensitive code or credentials or even system architecture, which can give you a better understanding of the target at hand and where to go from there.

Finally dive into code with Static Analysis

Once you’ve done dynamic analysis and some OSINT, it’s time to dive in with static analysis. First, as a hacker, you approach it with some linting or tools to find bad-smelling code. In other words, you are looking for areas where typecasting happens with no error checkings (this can cause things to crash and cause a DoS attack) errors or other low-hanging fruit vulnerabilities. 

Then, that’s followed up with grep in combination with any regular expression patterns to find sensitive keys, dangerous functions, etc.

Spencer’s Pro-tip to use semgrep: “I found that grep and regex can be false-positive prone. So for me, I think Semgrep is an excellent tool to use. It’s a fast open-source tool used for finding bugs. It comes with pre-written rules to find common vulnerabilities, secret keys and common misconfigurations while using regex-likes patterns. These patterns can be specific to each language, making it more intuitive and less false positive prone.”

Looking at the sources and sinks

Looking at sources and sinks allows you to see how data is passed around an application. 

Going from “source to sink”

When you start with the sources in a code, you’ll observe how data is passed to the sink. Fair warning here: you can go down a rabbit hole, but you’ll understand more of the application and codebase in this way, and it might find juicier bugs. 

Going from the sink to source

When you start with the sink and work your way back to the source, you might find more bugs, but they may not be as impactful. Going from this perspective doesn’t give you a complete understanding of the application. 

Detectify’s Approach to OSS:

Detectify is a website app scanner, but unlike the other companies, we collaborate and crowdsource information from hackers worldwide. Having an ethical hacker community on your side means that you can take a crowd-based approach to find vulnerability information faster and with more accuracy. They submit vulnerabilities from zero-days, critical CVEs, published research, or private research as well. 

And we don’t do signature-based detection. Crowdsource hackers send a PoC that will eventually become a module. The security testing modules in Detectify’s scanning engines send a  HTTP request and we then look for various things in the response to verify the vulnerability exists. Unlike the other platforms, we give a per-hit payout. The more popular a module is the more hits and the more payouts a hacker gets in the long run.

Ask and you shall receive

At times, there’s a specific vulnerability that’s spreading like wildfire across the Internet, and it’s something we want to be able to detect in our users’ scan profile and alert them in time. When that happens, we ask our hacker community to submit working POCs and set a bounty for it to get it into our testbed as soon as possible such as Drupal RCE CVE-2019-6340, CVE-2021-26855: Microsoft Exchange SSRF, etc.

For example, a couple of months ago, the Microsoft Exchange SSRF – CVE-2021-26855: Microsoft Exchange SSRF – was being exploited by a Chinese state-sponsored group targeting many multinational companies in the US. Within a couple of days of setting a bounty for this actively exploited vulnerability, a Crowdsource hacker successfully found Microsoft’s implementation of detecting this vulnerability in a nmap script. This was developed and released to our user base ASAP.

What else does a hacker submit?

Our coverage also includes sensitive keys or API keys. A Detectify user or hacker can submit an endpoint to a sensitive file path such as aws.json and then submit specific regexes to check for API keys in the file.

Detectify’s in-house research on OSS:

The Detectify Security Research team also conducts security research on open-source technologies like nginx. The team analyzed almost 50,000 unique NGINX configuration files downloaded from GitHub with Google BigQuery and discovered common misconfigurations that left your website vulnerable to attack if left unchecked. Detectify security advisor Frans Rosen also had a deep dive into middleware over on Detectify Labs. 

How can Detectify help find vulns in OSS?

What you find in production is what matters because new vulnerabilities are discovered by researchers each day. Detectify web vulnerability scanner checks for secrets and tokens exposed in repos, CVEs and other actively exploited vulnerabilities in open-source technologies, common misconfigurations and more. We collaborate with expert ethical hackers to crowdsource research and relay weaponized-payload testing from hacker-to-scanner in as fast as 25 minutes. This means faster, verified results and clearer visibility with less noise. 

With Detectify, you will bring security up to speed and scale with the business and go-to-market safer. See it for yourself with a free 2-week trial. Sign up today.