95

imgproxy: Resize your images instantly and securely — Martian Chronicles, Evil M...

 6 years ago
source link: https://evilmartians.com/chronicles/introducing-imgproxy
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Learn how to get started with imgproxy—a fast and secure standalone server written in Go. It does only one thing and does it well: takes care of all your image resizing needs.

Is image worth a thousand lines?

You have an idea for a startup; maybe you are already working on it. You have scaffolded your MVP, and everything works fine under a minimal load. You have users who can do things in your app. And now you want to accept images from them—so anyone can upload an avatar or a profile page cover. Everyone does it, right? Why should not you?

Evil Martians Twitter account

A small rounded avatar and a big rectangular background cover—minimal personalization users expect from your startup today.

By now you probably know that image upload is not a walk in the park. So many questions arise instantly, mainly:

  • Where do I keep my images? Locally or remotely?
  • How do I handle image transformations? Where do I keep all versions of the same image?
  • How do I plug everything into my existing app?
  • Last but not least: how do I protect myself against image bombs (the sub-class of decompression bombs)?

Sure, there is a plethora of ready-made solutions available. So you pick one.

And everything goes smoothly until you end up with few thousand images on your hand—and suddenly your design requirements change.

You want your avatars to be more high-res! You want to change the crop mode and dimensions. One thing is writing new code, asking tools like ImageMagick to, well, do the magic. Another thing: at every requirement change you have to waste time, storage and processing power to handle all the transformations again and again. Requirements in startups do change, and quite often so.

Let’s try something different

What if the image hosting problem (and maybe you don’t want to host anything—just re-use images from a remote source?) was the only problem you had to solve? What if you could offload image resizing not to some foreign cloud, but to a microservice under your full control? And what if that tool was completely language and framework agnostic, meaning you can use it right away, no matter the stack?

Meet imgproxy—a tool created exactly for these purposes. You can quickly test it out yourself right now, by following simple steps below:

  • Go to imgproxy’s GitHub repository and read installation instructions. If you’re comfortable with Go and Docker, you can set it up on a local machine or a remote server right away. If that is not your cup of tea—no problem, just click on “Deploy to Heroku” button inside a repo’s README, and you will instantly get an up-and-running instance of imgproxy. That is how it looks like:

Deploying imgproxy to Heroku

Deploying imgproxy to Heroku

  • Now all you have to do is pick a name for your app and note down IMGPROXY_KEY and IMGPROXY_SALT (they are already generated, just cut and paste them somewhere). Together they make up a powerful security feature that no imgproxy alternative currently has. We will talk about it later.

NB: for an actual production you want to create your own unique set of keys, any random hex-encoded strings will do. Here’s some *nix magic to quickly generate such a string (try it in your shell):

$ xxd -g 2 -l 64 -p /dev/random | tr -d '\n'
  • Once you have a URL to your imgproxy instance and a key/salt pair close at hand, go to imgproxy URL generator. Now pick a random remote image: this 4000x2000 NASA’s image of Martian landscape will do perfectly. Note that by default imgproxy will reject images with dimensions larger than 4096 pixels on either side, but you can change it through settings (more on that later).

imgproxy URL generator

Testing imgproxy in action

Paste this URL along with your imgproxy host URL, key, and salt in respective fields. Set desired dimensions and crop mode and hit “Generate.” Now copy the resulting URL and paste it in a new browser window.

  • Bingo! Your very own imgproxy instance has just resized a heavy remote image on the fly in no time.

imgproxy does resizing

Original 4000x2000 image weighing over 650KB; creating imgproxy URL; resized 500x250 image weighing 19KB

Three “S”: Simplicity, Speed, Security

Now you see that imgproxy receives the image source and instructions on what to do with it as a cryptographically signed URL. Steps for creating the signature are the following:

  • Take the path after the signature — /%resizing_type/%width/%height/%gravity/%enlarge/%encoded_url.%extension;
  • Add salt to the beginning;
  • Calculate the HMAC digest using SHA256;
  • Encode the result with URL-safe base64.

You don’t have to be familiar with cryptography. Even if terms like HMAC, base64, and SHA256 don’t tell you much, just take our word: implementing those steps is a trivial task in any modern programming language. Once you do that, you can be confident that no one will be able to use your imgproxy server for their own needs, unless they know both your key and your salt.

Project’s repo contains a folder with scripts for generating proper URLs in Ruby and Go, the web form we used for the demo is implemented in frontend JavaScript with the help of jsSHA cryptographic library. Of course, in the real world, you would like to keep your cryptographic calculations to your backend exclusively.

As an extra layer of security, you can set up an authorization token to be used in the header of all incoming HTTP requests.

What about those image bombs we discussed in the beginning? Well, imgproxy also checks image type and “real” dimensions when downloading, so the image will not even be fully downloaded if it has an unknown format or the dimensions are too big.

Speed and memory footprint is also something to take into consideration. libvips, the library behind imgproxy that performs image transformations, is arguably the fastest out there.

In siege testing with 100 concurrent users performing 50 transformations each, imgproxy only used 200MB of RAM. Under a real-world heavy load, the service rarely consumes more than half of that.

As a bonus, imgproxy follows the Twelve-Factor App paradigm to the letter and can be fully configured using environment variables only. Among other things, you can set read, write, and download timeouts, maximum dimensions for accepted images, and the number of image requests to be processed simultaneously (default: 100).

Sounds great! Any alternatives?

Sure, imgproxy is not groundbreaking in its concept. Quick Google search for “image proxy” or “thumbnail service” will yield tools like Thumbor, imageproxy (don’t confuse with our imgproxy), picfit or imaginary—all doing pretty much the same thing: resizing images. If so, why write yet another one? As imgproxy’s README states: “imgproxy does one thing — resizing remote images — and does it well.” The trick was to implement just the must-have features but to do it right.

For example, it would be great to:

  • …be able to rotate, flip and apply masks to images. Still, it is easier with CSS.
  • …have built-in HTTP caching of some kind, but it is way better to use a CDN or a caching proxy server for this, as you will need it anyway once your project grows.
  • …have an HTTPS support built-in, but it makes more sense to give this task to a proxy server such as nginx.

Cutting extra features and focusing on core functionality allows for a very lightweight and memory-efficient solution. Thumbor, for instance, is very powerful, but its Docker container is 300 times bigger than imgproxy’s. It also does not support 12factor and uses GraphicsMagick as an underlying library, which is considerably slower than vips. Imaginary does not focus on security; anyone can compose a right URL and piggyback on your service. The only protection this tool gives you is a way to set up Authorization HTTP header, but that approach, if not coupled with other security features, makes you vulnerable for a “man-in-the-middle” attack. You are not protected from image bombs either. Picfit uses storage to keep track of transformations. Pilbox does not use vips, does not support 12factor and relies on Tornado web server, that you might not need. Thumbor, by the way, also expects you to use additional software for caching. imgproxy has no such dependencies.

imgproxy and eBay: real-life success story

We did not start working on imgproxy out of sheer spite: it was a conscious decision in response to our client’s needs. For one of our most technically challenging projects, eBay Social, we started out with CarrierWave and had to carefully consider all alternatives described above once we ran into first scaling problems. Consider this: only in the last three months we had to transform 1 217 934 images coming from parent eBay. At this rate, storing (and re-transforming) all those thumbnails becomes a daunting task very quickly and can easily take up the whole day if you are keeping image versions locally.

imgproxy at ebaysocial.ru

imgproxy URLs at ebaysocial.ru. Caching is performed with CloudFront CDN.

Using imgproxy in production allowed to reduce the time needed for updating the whole image collection to, well, zero: as images come from a remote source, only those that were requested would be transformed on the fly and cached by a CDN for further requests.


imgproxy is open source and can be used in any project, big or small, to deal with resizing remote images. It will help your team embrace microservice approach and reduce the overhead from handling image assets. imgproxy is easy to set up with Heroku or Docker and is completely language or framework agnostic—any web developer can benefit from it.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK