5

The Petabyte Pi Project

 2 years ago
source link: https://www.jeffgeerling.com/blog/2022/petabyte-pi-project
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

The Petabyte Pi Project

May 18, 2022

I haven't had time to write up the details yet, but I wanted to share a project that's been many months in the making: The Petabyte Pi Project on YouTube.

I'm still doing follow-up testing based on feedback from Broadcom storage engineers, and will put out a much more in-depth blog post later, but the gist is:

Can a single Raspberry Pi cosplay as an 'enterprise' storage server, directly addressing 1 PB of storage?

Now... caveats abound here. What does 'enterprise' mean? And what does 'directly addressing' mean? Those things are all answered in the video linked above.

But to give a tl;dr: The Pi does not perform swimmingly. But... I did get a single array of 60 hard drives—20TB Exos HDDs to be exact—working in a 45Drives Storinator XL60 chassis, controlled only through a single Raspberry Pi Compute Module 4. Of course I had to rip out the Xeon guts and replace them with said Pi:

Raspberry Pi Compute Module 4 and Broadcom LSI HBAs inside Storinator XL60 chassis

Some snarky commenters will say "oh of course the Pi couldn't cope that well! It's a tiny SBC that's not built for it!"

But you're talking to the guy who's spent two years getting a GPU to work (barely) on a Raspberry Pi. And the guy who built a $5,000 all-SSD Pi storage server. If there was even a tiny chance I could get my hands on a raw petabyte of storage, I was going to run this thing to its logical conclusion.

Watch the video linked above for the whole story (for now), and I have to say an incredibly huge thank you to 45Drives, who sponsored this project and helped in so many ways (and to some Broadcom engineers who have been immensely helpful along the way).

45Drives even shipped a custom faceplate enshrining Red Shirt Jeff's favorite weapon of destruction on the front of the thing:

Storinator XL60 chassis - Red Shirt Jeff edition

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK