Silent Fanless FreeBSD Server – Redundant Backup
source link: https://www.tuicool.com/articles/hit/3iErYbm
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
I brought up this topic in the past. It was in the form of more theoretical Silent Fanless FreeBSD Desktop/Server post and more hands-on Silent Fanless FreeBSD Server – DIY Backup article.
One of the comments after the latter was that I compared non-redundant backup solution (single disk) to redundant backup in the cloud. Today – as this is my main backup system – I would like to show you redundant backup solution with two disks in ZFS mirror along with real power usage measurements. This time I got ASRock J3355B-ITX motherboard with only 10W TDP which includes 2-core Celeron J3355 2.0-2.5 GHz CPU and small shiny REALAN H80 Mini ITX case. It looks very nice and comes from AliExpress at very low $33 price for new unit along with free shipping.
Build
Here is how the REALAN H80 case looks like.
The ASRock J3355B-ITX motherboard.
Same as with the earlier build the internal Seagate BarraCuda 5TB 2.5 SATA drives costs about $200. The same Seagate Backup Plus 5TB 2.5 disk in external case with USB 3.0 port costs nearly half of that price – only $120 – at least in the Europe/Poland location. I took the decision to buy external ones and rip off their cases. That saved me about $160.
Here is the simple performance benchmark of these 2.5 disks.
% <strong>which pv</strong> pv: aliased to pv -t -r -a -b -W -B 1048576 % <strong>pv < /dev/ada0 > /dev/null</strong> 1.35GiB 0:00:10 [ 137MiB/s] [ 137MiB/s] ^C % <strong>dd < /dev/ada0 > /dev/null bs=8M</strong> 127+0 records in 127+0 records out 1065353216 bytes transferred in 7.494081 secs (142159287 bytes/sec) ^C
About 135MB/s per disk.
The ripped of parts of Seagate Backup Plus USB cases.
What made me laugh was that as I got different cases colors (silver and gray) the disks inside also had different colors (green and blue) :>
… but their part number is the same, here they are mounted on a REALAN H80 disks holder.
For the record – several REALAN H80 case real shots (not renders). First its front.
Back.
Side with USB port.
Bottom.
Top.
Case parts.
Generally the REALAN H80 looks really nice. Little lower REALAN H60 (without COM slots/holes in the back) looks even better but I wanted to make sure that I will have room and space for hot air in that case – as space was not a problem for me.
Cost
The complete price tops at $220 total. Here are the parts used.
<strong>PRICE COMPONENT</strong> $49 CPU/Motherboard ASRock J3355B-ITX Mini-ITX $10 RAM 4GB DDR3 $13 PSU 12V 7.5A 90W Pico (internal) $2 PSU 12V 2.5A 30W Leader Electronics (external) $33 Supermicro SC101i $3 SanDisk Fit 16GB USB 2.0 Drive (system) $120 Seagate 5TB 2.5 drive (ONE) $120 Seagate 5TB 2.5 drive (TWO) $350 TOTAL
That is $110 for the ‘system’ and additional $240 for ‘data’ drives.
Today I would probably get the ASRock N3150DC-ITX motherboard instead because of 1 DC jack slot (compatible with 19V power adapter) on its back. This will eliminate the need for additional internal Pico PSU power supply …
The ASRock N3150DC-ITX with builtin DC jack.
PSU
As in the earlier Silent Fanless FreeBSD Server – DIY Backup article I used small 12V 2.5A 30W compact and cheap external PSU instead of the large 90W PSU from FSP Group . As these low power motherboard does not need a lot of power.
New Leader Electronics PSU label.
The internal power supply is Pico PSU which now tops as 12V 7.5A 90W power.
Power Consumption
I also measured the power consumption with power meter.
The whole box with two Seagate BarraCuda 5TB 2.5 drives for data on ZFS mirror and SanDisk 16GB USB 2.0 system drive used about 10.4W in idle state.
I used all needed settings from my earlier The Power to Serve – FreeBSD Power Management article with CPU speed limited between 0.4GHz and 1.2GHz.
The powerd(8) settings in the /etc/rc.conf file are below.
powerd_flags="-n hiadaptive -a hiadaptive -b hiadaptive -m 400 -M 1200"
I used python(1) [1] to load the CPU and dd(8) to load the drives. I used dd(8) on the ZFS pool so 1 disk thread will read [2] and write [3] from/to both 2.5 disks. I temporary disabled LZ4 compression for the write tests.
<strong>[1]</strong> # echo '999999999999999999 ** 999999999999999999' | python <strong>[2]</strong> # dd < /data/FILE > /dev/null bs=1M <strong>[3]</strong> # dd > /data/FILE < /dev/zero bs=1M
<strong>POWER CPU LOAD I/O LOAD</strong> 10.4 W IDLE IDLE 12.9 W IDLE 1 DISK READ Thread(s) 14.3 W IDLE 1 DISK READ Thread(s) + 1 DISK WRITE Thread(s) 17.2 W IDLE 3 DISK READ Thread(s) + 3 DISK WRITE Thread(s) 11.0 W 8 CPU Thread(s) IDLE 13.4 W 8 CPU Thread(s) 1 DISK READ Thread(s) 15.0 W 8 CPU Thread(s) 1 DISK READ Thread(s) + 1 DISK WRITE Thread(s) 17.8 W 8 CPU Thread(s) 3 DISK READ Thread(s) + 3 DISK WRITE Thread(s)
That’s not much remembering that 6W TDP power motherboard ASRock N3150B-ITX with just single Maxtor M3 4TB 2.5 USB 3.0 drive used 16.0W with CPU and I/O loaded. Only 1.8W more (on loaded system) with redundancy on two 2.5 disks.
Commands
The crypto FreeBSD kernel module was able to squeeze about 68MB/s of random data from /dev/random as this CPU has built in hardware AES-NI acceleration. Note to Linux users – the /dev/random and /dev/urandom are the same thing on FreeBSD. I used both dd(8) and pv(1) commands for this simple test. I made two tests with powerd(8) enabled and disabled to check the difference between CPU speed at 1.2GHz and at 2.5GHz with Turbo mode.
Full speed with Turbo enabled (note 2001 instead of 2000 for CPU frequency)..
# <strong>/etc/rc.d/powerd stop</strong> Stopping powerd. Waiting for PIDS: 1486. % <strong>sysctl dev.cpu.0.freq</strong> dev.cpu.0.freq: 2001 % <strong>which pv</strong> pv: aliased to pv -t -r -a -b -W -B 1048576 % <strong>dd < /dev/random bs=8M | pv > /dev/null</strong> 1.91GiB 0:00:31 [68.7MiB/s] [68.1MiB/s] 265+0 records in 265+0 records out 2222981120 bytes transferred in 33.566154 secs (70226864 bytes/sec) ^C
CPU limited to 1.2GHz with powerd(8) daemon was able to squeeze about 24MB/s.
# <strong>service powerd start</strong> Starting powerd. % <strong>which pv</strong> pv: aliased to pv -t -r -a -b -W -B 1048576 % <strong>dd < /dev/random bs=8M | pv > /dev/null</strong> 568MiB 0:00:23 [25.3MiB/s] [24.7MiB/s] 71+0 records in 71+0 records out 595591168 bytes transferred in 23.375588 secs (25479195 bytes/sec ^C
Below I will show you the data from dmesg(8) about the used USB and 2.5 drives.
The dmesg(8) information for the SanDisk Fit USB 2.0 16GB drive.
# <strong>grep da0 /var/run/dmesg.boot</strong> da0 at umass-sim1 bus 1 scbus3 target 0 lun 0 da0: Removable Direct Access SPC-4 SCSI device da0: Serial Number 4C530002030502100093 da0: 400.000MB/s transfers da0: 14663MB (30031250 512 byte sectors) da0: quirks=0x2
… and two Seagate BarraCuda 5TB 2.5 drives.
# <strong>grep ada /var/run/dmesg.boot</strong> ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ACS-3 ATA SATA 3.x device ada0: Serial Number WCJ0DRJE ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 4769307MB (9767541168 512 byte sectors) ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ACS-3 ATA SATA 3.x device ada1: Serial Number WCJ0213S ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 4769307MB (9767541168 512 byte sectors)
The whole /var/run/dmesg.boot content (without disks) is shown below.
# <strong>cat /var/run/dmesg.boot</strong>
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.2-RELEASE-p7 #0: Tue Dec 18 08:29:33 UTC 2018
[email protected]:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)
VT(vga): resolution 640x480
CPU: Intel(R) Celeron(R) CPU J3355 @ 2.00GHz (1996.88-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x506c9 Family=0x6 Model=0x5c Stepping=9
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x4ff8ebbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x101<LAHF,Prefetch>
Structured Extended Features=0x2294e283<FSGSBASE,TSCADJ,SMEP,ERMS,NFPUSG,MPX,PQE,RDSEED,SMAP,CLFLUSHOPT,PROCTRACE,SHA>
XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
TSC: P-state invariant, performance statistics
real memory = 4294967296 (4096 MB)
avail memory = 3700518912 (3529 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table:
WARNING: L1 data cache covers less APIC IDs than a core
0 < 1
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
ioapic0 irqs 0-119 on motherboard
SMP: AP CPU #1 Launched!
Timecounter "TSC" frequency 1996877678 Hz quality 1000
random: entropy device external interface
kbd1 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff80ff4580, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
nexus0
vtvga0: on motherboard
cryptosoft0: on motherboard
acpi0: on motherboard
unknown: I/O range not supported
cpu0: on acpi0
cpu1: on acpi0
attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: port 0x70-0x77 on acpi0
atrtc0: Warning: Couldn't map I/O.
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: iomem 0xfed00000-0xfed003ff irq 8 on acpi0
Timecounter "HPET" frequency 19200000 Hz quality 950
Event timer "HPET" frequency 19200000 Hz quality 550
Event timer "HPET1" frequency 19200000 Hz quality 440
Event timer "HPET2" frequency 19200000 Hz quality 440
Event timer "HPET3" frequency 19200000 Hz quality 440
Event timer "HPET4" frequency 19200000 Hz quality 440
Event timer "HPET5" frequency 19200000 Hz quality 440
Event timer "HPET6" frequency 19200000 Hz quality 440
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
vgapci0: port 0xf000-0xf03f mem 0x90000000-0x90ffffff,0x80000000-0x8fffffff irq 19 at device 2.0 on pci0
vgapci0: Boot video device
hdac0: mem 0x91210000-0x91213fff,0x91000000-0x910fffff irq 25 at device 14.0 on pci0
pci0: at device 15.0 (no driver attached)
ahci0: port 0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0x91214000-0x91215fff,0x91218000-0x912180ff,0x91217000-0x912177ff irq 19 at device 18.0 on pci0
ahci0: AHCI v1.31 with 2 6Gbps ports, Port Multiplier supported
ahcich0: at channel 0 on ahci0
ahcich1: at channel 1 on ahci0
pcib1: irq 22 at device 19.0 on pci0
pci1: on pcib1
pcib2: irq 20 at device 19.2 on pci0
pci2: on pcib2
re0: port 0xe000-0xe0ff mem 0x91104000-0x91104fff,0x91100000-0x91103fff irq 20 at device 0.0 on pci2
re0: Using 1 MSI-X message
re0: Chip rev. 0x4c000000
re0: MAC rev. 0x00000000
miibus0: on re0
rgephy0: PHY 1 on miibus0
rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
re0: Using defaults for TSO: 65518/35/2048
re0: Ethernet address: 70:85:c2:3f:53:41
re0: netmap queues/slots: TX 1/256, RX 1/256
xhci0: mem 0x91200000-0x9120ffff irq 17 at device 21.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
isab0: at device 31.0 on pci0
isa0: on isab0
acpi_button0: on acpi0
acpi_tz0: on acpi0
atkbdc0: at port 0x60,0x64 on isa0
atkbd0: irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: cannot reserve I/O port range
est0: on cpu0
est1: on cpu1
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
hdacc0: at cad 0 on hdac0
hdaa0: at nid 1 on hdacc0
ugen0.1: <0x8086 XHCI root HUB> at usbus0
uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
pcm0: at nid 21 and 24,26 on hdaa0
pcm1: at nid 20 and 25 on hdaa0
pcm2: at nid 27 on hdaa0
hdacc1: at cad 2 on hdac0
hdaa1: at nid 1 on hdacc1
pcm3: at nid 3 on hdaa1
uhub0: 15 ports with 15 removable, self powered
ugen0.2: at usbus0
uhub1 on uhub0
uhub1: on usbus0
uhub1: 4 ports with 4 removable, self powered
Trying to mount root from zfs:zroot/ROOT/default []...
random: unblocking device.
re0: link state changed to DOWN
ZFS Pool Configuration
To get higher LZ4 compression ratio I use larger blocksize (1MB) on this ZFS mirror pool. Here is the ZFS pool status.
% <strong>zpool status data</strong>
pool: data
state: ONLINE
scan: scrub repaired 0 in 44h14m with 0 errors on Mon Feb 11 07:13:42 2019
config:
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
label/WCJ0213S ONLINE 0 0 0
label/WCJ0DRJE ONLINE 0 0 0
errors: No known data errors
I get 4% compression ( 1.04x ) on that ZFS pool. Its about 80% filled with lots of movies and photos so while such compression ratio may not be great it gives a lot of space. For example 4% of 4TB of data is about 160GB of ‘free’ space.
% <strong>zfs get compressratio data</strong> NAME PROPERTY VALUE SOURCE data compressratio 1.04x -
Here is the ZFS pool configuration.
# <strong>zpool history</strong> History for 'data': 2018-11-12.01:18:33 zpool create data mirror /dev/label/WCJ0229Z /dev/label/WCJ0DPHF 2018-11-12.01:19:11 zfs set mountpoint=none data 2018-11-12.01:19:16 zfs set compression=lz4 data 2018-11-12.01:19:21 zfs set atime=off data 2018-11-12.01:19:34 zfs set primarycache=metadata data 2018-11-12.01:19:40 zfs set secondarycache=metadata data 2018-11-12.01:19:45 zfs set redundant_metadata=most data 2018-11-12.01:19:51 zfs set recordsize=1m data (...)
We do not need redundant_metadata as we already have two disks, its useful only on single disks configurations.
Self Solution Cost
As in the earlier post I will again calculate how much energy this server would consume. Currently 1kWh of power costs about $0.20 in Europe/Poland (rounded up). This means that running computer with 1000W power usage for 1 hour would cost you $0.20 on electricity bill. This system uses 10.4W idle and 12.9W when single disk read occurs. For most of the time server will be idle so I assume 11.0W average for the pricing purposes.
That would cost us $0.0022 for 11.0W device running for 1 hour.
Below you will also find calculations for 1 day (24x multiplier), 1 year (another 365.25x multiplier) and 3 years (another 3x multiplier).
<strong> COST TIME</strong> $0.0022 1 HOUR(S) $0.0528 1 DAY(S) $19.285 1 YEAR(S) $57.856 3 YEAR(S) $96.426 5 YEAR(S)
Combining that with server cost ($350) we get TCO for our self hosted 5TB storage service.
<strong> COST TIME</strong> $369.29 1 YEAR(S) $407.86 3 YEAR(S) $446.43 5 YEAR(S)
Our total 3 years TCO is $407.86 and 5 years is $446.43. Its for running system non-stop. We can also implement features like Wake On LAN to limit that power usage even more.
Cloud Storage Prices
This time after searching for cheapest cloud based storage I found these services.
- Amazon Drive
- Amazon S3 Glacier Storage
- Backblaze B2 Cloud Storage
- Google One
Here is its cost summarized for 1 year period for 5TB of data.
<strong>PRICE TIME SERVICE</strong> $300 1 YEAR(S) Amazon Drive $310 1 YEAR(S) Google One $240 1 YEAR(S) Amazon S3 Glacier Storage $450 1 YEAR(S) Backblaze B2 Cloud Storage
For the Backblaze B2 Cloud Storage I assumed average between upload/download price because upload is two times cheaper then download.
Here is its cost summarized for 3 year period for 5TB of data.
<strong>PRICE TIME SERVICE</strong> $900 3 YEAR(S) Amazon Drive $930 3 YEAR(S) Google One $720 3 YEAR(S) Amazon S3 Glacier Storage $1350 3 YEAR(S) Backblaze B2 Cloud Storage
Here is its cost summarized for 5 year period for 5TB of data.
<strong>PRICE TIME SERVICE</strong> $1500 5 YEAR(S) Amazon Drive $1550 5 YEAR(S) Google One $1200 5 YEAR(S) Amazon S3 Glacier Storage $2250 5 YEAR(S) Backblaze B2 Cloud Storage
Now lets compare costs of our own server to various cloud services.
If we would run our server for just 1 year the price will be similar.
PRICE TIME SERVICE
$369 1 YEAR(S) Self Build NAS
$300 1 YEAR(S) Amazon Drive
$310 1 YEAR(S) Google One
$240 1 YEAR(S) Amazon S3 Glacier Storage
$450 1 YEAR(S) Backblaze B2 Cloud Storage
It gets interesting when we compare 3 years costs. Its two times cheaper to self host our own server then use cloud services. One may argue that clouds are located in many places but even if we would buy two such boxes and put one – for example in our friends place at Jamaica – or other parts of the world.
PRICE TIME SERVICE
$408 3 YEAR(S) Self Build NAS
$528 3 YEAR(S) Self Build NAS (assuming one of the drives failed)
$900 3 YEAR(S) Amazon Drive
$930 3 YEAR(S) Google One
$720 3 YEAR(S) Amazon S3 Glacier Storage
$1350 3 YEAR(S) Backblaze B2 Cloud Storage
… but with 5 years using cloud service instead of self hosted NAS solution is 3-5 times more expensive … and these were the cheapest cloud services I was able to find. I do not even want to know how much would it cos on Dropbox for example
PRICE TIME SERVICE
$447 5 YEAR(S) Self Build NAS
$567 5 YEAR(S) Self Build NAS (assuming one of the drives failed)
$1500 5 YEAR(S) Amazon Drive
$1550 5 YEAR(S) Google One
$1200 5 YEAR(S) Amazon S3 Glacier Storage
$2250 5 YEAR(S) Backblaze B2 Cloud Storage
… and ‘anywhere’ access is not an argument for cloud services because you can get external IP address for you NAS or use Dynamic DNS – for free. You may also wonder why I compare such ‘full featured NAS’ with S3 storage … well with rclone (rsync for cloud storage) you are able to synchronize your files with almost anything
Not to mention how much more privacy you have with keeping all your data to yourself … but that is priceless.
You can also setup a lot more services on such hardware – like FreeNAS with Bhyve/Jails virtualization … orNextcloud instance … orSyncthing … while cloud storage is only that – a storage in the cloud.
Summary
Not sure what else could I include in this article. If you have an idea what else could I cover then let me know.
EOF
Advertisements
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK