Using NAS with RAID as the main home storage

As everybody probably has read more than once, is that memory cards and hard drives will fail sooner or later, so some kind of redundancy and/or backup is necessary. You may look at redundancy as a kind of safety net that will “catch” your data if something goes wrong, as the data is stored in more than one place at the same time. There are several ways to have a storage with redundancy and I will mention specially a NAS setup with RAID.

We have to acknowledge that most people working with photography are using laptops rather than a stationary computer, although the need for safe storage is the same, the solution may vary. For stationary computers you could just add more drives in your cabinet and set up a RAID inside the cabinet, many mainboards support RAID nowadays. Most stationary medium size and larger cabinets have bays for 5-6 3.5″ (or 2.5″) drives and a couple of 2.5″ drives, means you could use up to 6 of them in a RAID. The drawbacks of setting up a RAID inside the “main” cabinet is that if something catastrophically happens like a mainboard or power supply failure, you may fry everything that is connected inside the cabinet. Of course, you may also be lucky and nothing gets fried, but you will only know for sure when it’s potentially too late. The second drawback is that a lot of drives makes a lot of unwanted noise and vibration (the drives are not just running when you are using it or there is a backup running).

With a laptop you will of obvious reasons have limited space to have any serious multi drives inside, although some larger laptops have room for two drives, the first one is normally an OS-drive and the second for storage, making it vulnerable right out of the box. An obvious solution is to have a cloud backup, but for photographers who shot RAW, that is not a good solution as RAW-files normally is in the 20-45 MB range in lossless compress and even 100MB for a D850-file uncompressed, or others working with large files, making it a painfully slow process to work via the cloud. A better option is a USB3/Thunderbolt external drives, but then you need a very rigid system of doing backup regularly so you don’t risk losing any critical data between backups. Other option is to have a wireless drive, like a USB3 drive attached to the wireless router, but then you are limited to the speed of your home network, witch in most cases are much slower than the transfer speed of a magnetic drive, and it will slow down you surf speed if you or someone else in the household are using wi-fi to access the internet or the same drive at the same time. For smaller files like documents and such, it shouldn’t be a huge problem though. But one external drive attached to the router is still a poor substitute for a NAS should that one drive fail.

Possible one of the best options is to have a docking station for the laptop and cabled network in the house to the router or a switch, and then cabled to a NAS (Network Attached Storage). I recommend a docking station because it makes your life much easier as it charges the laptop, connects to home network and internet, connects to USB’s attached to the docking (can be printer, scanner, mouse and keyboard, drawing tablet, card reader and so on) and one or more external monitors with just one cable (there are different connection and options to docking stations). Here is when this solution merges with a stationary computer with cabled network to the same router/switch.

Continuing will get somewhat more technical, but try to follow as the technicality is the reasons behind the choices.

As of writing, almost all wireless routers, mainboards and NAS only has 1Gb/s EJ45 Ethernet cable sockets (truth, more NAS comes now with 10Gb/s sockets or has it as an option), making it the limiting factor speed vise. 1Gb/s is the same as 125 MB/s and again the same as 1000Mb/s, and is more or less the same of a traditional magnetic drive (HDD) can do over time. Top speed of an HDD is about 250 MB/s and with a big buffer maybe more, but for mixed us, 125 MB/s should be about what it can do, so you should basically not notice much difference between a magnetic drive attached inside your computer or via an Ethernet cable. Better still though, is the 10Gb/s protocol that uses the same EJ45 socket and plug, but you need cat-6 cables or higher (preferably cat6a) in your house. As mentioned, hardly any mainboards or wireless routers has 10 Gb/s adapters built-in yet, even though the standard is more than 10 years old!

For comparison a SATA SSD has about 550MB/s speed while the newer M.2 SSD NVMe has up to 3500 MB/s. Notice, many mainboards have two M.2 sockets, but often only one of them uses the PCI Express bus, and the other may only use the SATA bus, so be careful adding a M.2 NVMe SSD in your computer, so that you don’t strangle it to SATA 6Gb/s speed only.

Before continuing, what is the real-life speed you could expect from your wi-fi/wireless home network then?

If you have an old “g” router, you should expect about 20 Mb/s.

If you have the not so old “n” router, you should expect about 50 Mb/s.

If you have the newer “ac” router, you should expect about 100 Mb/s.

The next step will be “ax” routers that are just out on the market with about 800 Mb/s.

Mark that if your router has several antennas AND your laptop or stationary computer (or other devices) support multiple streams, the speed could increase accordingly. If your router is an “n” type and your laptop support “ac”, you will still be limited by the “n” router as the slowest device.

Connecting a NAS to the wireless router with an Ethernet cable will limit your speed to about 125 MB/s as mentioned several times already because the router is limited to the 1Gb/s protocol. If you choose Ethernet cable or wireless from the router, is obviously a choice you need to make, but there is no doubt that an Ethernet cable is more stable and won’t interfere with the wi-fi/wireless capacity for other usage, like internet. If you on the other hand go “all in” with a 10Gb/s NAS and a 10 Gb/s switch, you will need to go for an Ethernet cable all the way to get the most out of it, as not any commercially available wireless system can compete with this speed yet.

I’ve tested the speed between the Asus RT AC87U router to the computer with an Asus PCE-AC88 wireless network card, and got a maximum of 400 Mb/s read or write from/to the NAS connected wirelessly. Not bad speed at all, but then you are occupying the whole capacity of your ac-net (you may still have the n-net running at the same time, but computers and other gadgets doesn’t switch between when first connected to one of them). Even if you live alone or have your own private wireless network, it may be times when you feel like then connection is slow, and if you have others in the household I would guess it will be complaints some times.

Because of “only” 400Mb/s read/write speed via wireless and potentially occupying the wireless bandwidth, I opted to not only connect the NAS with one Ethernet cable to the router, but two, and then connect both stationary computers with Ethernet cable to the same router. Then we have moved all the heavy lifting (included internet connection to the stationary computers) over to the Ethernet cables and have the Wi-Fi available to everything else.

I mentioned two cables from the NAS to the router, and the reason why is that 1Gb/s is not bad if you can have it for yourself, but sharing it with one more cut it in half, and then it gets noticeably slower. So, to help this I’ve connected two Ethernet cables from the NAS to the router. Note, the NAS needs to support so-called link aggregation and the same with the router, where you basically double the bandwidth with two cables, so you can use the NAS at full 1 Gb/s from two separate computers at the same time via the router.

When you start looking into NAS you will also quickly get in touch with “RAID”, Redundant Array of Independent Disks. So, to begin with a short explanation about the possibility with different RAID setups. These are the most common RAID options.

RAID 0 “Stripe” Needs minimum 2 drives, but is not really suited for safe storage as it doesn’t have any safety net. It’s a pure speed advantage as data is split and shared between 2 drives at the same time. Can be increased with one and one drive.

RAID 1 “Mirror” Needs minimum 2 drives and as the subtitle suggests, the 2 drives are exactly the same, so should one fail, the other should have all the data. This is the easiest and cheapest way to have any redundancy. Can be expanded with one and one drive, but then it’s better to go for RAID 5 or higher.

RAID 5 “Parity” Needs minimum 3 drives where all data is spread across several drives, including parity, to be able to recover data from 1 failed drive out of 3 drives. Can be expanded with one and one drive.

RAID 6 “Double parity” Needs minimum 4 drives where all data is spread across several drives, including double parity, to be able to recover data from 2 simultaneously failed drives out of 4 drives. Can be expanded with one and one drive.

RAID 10 “Mirror + Stripe” Needs minimum 4 drives where data is striped and mirrored. Can be expanded in pairs only, means 4-6-8-10 and so on drives.

Out of those, RAID 2 is the cheapest, RAID 5 is safer, RAID 6 even more safe and RAID 10 is fastest and still pretty safe, but you will have less space to use, as you will have only half of the total drive capacity available. If you would like to know the read and write speed of any combination and RAID type, you need to search for a RAID calculator that also calculate transfer speeds, as this will vary with number and type of drives, and type of RAID.

I have three recommendations here for type of RAID:

1. RAID 1 with two drives internally in your computer cabinet, and cloud backup of the most important files. This is the cheapest alternative and is relatively silent as it’s only two drives and are not running all of the time. Alternatively, two drives in a NAS with cloud backup (better option for noise).

2. RAID 6 in a NAS. You will have good redundancy with the security of possibility of two drives fail at the same time (or before you are able to rebuild your NAS after first drive failure). You can locate your NAS in another room so you won’t hear any noise from it. When expanding from four drives, you will still only have two drives used as protection, so for each drive you add above the mandatory four, it will add to the available capacity.

3. RAID 10 in a NAS. Highly expensive RAID, but also high performance and security as two and two drives are equal. If scaled to higher number of drives it will always utilize only half of the total capacity, so best used with four larger drives than more than four smaller ones.

Brand and type of drives?

If you chose a RAID with many drives, notice that most consumer NAS drives are recommended for a limited number of drives together in a NAS, like up to 5-7 drives the Seagate Ironwolf or Western Digital Red are a good choice together with HGST NAS drives, but HGST drive are more noisy than Seagate and WD drives. While Seagate Ironwolf PRO and Western Digital Red PRO drives are made for working together in numbers up to about 12 drives, notice that Pro-drives are more noisy than “no-pro” drives. For NAS with more than 12 drives you are recommended to go for enterprise drives like Seagate Exos or Western Digital Gold. Mark also that “normal” desktop drives are not recommended used in a NAS.

For fun, and also to check how it would be to use the NAS as a main storage drive compared to the internal drives in the cabinet, I did a test of batch processing using Nikon Capture NX-D 1.5.0. I had copied some RAW-files from the Nikon D810 over to a separate folder and saved the converted jpeg’s back into the source folder, computer was restarted between each run to clear any buffer or cache:

Files from and to the NAS

Time was 8 min 25 seconds for the whole process with spikes in receiving files from the NAS ranging from 250 Mb/s (31MB/s) up to 500 Mb/s (62MB/s), so not much stress for the NAS doing this. Sending was much smaller, as it was just jpegs, with about 50 Mb/s (6MB/s).

Files from and to the internal SATA SSD

The same files on the SATA SSD, converted back to the source folder took 3 min 17 seconds, with read speed at 320Mb/s (40 MB/s) with less spikes but more wave forms, and write at 80Mb/s (10 MB/s).

Files from and to the internal M.2 NVMe SSD

The same files on the NVMe SSD (system drive), converted back to the source folder took 3 min 18 seconds with even less spikes and more wave form than from the SATA SSD, but the final time was for all practical measures the same for the NVMe SSD and the SATA SSD.

Comment on the speed test.

Capture NX-D has never been a speed demon, but never the less the converter I use all of the time because of both the initial and the final image quality. On the other hand, I’m writing this article in Word, saved on the NAS, and I wouldn’t notice any difference if it was stored locally in the computer as its such a small file. Obviously working on bigger files or batch processing will take more time over the 1Gb/s Ethernet cable, compared to a locally attached SSD. The speed from the NAS also depends on RAID setup, type of drives and number of drives, so experienced speed will differ from setup to setup. Why the read speed from the NAS wasn’t higher, I don’t know, as the highest read and write speed to the NAS is limited only by the 1Gb/s Ethernet cable. Even though it’s much faster from and to the internal drives, not even there the maximum read or write speed was achieved. If you have an idea about it, tell me in the comments below.

Where should you place your NAS?

As for me, one of the reasons to move the magnetic drives out of the computer cabinet, was to get rid of the noise and vibration of the spinning hard drives, so placing the NAS on, next to, or even under the desk where I’m working is not an option for me. In another room or even on a different floor is a good idea if its possible. I’ve read some putting their NAS inside a closet, and I’m not sure that is a good solution in the long run as the NAS will always give off some heat, and in enclosed space it could be too hot and reduce the lifetime of the drives.

Benefits with a NAS with RAID

– You have options ranging from one bay NAS to up to thirty bays, but most common for home usage is between two and eight bays.

– You can have everything from two small drives to massive drives in a multi bay NAS.

– You can connect it to your wireless router and have access to it from any computer or device in range, and even via internet away from home.

– If connected with Ethernet cable you have speed equal 1 Gb/s or more, or via wireless as fast as your connection can manage, witch all are considerably faster than using cloud storage.

– You can place the NAS away from the computer, somewhere the noise doesn’t bother you.

– With all RAID setups, apart from RAID 0, you will have a safety-net in the parity in the RAID, means you may have one or more simultaneously drive failures, and still don’t lose data.

– If you find your self in need of more space, you can either buy bigger drives and “hot swapping” them, or you may buy an expansion unit if your NAS supports it.

Drawbacks with a NAS with RAID

– Apart from RAID 0, you will lose the storage capacity of one or more drives as safety.

– The NAS box is relatively expensive, to just house some drives and to distribute and collect data, but you may save some money by buying a used one.

– Even though not terribly difficult to set up, a certain level of knowledge is needed to do the installation and setup (lots of guides are to be found on producers websites, on YouTube and elsewhere).

– If connected via wi-fi, the transfer speed is in most cases considerably slower than the NAS is capable of, or from a local drive.

Should you schedule restart or even schedule to turn off your NAS at certain hours?

I guess most people just have the NAS running all of the time without restart, sleep or anything. Will raise the question though, what if you go to work in the morning and won’t need to access your NAS before late afternoon or evening? Why do I ask? Drive failure is mostly related to high temperature and/or running hours. Operating temperature should preferably be below 40°C and running hours, well, runs as long as the NAS is on and the drives are not sleeping. So, if you can schedule turn off for eight hours a day, you may extend the lifetime (hours) of the drives by 30%. The schedule could even be 12 hours in the weekdays and less in the weekends. It will also save you some on the electricity bill. Talking about lifetime for NAS drives, although a drive can fail at any time, the chance of failure increases with each hour running, and according to data center analyses, the time you should really expect a drive to fail is after about the equivalent to five years of continuous run.

Conclusion

As mentioned throughout this blog, my goal is to have a powerful computer that is also as silent as possible (within reasonable measures). The reason is that if you are working on a noisy computer for hours and then turn it off, it will feel like a relief when the drives and fans spins down to a stop. So why not try to get that peaceful silence even when the computer is running? Then all old-style magnetic drives need to go, and fans and general cooling set up with least possible noise in mind. The drawback with getting rid of the magnetic drives, is that SSD’s are still pretty expensive in comparison. Solution for me is to have all the files stored at the NAS, but keep the latest and most used ones temporarily on the local SSD as well. To explain the process better, new files are always transferred from memory cards and over to the local SSD, and once a day the NAS is automatically updated with the new files from the SSD. When the local SSD are getting filled up or I’m done with a batch of photos, I’ll just delete them from the SSD as they are already backed up on the NAS, to free up space on the SSD for newer projects. Then I will always (hopefully) have space on the local SSD. If you set up the NAS and your SSD with equal folder system, the files and folders just add where they should on the NAS and will be easily found when needed on a later time. For scheduling backup from the local SSD to the NAS, I’ve used a small program called “Karen Replicator” for more than ten years, and it’s easily customizable to check, compare, exclude or whatever your need, and I’ve set it up to run once a day.

Afterword.

Many people would disagree that a solution with a NAS with RAID is actually a “backup”, and I agree, having a RAID setup as safety is not “backup”. That’s why the most important files (means all photos, digitized CD-collection, other work-related files and more) are backed up to the cloud regularly.

The article became very long and includes a lot of consideration, but I feel all of it is needed to explain the choices I’ve done. If you have any questions or comments, feel free to add them in the comment section below.

Update 2020.
In the article I mentioned that I’ve connected the NAS to the wireless router with dual cables. This changed about a year ago because I relocated the NAS to a storage room in the basement where it’s colder and I would never hear the noise from the NAS. Out of sight, out of mind and just working is a way to describe it now. I use cabled network for everything stationary like TV’s, BD-players, PlayStation, surround processor, stationary computers, and use the wireless only for laptops and phones.

All photos in the article are from the respective producers website.

Thanks for visiting!