Andre's Blog
Perfection is when there is nothing left to take away
Storage tale

When I just started taking digital pictures, I thought a 320GB hard drive coupled with an occasional backup will last me for years, which worked quite well for the first 5-10 MP digital cameras I owned, but as soon as I upgraded to an 18 MP DSLR and started shooting RAW images at a high frame rate, I quickly realized that I need to find a better storage solution.

Just as many others, I first set my eyes on a hardware RAID device and bought the Addonics RTM435R5 four-bay enclosure. I set it up as a RAID1 mirror with four 2TB drives, which gave me 4TB of mirrored storage for me and my family. I honestly thought that this is the last time I bought storage, but once again things didn't quite work out as well as I hoped.

Curved mirrors

First signs of trouble started to show when I had to flip the power switch on the RTM435R5 a couple of times before the enclosure would power up. The unit was just over a year old at the time and out of warranty. I called Addonics and asked if I can buy a replacement unit and was surprised with a response that they don't make these enclosures anymore and the new ones use a different chipset, which will not read data from my exiting hard drives.

This made me realize that any hardware form of mirroring that does not offer guaranteed replacement parts from different vendors is unreliable on the long run and it is time to switch to a hardware-independent replication. The obvious choice was a software mirror, but the edition of Windows on the computer I used for image processing didn't have this option and I opted for manual replication.

Manual labor

I bought a Sabio DM4 and a Sans Digital's TR4UTBPN four-bay enclosures and striped four 2TB drives to have 8TB of fast storage in each enclosure. On a daily basis I copied images from the camera to the primary enclosure and then moved the same images to the secondary enclosure, so I always had two copies of everything. Once in a while I ran robocopy against the primary enclosure to keep images I edited or deleted synchronized with the copies on the secondary enclosure.

Using striping for long-term storage was not a very smart choice and after a while I reconfigured TR4UTBPN to access each drive individually and replaced the four 2TB drives in it with two 4TB drives, which allowed me to move these drives to a different enclosure if I needed. The Sabio's DM4 only supported RAID modes, so I kept it as four striped 2TB disks.

This approach worked quite well for a few years, until I filled up each drive almost to the top and faced the fact that even if I bought a couple of 4TB drives to fill in empty slots in TR4UTBPN, I would still have to manage copying and synchronizing files between multiple locations, which was simply a non-starter. On top of that, I was not very happy with a high-risk striped DM4 in my storage configuration.

Storage pools

This is where the free Windows 10 upgrade came to the rescue. Talk about good timing. After upgrading my Windows 7 computer to Windows 10, I started reading about Storage Spaces and in a matter of minutes I realized that Storage Spaces offers exactly what I was looking for.

The basic concept behind Storage Spaces is quite simple. You create a pool of drives and then one or more storage spaces that draw storage from the pool. Each storage space may be configured to maintain one, two or three copies of your data and they look like ordinary Windows drives.

Storage spaces that keep two or three copies of your data are called two-way and three-way mirrors. There is also a mode called parity that offers more usable space than the two-way mirror, but it comes at a price of reduced performance and some maintenance features becoming unavailable, so it is not very practical to use this mode.

A storage space drive may be formatted as an NTFS or a ReFS volume. The latter is the new file system format developed by Microsoft, which stands for Resilient File System and offers better data protection than NTFS. If ReFS detects data corruption in a storage space, it goes to the pool and requests a new usable storage block, which is then populated using the good copy of the data from the other drive and the bad storage block is marked as such.

You can add more drives of any size to the storage pool and expand your storage space without having to copy existing data. You can also move any drive to a different enclosure or just keep them in different enclosures.

Putting it all together

Enclosure

Even though you can use multiple enclosures for a single storage pool, it is more practical to keep all drives that comprise one storage pool in a single physical unit. Not only it's easier to turn it on and off, but this way you can carry just one enclosure if you want to transport all storage spaces in that storage pool to a different location.

When picking an enclosure for a storage pool, make sure to buy one that allows each drive accessed individually. Enclosures that support this mode will typically list the JBOD mode (Just a Bunch of Disks) in their specification, although some manufacturers label disk concatenation as JBOD, so make sure to keep all your receipts and packaging until you are sure all components work nicely together.

Out of the two enclosures I had, I could only use Sans Digital's TR4UTBPN for a storage pool because it could be configured as JBOD. Sabio's DM4 only supported RAID modes and was not compatible with Storage Spaces, so I had to buy a new enclosure. I almost bought another tried-and-true TR4UTBP, but these units have the same power supply as the one in my failed Addonics enclosure and there is a lot of buzz on the Internet about how bad these power supplies are, so I looked for an alternative.

After reviewing many enclosures from different manufacturers, I set my eyes on Sans Digital's MS4UM+(B). This enclosure comes with an external power supply, which is much easier to replace than an internal power supply and has a much better build quality than other enclosures in this price range (~$200 CAD). Some of the features not commonly found in other enclosures include a temperature-controlled variable speed fan, non-obnoxious LED lights (most enclosures will illuminate your room with eerie blue or green light at night) and an automatic power-off feature when the computer is turned off.

Hard Drives

I decided to build two separate storage pools, one per enclosure, and picked up two new 4TB drives, so each enclosure would contain two 4TB drives and two 2TB drives and would give me 6TB of usable mirrored space per enclosure.

Storage Spaces are designed to work with any consumer-grade drives, but I had such bad experience with WD Caviar Green drives (two out of four I bought went bad within a year) that I opted for more expensive Caviar Red NAS drives. I also had similar in specs Seagate NAS drives in the other enclosure. NAS drives are better built, have their internal maintenance processes optimized for multi-drive operation and also run with less vibration, which works well in multi-bay enclosures.

Update (2016-01-17)

WD Caviar Green also have a very annoying feature that tries to conserve power and it puts drives to a sleep mode after a few minutes of inactivity, without any way to configure the inactivity the timeout from the operating system. After entering the sleep mode, it takes up to 20 seconds to spin up drives and all this time any application that tries to use any files on the drives, such as an image editor, a photo viewer, Windows Explorer, etc, becomes unresponsive.

Interface

TR4UTBPN and MS4UM+(B) support USB v3 and eSATA interfaces. My computer only has USB v2 ports, which can be easily saturated with storage traffic, and two eSATA ports, so I opted for eSATA.

While trying to create a storage pool, I ran into a very annoying problem when Windows reported a cryptic error and failed to create a pool. After days of investigating, I finally figured out that the built-in SATA controller, which used JMicron's JMX36 chipset, misreported all hard drives in a single enclosure as having the same unique identifier. JMicron did not offer any updates that would fix this problem and I had to look for an alternative eSATA adapter. I picked up Vantec's non-RAID two-channel UGT-ST622 adapter, which supports port multipliers and it worked well for my configuration.

You can run this PowerShell command to check if your configuration is compatible with Storage Spaces:

Get-PhysicalDisk | ft FriendlyName,Size,UniqueId

If you see any duplicate values in the UniqueId column, then you cannot create a storage pool in this configuration.

Storage Spaces Configuration

Once you have all the hardware connected, configuring storage spaces is a very straightforward process. Go to Control Panel, select Storage Spaces and click Create a new pool and storage space link. Select the hard drives you allocated for the storage pool from the list and click the Create pool button.

You will need at least two drives for a two-way mirror and at least five drives for a three-way mirror. Configure the storage space as a two-way or a three-way mirror, change the default NTFS file system to ReFS and assign storage space a drive letter. Give the new storage space a meaningful name. This name will be used as the drive label in Windows. See below for storage space size considerations. Click Create storage space to finish.

The default storage pool name is not very descriptive and it's better to rename it, especially if you plan to use more than one storage pool. Click Rename pool link that will be visible after you created the pool and the first storage space and change the pool name to something meaningful.

You can allocate one storage pool for the entire household and then create a storage space for each person in the household, but then if you need to take your drives elsewhere, you will haul other people's data with you. Creating a pool per enclosure and per person is a better alternative, but it will require more hardware. Consider buying two-bay enclosures for smaller storage pools (e.g. Vantec's NexStar MX).

Note that Windows does not restrict you from configuring storage space size larger than the pool size, which is called thin provisioning. Once you use up about 70% of the storage available in the pool, Windows will shut down all storage spaces in this pool and will ask you to add more drives. You can still activate the pool as a read-only drive to access your data. I find it more convenient to set the storage space size to the exact value available in the storage pool or a smaller value. While I was moving my data from the old storage, I reached the 70% mark and Windows asked me to add more drives, but I was still able to copy more data into that storage space before adding two more drives. However, never fill up the storage pool because Windows will not be able to find spare storage if it finds any bad blocks in your data.

As a final note, do not change the drive label managed by Storage Spaces in Windows Explorer - it will ask you for administrative access and attempt to change the label, but then it will revert your changes. Use Storage Spaces to change the storage space name, which will also change the volume label.

Do I still need backups?

The simple answer is that any form of mirroring is not a replacement for periodic backups. Mirroring may protect you against disk and media failures, but it will not provide any protection against data corruption caused by hardware, software or user errors.

For example, if your image editor misbehaved and corrupted an image file, both mirrored copies of this file will be corrupted and if you don't have an earlier backup, the image will be gone forever. Similarly, if the storage controller fails in the way that corrupts data on the way to the disks, there will be no way to recover the original data.

If you have less than a terabyte of data or so, there is no excuse for not having a periodic backup running against your data. However, from the practical standpoint, the cost of backing up terabytes worth of data on a schedule would be prohibitive for most environments. Consequently, my only backup plan for the future is to find or write a utility to copy four- and five-star rated images to some backup storage.

Conclusion

While hardware RAID sounds like the big boys' game, RAID enclosures will eventually fail and you will have to go the extra mile to find replacement parts for your failed unit. Sometimes, it just will not be possible because either the hardware is too old or the manufacturer is out of business and you will have to invest into data recovery software or services. That is a lot of headache and sometimes a good chunk of money. As far as long-term storage is concerned, stay away from any hardware that you cannot replace overnight.

Software replication will not be as fast as a hardware solution, but it is not dependent on any specific hardware and if a JBOD enclosure fails in a non-destructive way, you will simply pick up a new enclosure, move your drives, turn it on and continue working with your pictures where you left it off on the day the old enclosure failed.

Update 2016-11-12

One annoying shortcoming of Storage Spaces is that it's impossible to safely eject the associated drive, which makes it very inconvenient for laptop users because one would need to shut down the computer before disconnecting the drives.

Comments:
Name:

Comment: