QNAP QTS Hero for ZFS on your NAS

QTS Hero – Bringing ZFS to your QNAP NAS

QNAP has revealed something quite special at their worldwide TechDay 2020 event that whether you are a new or old NAS user will be great news. In short, QNAP is releasing anew version of its QTS system and GUI, but this time rather than EXT4, they will be using the ZFS enterprise File System. There was alot to unpack at the event, but the short answer is that if you are using a quad-core modern Intel/AMD CPU equipped QNAP NAS, with at least 4/8GB of memory (factoring in deduplication), then in 2020, you will be able to make the move onto ZFS. But there is so much more to this new upgrade than just hoping onto ZFS. With enhancements to the QTS platform, as well as absorbing the RAID, caching and Integrity benefits of ZFS, this will certainly be great news for businesses that require the integrity of their data to be 100% clean all the way down to the bit. Who want multi-tiered backups, but without an impact of performance (i.e. Virtual Machines, Data Centers, Video Editing, Creative Workflows, etc). All the while wanting the best possible rad and write that can be achieved, without fear of invisible bottlenecks. In short, we are talking about a whole new kind of QNAP NAS system. Let’s go.

QNAP QTS Hero and SSD Support

Thanks to the improved support and functionality for SSDs, SSHDs and HDDs on the ZFS platform, QTS Hero has access to these abilities, leading to a better configured hybrid storage media system, more space being available, whilst still maintaining the speed and access times you need.As well as the vastly better data integrity measures in place within ZFS compared with EXT4 and BTRFS. Such as file self-healing in the event of data corruption, so it is spotted and remedied automatically long before it becomes a problem. However, in order to understand how the HERO file system handles data differently in real-time (In-line), we need to discuss how data travels from the client machine and into the NAS.

ZIL / L2ARC provide improved caching performance

QTS Hero lets you safely and reliably use Memory/RAM as large scale Cache. Using L2ARC in conjunction with SSDs means that you will have fantastically high throughput, as well as working will with hybrid storage where data is moved around constantly internally. Not a new ideal and good in the short term, but flushes/empties constantly with power on/off. ZIL works with the Memory Cache, logging the data that is held there.

This helps with maintain the data record there and preventing corruption with that log. QTS has one-off memory/ram caching, but it cannot hold it safely or has the log record to ensure it is complete. Additionally, there are some high-end enterprise QNAP NAS (ES series) available with QES that support battery backup units (BBU) for long term memory caching options, but QTS Hero and the ZFS system using ZIL should allow very similar levels of memory caching performance, or as close as to make little difference in the real world.

Data Write Coalescing

When data is written to the NAS (via the memory caching mentioned), the system will write this data in 256kb blocks. This is because SSD blocks are always in multiples of 256 – meaning data will always fit neatly onto the available chips.

The benefit of this is that the maximum block space is used, resulting in better performance, availability and survival of the SSD NAND – with the potential increase of 3x Maximum IOPS. For industries that thrive on big data, fast data and multiple usages of data at one time, these varying methods of data handling in the cache before drive writing will be enormously beneficial.

Inline Compression and Inline Deduplication of data

Data is de-duplicated BEFORE it is written to the drives, which leads to improved performance. As well as it happening before caching, leading to less super-fast cache space being utilized. Recommended Memory of 1GB of Memory to compliment every 1TB of storage in deduplication, as file reading for multiple sources will rely HEAVILY on the metadata – which is held in the Memory/RAM.

How Inline Data Deduplication and Compression work in practice

Data written to the NAS, go straight to the Memory/RAM, when it reaches it, it gets compressed, deduplicated (where appropriate) and compact the data – THEN it is written to the drive. The result is both a smaller storage footprint on your main storage RAID + less spacing in the data to speed up access.

Why is Data Deduplication and Compression Important

Typically, the understanding has always been that you cannot read/write your data faster than the speed of your drive within the RAID. Ie typically

Typical Enterprise HDD = 200MB/s performance

10x Enterprise HDDs = 1000MB/s (in an ideal situation)

Data Deduplicated/Compressed in the memory first = FASTER send/receive, then the compacted/reduced data goes to the HDDs

LZ4 Compression (Best in the market) for vastly improved compressed data performance. Additionally, inline Data Compression and DeDuplication Can be enabled/Disabled easily and at any time, as different NAS users will see the benefits of one/both at different times or when Memory/RAM is abundant or limited.

ZFS and QTS Hero RAID Types

As well as traditional RAID Types like RAID 0, 1, RAID 10, 50 and 60, etc QTS Hero supports the following:

RAIDZ1 (aka RAID 5)

RAIDZ2 (aka RAID 6)

RAIDZ3 (aka RAID 7)

That third one is the most important, as it provides a 3 disk parity system (3 disk safety net). It will impact your performance in the same wasy RAID 5/6 or RAIDZ1/2 would, but because of the read/write nature of QTS Hero, it is comparable to the latter.

Greater Numbers and Shared Snapshots in QTS Hero

Snapshots are handled similarly to QTS, but in far, far greater number (65,000). ZFS native snapshots within QTS Hero allow this volume of snapshots, made with far lesser impact on the system resources in general, thanks to the writing methods of this file system.

QTS Hero and Write once Read many (WORM) support

To avoid the deletion of accidental permanent modification of saved data within the NAS, QTS Hero supports WORM and it allows data to be read, but impossible to delete remotely and can only be changed locally by taking the storage pool offline. Along with this as a feature for data safety, it can be hugely beneficial to business users who need data to be ‘as of the time’ unchanged for business compliance, proof of submission or long term legality.

BitRot protection in QTS Hero

Protection from data corruption due to poor write conditions, media degradation and data compression is something that modern business needs to be massively aware of. There are already systems in place, from ECC memory and SMART testing of drives, to existing file system check in EXT4 and QTS features bad-bit checks that can be scheduled or actioned manually as needed, as this can affect overall system performance. QTS Hero and ZFS take this a massive step forward with real-time integrity tests at ALL TIMES of data being written/read, making corrections immediately and automatically. This helps prevent silent silent data corruption.

This is done via a method of a checksum being created for each block of data, making comparisons between the sum (content) of the block and the checksum at the start and finish of data operation. This works between drives in a RAID, ensuring the bad block is not duplicated in the parity of a RAID. This means that the file check system typically used in EXT4 is unnecessary (the operation that checks that data integrity is periodically checked at boot) and not performed.

Faster RAID rebuilding thanks to ZFS and QTS Hero

Thanks to ZFS and Resilvering, the RAID rebuild time is drastically reduced. In the event of a drive failure, the only part of the drive that needs to be rebuilt is the part that held the data (rather than a full bit by bit (or block by block) recreation including empty space. This potentially rebuilds the drive in minutes vs hours.

Even if a RAID drive that is being replaced needs FULL capacity to be rebuilt, it will still be faster than a traditional EXT4 RAID rebuild, thank to the block sizes and write operations that have been used.

QNAP QTS and ULINK for Drive Failure Prediction and Data Loss Prevention

AI-Powered and software-based drive failure prediction service. Your individual drives, internal system performance, media health and drive hardware are monitored within the system and there is an app/GUI that shows you realtime statistics about the drives. This is cross-referenced against a huge archive of healthy/unhealthy drive history and if a drive shows historical evidence/indicators, you will be able to migrate your data form the warned drive to a new drive (or Hot Spare) in the background and completely eliminate RAID Rebuilding later down the line if that original drive failed. This feature will be available in both QTS and QTS Hero.

Improved Windows ACL Permissions

With a greater degree of permissions, rules and access for Windows ACL than any other NAS brand in their own software system, allowing a hugely customizable and unique windows control setup

Which NAS Models will Support QNAP QTS Hero?

In order to run QTS Hero to it’s fullest potential, you will need to look at NAS that featuring high-end Intel CPUs and Memory capacities upto and exceeding 64GB. In actual fact, the minimum requirements are just 4GB of memory (with deduplication requiring 8GB minimum), but you would hit an instant bottleneck.The QNAP TS-x77XU series has been highlighted, along with the TS-x83XU range. The CPU needs some real power to assist the memory cache.

You will need at least a Xeon or Ryzen based CPU for the very best QTS Hero performance, but QNAP state that that the following NAS will support the upgrade from QTS to QTS Hero at launch:

• TS-x53B/Be/BT3/BU (4-8GB Upgrade needed)
• TS-x77X
• TVS-x82
• TVS-x82T3
• TVS-x72XT
• TVS-x72N
• TS-x83XU
• TVS-x71
• TVS-x71T
• TS-x85

Other NAS devices will be added and confirmed closer to the launch of the platform, but be sure to have enough memory to support the cache operations (with QNAP confirming 1GB of memory to compliment ever 1TB of storage, as well as 8GB minimum for deduplication). Finally, it is worth remembering that you will need to fully reinitialise the NAS to move over to this new file system, so you will need to ensure your NAS Data is backed up elsewhere before upgrading safely. Otherwise in a fresh installation, at the choice will be available at setup to choose between QTS and QTS Hero for your chosen file system.

What about QES? Compared with QTS Hero

QES, though ZFS and Business based, loses a lot of the software applications and features of QTS, so QTS Hero in 2020 will replace QES on these enterprise models via a firmware update. This will not effect files, shares, setups or setting. It will act as a system-wide update and providing the QTS/Hero functionality to existing users of the QES Enterprise platforms. It will arrive in the form of a upgrade pass and will even allow QTS users on NAS such as the 77XU series to move over to QTS Hero with ZFS.

📧 SUBSCRIBE TO OUR NEWSLETTER 🔔

🔒 Join Inner Circle

This description contains links to Amazon. These links will take you to some of the products mentioned in today's content. As an Amazon Associate, I earn from qualifying purchases. Visit the NASCompares Deal Finder to find the best place to buy this device in your region, based on Service, Support and Reputation - Just Search for your NAS Drive in the Box Below

Need Advice on Data Storage from an Expert?

Finally, for free advice about your setup, just leave a message in the comments below here at NASCompares.com and we will get back to you. Need Help? Where possible (and where appropriate) please provide as much information about your requirements, as then I can arrange the best answer and solution to your needs. Do not worry about your e-mail address being required, it will NOT be used in a mailing list and will NOT be used in any way other than to respond to your enquiry. TRY CHAT Terms and Conditions   Alternatively, why not ask me on the ASK NASCompares forum, by clicking the button below. This is a community hub that serves as a place that I can answer your question, chew the fat, share new release information and even get corrections posted. I will always get around to answering ALL queries, but as a one-man operation, I cannot promise speed! So by sharing your query in the ASK NASCompares section below, you can get a better range of solutions and suggestions, alongside my own.

☕ WE LOVE COFFEE ☕

  This description contains links to Amazon. These links will take you to some of the products mentioned in today’s video. As an Amazon Associate, I earn from qualifying purchases