What is exFAT? And why is it so important for removable storage?

One of the most interesting new features in Macrium Reflect 8 is exFAT support. If you’re not an operating system enthusiast, that might not mean much to you — however, it’s significant because it gives Reflect the ability to backup removable storage devices.

In this post we’ll explore the relationship between the ExFAT filesystem and removable storage. It’s a story about a popular, if historically flawed hardware format, and how one innovation can enable others.

FAT and NTFS

Before we go further, it’s worth going back in time to understand the prehistory of exFAT and how it came about. To do that, we need to go back to the FAT file system.

The FAT filesystem and removable storage

The FAT file system first emerged on the scene in 1977. It gained traction when it was adopted as the filesystem for MS-DOS and Windows operating systems. However, it was later replaced by NTFS. NTFS was first introduced in 1993, and over the following decade its superiority over FAT led it to become widely adopted in Microsoft Windows operating systems, including Windows XP in 2003.

NTFS was better suited to larger disks and faster access. It was also more reliable and secure thanks to the use of journaling and ACLs.

However, NTFS never killed FAT. This is because FAT has two key advantages that make it suited to a number of specific uses. Its simplicity means that it can be easily supported by embedded systems such as a digital camera and bootstrap firmware which can be used to boot fully featured operating systems, while the fact it is supported by all operating systems gives it a level of flexibility and compatibility that you don’t get with NTFS.

It’s for these reasons that FAT became the default choice for memory cards and USB sticks.

Microsoft introduces exFAT

However, using FAT in removable media started to show its age. This is because the market evolved, with removable media exceeding FATs limitations of 32GB storage with a 4GB file size limit. To respond to this, Microsoft introduced exFAT in 2006.

exFAT shares some characteristics with FAT to enable continued simplicity of support, but it does not contain any limits on storage and file sizes. The fact that it also includes richer metadata and optimizations to better operate on removable media made it an obvious choice for the evolution of the removable storage market.

This isn’t to say adoption was swift; initial acceptance was, in fact, slow. It took the filesystem documentation to be released to the public, and the acknowledgement of the lack of any viable alternative for it to be the default choice for removable media at a range of sizes.

exFAT and the removable media market

The introduction of exFAT has made the removable media market more stable than it was in the past. For a time it was a bit of a wild west.

For example, flash media was often preformatted without any partition table. This meant the operating system had no choice but to guess the filesystem type. Sometimes, the first few sectors were read-only.

Furthermore, counterfeits were not unusual. You’d see flash storage sold with a much larger advertised capacity than existed. Unfortunately, you’d only realise when Windows tried to position a file outside the extent of the ‘real’ storage.

However, such problems are rare these days. This meant it was the right time to add exFAT support to Macrium Reflect — indeed, some might say it’s long overdue!

Adding exFAT and removable device support to Macrium Reflect

While we’d like to say we spent ages working to add exFAT support to Macrium Reflect 8, there is also a more practical reason it features in the new version. To put it simply, because we had already done the work to add ReFS support (which you can read about here), adding in exFAT support was straightforward. While adding ReFS took considerable effort, it effectively laid the foundations that made it easy to do the work required for exFAT.

Further work was required to support removable devices; Windows treats removable devices somewhat differently to standard storage devices mostly to accommodate the likelihood of the device being removed without notice. This is one of the reasons why even FAT formatted devices haven’t been supported in existing versions of Reflect.

We think exFAT and removable device support is an incredibly valuable addition to Macrium Reflect’s features. We know many users depend on removable storage to store data; moreover, there’s also a growing new use case thanks to the growth of ARM microcomputers such as Raspberry Pi. Ultimately, having the ability to protect that data with reliable and consistent backups is absolutely essential.