Next gen file systems for Open Source

BANGALORE, INDIA: Open Source treats everything as either a file or a directory. Even hardware is considered a file and is kept in a directory. Therefore, a file system is an organization of data and metadata on a storage device and is expected to provide quick transfer and storage of data without corrupting it.

Linux file system interface is implemented as a layered architecture, separating the user interface layer from the file system implementation from the drivers that control the storage devices. To begin with Linux file systems are expected to perform your day to day tasks with some of the latest file systems. Some of the key next gen file systems for open source are discussed below.

BTRFS

BTRFS is also known as B-tree file system and is a popular next gen file system for Linux, available with a GPL license. It is developed by Oracle in association with contributors from the Linux community. BTRFS provides a number of features that make it a very attractive file system solution for local storage.

It is designed for large files and file system and helps in easy administration, integrated raid and volume engagement. It also detects and fixes data and files system corruption, improves backup operations, easy searching for files and allows quick rollback of software and OS upgrades, improves storage capacity.

BTRFS is intended to address the lack of pooling, snapshots, checksums and integral multi-device spanning in Linux file systems as the use of Linux scales upward into larger storage configurations common in the enterprise. It is structured as several layers of trees, all using the same B-tree implementation to store their various data types as generic items sorted on a 136-bit key. The first 64 bits of the key are a unique object ID.

The middle 8 bits is an item type field; its use is hardwired into code as an item filter in tree lookups. Objects can have multiple items of multiple types. The remaining right-hand 64 bits are used in type-specific ways.

{#PageBreak#}

ZFS

ZFS is the feature rich file system developed by Sun for its UNIX version, Solaris. ZFS allows quick and easy snapshots of data, data check summing, and integration of several tools to manage disks and file systems. It is based upon a copy-on-write design that writes a new copy of the data every time it changes.

Once the new version of the data is written the old version is marked as deleted and the space can be reclaimed. To implement a snapshot system you need to instruct the OS to not mark the old data as deleted and changes are preserved.

All data that is written to a ZFS file system is check summed to ensure its validity. Hard drive corrupting data has always been an issue but due exponential growth in storage requirements data corruption has become a common phenomenon. To help mitigate the risk of silent data corruption ZFS stores a checksum of all the data it stores and validates the data again before relaying it onto the operating system. If one copy of the data has been corrupted it is identified on read and seamlessly copied from another source.

NILFS-2

NILFS-2 is a reprisal of a log-structured file system developed by Nippon Telegraph and Telephone. The first version of NILFS appeared in 2005 but lacked any form of trash collection. In mid-2007, version 2 was first released, which included a trash collector and the ability to create and maintain multiple snapshots. The NILFS-2 file system entered the mainline kernel and can be enabled simply by installing its loadable module.

An interesting aspect of NILFS-2 is its technique of continuous snap-shooting. As NILFS is log structured, new data is written to the head of the log while old data still exists. Because the old data is there, you can step back in time to inspect epochs of the file system. These epochs are called checkpoints in NILFS-2 and are an integral part of the file system. NILFS-2 creates these checkpoints as changes are made. It is one of the many file systems that incorporate snapshot behaviour. Other file systems that include snapshots are ZFS, LFS etc.

Click here to continue reading!