Disco Lab - Rutgers University

Overview	FedFS	Status	Software
People	Publications	Related Links	Funding

Overview

Our research focuses on the design and development of next-generation file systems that take advantage of emerging Remote Direct Memory Access (RDMA) interconnect technologies such as Virtual Interface Architecture (VIA), Infiniband and Myrinet.

Using these RDMA-based interconnects, our goals are two-fold:

Build Direct Access File Systems (DAFS) that provide low overhead access to network attached storage. The DAFS protocol is a new file-access protocol designed to take advantage of RDMA interconnect technologies. We study the feasibility of building direct access file systems by building prototypes using various RDMA interconnects including Infiniband, VIA and Myrinet.

Design and implement an architecture for a scalable federation of storage distributed across a cluster, that results in improved performance and availability of server applications in data center environments. We call this architecture Federated File Systems (FedFS). The most important application of FedFS is to aggregate servers implementing standard file system protocols like NFS and DAFS.

Federated File System over an RDMA interconnect

Federated DAFS is an architecture in which FedFS is used as a low-overhead clustering layer to build a scalable federation of DAFS servers.

Top

FedFS

There has been an increasing demand for better performance and availability in storage systems. In addition, as the amount of available storage becomes larger, and the access pattern more dynamic and diverse, the maintenance properties of the storage system have become as important as performance and availability. We define a loose clustering of the local file systems of the cluster nodes as an ad-hoc global file space to be used by a distributed application. We call this distributed file system architecture, a Federated File System (FedFS). FedFS provides improved availability and performance by supporting dynamic reconfiguration, file migration for load redistribution, and replication for failover. The goal of the FedFS project is to design and implement an architecture for scalable federation of storage distributed across a cluster with the following features:

Location independent global file naming while preserving the autonomy of the local file systems. FedFS provides a per-application global file naming facility that applications can use to access files across the cluster in a location-independent manner


                      Location independent file naming using Virtual Directories
Dynamic load balancing by combining caching, request forwarding and file migration

                                         Load balancing using file migration
Consistent file replication for fault tolerance
Dynamic reconfiguration of cluster supporting join/leave of nodes at run time with minimum disruption of service performance

FedFS provides all these features on top of autonomous local file systems. Autonomy means that local file systems are not changed in order to participate in a federation, and no FedFS specific metadata is stored in the local file systems. To achieve this, FedFS I/O operations translate into local file system operations and the global file space metadata becomes soft state that can be stored in volatile memory of the cluster nodes. As a result, a local file system can join or leave a federation anytime, without requiring any preparation, and without carrying out persistent global state operations. In this way, local file systems can simultaneously operate in a standalone fashion, and as part of one or more federations.

Top

Status

DAFS Prototype We have developed an implementation of the DAFS protocol in user-space, that includes support for most of the session management operations specified in the DAFS API. Our prototype includes support for various transports including Infiniband, VIA and Myrinet.FedFS Prototype We have developed a user-level FedFS prototype that supports Infiniband (IB) and Virtual Interface Architecture (VIA), and work is underway to add support for Myrinet. The FedFS prototype currently supports location independent global naming and a file migration mechanism for load redistribution. We are working on enhancing our prototype to support dynamic reconfiguration of the cluster.Our prototypes have been developed and evaluated on a cluster of PCs running Linux.

Top

Software

Coming Soon!

	Source code for our user-level DAFS prototype
	Source code for the FedFS prototype to aggregate file systems on cluster nodes

The prototypes will include support for

	Infiniband (uDAPL, VAPI and IBAL)
	Myrinet (GM)
	Virtual Interface Architecture (VIA)

Top

Funding

NSF - CAREER grant (CCR 0133366) for Prof. Iftode to investigate the impact of new I/O technologies on the operating system architecture of network servers.

New cluster(front and back) of eight Dell Poweredge 2600 machines (Dual Xeon 2.4GHz CPUs, 36 GB 15K RPM SCSI and 1 GB DDR SDRAM)

Mellanox Technologies - Research partnership and Hardware support

Faculty

Graduate Students

Alumni

Suresh Gopalakrishnan

Top

Publications

Papers

	Building a User-level Direct Access File System over Infiniband Murali Rangarajan and Liviu Iftode, Accepted for publication at the 3rd Workshop on Novel Uses of System Area Networks (SAN-3), February 2004.
	Federated DAFS: Scalable Cluster-based Direct Access File Servers Murali Rangarajan, Suresh Gopalakrishnan, Ashok Arumugam, Rabita Sarker and Liviu Iftode, Proceedings of the 2nd Workshop on Novel Uses of System Area Networks (SAN-2), February 8, 2003. Rutgers University Technical Report, DCS-TR-516
	Federated File Systems for Clusters with Remote Memory Communication Suresh Gopalakrishnan, Ashok Arumugam, Liviu Iftode. Rutgers University Technical Report DCS-TR-472, December 2001

Presentations

	Federated DAFS: Scalable Cluster-based Direct Access File Servers Presented at the SAN-2 Workshop. February 8, 2003. Also presented as a Work in Progress at FAST 2003
	Federated File Systems for Clusters with Remote Memory Communication Presented as a Work in Progress at FAST 2002

Patents

Federated File Systems over Internet - Submitted for a patent.

Top