The original goal of Remote Direct Memory Access (RDMA) technology
was to enable efficient user-level communication without involving the
OS. With performance critical applications increasingly being deployed
in the OS kernel, RDMA is increasingly being used not only for
user-level but also for in-kernel applications.
Unlike user-level applications, which perform a limited set of
independent tasks, the OS kernel is a large program with a single
resource pool. To enable efficient RDMA inside the kernel requires
unified management of resources like communication endpoints, buffers
used for data transfers, and the kernel virtual address space. Our goal
is to develop an RDMA Service
Platform (RSP), which manages such resources without
performance overheads. In RSP, we provide resource management and a
service for remote address descriptor exchange. We developed two
network storage protocols over RSP: NFS, a file based network attached
storage (NAS) protocol, and iSCSI, a block based storage area network
(SAN) protocol.
RDMA Service Platform is the basic building block of RDMA based
kernel services. It provides buffer and endpoint management, flow
control, and a service for remote memory descriptor exchange. Using
RSP, global resource management decisions are possible and multiple
kernel services can be built without overallocation or
underutilization of allocated resources. RSP defines a generic API for
in-kernel services without imposing significant performance
overhead. It also defines a generic external service for remote memory
descriptor exchange essential for RDMA communication. The goal of our
research in RSP is to study the effectiveness of multiple resource
management mechanisms and policies when used by multiple kernel based
services.
[Top]
NFS is arguably the most prevalent NAS protocol. We are currently
designing and building an in-kernel RDMA based NFS service (client and
server). Our study of this architecture will begin with a thorough
evaluation of the performance characteristics of NFS over RDMA (with a
comparison to standard NFS over TCP/IP). We will also evaluate the
programmability of NAS services over our RDMA Service Platform (see
"RSP" above), using NFS over RSP as a case
study.
[Top]
SCSI over IP (iSCSI) is a popular SAN protocol. We are currently
extending iSCSI to incorporate support for RDMA based transports. Our
implementation of iSCSI over RDMA is built on top of RSP. The goal of
our research is to evaluate the benefits of RDMA and zero-copy
transports as well as design novel caching strategies to improve the
performance of the iSCSI protocol over RDMA.
[Top]
[Top]
[Top]
[Top]
[Top]
