Rethinking Storage and Networking in Next Generation Racks

A standard data center rack today is composed of shared-nothing commodity servers. Recent data center hardware trends suggest a shift in the rack design, enabling high density “rack-scale computers” optimized and managed at rack-scale and targeted at data center workloads.

In this talk, I will describe two projects that illustrate the potential and challenges of this approach. First, Pelican is a disk-based rack-scale cold data store. It has a converged rack-scale design in which resources, such as power and cooling are provisioned just for the requirements of the workload. Thanks to that, Pelican has a low total cost of ownership comparable to tape and has a much lower latency than tape. Achieving that requires a novel storage stack that manages complex resource constraints at rack scale. Second, XFabric is a rack-scale network in which the physical network topology can be dynamically reconfigured. XFabric is a packet-switched network that operates over a physical circuit-switched topology. XFabric is managed by a rack-scale controller that optimizes the in-rack topology for the traffic demand at runtime. XFabric uses low-cost electrical circuit switches, supports about 300 servers per rack and significantly outperforms static topologies.