The document summarizes research on developing a "storage fabric" for computational grids. It describes a network storage stack including the Logistical Backbone (L-Bone) for resource discovery, the Internet Backplane Protocol (IBP) for allocating and managing network storage, the exNode data structure, and the Logistical Runtime System (LoRS) for aggregation tools. The research aims to provide scalability, flexibility, fault-tolerance and composability through an approach modeled on the IP network stack.

1. The Storage Fabric of the Grid: The Network Storage Stack James S. Plank Director: Lo gistical C omputing and I nternetworking ( LoCI ) Laboratory Department of Computer Science University of Tennessee Cluster and Computational Grids for Scientific Computing: September 12, 2002, Le Chateau de Faverges de la Tour, France

2. Grid Research & The Fabric Layer Middleware Application Resources The “Fabric” Layer

5. LoCI’s Research Agenda Redefine the fabric layer based on End-to-End Principles Communication Storage Computation Data / Link / Physical Network Transport Application Access / Physical IBP Depot exNode LoRS Application Access / Physical IBP NFU exProc LoRS Application

11. The Network Storage Stack The L-bone : Resource discovery & proximity queries IBP (Internet Backplane Protocol) : Allocating and managing network storage The exNode : A data structure for aggregation LoRS: The Logistical Runtime System : Aggregation tools and methodologies

20. Typical IBP usage scenario

21. Logistical Networking Strategies Sender Receiver IBP Network Sender Receiver IBP Network Sender Receiver IBP IBP IBP #2 #1 #3 Sender Receiver IBP #4 IBP IBP

24. The Network Storage Stack The L-bone : Resource Discovery & Proximity queries IBP : Allocating and managing network storage (like a network malloc) The exNode : A data structure for aggregation LoRS: The Logistical Runtime System : Aggregation tools and methodologies

26. Snapshot: August, 2002 Approximately 1.6 TB of publicly accessible storage (Scaling to a petabyte someday…)

27. The Network Storage Stack The L-bone : Resource Discovery & Proximity queries IBP : Allocating and managing network storage (like a network malloc) The exNode : A data structure for aggregation LoRS: The Logistical Runtime System : Aggregation tools and methodologies

29. The exNode (XML-based) A B C 0 300 200 100 IBP Depots Network

30. The Network Storage Stack The L-bone : Resource Discovery & Proximity queries IBP : Allocating and managing network storage (like a network malloc) The exNode : A data structure for aggregation LoRS: The Logistical Runtime System : Aggregation tools and methodologies

33. Upload

34. Augment to Tennessee

35. Augment to Santa Barbara

36. Stat (ls)

37. Failures do happen.

38. Download

39. Trimming (dead capability removal)

41. Parity / Coding IBP Buffers Network = + + = + 2 + 3 ExNode with Coding

45. The Storage Fabric of the Grid: The Network Storage Stack James S. Plank Director: Lo gistical C omputing and I nternetworking ( LoCI ) Laboratory Department of Computer Science University of Tennessee Cluster and Computational Grids for Scientific Computing: September 12, 2002, Le Chateau de Faverges de la Tour, France

46. Replication: Experiment #1 UCSB UCSD TAMU UTK UNC Harvard Turin, IT Stuttgart, DE 3 MB file 0 3 MB UTK 2 UTK 5 UTK 6 UTK 3 UTK 4 UTK 1 UCSB 1 UCSB 2 UCSB 3 UCSD 1 UCSD 3 Harvard UNC UTK 5 UTK 2 UTK 5 UTK 6 UTK 3 UCSB 1 UCSB 2 UCSB 3

47. Replication: Experiment #1 Depot Availability at UTK Depot Availability at UCSD Depot Availability at Harvard 860 Download Attempts 100% Success 857 Download Attempts 100% Success 751 Download Attempts 100% Success 0 10 20 30 40 50 70 80 90 100 60 UTK 99.85 UCSD UCSB Harvard UNC 99.71 95.31 59.77 99.88 Fragment Availability (%) 0 10 20 30 40 50 70 80 90 100 60 UTK 96.27 UCSD UCSB Harvard UNC 98.60 88. 60 57.29 97.20 Fragment Availability (%) 0 10 20 30 40 50 70 80 90 100 60 UTK 99.87 UCSD UCSB Harvard UNC 99.80 90.47 57.45 99.87 Fragment Availability (%)

48. Most Frequent Download Path From UTK From Harvard From UCSD