SC|06 Powerful Beyond Imagination
SC06 is the International Conference for High Performance Computing Networking and Storage

About Registration Conference Technical Program Exhibits News and Press Travel

Home Conference HPC Competitions HPC Storage Challenge






HPC Storage Challenge

Submissions Now Open
April 11, 2006

Questions

Final Project Report Due
November 14, 2006

The SC06 HPC Storage Challenge is now in its final stage. Four Project Proposals have been selected as finalists and they now have until the start of the conference to complete their projects. The projects fall into 2 categories with 2 projects in each category. The categories are:

  • Large System
  • Small System

Finalists will present reports describing their project to the conference in a technical session as outlined below. One winner in each category will be announced during the awards session of the conference.

Please come join the excitement and hear what the finalists have done as they present their work during the HPC Storage Challenge session!

Tuesday, Nov 14

Time Session Event Chair/Speaker Location
10:30AM - 10:50AM Storage Challenge Trading Memory for Disk: Using Parallel Access to Fast InfiniBand Disk Arrays for Large Computational Chemistry Applications Troy Benjegerdes, Brett Bode, Kyle Schochenmaier 24-25
10:50AM - 11:10AM Storage Challenge Scaling NFS through RDMA for Cluster Computing Dov Cohen, Jeff Decker, Noah Fischer, Helen Y. Chen, Jackie H. Chen 24-25
11:10AM - 11:30AM Storage Challenge HUSt: A Heterogeneous Unified Storage System for GIS Grid Lingfang Zeng, Ke Zhou, Zhan Shi, Dan Feng, Fang Wang, Changsheng Xie, Zhitang Li, Zhanwu Yu, Jianya Gong, Qiang Cao, Zhongying Niu, Lingjun Qin, Qun Liu, Yao Li, Hong Jiang 24-25
11:30AM - 11:50AM Storage Challenge High Performance Data Analysis for Particle Physics using the Gfarm File System Osamu Tatebe, Nobuhiko Katayama, Mitsuhisa Sato, Taisuke Boku, Akira Ukawa, Shohei Nishida, Ichiro Adachi 24-25



HPC Storage Challenge Description
HPC systems are comprised of 3 major subsystems: processing, networking and storage. In different applications, any one of these subsystems can limit the overall system performance. The HPC Storage Challenge is a competition showcasing effective approaches using the storage subsystem, which is often the limiting system, with actual applications. Participants must describe their implementations and present measurements of performance, scalability, and storage subsystem utilization. Judging will be based on these measurements as well as innovation and effectiveness; maximum size and peak performance are not the sole criteria. Finalists will be chosen on the basis of submissions which are in the form of a proposal; submissions are encouraged to include reports of work in progress. Finalists will present their completed results in a technical session during the conference from which the winners will be selected. Participants with access to either large or small HPC systems are encouraged to enter this challenge.

Click here to make submissions

Questions

ADDITIONAL INFORMATION

Participating in the HPC Storage Challenge
Teams participating in the SC06 HPC Storage Challenge will begin work at their earliest convenience and must submit a Project Proposal by July 31, 2006; n.b., the Project Proposal is the Submission. Teams are requested, but not required, to submit a Notice of Intent by May 22, 2006. The Project Proposals will be used to select a small number of finalist teams who will be notified of their status by August 31, 2006. Finalist teams may continue working on their challenge projects after submitting their Project Proposal until the Final Project Report is due (November 14, 2006). During the conference, finalist teams will make a "challenge finalist presentation" in a technical session that will be part of the Storage Challenge judging. They will also participate in a private follow up interview with the judges. Titles, abstracts and team member names will be posted in the Conference Program and the winners will be announced at the conference awards session.

HPC Storage Challenge Resources
Teams participating in the SC06 HPC Storage Challenge will work on their challenge projects in advance of the conference with resources they can obtain outside of the conference. Due to space limitations, the conference can not supply an aggregation of storage as it provided for the StorCloud challenge in previous years.

Award Categories
As the HPC community matures, a large variance in the size of HPC systems is emerging. In recognition of this fact, two award categories are recognized; the Large Systems award and the Small Systems award. Size is a relative term and difficult to define in isolation without being arbitrary. Within the context of the SC06 HPC Storage Challenge, the definition of size is based on the size of the entire system (i.e., number of nodes * number of processors per node, size of the network, amount of storage, bandwidth potential for the storage, etc.), not just the storage system. It will be left to the team to declare whether they wish to compete for the Small System award or the Large System award, but the judges reserve the right to reclassify submissions if in their judgment it is warranted. A simple example follows to illustrate what might be considered small and large systems.

  • Small System
    • 64 Xeon nodes, GbE network, 1 storage controller, 5 TB of disk
    • two 64 CPU SMP systems with 1 storage controller, 10 TB of disk
  • Large System
    • 1024 Opteron blade system with several storage controllers and 50 TB of disk
    • ten 64 CPU SMP system with several storage controllers and 200 TB of disk

Finalists and Winners
Finalists will be notified by August 31, 2006 that they have been selected. Their sponsoring institutions may use this information for publicity as they see fit. Each finalist team will be expected to make a Challenge Finalist Presentation in a technical session during the conference. The Conference Program will include a 150 word abstract and team member names along with their sponsoring institutions. The winners will be announced in the conference awards session. Finalists and winners will receive certificates for their accomplishments in this challenge.

Reports
Two principal reports must be submitted as part of the SC06 HPC Storage Challenge. The first one is due with the Project Proposal submission and the second one is due with the Final Project Report submission. An extended abstract is due with the Notice of Intent.

Notice of Intent To Participate
Teams participating in the SC06 HPC Storage Challenge are asked to complete a Notice of Intent form posted on the SC06 conference submission website before May 22nd. An extended abstract in the form of a PDF file with at most 4 pages is to be included as part of this submission. The purpose of the Notice of Intent is to provide the Challenge Committee guidance on the number of expected participants in this year's challenge. Submission of an Notice of Intent is not required, nor binding. it is merely an indication of a "serious" intention to participate in the Storage Challenge. A team's actual participation in the challenge may deviate materially from what was stated in their Notice of Intent. Failure to submit a Notice of Intent will not prevent a team from participating in the challenge.

Project Proposal
Teams participating in the SC06 HPC Storage Challenge are required to submit a Project Proposal by July 31, 2006 in the submission website which will be used by the judges to select finalists. Teams are encouraged to include results from proof of concept tests or works in progress to lend credibility to assertions made in their Project Proposal. A report in the form of a PDF file with at most 12 pages is to be included as part of the submission. A suggested format for this report follows
  • Title
  • Team Information
  • Abstract
  • Problem Statement (including hypothesis)
  • Hardware Configuration
  • Data and Storage Layout
  • Software Used (including application and systems software)
  • Description of the solution to the problem
  • Experiments to be done, and measurements and quantitative results to be collected
    • Include results from proof of concept tests or works in progress
  • Claims (how this project satisfies judging criteria)
    • Overall Performance
    • System and/or Application Scalability
    • Storage Resource Utilization
    • Innovation
    • Effectiveness
  • Acknowledgements
    • List of individuals or agencies not listed in the team information who made significant contributions to, or supported the work being reported
Final Project Report
Finalist teams in the SC06 HPC Storage Challenge are required to submit a Final Project Report by November 14, 2006 in the submission website which will be used by the judges to select the winner in each category. A report in the form of a PDF file with at most 12 pages is to be included as part of the submission. It must also include slides for the challenge finalist presentation in an appendix. Other appendices may also be included, but it is not guaranteed that they will be reviewed by the judges. The appendices are not included in the 12 page upper limit. A suggested format for this report follows
  • Title
  • Team Information
  • Abstract
  • Problem Statement (including hypothesis)
  • Hardware Configuration
  • Data and Storage Layout
  • Software Used (including application and systems software)
  • Description of the solution to the problem
  • Experiments, measurements and quantitative results of solution
    • Describe actual experiments and results
    • Describe how the actual measurements were made
    • Explain how they support the hypothesis
  • Describe conclusions based on the experiments
  • Describe how the judging criteria apply to the project
    • Overall Performance
    • System and/or Application Scalability
    • Storage Resource Utilization
    • Innovation
    • Effectiveness
  • Describe any deviations from the Project Proposal
    • Deviations from the Project Proposal as part of the exploratory process are anticipated, but the work must be in the same spirit as proposed
  • Price/performance information (see explanation below)
  • Appendices
    • Slides for challenge finalist presentation (required)
    • Other optional appendices
    • Acknowledgements: list of individuals or agencies not listed in the team information who made significant contributions to, or supported the work being reported
Interviewing of Finalist Teams
Finalist teams in the SC06 HPC Storage Challenge are to participate in a private interview session with the Challenge judges. It is tentatively scheduled the morning before the awards session (November 16, 2006). The purpose of this interview is provide judges an opportunity to ask questions and seek clarifications of that might arise from the Storage Challenge Finalist Presentations and/or the Final Project Report.

Judging
Project Proposal submissions and Final Project Report submissions to the SC06 HPC Storage Challenge will be evaluated by the judges based on a mixture of quantitative and qualitative criteria. The judges decisions are final.

Judging will be based on 3 broad categories: performance and scalability, effective use of storage and storage related resources, and innovation. The following list presents examples of things to be considered within each of these categories. These lists are neither exclusive nor exhaustive; it will be up to each team select the appropriate measurements and analyses to make the case for their project.
  1. Performance and Scalability
    1. File size, application I/O request, file system block size
    2. Application measurements (select appropriate measurement)
      1. I/O rate in MB/s (both as an aggregate and harmonic mean)
      2. IOPs (both as an aggregate and harmonic mean)
      3. File creations per second
      4. Bandwidth per kilometer
    3. System measurements
      1. Bandwidth tools (e.g., iostat, netstat)
      2. Latency tools (e.g., histograms of record size vs. time)
    4. Storage system or vendor tools
    5. Scalability; ability to expand solution to increasing numbers of
      1. Disks and disk controllers
      2. Nodes
    6. Ensure that the magnitude of I/O operations is large enough to actually use disk
  2. Effective use of resources
    1. Measure storage and storage related resource utilization
      1. disk/disk controllers, SAN, LAN
      2. percent of peak BW per disk controller
      3. percent of SAN/LAN utilization
      4. assumes the application is storage I/O intensive
    2. Requires "compelling" baseline measurements
  3. Innovation
    1. Novel ways to solve traditional and/or difficult problems
    2. Effective use of new technologies
    3. Well designed application
    4. Best practices
    5. The "Wow, that's COOL!" factor
Price/Performance Information Is Required But Will Not Be Used In Judging
The HPC Storage Challenge Committee is considering using price/performance as a judgment criteria for SC07 but is divided on how best to assess this or even if a consistent enough measurement can be devised in order to meaningfully evaluate this. In order to help the committee resolve this issue, SC06 HPC Storage Challenge submissions are being asked to in include a short section assessing the price/performance characteristics of their submission in the Final Project Report. This will not be included as part of the judging criteria during SC06, but is being included on an informational basis only. The price/performance analysis needs to be based as far as possible on the complete system (e.g., hardware, software, power consumption) using list prices so as to generate a reasonably consistent basis for comparison.

Good luck to all teams and have fun!

Click here to make submissions

Questions



Home | About | Contact Us | Registration | Sitemap
IEEEComputer SocietyACM