SAN JOSE: With the inaugural Dell Enterprise Forum, the expanded and rebranded storage event being held here this week, the company that made its reputation in the commodity PC business, wants to provide its customers and partners a broader and deeper perspective on its portfolio of datacenter hardware, software and services. So it seemed appropriate to talk about Dell’s foray into high-performance computing and the University of Florida HiPerGator supercomputer.
Officially announced less than a month ago, the HiPerGator, together with a tenfold increase in the size of the university’s data pipeline, will be used to help researchers find life-saving drugs, make decades-long weather forecasts and improve armor for troops. The supercomputer has a peak speed of 150 trillion calculations per second, and is the result of collaboration between UF, Dell, Terascala, Mellanox and AMD. It features 16,384 AMD processing cores, a Dell/Terascala HPC storage solution (DT-HSS 4.5) with the industry’s fastest open-source parallel file system, and Mellanox’s FDR 56Gb/s InfiniBand interconnects that provide the highest bandwidth and lowest latency.
HiPerGator was purchased and assembled for $3.4 million, and the 25,000-square-foot UF Data Center built to house it cost $15 million. The university also announced that it is the first in the nation to fully implement the Internet2 Innovation Platform, a combination of new technologies and services that will further speed research computing. About 30 other universities are working to fulfill the requirements to use the Internet2 Innovation Platform, but UF is the first to achieve the three changes required in research computing architecture: a 100 Gbps connection to Internet2, a Science DMZ, and use of software-defined networking, or SDN.
“This is really a big fireworks result and conclusion of a decade-long project that started in 2004,” said UF’s Erik Deumens, Director of Research Computing. “We realized in that decade that it’s not just about high-performance computing in the traditional sense. There are a lot more people that really need high-performance computing but don’t think of it as high-performance computing.”
The university set out to create an environment where people could work together more efficiently, but it wasn’t until 2010 and a new CIO that everything came together in a five-year commitment to build a research computing environment that was matching the needs of UF, he said.
“We needed a good network infrastructure because research today is all about collaboration… in Florida, across the country and world… and then give them a competitive advantage.” A campus network was built in 2005, and the research network was upgraded from 20 to 200Gbps in March. It was expensive to upgrade all these closet-based clusters, which led to the creation of the datacenter which opened last month.
As for the computing platform, Deumens said the initial objective was to build an environment for research for everybody, engineers, physicists and people in the humanities, not a supercomputer. A number of people and projects are taking advantage of the HiPerGator, but he expects a lot more who are continuing to use more traditional tools to jump on.
The raw horsepower available is incredible, able to reduce work that took months down to hours. Deumens said one initiative to identify side effects of drugs for pharmaceutical companies involved the analysis of up to 140,000 possible compounds. Just testing one used to take 8 hours, and now all 140,000 can be done in 10-12 hours.
As for the answer to the million dollar question – $3.4 million in this case – why Dell, Deumens said his staff had extensive experience building a cost-effective computer platform. “We needed a system that has all these abilities of communications, has to have stability, and be manageable, and by the way doesn’t cost too much.” UF had been working with Dell on a number of smaller projects over the years, but for something to scale 10X, they worked with both Dell and Terascale to design HiPerGator.