stevejones at stanford.edu
Thu Apr 6 13:35:31 PDT 2006
We're heavy Panasas users and keep buying shelves. I've done benchmarking,
sequential using bonnie++ & IOR, and parallel using IOR (ASCI Purple
Benchmarks) extensively. We're now using a few shelve for fourier
transforms and post-processing of data sets generated from a 65,000
processor run that have snapshots approx. 440GB each and have great file
Here's a slide desk on panasas integration:
Panasas is a snap to set up, easy to manage, and I've done just about
everything to lose data (hosed networking, failed director blades on single
shelf config's, etc) on one shelf and have still recovered every time within
I'm not sure of the issues Joe faced, but I'd sure like to hear about them.
My two cents...
Quoting "Joseph L. Kaiser" <jlkaiser at fnal.gov>:
> I work at a large national lab in the midwest. We have had unpleasant
> (i.e. bad) experiences with Panasas and Ibrix. We have not tried GFS
> because it doesn't really look like it scales. If you really want to
> have a manageable name space with good throughput then do it with
> hardware. Netapp, Bluearc, EMC. Those guys. Pay the money. You'll be
> Although, how much risk can your enterprise stand? You can try Lustre
> though it still looks like it's fearsome to make work. PVFS2? Not
> quite ready for prime time but some on this list have installed it. In
> production anyone?
> Debbie Tropiano wrote:
> >Hello again -
> >As we try to slog thru our requirements, it has become apparent
> >that our current configuration will have a storage problem. Not
> >with how much but with thru put to the compute nodes, so I'm
> >here seeking ideas on how best to proceed. I should note that
> >our processing is non-standard in that we have very large files
> >and thus stream the data in large blocks to long running jobs.
> >We currently have 10 nodes but expect it to grow once we meet our
> >unusual security requirements and get it fully deployed. We need
> >3-6TB of storage with 40-80MB/sec/node thru put (ideally our users
> >would like 80-120MB/sec/node but will be satisfied with 40-80MB :-).
> >This would be on our GigE connect, not IB.
> >What options are there? We're considering having several I/O nodes
> >(how are those configured BTW? .. I couldn't find anything in the
> >docs) and have looked at DataDirect and Panasas (waiting to get
> >pricing info from Panasas). Any other options out there that we
> >haven't looked at yet? We're willing to be creative, but also
> >don't want something that requires alot of effort to configure
> >and manage, at the same time we're not made of money.
> >Thanks for any information,
More information about the npaci-rocks-discussion