Re: [netcdfgroup] Informations about parallel netCDF4

On Wed, May 07, 2014 at 09:02:23AM -0500, Rob Latham wrote:
> i'm not entirely sure what you're asking here.  Most parallel I/O
> libraries carry out I/O to different regions of the file
> simultaneously (in parallel), and thereby extract more aggregate
> performance out of the storage system.
> 
> for any application using any I/O library, the trickiest part is how
> to decompose your domain over N parallel processes and how to
> describe that decomposition.

To clarify: the way I see it, you can do parallel I/O in three different ways. 
The first is to reserve a process which will only deal with I/O and other
process will exchange data to read/write with it.
The second is to have each process read/write independantly.
The third is to aggregate the I/O for several processes to improve performances.

So my question was: in practice, which approach does parallel netCDF use ?

> in strict performance terms -- which in the end is not really the
> be-all end all -- Argonne-Northwestern Parallel-NetCDF will be hard
> to beat, unless you are working with record variables.  
Do you speak from personal experience ? I would be very interested in seeing
some data or benchmark about it.


-- 
 Alexis Praga   
_________________________________________________________
                               Ph.D Student                  
                               Aviation et Environnement 
                               CERFACS
                               alexis.praga@xxxxxxxxxx
                               (33) 05 61 19 31 74

Attachment: pgpwabik44XfR.pgp
Description: PGP signature