 
  
   
    2012 Unidata NetCDF Workshop  > Chunking and Deflating Data with NetCDF-4
 
2012 Unidata NetCDF Workshop  > Chunking and Deflating Data with NetCDF-4  
 
19.1 Choosing Chunksizes
How do you pick chunksizes?
- Choosing good chunksizes depends on the access patterns of your
  data. Are you trying to optimize writing, reading, or both? What are
  the access patterns at I/O bottlenecks?
- Choose chunksizes so that the subsets of data you are accessing
  fit into a chunk. That is, the chunks should be as large, or larger
  than, the subsets you are reading/writing.
- The chunk cache size must also be adjusted for good performance. The
  cache must be large enough to hold at least one chunk.
- Setting a larger cache (for example, big enough to hold tens or
  hundreds of chunks) will pay off only if the access patterns support
  it.
- On today's high-performance systems, large amounts of memory are
  available (both to the user and as internal hardware caching.) This
  suggests that chunks and caches should be large, and programs should
  take large sips of data.
 
 
 
  
   
    2012 Unidata NetCDF Workshop  > Chunking and Deflating Data with NetCDF-4
  
2012 Unidata NetCDF Workshop  > Chunking and Deflating Data with NetCDF-4