Re: [netcdf-java] fastest way to determine feature type

Just as a followup, the attached program tests the speed of opening a file using the method in FeatureScan vs. GridDataset.open. In my test, the latter is actually faster by a few milliseconds. The real slowdown is the initial os caching of the file (in this case a 3.3 GB file). Once the file is in the OS cache, both methods are pretty quick. 

Thanks to John (and Roland) for their help.

Don

On 6/20/12 8:14 PM, John Caron wrote:

On 6/19/2012 3:19 PM, Don Murray wrote:

Hi-

I have a bunch of netCDF files and I want to quickly determine whether
they are grids, trajectories, or point features.  For grids, I've been
using GridDataset gds = GridDataset.open(path) and catch the exception
if it's not a grid, but for a 3.3 GB file, that can take 2 minutes (or
longer) to open and create the dataset if it is a grid.  I was
wondering if there's a quicker method of determining the feature type
of a netCDF file.

Thanks for your help.

Don


Hi Don:

The most convenient thing is to use ToolsUI / FeatureTypes /
FeatureScan, and give it a file or directory. It will try to figure out
the type and report on what it finds.

The code is in ucar.nc2.ft.scan.FeatureScan.java, you can copy the parts
you need.

Its an ongoing process, i think im not doing it as well as it can be
done. Send me reports on files it misidentifies.

John

_______________________________________________
netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/


--
Don Murray
NOAA/ESRL/PSD and CIRES
303-497-3596
http://www.esrl.noaa.gov/psd/people/don.murray/



import ucar.nc2.ft.scan.*;
import ucar.nc2.ft.*;
import ucar.nc2.dataset.NetcdfDataset;
import ucar.nc2.constants.FeatureType;
import ucar.nc2.dt.grid.*;

public class TestOpen {

    public static void main(String[] args) throws Exception {

        if (args.length == 0) {
           System.err.println("Need to supply a filename");
           System.exit(1);
        }
        String file = args[0];
        long start = System.currentTimeMillis();
        long end = start;
        if (args.length == 1) {
          NetcdfDataset ds = null;
          ds = NetcdfDataset.openDataset(file);
          end = System.currentTimeMillis();
          System.out.println("dataset open took "+(end-start)+" ms");
          start = end;
          FeatureDataset featureDataset = 
FeatureDatasetFactoryManager.wrap(null, ds, null, null);
          end = System.currentTimeMillis();
          System.out.println("feature determination took " + (end-start) + " 
ms");
          if (featureDataset != null) {
            FeatureType featureType = featureDataset.getFeatureType();
            if (featureType != null)
              System.out.println("type: "+featureType.toString());
           } else {
              System.out.println("unknown type");
           }
         } else {
            GridDataset gds = GridDataset.open(file);
            end = System.currentTimeMillis();
            System.out.println("griddataset open took "+(end-start)+" ms");
         }
    }
}