Re: forecast aggregations



Steven.Anthony wrote:

Hey John,
The dev server with the FMRC on it is not accessible outside of the building, 
however, I have attached the FMRC catalog & the directory listing.
If you need anything else, feel free to ask.





Thanks for the quick response.
- Steven

John Caron said the following on 2/22/2007 1:21 PM:


Hi Steven:

Can you send me a directory listing and the resulting FMRC catalog?

If the server is running and I can access, just send me the catalog URL

thanks

Glenn.Rutledge wrote:


The following from Steve Anthony- Glenn

Hey,
Our problems with the FMRC Aggregation are from a date issue. It appears that when the data is read, the dates are getting thrown off by an offset.
In the catalog.xml file, we have copied (almost verbatim) the syntax that UCAR uses for its Motherlode server. 
from catalog.xml:
<datasetFmrc name="gfs-test-FMRC" collectionType="ForecastModelRuns" harvest="true" path="fmrc/gfs_3"> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; enhance="true"> <aggregation dimName="run" type="forecastModelRunSingleCollection" fmrcDefinition="NCEP-GFS-Global_onedeg.fmrcDefinition.xml" timeUnitsChange="true" > <scanFmrc location="/nomads3_data/raid2/noaaport/merged/gfs-hi/200611/20061129" 
       regExp=".*\.grb$"
       runDateMatcher="#gfs_3_#yyyyMMdd_HH"
       forecastOffsetMatcher="HHH#.grb#"/>
         </aggregation>
</netcdf>
<fmrcInventory location="/nomads3_data/raid2/noaaport/merged/gfs-hi/200611/20061129" suffix=".grb" fmrcDefinition="NCEP-GFS-Global_onedeg.fmrcDefinition.xml" /> 
</datasetFmrc>
When I open up the "Forecast Model Run" folder in the FMRC dataset, I see 4 links (like I should) but the run dates are horribly off, as you can see below, the run dates are jumping by almost a month! - these should only be 6 hours apart. 
gfs-test-FMRC_RUN_2007-02-12T00:00:00Z
gfs-test-FMRC_RUN_2007-01-18T00:00:00Z
gfs-test-FMRC_RUN_2006-12-24T00:00:00Z
gfs-test-FMRC_RUN_2006-11-29T00:00:00Z


i dont see anything obviously wrong. Are there any error or debug messages in 
catalina.out, or in thredds/content/logs/threddsServlet.log?

Do you know how to change log4j debugging on ucar.nc2.ncml to DEBUG (from 
thredds debug page is easiest) ? Else from WEB-INF/log4j.xml, add

 <logger name="ucar.nc2.ncml">
   <level value="DEBUG"/>
   <appender-ref ref="threddsServlet"/>
 </logger>

then look at threddsServlet.log after accessing the fmrc dataset.
The "Constant Forecast Offset" folder appears as it should, forecast offsets out to 180 hours, and at a 3 hour interval.
The "Constant Forecast Date" folder, however, also has problems, but they are a combination of the 2 previous ones. 
gfs-test-FMRC_ConstantForecast_2006-11-29T00:00:00Z
gfs-test-FMRC_ConstantForecast_2006-11-29T03:00:00Z
gfs-test-FMRC_ConstantForecast_2006-11-29T06:00:00Z
...
gfs-test-FMRC_ConstantForecast_2006-12-06T12:00:00Z << looks normal up to this point gfs-test-FMRC_ConstantForecast_2006-12-24T00:00:00Z << then it jumps 18 days, and does this 2 more times in the dataset. 
gfs-test-FMRC_ConstantForecast_2006-12-24T03:00:00Z
gfs-test-FMRC_ConstantForecast_2006-12-24T06:00:00Z
...

Questions:
Is my syntax wrong in the catalog.xml file? (we have one grib1 file per forecast hour per model run) What is in the NCEP-GFS-Global_onedeg.fmrcDefinition.xml file, and where is it? 

why dont you remove this for now, and we can come back to it later. It may be 
interfering.
So remove the <fmrcInventory> element and the fmrcDefinition attribute.
also, what version of TDS are you running? 
is there any way you can run TDS on a public server, with enough data files to 
duplicate the proble?

otherwise, can you make accessible to me as many as the files as needed to 
where the problem starts to occur. I see from my testing i only had 3 files, 
all from the same run.

thanks