Re: [cf-satellite] cf-satellite Digest, Vol 14, Issue 1

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

So that everyone can better follow this discussion, I attach the ncdump
of a seviri file.

Best regards,
Martin

On 24/10/11 17:08, Martin Raspaud wrote:
> On 24/10/11 15:35, Peter Miu wrote:
>> Hi Martin,
> 
> Hi Pete,
> 
>> Thanks for the fast feedback.  Before I answer your questions, I would just 
>> like to let you know
>> that the EUMETSAT Data Centre is currently planning the implementation 
>> NetCDF formats as the 
>> common delivery format for all orderable data sets from the Archive.  
> 
>> A pilot has already been performed for the level 1B and 2 products from the 
>> Metop-A ASCAT instrument
>> and these are available for user feedback under the following URL:
> 
>> http://gsics.eumetsat.int/thredds/ascat.html
> 
>> The ASCAT NetCDF products were also presented to users in this year's 
>> EUMETSAT user conference held in Oslo.
> 
>> The development of usable NetCDF formats is our top priority as we want to 
>> ensure that all users (novice and expert) 
>> can immediately use our data so it is very important that guidelines 
>> regarding best practises, filename conventions, 
>> CF-Conventions and standards are followed.  A Format Advisory Group has been 
>> set up at EUMETSAT to discuss these 
>> guidelines with a view to publishing them so that users are aware how our 
>> data is organised and also to 
>> propose updates to the CF-conventions for satellite data as work is needed 
>> here (hence the existence of this mailing list! :o)
> 
> I was at the EUMETSAT conference this year :)
> 
>> Here are the answers to your questions:
> 
>> - Why are southMostLine, eastMostPixel, northMostLine, westMostPixel, and 
>> numberOfChannels dimensions and not attributes ?
> 
>> Apart from the HRV channel, these values remain constant for all channels in 
>> the data sets so we can be specify 
>> them as global attributes.  If the channel arrays store different 'sub-area' 
>> then we would need these value to be stored
>> as variable attributes for each channel array.  This is not the case for our 
>> NetCDF data set i.e. all channel arrays hold the 
>> same area for the same time.
> 
>> Note, the NetCDF file follows the Unidata recommendation for identifying the 
>> coordinates of the data in the channel arrays 
>> (Coordinate System Object Model - Current Encodings). Longitude and latitude 
>> arrays exist in the data set and 
>> their contents are used to geo-locating the pixel counts for each channel 
>> onto a projection.
> 
>> See: http://www.unidata.ucar.edu/software/netcdf-java/CDM/
> 
> Ok. I don't really understand the interest of having these
> dimensions/attributes though, since I guess you will have to define
> lon/lats anyway for each different resolution...
> 
>> - - How would you deal with the inclusion of the HRV channel ?
> 
>> We have not implemented the HRV channel in our NetCDF data sets as GSICS 
>> does not use this channel for calibration.
> 
>> To implement this channel in our NetCDF data sets, we would just need to add 
>> another longitude and latitude array 
>> specifying the location of each HRV pixel counts and a 2D array holding the 
>> HRV pixel counts.
> 
> Sounds good, but the HRV has 2 sub-areas: would you have a different
> variable for each or would you group the in one big channel ? (We
> implemented the latter, since the netcdf compression works well in this
> case)
> 
>> - - Are the longitudes and latitudes values different from what a "vertical 
>> perspective" with right parameters would provide ? If they do, why ? If they 
>> don't, why not include a grid-mapping definition ?
> 
>> Not sure what you mean here, perhaps the following explain it:
> 
>> For the pixel count in ch1( y, x ), it is located at latitude lat( y, x ) 
>> and longitude lon( y, x ).
>> By assigning the lat and lon as the coordinates system for any channel 
>> array, applications know how to geo-locate its contents.
> 
>>   float lat(y,x);
>>   float lon(y,x);
>>   int ch1(y,x);
>>   ch1:coordinates="lat lon";
> 
>>   int ch2(y,x);
>>   ch2:coordinates="lat lon";
> 
> My question was if the lon/lats differ from what one would get from
> running proj.4 with parameters like
> +proj=geos +lon0=0 +a=... etc.
> If one would get the same lon/lat data, then I would recommend to record
> also these parameters in a grid mapping variable. In this case, the
> lon/lats arrays would even become optional.
> 
>> - - On the cf-satellite list, we have long been discussing "band"s. Did you 
>> consider using those ?
> 
>> Sorry, I don't know what bands are.
> 
> Well, this is a bit sad :(. We have been working quite a lot on this
> list toward having band variables that would correctly describe
> satellite channels (or bands). My colleagues and I have tried to drag
> Eumetsat's attention to this list and the work done here on several
> occasions. Tom Whittaker was also at the Eumetsat conference last year
> in Cordoba to present CF conventions and the possibility to apply these
> to satellite. But apparently that was not enough...
> 
> You can read more on the band variable and related questions here:
> http://www.unidata.ucar.edu/mailing_lists/archives/cf-satellite/2011/thread.html
> 
>> - - Did you consider grouping channels in 3d arrays instead of having them 
>> separated ?
> 
>> I'm not sure what value a 3D arrays would give to the data.  Applications 
>> like the Unidata can 
>> plot the arrays on top of each other if needed.
> 
>> As mentioned above, we want our data to be immediately usable by all users 
>> and by following guidelines,
>> we add values to the data as the contents can be examined and plotted by 
>> existing free NetCDF applications
>> without any further processing by the users.
> 
>> - - Could you provide in the file calibration coefficients needed to convert 
>> radiances to reflectances and brightness temperatures ?
> 
>> Yes we can add any variable attribute to the channel arrays to support the 
>> needs of our users.
>> The power of NetCDF is as long as we don't remove or reorganise any 
>> attributes and/or arrays, 
>> existing users and applications will not 'break' by the change.  The only 
>> concern here is we need to 
>> ensure that a standard convention is followed for representing measurements 
>> that can be stored in different units 
>> (e.g. temperature can be stored as Celsius, Fahrenheit, Kelvin, ... ).  What 
>> unit is used should be
>> based on the needs of the 'core' users and this unit with a standard name 
>> should be included in the CF-conventions.
> 
> That would be nice :) Thanks a lot.
> 
> Best regards,
> Martin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Red Hat - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOpkyaAAoJEBdvyODiyJI4njoIAKgFcS4f3JKNlcBV1bM3be40
o7Rjl+f1s5RUr547iPaK7dTGPMpSLII+EuC6fbBdpWUNvYR1J0Dz05VjrcuJzw8Z
RFTT/eO8gLKjhkd8NlfQakORb7zj64QTIyoNGGG1aDy18QSC3aPl2qzXaPZpXZYC
R88xddqIAfYrEXzjRQvV0T9M4wzgAGygIx3YhJiheQQhsb0ZcGPUnJ/6RkVPXYmC
ZSoMt/fXAqwUqyIR2w92mJLcLWfKbiuZfgeBiXGr4S+Z4byOufA0bG04944i6xPo
AHuIu91wlaLjgUukqHSa9kJH8o38QkWGatYPyvugRc0ykWdFKkIG9nCnY/ISM/4=
=vuE+
-----END PGP SIGNATURE-----

netcdf 
W_XX-EUMETSAT-Darmstadt\,VIS+IR+IMAGERY\,MSG2+SEVIRI_C_EUMG_20111024221511 {
dimensions:
        yc = 2345 ;
        xc = 2356 ;
        southMostLine = 686 ;
        eastMostPixel = 679 ;
        northMostLine = 3030 ;
        westMostPixel = 3034 ;
        numberOfChannels = 11 ;
        totalNumberOfSeviriChannels = 12 ;
variables:
        short ch1(yc, xc) ;
                ch1:standard_name = 
"satellite_geo_meteosat_vis_0.6_pixel_count" ;
                ch1:long_name = "SEVIRI visible 0.6 micron pixel counts" ;
                ch1:scale_factor = 0.0204191002994776 ;
                ch1:add_offset = -1.04137411527336 ;
                ch1:units = "1" ;
                ch1:valid_min = 0s ;
                ch1:valid_max = 1023s ;
                ch1:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch2(yc, xc) ;
                ch2:standard_name = 
"satellite_geo_meteosat_vis_0.8_pixel_count" ;
                ch2:long_name = "SEVIRI visible 0.8 micron pixel counts" ;
                ch2:scale_factor = 0.0261677000671625 ;
                ch2:add_offset = -1.33455270342529 ;
                ch2:units = "1" ;
                ch2:valid_min = 0s ;
                ch2:valid_max = 1023s ;
                ch2:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch3(yc, xc) ;
                ch3:standard_name = "satellite_geo_meteosat_ir_1.6_pixel_count" 
;
                ch3:long_name = "SEVIRI infrared 1.6 micron pixel counts" ;
                ch3:scale_factor = 0.0223222002387047 ;
                ch3:add_offset = -1.13843221217394 ;
                ch3:units = "1" ;
                ch3:valid_min = 0s ;
                ch3:valid_max = 1023s ;
                ch3:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch4(yc, xc) ;
                ch4:standard_name = "satellite_geo_meteosat_ir_3.9_pixel_count" 
;
                ch4:long_name = "SEVIRI infrared 3.9 micron pixel counts" ;
                ch4:scale_factor = 0.00365866686960072 ;
                ch4:add_offset = -0.186592010349637 ;
                ch4:units = "1" ;
                ch4:valid_min = 0s ;
                ch4:valid_max = 1023s ;
                ch4:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch5(yc, xc) ;
                ch5:standard_name = "satellite_geo_meteosat_wv_6.2_pixel_count" 
;
                ch5:long_name = "SEVIRI water vapour 6.2 micron pixel counts" ;
                ch5:scale_factor = 0.0083181111898559 ;
                ch5:add_offset = -0.424223670682651 ;
                ch5:units = "1" ;
                ch5:valid_min = 0s ;
                ch5:valid_max = 1023s ;
                ch5:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch6(yc, xc) ;
                ch6:standard_name = "satellite_geo_meteosat_wv_7.3_pixel_count" 
;
                ch6:long_name = "SEVIRI water vapour 7.3 micron pixel counts" ;
                ch6:scale_factor = 0.0386219681754982 ;
                ch6:add_offset = -1.96972037695041 ;
                ch6:units = "1" ;
                ch6:valid_min = 0s ;
                ch6:valid_max = 1023s ;
                ch6:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch7(yc, xc) ;
                ch7:standard_name = "satellite_geo_meteosat_ir_8.7_pixel_count" 
;
                ch7:long_name = "SEVIRI infrared 8.7 micron pixel counts" ;
                ch7:scale_factor = 0.126744318685915 ;
                ch7:add_offset = -6.46396025298166 ;
                ch7:units = "1" ;
                ch7:valid_min = 0s ;
                ch7:valid_max = 1023s ;
                ch7:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch8(yc, xc) ;
                ch8:standard_name = "satellite_geo_meteosat_ir_9.7_pixel_count" 
;
                ch8:long_name = "SEVIRI infrared 9.7 micron pixel counts" ;
                ch8:scale_factor = 0.103960910697578 ;
                ch8:add_offset = -5.30200644557646 ;
                ch8:units = "1" ;
                ch8:valid_min = 0s ;
                ch8:valid_max = 1023s ;
                ch8:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch9(yc, xc) ;
                ch9:standard_name = 
"satellite_geo_meteosat_ir_10.8_pixel_count" ;
                ch9:long_name = "SEVIRI infrared 10.8 micron pixel counts" ;
                ch9:scale_factor = 0.20503567620766 ;
                ch9:add_offset = -10.4568194865907 ;
                ch9:units = "1" ;
                ch9:valid_min = 0s ;
                ch9:valid_max = 1023s ;
                ch9:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + ( 
pixelCount * scale_factor )" ;
        short ch10(yc, xc) ;
                ch10:standard_name = 
"satellite_geo_meteosat_ir_12.0_pixel_count" ;
                ch10:long_name = "SEVIRI infrared 12.0 micron pixel counts" ;
                ch10:scale_factor = 0.222311146680895 ;
                ch10:add_offset = -11.3378684807256 ;
                ch10:units = "1" ;
                ch10:valid_min = 0s ;
                ch10:valid_max = 1023s ;
                ch10:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + 
( pixelCount * scale_factor )" ;
        short ch11(yc, xc) ;
                ch11:standard_name = 
"satellite_geo_meteosat_ir_13.4_pixel_count" ;
                ch11:long_name = "SEVIRI infrared 13.4 micron pixel counts" ;
                ch11:scale_factor = 0.157606896877807 ;
                ch11:add_offset = -8.03795174076818 ;
                ch11:units = "1" ;
                ch11:valid_min = 0s ;
                ch11:valid_max = 1023s ;
                ch11:comment = "Radiance in mW m-2 sr-1(cm-1)-1 = add_offset + 
( pixelCount * scale_factor )" ;
        byte channelAvailableFlags(totalNumberOfSeviriChannels) ;
                channelAvailableFlags:standard_name = 
"satellite_geo_channels_available_flags" ;
                channelAvailableFlags:valid_max = "1b" ;
                channelAvailableFlags:flag_meaning = 
"channel_not_available_in_file channel_available_in_file" ;
                channelAvailableFlags:long_name = "Channel Available in File 
Flags" ;
                channelAvailableFlags:flag_masks = "0b, 1b" ;
                channelAvailableFlags:valid_min = "0b" ;
                channelAvailableFlags:units = "1" ;
        float lat(yc, xc) ;
                lat:missing_value = -999.f ;
                lat:_CoordinateAxisType = "Lat" ;
                lat:standard_name = "latitude" ;
                lat:valid_max = 90.f ;
                lat:long_name = "Latitudes for each pixel count" ;
                lat:valid_min = -90.f ;
                lat:units = "degrees_north" ;
        float lon(yc, xc) ;
                lon:missing_value = -999.f ;
                lon:_CoordinateAxisType = "Lon" ;
                lon:standard_name = "longitude" ;
                lon:valid_max = 180.f ;
                lon:long_name = "Longitude for each pixel count" ;
                lon:valid_min = -180.f ;
                lon:units = "degrees_east" ;

// global attributes:
                :Conventions = "CF-1.4" ;
                :Metadata_Conventions = "Unidata Dataset Discovery v1.0" ;
                :title = "MSG15 channel data in NetCDF." ;
                :summary = "MSG15 channel pixel counts with calibration 
coefficients and geo-location values." ;
                :keywords = "EUMETSAT, GSICS, ARCHIVE, UNIDATA, SEVIRI, 
CALIBRATION COEFFICIENT - SLOPE-OFFSET, NetCDF" ;
                :history = "V1.0 - EUMETSAT COPYRIGHT 2009" ;
                :comment = "OPERATIONAL VERSION" ;
                :creator_name = "EUMETSAT Archive" ;
                :creator_url = "http://archive.eumetsat.int"; ;
                :creator_email = "ops@xxxxxxxxxxxx" ;
                :institution = "EUMETSAT" ;
                :time_converage_start = "2011-10-24T22:15:11Z" ;
                :time_converage_end = "2011-10-24T22:27:41Z" ;
                :license = "CopyRight EUMETSAT 2009" ;
                :references = "Unidata NetCDF, Climate Format Conventions, 
EUMETSAT MSG1.5 Native Format Guide" ;
                :format_authors = "EUMETSAT" ;
                :format_version = "1.0" ;
                :source_formats = 
"MSG2-SEVI-MSG15-0100-NA-20111024222741.515000000Z-2514381.tmp" ;
                :format_name = "SEVIRI_CALIBRATION_COEFFICIENT" ;
                :satellite_identifier = "MSG2" ;
                :instrument = "SEVI" ;
                :disposition_mode = "OPE" ;
                :quality = "OK" ;
                :wmo_filename = 
"W_XX-EUMETSAT-Darmstadt,VIS+IR+IMAGERY,MSG2+SEVIRI_C_EUMG_20111024221511.nc" ;
                :wmo_data_category = "101" ;
                :wmo_international_data_sub_category = "000" ;
                :subSatellitePoint = "000.0000" ;
                :originalSubSatellitePoint = "0.0" ;
                :scanMode = "ALTHRV" ;
                :fullScanTime = "742.4_seconds" ;
                :rsScanTime = "246_seconds" ;
                :pixelScanTimeFormula = 
"StartScanTime+(LineNumber/NumberOfLines)*NorthSouthScanTime" ;
                :channel_available_flag_1 = "VIS06" ;
                :channel_available_flag_2 = "VIS08" ;
                :channel_available_flag_3 = "IR16" ;
                :channel_available_flag_4 = "IR39" ;
                :channel_available_flag_5 = "WV62" ;
                :channel_available_flag_6 = "WV73" ;
                :channel_available_flag_7 = "IR87" ;
                :channel_available_flag_8 = "IR97" ;
                :channel_available_flag_9 = "IR108" ;
                :channel_available_flag_10 = "IR120" ;
                :channel_available_flag_11 = "IR134" ;
                :channel_available_flag_12 = "NotUsedAlwaysSetToZero" ;
}

Attachment: seviri.txt.sig
Description: PGP signature

begin:vcard
fn:Martin Raspaud
n:Raspaud;Martin
org:SMHI
adr;quoted-printable;quoted-printable:;;Folkborgsv=C3=A4gen 
1;Norrk=C3=B6ping;;60176;Sweden
email;internet:martin.raspaud@xxxxxxx
tel;work:+46 (0)11 495 8261
tel;cell:+46 (0)11 495 8261
url:www.smhi.se
version:2.1
end:vcard