Christian,
If you don't want mess with making queues and the LDM, bypass it going
directly to GDBM. YOu should look at the man page for gdbm. You might
have to make a c programs to do the gdbm work. I modified the script, ie
>
> #!/local/bin/perl
> #
$fname = $ARGV[0] ;
$gdbm = gdbm_open ($fname ....
> # Now begin parsing file and decoding observations breaking on cntrl C
> $/ = "\cC" ;
>
> while( <STDIN> ) {
> # extract product ID
> /(\w{4}\d{2} \w{4} \d{6}})/s
> $prodID = $1 ; #you might want to change the id
gdbm_store ($gdbm, $prodID, $_, "0664") ;
> }
gdbm_close( $gdbm ) ;
Robb...
On Wed, 27 Nov 2002, [ISO-8859-1] Christian Pagé wrote:
Hi Robb,
I understand your suggested way. But I have some more technical
concerns:
I must create then an ldm queue to hold the products, and have an ldmd
process (and pqsurf) to take care of processing the bulletins.
Since I do ingest realtime data on that machine, which I don't want to
interfere, how can I set up a special queue, and a specific run of ldmd
to transfer all those bulletins without interfering the realtime data
flux?
Thanks again for your help,
On Wednesday, November 27, 2002, at 11:33 AM, Robb Kambic wrote:
>>> To: support@xxxxxxxxxxxxxxxx
>>> From: =?ISO-8859-1?Q?Christian_Pag=E9?= <page.christian@xxxxxxx>
>>> Subject: converting old wmo metar data to gdbm?
>>> Organization: UCAR/Unidata
>>> Keywords: 200211011624.gA1GO6X00936
>>
>> Hi,
>>
>> Is there a way to convert old metar archived data from raw wmo format
>> to gdbm format?
>> In realtime, it uses pqsurf with a DBFILE action.
>
> Christian,
>
> You are asking a tough question. It can be done with some work
> though. I
> read your support question from July 31 about a similiar problem.
> Here's
> some background info that really important. For pqsurf to work off a
> ldm
> queue it needs to have the bulletins have the same name as the original
> raw bulletin name. Otherwise, pqsurf will not find them and not perform
> any actions on them. What this means is that you can't take an hours
> worth of bulletin and pqinsert into an ldm queue because they would
> have
> a wrong name and pqsurf will not act on it.
>
> Solution:
>
> You have to break up the hourly files into the original bulletins, ie
> starting with the ^A and ending with the ^C.
>
> ^A^M^M
> 981 ^M^M
> SAFR31 LFPW 311230^M^M
> LFJL 311230Z AUTO 26010KT 9999 SCT058 SCT070 30/15 Q1022=^M^M
> LFBG 311230Z 36006KT 320V040 CAVOK 31/16 Q1024=^M^M
> ^M^M
> ^C
>
> Using pqinsert, send it to the ldm with name "SAFR31 LFPW 311230"
>
> ie
>
> % pqinsert -vl - -s 999 -f IDS -p "SAFR31 LFPW 311230"
> <rawbulletinFile>
>
> where <rawbulletinFile> contains:
>
> ^A^M^M
> 981 ^M^M
> SAFR31 LFPW 311230^M^M
> LFJL 311230Z AUTO 26010KT 9999 SCT058 SCT070 30/15 Q1022=^M^M
> LFBG 311230Z 36006KT 320V040 CAVOK 31/16 Q1024=^M^M
> ^M^M
> ^C
>
> Code:
>
>
> One could make a perl script to do the work. Something like this:
> Script name bulletinInsert
>
> #!/local/bin/perl
> #
> # Now begin parsing file and decoding observations breaking on cntrl C
> $/ = "\cC" ;
>
> while( <STDIN> ) {
> # extract product ID
> /(\w{4}\d{2} \w{4} \d{6}})/s
> $prodID = $1 ;
> open( OUT,">raw" ) ;
> pring OUT $_ ;
> close OUT ;
> `pqinsert -vl - -s 999 -f IDS -p "$prodID" raw` ;
> unlink "raw" ;
> }
>
> To run:
>
> % bulletinInsert < <raw hourly files>
>
> I would extract one bullletin and test pqinsert first, then look at the
> queue using pqcat to make sure the prodID is correct. Then use
> bulletinInsert and check queue again with pqcat. I didn't debug the
> script, you might want to run it in debug mode to check it.
>
> % perl -d bulletinInsert < <raw hourly files>
>
> Hopes this works for you,
> Robb...
>
>
>>
>> Thanks,
>>
>> Christian Pagé
>> page@xxxxxxxxxxx
>> http://meteocentre.com/ http://meteoalerte.com/
>>
>> Etudiant au Doctorat en Sciences de l'environnement UQAM
>> +1 514 987 3000 ext. 2376
>>
>>
>> ------- End of Forwarded Message
>>
>>
>
> =======================================================================
> =======
> Robb Kambic Unidata Program Center
> Software Engineer III Univ. Corp for Atmospheric
Research
> rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/
> =======================================================================
> =======
>
>
Christian Pagé
page@xxxxxxxxxxx
http://meteocentre.com/ http://meteoalerte.com/
Etudiant au Doctorat en Sciences de l'environnement UQAM
+1 514 987 3000 ext. 2376
==============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/
==============================================================================
