Focal Point
[SOLVED] GETTOK getting wrong values from csv File

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/3737030176

June 06, 2014, 02:01 AM
Fady
[SOLVED] GETTOK getting wrong values from csv File
Hi!

I am using GETTOK to get the date field from a csv file and print it in another file, the result is not consistent

The code I am using is:
DAT1/A20 =GETTOK(LINE1,512,11,'"',19,'A20');
WDDAY/A8 = EDIT(DAT1,'9999$99$99$$$$$$$$');

The date format in the csv file is "yyyy/mm/dd hh:mm:ss" and I am storing it as "YYYYMMSS" in another file, however the out come it "mosty" correct. i.e. if the date is 2014/06/04 I get the following results :

DATE CNT
---- ---
2 2
201 1
2014 3
20140 2
201406 1
20140604 143

The errors are in random lines with no clear pattern as why it's happening.

Any ideas?

Many Thanks

Fady

This message has been edited. Last edited by: Fady,


Webfocus 7.7, Windows
June 06, 2014, 03:23 AM
Waz
You seem to be getting the 11th token, is that correct ?

What is the values of LINE1 ?


Waz...

Prod:WebFOCUS 7.6.10/8.1.04Upgrade:WebFOCUS 8.2.07OS:LinuxOutputs:HTML, PDF, Excel, PPT
In Focus since 1984
Pity the lost knowledge of an old programmer!

June 06, 2014, 03:32 AM
Fady
Yes the 11th Token, I am processing the csv file line by line.
An example of the line:
"Attendo WEWABM";"Kronoby fortsätter med Attendo";"ATTENDOWEWABM130266143";;"Svenska YLE - Yle Fem - Radio Vega";"http://svenska.yle.fi/artikel/2014/06/04/kronoby-fortsatter-med-attendo";"2014/06/04 17:04:00";"Kronoby fortsätter med AttendoPublicerad 04.06.2014 - 16:05.Arkivbild. Dalsbruks hälsostation Bild: YLE/ Tiina GrönroosKronoby hälsocentraler fortsätter med läkare från Attendo.På sikt är tanken att gå över till egna läkare, men ännu klarar man inte det. Därför går social- och..."

In General the token is fetched correctly, in random occasions it's not. I couldn't find any problem in the lines that generated the error.


Webfocus 7.7, Windows
June 06, 2014, 03:44 AM
Waz
I would suggest using the CSV delimiter to get the token. ";".

I think it is safer to do it that way. Assuming that there are no delimiters within the text.

It may also be safer to write a master file to read the CSV as data and not a single line.


Waz...

Prod:WebFOCUS 7.6.10/8.1.04Upgrade:WebFOCUS 8.2.07OS:LinuxOutputs:HTML, PDF, Excel, PPT
In Focus since 1984
Pity the lost knowledge of an old programmer!

June 06, 2014, 04:32 AM
Fady
Waz;

Thank you for your reply and please bare with me as I am still VERY green with WebFocus.

I am using the the quotation mark as I thought it would be safer in case the file came with a comma instead of the semi colon, and it would make it easier to handle the token without the extra quotation mark to ignore.

I am not sure what you mean with the master file, I am using my code to generate a "master" file that will be processed and added to a database.

All the fields seem to be fetched normally, the only field I am having an issue with is the date.

SET MORE=OFF, MSG=OFF

FILEDEF WWDAT DISK C:\IBI\APPS\ESMERK\WEWADAT.CSV (LRECL 512
-RUN

DEFINE FILE WWDAT
APUPROJ/A35 =GETTOK(LINE1,512,1,'"',35,APUPROJ);
PROJECT/A35 =GETTOK(APUPROJ,35,1,'_',35,PROJECT);
HEADLINE/A80 =GETTOK(LINE1,512,3,'"',80,HEADLINE);
WDATID/A35 =GETTOK(LINE1,512,5,'"',35,WDATID);
SOURCE/A30 =GETTOK(LINE1,512,7,'"',30,SOURCE);
DAT1/A20 =GETTOK(LINE1,512,11,'"',19,'A20');
WDDAY/A8 = EDIT(DAT1,'9999$99$99$$$$$$$$');
END
TABLE FILE WWDAT
SUM CNT.APUPROJ AS 'CNT'
BY WDDAY AS 'DATE'


Webfocus 7.7, Windows
June 06, 2014, 09:30 AM
Waz
What I am saying is that it may be esier for you to use the master file to read the csv as a csv.

e.g.
EX -LINES * EDAPUT MASTER,test,CV,FILE
FILENAME=test, SUFFIX=DFIX,$
SEGNAME=test, $
  FIELD=FIELD1 ,ALIAS=  ,A255 ,A255 ,$
  FIELD=FIELD2 ,ALIAS=  ,A255 ,A255 ,$
  FIELD=FIELD3 ,ALIAS=  ,A255 ,A255 ,$
  FIELD=FIELD4 ,ALIAS=  ,A255 ,A255 ,$
  FIELD=FIELD5 ,ALIAS=  ,A255 ,A255 ,$
  FIELD=FIELD6 ,ALIAS=  ,A255 ,A255 ,$
  FIELD=FIELD7 ,ALIAS=  ,A21,A21 ,$
  FIELD=FIELD8 ,ALIAS=  ,A500,A500,$
EDAPUT*

EX -LINES * EDAPUT ACCESS,test,CV,FILE
SEGNAME=test, DELIMITER=;, HEADER=NO, $
EDAPUT*

FILEDEF TEST DISK test.ftm

-RUN

-WRITE TEST "Attendo WEWABM";"Kronoby fortsätter med Attendo";"ATTENDOWEWABM130266143";;"Svenska YLE - Yle Fem - Radio Vega";"http://svenska.yle.fi/artikel/2014/06/04/kronoby-fortsatter-med-attendo";"2014/06/04 17:04:00";"Kronoby fortsätter med AttendoPublicerad 04.06.2014 - 16:05.Arkivbild. Dalsbruks hälsostation Bild: YLE/ Tiina GrönroosKronoby hälsocentraler fortsätter med läkare från Attendo.På sikt är tanken att gå över till egna läkare, men ännu klarar man inte det. Därför går social- och..."

-RUN
TABLE FILE TEST
PRINT
FIELD1
FIELD2
FIELD3
FIELD4
FIELD5
FIELD6
COMPUTE DATE/A8 = EDIT(FIELD7,'$9999$99$99') ;
FIELD8
END



Waz...

Prod:WebFOCUS 7.6.10/8.1.04Upgrade:WebFOCUS 8.2.07OS:LinuxOutputs:HTML, PDF, Excel, PPT
In Focus since 1984
Pity the lost knowledge of an old programmer!

June 09, 2014, 03:34 PM
susannah
[quote]

FILE=TEST, SUFFIX=DFIX
SEGNAME=TEST, SEGTYPE=S0, $
FIELD=FIELD1 ,ALIAS= ,A255 ,A255 ,$
FIELD=FIELD2 ,ALIAS= ,A255 ,A255 ,$
FIELD=FIELD3 ,ALIAS= ,A255 ,A255 ,$
FIELD=FIELD4 ,ALIAS= ,A255 ,A255 ,$
FIELD=FIELD5 ,ALIAS= ,A255 ,A255 ,$
FIELD=FIELD6 ,ALIAS= ,A255 ,A255 ,$
FIELD=FIELD7 ,ALIAS= ,A21,A21 ,$
FIELD=FIELD8 ,ALIAS= ,A500,A500,$
FIELDNAME=DELIMITER ,ALIAS=';' ,USAGE=A1 ,ACTUAL=A1 ,$

the double quotes are fine.
just write a master file that says, this is a delimited file and the delimiter is a semicolon.
so i've tweaked waz's code above
then just pull the field that is supposed to be a date. and work on it alone. Left adjust it, measure its length (ARGLEN function), etc...




In Focus since 1979///7706m/5 ;wintel 2008/64;OAM security; Oracle db, ///MRE/BID
June 16, 2014, 09:50 AM
Fady
Waz, Susannah

thank you very much, works like a charm.

is there a way to get the computed field (date) in a variable to use in different places in the report?

Fady


Webfocus 7.7, Windows
June 16, 2014, 11:03 AM
susannah
fady, not sure what you mean.
is 'the report' to be made from this .csv file?
or are you just grabbing the date from this .csv file and then using it in some other report?
if so, sure. but confirm first before i contine.




In Focus since 1979///7706m/5 ;wintel 2008/64;OAM security; Oracle db, ///MRE/BID
June 16, 2014, 03:03 PM
Fady
Hi Susannah,

I am grabbing the data from the cvs to generate another report. I was having a problem with the GETTOK as in some instances it was getting truncated dates. The master file approach seems to solve the issue, however I lost the variable that had the date inside and I was using in other parts of the report.


Webfocus 7.7, Windows
June 16, 2014, 05:29 PM
Waz
Was the original variable an amper variable ?

Can you post the original code ?


Waz...

Prod:WebFOCUS 7.6.10/8.1.04Upgrade:WebFOCUS 8.2.07OS:LinuxOutputs:HTML, PDF, Excel, PPT
In Focus since 1984
Pity the lost knowledge of an old programmer!

June 17, 2014, 01:53 AM
Fady
Hi Waz;

the original code:
FILEDEF WWDAT DISK C:\IBI\APPS\ESMERK\WEWADAT.CSV (LRECL 512
-RUN

DEFINE FILE WWDAT
APUPROJ/A35 =GETTOK(LINE1,512,1,'"',35,APUPROJ);
PROJECT/A35 =GETTOK(APUPROJ,35,1,'_',35,PROJECT);
HEADLINE/A80 =GETTOK(LINE1,512,3,'"',80,HEADLINE);
WDATID/A35 =GETTOK(LINE1,512,5,'"',35,WDATID);
SOURCE/A30 =GETTOK(LINE1,512,7,'"',30,SOURCE);
DAT1/A26 =GETTOK(LINE1,512,11,'"',26,DAT1);
DT_FROM_ALPHA/HYYMDS = HINPUT(26, DAT1, 8, 'HYYMDS');
WDDAY/A8 = HCNVRT(DT_FROM_ALPHA,'(HYYMD)',8, 'A8');
END


TABLE FILE WWDAT
SUM CNT.APUPROJ AS 'CNT'
BY WDDAY AS 'DATE'
HEADING
"***** WEWADAT.CSV MATERIAL BY DATE *****
" "
END
-RUN
TABLE FILE WWDAT
PRINT PROJECT WDDAY SOURCE HEADLINE
BY WDATID
BY PROJECT NOPRINT
IF WDATID NE ' '
IF WDDAY NE ' '
ON TABLE SAVE
END
-RUN

So as you see I am using the WDDAY in another file as a sum and then as a field


Webfocus 7.7, Windows
June 17, 2014, 05:40 PM
Waz
You can still do something similar to what you are doing now.
EX -LINES * EDAPUT MASTER,WWDAT,CV,FILE
FILENAME=WWDAT, SUFFIX=DFIX,$
SEGNAME=WWDAT, $
  FIELD=APUPROJ,ALIAS=  ,A35 ,A35 ,$
  FIELD=HEADLINE,ALIAS=  ,A80 ,A80 ,$
  FIELD=WDATID,ALIAS=  ,A35 ,A35 ,$
  FIELD=SOURCE,ALIAS=  ,A30 ,A30 ,$
  FIELD=FIELD5 ,ALIAS=  ,A255 ,A255 ,$
  FIELD=FIELD6 ,ALIAS=  ,A255 ,A255 ,$
  FIELD=DAT_FROM,ALIAS=  ,A21,A21 ,$
  FIELD=FIELD8 ,ALIAS=  ,A500,A500,$
  FIELDNAME=DELIMITER ,ALIAS=';' ,USAGE=A1 ,ACTUAL=A1 ,$
EDAPUT*

FILEDEF WWDAT DISK test.ftm

-RUN

-WRITE WWDAT "Attendo WEWABM";"Kronoby fortsätter med Attendo";"ATTENDOWEWABM130266143";;"Svenska YLE - Yle Fem - Radio Vega";"http://svenska.yle.fi/artikel/2014/06/04/kronoby-fortsatter-med-attendo";"2014/06/04 17:04:00";"Kronoby fortsätter med AttendoPublicerad 04.06.2014 - 16:05.Arkivbild. Dalsbruks hälsostation Bild: YLE/ Tiina GrönroosKronoby hälsocentraler fortsätter med läkare från Attendo.På sikt är tanken att gå över till egna läkare, men ännu klarar man inte det. Därför går social- och..."

-RUN

DEFINE FILE WWDAT
PROJECT/A35 =GETTOK(APUPROJ,35,1,'_',35,PROJECT);
WDDAY/A8 = EDIT(DAT_FROM,'$9999$99$99') ;
END

TABLE FILE WWDAT
SUM CNT.APUPROJ AS 'CNT'
BY WDDAY AS 'DATE'
HEADING
"***** WEWADAT.CSV MATERIAL BY DATE *****
" "
END
-RUN
TABLE FILE WWDAT
PRINT PROJECT WDDAY SOURCE HEADLINE
BY WDATID
BY PROJECT NOPRINT
IF WDATID NE ' '
IF WDDAY NE ' '
ON TABLE SAVE
END
-RUN



Waz...

Prod:WebFOCUS 7.6.10/8.1.04Upgrade:WebFOCUS 8.2.07OS:LinuxOutputs:HTML, PDF, Excel, PPT
In Focus since 1984
Pity the lost knowledge of an old programmer!