Focal Point
Flat File Problem

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/7041059952

December 03, 2007, 09:15 AM
linus
Flat File Problem
We have a flat file that we maintain manually by ftp'ing to our hard drives on our laptops, make the changes, then ftp back to the HP-UNIX box. We've been doing this for the past couple months, and recently our file seems to have three extra blank lines in it when you query it, but if you open the file and view it you don't see these blank lines. I've tried highlighting all the rows in the file to see if there were blank spaces some where in the file causing problems and there doesn't seem to be any. These three lines have reaked havoc on my programs that read this file and I've had to add code to accommodate them. Does anyone have any ideas of what happene and how to resolve the problem?

Thanks.

WF 7.1.4, Client=Windows 2003, Reports Server=HP-UNIX


WF 7.7.05
HP-UX - Reporting Server, Windows 2008 - Client, MSSQL 2008, FOCUS Databases, Flat Files
HTML, Excel, PDF
December 03, 2007, 09:56 AM
Frans
Stuff like this can happen when you work with different editors on different platforms. Some charachters can not be translated and can be interpreted as a blank line.

Have you tried to open it with VI? Or is the file too big to manually search for blank lines?


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
December 03, 2007, 11:00 AM
Francis Mariani
Are those three blank lines just three carriage returns and not blank spaces?


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
December 03, 2007, 11:57 AM
Darin Lee
My guess would be along the lines of Francis' suggestion. One other possibility - when working with flat files, if a line contains additional spaces (more than are described in the MFD), it looks like there are additional records when you run a query. It thinks if it has filled all the fields described in the MFD, then any additional characters must belong to the next record - even though there is no end-of-line or carriage return. (Flat files are funny this way). Check to make sure there are no carriage returns or trailing spaces on the last record(s).

This message has been edited. Last edited by: Darin Lee,


Regards,

Darin



In FOCUS since 1991
WF Server: 7.7.04 on Linux and Z/OS, ReportCaster, Self-Service, MRE, Java, Flex
Data: DB2/UDB, Adabas, SQL Server Output: HTML,PDF,EXL2K/07, PS, AHTML, Flex
WF Client: 77 on Linux w/Tomcat
December 03, 2007, 12:01 PM
Francis Mariani
Darin's guess is a good one as well...


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
December 04, 2007, 10:28 AM
linus
I highlighted the entire contents of the file to see if there were blank spaces, but I did this on my hard drive and not it VI, I'm relatively new to Unix so I'm not very good with VI. I will take a look at the file in VI and see if I see any blank spaces or carriage returns. Thanks for the quick responses.


WF 7.7.05
HP-UX - Reporting Server, Windows 2008 - Client, MSSQL 2008, FOCUS Databases, Flat Files
HTML, Excel, PDF
December 04, 2007, 12:18 PM
ira
they MIGHT actually be unprintable characters, and not visible to the naked eye. if you can view hex notation you may find they are not '40' (spaces), but some unprintable chars.
try using HEXBYT user-written subroutine

Ira
AIX 533 WF 5.3.6


aix-533,websphere 5.1.1,apache-2.0,
wf 538(d), 537 (p),
==============
7.6.11 (t) aix 5312
websphere 6.1.19
apache 2.0
December 05, 2007, 08:39 AM
linus
Thanks for the suggestion. I'll try that.


WF 7.7.05
HP-UX - Reporting Server, Windows 2008 - Client, MSSQL 2008, FOCUS Databases, Flat Files
HTML, Excel, PDF
December 05, 2007, 09:18 AM
GinnyJakes
On Unix, spaces would be ASCII '20'.


Ginny
---------------------------------
Prod: WF 7.7.01 Dev: WF 7.6.9-11
Admin, MRE,self-service; adapters: Teradata, DB2, Oracle, SQL Server, Essbase, ESRI, FlexEnable, Google
December 05, 2007, 12:49 PM
susannah
try reading your file using the LIST verb instead of PRINT, and you'll get line numbers for each line.
them open your file in, say, TextPad, which will give you line numbers...then compare and see where its screwing up.
I remember having a problem with unpredictable record lengths (to Darin's point) in a flat file and solving it using an mfd with SUFFIX=DFIX..
If i can find that code from several clients ago, i'll update this post.




In Focus since 1979///7706m/5 ;wintel 2008/64;OAM security; Oracle db, ///MRE/BID
December 05, 2007, 01:57 PM
linus
I resolved the issue. There were three rows in the file that each had a character way to the right of the last field in each record. Thanks for all the ideas.


WF 7.7.05
HP-UX - Reporting Server, Windows 2008 - Client, MSSQL 2008, FOCUS Databases, Flat Files
HTML, Excel, PDF
December 07, 2007, 11:34 AM
Darin Lee
Yup. exactly the point.
quote:
There were three rows in the file that each had a character way to the right of the last field in each record

Yup. Exactly the point. This is almost always the case when reading flat files and you're getting too many records.


Regards,

Darin



In FOCUS since 1991
WF Server: 7.7.04 on Linux and Z/OS, ReportCaster, Self-Service, MRE, Java, Flex
Data: DB2/UDB, Adabas, SQL Server Output: HTML,PDF,EXL2K/07, PS, AHTML, Flex
WF Client: 77 on Linux w/Tomcat