Focal Point
[CLOSED] Comparing two files and outoputting the difference

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/707109622

September 01, 2009, 02:19 PM
Greaseman
[CLOSED] Comparing two files and outoputting the difference
This most likely has been asked before, but I couldn't find it.

Here is the scenario: On Sunday, I created a data file and held it aside with a name of PREV (for Previous day's data). On Monday, I again create a file, but this one contains both Sunday's data and additional data for today. I want to be able to compare the file from Sunday to the file from Monday, and output only the records that are new for Monday. I then want to append the new Monday records to the Sunday file, so that on Tuesday I will then compare the Tuesday file against the records in the combined Sunday-Monday file, and so on into the future. In other words I want to be able to do an ongoing data compare and only produce a resultant new records file each day.

My original Sunday file will be a flat file. with an MFD that has a format=FIX.

Can anyone provide help or guidance on what needs doing? Better yet, if there is a sample out there, can you point me to it?

Thanks very much!

This message has been edited. Last edited by: Kerry,


Mainframe FOCUS, Rel 7.1.1 running on MVS/TSO
Windows XP, Mainframe VM/CMS and MVS/TSO
outputs in Reports, Excel, Flat file
September 01, 2009, 02:33 PM
Doug
Just two words on this: "MATCH FILE".




   In FOCUS Since 1983 ~ from FOCUS to WebFOCUS.
   Current: WebFOCUS Administrator at FIS Worldpay | 8204, 8206
September 01, 2009, 03:27 PM
Greaseman
Um.... perhaps a bit fuller explanation would be more helpful....


Mainframe FOCUS, Rel 7.1.1 running on MVS/TSO
Windows XP, Mainframe VM/CMS and MVS/TSO
outputs in Reports, Excel, Flat file
September 01, 2009, 03:37 PM
Francis Mariani
Um... how about a bit of digging into the documentation?

Creating Reports With WebFOCUS Language > Merging Data Sources > Merging Data


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
September 01, 2009, 03:42 PM
Greaseman
Um... thanks.


Mainframe FOCUS, Rel 7.1.1 running on MVS/TSO
Windows XP, Mainframe VM/CMS and MVS/TSO
outputs in Reports, Excel, Flat file
September 02, 2009, 04:59 AM
Danny-SRL
Greaseman,

Just a thought. Why would you want to append Monday's new records to the Sunday file in order to compare the Tuesday file with the appended file? Wouldn't the appended file be exactly the Monday file?


Daniel
In Focus since 1982
wf 8.202M/Win10/IIS/SSA - WrapApp Front End for WF

September 02, 2009, 10:51 AM
Greaseman
Danny,

In a way, you are correct, but here is an outline of my entire work flow, that might help to explain what I'm interested in doing:

01) Create Sunday file
02) On Monday, run program to extract all data plus any new data.
03) Compare Sunday File against Monday file.
04) Pull off unmatching records from Monday file into a "Difference" file.
05) Make an output Excel file of the unmatched records for subsequent FTP-ing to a server.
06) Append the unmatched records to the Sunday file.

07) On Tuesday, repeat the process.

The user is only interested in the unmatched records that get FTP-ed to the server.

I may be confusing myself, but my thinking is that the Sunday file would be a good baseline to start with and then to process each subsequent day's data against what I started with.

Is how I'm planning this a good approach, or should I maybe try doing this in a different way? I'm open to suggestions.

Thank you for writing.


Mainframe FOCUS, Rel 7.1.1 running on MVS/TSO
Windows XP, Mainframe VM/CMS and MVS/TSO
outputs in Reports, Excel, Flat file
September 02, 2009, 01:19 PM
M Meagher
I would think Tuesday is just Monday plus anything new ... but I don't know your data. MATCH FILE is definitely the way to go.

MATCH FILE SUNDAY
SUM
  V1
  V2
  V3
BY V4
BY V5
FILE MONDAY
SUM
  V1
  V2
  V3
BY V4
BY V5
AFTER MATCH HOLD AS DIFF_MATCH NEW-NOT-OLD
END


This will give you the difference file. You can create Excel file from DIFF_MATCH.
September 02, 2009, 01:21 PM
M Meagher
Oops I left out a RUN ...
 MATCH FILE SUNDAY
SUM
  V1
  V2
  V3
BY V4
BY V5
RUN
FILE MONDAY
SUM
  V1
  V2
  V3
BY V4
BY V5
AFTER MATCH HOLD AS DIFF_MATCH NEW-NOT-OLD
END
 

September 02, 2009, 06:45 PM
Prash1983
After the process explained by "M Meagher", you will need a "MORE" command to append and get a single file. You might be knowing this, but I thought I should post it.


webFOCUS 7.6.x,
z/OS(Mainframes),
Excel and HTML
September 02, 2009, 10:37 PM
Doug
Um... Thanks Francis and M Meagher for your support on my "Hint" towards the solution...

"04) Pull off unmatching records from Monday file into a "Difference" file." Sounds like a MATCH FILE to me...

Another thought: Don't the Sunday and Monday records have some timestamps to ID them? Can those timestamps be used in this quandary?
September 03, 2009, 10:11 AM
Greaseman
Question.... will the way I've got my logic described in my response to Danny work? Opinions? (And please...without the "um..." which I used once since I thought it was best at the time to write like I would talk).


Mainframe FOCUS, Rel 7.1.1 running on MVS/TSO
Windows XP, Mainframe VM/CMS and MVS/TSO
outputs in Reports, Excel, Flat file
September 03, 2009, 11:28 AM
Prash1983
As long as user is only interested in the unmatched records, the logic explained will work.


webFOCUS 7.6.x,
z/OS(Mainframes),
Excel and HTML
September 03, 2009, 02:25 PM
Greaseman
Understood.... however my question wa would the logic I outlined in my response to Danny also work?


Mainframe FOCUS, Rel 7.1.1 running on MVS/TSO
Windows XP, Mainframe VM/CMS and MVS/TSO
outputs in Reports, Excel, Flat file
September 04, 2009, 04:34 AM
Tony A
Greaseman,

Logically yes, although I would be inclined to change 6) to replace Sunday file with Monday file - less processing involved and produces the same result.

T



In FOCUS
since 1986
WebFOCUS Server 8.2.01M, thru 8.2.07 on Windows Svr 2008 R2  
WebFOCUS App Studio 8.2.06 standalone on Windows 10 
September 04, 2009, 10:00 AM
Greaseman
Oops! I believe I left off a big part of my logic flow.... where I said the process repeats on Tuesday, it also gets repeated on Wednesday, Thursday and Friday..in other words, the whole process repeats every day of the week.

That's why I thought it would be easier to make a base file on Sunday and then add to that base file each subsequent day, as opposed to changing the program or JCL to refer to a different file each day.

Thanks for replying.


Mainframe FOCUS, Rel 7.1.1 running on MVS/TSO
Windows XP, Mainframe VM/CMS and MVS/TSO
outputs in Reports, Excel, Flat file
September 11, 2009, 12:06 AM
TexasStingray
Are you just trying to do a change data capture routine. compare file 1 with file 2 and mark records as I insert for new, D Deleted, C changed or blank for no change.




Scott

Does anyone use SET ALL=PASS anymore?


"There is no limit to what you can achieve ... if you don’t care who gets the credit." Roger Abbott
Bonjour,
SET ALL=ON, SET ALL=OFF, SET ALL=PASS made me sweat a great deal, because somebody who didn't have a telephone number was forgotten in my lists of people who owed more than XXXX Francs (at the time).
A nice and easy way not to pay one's debt.
At the time, a specialist of IBF, took the time to explain me everything about those powerful Sets ( Including Useless on SEGTYPE = U )
Those Sets are obviously Focus+ level. It does not at all mean they are Useless.
They are somewhere in the very Heart of Focus and it's way to turn into Procedural Action its Global and Biblic langage ( "Reading and Filtering Data : How the internal Matrix is constituted" - Myownmemory.My-Important-Focuseries.fr )
Cordially and Focusely
PS : Since when are the LEFT Outer Join available in DB2 ? Our Focus Sets are older


Focus Mainframe 7.6.11
Dev Studio 7.6.11 and !!!
PC Focus, Focus for OS/2, FFW Six, MSO
I'm not saying I use it every day, but it is certainly not a forgotten setting in my book.
Why do you ask?


GamP

- Using AS 8.2.01 on Windows 10 - IE11.
in Focus since 1988
In my experience, SET ALL=PASS is a mess with DB2. It does not generate the correct SQL, even with SQL DB2 SET SQLJOIN OUTER ON.


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
I might be off base and if I am, I apologize. However, the last ferw posting seem to gotten away from the original subject of this posting. Can we get back to the posting's issue, please?


Mainframe FOCUS 7.0
VM/CMS and MVS/TSO
Here is the thrust of the original posting:

01) Create Sunday file
02) On Monday, run program to create a Monday file.
03) Compare Sunday File against Monday file.
04) Pull off unmatching records from Monday file into a "Difference" file.
05) Make an output Excel file of the unmatched records for subsequent FTP-ing to a server.
06) Append the unmatched records from the Monday file to the Sunday file.

07) On Tuesday, repeat the process, this time using the Tuesady file agaisnt the Sunday file.

08) On Wednesday, repeat the process, this time, using the Wednesday file against the Sunday file.

09) On Thrusday, repeat the process, this time using the Thursday file against the Friday file.

10) On Friday, repeat the process, this time using the Friday file against the Sunday file.

11) The following week, DO NOT create a new Sunday file... we're going to utilize the original Sunday file.

12) On Monday of the following week, compare the following week's Monday file against the original Sunday file.

13) On Tuesday, repeat the process.

14) In other words, once an original Sunday file has been created, then each subsequent day's file will be processed against it.

Hopefully this clears up some of the confusions my posting seems to have created.

Thank you to all who have responded.


Mainframe FOCUS 7.0
VM/CMS and MVS/TSO
I wasn't going away from the subject. Example:
-* example sunday file.
TABLE FILE CAR
SUM
     DEALER_COST
     RETAIL_COST
     SALES
BY COUNTRY
BY CAR
BY MODEL
WHERE MODEL OMITS 'AUTO';
ON TABLE HOLD AS SUNDAY FORMAT ALPHA
END

-* example monday file.
TABLE FILE CAR
SUM
     DEALER_COST
     RETAIL_COST
     SALES
BY COUNTRY
BY CAR
BY MODEL
ON TABLE HOLD AS MONDAY FORMAT ALPHA
END

-* join 'em ... note the all
JOIN 
	COUNTRY AND CAR AND MODEL IN MONDAY TO ALL
	COUNTRY AND CAR AND MODEL IN SUNDAY AS J1.
END

-* set all to pass
SET ALL=PASS

TABLE FILE MONDAY
PRINT MONDAY.DEALER_COST MONDAY.RETAIL_COST MONDAY.SALES
BY MONDAY.COUNTRY 
BY MONDAY.CAR
BY MONDAY.MODEL
-* enforce the all=pass with a B/S condition
WHERE SUNDAY.COUNTRY EQ 'NOTHING';
END
  



"There is no limit to what you can achieve ... if you don’t care who gets the credit." Roger Abbott
Understood... actually, it was some of the subsequent responses that seemed to get a bit of the subject. Thanks for replying.


Mainframe FOCUS 7.0
VM/CMS and MVS/TSO