This most likely has been asked before, but I couldn't find it.

Here is the scenario: On Sunday, I created a data file and held it aside with a name of PREV (for Previous day's data). On Monday, I again create a file, but this one contains both Sunday's data and additional data for today. I want to be able to compare the file from Sunday to the file from Monday, and output only the records that are new for Monday. I then want to append the new Monday records to the Sunday file, so that on Tuesday I will then compare the Tuesday file against the records in the combined Sunday-Monday file, and so on into the future. In other words I want to be able to do an ongoing data compare and only produce a resultant new records file each day.

My original Sunday file will be a flat file. with an MFD that has a format=FIX.

Can anyone provide help or guidance on what needs doing? Better yet, if there is a sample out there, can you point me to it?

Thanks very much!

This message has been edited. Last edited by: Kerry,

Just two words on this: "MATCH FILE".

Um.... perhaps a bit fuller explanation would be more helpful....

Um... how about a bit of digging into the documentation?

Creating Reports With WebFOCUS Language > Merging Data Sources > Merging Data

Um... thanks.

Greaseman,

Just a thought. Why would you want to append Monday's new records to the Sunday file in order to compare the Tuesday file with the appended file? Wouldn't the appended file be exactly the Monday file?

Danny,

In a way, you are correct, but here is an outline of my entire work flow, that might help to explain what I'm interested in doing:

01) Create Sunday file
02) On Monday, run program to extract all data plus any new data.
03) Compare Sunday File against Monday file.
04) Pull off unmatching records from Monday file into a "Difference" file.
05) Make an output Excel file of the unmatched records for subsequent FTP-ing to a server.
06) Append the unmatched records to the Sunday file.

07) On Tuesday, repeat the process.

The user is only interested in the unmatched records that get FTP-ed to the server.

I may be confusing myself, but my thinking is that the Sunday file would be a good baseline to start with and then to process each subsequent day's data against what I started with.

Is how I'm planning this a good approach, or should I maybe try doing this in a different way? I'm open to suggestions.

Thank you for writing.

I would think Tuesday is just Monday plus anything new ... but I don't know your data. MATCH FILE is definitely the way to go.

MATCH FILE SUNDAY
SUM
  V1
  V2
  V3
BY V4
BY V5
FILE MONDAY
SUM
  V1
  V2
  V3
BY V4
BY V5
AFTER MATCH HOLD AS DIFF_MATCH NEW-NOT-OLD
END

This will give you the difference file. You can create Excel file from DIFF_MATCH.

Oops I left out a RUN ...

 MATCH FILE SUNDAY
SUM
  V1
  V2
  V3
BY V4
BY V5
RUN
FILE MONDAY
SUM
  V1
  V2
  V3
BY V4
BY V5
AFTER MATCH HOLD AS DIFF_MATCH NEW-NOT-OLD
END
 

After the process explained by "M Meagher", you will need a "MORE" command to append and get a single file. You might be knowing this, but I thought I should post it.

Um... Thanks Francis and M Meagher for your support on my "Hint" towards the solution...

"04) Pull off unmatching records from Monday file into a "Difference" file." Sounds like a MATCH FILE to me...

Another thought: Don't the Sunday and Monday records have some timestamps to ID them? Can those timestamps be used in this quandary?

Question.... will the way I've got my logic described in my response to Danny work? Opinions? (And please...without the "um..." which I used once since I thought it was best at the time to write like I would talk).

As long as user is only interested in the unmatched records, the logic explained will work.

Understood.... however my question wa would the logic I outlined in my response to Danny also work?

Greaseman,

Logically yes, although I would be inclined to change 6) to replace Sunday file with Monday file - less processing involved and produces the same result.

T

Oops! I believe I left off a big part of my logic flow.... where I said the process repeats on Tuesday, it also gets repeated on Wednesday, Thursday and Friday..in other words, the whole process repeats every day of the week.

That's why I thought it would be easier to make a base file on Sunday and then add to that base file each subsequent day, as opposed to changing the program or JCL to refer to a different file each day.

Thanks for replying.

Are you just trying to do a change data capture routine. compare file 1 with file 2 and mark records as I insert for new, D Deleted, C changed or blank for no change.

Does anyone use SET ALL=PASS anymore?

Bonjour,
SET ALL=ON, SET ALL=OFF, SET ALL=PASS made me sweat a great deal, because somebody who didn't have a telephone number was forgotten in my lists of people who owed more than XXXX Francs (at the time).
A nice and easy way not to pay one's debt.
At the time, a specialist of IBF, took the time to explain me everything about those powerful Sets ( Including Useless on SEGTYPE = U )
Those Sets are obviously Focus+ level. It does not at all mean they are Useless.
They are somewhere in the very Heart of Focus and it's way to turn into Procedural Action its Global and Biblic langage ( "Reading and Filtering Data : How the internal Matrix is constituted" - Myownmemory.My-Important-Focuseries.fr )
Cordially and Focusely
PS : Since when are the LEFT Outer Join available in DB2 ? Our Focus Sets are older

I'm not saying I use it every day, but it is certainly not a forgotten setting in my book.
Why do you ask?

In my experience, SET ALL=PASS is a mess with DB2. It does not generate the correct SQL, even with SQL DB2 SET SQLJOIN OUTER ON.

I might be off base and if I am, I apologize. However, the last ferw posting seem to gotten away from the original subject of this posting. Can we get back to the posting's issue, please?

Here is the thrust of the original posting:

01) Create Sunday file
02) On Monday, run program to create a Monday file.
03) Compare Sunday File against Monday file.
04) Pull off unmatching records from Monday file into a "Difference" file.
05) Make an output Excel file of the unmatched records for subsequent FTP-ing to a server.
06) Append the unmatched records from the Monday file to the Sunday file.

07) On Tuesday, repeat the process, this time using the Tuesady file agaisnt the Sunday file.

08) On Wednesday, repeat the process, this time, using the Wednesday file against the Sunday file.

09) On Thrusday, repeat the process, this time using the Thursday file against the Friday file.

10) On Friday, repeat the process, this time using the Friday file against the Sunday file.

11) The following week, DO NOT create a new Sunday file... we're going to utilize the original Sunday file.

12) On Monday of the following week, compare the following week's Monday file against the original Sunday file.

13) On Tuesday, repeat the process.

14) In other words, once an original Sunday file has been created, then each subsequent day's file will be processed against it.

Hopefully this clears up some of the confusions my posting seems to have created.

Thank you to all who have responded.

I wasn't going away from the subject. Example:

-* example sunday file.
TABLE FILE CAR
SUM
     DEALER_COST
     RETAIL_COST
     SALES
BY COUNTRY
BY CAR
BY MODEL
WHERE MODEL OMITS 'AUTO';
ON TABLE HOLD AS SUNDAY FORMAT ALPHA
END

-* example monday file.
TABLE FILE CAR
SUM
     DEALER_COST
     RETAIL_COST
     SALES
BY COUNTRY
BY CAR
BY MODEL
ON TABLE HOLD AS MONDAY FORMAT ALPHA
END

-* join 'em ... note the all
JOIN 
	COUNTRY AND CAR AND MODEL IN MONDAY TO ALL
	COUNTRY AND CAR AND MODEL IN SUNDAY AS J1.
END

-* set all to pass
SET ALL=PASS

TABLE FILE MONDAY
PRINT MONDAY.DEALER_COST MONDAY.RETAIL_COST MONDAY.SALES
BY MONDAY.COUNTRY 
BY MONDAY.CAR
BY MONDAY.MODEL
-* enforce the all=pass with a B/S condition
WHERE SUNDAY.COUNTRY EQ 'NOTHING';
END
  

Understood... actually, it was some of the subsequent responses that seemed to get a bit of the subject. Thanks for replying.