Focal Point
[CLOSED] Incremental loading

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/1381057331/m/8217048086

February 18, 2016, 09:14 AM
Aimar Arak
[CLOSED] Incremental loading
HI,
I have table on my MySQL source for which I have to create flow to incrementally load data once a day. How can I implement it as I have timestamp column there for this purpose.

This message has been edited. Last edited by: Tamra,


WebFOCUS 8.2.XX
February 22, 2016, 08:25 AM
Clif
If I understand what you want to do you could use a WHERE condition with a sub-select to find the last timestamp in the target table.

sourcetimestamp > SELECT MAX(targettimestamp) FROM targettable

Does that do it?

I suspect that MySQL will run that sub-select for every row of your source which may be slow for large data volumes. If so that could be avoided by obtaining the filter value first. Do you need do that?


N/A
February 23, 2016, 03:33 AM
Aimar Arak
How can I do it if my target DBMS is Postgre? I cannot use data from two different adapters in SQL flow. Or can I?


WebFOCUS 8.2.XX
February 23, 2016, 09:56 AM
Clif
Not in a DBMS SQL Flow. In a Data Flow? Of course you can!


N/A