We need to setup an iWay Data Migrator process to move data from several tables in Oracle DB to DataWarehouse on daily basis.

If a source table contains a Datetime column, we can base on the date time to identify and easily move these records.
However, when a source table doesn't contain any date time column, what would be a proper way to select the newly inserted/updated records?

One way to solve this is to build triggers on the source tables to keep track of what being insert/update. Then iWay will base on this info to move data. This will work but i don't think this is a good approach.

Could you recommend any alternatives? Thanks a lot.

This message has been edited. Last edited by: Tamra,

You'll have to double check the manual but I belive that Oracle supports Change Data Capture.

Then you can set up DM to only get the changed data.

If that doesn't work then one hack is to create hashes of the concatenated fields you are interested in and compare that against your warehouse - not exactly efficient but much better than comparing field to field.

When I've used this I've added the hash as a separate field in the warehouse to speed up comparison.

Cheers

Stu

Hi David,

My suggestion to handle your ETL to tables without Datetime column is:

- Access your DW table, create a HOLD with IDs or max PK ( If its is auto increment );
- Access your Oracle DB filtering IDs NE (HOLD) or PK GT (HOLD)

I hope it helps you. Good lucky!

There is an optional add-on to DataMigrator that we call Change Data Capture that reads from the ORACLE Archive Logs to identify rows that have changed (insert, update or delete). It's documented in the Data Migrator User's Guide, Chapter 12, "Change Data Capture."

Note that for an ORACLE source it's only required that full logging is enabled. It does not require what ORACLE calls Change Data Capture (which is being deprecated) or Golden Gate.

While CDC is available in Release 7704 it's been improved since that release was first posted. I'd recommend using the current production Release 7706.

This message has been edited. Last edited by: Clif,

Thanks a lot for all of your suggestion. I really appreciate it.

I'm taking a look at using the Change Data Capture Method in iWay 7704 for now.

We are also in a process of upgrading to iWay 7706 this year.