Hi all, I have a performance issue with a Slowly Changing Dimension flow taking an unacceptable amount of time to load; and i'm looking for some feedback/advice.
It is a relatively simple flow, with one source file with only 63000 records.
The target table has 1.3 million rows.
I have 4 fields designated as logical key fields (Company, Program, Prefix Code, and Unit#).
Does the flow process have to read every record in the target table? every record for each logical? (1.3 mill X 4)?
Would it be beneficial to create one Unique ID in the source to use as "1" logical key field to the target (create the same Unique ID in Target)?
Does anyone have any similar experiences, troubleshooting, etc..?
thank you, James GoldenThis message has been edited. Last edited by: Kerry,
WebFocus Developer Studio v. 7610 MySQL/Linux All different outputs including map technology and using the DMC for our ETL process as well
July 06, 2009, 05:37 PM
Darin Lee
You might get a better answer if you post this to the iWay Forum where the ETL users mingle. I have a little experience with our ETL processes but not enough to answer your question. Sorry.
Regards,
Darin
In FOCUS since 1991 WF Server: 7.7.04 on Linux and Z/OS, ReportCaster, Self-Service, MRE, Java, Flex Data: DB2/UDB, Adabas, SQL Server Output: HTML,PDF,EXL2K/07, PS, AHTML, Flex WF Client: 77 on Linux w/Tomcat
July 07, 2009, 08:23 AM
jgolden
thanks for the feedback
WebFocus Developer Studio v. 7610 MySQL/Linux All different outputs including map technology and using the DMC for our ETL process as well
July 07, 2009, 10:01 AM
<JG>
quote:
key
Darin is quite correct, Wrong forum, but INDEX for the purpose is the answer.