Focal Point Banner


As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only.

Join the TIBCO Community
TIBCO Community is a collaborative space for users to share knowledge and support one another in making the best use of TIBCO products and services. There are several TIBCO WebFOCUS resources in the community.

  • From the Home page, select Predict: WebFOCUS to view articles, questions, and trending articles.
  • Select Products from the top navigation bar, scroll, and then select the TIBCO WebFOCUS product page to view product overview, articles, and discussions.
  • Request access to the private WebFOCUS User Group (login required) to network with fellow members.

Former myibi community members should have received an email on 8/3/22 to activate their user accounts to join the community. Check your Spam folder for the email. Please get in touch with us at community@tibco.com for further assistance. Reference the community FAQ to learn more about the community.


Focal Point    Focal Point Forums  Hop To Forum Categories  iWay Software Product Forum on Focal Point    [SOLVED]Insert Records From Memory is Slow

Read-Only Read-Only Topic
Go
Search
Notify
Tools
[SOLVED]Insert Records From Memory is Slow
 Login/Join
 
DeeJay
Member
posted
Inserting 17 million records into table with only a clustered index on identity column is taking 66 minutes. Not doing any transforms, literally just selecting all columns (approx. 100) from source table for insertion into target. Source and target are the same layout.

*Load Type = Insert Records From Memory
*Commit every row(s) = Off
*Block size = 10000


For comparison purposes I copied the SQL out of Data Migrator and pasted into SQL Server Management Studio. Added the INSERT INTO portion and executed. This completes in 6 minutes.

Any ideas on why Data Migrator is so slow?

Thanks.

This message has been edited. Last edited by: Tamra,


WebFOCUS 7.7.06
Windows, All Outputs
 
Posts: 6 | Location: Des Moines, IA | Registered: April 10, 2014Report This Post
Clif
Guru
posted Hide Post
DataMigrator doesn't currently generate INSERT INTO ... SELECT commands in MS SQL Server.

When DataMigrator was originally developed it was mostly used to copy data from one database to another. So the emphasis was on approaches that would allow any source (indeed multiple sources) and targets. Not all databases supported SELECT ... INTO or INSERT INTO ... SELECT commands (although now most do) but by definition that would only work for a source and target in the same database so we developed alternatives including Insert Records from Memory, Bulk Load, Extended Bulk Load to take advantage of different databases capabilities.

The forthcoming server Release 7.7 adds a limited capability in the Web Console to do a custom copy to a new table in the same database using INSERT INTO ... SELECT.

If you would like to see such functionality incorporated into a future release of DataMigrator please open a Hottrack case as a New Feature Request.


N/A
 
Posts: 397 | Location: New York City | Registered: May 03, 2007Report This Post
DeeJay
Member
posted Hide Post
Thank you for your response. I was just thinking an insert from memory in data migrator and an insert into select from in sql server were basically doing the same thing.... selecting records from a source and inserting into a target. I just didn't understand how one can be 11 times faster than the other when all it's doing is moving data from one table to another.


WebFOCUS 7.7.06
Windows, All Outputs
 
Posts: 6 | Location: Des Moines, IA | Registered: April 10, 2014Report This Post
  Powered by Social Strata  

Read-Only Read-Only Topic

Focal Point    Focal Point Forums  Hop To Forum Categories  iWay Software Product Forum on Focal Point    [SOLVED]Insert Records From Memory is Slow

Copyright © 1996-2020 Information Builders