Focal Point Banner


As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only.

Join the TIBCO Community
TIBCO Community is a collaborative space for users to share knowledge and support one another in making the best use of TIBCO products and services. There are several TIBCO WebFOCUS resources in the community.

  • From the Home page, select Predict: WebFOCUS to view articles, questions, and trending articles.
  • Select Products from the top navigation bar, scroll, and then select the TIBCO WebFOCUS product page to view product overview, articles, and discussions.
  • Request access to the private WebFOCUS User Group (login required) to network with fellow members.

Former myibi community members should have received an email on 8/3/22 to activate their user accounts to join the community. Check your Spam folder for the email. Please get in touch with us at community@tibco.com for further assistance. Reference the community FAQ to learn more about the community.



Read-Only Read-Only Topic
Go
Search
Notify
Tools
Bulk Insert
 Login/Join
 
vaayu
Guru
posted
I need to update tables (sql server2k) on a frequent basis which are pretty huge(about 80Mil rows)
Right now, I use a fex which takes the data from a table(40mil rows)and does all the necessary calculations and holds as .ftm file then I use MODIFY FILE.. ..ON NOMATCH INCLUDE ON MATCH UPDATE to insert into that huge table. But, these tables keep growing every month so I need to come up with an optimum solution
Can anyone tell me if there is any effective/efficient solution for this kind of situation? I hope I'm not confusing anyone.

Thanks
 
Posts: 289 | Location: Houston,TX | Registered: June 11, 2004Report This Post
<Grzegorz>
posted
As I know MODIFY performs very well, but on the hierarchical databases (especially FOCUS on MVS).
With the relational interfaces (including MS SQL) MODIFY INCLUDE syntax is translated to the series of INSERT statement which is not very effective for the large number of rows (I think it is not intended to use in such cases). It is better to use "native bulk load utility", in this case MS bcp utility, or MS SQL script/procedure with the BULK INSERT instruction. If memory serves me iWay ETL Manager uses bcp for MS SQL bulk load. You can invoke bcp with the ! syntax:


TABLE FILE LARGE_TABLE
PRINT *
ON TABLE SAVE AS LDFILE FORMAT COM
END
-RUN
! bcp with-the-appropriate-options

I do not remember the options and syntax of these utilities, but you can find information and examples with the Google, or on the Microsoft sites and "MS SQL Books Online".
An article about loading data into SQL Server

Another option is to SAVE the file to the location which is "known" for the MS SQL and use MS SQL DTS to load the data.

Hope this helps
Grzegorz

This message has been edited. Last edited by: <Mabel>,
 
Report This Post
vaayu
Guru
posted Hide Post
I am still confused with the syntax with using bcp. This is what I found

bcp [[database_name.]owner.]table_name {in | out} datafile
[/m maxerrors] [/f formatfile] [/e errfile]
[/F firstrow] [/L lastrow] [/b batchsize]
[/n] [/c] [/E]
[/t field_term] [/r row_term]
[/i inputfile] [/o outputfile]
/U login_id [/P password] [/S servername] [/v] [/a packet_size]

but, my question is if I save my data as HOLD FILE FORMAT COM, how do I tell bcp (the datafile path). May be Im confused bcos I never used COM format to HOLD a file.
Please Help me in uderstanding the situation

Thanks
 
Posts: 289 | Location: Houston,TX | Registered: June 11, 2004Report This Post
GCohen
Platinum Member
posted Hide Post
The WebFocus ETL solution uses Bulk Loading.
Since you have many of the WebFocus components
ETL would be an easy install.
 
Posts: 226 | Registered: June 08, 2003Report This Post
  Powered by Social Strata  

Read-Only Read-Only Topic


Copyright © 1996-2020 Information Builders