As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only.
Join the TIBCO Community TIBCO Community is a collaborative space for users to share knowledge and support one another in making the best use of TIBCO products and services. There are several TIBCO WebFOCUS resources in the community.
From the Home page, select Predict: WebFOCUS to view articles, questions, and trending articles.
Select Products from the top navigation bar, scroll, and then select the TIBCO WebFOCUS product page to view product overview, articles, and discussions.
Request access to the private WebFOCUS User Group (login required) to network with fellow members.
Former myibi community members should have received an email on 8/3/22 to activate their user accounts to join the community. Check your Spam folder for the email. Please get in touch with us at community@tibco.com for further assistance. Reference the community FAQ to learn more about the community.
I'm wanting to submit a report (deferred) for a large extract (27GB) and write the output directly to a shared drive (not on the webfocus server).
I can do this successfully using the correct FILDEF, for smaller sizes and it works fine. What I don't want to happen is for webfocus to spool (or make an internal matrix) to the webfocus server's drive first, then move the output to the shared drive.
How can I avoid this or know this is NOT going to happen? Does FILEDEF on the HOLD ddname make all the work take place on the shared drive? Will I be forced to use TABLEF ?
The webfocus server's total storage is 30GB (permanent and temp). So, if the large job was run, it would crash the webfocus server.
Hope I'm explaining well enough.
Thanks, --wgThis message has been edited. Last edited by: Kerry,
WF 8009m, Clustered vm Windows2008r2 reporting servers; Web interface: tomcat; Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005
The only way to not create an internal matrix is to use TABLEF. Any other method will create the infamous focsort file - the matrix - and in that case you'll need the extra storage. Fortunately focsort is not limited to 2GB anymore ...
GamP
- Using AS 8.2.01 on Windows 10 - IE11.
in Focus since 1988
Posts: 1961 | Location: Netherlands | Registered: September 25, 2007
the easiest way is probable to add a larger or additional disk to the server. A couple of TB does not cost very much. Most likely cheaper than tweaking the code.
I was hoping someone would say I could FILEDEF FOCSORT, HOLDMAS, FOCTEMP and/or whatever is involved to the shared drive, for this one fex and not crash our server.
Our disks drives must be made of gold/diamonds/oil because they're very expensive.
Our servers are not a single physical box (wish they were).
Thanks, --wg
WF 8009m, Clustered vm Windows2008r2 reporting servers; Web interface: tomcat; Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005
Our disks drives must be made of gold/diamonds/oil because they're very expensive.
The expensive part is likely the team of people paid to get up and fix things in the middle of the night when something goes bump. As well as the service contract with the vendor that will be there with new hardware before the dew burns off.
To your original point, I don't know if this will help you at all, but we had a job that kept 11GB of held files and they were all essentially massages of the previous data or joins to external files -- nothing that had to be reused later in the program. this was also filling up our disk and crashing the machine (and boy is WebFOCUS ugly when you run out of disk space). I wrote a routine to delete the temp file from the previous step and a process document for how to use it (below). Again, your needs are a bit different.
Is it something that you could write out pieces of data and then stitch them back up at the OS level (thinking of a CAT or TYPE command)?
quote:
Process: Deleting Hold Files As Your Program Runs
Background: As a Focus Executable runs, it creates a directory on the server to hold all the data it gathers for immediate and later usage. These files are created as HOLDNAME.ftm (where HOLDNAME is the name you gave it in the 'ON TABLE HOLD AS HOLDNAME' statement). These files can be quite large and can add up in the space consumed on the server. On more than one occasion, we have had situations where all the hold files together caused the server to run out of disk space.
Beginning immediately, it will be considered a best programming practice to clean up large hold files during program execution.
File: Server > Apps > Standards > DELHOLD.FEX
The program is documented well and fills in some blanks.
When to Use: Use this in programs where you will generate large datasets and have the potential to run for a long time. Only use this cleanup process when you are done with the file and will not be joining it back or using it again (i.e. via TABLE FILE).
Usage:
TABLE FILE CAR
PRINT
*
ON TABLE HOLD AS FILE1
END
-RUN
-* THE &VERBOSE_DELHOLDFILE PARAMETER IS OPTIONAL. IF YOU DON'T WANT
-* SYSTEM MESSAGES BACK, YOU CAN LEAVE IT OUT.
-SET &VERBOSE_DELHOLDFILE = 'ON';
-* THIS VARIABLE MUST MATCH THE HOLDFILE NAME YOU WANT TO DELETE!!!
-SET &HOLDFILETODELETE = 'FILE1';
-MRNOEDIT -INCLUDE DELHOLD
DELHOLD.fex:
-*******************************************************************************
-* DELHOLD.FEX
-*******************************************************************************
-* AUTHOR: Blake Thompson
-*
-* DATE: 20110715
-*
-* PURPOSE: Keep temporary agent directories clean of old hold files during
-* long running processes or processes that consume lots of diskspace.
-*
-*******************************************************************************
-* REQUIREMENTS
-*******************************************************************************
-* SET THE &HOLDFILETODELETE variable with the name of the holdfile you specified
-* in your program.
-*
-* You MUST set a -RUN before calling this (i.e. after your ON TABLE HOLD statement).
-*
-*******************************************************************************
-* USAGE:
-*******************************************************************************
-*
-* TABLE FILE CAR
-* PRINT
-* *
-* ON TABLE HOLD AS FILE1
-* END
-* -RUN
-*
-* -SET &HOLDFILETODELETE = 'FILE1';
-* -MRNOEDIT -INCLUDE DELHOLD
-*
-*******************************************************************************
-* OPTIONS:
-*******************************************************************************
-* -SET &VERBOSE_DELHOLDFILE = 'ON';
-* TO SEE SYSTEM MESSAGES (GOOD FOR DEBUG)
-*
-* YOU CAN IGNORE THE VARIABLE IF YOU YOU DON'T WANT FEEDBACK.
-*
-*******************************************************************************
-*TABLE FILE CAR
-*PRINT
-**
-*ON TABLE HOLD AS FILE1
-*END
-*-RUN
-DEFAULT &VERBOSE_DELHOLDFILE = 'OFF';
-*-SET &HOLDFILETODELETE = 'FILE1';
-SET &HOLDFILETODELETE = &HOLDFILETODELETE | '.ftm';
-IF &VERBOSE_DELHOLDFILE = 'ON' THEN GOTO DOMSG1 ELSE GOTO SKIPMSG1;
-DOMSG1
-WINNT echo =========BEFORE============
-WINNT cd
-WINNT dir /b
-SKIPMSG1
-WINNT del &HOLDFILETODELETE
-IF &VERBOSE_DELHOLDFILE = 'ON' THEN GOTO DOMSG2 ELSE GOTO SKIPMSG2;
-DOMSG2
-WINNT echo.
-WINNT echo =========AFTER=============
-WINNT cd
-WINNT dir /b
-SKIPMSG2
Zip wouldn't work, because it would first spool to the webfocus srv drive (it won't fit)!
ABT, I to have had to do this for a multiple step job. In between steps, had to cleanup hold files, otherwise the server would nearly crash (crawl).
The job that I'm faced with is very large single query (sql pass-thru) that returns a single hold file. I need to be able to tell webfocus to work on the shared drive (for this single job).
If it's not possible then I'll need to increase my storage on the webfocus servers.
However, 30GB has worked well since 2004!
--wg
WF 8009m, Clustered vm Windows2008r2 reporting servers; Web interface: tomcat; Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005
shooting from the hip, i think you may be stuck with alternatives. I'd wait from the final word from IBI or someone smarter/more experienced than I am, but it seems like you may have to run it directly on Oracle and save the output as a file. In SQL Server, you'd use DTS or SSMS packages, not sure what the Oracle equivalent is. The pass through SQL will process entirely on the remote server. You'd have to hold it there and devise a way to get it to the destination. Maybe a timed cron or windows scheduler job?
We could always do that, but means I'd have to be involved. I'm really wanting the end-user to control this, since they are familiar with the TAX data they're extracting.
You may be right that webfocus is NOT the tool for this kind of job.... ugh.
Thanks, --wg
WF 8009m, Clustered vm Windows2008r2 reporting servers; Web interface: tomcat; Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005