Focal Point Banner


As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only.

Join the TIBCO Community
TIBCO Community is a collaborative space for users to share knowledge and support one another in making the best use of TIBCO products and services. There are several TIBCO WebFOCUS resources in the community.

  • From the Home page, select Predict: WebFOCUS to view articles, questions, and trending articles.
  • Select Products from the top navigation bar, scroll, and then select the TIBCO WebFOCUS product page to view product overview, articles, and discussions.
  • Request access to the private WebFOCUS User Group (login required) to network with fellow members.

Former myibi community members should have received an email on 8/3/22 to activate their user accounts to join the community. Check your Spam folder for the email. Please get in touch with us at community@tibco.com for further assistance. Reference the community FAQ to learn more about the community.


Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     [CLOSED] Internals question

Read-Only Read-Only Topic
Go
Search
Notify
Tools
[CLOSED] Internals question
 Login/Join
 
Gold member
posted
Hello,

I'm wanting to submit a report (deferred) for a large extract (27GB) and write the output
directly to a shared drive (not on the webfocus server).

I can do this successfully using the correct FILDEF, for smaller sizes and it works fine.
What I don't want to happen is for webfocus to spool (or make an internal matrix) to the
webfocus server's drive first, then move the output to the shared drive.

How can I avoid this or know this is NOT going to happen?
Does FILEDEF on the HOLD ddname make all the work take place on the shared drive?
Will I be forced to use TABLEF ?

The webfocus server's total storage is 30GB (permanent and temp). So, if the large job
was run, it would crash the webfocus server.

Hope I'm explaining well enough.

Thanks,
--wg

This message has been edited. Last edited by: Kerry,


WF 8009m, Clustered vm Windows2008r2 reporting servers;
Web interface: tomcat;
Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
 
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005Report This Post
Virtuoso
posted Hide Post
The only way to not create an internal matrix is to use TABLEF. Any other method will create the infamous focsort file - the matrix - and in that case you'll need the extra storage. Fortunately focsort is not limited to 2GB anymore ...


GamP

- Using AS 8.2.01 on Windows 10 - IE11.
in Focus since 1988
 
Posts: 1961 | Location: Netherlands | Registered: September 25, 2007Report This Post
Guru
posted Hide Post
Winfred

the easiest way is probable to add a larger or additional disk to the server. A couple of TB does not cost very much. Most likely cheaper than tweaking the code.

Håkan


WebFOCUS DS 8.0.06/08 DS/AS
WebFOCUS RS 8.0.08 (Linux/IBM i)
WebFOCUS Client 8.0.06 (Linux)
 
Posts: 319 | Location: Stockholm, Sweden | Registered: February 04, 2004Report This Post
Gold member
posted Hide Post
Thanks Guys,

I was hoping someone would say I could FILEDEF FOCSORT, HOLDMAS, FOCTEMP and/or whatever is involved to the shared drive, for this one fex and
not crash our server.

Our disks drives must be made of gold/diamonds/oil because they're very expensive.

Our servers are not a single physical box (wish they were).

Thanks,
--wg


WF 8009m, Clustered vm Windows2008r2 reporting servers;
Web interface: tomcat;
Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
 
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005Report This Post
Expert
posted Hide Post
OR Zip it up, FTP the zip file to the Shared drive, and have the OS UNzip it...


Tom Flynn
WebFOCUS 8.1.05 - PROD/QA
DB2 - AS400 - Mainframe
 
Posts: 1972 | Location: Centennial, CO | Registered: January 31, 2006Report This Post
Master
posted Hide Post
quote:
Our disks drives must be made of gold/diamonds/oil because they're very expensive.

The expensive part is likely the team of people paid to get up and fix things in the middle of the night when something goes bump. As well as the service contract with the vendor that will be there with new hardware before the dew burns off.

To your original point, I don't know if this will help you at all, but we had a job that kept 11GB of held files and they were all essentially massages of the previous data or joins to external files -- nothing that had to be reused later in the program. this was also filling up our disk and crashing the machine (and boy is WebFOCUS ugly when you run out of disk space). I wrote a routine to delete the temp file from the previous step and a process document for how to use it (below). Again, your needs are a bit different.

Is it something that you could write out pieces of data and then stitch them back up at the OS level (thinking of a CAT or TYPE command)?

quote:

Process: Deleting Hold Files As Your Program Runs

Background:
As a Focus Executable runs, it creates a directory on the server to hold all the data it gathers for immediate and later usage. These files are created as HOLDNAME.ftm (where HOLDNAME is the name you gave it in the 'ON TABLE HOLD AS HOLDNAME' statement). These files can be quite large and can add up in the space consumed on the server. On more than one occasion, we have had situations where all the hold files together caused the server to run out of disk space.

Beginning immediately, it will be considered a best programming practice to clean up large hold files during program execution.


File:
Server > Apps > Standards > DELHOLD.FEX

The program is documented well and fills in some blanks.


When to Use:
Use this in programs where you will generate large datasets and have the potential to run for a long time. Only use this cleanup process when you are done with the file and will not be joining it back or using it again (i.e. via TABLE FILE).


Usage:
TABLE FILE CAR
PRINT
*
ON TABLE HOLD AS FILE1
END
-RUN

-* THE &VERBOSE_DELHOLDFILE PARAMETER IS OPTIONAL.  IF YOU DON'T WANT 
-* SYSTEM MESSAGES BACK, YOU CAN LEAVE IT OUT.
-SET &VERBOSE_DELHOLDFILE = 'ON';

-* THIS VARIABLE MUST MATCH THE HOLDFILE NAME YOU WANT TO DELETE!!!
-SET &HOLDFILETODELETE    = 'FILE1';

-MRNOEDIT -INCLUDE DELHOLD



DELHOLD.fex:
-*******************************************************************************
-* DELHOLD.FEX
-*******************************************************************************
-* AUTHOR:  Blake Thompson
-*
-* DATE:    20110715
-*
-* PURPOSE: Keep temporary agent directories clean of old hold files during
-*          long running processes or processes that consume lots of diskspace.
-*
-*******************************************************************************
-* REQUIREMENTS
-*******************************************************************************
-* SET THE &HOLDFILETODELETE variable with the name of the holdfile you specified
-* in your program.
-*
-* You MUST set a -RUN before calling this (i.e. after your ON TABLE HOLD statement).
-*
-*******************************************************************************
-* USAGE:
-*******************************************************************************
-*
-*  TABLE FILE CAR
-*  PRINT
-*  *
-*  ON TABLE HOLD AS FILE1
-*  END
-*  -RUN
-*
-*  -SET &HOLDFILETODELETE = 'FILE1';
-*  -MRNOEDIT -INCLUDE DELHOLD
-*
-*******************************************************************************
-* OPTIONS:
-*******************************************************************************
-* -SET &VERBOSE_DELHOLDFILE = 'ON';
-* TO SEE SYSTEM MESSAGES (GOOD FOR DEBUG)
-*
-* YOU CAN IGNORE THE VARIABLE IF YOU YOU DON'T WANT FEEDBACK.
-*
-*******************************************************************************
 
 
-*TABLE FILE CAR
-*PRINT
-**
-*ON TABLE HOLD AS FILE1
-*END
-*-RUN
 
-DEFAULT &VERBOSE_DELHOLDFILE = 'OFF';
 
-*-SET &HOLDFILETODELETE = 'FILE1';
-SET &HOLDFILETODELETE = &HOLDFILETODELETE | '.ftm';
 
-IF &VERBOSE_DELHOLDFILE = 'ON' THEN GOTO DOMSG1 ELSE GOTO SKIPMSG1;
-DOMSG1
-WINNT echo =========BEFORE============
-WINNT cd
-WINNT dir /b
-SKIPMSG1
 
 
-WINNT del &HOLDFILETODELETE
 
 
-IF &VERBOSE_DELHOLDFILE = 'ON' THEN GOTO DOMSG2 ELSE GOTO SKIPMSG2;
-DOMSG2
-WINNT echo.
-WINNT echo =========AFTER=============
-WINNT cd
-WINNT dir /b
-SKIPMSG2


------------------------------------
WF Environment:
------------------------------------
Server/Client, ReportCaster, Dev Studio: 7.6.11
Resource Analyzer, Resource Governor, Library, Maintain, InfoAssist
OS: Windows Server 2003
Application/Web Server: Tomcat 5.5.25
Java: JDK 1.6.0_03
Authentication: LDAP, MRREALM Driver
Output: PDF, EXL2K, HTM

------------------------------------
Databases:
------------------------------------
Oracle 10g
DB2 (AS/400)
MSSQL Server 2005
Access/FoxPro
 
Posts: 561 | Registered: February 03, 2010Report This Post
Gold member
posted Hide Post
Thanks Tom and ABT,

Zip wouldn't work, because it would first spool to the webfocus srv drive (it won't fit)!

ABT, I to have had to do this for a multiple step job. In between steps, had to cleanup hold files, otherwise the server would nearly crash (crawl).

The job that I'm faced with is very large single query (sql pass-thru) that returns a single hold file. I need to be able to tell webfocus to work on the shared drive (for this single job).

If it's not possible then I'll need to increase my storage on the webfocus servers.

However, 30GB has worked well since 2004!

--wg


WF 8009m, Clustered vm Windows2008r2 reporting servers;
Web interface: tomcat;
Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
 
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005Report This Post
Master
posted Hide Post
shooting from the hip, i think you may be stuck with alternatives. I'd wait from the final word from IBI or someone smarter/more experienced than I am, but it seems like you may have to run it directly on Oracle and save the output as a file. In SQL Server, you'd use DTS or SSMS packages, not sure what the Oracle equivalent is. The pass through SQL will process entirely on the remote server. You'd have to hold it there and devise a way to get it to the destination. Maybe a timed cron or windows scheduler job?


------------------------------------
WF Environment:
------------------------------------
Server/Client, ReportCaster, Dev Studio: 7.6.11
Resource Analyzer, Resource Governor, Library, Maintain, InfoAssist
OS: Windows Server 2003
Application/Web Server: Tomcat 5.5.25
Java: JDK 1.6.0_03
Authentication: LDAP, MRREALM Driver
Output: PDF, EXL2K, HTM

------------------------------------
Databases:
------------------------------------
Oracle 10g
DB2 (AS/400)
MSSQL Server 2005
Access/FoxPro
 
Posts: 561 | Registered: February 03, 2010Report This Post
Gold member
posted Hide Post
ABT,

We could always do that, but means I'd have to be involved. I'm really wanting the end-user to control this, since they are familiar with the TAX data they're extracting.

You may be right that webfocus is NOT the tool for this kind of job.... ugh.

Thanks,
--wg


WF 8009m, Clustered vm Windows2008r2 reporting servers;
Web interface: tomcat;
Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
 
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005Report This Post
Expert
posted Hide Post
What if you set APP HOLD to the shared drive ?

Does the SQLOUT file go there are well ?

Combined with TABLEF and no BY fields, should stop and sort files fromm being produced and the temp files won't be local.


Waz...

Prod:WebFOCUS 7.6.10/8.1.04Upgrade:WebFOCUS 8.2.07OS:LinuxOutputs:HTML, PDF, Excel, PPT
In Focus since 1984
Pity the lost knowledge of an old programmer!

 
Posts: 6347 | Location: 33°49'23.0"S, 151°11'41.0"E | Registered: October 31, 2006Report This Post
Guru
posted Hide Post
What about having Oracle create a sorted temp table which will sit on the db server and the run a TABLEF from. Would that help?

Håkan


WebFOCUS DS 8.0.06/08 DS/AS
WebFOCUS RS 8.0.08 (Linux/IBM i)
WebFOCUS Client 8.0.06 (Linux)
 
Posts: 319 | Location: Stockholm, Sweden | Registered: February 04, 2004Report This Post
Gold member
posted Hide Post
Waz and Hakan,

Thanks.... both good ideas.
I'll try the APP HOLD and see.

More when I know,
--wg


WF 8009m, Clustered vm Windows2008r2 reporting servers;
Web interface: tomcat;
Output: EXCEL, HTML, PDF; dbms: Oracle 10, db2 on mvs, mssql
 
Posts: 81 | Location: Monroe LA | Registered: January 07, 2005Report This Post
  Powered by Social Strata  

Read-Only Read-Only Topic

Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     [CLOSED] Internals question

Copyright © 1996-2020 Information Builders