Focal Point
Max No Records WF can Scan Through

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/5911014562

January 10, 2008, 09:00 AM
TCI
Max No Records WF can Scan Through
Hi,
I would like to know what is the optimal number of records that webfocus can scan through. What are the implications if i have WF scan through 500 Million Records ?

Thanks,
TCI


WF 7.1.4 Windows/UNIX, MRE/DEV Studio
January 10, 2008, 09:25 AM
GinnyJakes
It is not the number of records that you scan that is the issue, though 500 million would take a long time. The issues are:

Are there any WHERE clauses in your request?

Is the data source indexed on the column being screened?

You run the risk of timing out if you try to bring back too many or it takes too long to retrieve the set you actually want.


Ginny
---------------------------------
Prod: WF 7.7.01 Dev: WF 7.6.9-11
Admin, MRE,self-service; adapters: Teradata, DB2, Oracle, SQL Server, Essbase, ESRI, FlexEnable, Google
January 10, 2008, 09:40 AM
Leah
As Ginny says, on the WHERE, works best on an index field, the amount of data you are retrieving if sorted on the application server may exceed the sort space allowed, But to look through millions of rows is not an issue that I've seen other than the browser time out issue if you are doing it 'live' rather than deferred or using report caster.


Leah
January 10, 2008, 09:57 AM
TCI
quote:
Reply

Thanks for your quick responses, I do have where clauses but my concern is most of these reports will work from the self service applications. Does it become mandate to have them run through deferred mode? I also want to know if we have any predefined information from IBI regarding the data limits.


WF 7.1.4 Windows/UNIX, MRE/DEV Studio
January 10, 2008, 10:04 AM
GinnyJakes
I would try to run it several times from a browser to get timings and see if the performance is acceptable for a user. If not, then deferred would be a good way to go.

You can check with IBI but I worked for them for a long time and the line was/is that the answer is too dependent on platform, memory, request particulars, data source, etc. to give any kind of limits or performance expectations.

Try it to see if it works for you.


Ginny
---------------------------------
Prod: WF 7.7.01 Dev: WF 7.6.9-11
Admin, MRE,self-service; adapters: Teradata, DB2, Oracle, SQL Server, Essbase, ESRI, FlexEnable, Google
January 11, 2008, 09:32 AM
cburtt
TCI:

An old FOCUS 'best practice' is to NEVER ever 'WHERE' on a DEFINEd field (unless it's a small table). Such a suntax forces FOCUS to do a sequential scan of all records, to compute the DEFINEd field on each so that it can determine if the record is to be included or excluded.

With correctly stated WebFOCUS and proper indexing of the data it is possible to retrieve from multi-million row tables in a reasonable amount of time.
Activate the proper traces, run the FEX, and show the generated SQL to your DBA.
Together you can get it done.

Also, querying not the RDBS table itself, but a view defined for your needs can quicken the retrieval process.

Chris Burtt


WIN/2K running WF 7.6.4
Development via DevStudio 7.6.4, MRE, TextEditor.
Data is Oracle, MS-SQL.