Focal Point Banner


As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only.

Join the TIBCO Community
TIBCO Community is a collaborative space for users to share knowledge and support one another in making the best use of TIBCO products and services. There are several TIBCO WebFOCUS resources in the community.

  • From the Home page, select Predict: WebFOCUS to view articles, questions, and trending articles.
  • Select Products from the top navigation bar, scroll, and then select the TIBCO WebFOCUS product page to view product overview, articles, and discussions.
  • Request access to the private WebFOCUS User Group (login required) to network with fellow members.

Former myibi community members should have received an email on 8/3/22 to activate their user accounts to join the community. Check your Spam folder for the email. Please get in touch with us at community@tibco.com for further assistance. Reference the community FAQ to learn more about the community.


Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     Datawarehouse desing

Read-Only Read-Only Topic
Go
Search
Notify
Tools
Datawarehouse desing
 Login/Join
 
Frans
Guru
posted
Hi all,

Not really a WebFOCUS question, build I would like to have your opinion.

We're now reporting on a Oracle 9i DWH. The fact table is about 35,000,000 records. The performance is not good (30 seconds - 10 minutes to retrieve 700 rows)

All keys in this fact table are VARCHAR and indexes are on these varchars. I know that alfanumeric / natural keys should be in the dimension and the fact table should only contain numeric surrogated keys with preferable bitmap indexes.

Another thing is that every column has a seperate index. I think multi column indexes would be a better option since some columns are always queried. But I'm not sure what's preferable in Oracle.

I want your opinion if this using of varchar's and indexes on varchars in the fact table has a big influence on performance. And if multi column indexes would be better that 15 different single comlumn indexes.

Thanks


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
 
Posts: 454 | Location: Europe | Registered: February 05, 2007Report This Post
TexasStingray
Master
posted Hide Post
Frans,

If performance is slow be sure you check the SQL being generated by WebFOCUS using the following commands to view the SQL. You want the database to do as much of the work as possible at that's what they are good for. You only want to see 1 SQL statement. make sure that your where statements are being passed to the RDBMS.

SET TRACEOFF=ALL
SET TRACEON=STMTRACE//CLIENT
SET TRACEON=SQLAGGR//CLIENT
SET TRACEUSER=ON
SET XRETRIEVAL=OFF


Hope this helps.

PS:If you send the SQL thru SQLPlus how does it perform?

Scott




Scott

 
Posts: 865 | Registered: May 24, 2004Report This Post
RSquared
Guru
posted Hide Post
Frans,

I would also check to see how many defines are being done on the Tables. It may be much faster to first extract the data, save as a hold file and then do all youy defines on the hold file.


WF 7.6.11
Oracle
WebSphere
Windows NT-5.2 x86 32bit
 
Posts: 398 | Registered: February 04, 2008Report This Post
Frans
Guru
posted Hide Post
Hi TexasStingray and RSquared,

I've checked the SQL which is generated by WebFOCUS and also put in in the analyzer tool from Oracle. The analyser tool didn't have any SQL improvement suggestions. All fields in the selection have an index.

My query is like this:
WHERE T1.HOUSE ='0058'
AND T1.LOCATION='LNL'
AND(T1.SUBMIT BETWEEN 24091 AND 24097
AND T1.TYPE =9
OR T1.SUBMIT = 24098
AND T1.START >= 24098
AND T1.START <= 24102);

There are no defines or sorts messing up anything so I guess this is not a FOCUS problem.

Any suggestions?


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
 
Posts: 454 | Location: Europe | Registered: February 05, 2007Report This Post
RSquared
Guru
posted Hide Post
Frans,

Have you tried to change your selection to separate the where clauses. I find the combination of 'and's and 'or's confusing.

Do you want either when thes conditions are met
T1.SUBMIT BETWEEN 24091 AND 24097
AND T1.TYPE =9
or when these coditions are met
T1.SUBMIT = 24098
AND T1.START >= 24098
AND T1.START <= 24102

or something different?


WF 7.6.11
Oracle
WebSphere
Windows NT-5.2 x86 32bit
 
Posts: 398 | Registered: February 04, 2008Report This Post
Frans
Guru
posted Hide Post
Hi RSquared,

First I want all records with house 0058 and location LNL.

From that recordset I only want records with submit between 24091 and 24097 and type 9

Or records which have a submit of 24098 and have a start between 24098 and 24102

Is this more clear?


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
 
Posts: 454 | Location: Europe | Registered: February 05, 2007Report This Post
RSquared
Guru
posted Hide Post
Frans,

In that case you can try to either change your code to
WHERE T1.HOUSE ='0058'
AND T1.LOCATION='LNL'
WHERE(T1.SUBMIT BETWEEN 24091 AND 24097
AND T1.TYPE =9)
OR (T1.SUBMIT = 24098
AND T1.START >= 24098
AND T1.START <= 24102);
Or
Extract to a hold file using the first Where WHERE T1.HOUSE ='0058'
AND T1.LOCATION='LNL'
and then apply the second logic to the hold file.


WF 7.6.11
Oracle
WebSphere
Windows NT-5.2 x86 32bit
 
Posts: 398 | Registered: February 04, 2008Report This Post
Frans
Guru
posted Hide Post
RSquared,

Adding another WHERE in WebFOCUS will change it to AND in SQL code to Oracle.

Retrieving data without the date selection results in 10K+ rows, so unfortunately no performance improvement Frowner


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
 
Posts: 454 | Location: Europe | Registered: February 05, 2007Report This Post
RSquared
Guru
posted Hide Post
Did you try adding the brackets to separate the two conditions.


WF 7.6.11
Oracle
WebSphere
Windows NT-5.2 x86 32bit
 
Posts: 398 | Registered: February 04, 2008Report This Post
hammo1j
Master
posted Hide Post
Hi Frans

Used to be an Oracle dba for a bit with Oracle dwh. Can you check that you do not have bitmap indicies (recommended by Oracle but they do not work at all well) and try replacing them with conventional indicies to see if that works.

btw if you want performance x20 of Oracle build your dwh as XFOCUS. I guess probably not practical in terms of time and resource but worth knowing you can get this level of performance if needed.

Regards John.


Server: WF 7.6.2 ( BID/Rcaster) Platform: W2003Server/IIS6/Tomcat/SQL Server repository Adapters: SQL Server 2000/Oracle 9.2
Desktop: Dev Studio 765/XP/Office 2003 Applications: IFS/Jobscope/Maximo
 
Posts: 888 | Location: Airstrip One | Registered: October 06, 2006Report This Post
Frans
Guru
posted Hide Post
Hi John,

This statement uses nested loops on 1 unique index, 1 range scan. A hash join with 2 full table scans (this takes very little performance cost (7) will the nested loops take take 4516 cost)

I've made a virtual index in the analyzer tool, and a bitmapped index show a lot of estimated improvement?

I'll try to do a test and put it in XFOCUS, I'm curious what the results will be....


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
 
Posts: 454 | Location: Europe | Registered: February 05, 2007Report This Post
ptp
Silver Member
posted Hide Post
You may need to check the licensing implications of using XFOCUS and more importantly Multi-Dimensional Indices with your local Information Builders Office.

Thanks,
P
 
Posts: 42 | Location: UK | Registered: October 23, 2005Report This Post
hammo1j
Master
posted Hide Post
Would not bother immediately with XFOCUS since you'll have to use hierarchial design principles to get the speed.

The point about the bitmaps is that they DONT work very well in 9i. I would advise trying it WITHOUT the Bitmap indicies to see if it improves. It shouldn't take too long to rebuild them as btree and try the query again.


Server: WF 7.6.2 ( BID/Rcaster) Platform: W2003Server/IIS6/Tomcat/SQL Server repository Adapters: SQL Server 2000/Oracle 9.2
Desktop: Dev Studio 765/XP/Office 2003 Applications: IFS/Jobscope/Maximo
 
Posts: 888 | Location: Airstrip One | Registered: October 06, 2006Report This Post
Frans
Guru
posted Hide Post
ptp, I can create XFOCUS right now, so I guess the license would be OK.


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
 
Posts: 454 | Location: Europe | Registered: February 05, 2007Report This Post
TexasStingray
Master
posted Hide Post
Frans,

When you run it thru SQLPlus, how long does the request take. then try just running the request thru WebFOCUS and not create the focus db just dump it to a file or something how long does it take. the time may be in creating the FOCUS database.

tex




Scott

 
Posts: 865 | Registered: May 24, 2004Report This Post
Darin Lee
Virtuoso
posted Hide Post
FYI - being "able" to do something and being "licensed" to do something are NOT the same thing. You are "able" to configure as many data adapters as you want, but if you are not licensed for them and get caught, it's lights out.


Regards,

Darin


In FOCUS since 1991
WF Server: 7.7.04 on Linux and Z/OS, ReportCaster, Self-Service, MRE, Java, Flex
Data: DB2/UDB, Adabas, SQL Server Output: HTML,PDF,EXL2K/07, PS, AHTML, Flex
WF Client: 77 on Linux w/Tomcat
 
Posts: 2298 | Location: Salt Lake City, Utah | Registered: February 02, 2007Report This Post
GinnyJakes
Expert
posted Hide Post
Actually, Darin, you can't do that anymore either at least on AIX. The Reporting server license key has 4 parts now instead of 3 and the last one regulates how many adapters you can configure. If you try to configure the nth+1 adapter, you get an error message.


Ginny
---------------------------------
Prod: WF 7.7.01 Dev: WF 7.6.9-11
Admin, MRE,self-service; adapters: Teradata, DB2, Oracle, SQL Server, Essbase, ESRI, FlexEnable, Google
 
Posts: 2723 | Location: Ann Arbor, MI | Registered: April 05, 2006Report This Post
Frans
Guru
posted Hide Post
True, I've been at curstomers where there was a maximum of 2 adapters. Also active reports, visual discovery etc needs extra licensing. For XFOCUS there isn't.

Anyway, I made some performance improvement by making a multi column index (B tree) which made it faster (5 minutes to 40 seconds).

Then I dumped 8 million records in an indexed FOCUS file and my request only took 4 seconds Cool

John, there are no bitmapped indexes at the moment, only single column B tree indexes.


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
 
Posts: 454 | Location: Europe | Registered: February 05, 2007Report This Post
  Powered by Social Strata  

Read-Only Read-Only Topic

Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     Datawarehouse desing

Copyright © 1996-2020 Information Builders