Focal Point
issue in rendering huge volume of data

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/8081074772

March 24, 2008, 08:19 AM
<Arun>
issue in rendering huge volume of data
Hi,

We have an issue in rendering huge volume of data to the front end page .

We have a query which fetches around 90,000 rows. Only when all 90,000 rows are
loaded into the report page , we see the report output (HTML format). Even if 10000 rows are loaded into the page, data is not shown.

But our requirement is that, as soon as some thousand rows are loaded into the page, we
need to show them in the front page. Then once the next set of rows are ready,
we need to render them to the front page.

Is there any way to stream the datas?

Thanks,
Arun
March 24, 2008, 08:35 AM
FortuneCookie
You may want to try "On-demand Paging".

Add "SET WEBVIEWER=ON" to you request, see if that gives you the desirable results. With that at least data will start streaming down and you won't have to sit there with a blank screen until all 90k is completed.
March 24, 2008, 10:38 AM
FrankDutch
Arun

You might have seen this remark before, but would you be so kind to update your signature with some information about version in use, platform, systems, etc so we may be able to help better.

According to your question, the solution is ok, but I always wonder why people put 90.000 rows of data into a report. Beware of printing this, in a report of 60 rows per page this is a printed book with 1500 pages, who is going to read that.
Try to bring it down to a small one page report with the headlines. Then drill down to a second level of information.

I'm sorry, this is how I see my job in our company. Advices about the use, not just pushing data.




Frank

prod: WF 7.6.10 platform Windows,
databases: msSQL2000, msSQL2005, RMS, Oracle, Sybase,IE7
test: WF 7.6.10 on the same platform and databases,IE7

March 24, 2008, 11:32 AM
<Arun>
Sorry for missing those informations.

I use WF 7.1.7 version , linux platform
March 25, 2008, 05:42 AM
<Arun>
SET WEBVIEWER = ON works gr8 . Thanks all.

this resolved only HTML streaming issue. Do we have similar kind of options for PDF and XL??

Appreciate ur responses!!!!

Thanks,
Arunvijay
March 25, 2008, 07:00 AM
FrankDutch
Arun

Thanks for updating your signature.....it will help.

In pdf or excel 90.000 rows?????

See my other remark. PDF is for printing....please do not spoil trees.
Excel might work if you have the correct version, but people who ask for 90.000 rows in a spreadsheet tend to be not end users who are going to create there own graphics, detailed reports.

But if yo really want to do it....use reportcaster to send the report directly to the end user by mail.




Frank

prod: WF 7.6.10 platform Windows,
databases: msSQL2000, msSQL2005, RMS, Oracle, Sybase,IE7
test: WF 7.6.10 on the same platform and databases,IE7

March 25, 2008, 07:04 AM
Tony A
Arun,

Yes, my suggestion would be to educate your users that 90,000 rows of data is not a report, it is a book!! Exactly as Frank suggests above! If they think that they can analyse that (even a page at a time) to make an informed decision then they are only deluding themselves - and possibly their business!!

Failing that then I would suggest using more parameters in the request so that the "report" size is reduced.

You could also use such code as BYTOC or DRILLTHROUGH.

T



In FOCUS
since 1986
WebFOCUS Server 8.2.01M, thru 8.2.07 on Windows Svr 2008 R2  
WebFOCUS App Studio 8.2.06 standalone on Windows 10 
March 25, 2008, 08:14 AM
<Arun>
Frank/Tony,

Since our reports are the only place where business people can get these information, they need us to provide those huge datas.

* we arrive at 90k records only after applying filters. so there is no way to filter any more.

* Since reportcaster is not part of our release/effort , we cant get help from reportcaster too.

Any suggesstions for providing streaming in PDF/XLS format?

Thanks,
Arunvijay
March 26, 2008, 12:43 PM
Darin Lee
Sorry, there is no such thing as streaming PDF/XLS. By definition a PDF file is a document - something that has already been created. Adobe Reader cannot open a document until it is complete. Excel cannot open a spreadsheet that is in the process of being created.

And it sounds like you need to provide your users with a reporting tool, not a report. In my opinion (though it may not be a humble one in this case), there is no such thing as a 90,000 row "report." A report is something that someone will read, become informed, and make decisions based upon its contents. There are not many users (sane ones, at least) that are going to read a 90,000 row, 1500 page report containing columns of data. They are going to be scanning for anomalies, looking for trends, etc. all of which should be built into a report, or multiple reports if necessary. What you are giving the users is a database. You need to get more information about what the users are really looking for, or provide them access via WF or DevStudio for them to get exactly the information they need.

I'm not meaning this to sound like criticism as much as trying to be helpful. Frank, Tony, myself and others who may respond have years of experience in IT, using these tools, and creating "useful" reports for users. There are situations where users just want all the data. That still is not a report - its just data. What we are saying is that it is sometimes easier to throw ourselves into that category and give users all the data and let them figure it out themselves. That is not helpful to them and it is not productive (or rather very wasteful) for IT and IT resources. And it causes us to run into these problems, such as yours, that have no solution.


Regards,

Darin



In FOCUS since 1991
WF Server: 7.7.04 on Linux and Z/OS, ReportCaster, Self-Service, MRE, Java, Flex
Data: DB2/UDB, Adabas, SQL Server Output: HTML,PDF,EXL2K/07, PS, AHTML, Flex
WF Client: 77 on Linux w/Tomcat
March 31, 2008, 06:26 AM
<Arun>
quote:
SET WEBVIEWER = ON


Guys,

SET WEBVIEWER = ON is working fine in the console only . But it is not working when i run the report by hitting the WFservlet using a url. only the first page comes up , and remaining pages doesnt come up.

any reason for this behaviour??

Appreciate any resolution for this issue!

Thanks,
Arunvijay