[CLOSED] Running large report with EXCEL output

Focal Point

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/127101705

August 12, 2010, 11:57 AM

fatboyjim

[CLOSED] Running large report with EXCEL output

Hi,

I'm currently running a simple query report, but with large amounts of data in the output in EXCEL (approx 150mb).

I have used FILEDEF to save into a shared location.

But I keep getting the following error:
This report is invalid because the data retrieval has been killed or the job has been stopped.

Has anyone have similar problems with it or any alternatives to large excel files produced?

Thanks.

Best Regards,

Jimmy Pang

This message has been edited. Last edited by: fatboyjim,

August 12, 2010, 01:08 PM

Francis Mariani

Biggest clue: "150mb".

Which version of Excel? Excel 2003 and older has a limit of 65000 rows per tab (and some number of columns).

I would create a tab or comma delimited file on that shared drive and import that into Excel, remembering the limits of Excel.

Francis

Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server

August 12, 2010, 01:22 PM

fatboyjim

Yes, I know, but the user requirement still want all the data at once (~approx 20% of the data in the table).

There are many columns and that report has over 100 000 rows. It is under the assumption that they have EXCEL 2007 installed into their PC as part of the specs. The reason for the EXCEL to for formatting purposes.

Would it be a much smaller file if it was comma delimited?

Best Regards,

Jimmy Pang

August 12, 2010, 02:20 PM

RSquared

Can you somehow break on any field that would limit the number of records per tab ? If yes, then save as EXL2K BYTOC may help

August 12, 2010, 03:09 PM

fatboyjim

Hi,

Thanks for the suggestion, but in the end, it's still the same EXCEL file and the size would still be very large.

Found out that the transfer from the application server to local or shared file location was taking up the CPU usage (+90%) and the admin console has a CPU_LIMIT configuration setting that kills the agent when it reaches the limit. I believe it is set to 120 seconds by default.

Tested out with a larger limit and was able to get my report out successfully (500mb in size), but of course that increase the risk of having run away jobs or overload the server if the setting changes.

Luckily, was able to convince them that it was too big and even if the report came out, can't open it up.

Solution: Refine the search criteria.

Thanks for everyone's help.

Best Regards,

Jimmy Pang