Focal Point
[SOLVED] Need to handle 7-8 million records in excel or pdf!!

This topic can be found at:
http://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/4937055476

February 03, 2015, 02:44 AM
Maddy1
[SOLVED] Need to handle 7-8 million records in excel or pdf!!
Hello All,

we have a very huge data almost 7-8 million of records,As part of my requirement in worst cases i need to load all the records in a excel or pdf file,Is it possible? with out breaking or stop....

This message has been edited. Last edited by: <Kathryn Henning>,


WebFOCUS 7.7.03 Windows 7, Excel,HTML,PDF
February 03, 2015, 02:53 AM
Dave
My first advice would be : "don't !".

My 2 cents:
You might want to ask what the end user wants to do with so much data. Postprocessing is probably the answer. Question is, can you do that for him/her and reducing the output volume.

...

If you save the output in a flatfile ( .txt or .csv ) it will be a long big breakless file.

But it's not possible to open it in excel anyway ( it doesn't support so many lines ).

Good luck,
Dave


_____________________
WF: 8.0.0.9 > going 8.2.0.5
February 03, 2015, 03:36 AM
Ram Prasad E
First it is not advisable. Second in Excel it is not possible. Third, even if PDF allows, think about the size of the file and consider if end users system resource capacity to handle such a huge file. You may also need to consider Network bandwidth limit, browser timeout issue too.


WebFOCUS 8.1.05
Windows
http://ibiwebfocus.wordpress.com
https://www.facebook.com/groups/ibi.webfocus/
February 03, 2015, 09:03 AM
eric.woerle
Maddy,

First off, I agree with everyone else. You don't want 7 to 8 million record outputs. In my experience, these users just want to skip your BI system and work with these in their own excel environments. This cause problems for many reasons which I wouldn't get into here.

As for "This isn't possible in excel." Thats not entirely true. You would need to create a row count for each row, hold your outpput, then write a loop to compound each group of million records onto their own page. We used to do this all the time before excel could handle 1mil records per sheet.

You still have to consider the users computer and its capabilities. an 8 million excel file will take a huge hit on the persons resources. They could open it in access, but again, there will be other issues associated with that.


Eric Woerle
8.1.05M Gen 913- Reporting Server Unix
8.1.05 Client Unix
Oracle 11.2.0.2
February 03, 2015, 09:26 AM
Doug
I agree with the above. However, perhaps you could use the Migration / Export of that file. It does sound, like Eric says, that: "these users just want to skip your BI system and work with these in their own excel environments. This cause problems for many reasons which I wouldn't get into here."
February 03, 2015, 09:26 AM
krish
I strongly believe there is be always scope for filter. if at all Users are adamant you can try for Burst option and try to ftp the file to some shared server.

WF 8007
Solaris
February 03, 2015, 09:30 AM
Maddy1
Ok,Thank you for your valuable suggestions to All.


WebFOCUS 7.7.03 Windows 7, Excel,HTML,PDF