Focal Point
[CLOSED] HOLD COMPRESSED; 30 MILLION ENTRIES

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/8527086096

August 13, 2018, 06:27 AM
jenni
[CLOSED] HOLD COMPRESSED; 30 MILLION ENTRIES
Hi,

is it possible to hold a table from Oracle in an xfocus Format, with some extra space saveing Options? When I create the hold, I think this would be around 65 GB, which is too much. Using compress doesn't help...

Any ideas are welcome.

We also got hyperstage licensed, but want to try it without, because we are still in 8.1.05 and Need some more time to migrate to 8.2.03, where we got hyperstage licensed...


thank!!

This message has been edited. Last edited by: FP Mod Chuck,


WebFOCUS 8.1.05
Windows, All Outputs
August 13, 2018, 08:40 AM
BabakNYC
I'm not sure if this will compress it as much as you'd like but you can try HOLD FORMAT DATREC. According to the manual: DATREC format saves report output as a sequential file with a Master File, and stores numeric fields as binary numbers without aligning them on fullword boundaries. The last field consists of one byte for each of the other fields in the Master File that indicates whether the corresponding field is missing. The output is a binary hold file with an FTM extension and a corresponding MAS.

However, I think if you've license Hyperstage in 8.2 you're probably allowed to also use it in the earlier release. The license is associated with your site code not version. You might want to ask IB tech support if that's true.


WebFOCUS 8206, Unix, Windows
August 13, 2018, 11:57 AM
j.gross
I worked on QA testing before the initial release of XFOCUS. The internal structure of XFOCUS files (unlike FOCUS files) is designed to support variable-length records -- records include a leading byte-count, and are indexed within the page by byte-offset, much like IDMS page structure. Thus, there is a potential for WF to store AnV fields in trimmed (ragged rather than Procrustean) form, compressing out the trailing whitespace.

So, if your data records include long character fields, with significant trailing blanks, see what happens if you apply RTRIM() in the request, or before storing in the XFOCUS file.

This message has been edited. Last edited by: j.gross,


- Jack Gross
WF through 8.1.05
August 13, 2018, 01:57 PM
j.gross
Also, if your data naturally factor into a multi-segment [hierarchical] structure, that can save a lot of room compared to storing in a one-segment (flat) structure.

Table ...
sum [columns dependent on just key_1]
by key_1
sum [columns dependent on key_1 and key_2]
by key_1
by key_2
on table hold ... format xfoc

will build and populate the two-segment structure dynamically.

This message has been edited. Last edited by: j.gross,


- Jack Gross
WF through 8.1.05
August 14, 2018, 09:20 AM
jgelona
Another option would be if your data can easily be separated into logical groups, i.e. years. Then you could create a file for each year.
USE
  MYFILE_2018.FOC AS MYFILE
  MYFILE_2017.FOC AS MYFILE
  MYFILE_2016.FOC AS MYFILE
END

The FOCUS language will treat all 3 files as one file.


In FOCUS since 1985. Prod WF 8.0.08 (z90/Suse Linux) DB (Oracle 11g), Self Serv, Report Caster, WebServer Intel/Linux.