Focal Point
[CLOSED] Help needed with large XFOCUS databases on zOS

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/3437005506

June 15, 2011, 09:50 AM
Francis Mariani
[CLOSED] Help needed with large XFOCUS databases on zOS
I load a very large XFOCUS DB in an IBM Mainframe environment (zOS). The load is done with a Master without indexes. Then a REBUILD INDEX is done with a Master with four indexes. These two steps ran without error. I tried a TABLE FILE SUM on this DB and I get an I/O error (It's strange that the load and rebuild steps ran without an error and now I get this error):
(FOC198) FATAL ERROR IN DATABASE I/O. FOCUS TERMINATING  ERROR READING
BRANCHDB, PAGE 197895, CODE 0x0000000c

My fear is that this DB is too large, even for an XFOCUS DB - it expanded on to a second volume with a total of around 7000 cylinders. The manual states an XFOCUS database can be 16 gigabytes. How does one calculate cylinders from gigabytes? How does one calculate how much disk space is added by the indexes?

I may have to break this thing into multiple partitions - I've never done this before so I would appreciate some tips. Is there any documentation on XFOCUS database partitioning?

Thanks,

This message has been edited. Last edited by: Francis Mariani,


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
June 15, 2011, 10:14 AM
Tom Flynn
Hi Francis,

If you have your DB segmentized, create the DB/Master with the LOCATION attribute for each segment, which, will put each segment into it's own FOCUS file. They will have to be DD'd in your JCL. That's how we accomplished loading large data sources... A very long time ago though!

hth

Tom


Tom Flynn
WebFOCUS 8.1.05 - PROD/QA
DB2 - AS400 - Mainframe
June 15, 2011, 10:21 AM
Alan B
Francis
Cannot remember the MF cylinders etc.

For Intelligent partitions you need an access file for the MFD. In the access file you can have something like:
MASTER=MEASURES ,$
$
  DATA=xfocus/meas1038.foc ,
  WHERE=WEEK_NUMBER EQ '201038';,$
$
  DATA=xfocus/meas1037.foc ,
  WHERE=WEEK_NUMBER EQ '201037';,$
$
  DATA=xfocus/meas1036.foc ,
  WHERE=WEEK_NUMBER EQ '201036';,$
$
  DATA=xfocus/meas1035.foc ,
  WHERE=WEEK_NUMBER EQ '201035';,$
$
  DATA=xfocus/meas1034.foc ,
  WHERE=WEEK_NUMBER EQ '201034';,$
$

(this is windows, but principle is the same.)

and declare the access file in the MFD
FILENAME=MEASURES, SUFFIX=XFOCUS, ACCESSFILE=MEASURES,
 REMARKS='Merchandising Measures Week / Item / Store', $


For IP to work effectively, you need to choose a field that will ALWAYS be used as screening in a report. This one is week number.

For reporting FOCUS will work out which partitions to use and access based on the screening condition (basically a USE in the background).

For MODIFY / MAINTAIN you have to do it all manually. You need to ALLOC and USE the correct data file.

To JOIN to the file I use an External Index which can be built over all partitions. I prefer this to multi-dimension join. If I need a multi-field JOIN I use a DEFINE in the mfd to create a single field and index the define field.

It really is as simple as that.


Alan.
WF 7.705/8.007
June 15, 2011, 10:26 AM
<JG>
3390 disks

56,664 bytes per track,
15 tracks per cylinder,
849,960 Bytes per Cylinder.

for 7000 cylinders that makes 5949720000 bytes or 5.54 gig(ish)
June 15, 2011, 10:38 AM
Francis Mariani
Thanks for the quick response! I guess we're the only ones not at Summit!

Tom, it's a one-segment DB so I don't think I can segment it with the location attribute.

JG, thanks for the calculation - my 5.54 gig(ish) DB is well within the maximum.

Could it be a real I/O error on a mainframe device?

I haven't worked with FOCUS DBs for a very long time so I'm quite rusty at this.

Alan, your suggestions look like the method I should be using. The database was designed previous to my involvement. It is one segment, with a Year/Month field that would be the candidate for the WHERE statement in the ACCESS file. Even though we always screen on this field, we do need to include more than one value:

WHERE YRMTH FROM '200901' to '201111' OR YRMTH EQ '2010Q1'...

I do have some further questions:

Is "Intelligent Partitions" a FOCUS term, a database design term or your intelligent term?

I need to join from this DB to other lookup tables. How is the External Index built (I promise to look for the documentation!)?

Thanks so much!


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
June 15, 2011, 11:10 AM
Alan B
Francis

For a simple one:
APP HOLD XFOCUS
REBUILD
EXTERNAL INDEX
NEW
MEASINDX
MEASURES
ITEM_NUMBER
NO
NO


rebuild / external index / into a new file / called measindx / on file measures / using field ITEM_NUMBER / Not associated with a field / No screening conditions.

You will need a USE prior to a JOIN (wish this was also in access file).
USE
XFOCUS/MEASINDX WITH MEASURES
END


And somewhere there is a pile of docs (1 page) for ALLOCating an index file.

Cannot use DKU, has to be JOIN, (wish this would change also).

A simple complete example for external Index:
DEFINE FILE CAR
PROFIT/D12.2 = RCOST-DCOST
END
TABLE FILE CAR
PRINT PROFIT
BY COUNTRY
BY CAR
BY MODEL
BY BODYTYPE
ON TABLE HOLD AS PROFIT
END
-RUN

DEFINE FILE CAR
JOIN_FLD/A62 = COUNTRY | CAR | MODEL | BODYTYPE;
END
-RUN
REBUILD
EXTERNAL INDEX
NEW
CARINDEX
CAR
JOIN_FLD
YES
BODYTYPE
NO
USE
CARINDEX.IDX WITH CAR
END
-RUN
JOIN COUNTRY AND CAR AND MODEL AND BODYTYPE IN PROFIT TO JOIN_FLD IN CAR AS J1.
TABLE FILE PROFIT
PRINT PROFIT RCOST DCOST
BY COUNTRY
BY CAR
BY MODEL
BY BODYTYPE
END
-RUN


Intelligent partition is an IB term with Intelligent Horizontal and Vertical Partitions!


Alan.
WF 7.705/8.007
June 15, 2011, 11:27 AM
Francis Mariani
Alan,

Thanks very much! This is great information.

Regards,


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
June 15, 2011, 11:33 AM
Sandy Weller
Francis,

These may be so obvious, but...

Are LRECL and BLKSIZE set to 16384 for files and I think you have to explicitly set

SET XFOCUS = ON

for it to work.

I apologize if you already did this but..


Signature changes from project to project
June 15, 2011, 11:48 AM
j.gross
My recolection is that Focus pages (or blocks in general) cannot span tracks, so you get just 3 xfocus pages per track (the remaining 7,512 bytes are unused), or 45 pages/cyl. So 7000 cyls holds 315,000 pages, which comes to precisely 5 gb (no -ish about it).


- Jack Gross
WF through 8.1.05
June 15, 2011, 11:48 AM
Francis Mariani
Sandy, yes, thanks, I have these settings.

Regards,


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
June 15, 2011, 11:51 AM
Francis Mariani
Jack,

I guess the number of bytes in each record make no difference - 35 byte or 350 byte records just fill up an XFOCUS databse page and it's the number of pages that counts...

Thanks,


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
June 15, 2011, 12:08 PM
Sandy Weller
More "you probably did this but..."

Can you load fewer records to the same setup and get it to work?


Signature changes from project to project
June 15, 2011, 12:18 PM
Francis Mariani
Sandy, yes, less rows - no problem.

When I allocated the file, I could allocate a file of only 4300 cylinders - trying 4400 gave me an error ("PRIMARY SPACE EXCEEDS 65535 TRKS"). So I allocated 4300 cylinders as the primary quantity and 200 as the secondary quantity. Is it possible, that as the db file grows it sometimes spans more than one volume - could that cause the problem?


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server