Hi all,

We have been experiencing random insufficient memory failures on various reports. There does not seem to be any pattern to this error appearing and it can be generated when any of our reports are called, either interactively or via the Report Caster. The error usually appears pretty much instantaneously on the program fex being called. When running a report interactively the user may have to submit the report 3 or 4 times before it will be accepted and produce the report.

All of our reports start by calling a Startup.fex, which does several things, such as checking that the User is pointing to the correct data and also writes out records to log files etc. I therefore thought that it must be this common program that contains something that WebFocus does not like.

To try and trap the error I put several 'marker' points into the startup program (using the TYPE command), so hopefully the log would show me how far through the program it got before failing. We had our first error since making this change today and it looks like none of these markers were reached.

Here is an example of a log for a successful report showing markers:

Starting worker thread
Starting task: Call Report
Task type: EDA RPC
Procedure name: abc.fex
Connecting to server EDASERVE with execution id iadmin
Executing focexec.
STARTUP_MARKER_1
STARTUP_MARKER_2
STARTUP_MARKER_3
STARTUP_MARKER_4
STARTUP_MARKER_5
STARTUP_MARKER_6
Task finished.
Retrieving distribution list from repository
abc.zip distributed to abc@abc.com


And here is the log from todays failure:

Starting worker thread
Starting task: Call Report
Task type: EDA RPC
Procedure name: abc.fex
Connecting to server EDASERVE with execution id iadmin
Executing focexec.
INSUFFICIENT MEMORY IS AVAILABLE FOR PROGRAM/CACHE
Task error: INSUFFICIENT MEMORY IS AVAILABLE FOR PROGRAM/CACHE
No report to distribute.

So it appears that the error was generated before anything happened as not even the first marker is present in the log? Usually when a report falls over, we get an error code, a clue as to what caused it and at what line and the error report will show all the marker points. As none of these are present, it looks to me that it is not any code causing the problem but a setup/system issue?

Has anyone else had this experience or any thoughts as to what we can do next?

Many thanks!

Mark

This message has been edited. Last edited by: Kerry,

Mark,
There might be problem of the log memory..!
Where this log record gets stored of the startup.fex?

You can try clearing the log memory?
Hope this helps...

A

There can be many reasons for this message (it is foc206 isn't it?).
Reasons may vary from having too many match conditions in modify to exceeding the 2gb limit for writing to a file.
It could also be that the error message gets precedence over the type messages, so that you wrongly think it does not reach any marker.
If that is the case, you could try to write the marker messages to an external file in stead of to the caster log file (use -WRITE in stead of -TYPE).

- Does this also happen outside of report caster?
- Does restarting the server help in any way?
- Is there anything in the edaprint file that might give an indication of the problem?
- Have you tried switching on tracing on the server - this is possibly problematic, since the error does not occur always, it's not reproducable.

quote:

Originally posted by Anmol:
Mark,
There might be problem of the log memory..!
Where this log record gets stored of the startup.fex?

You can try clearing the log memory?
Hope this helps...

A

Hi Anmol,

Sorry, I do understand the 'problem of the log memory'? The startup script writes a record to an external file on the iSeries that we use for usage logging. This file contains a few hundred thousand records, so not a large file at all.

Again, sorry I do not understand how I would 'clear the log memory'?

quote:

Originally posted by GamP:
There can be many reasons for this message (it is foc206 isn't it?).
Reasons may vary from having too many match conditions in modify to exceeding the 2gb limit for writing to a file.
It could also be that the error message gets precedence over the type messages, so that you wrongly think it does not reach any marker.
If that is the case, you could try to write the marker messages to an external file in stead of to the caster log file (use -WRITE in stead of -TYPE).

- Does this also happen outside of report caster?
- Does restarting the server help in any way?
- Is there anything in the edaprint file that might give an indication of the problem?
- Have you tried switching on tracing on the server - this is possibly problematic, since the error does not occur always, it's not reproducable.

Hi Gamp,

FOC206 is sometimes displayed and other times we get no error code at all? When we write the record to the log file we do perform a match on 6 keys. I did wonder if this was the cause for the error and this is what I hoped to identify by using the markers.

I did find a document on the technical site which reported the memory issue but along with the error 'Too may matches on non-key fields'. I therefore ensured we have an index on our log file for the appropiate keys, even though we hadn’t seen this more detailed error message.

I think you could well be correct that the error message is taking precedence over the typed messages and I have therefore added some new markers that WRITE to an external file. Good suggestion - thanks!

As for the other questions:

-Yes happens both interactive and report caster
-Not sure, we IPL the box every 2 weeks. The errors are very temperamental. We may have non for 2 weeks and then 10 the next week
-Not sure how to find the edaprint file.
-No not tried tracing. (Sorry we have very limited WebFocus technical knowledge here!)

Thank you both for your inputs and I hope that my new external markers will identify the problem area…I will report back!

Cheers

Mark

The fact that it is, as you say, very temperamental, might also indicate that the problem will only start after the server (or agent) has done a certain amount of work. I've seen that happen before, with jobs failing with some strange error message that did not seem to be right. In these cases we managed to overcome that behavior by specifying that the agent should be recycled after every request. You do that by adding the line 'agent_refresh=1' to your edaserve config file (not sure how it is called on as400) - it can be done through the server console pages. It is an option on the properties page of the default data service.

Hi GamP,

Well since implementing the external log file to log my markers we haven't had an Insufficient memory error on program submission...typical!

We have however had two instances of the following error:

"THE MAXIMUM SPACE FOR COLUMN TITLES HAS BEEN EXCEEDED.
INSUFFICIENT MEMORY FOR STYLESHEET PROCESSING"

This error has been thrown by two different reports that are run 50+ times a day without issue. Both the errors came from scheduled versions. I have checked the program and we have no long defined fields or long titles? One of the reports in question has run over 20,000 times without error!

So I agree that this is a very tempremental error and I think the agent suspicion could be well founded. I checked our agent_refresh value and it was set to 100 (default), thanks for the instructions on how to find it

We have now reduced this to 10 and will continue to monitor. Fingers crossed!

Thanks again for your assistance.

quote:

"THE MAXIMUM SPACE FOR COLUMN TITLES HAS BEEN EXCEEDED.
INSUFFICIENT MEMORY FOR STYLESHEET PROCESSING"

Do the reports by any chance have a variable number of columns dependant on date for example.
For this error it is possible you are really exceeding the limit in a situation like that.

Another possibility that I have seen looking at past hottracks

SET CACHE=

Not used in WebFOCUS but is a FOCUS feature, Are these reports originally ported
from FOCUS.

It should be removed if it is being used.

This message has been edited. Last edited by: <JG>,

Hi JG,

One of the two reports is static and the columns do not change.

The other report does allow users to select which columns to report from the HTML screen, but not in this instance when being called by the Report Caster. The scheduled version selects the same columns each day without issue.

Reference the SET CACHE, we have already checked for this and we have no instances in any of our programs.

So far no errors since changing agent_refresh to 10. Fingers are still crossed!