Focal Point
[CLOSED] Logic check on approach to building out multi procedure report

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/1717089096

December 14, 2018, 11:16 AM
Sean Sweeney
[CLOSED] Logic check on approach to building out multi procedure report
I want to reach out regarding my approach to my deliverable to see if anyone has any comments. I'm working through some issues and I want to see if my process so far is the proper / best route.

I have a report that presents a series of metrics. These metrics work on a subset of the entire data set so I create a cache of the relevant guids in foccache. The metrics are setup as individual procedures as each metric queries the DB using the cached guids for specific field / value combinations and performs a count. These are setup through Tasks & Animations as invidiual requests.

Some points that are important:
- the cache is unique to a user; each user can have their own unique list of guids
- the cache should be refreshed on each reqeust of the report as it has the potential for being updated by the minute and the metrics should reflect the latest data. Therefore it can be said the cache is purely for improving performance within a request, not for subsequent requests

So, I am concerned I am getting a race condition between procedures, (which seem to be async at least in terms of the listing I see in Tasks & Animations, they don't seem to wait for each other), competing to build the cache. I see evidence of this in report as my counts can bounce between values between requests.

A process flow on how I see this working:
1. The report submit button is clicked
2. The first procedure/metric checks to see if foccache has been cleared through a global variable, (Q: does the global variable maintain existence across "Requests/Actions" as defined in "Tasks & Animations")
3. If the variable is 0, clear the cache and set the variable to 1
4. Setup the cached data and create a global variable for the count of said data
5. Execute the metric
6. Proceed to the next metric

I guess I'm looking for confirmation my approach of using foccache is the best route and whether there are any concurrency concerns in what I have presented. If there are, what do you think should be changed?

Thanks for any assistance!

This message has been edited. Last edited by: FP Mod Chuck,


Windows 8203 All output formats
December 18, 2018, 05:11 AM
Frans
If it is one report and the data should not be saved, then there is no reason to use foccache.


Test: WF 8.2
Prod: WF 8.2
DB: Progress, REST, IBM UniVerse/UniData, SQLServer, MySQL, PostgreSQL, Oracle, Greenplum, Athena.
December 19, 2018, 01:47 PM
Sean Sweeney
Thanks for your input! So what structure should I look at using? How should my approach change?


Windows 8203 All output formats
December 19, 2018, 01:56 PM
Sean Sweeney
So I think the obvious alternative is using a HOLD file?

If so, would / can this respect my requirement of it being user/session specific and further more specific to one request of the report, (so a subsequent report request starts with a brand new HOLD which is used for all the Requests/Actions for that report)?


Windows 8203 All output formats
December 19, 2018, 02:22 PM
FP Mod Chuck
Sean

The answer is yes as long as it is all in the same procedure and you don't use the APP HOLD command. On the reporting server each request uses a temporary folder assigned to the agent running the request and the hold files are deleted when the request is done. You do have the overhead of recreating the hold file with each request but hopefully those are quick to run.


Thank you for using Focal Point!

Chuck Wolff - Focal Point Moderator
WebFOCUS 7x and 8x, Windows, Linux All output Formats
December 19, 2018, 02:32 PM
Sean Sweeney
So maybe that's the catch - It's not all done in the same procedure.

The report will present a series of metrics organized into groups. The metrics in the report have been setup as individual procedures as some of them can be quite complicated. Being there will be upwards of 30 metrics and some of these will be the same procedure but with different parameters being passed, I think from a readability / maintainability point of view the separate procedures is the best way forward.

So in this case, is the HOLD file approach not an option?

Thanks again!


Windows 8203 All output formats
December 20, 2018, 04:05 AM
Wep5622
Your "individual requests" mean that we are talking about multiple reports here, and run independent of each other, no less.

That is why you may indeed be seeing a race condition: both requests start when there isn't a session yet, and each independently creates one.

You can verify that is what's happening by applying WHENCE to the foccache hold-file you create in each request and then -EXITing. In the output, you would then see different foccache directories being in use between those requests.

We've had this same issue, but I don't think we managed to work around it, even though to my understanding our approach to solving the problem should have worked:

The idea is that if you start a session before the asynchronous requests from the launch page fire, those should see the existing session and use it. For that purpose, we created a procedure that includes the launch page (using -HTMLFORM) and also stores some arbitrary data in the foccache - thereby creating a session.

Why that didn't work for us I cannot say, I didn't do that implementation myself (perhaps I should have...).

In the end we opted for a different solution: From the launch page we call a single request/procedure that includes the original requests. That makes them run sequentially, and thus the first request in the wrapper procedure creates the foccache and the others pick up the existing session.


WebFOCUS 8.1.03, Windows 7-64/2008-64, IBM DB2/400, Oracle 11g & RDB, MS SQL-Server 2005, SAP, PostgreSQL 11, Output: HTML, PDF, Excel 2010
: Member of User Group Benelux :
December 21, 2018, 11:23 AM
Sean Sweeney
Very interesting suggestion. I've implemented this idea and I am no longer seeing any issues - this may work out perfectly.

Some more questions...

1) Are all caching options available in AppStudio through disk read/write? Or are there in memory options?

2) If I perform "ON TABLE HOLD FORMAT HTMTABLE AS X" for a result is that confined to just the requesting session? Or is it stored in a more globally accessible location? In which case I would still want to do: "ON TABLE HOLD FORMAT HTMTABLE AS 'foccache/abc'"


Windows 8203 All output formats
December 27, 2018, 12:38 PM
FP Mod Chuck
Sean

foccache is as close to in memory as you can get with WebFOCUS.

Although I haven't tried number 2 it seems to me that would be a good assumption to use foccache for the session.


Thank you for using Focal Point!

Chuck Wolff - Focal Point Moderator
WebFOCUS 7x and 8x, Windows, Linux All output Formats
January 03, 2019, 11:43 AM
Sean Sweeney
Thanks for the information Chuck!


Windows 8203 All output formats