Focal Point Banner


As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only.

Join the TIBCO Community
TIBCO Community is a collaborative space for users to share knowledge and support one another in making the best use of TIBCO products and services. There are several TIBCO WebFOCUS resources in the community.

  • From the Home page, select Predict: WebFOCUS to view articles, questions, and trending articles.
  • Select Products from the top navigation bar, scroll, and then select the TIBCO WebFOCUS product page to view product overview, articles, and discussions.
  • Request access to the private WebFOCUS User Group (login required) to network with fellow members.

Former myibi community members should have received an email on 8/3/22 to activate their user accounts to join the community. Check your Spam folder for the email. Please get in touch with us at community@tibco.com for further assistance. Reference the community FAQ to learn more about the community.


Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     [Solved] Calculation Error

Read-Only Read-Only Topic
Go
Search
Notify
Tools
[Solved] Calculation Error
 Login/Join
 
Joel Elscott
Gold member
posted
Can anyone explain why my calculation for "PTM_NOT_SHIPPED_IMAGES" is wrong? It's calculating a value of 1,273,022, but it should be 1,273,333 (276,211 * 4.61). 

  
-SET &Processed_Images = '2232254';
-SET &Processed_Stmts = '484338';
-SET &PTM_Stmts = '208127';

TABLE FILE CAR
SUM CAR NOPRINT
	COMPUTE IMGS_PER_STMT/D3.2 = &Processed_Images / &Processed_Stmts; AS 'Images per Statement' 
	COMPUTE PTM_NOT_SHIPPED/D12 = &Processed_Stmts - &PTM_Stmts; AS 'Statements Not Shipped'
	COMPUTE PTM_NOT_SHIPPED_IMAGES/D12 = PTM_NOT_SHIPPED * IMGS_PER_STMT; AS 'Images Not Shipped'
END

This message has been edited. Last edited by: Joel Elscott,


WebFOCUS 8.2.03
z/OS
 
Posts: 66 | Registered: May 20, 2013Report This Post
Tom Flynn
Expert
posted Hide Post
Change D3.2 to D5.2...

 
Images per Statement Statements Not Shipped Images Not Shipped 
                4.61    276,211                      1,273,022


Tom Flynn
WebFOCUS 8.1.05 - PROD/QA
DB2 - AS400 - Mainframe
 
Posts: 1972 | Location: Centennial, CO | Registered: January 31, 2006Report This Post
Joel Elscott
Gold member
posted Hide Post
I tried D5.2 and D3.2 but get same result of 1,273,022.

4.61 * 276,211 should equal 1,273,333.


WebFOCUS 8.2.03
z/OS
 
Posts: 66 | Registered: May 20, 2013Report This Post
MartinY
Virtuoso
posted Hide Post
Try /P format instead of /D such as /P3.2
Change them all to /P


WF versions : Prod 8.2.04M gen 33, Dev 8.2.04M gen 33, OS : Windows, DB : MSSQL, Outputs : HTML, Excel, PDF
In Focus since 2007
 
Posts: 2409 | Location: Montreal Area, Qc, CA | Registered: September 25, 2013Report This Post
Joel Elscott
Gold member
posted Hide Post
That worked! Thank you!

Dare I ask why Packed vs. Decimal would make a difference?


WebFOCUS 8.2.03
z/OS
 
Posts: 66 | Registered: May 20, 2013Report This Post
Tony A
Expert
posted Hide Post
Hi Joel,

It's the way that the data is held.

Packed decimal format will only store the decimal component to the number of decimal places given within the format (in your case, 2 decimal places) rounding as per rules.

Double Precision format (not decimal) will hold the result at more decimal places than given in the format - which is essentially a "usage" format in this instance.

So, with double precision, your calculation is using 4.60887644579 etc. and not 4.61 which is what packed decimal format provides.

T


In FOCUS
since 1986		WebFOCUS Server 8.2.01M, thru 8.2.07 on Windows Svr 2008 R2  
WebFOCUS App Studio 8.2.06 standalone on Windows 10 
 
Posts: 5694 | Location: United Kingdom | Registered: April 08, 2004Report This Post
Joel Elscott
Gold member
posted Hide Post
Thank you Tony, I appreciate your detailed explanation!


WebFOCUS 8.2.03
z/OS
 
Posts: 66 | Registered: May 20, 2013Report This Post
  Powered by Social Strata  

Read-Only Read-Only Topic

Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     [Solved] Calculation Error

Copyright © 1996-2020 Information Builders