Can anyone explain why my calculation for "PTM_NOT_SHIPPED_IMAGES" is wrong? It's calculating a value of 1,273,022, but it should be 1,273,333 (276,211 * 4.61).

  
-SET &Processed_Images = '2232254';
-SET &Processed_Stmts = '484338';
-SET &PTM_Stmts = '208127';

TABLE FILE CAR
SUM CAR NOPRINT
	COMPUTE IMGS_PER_STMT/D3.2 = &Processed_Images / &Processed_Stmts; AS 'Images per Statement' 
	COMPUTE PTM_NOT_SHIPPED/D12 = &Processed_Stmts - &PTM_Stmts; AS 'Statements Not Shipped'
	COMPUTE PTM_NOT_SHIPPED_IMAGES/D12 = PTM_NOT_SHIPPED * IMGS_PER_STMT; AS 'Images Not Shipped'
END

This message has been edited. Last edited by: Joel Elscott,

Change D3.2 to D5.2...

 
Images per Statement Statements Not Shipped Images Not Shipped 
                4.61    276,211                      1,273,022 

I tried D5.2 and D3.2 but get same result of 1,273,022.

4.61 * 276,211 should equal 1,273,333.

Try /P format instead of /D such as /P3.2
Change them all to /P

That worked! Thank you!

Dare I ask why Packed vs. Decimal would make a difference?

Hi Joel,

It's the way that the data is held.

Packed decimal format will only store the decimal component to the number of decimal places given within the format (in your case, 2 decimal places) rounding as per rules.

Double Precision format (not decimal) will hold the result at more decimal places than given in the format - which is essentially a "usage" format in this instance.

So, with double precision, your calculation is using 4.60887644579 etc. and not 4.61 which is what packed decimal format provides.

T

Thank you Tony, I appreciate your detailed explanation!