Focal Point
[SOLVED] Packed Fields Causing Rounding Errors

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/3587015396

May 29, 2020, 07:06 PM
AMC2
[SOLVED] Packed Fields Causing Rounding Errors
I use a couple of packed decimal fields formatted as P15.2 (VBR_PD_ELG_AMT) and P11.2 (VBR_GAIN_AMT) in the database in two DEFINE fields and define them as P15!d and P11!d because I want the whole dollar amount with the dollar sign. The code is below.

Most of the amounts are coming out fine. There are a few that are not rounding as they should. For example, 1852.12 comes out as $1853 oddly. There are only a few cases like this, but most of the amounts are rounding fine. Should I try to convert the packed decimal double precision fields in the database to something else before formatting them to a whole amount with the dollar sign..if anyone thinks?

  
DEFINE FILE pmr/pmr_prov_val_base_cluster
CLDR_DT/MDYY = DATECVT(ICURD_CLDR, 'I8YYMD', 'MDYY');
VBR_PD_ELG_AMT_P/P15!d = VBR_PD_ELG_AMT;
VBR_GAIN_AMT_P/P11!d = VBR_GAIN_AMT;
-*P_VBR_PD_ELG_AMT/P15!d = VBR_PD_ELG_AMT_P;
-*P_VBR_GAIN_AMT/P11!d = VBR_GAIN_AMT_P;
END

TABLE FILE pmr/pmr_prov_val_base_cluster
SUM VBR_PD_ELG_AMT
    PMR_PROV_VAL_BASE_CLUSTER.PROV_VAL_BASE_PMT_FCT.VBR_PD_ELG_AMT_P AS 'Potential'
	VBR_GAIN_AMT
    PMR_PROV_VAL_BASE_CLUSTER.PROV_VAL_BASE_PMT_FCT.VBR_GAIN_AMT_P AS 'Actual'
-*BY PMR_PROV_VAL_BASE_CLUSTER.PROV_VAL_BASE_PMT_FCT.ICURD_CLDR
BY CLDR_DT AS 'Month
WHERE VBR_PRCG_PCT_VAL NE 0;
WHERE SPEC_CD EQ 'AC' OR 'PD' OR 'OB';
WHERE PRIM_LCTN_SW EQ 'Y'
WHERE PMR_PROV_VAL_BASE_CLUSTER.PROV_DMN.PROV_NPI EQ &NPI;
END
-RUN
-EXIT

This message has been edited. Last edited by: AMC2,


WebFOCUS 8.2.06
SQL Server
HTML, PDF, Excel, etc
May 31, 2020, 07:48 AM
Danny-SRL
AMC2,

quote:
VBR_PD_ELG_AMT_P/P15!d = VBR_PD_ELG_AMT;


Try the following:
  
VBR_PD_ELG_AMT_P/P15!d = INT(VBR_PD_ELG_AMT + 0.5);



Daniel
In Focus since 1982
wf 8.202M/Win10/IIS/SSA - WrapApp Front End for WF

June 01, 2020, 08:12 AM
MartinY
Since you reassign each single value from a DEFINE, each of them are rounded to then be added together by the SUM which may explain the "bad" rounding.
Try with the following where it is rounded and formatted only at display
TABLE FILE pmr/pmr_prov_val_base_cluster
SUM PMR_PROV_VAL_BASE_CLUSTER.PROV_VAL_BASE_PMT_FCT.VBR_PD_ELG_AMT_P/P15!d AS 'Potential'
    PMR_PROV_VAL_BASE_CLUSTER.PROV_VAL_BASE_PMT_FCT.VBR_GAIN_AMT_P/P15!d   AS 'Actual'
BY CLDR_DT AS 'Month
WHERE VBR_PRCG_PCT_VAL NE 0;
WHERE SPEC_CD EQ 'AC' OR 'PD' OR 'OB';
WHERE PRIM_LCTN_SW EQ 'Y'
WHERE PMR_PROV_VAL_BASE_CLUSTER.PROV_DMN.PROV_NPI EQ &NPI;
END
-RUN
-EXIT



WF versions : Prod 8.2.04M gen 33, Dev 8.2.04M gen 33, OS : Windows, DB : MSSQL, Outputs : HTML, Excel, PDF
In Focus since 2007
June 08, 2020, 10:07 AM
AMC2
Thanks MartinY - By the way, I just rounded and formatted at the display and that did take care of those few instances were the rounding was off. Thanks for your suggestion also Danny-SRL.


WebFOCUS 8.2.06
SQL Server
HTML, PDF, Excel, etc
June 08, 2020, 10:43 AM
MartinY
AMC2,

According to the Forum Guidelines you should have marked you post as [SOLVED] and not [CLOSED]
Following these guidelines helps for further research


WF versions : Prod 8.2.04M gen 33, Dev 8.2.04M gen 33, OS : Windows, DB : MSSQL, Outputs : HTML, Excel, PDF
In Focus since 2007
June 08, 2020, 11:36 AM
AMC2
I did not mark it as [CLOSED]. Maybe the forum admin marked it as [CLOSED] maybe because I took a while to respond. I changed it to [SOLVED]. Thanks.


WebFOCUS 8.2.06
SQL Server
HTML, PDF, Excel, etc
June 08, 2020, 12:45 PM
FP Mod Chuck
Yes it was me as I didn't see any activity for 7 days and no acknowledgment that the last post from June 1st helped.


Thank you for using Focal Point!

Chuck Wolff - Focal Point Moderator
WebFOCUS 7x and 8x, Windows, Linux All output Formats