[SOLVED] Keeping a formatted value for use in a compute

March 01, 2017, 01:17 PM

David M

Code snippet

TABLE FILE DETAIL
SUM
COMPUTE AVGDispatchesPerTrailer/D12.2=DispatchesPerTrailer/13;
COMPUTE PCTCHG/D8.2%=( &CURRENTDISPATCHESPERTRAILER - AVGDispatchesPerTrailer ) / AVGDispatchesPerTrailer * 100;
ON TABLE HOLD AS AVGDATA

&CURRENTDISPATCHESPERTRAILER = 2.45
AVGDispatchesPerTrailer = 2.38
PCTCHG = 2.94

I am having a problem with PCTCHG. This is what I want...
PCTCHG = (2.45 - 2.38) / 2.38 * 100 = 2.94

This is what is happening...
PCTCHG = (2.45 - 2.38391) / 2.38391 * 100 = 2.77

DispatchesPerTrailer/13 = 2.38391 which I format to 2 decimal places

When I compute PCTCHG, I want to use that number with only 2 decimals, but it remains at 5.
How can I get AVGDispatchesPerTrailer to equal 2.38?

This message has been edited. Last edited by: David M,

March 01, 2017, 01:30 PM

tomatosauce

Use AVGDispatchesPerTrailer/D12.2=DispatchesPerTrailer/13; in a DEFINE instead of COMPUTE.

Something like this.

 
DEFINE FILE DETAIL
AVGDispatchesPerTrailer/D12.2 = DispatchesPerTrailer/13;
END

TABLE FILE DETAIL
SUM
COMPUTE PCTCHG/D8.2%=( &CURRENTDISPATCHESPERTRAILER - AVGDispatchesPerTrailer ) / AVGDispatchesPerTrailer * 100; 
ON TABLE HOLD AS AVGDATA

March 01, 2017, 01:48 PM

David M

Thanks tomatosauce for your quick response, but I am still getting PCTCHG = 2.77

Does the statement..
AVGDispatchesPerTrailer/D12.2=DispatchesPerTrailer/13;
simply display AVGDispatchesPerTrailer as 2 decimals or does it convert it?

March 01, 2017, 01:56 PM

Francis Mariani

It displays 2 decimals but retains the full value. Try using P8.2.

From the docs:

Describing Data With WebFOCUS Language > Describing an Individual Field > The Displayed Data Type: USAGE > Numeric Display Options > Rounding

quote:
The details of rounding are handled in the following ways for the following numeric formats:

Integer format When a value with decimal places is assigned to an integer field, the value is rounded before it is stored. If the value is assigned using a DEFINE or COMPUTE command, the decimal portion of the value is truncated before it is stored.

Packed-decimal format When a value is assigned to a packed-decimal field, and the value has more decimal places than the field format specifies, the value is rounded before it is stored.

Floating-point single-precision and double-precision formats When a value is assigned to one of these fields, and the value has more decimal places than the field format specifies, the full value is stored in the field (up to the limit of precision determined by the field internal storage type). When this value is later displayed, however, it is rounded.

Note that if the decimal portion of a floating-point value as it is internally represented in hexadecimal floating-point notation is repeating (that is, non-terminating), the repeating hexadecimal number is resolved as a non-repeating slightly lower number, and this lower number is stored as the field value. In these situations, if in the original value of the digit to be rounded had been a five (which would be rounded up), in the stored lower value, it would become a four (which is rounded down).

March 01, 2017, 02:06 PM

MartinY

Your issue may probably be somewhere else than changing the COMPUTE to a DEFINE.

Look at this sample where I kept 5 decimals and my result only have 2 no matter if I use DEFINE or COMPUTE:

-SET &VAR = 2.45;
DEFINE FILE CAR
RCST1/P12.5 = RETAIL_COST / 1500;
END
TABLE FILE CAR
SUM RCST1
    COMPUTE RCST2 /P12.5 = RETAIL_COST / 1500;
    COMPUTE AVG1  /P12.2 = (&VAR - RCST1) / RCST1 * 100;
    COMPUTE AVG2  /P12.2 = (&VAR - RCST2) / RCST2 * 100;
BY COUNTRY
BY CAR
END

Post your whole code.

Using /D instead of /P have same result. It depends on the input field as Francis refer and more a question of rounding function.