[CLOSED] Rounding off giving wrong result in 7.6.11

Focal Point

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/364106635

September 29, 2010, 05:51 AM

DeepakD

[CLOSED] Rounding off giving wrong result in 7.6.11

Hello Friends,

We have just installed WF 7.6.11 and migrated our applications from WF 7.1.4. But some of our reports started giving wrong values. e.g. We have one define field

FIELD/A22.1=TABLE_FIELD/1000000;

The value of TABLE_FIELD/1000000 comes out to be 4.4861 but the field displayed in 7.6.11 is 4.4 which should be 4.5 as it is coming in 7.1.4.

Has anybody faced a similar issue with 7.6.11?

regards,
Deepak

This message has been edited. Last edited by: Kerry,

September 29, 2010, 07:18 AM

Wep5622

A22.1 is not a valid format specifier. Did you mean D22.1?

It works fine here:

APP APPENDPATH IBISAMP
-RUN

DEFINE FILE CAR
	VALUE/D22.1 = 4486100/1000000;
END
TABLE FILE CAR
	PRINT VALUE CAR
WHERE RECORDLIMIT EQ 1;
END

WebFOCUS 8.1.03, Windows 7-64/2008-64, IBM DB2/400, Oracle 11g & RDB, MS SQL-Server 2005, SAP, PostgreSQL 11, Output: HTML, PDF, Excel 2010
: Member of User Group Benelux :

September 29, 2010, 09:55 AM

Francis Mariani

4.5 in v7.6.8.

It's called "rounding", so, is it "wrong" or is it just "different"?

Francis

Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server

September 29, 2010, 01:24 PM

Darin Lee

I'm getting 4.5 in v7.6.11 as well. Something else must be going on

Regards,

Darin

In FOCUS since 1991
WF Server: 7.7.04 on Linux and Z/OS, ReportCaster, Self-Service, MRE, Java, Flex
Data: DB2/UDB, Adabas, SQL Server Output: HTML,PDF,EXL2K/07, PS, AHTML, Flex
WF Client: 77 on Linux w/Tomcat

September 30, 2010, 09:48 AM

jgelona

I don't think D22 is valid either. I thought the max was D15.

October 01, 2010, 06:29 AM

<JG>

The documentation all says 16 display characters for D format including the minus sign.

But I believe that this is nolonger correct.

20 display characters is valid and is generally generated against relational tables for numbers
as D20.2.

However the numeric funtions such as FTOA still only accept a maximum of 15.