Focal Point
[CLOSED]Compute or Define in masters

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/5111044852

November 22, 2007, 05:09 PM
FrankDutch
[CLOSED]Compute or Define in masters
I wonder what the difference is, but I also had a problem today.
I changed a master by adding 4 new fields in one segment and to keep several reports working I made a computed field that summed these four fields together.
The result ? bad...I got stars in most of my reports, so it seemed to be that the description was to small to hold the values.

FIELDNAME=VALUE1, ACTUAL=D20.2,..,$
FIELDNAME=VALUE2, ACTUAL=D20.2,..,$
FIELDNAME=VALUE3, ACTUAL=D20.2,..,$
DEFINE VALUE4/D20.2=VALUE1+VALUE2+VALUE3;


Is there a limitation in the size?

This message has been edited. Last edited by: <Emily McAllister>,




Frank

prod: WF 7.6.10 platform Windows,
databases: msSQL2000, msSQL2005, RMS, Oracle, Sybase,IE7
test: WF 7.6.10 on the same platform and databases,IE7

November 22, 2007, 08:02 PM
jnc
Isn't the limit for D format 15. If you need larger, then P is good for 31.


WebFocus 7x, 8x, Win / Linux, any output format
November 23, 2007, 02:34 AM
Tony A
Actually, the limit for D is 16 chars including decimal point and leading minus sign as applicable. D20.2 is often used though (just like I10 used to function when the max was I9 - I think) - it is now documented as max I11.

The limit for P is 33 characters, again including the decimal point and leading sign.

On the other hand, F limit is 9 with the same provisions as for other non-integer numbers

So, in theory you can declare (as of Version 5.3) -

D16 - using D16.14 to include signage
      through to D16 without decimal places
P33 - using P33.31 to include signage
      through to P33 (if you do not have any decimals)
F9  - using F9.7 to include signage
      etc. etc.
I11 - using I10 to include signage

Of course, many folks will require the Double precision format due to the way the original value is retained irrespective of the usage format. Especially in accounts processing.

T



In FOCUS
since 1986
WebFOCUS Server 8.2.01M, thru 8.2.07 on Windows Svr 2008 R2  
WebFOCUS App Studio 8.2.06 standalone on Windows 10 
November 23, 2007, 10:47 AM
GamP
And then there is of course also the scientific notation, which has a somewhat more extended display range: Maximally 33 positions.
For instance: COMPUTE FLD/E33.25=DCOST;
This will result in figures like: 0.4292000000000000000000000E+04

Hope this helps.


GamP

- Using AS 8.2.01 on Windows 10 - IE11.
in Focus since 1988
November 26, 2007, 08:57 AM
PBrightwell
What type of database? With Oracle D20.2 is the default for a numeric field and I am sure it will work in 5.3 and higher. When you added the 3 new fields did you rebuild the file? Is it possible that you are hitting nulls or alpha data? Does your MFD match the metadata?


Pat
WF 7.6.8, AIX, AS400, NT
AS400 FOCUS, AIX FOCUS,
Oracle, DB2, JDE, Lotus Notes
November 26, 2007, 10:08 AM
FrankDutch
thx for thinking...

Pat, I did a rebuild the file, that was not the issue. I resized the 4 fields to D12.2, and that helped.




Frank

prod: WF 7.6.10 platform Windows,
databases: msSQL2000, msSQL2005, RMS, Oracle, Sybase,IE7
test: WF 7.6.10 on the same platform and databases,IE7

November 29, 2007, 09:49 PM
Brian Suter
D or F format inside the computer looks something like scientific notation. You can display F20 if you like but you can only have 7 significant digits (because they use 24 bits for the digits, 7 bits for the power of 16, and one bit for the sign). A D format has 56 bits available for significant digits which means you can go up to 72,057,594,037,927,936 before you start loosing single units of precision.


Brian Suter
VP WebFOCUS Product Development
May 05, 2016, 10:56 AM
Doug
Hi Tony,

What's the max size for D##.# now, / 8.1.0.7?

Decimal: The GUI shows D(1-20).(0-18).

Packed : The GUI shows P(1-33).(0-31).

So, is D30.2 or D22.2 valid?

Thanks, Doug