Focal Point Banner


As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only.

Join the TIBCO Community
TIBCO Community is a collaborative space for users to share knowledge and support one another in making the best use of TIBCO products and services. There are several TIBCO WebFOCUS resources in the community.

  • From the Home page, select Predict: WebFOCUS to view articles, questions, and trending articles.
  • Select Products from the top navigation bar, scroll, and then select the TIBCO WebFOCUS product page to view product overview, articles, and discussions.
  • Request access to the private WebFOCUS User Group (login required) to network with fellow members.

Former myibi community members should have received an email on 8/3/22 to activate their user accounts to join the community. Check your Spam folder for the email. Please get in touch with us at community@tibco.com for further assistance. Reference the community FAQ to learn more about the community.


Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     [SOLVED] decimal calculation

Read-Only Read-Only Topic
Go
Search
Notify
Tools
[SOLVED] decimal calculation
 Login/Join
 
Ponnakudam
Member
posted
Hi,

i have a issue in below calculation in mainframe focus. below are 2 computational lines.

Value is getting correctly calculated to TOT in 1st computational line

in the 2nd computational line i am taking the average of TOT and putting to temp variable called AVG. But the value in AVG is not getting correctly displayed. it gives just ****

Let me know how can i fix it

Statements

TOT/D3=MARK_1 + MARK_2 + MARK_3;

AVG/D4.2=TOT / 3;

This message has been edited. Last edited by: Kerry,


WebFOCUS 7.6
Windows, All Outputs
 
Posts: 9 | Registered: August 19, 2011Report This Post
Wep5622
Virtuoso
posted Hide Post
That means the value overflowed the data-type.

And that makes sense too. For example:
TOT/D3 = 999
AVG/D4.2 = 999/3 = 333, which doesn't fit a D4.2.


WebFOCUS 8.1.03, Windows 7-64/2008-64, IBM DB2/400, Oracle 11g & RDB, MS SQL-Server 2005, SAP, PostgreSQL 11, Output: HTML, PDF, Excel 2010
: Member of User Group Benelux :
 
Posts: 1669 | Location: Enschede, Netherlands | Registered: August 12, 2010Report This Post
GamP
Virtuoso
posted Hide Post
D4.2 means:
- Decimal number.
- 4 digits in total
- of which one can be the minus-sign in case of a negative number
- of which one is the dot
- of which 2 are the numerals to the right of the dot
- which leaves just 1 numeral to the left of the dot for a positive number and none for a negative.


GamP
- Using AS 8.2.01 on Windows 10 - IE11.
in Focus since 1988
 
Posts: 1961 | Location: Netherlands | Registered: September 25, 2007Report This Post
jimster06
Guru
posted Hide Post
quote:
D4.2 means:
- Decimal number.
- 4 digits in total
- of which one can be the minus-sign in case of a negative number
- of which one is the dot
- of which 2 are the numerals to the right of the dot
- which leaves just 1 numeral to the left of the dot for a positive number and none for a negative.

To elaborate on GamP's fine explanation, the D indicates a Double precision floating point number. Similarly, single precision floating point use F to designate the format. Another numeric format that can be used to express decimal number is Packed Decimal where the format designator (or data type) is P.
I recommend the Describing Data manual for a more thorough discussion of topic.
HTH


jimster06
DevStu WF 7.6.11
W7
HTML, PDF, EXL2K
 
Posts: 252 | Location: USA | Registered: April 15, 2003Report This Post
George Patton
Master
posted Hide Post
Smiler I think my learned colleagues are suggesting you might want to change it to AVG/D6.2 or maybe even AVG/D7.2.

This is ever-so-basic stuff and you're lucky you didn't get a suggestion to RTFM !

People here are extremely helpful, but understandably get slightly annoyed when the question is one that is thoroughly discussed in the most elementary manuals.


WebFOCUS 7.7.05 Windows, Linux, DB2, IBM Lotus Notes, Firebird, Lotus Symphony/OpenOffice. Outputs PDF, Excel 2007 (for OpenOffice integration), WP
 
Posts: 674 | Location: Guelph, Ontario, Canada ... In Focus since 1985 | Registered: September 28, 2010Report This Post
Ponnakudam
Member
posted Hide Post
solved Smiler


WebFOCUS 7.6
Windows, All Outputs
 
Posts: 9 | Registered: August 19, 2011Report This Post
  Powered by Social Strata  

Read-Only Read-Only Topic

Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     [SOLVED] decimal calculation

Copyright © 1996-2020 Information Builders