Focal Point
PRECISION REDUCTION...

This topic can be found at:
https://forums.informationbuilders.com/eve/forums/a/tpc/f/7971057331/m/4901065822

May 18, 2007, 12:12 PM
gregson06
PRECISION REDUCTION...
Okay, here's the scenario. I have a field which is formated to be D23.9, but I need to shrink it to D1.2 for some data file specs I need to match. I've tried doing an FTOA to A32, then an EDIT of the 4 characters that hold the number I need, but when I try to convert back to decimal, Webfocus tells me that it as an invalid format. It seems to not like ATODBL to D1.2. Is there another way to do this?

Thanks!
May 18, 2007, 12:21 PM
Leah
In the D format, you have to allow in the first number the total number of digits, so do you want say 3456789.123456 to become 3.45?


Leah
May 18, 2007, 12:37 PM
gregson06
Well, that's just the thing, the number is never greater than the one's digit, but can have lots of trailing decimals. For example. it can be 9.999, but not 10.0. But the format of the field allows it to be 99999999.999, I'd only want the 9.99.
May 18, 2007, 12:45 PM
Alan B
Remember that in the format D6.4, the 6 is the total characters including the decimal point, so you could display 1.3456, but not 12.4567, which would show as *s'


Alan.
WF 7.705/8.007
May 18, 2007, 12:57 PM
gregson06
Well, I am just not sure if I am going about it the right way. It seems that because I am changing the large decimal to alpha, that when I do the 'EDIT' that this is when it gets messy. What is happening now is that instead of a number being, for example, 2.38, I only get 2.00 or 2.30 (sometimes). Would it matter if I am getting the decimal point with a 9 on the edit? Like EDIT(45.9393,'$9999'), this normally yields 5.93 in alpha, but when i do the decimal convert, it goes to 5.00 or 5.9 at best.
May 18, 2007, 01:16 PM
Alan B
I would use the following type of code:
NUMB/D16.3=123456789012.456;
ANUMB/A20=FTOA(NUMB,'(D16.3)',ANUMB);
ENUMB/A5=EDIT(ANUMB,'$$$$$$$$$$$$$$99999');
DNUMB/D5.3=ATODBL(ENUMB,'5',DNUMB);



Alan.
WF 7.705/8.007
May 18, 2007, 01:20 PM
ET
Try this.

Good luck

et

DEFINE FILE CAR
ALPHA1/A7 WITH COUNTRY='45.9393';
ALPHA2/A4=EDIT(ALPHA1,'$9999');
AMT1/D4.2=EDIT(ALPHA2);
END
TABLEF FILE CAR
PRINT ALPHA1 ALPHA2 AMT1
IF RECORDLIMIT EQ 1
END


ALPHA1 ALPHA2 AMT1
------ ------ ----
45.9393 5.93 5.93


FOCUS 7.6 MVS PDF,HTML,EXCEL
May 18, 2007, 01:23 PM
Darin Lee
As Alan says, D1.2 would not be valid. Minimum for what it looks like you need would be D4.2 (1 for ones, 1 for decimal point and 2 for decimal places. )


Regards,

Darin



In FOCUS since 1991
WF Server: 7.7.04 on Linux and Z/OS, ReportCaster, Self-Service, MRE, Java, Flex
Data: DB2/UDB, Adabas, SQL Server Output: HTML,PDF,EXL2K/07, PS, AHTML, Flex
WF Client: 77 on Linux w/Tomcat
May 18, 2007, 02:03 PM
gregson06
Thanks for the responses!

ET's solution works like a champ, it even rounds the decimal up on the last position.

Al, you're absolutely right, I think that the application takes into consideration total number of units in that first digit after D, so I had to use D4.2 and D1.2 will not work.

Thank you again for the assistance, this seems to be so many steps for such a "logical" procedure.
May 18, 2007, 02:11 PM
Alan B
Please beware of using EDIT , as in:
field/A10=EDIT(DFIELD);
when converting from numeric to alpha on numbers gt 9 digits, there is a maximum number which edit will deal with, which I forget, before you start getting errors.
NUMB/D16.3=123456789012.456;
ENUMB/A16=EDIT(NUMB);
will result in 1097262572 for example.


Alan.
WF 7.705/8.007
May 18, 2007, 02:22 PM
gregson06
Is this anomaly across all versions of WebFOCUS? I wonder why IBI hasn't patched it...
May 18, 2007, 02:57 PM
ET
I believe that this is because the edit function when converting from numeric to alpha, is limited to handling the largest nummber that can be stored in a 4 byte binary field. The bad thing is that when the number is exceeded it does not tell you by giving you an error. It just gives you the wrong number. I know it is in the documentation NOW. It wasn't in the documentation of real early mainframe focus manuals that I learned with. I still have a manual from 1983 where no mention is made of this fact. I had to find out the hard way when production applications sporadically had wrong numbers.

It does appear to work okay the other way (alpha to numeric) as the following code shows. Maybe those luckly ones attending Summit can ask your question as to when this can be corrected and post an answer here.

et


DEFINE FILE CAR
ALPHA1/A15 WITH COUNTRY='999999999999.11';
AMT1/P15.2=EDIT(ALPHA1);
END
TABLEF FILE CAR
PRINT ALPHA1 AMT1
IF RECORDLIMIT EQ 1
END
ALPHA1 AMT1
------ ----
999999999999.11 999999999999.11


FOCUS 7.6 MVS PDF,HTML,EXCEL
May 18, 2007, 02:59 PM
Darin Lee
Funny you should bring that up, Alan. We ran into that just yesterday. We were trying to convert a 10-digit policy number but were getting *s. The EDIT function would only handle up to 9 digits so it will convert 999999999 but not 9999999999.
Also, with the rounding issue, anything 9.995 and over would round to 10.00 which doesn't fit in D4.2 and would give an error. Doing the alpha with truncation and converting back would probably be safest.


Regards,

Darin



In FOCUS since 1991
WF Server: 7.7.04 on Linux and Z/OS, ReportCaster, Self-Service, MRE, Java, Flex
Data: DB2/UDB, Adabas, SQL Server Output: HTML,PDF,EXL2K/07, PS, AHTML, Flex
WF Client: 77 on Linux w/Tomcat
May 20, 2007, 08:26 PM
Piipster
How about this:

DEFINE FILE EMPDATA
D_D239/D23.9 = 000009.999999999;
-* The I format doesn't round.
-* Times 100 to get the significant digits
D_I3/I3 = D_D239 * 100;
-* back to D format for the decimals
D_OUT/D4.2 = D_I3 /100;
END
TABLE FILE EMPDATA
PRINT
D_D239
D_I3
D_OUT
BY PIN NOPRINT
IF RECORDLIMIT EQ 1


ttfn, kp


Access to most releases from R52x, on multiple platforms.