Focal Point Banner


As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only.

Join the TIBCO Community
TIBCO Community is a collaborative space for users to share knowledge and support one another in making the best use of TIBCO products and services. There are several TIBCO WebFOCUS resources in the community.

  • From the Home page, select Predict: WebFOCUS to view articles, questions, and trending articles.
  • Select Products from the top navigation bar, scroll, and then select the TIBCO WebFOCUS product page to view product overview, articles, and discussions.
  • Request access to the private WebFOCUS User Group (login required) to network with fellow members.

Former myibi community members should have received an email on 8/3/22 to activate their user accounts to join the community. Check your Spam folder for the email. Please get in touch with us at community@tibco.com for further assistance. Reference the community FAQ to learn more about the community.


Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     Help needed with MISSING data and WHERE statement

Read-Only Read-Only Topic
Go
Search
Notify
Tools
Help needed with MISSING data and WHERE statement
 Login/Join
 
Expert
posted
Something doesn't seem quite right here regarding MISSING data.

I have a simple fex where I want to compare two fields in a HOLD file generated by an ACROSS statement.

In the sample code below I have two sets of reports: REPORT 1A, REPORT 1B and REPORT 2A, REPORT 2B. (I've also thrown in REPORT 0 to display the data I'm working with.

REPORT 1A and REPORT 2A display the data without a WHERE statement.

REPORT 1B has a WHERE statement and it seems to work - it shows no results because no values are equal.

REPORT 2B has a WHERE statement which is not working - it shows results even though the values are different.

The only difference between the two sets of reports is the extra BY statement in the second set.

I have HOLDMISS set to ON because it "Allows you to store missing data in a HOLD file. When TABLE generates a default value for data not found, it generates missing values".

I would like my EQ test to differentiate between a zero and missing data. I'd like to understand why the following two rows appear in REPORT 2B:

SEATS BODYTYPE    SALENGLAND SALITALY 
----- ----------- ---------- --------
    2 CONVERTIBLE          0      ??? 
    2 HARDTOP              0      ??? 

The code:
SET BYDISPLAY=ON
SET PAGE=NOPAGE
SET NODATA='???'
SET HOLDFORMAT=ALPHA
SET HOLDLIST=PRINTONLY
SET ASNAMES=ON
SET HOLDMISS=ON
SET COMPMISS=ON

TABLE FILE CAR
SUM
SALES
BY COUNTRY
BY SEATS
BY BODYTYPE
WHERE COUNTRY IN ('ENGLAND', 'ITALY')
HEADING
"REPORT 0"
END

-*-----------------------

TABLE FILE CAR
SUM
SALES
BY SEATS
ACROSS COUNTRY
WHERE COUNTRY IN ('ENGLAND', 'ITALY')
ON TABLE HOLD AS H001
END

TABLE FILE H001
PRINT *
HEADING
"REPORT 1A"
END

TABLE FILE H001
PRINT *
WHERE SALENGLAND EQ SALITALY
HEADING
"REPORT 1B"
END

-*-----------------------

TABLE FILE CAR
SUM
SALES
BY SEATS
BY BODYTYPE
ACROSS COUNTRY
WHERE COUNTRY IN ('ENGLAND', 'ITALY')
ON TABLE HOLD AS H002
END

TABLE FILE H002
PRINT
*
HEADING
"REPORT 2A"
END

TABLE FILE H002
PRINT
*
WHERE SALENGLAND EQ SALITALY
HEADING
"REPORT 2B"
END


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
 
Posts: 10577 | Location: Toronto, Ontario, Canada | Registered: April 27, 2005Report This Post
Virtuoso
posted Hide Post
Hi Francis

Interesting behaviour. I can almost understand why this happens, not sure if it has the element of least surprise though.

SALES is an integer field. In the hold file, the character used to show missing is '.'. This can be controlled by HNODATA. A basic comparison between a zero and a '.' becomes true. Wondering if this is because '.' translates to 0, or to blank, which in a numeric field is 0.

Converting Sales to Alpha, this will work straight off.

Or, keeping with Integer, or any numeric, the only approach seems to be to go back to basics and use a DEFINEd FLAG, FLAG/A1 MISSING ON NEEDS ALL = IF SALENGLAND EQ SALITALY THEN 'Y' ELSE 'N';

Or, setting HNODATA to -1, would work, but then missing data shows as -1. Not perfect.

So if there was a WHERE_MISSING_ON_NEEDS_ALL … type option, this would not occur.


Alan.
WF 7.705/8.007
 
Posts: 1451 | Location: Portugal | Registered: February 07, 2007Report This Post
Virtuoso
posted Hide Post
Think you're right, Alan.
Because if you add in the last request'WHERE SALITALY NE MISSING AND SALENGLAND NE MISSING;' you then get the correct result. This means that the missing test is done correctly, but the eq test isn't quite correct, imho.


GamP

- Using AS 8.2.01 on Windows 10 - IE11.
in Focus since 1988
 
Posts: 1961 | Location: Netherlands | Registered: September 25, 2007Report This Post
Virtuoso
posted Hide Post
It looks like MISSING integer values get silently cast to 0 in equality tests.
That makes me wonder, if both are MISSING, is the result of the comparison still a match? What is the result of MISSING EQ MISSING?

I don't think the result of a comparison with MISSING should ever be true, just as it is with NULLs in SQL (http://www-cs-students.stanford.edu/~wlam/compsci/sqlnulls).
What would be the correct result for N EQ MISSING?

Is that FALSE?
If I don't have a pot of flowers and I want to know whether they're red, well, if they were red I could see them. They're not green or blue or any other colour either.

Or MISSING?
I'd love to tell you whether those flowers are red or not, but I don't have any flowers to compare so I can't answer that question at this time.

Or even UNKNOWN?
I don't know whether those flowers might have been red had I had them.


WebFOCUS 8.1.03, Windows 7-64/2008-64, IBM DB2/400, Oracle 11g & RDB, MS SQL-Server 2005, SAP, PostgreSQL 11, Output: HTML, PDF, Excel 2010
: Member of User Group Benelux :
 
Posts: 1669 | Location: Enschede, Netherlands | Registered: August 12, 2010Report This Post
Virtuoso
posted Hide Post
quote:
I don't think the result of a comparison with MISSING should ever be true

Ah, Philosophy 101.

The result of the FLAG above, with NEEDS ALL, is MISSING. With NEEDS SOME, is TRUE. So the NEEDS SOME, ignores the MISSING value, relying on the values that do exist, ergo there is a value so TRUE, whereas with NEEDS ALL, then there is no evaluation if any item does not exist, therefore becomes MISSING, or does not exist.


Alan.
WF 7.705/8.007
 
Posts: 1451 | Location: Portugal | Registered: February 07, 2007Report This Post
Member
posted Hide Post
I believe if you look at the source data some have sales of Zero and some have sales of missing so the Zeroes are overriding the missing.

TABLE FILE CAR
SUM
SALES
BY SEATS
ACROSS COUNTRY
WHERE COUNTRY IN ('ENGLAND', 'ITALY')
ON TABLE HOLD AS H001
END

TABLE FILE H001
PRINT *
END

TABLE FILE CAR
PRINT
SALES
BY SEATS
ACROSS COUNTRY
WHERE COUNTRY IN ('ENGLAND', 'ITALY')
END


Developer Studio Release : 7.6.11
 
Posts: 21 | Location: TataSteel Strip Products UK , Port Talbot , South Wales | Registered: November 28, 2006Report This Post
Guru
posted Hide Post
I'm sure you much more experienced focal pointers already thought of this, but a simple fix to me would be to compare apples to apples and make the
SET NODATA=0 
then use
   WHERE SALENGLAND EQ SALITALY AND (SALENGLAND NE 0 OR SALITALY NE 0 ); 
in report 2B.


WebFOCUS Server 8.1.05
Windows 2008 Server
WebFOCUS AppStudio 8.1.05
Windows 7 Professional
IE 11 and Chrome Version 43.0.2357.124 m.
Mostly HTML, PDF, Excel, and AHTML
 
Posts: 272 | Location: Kalamazoo, Michigan | Registered: September 30, 2010Report This Post
Expert
posted Hide Post
Thanks very much for this conversation!

FYI, the NODATA character is for display only and has no impact on the EQ test. I deliberately did not use zero so I could see what's happening. I also tried changing the format from integer to Dnn to mimic my real-world program and that made no difference.

I ended up with this:

-* Include values that are not equal
WHERE SC_AMT1 IS MISSING OR SC_SMT2 IS MISSING OR SC_AMT1 NE SC_AMT2;

In my data, I know I will never have missing values in both columns.

This message has been edited. Last edited by: Francis Mariani,


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
 
Posts: 10577 | Location: Toronto, Ontario, Canada | Registered: April 27, 2005Report This Post
Virtuoso
posted Hide Post
You could also use function ASIS, which seems to know the difference between zero and missing values:

WHERE ASIS(SALENGLAND) EQ ASIS(SALITALY);


WebFOCUS 7.7.05
 
Posts: 1213 | Location: Seattle, Washington - USA | Registered: October 22, 2007Report This Post
Platinum Member
posted Hide Post
As a matter of interest - if you simply use a WHERE clause like:
WHERE SALENGLAND EQ 0;
you will get both zeroes and missing values.
I recall bumping into this previously, and it is in fact documented that way in the manual.
Just BTW - I tried the WHERE clause with ASIS as per Dan's suggestion on WF 8.0.01, and it isn't very happy with me:
(FOC263) EXTERNAL FUNCTION OR LOAD MODULE NOT FOUND: ASIS


WebFOCUS 8.2.06 mostly Windows Server
 
Posts: 195 | Location: Johannesburg, South Africa | Registered: September 13, 2008Report This Post
Expert
posted Hide Post
I've always known ASIS to be a Dialogue-Manager-only function...


Francis


Give me code, or give me retirement. In FOCUS since 1991

Production: WF 7.7.05M, Dev Studio, BID, MRE, WebSphere, DB2 / Test: WF 8.1.05M, App Studio, BI Portal, Report Caster, jQuery, HighCharts, Apache Tomcat, MS SQL Server
 
Posts: 10577 | Location: Toronto, Ontario, Canada | Registered: April 27, 2005Report This Post
Platinum Member
posted Hide Post
Hi Francis,
Yes, that is what I thought too.
Explains why it does not work then.


WebFOCUS 8.2.06 mostly Windows Server
 
Posts: 195 | Location: Johannesburg, South Africa | Registered: September 13, 2008Report This Post
  Powered by Social Strata  

Read-Only Read-Only Topic

Focal Point    Focal Point Forums  Hop To Forum Categories  WebFOCUS/FOCUS Forum on Focal Point     Help needed with MISSING data and WHERE statement

Copyright © 1996-2020 Information Builders