Has anyone ever experienced an issue with the WHERE clause in which a hard-coded value to compare works great, but a defined variable with seemingly the same value doesn't? Objective here is to just grab data from the previous month.

I have a date field that is in Julian format, and if I set my where like the following then the report returns the correct data within 3 minutes:

   
WHERE ( J0.RR174_TMP00701.THEFDT GE '2016061' ) AND ( J0.RR174_TMP00701.THEFDT LE '2016091' );
 

Obviously I don't want to hard-code this because it will be ever-changing, so I set the defines as follows:

  
TODAY/MDYY='&DATEMDYY';
-*get previous month
PM_YYMD/YYMD=DATEADD(TODAY, 'M', -1);

-*convert begin of previous month to Julian as int
BOPM_YYMD/YYMD=DATEMOV(PM_YYMD, 'BOM');
BOPM_I8YYMD/I8YYMD=BOPM_YYMD;
BOPM_I8/I8=BOPM_I8YYMD;
BOPM_JUL/I7=JULDAT(BOPM_I8, 'I7');

-*convert end of previous month to Julian as int
EOPM_YYMD/YYMD=DATEMOV(PM_YYMD, 'EOM');
EOPM_I8YYMD/I8YYMD=EOPM_YYMD;
EOPM_I8/I8=EOPM_I8YYMD;
EOPM_JUL/I7=JULDAT(EOPM_I8, 'I7');
 

Then once I change my where clause to use these defined values the report runs forever (I killed it after an hour):

   
WHERE ( J0.RR174_TMP00701.THEFDT GE BOPM_JUL ) AND ( J0.RR174_TMP00701.THEFDT LE EOPM_JUL );
 

Things I have tried to no avail:
-THEFDT in WF is set as P8, so changed the "_JUL" defines to P8, no dice.
-Since my hard-coded values were alpha I tried converting my Julian date then to alpha like:

 EOPM_ALPHA/A7=EDIT(EOPM_JUL); 

and used that within my where clause but it threw a conversion error.
-I tried joining to a date conversion table we have and then comparing the YYMD equivalent in that table to the BOPM_YYMD instead of using THEFDT and I still kill it after 20 minutes. However once again if I hard code something like '2016/03/01' into the WHERE it returns in 3 minutes.
-Turned some of my joins into HOLD files and scraped off of those, no change.
-Used SET instead of defines for the variables, no change.
-Spit out the variables to the report to make sure the proper values are set, and indeed they are.

The next two things I'm going to try are setting defines to the value of THEFDT so I can then be sure the format is the same when compare with my WHERE, and if that doesn't work I'm going to work on paring down the data set even further.

I've spent over a day on this now, and this just doesn't make sense and I'm at wits end.

Any suggestions or theory's?

Thanks,
JC

This message has been edited. Last edited by: jcannavo,

Have you tried tracing the TABLE request to see what SQL WebFOCUS generates and why?

Before the TABLE request:

 SET TRACEOFF = ALL
 SET TRACEON = STMTRACE//CLIENT
 SET TRACEON = SQLAGGR//CLIENT
 SET TRACESTAMP = OFF
 SET TRACEUSER = CLIENT
-* Don't actually fetch the data
 SET XRETRIEVAL = OFF

And afterwards:

 SET TRACEOFF = ALL

Hello, I have not tried that. I'll give it a shot and report back.

Thanks

Using DEFINE to create a temporary column for a WHERE is an inefficient method to filter values in a database table: the DEFINE is processed for each database table row, some WebFOCUS functions cannot be translated to SQL.

A much better (and the correct) method is to use Dialogue Manager to compute the values you want in the WHERE.

-SET &TODAY = &YYMD;

-*get previous month
-SET &PM_YYMD = DATECVT(DATEADD(DATECVT (&TODAY,'I8YYMD','YYMD'),'M', -1), 'YYMD','I8YYMD');

-*convert begin of previous month to Julian as int
-SET &BOPM_YYMD = DATECVT((DATEMOV((DATECVT(&PM_YYMD,'I8YYMD','YYMD')),'BOM')),'YYMD','I8YYMD');
-SET &BOPM_JUL = JULDAT(&BOPM_YYMD, 'I7');

-*convert end of previous month to Julian as int
-SET &EOPM_YYMD = DATECVT((DATEMOV((DATECVT(&PM_YYMD,'I8YYMD','YYMD')),'EOM')),'YYMD','I8YYMD');
-SET &EOPM_JUL = JULDAT(&EOPM_YYMD, 'I7');

-? &BO
-? &EO

...

WHERE ( J0.RR174_TMP00701.THEFDT GE &BOPM_JUL ) AND ( J0.RR174_TMP00701.THEFDT LE &EOPM_JUL );

Note that Date functions work with Date fields, so, for Dialogue Manager, use DATECVT to convert the Dialogue Manager string to Date, then use the Date function, then convert back to string.

Francis and Wep,

Thanks for the quick responses! Definitely the trace returned back saying JULDAT and DATEMOV couldn't be converted to SQL, so those WHERE clauses were completely getting ignored which was then returning a crap-load of data back to check against. So that bloat was definitely the problem. I didn't know that tracing code so that is fantastic, thanks for that!

As I mentioned in my original post I did try to use SET commands for the variables, but I must have done something wrong because they didn't work yesterday but what Francis sent worked like a charm. I need to roll this into my main FEX now because I had broken this piece out to troubleshoot. Once I do that after my lunch hour and if everything works I'll report back and close the discussion.

Thanks again you two!

-JC

Rolled the set commands into my main FEX and viola, she works.

Thanks again!
Joe

This forum is so cool!

Right now (even indoors) it feels colder than cool! Something about a Polar Vortex.

Haven't seen the sun for days in the midwest.

What format is the juian date column ?

Hi Waz,

The format of the Julian date in WF was P8.

Its interesting that the format is P8 and you were testing against an alpha literal.

Yeah, I know. It somehow made the conversion and worked. I eventually removed the ticks to avoid any confusion, but now my variables are working so I need not worry about that. Happy Friday!