As of December 1, 2020, Focal Point is retired and repurposed as a reference repository. We value the wealth of knowledge that's been shared here over the years. You'll continue to have access to this treasure trove of knowledge, for search purposes only. Moving forward, myibi is our community platform to learn, share, and collaborate. We have the same Focal Point forum categories in myibi, so you can continue to have all new conversations there. If you need access to myibi, contact us at firstname.lastname@example.org and provide your corporate email address, company, and name.
In a Data Migrator dataflow, I am struggling trying to figure out how best to concatenate a series of columns when any number of them can be null on any given record. In my target transformation editor, I simply use an expression like: first_name || middle_name || last_name. In the transformation calculator, the Sample Data viewer gracefully handles if middle_name is null. However, the Test Transforms viewer returns null for the whole expression, as does the actual running of the dataflow. I could not find any transform function or operator that gracefully handles the null. The only thing I found was back in my SQL object, using the COALESCE() function on every column that may contain a null value. Are there any other ways to gracefully handle null values (i.e., treat null as an empty string) that isn't so manually intensive? Maybe using a different operator than || or some system / flow setting? My real world situation is much more complicated, where I will have dozens of dataflows handling hundreds of columns that may have nulls on any given record. I will literally be writing the COALESCE() function 1000+ times.
Thanks in advance for any insight or advice.This message has been edited. Last edited by: FP Mod Chuck,
Yes, MISSING=ON is present for both the source and target fields. In my (simplified) scenario, the source table has FIRST, MIDDLE, and LAST columns, any of which could be null. My target table has a FULL_NAME column. I actually know that FULL_NAME should never be null because I'll always have at least one source field populated. I just don't know which ones. The Oracle table backing my target has this column as nullable,and I'm guessing MISSING=ON is the default setting in this scenario.
Please let me know if any additional information would be helpful. Thanks.
I'm not a DM person but have you looked at using DEFINEs in the .mas?
In your example, I would create 4 DEFINE fields in the .mas file like this:
DEFINE PFNM/A25=IF FIRST IS MISSING THEN ' ' ELSE FIRST;
DEFINE PMNM/A25=IF MIDDLE IS MISSING THEN ' ' ELSE MIDDLE;
DEFINE PLNM/A25=IF LAST IS MISSING THEN ' ' ELSE LAST;
DEFINE FULL_NAME/A75=PFNM || PMNM || PLNM;
This way you do it once and it is always available. Would something like that work for you?
Another option is:
TABLE FILE FILE1
PRINT FIRST MIDDLE LAST
ON TABLE SET NODATA ' '
ON TABLE HOLD MISSING OFF
DEFINE FILE HOLD
FULL_NAME/A75=FIRST || MIDDLE || LAST;
In FOCUS since 1985. Prod WF 8.0.08 (z90/Suse Linux) DB (Oracle 11g), Self Serv, Report Caster, WebServer Intel/Linux.
Posts: 975 | Location: Oklahoma City | Registered: October 27, 2006
Thanks for the replies. I'm very new to Data Migrator so I had to research these suggestions as much as I could. If I understand correctly, a DEFINE would be needed for each column that could be null. That would probably end up being an even swap between writing a COALESCE vs. a DEFINE for each column, in my case. I could get a certain level of reusability using a DEFINE in the synonym instead of COALESCE in a flow, but still very manually intensive.
The second example of jgelona's sounds like it might work but it's definitely above my understanding of DM right now. Is the snippet representative of a stored procedure in the FOCUS language? I searched the user guide for the keyword NODATA and couldn't find any occurrences. There are a couple of posts on these forums that give me some clues, but can anyone point me to a document / resource that I can read up on how to write my own version of the example?
Another option I came up with is to write out to a delimited file as my target and then read each row back in as a single column. In my real-world scenario, I am trying end of with a target table that has a single column that holds a pipe-delimited string of several columns from a source table (ex. if middle name missing, ...|somevalue|john||smith|anothervalue|...). Creating a pipe delimited target sidesteps the null issue (no longer concatenating values) and having to learn / write in a language I don't know yet, but of course comes at the cost of passing through a set of data two times instead of once. The performance cost may be acceptable if it minimizes the maintenance cost of supporting the flows.
Thanks again for any ideas, directions, and insight.
DEFINE FUNCTION concatnull(s1/A100,s2/A100,s3/A100)
S1/A100V = IF s1 EQ MISSING THEN ' ' ELSE s1;
S2/A100V = IF s2 EQ MISSING THEN ' ' ELSE s2;
S3/A100V = IF s3 EQ MISSING THEN ' ' ELSE s3;
concatnull/A300V = LTRIM(RTRIM(S1 || S2 || S3)) ;
in your edasprof.prf or other commonly executed place.
Then you could use it, for example:
TABLE FILE systable
COMPUTE i/I9 = i + 1;
-* Make up some null/missing values to test with.
COMPUTE NAMEN/A128 MISSING ON = IF i EQ 1 THEN MISSING ELSE NAME;
COMPUTE TYPEN/A128 MISSING ON = IF i EQ 2 THEN MISSING ELSE TYPE;
COMPUTE CREATORN/A512 MISSING ON = IF i EQ 3 THEN MISSING ELSE CREATOR;
COMPUTE TESTCONCAT/A50 = concatnull(NAMEN, TYPEN, CREATORN) ;
WHERE READLIMIT IS 4
Posts: 210 | Location: Sterling Heights, Michigan | Registered: October 19, 2010
Scott, actually there are 2 SET options, NODATA and HNODATA. By default, in an alpha format hold file, if the source field is null FOCUS use a period to represent a null/missing data value. NODATA and HNODATA allow you to override that. Here's a link to the 8.2.01 SET commands.