Your signature says WebFOCUS 7.6 but since support for sub queries was added in DataMigrator Release 7.7 I'm going to assume that you are using the current release.
A FOC14019 is a SQL Translator general error. What relational database is your source? I think to diagnose this further it may be best to open a hottrack case and provide the three source synonyms and data flow.
Alternatively you could take another approach.
If PRODUCT_KEY is a the key of d_product it can't be NULL in the source table, so the only NULL values would be those created
by the OUTER JOIN
So it looks like what you want is all the rows f_rso_stg_ds where the VENDOR_KEY is not found in d_product.
If that's correct instead of a SUB-SELECT and a JOIN another way to obtain that result is with a WHERE condition:
NOT EXISTS(SELECT * FROM d_product SQ WHERE SQ.PRODUCT_KEY = VENDOR_KEY)
You could add another WHERE condition to check the xref_product table. Then you wouldn't need any JOINS to get the results.