Focal Point
[SOLVED] Data Migrator 8.2 -- READLIMIT in Sample Data?????

This topic can be found at:

December 03, 2019, 09:59 AM
[SOLVED] Data Migrator 8.2 -- READLIMIT in Sample Data?????
I'm moving materials from an 8.1 install to an 8.2 install, and I can't get my units to pull recent data when I'm running the Sample Data option. That is, "give me records from the last 8 hours." There's data in the table, but Data Migrator's Sample Data function keeps pulling zero records.

Is my unit broken, or does the tool simply not do what it does in the 8.1 version anymore? It just seems to be saying Nope for no reason, and I'm stuck trying to figure out if it's a tool problem or a source code problem.

This message has been edited. Last edited by: FP Mod Chuck,

December 03, 2019, 10:27 AM
Alright, so I told the tool to return 250,000,000 records for my sample, and I got results.

This is a really un-useful change in functionality for my location. Please tell me there's a way to configure this to run it as a RECORDLIMIT instead of a READLIMIT as it was in prior versions.

December 03, 2019, 11:53 AM
FP Mod Chuck

I reached out to product management and their recommendation is to open a case to get assistance, he was not sure why you would have to do that.

Thank you for using Focal Point!

Chuck Wolff - Focal Point Moderator
WebFOCUS 7x and 8x, Windows, Linux All output Formats
December 03, 2019, 12:03 PM
Yeah, about that . . . opening a case means I'm working for Information Builders instead of my client, but billing my client. I can't burn hours on their dime debugging Data Migrator.

When I pull up the last command from the agent on the console I see both record limit and read limit --


8.1 shows both as well, but returns the 2500 records even though they're at the far end of the database. 8.1 is functioning differently for identical units. 8.1 is also dramatically faster (like, 60 to 1 faster) so there's something going on with my 8.2 install that's pretty much in the way of upgrade at the moment.

Thanks for responding so quickly.

December 03, 2019, 01:38 PM
This is what is killing me --


That's in the WHERE clause in the transform, and it makes the transform slow as molasses in the run and produces a zero records in the sample data. When I hard coded today's date it produced sample data, and ran only about half as fast as 8.1 (a vast improvement and within my requirements).

So I need to figure out how to have is select today's records without using the select above, which takes about ten minutes to run.

I've opened a case and asked for a recommendation, hopefully someone will have one for tomorrow morning.

This message has been edited. Last edited by: John_Edwards,

December 04, 2019, 08:38 AM
It would be helpful to know what is the data source? Also what is the full version 8.2.??
December 04, 2019, 08:54 AM
It's 8204 reaching out to SQL*Server. I've opened a ticket, likely will land in your group. Thanks for responding.

December 04, 2019, 10:37 AM
The retrieval limits are pass through to SQL Server as SELECT TOP 2500 so I don't think that's the issue here. BTW you can see this from the Server Session Log after running a query.

We didn't have an 8204 release of DataMigrator so presumably that's a WebFOCUS release which would have used an 8203/7.7.08 Server which was GA in 04/2018.

I tested and back then your the in the WHERE condition it would have translated CURRENT_TIMESTAMP(3) to CONVERT(DATETIME,'2019-12-04 10:23:05.000', 120). Today that same function is translated to SYSDATETIME() which would be more expedient.

These kinds of incremental improvements are one of the reasons why it's so important to use the current production release, now 8207.
December 04, 2019, 11:22 AM
Gen #1337 8203M 64-bit

So . . . do you have a recommendation for me on 8203M? This is a job that pulls a couple of thousand records and it's taking 15 minutes in the new version. Is there a better way to say "all records created today"? I'm running some options but not getting significant improvement in speeds.

FYI -- It's costing my client tens of thousands of dollars to upgrade to 8204. I appreciate that IB doesn't charge for the upgrade, but it's a big expense in man-hours for the software owner. Upgrading is not something that you do every six months, not something that any customer with a budget does for every released version. We stay on older version for a reason. In state government it's as much about having enough warm bodies to make the transfer, as governments are running super-lean on staff all over the country.

December 04, 2019, 11:54 AM
You could use DB_EXPR(SYSDATETIME()) to get the current datetime.
December 04, 2019, 12:10 PM
That gets me to 52 seconds, which is fast enough for a two-minute update. Thanks for the rec!