Focal Point Banner
Community Center Education Summit Technical Support User Groups
Let's Get Social!

Facebook Twitter LinkedIn IB Bloggers
Focal Point    Focal Point Forums  Hop To Forum Categories  iWay Software Product Forum on Focal Point    [SOLVED]How to write DataMigrator logs to Vertica DB
Go
New
Search
Notify
Tools
Reply
  
[SOLVED]How to write DataMigrator logs to Vertica DB
 Login/Join
 
Member
posted
We have a number of flows and processes running daily through DataMigrator. We use Datamigrator to control both our ETL process as well to control the processing of our various analytics performed on our Vertica databases. Because of this, we'd like to retain logs for up to a month (or longer) to reference in case we need to debug a faulty procedure.

Instead of storing these log files on the local computer, we were wondering if we could instead direct the logs to write to a table in our Vertica Database. We could handle any archiving/retention policies easily there as well as save the space on our hard drive.

I've found a brief mention in the docs of the potential to do something like this, but haven't been able to find anything detailed. Has anyone had any experience with doing something like this?

This message has been edited. Last edited by: Tamra,


DataMigrator 7.7.06
Windows Server 2012
 
Posts: 4 | Location: United States | Registered: October 19, 2016Reply With QuoteReport This Post
Virtuoso
posted Hide Post
I've done this for very much the same reasons, but with a slight change. I don't believe that you can use anything other than the existing logging mechanism. So we left that alone. What we did was set up a reoccurring job to extract the log changes and load them into an alternate DB. We keep that live logs as small as possible, and delete data from the external log repository weekly.


"There is no limit to what you can achieve ... if you don’t care who gets the credit." Roger Abbott
 
Posts: 1029 | Location: Toronto, Ontario | Registered: May 26, 2004Reply With QuoteReport This Post
Member
posted Hide Post
Thanks dhagen. After a couple of days mulling it over. That's the approach we decided to take as well. Was hoping we could implement a more direct approach, but alas, this'll have to do.


DataMigrator 7.7.06
Windows Server 2012
 
Posts: 4 | Location: United States | Registered: October 19, 2016Reply With QuoteReport This Post
  Powered by Social Strata  
 

Focal Point    Focal Point Forums  Hop To Forum Categories  iWay Software Product Forum on Focal Point    [SOLVED]How to write DataMigrator logs to Vertica DB

Copyright © 1996-2017 Information Builders, leaders in enterprise business intelligence.