Clarity

Expand all | Collapse all

Data Migration Opinions?

  • 1.  Data Migration Opinions?

    Posted Sep 30, 2009 05:15 AM
    Hi, this is a cross-post to an interesting thread in  linkedin  about data migrations.  (The other reason is I want to upload an image to discuss at that site. Is this bad? )  The thread starts with question about using VBA, proceeds through several opinions into debate on if GEL is flexible or not (I vote it is)  Here I'm suggesting the opinion will be affected by 'desire to script' and 'frequency or volume of need'.    

    Attachment(s)



  • 2.  Re: Data Migration Opinions?

     
    Posted Oct 01, 2009 07:33 AM
    Thanks Paul.


  • 3.  Re: Data Migration Opinions?

    Posted Oct 02, 2009 07:19 AM
    This is how I perceived the discussion.  Martti K.  


  • 4.  Re: Data Migration Opinions?

    Posted Oct 03, 2009 05:03 AM
    Martti - thanks, I was struggling to arrive at the best axes - these are good.  


  • 5.  Re: Data Migration Opinions?

    Posted Oct 03, 2009 10:39 AM
    Just wondering again if there are more dimensions  Like one is definitely if the method and tool can be used as such or if code writing or scipting is needed or if in fact the tool is actually a development tool.  Another is the set up required. In the ideal world if you ask for the names and codes of the 20 top level units and their subunits on an Excel sheet you are likely to get the results at least in 20 different formats even if you have supplied the form. What I mean is the time after you have made the Excel sheet into one single format and   start getting your tool ready to process the Excel data until you hit go.  Data size could another dimension. For a single record manual edit might be the fastest way, but if you have thousands of records then I've used Excel formulas and mail merge. So that leads to combining data size to set up: set up time per record.   But then again that leaves still the data size a candidate for a dimension: the XOG.client GUI can't handle throusands of records. Another aspect of the is if the method and tool can process one record at a time so that there is not practical limit to the data size of will it load it all in memory when there is a limit.  Another thing to consider is that will the method and tool just create sort of XML, syntax checked XML or will it do the actual transfer as well.  That brings it back to the user: does the user need to understand more than to be able to read an Excel sheet and differentiate between a successfull run a failed run, or also   XML, syntax checking, client set up, java VM tuning and code generation with various development tools.    Then again, can it all be flattened to one dimension:  On one end there is OOTB as no method and at the other end is a tool with user friendly GUI which will take several different types of input, allow WYSIWYG editing, automatically checks XML schema and syntax, does the transfer for any amount of data without crashing the environment and which doesnot exist or emerge in the foreseeable future. Everything else is between those to, exactly where between those two depends which method is chosen and which skills does the user have. Then again apparently it cannot be flattened to one dimension.  Martti K.    


  • 6.  Re: Data Migration Opinions?

    Posted Oct 04, 2009 11:01 PM
    It is certainly a multi-dimensioned issue with the only reason for proposing the chart to help slow the expression of personal opinion and to concentrate on methods of solution.  For instance there are those who love VBA - I don't. Others prefer PERL (which is OK, I guess, it is XML aware). GEL is good in my view, GEL with XSLT has great potential. BUT: My need for a generic data mapping solution is also different to a typical user, because I do deal with many needs for "data load" with the enourmous variability that clients data is in being a big factor for me.  The thing is that we could spend time/effort with any of the possibilities (VBA, PERL, GEL, XSLT etc etc) - but is this the best approach? Note that "data load" isn't a unique characteristic of Clarity - introducing almost any new product is likely to have a similar need. So to me seeking existing generic approaches, open source or commercial (eg. altova's mapforce - which I have never tried I must say) is an attractive idea to me - instead of scripting at all.  


  • 7.  Re: Data Migration Opinions?

    Posted Oct 04, 2009 11:38 PM
    I was thinking that everybody would prefer just to get cracking with the least set up and without to need to do any coding with any tool. But maybe there are users who like coding more than preparing the data and doing the actual transfer successfully. Yes I've told that Mapforce can transfer the data into XML ready for XOG and would like to get the hang of it, but the threshold is higher than to the Office tools.My understanding is that you could get an idea with the eval version.  Martti K.


  • 8.  Re: Data Migration Opinions?

    Posted Oct 05, 2009 01:13 AM
    'everybody' is a broad term - there are those who love to develop "better" ways - and we should encourage this or we stagnate  'most' end-users really only want the results of a data load and are not interested at all in how it is achieved  'solution providers' (like myself) do have to concern ourselves with 'how' it is achieved (and how it is validated too)  I do not regard myself as expert in this area, but have had my time doing data loads into a variety of products using a variety of approaches - all with some degree of frustration.  I mention mapforce only because it was previously mentioned in the Linkedin thread - there are other commercial offerings - and as I mentioned before I have not used that product. It is available for evaluation by download - but who has the time for such luxuries?  


  • 9.  Re: Data Migration Opinions?

    Posted Oct 05, 2009 04:08 AM
    I regard myself a user, but have always wanted to do things better. Years ago I was told by a senior executive that all youg engineers have the same problem: when they see something they want to improve it.Time has told what became of that company.  But, yes I do want to use what is best and easiest process and use the best tool, but this is where the difference seems to be, I don't want to develop or code the tool, but pick one from the existing ones.My understanding of the Mapforce is that the exntreise version can read a variety of data source Excel and and another isntance of Clarity db among them for the data and defining the data source details and the xml elements and attributes happens in one process. It will also check the xml syntax ingeneral and against the schema. So that is one of the better tools you cna get without the need for coding.  Martti K.


  • 10.  Re: Data Migration Opinions?

    Posted Oct 05, 2009 09:43 AM
    Hi,  i think martti is very clear about the tool requirement,his experience speaks and that is correct,we need an userfriendly tool.  Altova Mapforce surely my favourite choice.we use altova xml spy for xogging and found it is very useful.  When we did the upgrade from Niku 6 - Clarity 7.5.2 migrating the custom forms,issues,risks we   encaged severn consultants (steve godrich - Tall guy),i eagerly waited for the migration.I thought that he will
    be doing via excel VBA templates.But he used Altova Map force with ease,i know that he finished the migration in a quick time than we expected (But only realeased on that scheduled day -- Business tricks) and the tool is very user friendly.There we can view the Database table columns directly and map with our columns in template.i recently downloaded the trial verion (But it will not have full features)but not used as we can only use an licenced one,but iam trying to install home PC.i   insisted my PM to buy that and happy that it is approved for our proposed v12 migration as the cost is also nominal (for companies). i will explore on the tool.                                               I feel CA should provide or Buy ot Takeover(that's famous now) an tool (May be altova) with the product itself(with little extra cost) which should be used with ease by technical / functional guys and it must have GUI Mode (Drag & Drop).The reason is that the techno / functional people are not programmers in the way(Now it is hard to do it in perl,vb,asp.net) and also we are notgiven enough time for coding(i hate perl) What we need is an user friendly tool......
    cheers,
    sundar


  • 11.  Re: Data Migration Opinions?

    Posted Oct 05, 2009 09:50 PM
    Hi, GuysIf you have any useful material about XML Spy, please share with our forum readers. This would help us to get real benefit from this kind of tools.


  • 12.  Re: Data Migration Opinions?

    Posted Oct 05, 2009 10:33 PM
    Altova products are athttp://www.altova.com/products.htmlYou can download 30 day trials.In 2006 there was   XML Spy home which was free, but it is discontinued.XML Spy is XML editor  Map Force is the tool which can read data from different sources and create the XML files for Clarity.Online training (free) is under supporthttp://www.altova.com/aot/online-training.aspx    Martti K.  


  • 13.  Re: Data Migration Opinions?

    Posted Oct 05, 2009 10:56 PM
      |   view attached
    Hi,  Pls find the user guide for altova xml spy 2008...  cheers,sundar

    Attachment(s)

    zip
    Altova Spy Tutorial.zip   1.57 MB 1 version


  • 14.  Re: Data Migration Opinions?

    Posted Jul 24, 2014 12:39 PM

    This is the image at the early part of the thread