Skip to end of metadata
Go to start of metadata

Purpose

Upgrading your Data Services repository from previous versions such as DS 3.x can be confusing. The goal of this page is to help clear things up and at the same time allow you to take advantage of new functionality available in Data Services 4.2.

Overview

This page will assist users by providing the necessary steps for upgrading both Local and Central repositories to DS 4.2.

NOTE: These instructions will suffice for moving from DI 11.x to DS 4.2. Where 3.x is referenced, substitute 11.x. Please keep in mind MANY changes have been made in the software from DI 11.x to DS 3.x to DS 4.2. There is a possibility some of your jobs may not work as expected. 

Getting Ready

  • Optimization of your jobs prior to migration will help you take advantage of new DS 4.2 functionality as well as prevent upgrade issues.
  • Make sure none of your work flows, data flows, or job names contain spaces. e.g. Change "Your Dataflow1" to "Your_Dataflow1"
  • If using Query transforms inside of your jobs, validate that none contain empty output schemas.
  • Do you have jobs that won't validate? You must either fix them, or delete them from BOTH your local and central repositories. Broken jobs very often cause repository upgrades to fail.
  • It is also suggested you use the Compact Repository feature on your local repositories (found only in versions prior to DS 4.0) before you migrate.
    (NOTE: if you run this tool on a central repository, it WILL affect object history.) 
  • Thorough testing of all your jobs in your current environment is highly recommended before you proceed. The upgrade process does not fix broken jobs.
  • If the hostname of your DS Jobserver will change as part of your upgrade, use the DS Server Manager to disassociate your DS repositories from your Jobserver PRIOR to repository upgrade.

Regarding Outer Joins

For versions prior to DS 4.0, it is HIGHLY recommended to add join rankings to the tables BEFORE migrating to DS 4.2. This will allow your design to get the same table order in the generated SQL statement. Please review the Performance Guide on starting on page 109 for further instruction on how to accomplish this:

http://help.sap.com/businessobject/product_guides/sboDS42/en/ds_42_perf_opt_en.pdf

Can't I just export my job/repo from my old system to an .atl file, then import it into DS 4.2? 

  • Although this might work for jobs/repos containing only Data Integrator (DI) transforms, this proceedure will break jobs/repos containing Data Quality (DQ) transforms.
  • Doing an export from the old and import into the new is NOT an upgrade, but only a "copy and paste".
  • If a transform in your job/repo has had functionality changes from the old version to the new, these new options will not be available to your old jobs/repos if you choose this "copy/paste" route.

The 3 Scenarios Below:

  • Help to make certain existing jobs in your older version of DS will be able to take advantage of new functionality within DS 4.2. This is especially important for DQ transforms.
  • Allow you to leave your older version of DS intact

Scenario 1 - Migrating a Central Repository (does not preserve history)

  1. Make certain all objects in the Central repository are checked in. Do NOT check in broken jobs (which fail validation) as they can cause the repository upgrade to fail.
  2. Use the Repository Manager from your older version of DS to create a new local repo which will be used for the upgrade.
  3. Access this new local repo with your older veraion of Designer and checkout all your objects from the Central Repository into the new local repo.
  4. Using the older version of Designer, make sure all your jobs are working correctly in the new local repo. The upgrade process does not fix broken jobs.
  5. From the Local Object Library > right-click > go to Repository > Export to file > and save a back-up of your repo to an .atl file. (This is just an additional backup. It will not be used in the upgrade process.)
  6. Close your older version of Designer.
  7. Using the DS 4.2 Repository Manager (Right-click DS Repository Manager and "Run as Administrator"), point to your new local repo (created in step 1) and click "Upgrade".
  8. Once complete, add the upgrade local repo to the 4.2 Central Management Console (CMC). If you were at an earlier version of DS 4.x, this step is not necessary.
  9. Access the upgraded local repo via the DS 4.2 Designer.
  10. IMPORTANT - If any of your jobs include the US Address Cleanse transform, please reference KB article 1618761 and follow the required steps. During this process, make sure you are pointing to the base "urac.atl", not any of the URAC files which include numbers in their names.
  11. Test all of your newly upgraded jobs.
  12. Once you are satisfied your jobs are functioning correctly, use the 4.2 Repository Manager to create a 4.2 Central Repository.
  13. Via the DS 4.2 Designer, log into you upgraded local repo and check all your objects into your new Central Repository so they can be checked out and used by others. 

Scenario 2 - Migrating a Central Repository (will preserve history)

  1. Make certain all objects in the Central repository are checked in. Do NOT check in broken jobs (which fail validation) as they can cause the repository upgrade to fail.
  2. Have your DBA make a copy of your Central Repository database, from your older version of Data Services, to a new database space.
  3. Using the DS 4.2 Repository Manager (Right-click DS Repository Manager and "Run as Administrator"), point to this copy and click "Upgrade".
  4. Once complete, add the upgraded Central Repository to the Central Management Console (CMC).
  5. If you are using a secure Central Repository, log into the DS Management Console to setup users and groups you'd like to be able to access it via the DS 4.2 Designer.

Scenario 3 - Migrating a Local Repository

When upgrading a Local repository, DO NOT use a database copy method. Instead:

  1. Log into your older version of Designer and export your local repo to an .atl file.
  2. Use the Repository Manager from your older version of DS to create a new local repo which will be used for the upgrade.
  3. Using your older version of Designer, log into your new local repo and import the .atl file you created.
  4. Make certain all your jobs perform correctly while logged into this repo. The upgrade process does not fix broken jobs.
  5. Close your older version of Designer.
  6. Using the DS 4.2 Repository Manager (Right-click DS Repository Manager and "Run as Administrator"), point to your new local repo (created in step 1) and click "Upgrade".
  7. Once complete, add the upgrade local repo to the 4.2 Central Management Console (CMC). If you were at an earlier version of DS 4.x, this step is not necessary.
  8. Access the upgraded local repo via the DS 4.2 Designer.
  9. IMPORTANT - If any of your jobs include the US Address Cleanse transform, please reference KB article 1618761 and follow the required steps. During this process, make sure you are pointing to the base "urac.atl", not any of the URAC files which include numbers in their names.
  10. Test all of your newly upgraded jobs.

Did you move your jobserver as part of your upgrade?

If you are using a different physical machine for your jobserver, you will likely need to use your DB editing tool and remove the old, extraneous jobserver entries from AL_MACHINE_INFO, found in your upgraded repository databases.

Have you upgraded any of your datastore databases?

Although upgrading your datastore databases isn't part of a repository upgrade, sometimes if you've upgraded the database your repositories are using during the move to a newer version of Data Services, you may have also upgraded the database your datastores are using. If this is indeed the case, you must make sure your datastore configuration information is changed to reflect this. Please reference KB article 1819246.

  • No labels

11 Comments

  1. Former Member

    Hi Julie, 

    this is very helpful.  

    in my upgrade scenario they are moving to a complete new platform.  we plan on taking our time since we have other projects going on. 

    So i had the DBA build new schemas- Repository and aud for IPS plus a new Local and Central for DS4.2.   i already installed IPS and DS but have not done any migration or upgrading yet. The 3.2 instance is still running and i would prefer not to connect to the old Local and Central DBs. 

    We were thinking that most of the jobs we run are fairly simple and do not need to upgrade yet but just to get everything on the same platform first.  How do you recommend going about this?   Maybe migrate first then upgrade at a later time?  

    They would like to shut down the old server since that hosted old BOBJ and DS.   we already upgraded BOBJ on a new platform and we chose not to use that CMS so went the IPS route. 

    Whats your thoughts? 

    THanks for any input!

     

    Daniel 

     

     

  2. Hi Daniel,

    I would not recommend what you are proposing, regardless of the simplicity of your jobs.

    Why? There have just been so many changes from DS 3.2 to DS 4.0 to DS 4.1 to DS 4.2, that a proper upgrade is the best way to go, especially if you have any Data Quality transforms inside your jobs.

    You mention not wanting to connect to your old Local and Central repository databases; however the purpose of the instructions above is to not only migrate properly, but also leave your DS 3.2 and those repository databases intact. This is particularly useful if you plan on doing any comparisions between DS 3.2 and DS 4.2.

    As you've said you intend on taking your time, you might as well follow the Best Practices to assure the smoothest migration possible. From the SAP Product Support perspective, when clients upgrade properly we're less likely to see a support incident generated in panic when something did not go as expected (smile) 

    I hope this helps. If you have more questions, please log a support incident with SAP Product Support via the SAP Support Portal.

    Enjoy your day!
    Julie

  3. Former Member

    Hi,

    Thank your for your useful post !

    I have a question concerning the migration of a central repository that preserves history and especially your first point of the "Scenario 2 - Migrating a Central Repository (will preserve history)". Currently we have a BODS 3.2 installation on Windows with MySQL 5.0 for local and central repositories.

    In the last upgrade guide, on page 16, I see the following option to upgrade a repository of Data Services 3.x :

    Option 1

    1. Before you uninstall Data Services XI 3.x or Data Integrator 11.7, back up the current MySQL data folder.
    2. Obtain a new copy of MySQL 5.0 (not 5.1) and install it.
    3. Copy the backup MySQL data folder to the new MySQL 5.0 location.
    4. Uninstall Data Services XI 3.x or Data Integrator 11.7. This also uninstalls the previous version of the bundled MySQL.
    5. Install Data Services 4.2.
    6. Upgrade the new MySQL 5.0 repository to version 14.2.0.0.

    My question is :

    -> The PAM indicates that MySQL 5.0 is deprecated, so it means that after the 6 steps above, we need to move the content of the upgraded MySQL 5.0 database to another DBMS (MySQL 5.1 or 5.5) ?

    Thanks !

    Jib

  4. Hello Jib,

    It looks like our documentation hasn't been upgraded to reflect the PAM for DS 4.2 SPx.

    Yes, you should be using a supported DBMS. MySQL 5.5 would be fine.

    Cheers!
    Julie

  5. Former Member

    Hi Julie,

    Thank you for your answer. I have another question.

    In the Upgrade Guide, page 55, I read this :

    "The batch files generated for scheduled jobs and exported execution commands contain a reference to the log folder location. In this version of Data Services, the location for log files has moved from <LINK_DIR>/log to <DS_COMMON_DIR>/log."

    Therefore, besides adding join ranks in our dataflows before the migration, I understand that we have also to change every occurence of "LINK_DIR" in our code to "DS_COMMON_DIR". Indeed, in BODS 3.2, we often use "LINK_DIR" to call other jobs.

    Do you confirm my point ?

    Thanks

    Jib

    1. Hi Again Jib (smile)

      What you make reference to on page 55 is specific to the log location moving. You have asked if you need to change every occurence of "LINK_DIR" in your code. If every time you use LINK_DIR you are referencing log location, then yes.

      But if when you use LINK_DIR you are referencing things other than log location, you need to read through both pages 54 & 55 to determine what you need to change based upon what your LINK_DIR references are using and what is shown as having changed.

      In Windows:

      <LINK_DIR> = C:\SAP\SAP BusinessObjects\Data Services
      <DS_COMMON_DIR> = C:\ProgramData\SAP BusinessObjects\Data Services

      In UNIX/Linux

      <LINK_DIR> = /INSTALL DIRECTORY/Data Services

      I hope that helps. If you have further questions please raise them via the Data Services and Data Quality Discussions page, or create a SAP Support Incident.

      Cheers!
      Julie

  6. Former Member

    Hopefully someone is still monitoring these comments.

    We're upgrading from 4.1 to 4.2.  We do not use a Central Repository, so I'm looking at your Scenario 3 instructions.  I do not see how one can perform step 3 without first assigning rights to the new repository, which is in the older version at that point.  However, if you try to set up the new, older version, repository in the IPCMC, it won't do it because the version is right.  It's a catch-22 scenario.  Or, I'm missing something.  Can you perhaps clarify this a bit for me, please?  Thanks.

    Henry

  7. Hello Henry,

    I can see there would be validity in providing more clarification on the steps given above, which might differ slightly if you are upgrading from DS 4.x vs DS 3.2. The information above was originally written for users going from DS 3.2 to DS 4.x. 

    Are you upgrading from DS 4.1 to DS 4.2 on the same set of servers, or are you moving to a new set of servers?

    If you're upgrading on the same set of servers, since DS 4.2 requires a different version of BI Platform/IPS, there would be no point in steps 1-5. Once the BI Platform/IPS is upgraded (which is done prior to the DS upgrade) DS 4.1 wouldn't work with it anyway.

    You'd have to test all the jobs in your local repositories on your current DS 4.1, fix an errors and/or delete any problem jobs, backup your local repositories (either db backups or atl backups), then proceed with step 6.

    If you're hoping to leave you DS 4.1 system intact to do comparison testing between DS 4.1 and DS 4.2, you'll need a separate set of servers for DS 4.2, including a separate BI Platform/IPS.

    I hope that helps/answers your question?

    Thanks,
    Julie

  8. Former Member

    We're upgrading on a separate server.  We originally copied the current server and then performed an in-place upgrade.  However, we now need to refresh our repositories, because for various reasons, this process has taken about a year and we're seriously out of synch now.  

    It's still unclear to me how I can get my 4.1 repository into my 4.2 server.

    Yesterday, I dropped my repo on the 4.2 server and, using the 4.1 Repository Manager, created a new one.  Thus, it's the 4.1 version.  I then attempted to use my 4.1 Designer to connect to the repo on the 4.2 server.  It wouldn't do it.  It didn't give me a version error, though.  It said something about not being able to access it.  I tried to take a look at the repository properties in the IPCMC on the 4.2 server, but when I tried to connect to it, then I got a version error.

    Everything's kind of in limbo now.

  9. Hi,

    Don't "drop" the 4.1 repo on 4.2. Do the following:

    • Log into your 4.1 Designer and export your local repo to an .atl file.
    • Log out of 4.1 Designer.
    • Log into the 4.1 Designer again, into the 4.1 repository you said you created, and import the atl file.
    • Test to be sure all the jobs are working.
    • Log out of 4.1 Designer.
    • Using the DS 4.2 Repository Manager (If you are not the Administrator on the box, right-click DS Repository Manager and "Run as Administrator"), point to the db space for the 4.1 repository you created and have imported the atl in to.
    • Click "Check Version" and it should show it is 14.1.x.xx
    • Then click "Upgrade".
    • Once the upgrade is done, you can use the "Check Version" again and it should show it is now 14.2.x.xx
    • Go to the DS 4.2 Server Manager and add your upgraded repository to a jobserver.
    • Log into the CMC for your DS 4.2 and add the upgraded repository under Data Services > Repositories
    • Then go to the CMC > Users and Groups, and give the appropriate users the rights to the repository.
    • At that point the users should be able to access that upgraded repository via 4.2 Designer.
    • A 4.1 Designer will never be able to access it.

    That should be it (smile)

    If you have further trouble please log a SAP Support Incident.

    Cheers!
    Julie

  10. Former Member

    I did open an Incident with SAP.  I'm going to put what worked for me here, in case someone in the future might need it.

    The missing step was to add the 4.1 repo on the 4.2 server to the 4.1 CMC.  That allowed me to log in to the 4.1 repo on the 4.2 server via my 4.1 Designer.  Without that step, I couldn't log in.  Once I could log in, I was able to finish up by importing the .atl file, then going to the 4.2 server and using the Repository Manager there to Upgrade the repo.  Then I had to go back and change the repo definition in the 4.1 CMC, to point it back to the 4.1 server database.

    Anyway, that all seemed to work.  Thanks for your responses to my questions.