Skip to end of metadata
Go to start of metadata

Deleting old transport requests

Why we need to delete the old transport requests?

  • Re-use the space in the file system
  • Large amount of data to be transported (lack of disk space)
  • Compliance
  • Increase the performance of the file system.

It is possible to delete the data which is no longer required from the sub directories data, log and cofiles of trans using the tp function clearold.

Note: It is recommended to archive rather than to delete directly.

Requirements

  • Check if olddata exists as subdirectory of trans

olddata under trans directory: Check If the subdirectory 'olddata' within the transport directory (/usr/sap/trans) does exist or not. If not create it and  have one of the SAP System administrator as the Owner and remaining SAP system administrators sharing the same trans directory should have write permission (i.e., write authorization for the group).

  • Parameter Settings: The minimum age of files in the transport directory that can be deleted can be set in the transport profile.

The transport profile is the file 'TPPARAM' in the subdirectory 'bin' in the transport directory.

In releases prior to 4.5A, this file was called TPPARAM

The new transport profile TP_DOMAIN_<SID>.PFL generated by TMS is used for all calls of the transport control program from the SAP System. The settings in the transport profile TPPARAM are ignored.  <SID> is the SID of the first Domain controller when STMS was configured.


 

The following parameters can be set (all durations are in days):

datalifetime is the minimum lifetime of the files in data

loglifetime is the minimum lifetime of the files in log

olddatalifetime is the minimum lifetime of the files in olddata

cofilelifetime is the minimum lifetime of the files in cofiles

Example:

datalifetime=30

loglifetime=60

olddatalifetime=60

cofilelifetime=30

  • If STMS was configured with EXTENDED TRANSPORT CONTROL and used Target groups, make sure that that below parameter are set globally.

CTC=1

NBUFFORM=1

Note: If above parameters are not set globally, the tp function "check all" will return below error.

ERROR: A target system group (/FROMDEV/) is used with a name longer than 3.

This is only possible with NBUFFORM=TRUE!

ERROR: EXIT(16) -> process ID is: 27830

tp returncode summary:

TOOLS: Highest return code of single steps was: 16

ERRORS: Highest tp internal error was: 0204

tp finished with return code: 204

meaning:

  parameter is missing

Transports clean-up can be performed in two steps

  • First to run 'tp check all' is to make a list of all files in the subdirectories 'log', 'cofiles', 'data' and 'oldata' that are no longer required. This list can be found in the file 'ALL_old.lis' in the 'tmp' subdirectory of the transport directory. The age of the files is not taken into account here.
  • In the second step (tp clearold all), the age of the files that are listed in the file 'ALL_old.lis' is checked. Files from the directory 'data' which have exceeded their lifetime are moved to the subdirectory 'olddata'. Files from the other directories ('log', 'cofiles', 'olddata') which have exceeded their lifetime are directly deleted.

"tp"  checks and takes action on below sub directories of trans.

/usr/sap/trans/sapnames

/usr/sap/trans/cofiles

/usr/sap/trans/data

/usr/sap/trans/olddata

/usr/sap/trans/log

Actions being performed with TP clearold command

  • Determines the removable files from cofiles-directory ... and removes corresponding entries from files in sapnames-directory ... (/usr/sap/trans/sapnames/..)
  • Deletes the cofiles that are determined in above step from /usr/sap/trans/cofiles/...
  • Determine and delete the transport log files from /usr/sap/trans/log/....
  • Determine and delete the data files present in /usr/sap/trans/olddata
  • Moves the data files from /usr/sap/trans/data/ to /usr/sap/trans/olddata/
  • Determine and delete the data files present in /usr/sap/trans/olddata, which met expiration date. This Step again checks if any of the data files moved from /usr/sap/trans/data/ to /usr/sap/trans/olddata

Steps to start old transports deletion

  • Log on as one of the SAP System administrators, Change to the subdirectory 'bin' in the transport directory: Start tp with the arguments 'check' and 'all'

devadm % cd /usr/sap/trans/bin

devadm % tp check all pf=TP_DOMAIN_DEV.PFL

  • Start tp with the arguments 'clearold' and 'all':

devadm % tp clearold all pf=TP_DOMAIN_DEV.PFL

  • Creates CLEAROLD.LOG in /usr/sap/trans/tmp
  • CLEAROLD.LOG contains the entire log of actions taken on the sub directories of /usr/sap/trans.

Optionally we can simulate the "tp clearold all" by using the tp function "tp testold all"

devadm %tp testold all pf=/usr/sap/trans/bin/TP_DOMAIN_DEV.PFL

  • Creates TESTOLD.LOG in /usr/sap/trans/tmp
  • TESTOLD.LOG contains the entire log of actions that will be taken by the "tp clearold all" on the sub directories of /usr/sap/trans.
  • This just creates the log and does not perform any physical actions

Preparing the list of transports which are between 30 to 60 days (data files which are available to restore from olddata directory)


Open the downloaded file /usr/sap/trans/tmp/CLEAROLD.LOG and search for the string "Move /usr/sap/trans/data/".  Select all the lines from the first occurrence of the string to last occurrence and then copy the contents to a spreadsheet.

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig1.gif

 [https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig2.gif
]

Now Find and Replace the String "Move /usr/sap/trans/data/" as in below Figure.

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig3.gif

"Replace with" should be kept blank and Click "Replace All"

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_Fig4.gif                 

Repeat replace for the string " to ." (Including the spaces)

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig05.gif                   

Select the Column "A". Go to Menu "Data" and select "Text to Columns..."

 https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig06.gif

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig07.gif                    

  • Choose Delimited Radio Button and click Next.

 https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig08.gif                            

  • Select "Other" and Enter "." (Dot) and click Next

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig09.gif                            

  • Now Hold the "Shift" Key and Select all the Columns and Select the Column Data format as "Text" and Click Finish.

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig10.gif

  • Enter Formula in Cell "C1" as =CONCATENATE(B1,A1)
  • Place the Cursor at Bottom - Right Corner of the Cell C1, Hold and Drag Fill Handle

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig11.gif

  • Copy the Contents in Column "C" and Right Click on Cell D1 and select "Paste Special" from the Pop-up menu.

  https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig12.gif                              

  • Select "Values" Radio Button and Click "Ok"

           

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig13.gif

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig14.gif

  • Delete the Columns "A", "B" and "C". Find and Replace "R" with "K" will produce the actual Transport Request Number. This will not apply to the transports which are generated while applying transports.

 https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig15.gif  

https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig16.gif
          

  • Save the spreadsheet



TP Restore Process


If there was any request to restore any transport, which was moved to "olddata" directory, we need to copy the corresponding data file from the path /usr/sap/trans/olddata to /usr/sap/trans/data

Note: Based on the parameters "olddatalifetime=60"  and "datalifetime=30" you can restore any transport whose age is between 30 to 60 days only. If you are archiving the datafiles before they are being deleted by "tp clearold all", then data file can be restored from the archive location.

We can identify the data file of a Transport as specified in below example

Eg:  If we need to restore the TP#DEVK950981 then corresponding data file will be R950981.DEV

<SID>    <K> <950981>

   DEV        K          #

SID         will be an extension to the data file

 K            will be replaced by R

 #             will remain unchanged

devadm 24% cd /usr/sap/trans/olddata

devadm 25% cp R950981.DEV /usr/sap/trans/data/

  • Below step will create the cofile for the transport request. This needs to be done as "tp clearold" will only backup the data file of the transport request.

devadm 31% tp CREATECOFILE DEVK950981 pf=/usr/sap/trans/bin/TP_DOMAIN_DEV.PFL

This is tp version 305.13.24 (release 46D) for ANY database

This is R3trans version 6.05 (release 46D - 29.03.07 - 10:05:00).

R3trans finished (0000).

tp finished with return code: 0

meaning:

  Everything OK

  •  Cofile will be created in /usr/sap/trans/cofiles as K950981.DEV
  •  If data file was not copied/moved to /usr/sap/trans/data output of the tp command will be as below. Also cofile cannot be created if we don't have data file.

devadm 34% tp CREATECOFILE DEVK950771 pf=/usr/sap/trans/bin/TP_DOMAIN_DEV.PFL

This is tp version 305.13.24 (release 46D) for ANY database

This is R3trans version 6.05 (release 46D - 29.03.07 - 10:05:00).

2EETW152 Cannot open file "/usr/sap/trans/data/R950771.DEV".

No such file or directory

R3trans finished (0012).

/usr/sap/trans/sapnames/ for transport: DEVK950771: Is a directory

HALT 20080805160810

ERROR: /usr/sap/trans/sapnames/ : cant open

: Is a directory

ERROR: EXIT(16) -> process ID is: 9365

tp returncode summary:

TOOLS: Highest return code of single steps was: 16

ERRORS: Highest tp internal error was: 0212

tp finished with return code: 212

meaning:

  could not access file as supposed (see SLOG for more details)

  • Once cofile has been created add the transport request to the buffer from STMS and import.

Script used to backup the data files to archive location. 

If decided to move/copy the data files to remote filesystem, it is better to perform the backup operation from /usr/sap/trans/data before using the "tp clearold all".  

By this, will have all the transport data files that can be used to restore the transport back if needed and also if "tp clearold all" has not been used for some months, as said above files which have met expiration will be moved to olddata and gets deleted from olddata as well based on the parameter set, in this case we will loose the datafiles which are moved to and deleted from olddata. 

Below shell script can be used to perform backup from /usr/sap/trans/data to remote filesystem. Script checks for each file on the source location and if it does not exist on destination location that file will be copied to remote filesystem. This script can be used in any area wherever needed to perform backup of files. Eg: oracle offline redo log files. 

Usage:  incbackup.shl <Source> <Destination> [stage:email address] 

Script expects at least 2 arguments to be passed and an optional email address. If email address passed as argument, log written by the above script will be e-mailed to specified e-mail. 

Note: Output of the script will be logged to a file in the current directory where it was triggered. 

Script name : incbackup.shl

# Shell script to do incremental backup from SOURDIR to DESTDIR

# Usage incbackup.shl <Absolute path of Source Dir> <Absolute path of Dest Dir> [email Address]

########################################################################################################
#  Variables:                                                                                                                                                                                          #
#                                                                                                                                                                                                            #
#  SOURDIR  = location of the Source files                                                                                                                                           #
#  DESTDIR  = location of the file destination                                                                                                                                        #
#  LOGSFILE = Name of the log file                                                                                                                                                       #
#  SMTP     = Mail id to which log need to be sent                                                                                                                                 #
   DSTAMP=`date +%y%m%d.%H%M`                                                                                                                                                 #
   LOGSFILE=incbackup                                                                                                                                                                        #
#                                                                                                                                                                                                            #
########################################################################################################

if [ "$#" -lt 2 ]; then

   echo "Specify absolute path of Source and Target directories" 2>and 1 >> $LOGSFILE.$DSTAMP.log

   echo "" 2>and 1 >> $LOGSFILE.$DSTAMP.log

   echo "Usage $0 <Source Dir> <Dest Dir> [stage:email Address]" 2>and 1 >> $LOGSFILE.$DSTAMP.log

   exit 1

else
  
    if [ "$#" -eq 2 ]; then
      
       SOURDIR=$1
       DESTDIR=$2

       if [ ! -d $SOURDIR ]; then

          echo "$SOURDIR is not a directory!" 2>and1 >> $LOGSFILE.$DSTAMP.log
          exit 1
       else

          if [ ! -d $DESTDIR ]; then

             echo "$DESTDIR is not a directory!" 2>and1    >> $LOGSFILE.$DSTAMP.log
             exit 1

          fi
       fi
    else

       if [ "$#" -gt 2 ]; then

          SMTP=$3
          SOURDIR=$1
          DESTDIR=$2

           if [ ! -d $SOURDIR ]; then

              echo "$SOURDIR is not a directory!" 2>and1 >> $LOGSFILE.$DSTAMP.log
              exit 1
           else

              if [ ! -d $DESTDIR ]; then

                 echo "$DESTDIR is not a directory!" 2>and1    >> $LOGSFILE.$DSTAMP.log
                 exit 1

              fi
           fi

       fi
 
    fi
fi

echo "Start of Incremental backup script - $0 at $TSTAMP " > $LOGSFILE.$DSTAMP.log

echo "" >> $LOGSFILE.$DSTAMP.log
echo " Variables passed to this script: " >> $LOGSFILE.$DSTAMP.log
echo " Source Directory      = $SOURDIR" >> $LOGSFILE.$DSTAMP.log
echo " Destination Directory = $DESTDIR" >> $LOGSFILE.$DSTAMP.log
echo " Log File              = $LOGSFILE" >> $LOGSFILE.$DSTAMP.log
echo "" >> $LOGSFILE.$DSTAMP.log

# Process each file in $SOURDIR, comparing it to $DESTDIR
missing=0
n=1
for filename in $SOURDIR/*; do
    fn=`(basename "$filename")`
    if [ -f "$filename" ]; then
       if [ ! -f "$DESTDIR/$fn" ]; then
             missing=`expr $missing + $n`
            cp $SOURDIR/$fn $DESTDIR/$fn 2>and1 >> $LOGSFILE.$DSTAMP.log
         if [ -f "$DESTDIR/$fn" ]; then
             echo "$SOURDIR/$fn is missing from $DESTDIR and copied \n" 2>and 1 >> $LOGSFILE.$DSTAMP.log
         else
             echo "$SOURDIR/$fn could not be copied to $DESTDIR" 2>and 1 >> $LOGSFIL.$DSTAMP.log

      if [ ! "$SMTP" = "" ]; then

             /bin/mailx -s "Incremental backup script --- Failure" $SMTP < $LOGSFILE.$DSTAMP.log

             fi
          exit
         fi         
       fi
    fi
done
echo "Total $missing files were missing and copied to $DESTDIR"  >> $LOGSFILE.$DSTAMP.log

if [ ! "$SMTP" = "" ]; then

/bin/mailx -s "Incremental backup script --- Success" $SMTP < $LOGSFILE.$DSTAMP.log

fi
exit

  • No labels