Deleting old transport requests
Why we need to delete the old transport requests?
- Re-use the space in the file system
- Large amount of data to be transported (lack of disk space)
- Compliance
- Increase the performance of the file system.
It is possible to delete the data which is no longer required from the sub directories data, log and cofiles of trans using the tp function clearold.
Note: It is recommended to archive rather than to delete directly.
Requirements
- Check if olddata exists as subdirectory of trans
olddata under trans directory: Check If the subdirectory 'olddata' within the transport directory (/usr/sap/trans) does exist or not. If not create it and have one of the SAP System administrator as the Owner and remaining SAP system administrators sharing the same trans directory should have write permission (i.e., write authorization for the group).
- Parameter Settings: The minimum age of files in the transport directory that can be deleted can be set in the transport profile.
The transport profile is the file 'TPPARAM' in the subdirectory 'bin' in the transport directory.
In releases prior to 4.5A, this file was called TPPARAM
The new transport profile TP_DOMAIN_<SID>.PFL generated by TMS is used for all calls of the transport control program from the SAP System. The settings in the transport profile TPPARAM are ignored. <SID> is the SID of the first Domain controller when STMS was configured.
The following parameters can be set (all durations are in days):
datalifetime is the minimum lifetime of the files in data
loglifetime is the minimum lifetime of the files in log
olddatalifetime is the minimum lifetime of the files in olddata
cofilelifetime is the minimum lifetime of the files in cofiles
Example:
datalifetime=30
loglifetime=60
olddatalifetime=60
cofilelifetime=30
- If STMS was configured with EXTENDED TRANSPORT CONTROL and used Target groups, make sure that that below parameter are set globally.
CTC=1
NBUFFORM=1
Note: If above parameters are not set globally, the tp function "check all" will return below error.
ERROR: A target system group (/FROMDEV/) is used with a name longer than 3.
This is only possible with NBUFFORM=TRUE!
ERROR: EXIT(16) -> process ID is: 27830
tp returncode summary:
TOOLS: Highest return code of single steps was: 16
ERRORS: Highest tp internal error was: 0204
tp finished with return code: 204
meaning:
parameter is missing
Transports clean-up can be performed in two steps
- First to run 'tp check all' is to make a list of all files in the subdirectories 'log', 'cofiles', 'data' and 'oldata' that are no longer required. This list can be found in the file 'ALL_old.lis' in the 'tmp' subdirectory of the transport directory. The age of the files is not taken into account here.
- In the second step (tp clearold all), the age of the files that are listed in the file 'ALL_old.lis' is checked. Files from the directory 'data' which have exceeded their lifetime are moved to the subdirectory 'olddata'. Files from the other directories ('log', 'cofiles', 'olddata') which have exceeded their lifetime are directly deleted.
"tp" checks and takes action on below sub directories of trans.
/usr/sap/trans/sapnames
/usr/sap/trans/cofiles
/usr/sap/trans/data
/usr/sap/trans/olddata
/usr/sap/trans/log
Actions being performed with TP clearold command
- Determines the removable files from cofiles-directory ... and removes corresponding entries from files in sapnames-directory ... (/usr/sap/trans/sapnames/..)
- Deletes the cofiles that are determined in above step from /usr/sap/trans/cofiles/...
- Determine and delete the transport log files from /usr/sap/trans/log/....
- Determine and delete the data files present in /usr/sap/trans/olddata
- Moves the data files from /usr/sap/trans/data/ to /usr/sap/trans/olddata/
- Determine and delete the data files present in /usr/sap/trans/olddata, which met expiration date. This Step again checks if any of the data files moved from /usr/sap/trans/data/ to /usr/sap/trans/olddata
Steps to start old transports deletion
- Log on as one of the SAP System administrators, Change to the subdirectory 'bin' in the transport directory: Start tp with the arguments 'check' and 'all'
devadm % cd /usr/sap/trans/bin
devadm % tp check all pf=TP_DOMAIN_DEV.PFL
- Start tp with the arguments 'clearold' and 'all':
devadm % tp clearold all pf=TP_DOMAIN_DEV.PFL
- Creates CLEAROLD.LOG in /usr/sap/trans/tmp
- CLEAROLD.LOG contains the entire log of actions taken on the sub directories of /usr/sap/trans.
Optionally we can simulate the "tp clearold all" by using the tp function "tp testold all"
devadm %tp testold all pf=/usr/sap/trans/bin/TP_DOMAIN_DEV.PFL
- Creates TESTOLD.LOG in /usr/sap/trans/tmp
- TESTOLD.LOG contains the entire log of actions that will be taken by the "tp clearold all" on the sub directories of /usr/sap/trans.
- This just creates the log and does not perform any physical actions
Preparing the list of transports which are between 30 to 60 days (data files which are available to restore from olddata directory)
Open the downloaded file /usr/sap/trans/tmp/CLEAROLD.LOG and search for the string "Move /usr/sap/trans/data/". Select all the lines from the first occurrence of the string to last occurrence and then copy the contents to a spreadsheet.
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig1.gif
[https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig2.gif
]
Now Find and Replace the String "Move /usr/sap/trans/data/" as in below Figure.
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig3.gif
"Replace with" should be kept blank and Click "Replace All"
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_Fig4.gif
Repeat replace for the string " to ." (Including the spaces)
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig05.gif
Select the Column "A". Go to Menu "Data" and select "Text to Columns..."
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig06.gif
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig07.gif
- Choose Delimited Radio Button and click Next.
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig08.gif
- Select "Other" and Enter "." (Dot) and click Next
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig09.gif
- Now Hold the "Shift" Key and Select all the Columns and Select the Column Data format as "Text" and Click Finish.
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig10.gif
- Enter Formula in Cell "C1" as =CONCATENATE(B1,A1)
- Place the Cursor at Bottom - Right Corner of the Cell C1, Hold and Drag Fill Handle
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig11.gif
- Copy the Contents in Column "C" and Right Click on Cell D1 and select "Paste Special" from the Pop-up menu.
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig12.gif
- Select "Values" Radio Button and Click "Ok"
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig13.gif
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig14.gif
- Delete the Columns "A", "B" and "C". Find and Replace "R" with "K" will produce the actual Transport Request Number. This will not apply to the transports which are generated while applying transports.
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig15.gif
https://weblogs.sdn.sap.com/weblogs/images/251731336/TPC_fig16.gif
- Save the spreadsheet
TP Restore Process
If there was any request to restore any transport, which was moved to "olddata" directory, we need to copy the corresponding data file from the path /usr/sap/trans/olddata to /usr/sap/trans/data
Note: Based on the parameters "olddatalifetime=60" and "datalifetime=30" you can restore any transport whose age is between 30 to 60 days only. If you are archiving the datafiles before they are being deleted by "tp clearold all", then data file can be restored from the archive location.
We can identify the data file of a Transport as specified in below example
Eg: If we need to restore the TP#DEVK950981 then corresponding data file will be R950981.DEV
<SID> <K> <950981>
DEV K #
SID will be an extension to the data file
K will be replaced by R
# will remain unchanged
devadm 24% cd /usr/sap/trans/olddata
devadm 25% cp R950981.DEV /usr/sap/trans/data/
- Below step will create the cofile for the transport request. This needs to be done as "tp clearold" will only backup the data file of the transport request.
devadm 31% tp CREATECOFILE DEVK950981 pf=/usr/sap/trans/bin/TP_DOMAIN_DEV.PFL
This is tp version 305.13.24 (release 46D) for ANY database
This is R3trans version 6.05 (release 46D - 29.03.07 - 10:05:00).
R3trans finished (0000).
tp finished with return code: 0
meaning:
Everything OK
- Cofile will be created in /usr/sap/trans/cofiles as K950981.DEV
- If data file was not copied/moved to /usr/sap/trans/data output of the tp command will be as below. Also cofile cannot be created if we don't have data file.
devadm 34% tp CREATECOFILE DEVK950771 pf=/usr/sap/trans/bin/TP_DOMAIN_DEV.PFL
This is tp version 305.13.24 (release 46D) for ANY database
This is R3trans version 6.05 (release 46D - 29.03.07 - 10:05:00).
2EETW152 Cannot open file "/usr/sap/trans/data/R950771.DEV".
No such file or directory
R3trans finished (0012).
/usr/sap/trans/sapnames/ for transport: DEVK950771: Is a directory
HALT 20080805160810
ERROR: /usr/sap/trans/sapnames/ : cant open
: Is a directory
ERROR: EXIT(16) -> process ID is: 9365
tp returncode summary:
TOOLS: Highest return code of single steps was: 16
ERRORS: Highest tp internal error was: 0212
tp finished with return code: 212
meaning:
could not access file as supposed (see SLOG for more details)
- Once cofile has been created add the transport request to the buffer from STMS and import.
Script used to backup the data files to archive location.
If decided to move/copy the data files to remote filesystem, it is better to perform the backup operation from /usr/sap/trans/data before using the "tp clearold all".
By this, will have all the transport data files that can be used to restore the transport back if needed and also if "tp clearold all" has not been used for some months, as said above files which have met expiration will be moved to olddata and gets deleted from olddata as well based on the parameter set, in this case we will loose the datafiles which are moved to and deleted from olddata.
Below shell script can be used to perform backup from /usr/sap/trans/data to remote filesystem. Script checks for each file on the source location and if it does not exist on destination location that file will be copied to remote filesystem. This script can be used in any area wherever needed to perform backup of files. Eg: oracle offline redo log files.
Usage: incbackup.shl <Source> <Destination> [stage:email address]
Script expects at least 2 arguments to be passed and an optional email address. If email address passed as argument, log written by the above script will be e-mailed to specified e-mail.
Note: Output of the script will be logged to a file in the current directory where it was triggered.
Script name : incbackup.shl
# Shell script to do incremental backup from SOURDIR to DESTDIR
# Usage incbackup.shl <Absolute path of Source Dir> <Absolute path of Dest Dir> [email Address]
########################################################################################################
# Variables: #
# #
# SOURDIR = location of the Source files #
# DESTDIR = location of the file destination #
# LOGSFILE = Name of the log file #
# SMTP = Mail id to which log need to be sent #
DSTAMP=`date +%y%m%d.%H%M` #
LOGSFILE=incbackup #
# #
########################################################################################################
if [ "$#" -lt 2 ]; then
echo "Specify absolute path of Source and Target directories" 2>and 1 >> $LOGSFILE.$DSTAMP.log
echo "" 2>and 1 >> $LOGSFILE.$DSTAMP.log
echo "Usage $0 <Source Dir> <Dest Dir> [stage:email Address]" 2>and 1 >> $LOGSFILE.$DSTAMP.log
exit 1
else
if [ "$#" -eq 2 ]; then
SOURDIR=$1
DESTDIR=$2
if [ ! -d $SOURDIR ]; then
echo "$SOURDIR is not a directory!" 2>and1 >> $LOGSFILE.$DSTAMP.log
exit 1
else
if [ ! -d $DESTDIR ]; then
echo "$DESTDIR is not a directory!" 2>and1 >> $LOGSFILE.$DSTAMP.log
exit 1
fi
fi
else
if [ "$#" -gt 2 ]; then
SMTP=$3
SOURDIR=$1
DESTDIR=$2
if [ ! -d $SOURDIR ]; then
echo "$SOURDIR is not a directory!" 2>and1 >> $LOGSFILE.$DSTAMP.log
exit 1
else
if [ ! -d $DESTDIR ]; then
echo "$DESTDIR is not a directory!" 2>and1 >> $LOGSFILE.$DSTAMP.log
exit 1
fi
fi
fi
fi
fi
echo "Start of Incremental backup script - $0 at $TSTAMP " > $LOGSFILE.$DSTAMP.log
echo "" >> $LOGSFILE.$DSTAMP.log
echo " Variables passed to this script: " >> $LOGSFILE.$DSTAMP.log
echo " Source Directory = $SOURDIR" >> $LOGSFILE.$DSTAMP.log
echo " Destination Directory = $DESTDIR" >> $LOGSFILE.$DSTAMP.log
echo " Log File = $LOGSFILE" >> $LOGSFILE.$DSTAMP.log
echo "" >> $LOGSFILE.$DSTAMP.log
# Process each file in $SOURDIR, comparing it to $DESTDIR
missing=0
n=1
for filename in $SOURDIR/*; do
fn=`(basename "$filename")`
if [ -f "$filename" ]; then
if [ ! -f "$DESTDIR/$fn" ]; then
missing=`expr $missing + $n`
cp $SOURDIR/$fn $DESTDIR/$fn 2>and1 >> $LOGSFILE.$DSTAMP.log
if [ -f "$DESTDIR/$fn" ]; then
echo "$SOURDIR/$fn is missing from $DESTDIR and copied \n" 2>and 1 >> $LOGSFILE.$DSTAMP.log
else
echo "$SOURDIR/$fn could not be copied to $DESTDIR" 2>and 1 >> $LOGSFIL.$DSTAMP.log
if [ ! "$SMTP" = "" ]; then
/bin/mailx -s "Incremental backup script --- Failure" $SMTP < $LOGSFILE.$DSTAMP.log
fi
exit
fi
fi
fi
done
echo "Total $missing files were missing and copied to $DESTDIR" >> $LOGSFILE.$DSTAMP.log
if [ ! "$SMTP" = "" ]; then
/bin/mailx -s "Incremental backup script --- Success" $SMTP < $LOGSFILE.$DSTAMP.log
fi
exit