Linux Shell Script to Archive Log Files

Most common task to manage your file system for Linux or Unix administrator to set up the script which will take care of your log files (or any output files).

There are many ways to archive log files, but I feel it is very hard to find out best one.

No hurdles, here you find one of the BEST, more generic & robust solution and it can be applicable to all your scenarios.

If you don’t want to go through all the details about archive script, then you may directly jump into log archive script.

Let’s take a following common scenario as an example.

  1. Move all your log files from main LOG directory to Archive directory
  2. Before you move your logs to Archive directory, create new folder under archive directory and then move everything inside that directory.
  3. Remove all logs files which are older than N days (for example, 7 days)

And you are good to manage your files without any manual interventions
or weekly/monthly activity.

This is your main LOG directory where all your log files present. Also make sure you should have the archive directory.

cd /local/apps/BatchServer/MyDIStudioJob1 ls

Log      Archive

LOG=/local/apps/BatchServer/MyDIStudioJob1/Log

Archive=/local/apps/BatchServer/MyDIStudioJob1/Archive

Variables used in the script:

LEVROOT: This is your root directory, in this example my root directory is “/local/apps/BatchServer/MyDIStudioJob1

LOG: This is your actual log directory. In this example my LOG directory is, “/local/apps/BatchServer/MyDIStudioJob1/Log

ARCHIVE: This is your archive directory. In this example my ARCHIVE directory is “/local/apps/BatchServer/MyDIStudioJob1/Archive

You could manage your archive folder by creating new folder (through script) each day and then move everything there in Archive-Subdirectory.

DATE: This is date variable which will hold todays date

TIME: This is time variable which will hold current time.

1. DEFINE your variables

LEVROOT=/local/apps/BatchServer/MyDIStudioJob1

DATE=$(date +%d%m%y)

TIME=$(date +%H%M)

ARCHIVE=“$LEVROOT/Archive/$DATE-$TIME”

LOG=”$LEVROOT/”

2.  Search and Delete files: Search the log files (along with the sub directories)  under Archive which are older than 7 days and delete it. (of course, you can              change that limit to 2 days or 15 days, etc. depending on your requirement.)

There many linux commands to search the files or sub directories but I’ve                found “find” command very easy and convenient to search files and “rm”              will clean up files or subdirectories.

find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} ;

You can verify this command by just listing down the file names. Run this                command:

find $LEVROOT/Archive/* -mtime +7

3. Create Subdirectory: Create subdirectories under Archive directory to manage your archived files efficiently. This is very useful if large numbers of log files getting generated under LOG directory.

mkdir $ARCHIVE

4. Move Log files: Move your files into Archive directory. Again, “find” command is very effective here as well.

               find $LOG -type f -print -exec mv {} $ARCHIVE ;

Linux/Unix Log Archive Script

Here is the complete files archiving script considering scenario discussed above. If you have any doubts in the script, then you may look at the scenario again or just leave your comment below.

#—————————————————————-

# Script for archiving of logs.

# Script          : logs_Archive.sh

# 02-01-2020 : Linux/Unix Admin

#—————————————————————–

#!/bin/sh

# Varibales Used:

LEVROOT=/local/apps/Batchserver/MyDIStudioJob1

DATE=$(date +%d%m%y)

TIME=$(date +%H%M)

ARCHIVE=”$LEVROOT/Archive/$DATE-$TIME”

LOG=”$LEVROOT/”

find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} ;

mkdir $ARCHIVE

find $LOG -type f -print -exec mv {} $ARCHIVE ;

echo $ARCHIVE

echo $LOG

Most common task to manage your file system for Linux or Unix administrator to set up the script which will take care of your log files (or any output files).

There are many ways to archive log files, but I feel it is very hard to find out best one.

No hurdles, here you find one of the BEST, more generic & robust solution and it can be applicable to all your scenarios.

If you don’t want to go through all the details about archive script, then you may directly jump into log archive script.

1 thought on “Linux Shell Script to Archive Log Files”

Leave a Comment