[Linux/Unix] Shell Script to Archive Logs By Flushing OUT it’s Content

You probably wondering, how log files to be archived especially when your system is running 24/7 and continuously updating the log files.

In a traditional way, we could do this by moving out all the log files from its original location to Archive directory and cleaning up there. You might want to check this log archiving script

Please Note – if your log files are continuously updating by your application and if you try to archive the logs by moving in another folder then there are high chances to get an error in the application.

The solution for this problem could be run this script outside business hours by taking 5-10 minutes downtime. But we should always look for BEST solution and it is not guaranteed that you will always get the system downtime approval from the client. 

So, here is the trick – Without taking any actual downtime we could first copy all the files to another folder and just flush out the content of log files present in original location.

If you don’t want to go through all the details about this archiving process, then you may directly jump into script.

What this script does?

1. COPY: Copy all your log files from main LOG directory to Archive directory.

2. MAKE DIR: Before you copy your logs to Archive directory, create new folder under Archive directory and then move everything inside that directory.

3. DELETE CONTENT: After copying all the files into Archive, flush out the content of log files present in original directory.

4. CLEAN UP ARCHIVE: Remove all logs files present under archive folder which are older than N days (for example, 7 days)

This is your main LOG directory where all your log files present. Also make sure you should have the archive directory. 

cd /local/apps/WebServer/Web ls

Log Archive

LOG=/local/apps/WebServer/Web/Log

Archive=/local/apps/logs/WebServer/Web/Archive

Variables used in the script:

LEVROOT: This is your root directory, in this example mu root directory is “/local/apps/WebServer/Web”

LOG: This is your actual log directory. In this example my LOG directory is

“/local/apps/WebServer/Web/Log”

ARCHIVE: This is your archive directory. In this example my ARCHIVE directory is

“/local/apps/WebServer/Web/Archive”

You could manage your archive folder by creating new folder (through script) each day and then move everything there in Archive-Subdirectory.

DATE: This is date variable which will hold todays date

TIME: This is time variable which will hold current time.

1. Define your variables

          LEVROOT=/local/apps/WebServer/Web

          DATE=$(date +%d%m%y)

          TIME=$(date +%H%M)

          ARCHIVE=“$LEVROOT/Archive/$DATE-$TIME”

          LOG=”$LEVROOT/Log”

2. Search the log files (along with the sub directories) under Archive which are older than 7 days and delete it. (of course, you can change that limit to 2 days or 15 days, etc. depending on your requirement.)

          find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} ;

You can verify this command by just listing down the file names. Run this command:

          find $LEVROOT/Archive/* -mtime +7

3. Create sub directory under Archive directory to manage your archived files efficiently. This is very useful if large numbers of log files getting generated under LOG directory.

          mkdir $ARCHIVE

4. COPY your log files into Archive directory

           find $LOG -type f -print -exec cp -p {} $ARCHIVE ;

 5. CLEAR the content of log files present in your main LOG directory.

           find $LEVROOT/*.* -type f -exec sh -c ‘>”{}”‘ ;

Linux/Unix Log Archive Script

Here is the complete files archiving script considering scenario discussed above.

If you have any doubts in the script, then you may look at the scenario again or just leave your comment below. 

#———————————————————————–
# Script for archiving of logs by flushing out its content
# Script          : logs_Archive.sh
# 02-01-2020 : Linux/Unix Admin
#———————————————————————-

 

#!/bin/sh

# Varibales Used:
LEVROOT=/local/apps/WebServer/Web
DATE=$(date +%d%m%y)
TIME=$(date +%H%M)
ARCHIVE=“$LEVROOT/Archive/$DATE-$TIME”
LOG=”$LEVROOT/Log”

#CLEAN up Archive directory

find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} ;

mkdir $ARCHIVE

#COPY files to archive directory

find $LOG/*.* -type f -print -exec cp -p {} $ARCHIVE ;

#remove content from log files

find $LOG/*.* -type f -exec sh -c ‘>”{}”‘ ;

echo $ARCHIVE

echo $LOG

 

 

Leave a Comment