sharadchhetri.com

Linux,Cloud Computing And DevOps

Secondary Logging : save all users history command output as log

by 16 Comments

The secondary logging will help you to save history command output as log. In this practical, we will set the secondary logging for all users including root. Secondary logging will help you in audit by this way you can find out on which date and time users hit the command.

It has been long time, I reworked on secondary logging method. And this time, I am sharing secondary logging method with two scenarios. Read our post on secondary logging , written a few years ago.

Practical Performed and tested On

Operating System : Linux ( RHEL , CentOS , Debian , Ubuntu , Linux Mint)
Arch : x86 and x86_64

Pros And Cons Of Using Secondary Logging

Pros:

1. We can save the history commands output of all users in system . For example in /var/log .
2. Even the user become super user by using command sudo su - or su - , secondary logging will save these users history command as logs.
3. When user become super user and as super user whatever it uses the commands, it will be logged too.
4. No package installation required.
5. Even if user try to remove secondary logging logs, it will be logged also.
6. You can use your own ideas and make it more innovative. Understanding of bash is required.
7. You can also save the secondary logging logs in remote location via rsyslog server , s3 bucket etc.

Cons:

The secondary logging do not save the history command logs in runtime. When user exit from terminal or get disconnected then only it save its log.

Setup Secondary Logging

1. Setting secondary logging for super user and root.
2. Setting secondary logging for other user except root/superuser.

Setup secondary logging for root and super user

The settings are quite simple and it will log the super user or root history command output as log. Here , we have advantage that when a user switch to super user and do the activity as root , the log will save its secondary log and it will also show which user become super user and log file name belongs to that user.

Let’s start setting secondary log for root and super user.

1. Create directory to save logs of secondary logging. 

mkdir -p /var/log/sudo_historylogs/

2. Edit /root/.bashrc

vi /root/.bashrc

And paste below given content in /root/.bashrc.

export HISTSIZE=10000
export HISTTIMEFORMAT='%F %T '
export HISTFILE=/var/log/sudo_historylogs/history-sudo-$(who am i | awk '{print $1}';exit)-$(date +%F)
export PROMPT_COMMAND='history -a'

3. Activate the settings .

source /root/.bashrc

That’s it. Now it will start logging . For test, login with user and become super user. run some command and exit. Check the new log file get created inside /var/log/sudo_historylogs . Just open the newly created log file and you will see commands hit by user is written over there.

Setup secondary logging for other users except root

The logic is same as described above only one change is there. We will create new log directory for other users who are not super user or root. As well as , we are not logging for root user.

Follow the given below steps.

1. Create directory for logging other users.

mkdir -p /var/log/users_historylogs/

And set the sticky bit on /var/log/users_historylogs/

chmod +t  /var/log/users_historylogs/

3. Create a new script inside /etc/profile.d/

vi /etc/profile.d/history_log.sh

And paste the below content in /etc/profile.d/history_log.sh file, save and exit.

_who_am_i=$(who am i|awk '{print $1}')
_ID=$(id -u $_who_am_i)
 
if [ "$_ID" > 0 ]
then
export HISTSIZE=10000
export HISTTIMEFORMAT='%F %T '
export HISTFILE=/var/log/users_historylogs/history-users-$(who am i | awk '{print $1}';exit)-$(date +%F)
export PROMPT_COMMAND='history -a'
fi

Set the permission.

chmod 770 /etc/profile.d/history_log.sh

3. Activate the script

source /etc/profile.d/history_log.sh

Now we have completed the settings for both secnarios i.e for super users and other users.

To test the secondary logging, login with any user other than root. Hit some commands and exit.
Login back with root user and check /var/log/users_historylogs/ directory . You will find some new log files are generated. Open the file and read it, you will find the history of commands used by your last login user.

Reader Interactions

Comments

  1. Ashok says

    Hi Sharad,

    Could you please let me know if this process would work for AIX servers since /etc/profile.d doesn’t exist on AIX OS? Secondary logging for root will work if we put the contents in /etc/profile but I’m not sure how secondary logging would work for non-root users on AIX servers.

    Thank you.

    Reply

  2. vivek says

    Hi Sharad your article is amazing could you please give me suggestion

    I want to show which user turns to root in history command or in logs as your article is doing, As we have multiple users which turn to root using sudo sudo -i and we are not able to track which users turns root and runs command when multiple users turn root at the same time

    Could you please help in suggesting a solution

    Reply

    • Sharad Chhetri says

      Hi Vivek,

      1. The Secondary logging settings should be applied for all users including root.
      2. In log directory, it will create logs for all users.

      When user switch to user root , it will run su command for it. This will be tracked in user’s log.
      Hope, it will help.

      Regards
      Sharad

      Reply

  3. Anand says

    Hey, thanks for the article. This is useful. But do you have any way to make sure once the user sudos as root, there is no way he can delete/modify the history log files?

    Reply

    • Sharad Chhetri says

      Hi Anand,

      Generally we can set permission on file but root can do all in system. For keeping best practice, you can setup remote log server(eg. rsyslog) to keep the backup of logs in sync.
      Apart of it, when user try to remove/modify the log that action will also be logged.

      Regards
      Sharad

      Reply

  4. Gaurav says

    Hi Sharad

    When tried secondary logging for users, we added the script in vi /etc/profile.d/history_log.sh and when tried to login with other user and execute the command this doesn’t show or save in /var/log/users_historylogs/ only ubuntu user details present rest all users details not getting saved.

    Reply

    • Sharad Chhetri says

      Hi Gaurav,

      The root and non-root user should be setup separately.
      If you can tell me your Operating System and its version, I will try to reproduce the issue and check again.

      Also note that if you are trying with other user, the user should have hit some command after this implementation then only it will log.

      Regards
      Sharad

      Reply

      • Gaurav Khandelwal says

        Hi Sharad

        The OS is ubuntu 16.04 LTS, When enabled secondary logging for users as mentioned and when tried logging in and execute commands, we see only su – xxxx is getting captured but not the commands executed from secondary user in /var/log/users_historylogs/ . Please check this and advise if this works for you .

        Reply

        • Sharad Chhetri says

          Hi Gaurav,

          I found the problem area. It is actually with PROMPT_COMMAND . When user is login and becoming super user onsudo su, the PROMPT_COMMAND is not working on Ubuntu. Probably it happened after change in bash versions. I am checking if any workaround on this in Ubuntu OS.

          Regards
          Sharad

          Reply

          • Sharad Chhetri says

            Hi Gaurav,

            Try this in your Ubuntu system. Instead of creating shell script in /etc/profile.d update /etc/bash.bashrc .
            In /etc/bash.bashrc add these new line .

            root@ubuntu:/home/sharad# cat /etc/bash.bashrc |tail -20


_who_am_i=$(who am i|awk '{print $1}')
_ID=$(id -u $_who_am_i)
 
if [ "$_ID" -gt 0 ]
then
export HISTSIZE=10000
export HISTTIMEFORMAT='%F %T '
export HISTFILE=/var/log/users_historylogs/history-users-$(who am i | awk '{print $1}';exit)-$(date +%F)
export PROMPT_COMMAND='history -a;date >> $HISTFILE'
fi

if [ "$_ID" == 0 ]
then
export HISTSIZE=10000
export HISTTIMEFORMAT='%F %T '
export HISTFILE=/var/log/sudo_historylogs/history-sudo-$(who am i | awk '{print $1}';exit)-$(date +%F)
export PROMPT_COMMAND='history -a'
fi

            Also change the permission to 777 of /var/log/sudo_historylogs/ and /var/log/users_historylogs .

            Check once and let me know.

            Remember that the non-root user when it changes to login by using sudo su command , at that time history will be saved in log.

            Regards
            Sharad

  6. deepak pandiyan says

    I have configured my server /etc/bashrc with these settings to capture sudo secondary logging
    export HISTSIZE=10000
    export HISTTIMEFORMAT=”%F %T ”
    export HISTFILE=/var/log/bashhist/root_history-$(who am i | awk ‘{print $1}’;exit)
    export PROMPT_COMMAND=’history -a’

    output
    In output am getting some random numbers instead date and timestamp
    #1471863084
    echo all is well
    #1471863110
    tail /var/log/bashhist/root_history-deepakp1
    #1471863124
    vim /etc/bashrc

    Reply

    • Sharad Chhetri says

      Hello Deepak,

      The date time is printed in epoch format. Given below is example to convert epoch format to human readable.

      sharad@linuxworld:~$ date -d "@1471863124"
Mon Aug 22 16:22:04 IST 2016
sharad@linuxworld:~$

      Regards
      Sharad

      Reply

      • deepak says

        Thanks Buddy !!!!!!!!!! Much appreciated. I have lot of lines to verified can we have any commands to verify or convert to whole into human readable

        Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: