The secondary logging will help you to save history command output as log. In this practical, we will set the secondary logging for all users including root. Secondary logging will help you in audit by this way you can find out on which date and time users hit the command.
It has been long time, I reworked on secondary logging method. And this time, I am sharing secondary logging method with two scenarios. Read our post on secondary logging , written a few years ago.
Practical Performed And Tested
In given below Operating System the setup was configured and tested.
Operating System : Linux ( RHEL , CentOS , Debian , Ubuntu , Linux Mint)
Arch : x86 and x86_64
Pros And Cons Of Using Secondary Logging
Pros:
1. We can save the history commands output of all users in system . For example in /var/log .
2. Even the user become super user by using command sudo su - or su - , secondary logging will save these users history command as logs.
3. When user become super user and as super user whatever it uses the commands, it will be logged too.
4. No package installation required.
5. Even if user try to remove secondary logging logs, it will be logged also.
6. You can use your own ideas and make it more innovative. Understanding of bash is required.
7. You can also save the secondary logging logs in remote location via rsyslog server , s3 bucket etc.
Cons:
The secondary logging do not save the history command logs in runtime. When user exit from terminal or get disconnected then only it save its log.
Setup Secondary Logging
1. Setting secondary logging for super user and root.
2. Setting secondary logging for other user except root/superuser.
Setup secondary logging for root and super user
The settings are quite simple and it will log the super user or root history command output as log. Here , we have advantage that when a user switch to super user and do the activity as root , the log will save its secondary log and it will also show which user become super user and log file name belongs to that user.
Let’s start setting secondary log for root and super user.
1. Create directory to save logs of secondary logging.
mkdir -p /var/log/sudo_historylogs/
2. Edit /root/.bashrc
vi /root/.bashrc
And paste below given content in /root/.bashrc.
export HISTSIZE=10000
export HISTTIMEFORMAT='%F %T '
export HISTFILE=/var/log/sudo_historylogs/history-sudo-$(who am i | awk '{print $1}';exit)-$(date +%F)
export PROMPT_COMMAND='history -a'
3. Activate the settings .
source /root/.bashrc
That’s it. Now it will start logging . For test, login with user and become super user. run some command and exit. Check the new log file get created inside /var/log/sudo_historylogs . Just open the newly created log file and you will see commands hit by user is written over there.
Setup secondary logging for other users except root
The logic is same as described above only one change is there. We will create new log directory for other users who are not super user or root. As well as , we are not logging for root user.
Follow the given below steps.
1. Create directory for logging other users.
mkdir -p /var/log/users_historylogs/
And set the sticky bit on /var/log/users_historylogs/
chmod +t /var/log/users_historylogs/
3. Create a new script inside /etc/profile.d/
vi /etc/profile.d/history_log.sh
And paste the below content in /etc/profile.d/history_log.sh file, save and exit.
_who_am_i=$(who am i|awk '{print $1}')
_ID=$(id -u $_who_am_i)
if [ "$_ID" > 0 ]
then
export HISTSIZE=10000
export HISTTIMEFORMAT='%F %T '
export HISTFILE=/var/log/users_historylogs/history-users-$(who am i | awk '{print $1}';exit)-$(date +%F)
export PROMPT_COMMAND='history -a'
fi
Set the permission.
chmod 770 /etc/profile.d/history_log.sh
3. Activate the script
source /etc/profile.d/history_log.sh
Now we have completed the settings for both secnarios i.e for super users and other users.
To test the secondary logging, login with any user other than root. Hit some commands and exit.
Login back with root user and check /var/log/users_historylogs/ directory . You will find some new log files are generated. Open the file and read it, you will find the history of commands used by your last login user.
Hi Sharad,
Could you please let me know if this process would work for AIX servers since /etc/profile.d doesn’t exist on AIX OS? Secondary logging for root will work if we put the contents in /etc/profile but I’m not sure how secondary logging would work for non-root users on AIX servers.
Thank you.
HI Ashok,
Currently I do not have AIX systems but I still remember this was implemented in AIX servers too.
Regards
Sharad
Hi Sharad your article is amazing could you please give me suggestion
I want to show which user turns to root in history command or in logs as your article is doing, As we have multiple users which turn to root using sudo sudo -i and we are not able to track which users turns root and runs command when multiple users turn root at the same time
Could you please help in suggesting a solution
Hi Vivek,
1. The Secondary logging settings should be applied for all users including root.
2. In log directory, it will create logs for all users.
When user switch to user root , it will run su command for it. This will be tracked in user’s log.
Hope, it will help.
Regards
Sharad
Hey, thanks for the article. This is useful. But do you have any way to make sure once the user sudos as root, there is no way he can delete/modify the history log files?
Hi Anand,
Generally we can set permission on file but root can do all in system. For keeping best practice, you can setup remote log server(eg. rsyslog) to keep the backup of logs in sync.
Apart of it, when user try to remove/modify the log that action will also be logged.
Regards
Sharad
Hi Sharad
When tried secondary logging for users, we added the script in vi /etc/profile.d/history_log.sh and when tried to login with other user and execute the command this doesn’t show or save in /var/log/users_historylogs/ only ubuntu user details present rest all users details not getting saved.
Hi Gaurav,
The root and non-root user should be setup separately.
If you can tell me your Operating System and its version, I will try to reproduce the issue and check again.
Also note that if you are trying with other user, the user should have hit some command after this implementation then only it will log.
Regards
Sharad
Hi Sharad
The OS is ubuntu 16.04 LTS, When enabled secondary logging for users as mentioned and when tried logging in and execute commands, we see only su – xxxx is getting captured but not the commands executed from secondary user in /var/log/users_historylogs/ . Please check this and advise if this works for you .
Hi Gaurav,
I found the problem area. It is actually with PROMPT_COMMAND . When user is login and becoming super user on
sudo su, the PROMPT_COMMAND is not working on Ubuntu. Probably it happened after change in bash versions. I am checking if any workaround on this in Ubuntu OS.Regards
Sharad
Hi Gaurav,
Try this in your Ubuntu system. Instead of creating shell script in /etc/profile.d update /etc/bash.bashrc .
In /etc/bash.bashrc add these new line .
root@ubuntu:/home/sharad# cat /etc/bash.bashrc |tail -20 _who_am_i=$(who am i|awk '{print $1}') _ID=$(id -u $_who_am_i) if [ "$_ID" -gt 0 ] then export HISTSIZE=10000 export HISTTIMEFORMAT='%F %T ' export HISTFILE=/var/log/users_historylogs/history-users-$(who am i | awk '{print $1}';exit)-$(date +%F) export PROMPT_COMMAND='history -a;date >> $HISTFILE' fi if [ "$_ID" == 0 ] then export HISTSIZE=10000 export HISTTIMEFORMAT='%F %T ' export HISTFILE=/var/log/sudo_historylogs/history-sudo-$(who am i | awk '{print $1}';exit)-$(date +%F) export PROMPT_COMMAND='history -a' fiAlso change the permission to 777 of /var/log/sudo_historylogs/ and /var/log/users_historylogs .
Check once and let me know.
Remember that the non-root user when it changes to login by using
sudo sucommand , at that time history will be saved in log.Regards
Sharad
Setup secondary logging for other users except root is not working for other user
Hi Rajesh,
Make a in file in /etc/profile.d/. It will apply to all.
Regards
Sharad
I have configured my server /etc/bashrc with these settings to capture sudo secondary logging
export HISTSIZE=10000
export HISTTIMEFORMAT=”%F %T ”
export HISTFILE=/var/log/bashhist/root_history-$(who am i | awk ‘{print $1}’;exit)
export PROMPT_COMMAND=’history -a’
output
In output am getting some random numbers instead date and timestamp
#1471863084
echo all is well
#1471863110
tail /var/log/bashhist/root_history-deepakp1
#1471863124
vim /etc/bashrc
Hello Deepak,
The date time is printed in epoch format. Given below is example to convert epoch format to human readable.
Regards
Sharad
Thanks Buddy !!!!!!!!!! Much appreciated. I have lot of lines to verified can we have any commands to verify or convert to whole into human readable