Install and configure rsyslog Centralized logging server in CentOS 6.5

In this tutorial we will learn, how to install and configure rsyslog 7.6 version on RHEL 6.5/CentOS 6.5 .The scenario is,install and setup rsyslog Centralized Logging Server in RHEL/CentOS 6.5. All the logs from client servers will be sent to Centralized logging server i.e rsyslog server.

rsyslog server

Check Pre-installed rsyslog package

Step 1: First of all check the rsyslog package is installed in your system.Generally by-default we get rsyslog version 5.x , after minimal installation of CentOS 6.x/ RHEL 6.x

We will install the latest rsyslog package. At the time of writing this post, rsyslog stable version 7.6 was available. You can find the latest package information from rsyslog official website

Note: By default, RHEL 6.x and CentOS 6.x has rsyslog version 5.x. So here we will update the rsyslog with new version.
You can get the rsyslog version information,by using below given two commands

rpm -qa|grep rsyslog
And
rsyslogd -v

See the below given screenshot

rsyslog version 7

Installing / Updating the rsyslog version 7.6 in RHEL 6.x / CentOS 6.x

For installing rsyslog version 7.6. Create a new yum client repo file and paste the below given contents. (With the same method,you can install other version of rsyslog [information link])

Creating new file /etc/yum.repos.d/rsyslog.repo (You can use your favorite editor )

vi /etc/yum.repos.d/rsyslog.repo

Paste the below given contents in file /etc/yum.repos.d/rsyslog.repo (In vi editor, for inserting content press i key,then paste content in file , for saving press key esc :wq )

[rsyslog-v7-devel]
name=Adiscon Rsyslog v7-devel for CentOS-$releasever-$basearch
baseurl=http://rpms.adiscon.com/v7-devel/epel-$releasever/$basearch
enabled=0
gpgcheck=0
protect=1

[rsyslog-v7-stable]
name=Adiscon Rsyslog v7-stable for CentOS-$releasever-$basearch
baseurl=http://rpms.adiscon.com/v7-stable/epel-$releasever/$basearch
enabled=1
gpgcheck=0
protect=1

For installing the rsyslog,in case the package is not available (rpm -qa|grep rsyslog). Run below given command

yum install rsyslog

For updating to new rsyslog version, run below given command

yum update rsyslog

Configure rsyslog in RHEL 6.x / CentOS 6.x

Step 1 : Enable the module.We will uncomment the below given line by removing #

Take the backup of original file

cp -pv /etc/rsyslog.conf /etc/rsyslog.conf.orig

Edit the file /etc/rsyslog.conf

vi /etc/rsyslog.conf

Uncomment by removing # in front of these module names

module(load="imuxsock") # provides support for local system logging (e.g. via logger command)
module(load="imklog")   # provides kernel logging support (previously done by rklogd)

Now, in same file , search for line *.emerg *. Modify the action (i.e *) with :omusrmsg:* . See below given reference

*.emerg                        :omusrmsg:*

Enable UDP port no. 514 for rsyslog .Uncomment below given lines by removing # sign

module(load="imudp") # needs to be done just once
input(type="imudp" port="514")

Now, at the end of file /etc/rsyslog.conf, paste the below given code(These are rsyslog templates)

$template TmplAuth, "/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log" 
$template TmplMsg, "/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log" 
authpriv.*   ?TmplAuth
*.info,mail.none,authpriv.none,cron.none   ?TmplMsg

Now , save and exit from file vi /etc/rsyslog.conf

Below given is the reference from our Server,after editing /etc/rsyslog.conf, it looks as given below (verify your file with below given reference)
Here, egrep -v '^#|^$' command will show only uncommented lines from the file.

[root@localhost /]# egrep -v '^#|^$' /etc/rsyslog.conf -v 
module(load="imuxsock") # provides support for local system logging (e.g. via logger command)
module(load="imklog")   # provides kernel logging support (previously done by rklogd)
module(load="imudp") # needs to be done just once
input(type="imudp" port="514")
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  /var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log
$template TmplAuth, "/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log"
$template TmplMsg, "/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log"
authpriv.*   ?TmplAuth
*.info,mail.none,authpriv.none,cron.none   ?TmplMsg
[root@localhost /]# 

Step 4: Now edit the file /etc/sysconfig/rsyslog .And set the SYSLOGD_OPTIONS with no parameter

SYSLOGD_OPTIONS=""

See the below given reference from our Server

[root@localhost ~]# cat /etc/sysconfig/rsyslog
# Options for rsyslogd
# Syslogd options are deprecated since rsyslog v3.
# If you want to use them, switch to compatibility mode 2 by "-c 2"
# See rsyslogd(8) for more details
SYSLOGD_OPTIONS=""
[root@localhost ~]# 

Step 5: Start/Restart the rsyslog service

For starting rsyslog

 /etc/init.d/rsyslog start 

For restarting rsyslog

 /etc/init.d/rsyslog restart 

[for stoping rsyslog, /etc/init.d/rsyslog stop]

Create new directory in /var/log

Create new directory in /var/log called rsyslog_custom. so that we will keep all Servers log in this directory.

mkdir -p /var/log/rsyslog_custom

Set SELINUX rules for rsyslog

Some system admin, disable the selinux .
If you want to keep SELINUX enable. Use the below given command
(Read this post,in case semanage command not found)

semanage fcontext -a -t syslogd_exec_t /sbin/rsyslogd
restorecon /sbin/rsyslogd

/usr/sbin/semanage fcontext -a -t var_log_t "/var/log/rsyslog_custom(/.*)?"
/sbin/restorecon -R -v /var/log/rsyslog_custom

Set IPTABLES for rsyslog

rsyslog service uses the UDP port number 514 .Hence we will set the iptable only for this port

Edit /etc/sysconfig/iptables

vi /etc/sysconfig/iptables

Put the given below RULE always above of any REJECT INPUT rule

-A INPUT -m state --state NEW -m udp -p udp --dport 514 -j ACCEPT

Now, save and exit. Restart the iptables service

/etc/init.d/iptables restart

Use iptables -nL command to check the iptables rule

Restart rsyslog and verify listening status of 514 port

/etc/init.rsyslog restart

Check listening status of port 514

 netstat -uanp|grep rsyslog

See below given reference from my server

[root@localhost ~]# netstat -uanp|grep rsyslog
udp        0      0 0.0.0.0:514                 0.0.0.0:*                               3266/rsyslogd       
udp        0      0 :::514                      :::*                                    3266/rsyslogd       
[root@localhost ~]# 

Configure rsyslog in client servers

To fetch the logs from remote client servers.We will edit the rsyslog.conf file in client machine. The method is applicable to Red Hat based and Debian based Operation system (eg. RHEL,CentOS,Debian,Ubuntu)

Syntax:
*.* @ip-address-of-rsyslog-server:514

Example:
Edit file

vi /etc/rsyslog.conf

Paste the below line (replace 192.168.56.102 with your rsyslog server ip address, here 514 is UDP port no.)

*.* @192.168.56.102:514

Save and exit from file /etc/rsyslog.conf and restart the rsyslog service

/etc/init.d/rsyslog restart

Now relogin in Client system so that we will capture the log.And same log information we will see in rsyslog server

Verify the log in rsyslog server

Change to directory /var/log/rsyslog_custom . You must see,a directory with hostname of your client machine. And inside that directory you will see some logs.

cd /var/log/rsyslog_custom

Reference from my system (Output will be different in your case)


[root@localhost ~]# ls -l /var/log/rsyslog_custom/
total 8
drwx------. 2 root root 4096 Mar  1 07:52 localhost
drwx------. 2 root root 4096 Mar  1 07:47 tuxworld
[root@localhost ~]# 
[root@localhost ~]# ls -l /var/log/rsyslog_custom/tuxworld/
total 32
-rw-------. 1 root root 193 Mar  1 07:46 CRON.log
-rw-------. 1 root root 619 Mar  1 07:47 dbus.log
-rw-------. 1 root root 255 Mar  1 07:47 dhclient.log
-rw-------. 1 root root   0 Mar  1 07:46 kernel.log
-rw-------. 1 root root 659 Mar  1 07:47 NetworkManager.log
-rw-------. 1 root root 120 Mar  1 07:46 rsyslogd-2039.log
-rw-------. 1 root root 149 Mar  1 07:46 rsyslogd.log
-rw-------. 1 root root 296 Mar  1 07:44 sudo.log
-rw-------. 1 root root 316 Mar  1 07:44 su.log
[root@localhost ~]# 

IMPORTANT Note: After setting all things,it is good practice to check /var/log/messages .In case of any rsyslog related error/warning is found

Note: This rsyslog server setup is recommended to use in secure internal network. The rsyslog server should not be publicly exposed.

In case you want to secure the rsyslog server.Use the iptable which only accept logs from your desired particular ip address/ network .

For EXAMPLE Only allow particular ip address to connect with rsyslog server for logging. The client machine has ip address 192.168.56.1 (replace 192.168.56.1 with your client machine ip address [ifconfig])

vi /etc/sysconfig/iptables

-A INPUT -m state --state NEW -m udp -p udp -s 192.168.56.1 --dport 514 -j ACCEPT

Restart the iptable after this /etc/init.d/iptables restart

For EXAMPLE. Only allow particular NETWORK to connect with rsyslog server for logging. The Network subnet is 10.0.0.0/255.255.255.0 (replace 10.0.0.0/255.255.255.0 from your network information )

vi /etc/sysconfig/iptables

-A INPUT -m state --state NEW -m udp -p udp -s 10.0.0.0/24 --dport 514 -j ACCEPT

Restart the iptable /etc/init.d/iptables restart

For further reading : Viewing and Managing Log Files

45 thoughts on “Install and configure rsyslog Centralized logging server in CentOS 6.5”

  1. Thanks Sharad for the great post. quite helpful. One thing which I would add is the need to merge rsyslog.conf.rpmnew created when we upgrade from syslog 5.x to rsyslog7 in RHEL 6.x

    Reply
  2. Hi,

    Everything working fine. But, every-time i will restart client pc rsyslog service latest file uploaded to Remote server.

    I want automatically upload every minute. What should i do now. Please mail me

    Regards
    S.Gandhi

    Reply
  3. Hi Sharad,

    Greetings

    I have seen your videos for syslog. it is wonderful. i am going rsylog setup with TLS.

    How to configure rsyslog with TLS configure.

    Please help me for this setup
    Thanks

    Reply
  4. Hi Sharad,

    I want forward messages from malty clients to malty remote servers

    Clients

    Client1
    Client2
    Client3

    Remote Servers

    RemoteServer1
    RemoteServer2
    RemoteServer3

    1.I want send logs like below

    Client1&2&3 –> RemoteServer1—>Remoteserver2

    First Client1&2&3 logs need to forward to RemoteServer1 for RemoteServer1 all logs (Client1&2&3 and RemoteServe1) to RemoteServer2 with TLS mode

    2.I want send logs like below

    Client1 –> RemoteServer1—>Remoteserver3

    Hear i want to send only specific clients to send RemoteServer3

    First Client1 logs need to forward to RemoteServer1 from RemoreServer1 only Client1 logs need to send to RemoteServer3 without TLS mode

    Please Suggest me

    Reply
  5. Hi Sharad,

    Everything is working fine after following your instructions, the rsyslog server got the log files from clients. However, I didn’t see most of log files under /var/log/* from my Ubuntu clients.

    Here’s the list of missing files

    /var/log/alternatives.log
    /var/log/auth.log
    /var/log/boot.log
    /var/log/dmesg
    /var/log/dpkg.log
    /var/log/faillog
    /var/log/lastlog
    /var/log/syslog
    …..

    How can i get these log files to rsyslog server?

    Thanks,
    Ben

    Reply
    • Hi Ben,

      The format should be like given below example in rsyslog.conf .
      Try once and do changes. for more understanding on template read this URL http://www.rsyslog.com/doc/v8-stable/configuration/templates.html

      *.info;mail.none;authpriv.none;cron.none /var/log/messages
      authpriv.* /var/log/secure
      mail.* /var/log/maillog
      cron.* /var/log/cron
      *.emerg :omusrmsg:*
      uucp,news.crit /var/log/spooler
      local7.* /var/log/boot.log
      $template TmplAuth, “/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log”
      $template TmplMsg, “/var/log/rsyslog_custom/%HOSTNAME%/%PROGRAMNAME%.log”
      authpriv.* ?TmplAuth
      *.info,mail.none,authpriv.none,cron.none ?TmplMsg

      Reply
  6. How an we add application logs to sync on rsyslog server?

    If mine application generating logs in file “/var/www/example/log/error.log”. so how can i add this to configuration to get it synced on rsyslog server?

    Reply
    • Hello Lokesh,

      Use the imfile rule .

      $ModLoad imfile
      $InputFileName /var/www/example/log/error.log
      $InputFileTag example-error
      $InputFileStateFile stat-example-error
      $InputFileSeverity error
      $InputFileFacility local3
      $InputRunFileMonitor
      local3.* @@:
      

      I will highly recommend you to also read about the Severity level in syslog. Syslog

      Here given below are other readings, which will help you.
      Rsyslog Replacer
      Rsyslog Filter

      Just try , it should work.I have not tested yet, just recalled from memories.

      Regards
      Sharad

      Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.