Harderning with SELinux
I finally got around to hardening my web server over the holiday break. A very important step in hardening linux is ensuring selinux is both enabled and enforcing. Of course the risk is that Security Enhanced Linux starts blocking things that should not be blocked…
I have been reluctant to have SELinux enforcing on all my instances of linux for a long time. It turns out that nowadays, with the maturity of the feature and its associated tools, this is pretty much an indefensible position.
Sure, back in the mid-noughties SELinux had its issues and caused much grief for sysadmins, but things have changed… a lot. This post documents me catching up, and will hopefully be useful to anyone trying to do the same!
SELinux is the most essential security tool available on linux. Period. No other single tool succeeds succeeds so well at securing a system from the inside out.
It originated from a 3-letter agency with a special interest in security. You can read more about it at wikipedia.
I used Centos 7 for this post. Most of my machines use Centos without a GUI installed. The tools should be identical on Red Hat (RHEL) and very similar on any modern linux distro. Of course the system commands may differ on your platform.
For the rest of the post I’ll assume you are using Centos 7.
[sysadmin@imago ~]$ cat /etc/centos-release CentOS Linux release 7.6.1810 (Core)
How SELinux Works
For those familiar with networking, SELinux works a little like a firewall, in that it controls activity between sources and targets based on a configured policy. That policy may include definitions of the source and target objects and rules about the types of activity allowed between them.
SELinux differs from a firewall in that it is interested in the activity within your system, not between systems on a network. And it does a very good job of this.
Further, source and target objects are not limited to hosts. They include a whole range of linux “things”, such as process, files, directories, devices and sockets. Basically, any element of a linux system.
SELinux has total visibiility and control of the activity between any of these objects. For example, when a process tries to read a file, or open a network socket.
KEY CONCEPT: TYPES
While SELinux can is very flexible and get very complicated, the basics are actually quite simple.
Every object on the system is labelled with a type. For example if you were to look at the files in a directory with the “-Z” option, you will see the SELinux attributes associated with each object (file or directory), including the type.
Filesystem OBJECT TYPES
Here is how my home directory looks:
[sysadmin@server ~]$ ls -aZ drwx------. sysadmin sysadmin system_u:object_r:user_home_dir_t:s0 . drwxr-xr-x. root root system_u:object_r:home_root_t:s0 .. -rw-------. sysadmin sysadmin system_u:object_r:user_home_t:s0 .bash_history -rw-r--r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 .bash_logout -rw-r--r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 .bash_profile -rw-r--r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 .bashrc -rw-------. sysadmin sysadmin system_u:object_r:user_home_t:s0 .lesshst drwx------. sysadmin sysadmin system_u:object_r:ssh_home_t:s0 .ssh -rw-rw-r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 tmp1 -rw-rw-r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 tmp2 -rw-------. sysadmin sysadmin unconfined_u:object_r:user_home_t:s0 .viminfo -rw-r--r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 .vimrc
The type here is denoted by “_t”. Most files here are type “user_home_t”.
The “_u” denotes user, and “_r” denote roles, but these related to SELinux’s role-based access system and can safely be ignored by most sysadmins.
Process Object Types
Processes also have a type, and simillarly, these can be shown with the “-Z” switch. Most of the standard utilities will use this switch:
[sysadmin@server ~]$ ps -aZ LABEL PID TTY TIME CMD unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 2744 pts/0 00:00:00 sudo unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 2748 pts/0 00:00:00 su unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 2749 pts/0 00:00:00 bash unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 3648 pts/0 00:00:00 tail unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 3649 pts/0 00:00:00 grep unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 5524 pts/1 00:00:00 ps
Here, all the processes my user has initiated are of type “unconfined_t”.
Rules basically boil down to which type of sources can access which types of targets. That’s pretty much all you need to know to get started.
For example can a process of type “unconfined_t” write to a directory with the “user_home_dir_t” type? Can it read from the same directory? Or modify a particular file in that directory?
Remember, these rules are separate from the file permissions system on the linux filesystem, and selinux runs in series. It won’t alter or override those permissions.
The TARGETED Policy
The default policy is also sufficient to get started.
Instead of changing the policy, we will attempt to resolve any issues with the system just by changing the type of any objects we encounter in our logs.
Checking the Status of SELinux
SELinux has two states enabled or disabled.
The enabled state has two modes: permissive or enforcing.
In permissive mode SELinux is doing everything except denying any activity. Log messages will appear just as though SELinux was enforcing policy on your system.
It is highly recommended to start in permissive mode and check your logs before going to enforcing, which we will do below. Permissive mode seems to be the default mode on most linux distros.
The most useful commands for checkiong the status of SELinux are:
On Centos 7, these utilities can be found in the following packages which are part of the base installation:
[sysadmin@server ~]$ sudo yum -q whatprovides /sbin/sestatus policycoreutils-2.5-29.el7.x86_64 : SELinux policy core utilities Repo : @base [sysadmin@server ~]$ sudo yum -q whatprovides /sbin/getenforce libselinux-utils-2.5-14.1.el7.x86_64 : SELinux libselinux utilies Repo : @base
In disabled mode, SELinux is not loaded into the kernel, and no functionality is available.
[sysadmin@server ~]# sestatus SELinux status: disabled
If your machine looks like above, then check the following file:
cat /etc/selinux/config # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # minimum - Modification of targeted policy. Only selected processes are protected. # mls - Multi Level Security protection. SELINUXTYPE=targeted
You will need to enable SELinux by changing “disabled” to “permissive” and then rebooting your machine.
sudo vim /etc/selinux/config SELINUX=permissive
This should be a safe action. I performed this without issue on a cloud machine on which I have no console access. The machine took a little longer to boot the first time, but came back as expected. Of course, there’s no guarantees so take care!
getenforce provides a simple one-line output, useful for confirming which enabled mode your system is in:
[sysadmin@server ~]$ getenforce Permissive
sestatus provides a more detailed output:
[sysadmin@server ~]$ sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: permissive Mode from config file: permissive Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy version: 28
The most useful information here is the mode (permissive) and the policy name (targeted).
From here we’ll assume you are in permissive mode.
How to find SELinux logs
On Centos, SELinux uses the linux audit daemon (service), known as auditd.
Auditd logs everything to /var/log/audit/audit.log which requires elevated privileges to read:
[sysadmin@server ~]$ cat /var/log/audit/audit.log cat: /var/log/audit/audit.log: Permission denied [sysadmin@server ~]$ sudo more /var/log/audit/audit.log type=USER_AUTH msg=audit(1546124135.971:451246): pid=30238 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:authentication grantors=? acct="root" exe="/usr/sbin/sshd" hostname=18.104.22.168 addr=22.214.171.124 terminal=ssh res=failed' type=USER_AUTH msg=audit(1546124138.373:451247): pid=30238 uid=0 auid=4294967295 ses=4294967295 msg='op=password acct="root" exe="/usr/sbin/sshd" hostname=? addr=126.96.36.199 terminal=ssh res=failed' <...>
On the plus side, everything you need is in one place. On the negative side, this file can get quite large, contains a log of messages unrelated to SELinux, and the output is a little cryptic.
As a first step we can just filter down to the relevant logs, which will begin with “type=AVC”:
[sysadmin@server ~]$ sudo cat /var/log/audit/audit.log | fgrep "type=AVC" | more
On some of my systems there were no “AVC” entries in the audit file. The rest of this post will focus on a cloud-located server that had multiple entries – in fact, the server hosting this blog.
Later on, we’ll use other tools to interpret these entries and even make recommendations.
In Part 2…
In the next part we will start looking at some example logs, where SELinux would have denied some activity on my system, had it been enforcing.