Troubleshooting SELinux – Part 1

Harderning with SELinux

I finally got around to hardening my web server over the holiday break. A very important step in hardening linux is ensuring selinux is both enabled and enforcing. Of course the risk is that Security Enhanced Linux starts blocking things that should not be blocked…

Undeserved Reputation?

I have been reluctant to have SELinux enforcing on all my instances of linux for a long time. It turns out that nowadays, with the maturity of the feature and its associated tools, this is pretty much an indefensible position.

Sure, back in the mid-noughties SELinux had its issues and caused much grief for sysadmins, but things have changed… a lot. This post documents me catching up, and will hopefully be useful to anyone trying to do the same!

Why SEliNux?

SELinux is the most essential security tool available on linux. Period. No other single tool succeeds succeeds so well at securing a system from the inside out.

It originated from a 3-letter agency with a special interest in security. You can read more about it at wikipedia.

PRE-REQUISITES

I used Centos 7 for this post. Most of my machines use Centos without a GUI installed. The tools should be identical on Red Hat (RHEL) and very similar on any modern linux distro. Of course the system commands may differ on your platform.

For the rest of the post I’ll assume you are using Centos 7.

[sysadmin@imago ~]$ cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core)

How SELinux Works

For those familiar with networking, SELinux works a little like a firewall, in that it controls activity between sources and targets based on a configured policy. That policy may include definitions of the source and target objects and rules about the types of activity allowed between them.

SELinux differs from a firewall in that it is interested in the activity within your system, not between systems on a network. And it does a very good job of this.

Further, source and target objects are not limited to hosts. They include a whole range of linux “things”, such as process, files, directories, devices and sockets. Basically, any element of a linux system.

SELinux has total visibiility and control of the activity between any of these objects. For example, when a process tries to read a file, or open a network socket.

KEY CONCEPT: TYPES

While SELinux can is very flexible and get very complicated, the basics are actually quite simple.

Every object on the system is labelled with a type. For example if you were to look at the files in a directory with the “-Z” option, you will see the SELinux attributes associated with each object (file or directory), including the type.

Filesystem OBJECT TYPES

Here is how my home directory looks:

[sysadmin@server ~]$ ls -aZ
drwx------. sysadmin sysadmin system_u:object_r:user_home_dir_t:s0 .
drwxr-xr-x. root root system_u:object_r:home_root_t:s0 ..
-rw-------. sysadmin sysadmin system_u:object_r:user_home_t:s0 .bash_history
-rw-r--r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 .bash_logout
-rw-r--r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 .bash_profile
-rw-r--r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 .bashrc
-rw-------. sysadmin sysadmin system_u:object_r:user_home_t:s0 .lesshst
drwx------. sysadmin sysadmin system_u:object_r:ssh_home_t:s0 .ssh
-rw-rw-r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 tmp1
-rw-rw-r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 tmp2
-rw-------. sysadmin sysadmin unconfined_u:object_r:user_home_t:s0 .viminfo
-rw-r--r--. sysadmin sysadmin system_u:object_r:user_home_t:s0 .vimrc

The type here is denoted by “_t”. Most files here are type “user_home_t”.

The “_u” denotes user, and “_r” denote roles, but these related to SELinux’s role-based access system and can safely be ignored by most sysadmins.

Process Object Types

Processes also have a type, and simillarly, these can be shown with the “-Z” switch. Most of the standard utilities will use this switch:

[sysadmin@server ~]$ ps -aZ
LABEL PID TTY TIME CMD
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 2744 pts/0 00:00:00 sudo
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 2748 pts/0 00:00:00 su
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 2749 pts/0 00:00:00 bash
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 3648 pts/0 00:00:00 tail
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 3649 pts/0 00:00:00 grep
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 5524 pts/1 00:00:00 ps

Here, all the processes my user has initiated are of type “unconfined_t”.

POLICY Rules

Rules basically boil down to which type of sources can access which types of targets. That’s pretty much all you need to know to get started.

For example can a process of type “unconfined_t” write to a directory with the “user_home_dir_t” type? Can it read from the same directory? Or modify a particular file in that directory?

Remember, these rules are separate from the file permissions system on the linux filesystem, and selinux runs in series. It won’t alter or override those permissions.

The TARGETED Policy

The default policy is also sufficient to get started.

Instead of changing the policy, we will attempt to resolve any issues with the system just by changing the type of any objects we encounter in our logs.

Checking the Status of SELinux

SELinux has two states enabled or disabled.

The enabled state has two modes: permissive or enforcing.

In permissive mode SELinux is doing everything except denying any activity. Log messages will appear just as though SELinux was enforcing policy on your system.

It is highly recommended to start in permissive mode and check your logs before going to enforcing, which we will do below. Permissive mode seems to be the default mode on most linux distros.

The most useful commands for checkiong the status of SELinux are:

sestatus
getenforce

On Centos 7, these utilities can be found in the following packages which are part of the base installation:

[sysadmin@server ~]$ sudo yum -q whatprovides /sbin/sestatus
policycoreutils-2.5-29.el7.x86_64 : SELinux policy core utilities
Repo : @base

[sysadmin@server ~]$ sudo yum -q whatprovides /sbin/getenforce
libselinux-utils-2.5-14.1.el7.x86_64 : SELinux libselinux utilies
Repo : @base
SELinux Disabled

In disabled mode, SELinux is not loaded into the kernel, and no functionality is available.

[sysadmin@server ~]# sestatus
SELinux status:                 disabled

If your machine looks like above, then check the following file:

cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted

You will need to enable SELinux by changing “disabled” to “permissive” and then rebooting your machine.

sudo vim /etc/selinux/config
SELINUX=permissive

This should be a safe action. I performed this without issue on a cloud machine on which I have no console access. The machine took a little longer to boot the first time, but came back as expected. Of course, there’s no guarantees so take care!

SELinux ENabled

getenforce provides a simple one-line output, useful for confirming which enabled mode your system is in:

[sysadmin@server ~]$ getenforce
Permissive

sestatus provides a more detailed output:

[sysadmin@server ~]$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: permissive
Mode from config file: permissive
Policy MLS status: enabled
Policy deny_unknown status: allowed
Max kernel policy version: 28

The most useful information here is the mode (permissive) and the policy name (targeted).

From here we’ll assume you are in permissive mode.

How to find SELinux logs

On Centos, SELinux uses the linux audit daemon (service), known as auditd.

Auditd logs everything to /var/log/audit/audit.log which requires elevated privileges to read:

[sysadmin@server ~]$ cat /var/log/audit/audit.log
cat: /var/log/audit/audit.log: Permission denied

[sysadmin@server ~]$ sudo more /var/log/audit/audit.log
type=USER_AUTH msg=audit(1546124135.971:451246): pid=30238 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:authentication grantors=? acct="root" exe="/usr/sbin/sshd" hostname=112.85.42.88 addr=112.85.42.88 terminal=ssh res=failed'
type=USER_AUTH msg=audit(1546124138.373:451247): pid=30238 uid=0 auid=4294967295 ses=4294967295 msg='op=password acct="root" exe="/usr/sbin/sshd" hostname=? addr=112.85.42.88 terminal=ssh res=failed'
<...>

On the plus side, everything you need is in one place. On the negative side, this file can get quite large, contains a log of messages unrelated to SELinux, and the output is a little cryptic.

As a first step we can just filter down to the relevant logs, which will begin with “type=AVC”:

[sysadmin@server ~]$ sudo cat /var/log/audit/audit.log | fgrep "type=AVC" | more

On some of my systems there were no “AVC” entries in the audit file. The rest of this post will focus on a cloud-located server that had multiple entries – in fact, the server hosting this blog.

Later on, we’ll use other tools to interpret these entries and even make recommendations.

In Part 2…

In the next part we will start looking at some example logs, where SELinux would have denied some activity on my system, had it been enforcing.

 

Power Scales – Orders of Magnitude in WiFi Signal Strength

WiFi Signal Strength

One thing in my WiFi journey I’ve found interesting, is how well the technology works over what seems a very large range of WiFi signal strength. A bit like Conor McGregor in the ring, WiFi just keeps coming at you, no matter how you try to run or shield your face from it’s blows..

Getting pumped up about WiFi signal strength.

Big Range

Let’s not get too Isaac Newton, but basically in the radio world, signal strength varies exponentially as distance varies linearlyThis is why when we look at WiFi using a linear scale, such as milliwatts, we see such small values alongside such large values.

Linear vs. Logarithmic Scales

And this why we prefer to use dBm as our scale. A logarithmic scale like dBm turns exponentially varying numbers into linearly varying numbers. Making them more usable to everyday engineer types.Ways to optimise the available airtime on your WiFi network to improve speed and performance.  Continue reading “Power Scales – Orders of Magnitude in WiFi Signal Strength”

Improving Wireshark for WiFi Packet Analysis

TLDR;

Download the Wireshark for WiFi profile now or read on.

Wireshark

Even if you’re an avid user of some of the premium packet analysis tools out there, such as Savvius’ excellent Omnipeek, every so often most people will be opening up the free Wireshark to look at a capture.

Unfortunately the default view included with Wireshark is very poorly suited to 802.11 packet analysis:

Continue reading “Improving Wireshark for WiFi Packet Analysis”

How Do I Increase Wi-Fi Airtime? Let Me Count The Ways..

Introduction

I recently attended a Certified Wireless Design Professional (CWDP) course, host by Spectrotech and trained by Wi-Fi expert and CWNE #1, Devin Akin.

CWDP Badge

One of the highlights of a week of intense training was an exercise on the last day. Devin asked the class to break into teams and each team was to come up with as many ways to increase Wi-Fi airtime as possible. We would then reconvene and deliver our results one-at-a-time in round-robin fashion. If one team gave an answer, the others couldn’t give the answer and had to come up with their own, until all responses had been delivered. The team with the most answers would be declared the winner.

Well, with so many great professionals in each team, it’s no wonder that Devin managed to fill the whiteboard and then had to declare a tie.

Ways To Increase Wi-Fi Airtime Whiteboard
The Whiteboard (credit: Scott Doorey)

Continue reading “How Do I Increase Wi-Fi Airtime? Let Me Count The Ways..”