This is going to be a longish post (for me) so for those with an attention span as short as mine you may want to skip ahead to a specific section that interests you:
- the discovery - how did I find it?
- the vulnerability - why does it work?
- the exploit - how can I exploit it? Including screenshots and video
- the security considerations - container escapes, unauthenticated RCE
- the timeline - how long did this all take?
SaltStack Salt is a popular tool used to automate and secure infrastructure. Its usage is split into two roles: one system is set up as the master and is responsible for controlling those systems that connect to it. One or more systems are then set up as minions that connect to the master and respond to any commands it issues.
Both master and minions are typically run as root on the systems they are installed on.
Whilst looking at the source code for SaltStack for one of the previously disclosed vulnerabilities at work, I decided to run the source code through Bandit, a security scanner for Python applications to see how many issues it would find.
I was expecting to see quite a few results, as it has a relatively large codebase and has existed for several years. However, as can be seen in the screenshot below Bandit showed even more issues than I was expecting including 117 high severity issues.
Of course, many of these issues are false positives or of little importance to us, and it can take significant time to parse through all the data. For a quick win, I decided to focus my time researching some potential command injections due to several instances of
subprocess.Popen used in conjunction with
shell=True in the codebase.
After looking at a few of these that turned out not to be in any way controllable I found one that could be controlled via some clever trickery using process names.
restartcheck is vulnerable to command injection via a crafted process name when this process has open file descriptors associated with
(deleted) at the end of a filename (note the leading space is required). This allows for a local privilege escalation to root from any user able to create files on the minion in a directory that is not explicitly forbidden.
The vulnerable code is at line 615 in restartcheck.py where
subprocess.Popen is called with
shell=True and a command that can be maniplulated by an attacker:
cmd = cmd_pkg_query + package
paths = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
package is formed from the process name and
cmd_pkg_query is one of the following depending on the OS:
If we can insert a bash control character such as
&& etc into the process name we can trigger the injection when this code is reached.
However, to reach this code, a process first needs to have a filehandler open to a file with a filename that ends in
(deleted), and this file needs to reside in a directory that is not explicitly forbidden.
The list of forbidden directories can be seen here. This deny list immediately rules out some of the more obvious places we might try to use such as
/dev/shm, however, there are a few common ones that a low privileged user may have access to such as:
During this research, it turned out that process names can be a surprisingly tricky thing to modify reliably, and a process name listed by
ps may not be the same returned by the Python psutil library.
In Linux, process names can contain any characters (apart from null). Any user can start processes on a system and the process itself can set the process name. They are a good target for command injection vulnerabilities as developers are unlikely to expect process names to contain special characters or injections.
It is possible to use
exec -a to directly set a process name; however, this doesn't work in
sh shells and doesn't appear to show the same name when using
psutil. It's also possible to modify the process name by directly manipulating procfs however this also leads to inconsistent results.
So in the end, the simplest and most consistent way to set a process name is to rename the binary or script being run. As filenames in Linux cannot contain
/ this restricts the commands we can inject, however using base64 encoding it is trivial to bypass this restriction as shown below:
# If we wanted to copy the shadow file to /tmp we could run this
cp /etc/shadow /tmp/shadow
# Convert it to a base64 string
echo cp /etc/shadow /tmp/shadow | base64 -w0
# The result of the conversion
# The new command we need to run
echo Y3AgL2V0Yy9zaGFkb3cgL3RtcC9zaGFkb3cK|base64 -d|sh -i
For this exploit to work, a writable directory that isn't explicitly forbidden by SaltStack is required. Running the proof of concept script with no arguments will search for directories that match this restriction.
Then by passing a suitable writable directory with the -w flag and a command with the -c flag, a process will be created containing the command injection in the process name and an open file handler that will cause the exploit to be triggered when the master calls restartcheck.
Let's step through the process of performing the exploit to demonstrate the vulnerability. We aim to create a simple file as the root user. First, we run the script providing the proper flags:
Now that we've confirmed the malicious process is running with a command injection in the name and a filehandler open, we can issue the
restartcheck.restartcheck command on the SaltStack master. Once this has completed, we can check for the existence of the hacked file in the root directory.
That's great but writing files is dull, lets do something a bit cooler. How about getting a shell as root? The video below shows one way of doing this by copying the find binary and making it suid:
For additional context the screenshot below shows how the exploit above appears from the master's point of view:
Additional Security Considerations
The above discusses the exploitation of this vulnerability locally to escalate privileges to the root user; however, there a couple of additional security concerns that should be considered:
As containerized processes are listed on the host machine, this exploit could be performed from within a container to gain command execution as root on the host machine.
Potential For Unprivileged RCE
While somewhat unlikely, there is potential for this attack to be performed by an attacker without local shell access. This is because, under certain circumstances, a remote user can influence process names.
SaltStack were very fast to respond and the whole timeline for discovery and disclosure can be seen below:
05 Nov 2020 — Vulnerability discovered
05 Nov 2020 — SaltStack notified
05 Nov 2020 — SaltStack investigating
06 Nov 2020 — CVE ID assigned
07 Nov 2020 — SaltStack confirmed vulnerability
18 Nov 2020 — SaltStack notified intention to release fix in Jan
22 Jan 2021 — Security fix announced for 4th Feb
04 Feb 2021 — Security fix delayed to 25th Feb
25 Feb 2021 — Security fix released (+91 days)
Developing and maintaining secure code is difficult and who the hell expects process names to be malicious? Hit me up on twitter if you have any questions or feedback, this was my first CVE so be kind.