Attacking management interfaces

29 09 2014

The management interfaces and everything that is incorporated into that software is, in the author’s opinion the most problematic area in virtualisation security today. There have been numerous attempts over the last few years to demonstrate how management interfaces can be breeched. The majority of these attacks are general attacks that use pre-existing attack methods such as brute forcing, MiTM (man-in-the-middle) and the numerous flaws with the PKI infrastructure. There are multiple proven attack methods available for exploiting management interfaces and below are descriptions of some of these attacks that have been discovered by researchers.

In an online blog (Mluft, 2011)  talks about how a brute force attack is achievable on the Amazon Web Services (AWS) portal by leveraging existing hacking software tools. In the attack (Mluft, 2011) demonstrated how it is possible to determine a successful logon using the exemplary payloads in the Burp Suite (Burp Suite, 2012). The use of the burp suite in this example is to simply automate the process of attempting logins to the interface. The payload in the software is also able to identify failed login attempts to the portal by returning a HTTP status code of 200 (Network Working Group , 1999). A correct password attempt is identified by a returned HTTP status code of 302. Using the documentation provided by Amazons services in relation to password policies (Mluft, 2011) created an appropriate wordlist and used the burp suite to attempt all the possible permutations. After 400,000 attempts the attack was paused and the results purged for the 302 status code. The code was found and also shown alongside was the value and the password attempted. This gave the attacker the username and password for the administration of all servers managed by that account. It should be noted that all of these attempts were originated from one IP address without the account being locked-out or subject to any account throttling.

As discussed earlier my earlier blog artical regarding the hypervisor, the Virtualization Assessment Toolkit (VASTO) has been developed to exploit multiple weaknesses, predominantly in the VMware family. As well as the identification module that returns the exact version of the server, it includes numerous attacks on virtual systems including a specific VMware brute forcing module, which mimics the attack on the AWS portal by (Mluft, 2011). One of the main contributors to the VASTO project (Criscione, 2010) demonstrated a number of the different functions found in VASTO at Blackhat USA 2010.  Although (Criscione, 2010) demonstrated how VASTO can be used at multiple layers of the virtual stack (Client, Hypervisor, Support, Management and internal), the majority concentrated on the management portion. (Criscione, 2010) confirms that although the (VMware, 2012) hardening guide recommends segmentation of management networks, these recommendations are often ignored and left situated on the same networks as traditional servers.

These servers that manage the entire fabric of the infrastructure have multiple attack vectors – from the operating systems they are installed on to the web services running the interfaces. Vulnerabilities in any one of these platforms can potentially jeopardise the security of an entire environment and should be taken very seriously.

The other element used in the VASTO modules which can target the management portion of the virtual infrastructure uses target flaws in the VMware components and implementation to expose threats in the infrastructure. One of the exploits that is included in the VASTO suite that best demonstrates how multiple components in these systems can be used for exploitation, originates via a flaw in the Jetty (Eclipse, 2012) web server that is used by vCenter Update manager. In the author’s opinion, this attack signifies how the complexity and code overhead that these management servers introduce, make securing virtual environments in an efficient manner, one that needs to be understood and prioritised. I will briefly give a breakdown of this attack to highlight the multiple elements that were used to complete the attack.

The Update Manager component of the vSphere suite is designed to secure the environment by automating the patching and updating process of hosts that fall under its management scope. However, (Criscione, 2010) recognised that the update manager requires a version of Jetty web server to operate. This is an additional component that is added to the total footprint of the management server. The version of Jetty installed prior to version 4.1 u1 (update 1) of the update manager was a version vulnerable to a directory traversal attack (Wilkins, 2009), which allowed attackers to view any files on a server that the Windows SYSTEM user has privileges. Consequentially vCenter stored a file on the server called “vpxd-profiler-*” which is a file used by administrators for debugging purposes. In this extensive file the, SOAP Session ID’s of all the users that have connected to that server are contained. With this ID the vmware_session_rider module, found in the VASTO toolkit, acts as a proxy server to allow the attacker to then connect through it into the vCenter server using the selected administrator SOAP ID. Once this is completed, the attacker is able to create a new admin credential within vCenter to ensure future access.

Another example of how different elements of the management interface could be used to gain access to vCenter is through VMware’s use of Apache Tomcat technology (The Apache Software Foundation, 2012). When navigating to a vCenter server through a web browser one is presented with the standard vSphere “Getting started” screen as is shown in figure 1

Web browser connection to vCenter server

Web browser connection to vCenter server

Connection to that same servers IP address, but specifying the default tomcat Tomcats index page port of “8443” over an SSL connection shows further information, including a link to login as the “Tomcat manager”. This page is shown in figure 2

The web interface seen when you navigate to vCenter with a port of 8443

The web interface seen when you navigate to vCenter with a port of 8443

In VMware version 4.1 there is a user named “VMwareAdmin” that is automatically added to the Tomcat server, which has full admin rights to the Tomcat service. In the earlier versions of VMware, the password for this admin account was 5 characters long starting with 3 uppercase, 1 number and one lowercase. This leaves an attacker with a number of options for an attacking perspective. The most obvious is to brute force the credentials with a compatible tools or script such as the Apache tomcat brute force tool (Snipt, 2011). A second (and more sophisticated attack) would be to use the folder traversal vulnerability introduced by the Jetty service to gain read access to the server. From here the attacker could navigate to the “tomcat-users.xml” file (C:\Program Files\VMware\Infrastructure\tomcat\conf) as shown in Figure 3, which is an XML file found in VMware 4.1 and which shows the clear text credentials of the account.

(left) The tomcat-users.xml file showing the username and password of a default admin account (Right) tomcat manager login prompt

(left) The tomcat-users.xml file showing the username and password of a default admin account (Right) tomcat manager login prompt

Using this access, an attacker is able to control elements of the web service with admin rights. As shown in Figure 4, one is able to change a number of settings through the tomcat interface, including the ability to upload custom WAR files, which can be created using Metaspolit to upload meterpreter payloads to the server.

Logged in to the tomcat manager using the credentials found on server

Logged in to the tomcat manager using the credentials found on server

Although some of the attacks using the VASTO toolkit are specific and use vulnerabilities that have almost all been patched by VMware (at the time of writing), the management interfaces are still vulnerable to more general network attacks that are not as fundamental to secure as simply applying a patch or updating to the newest version. As is explained briefly in by post on hypervisors, access to these interfaces are vulnerable to MiTM attacks and the implementations dependence on a highly insecure certificate/PKI model. These vulnerabilities are not directly the responsibility of the vendors, but certainly nothing has been done by them to address this issue.

I will not be explaining the process of how MiTM attacks and flaws in the certificate infrastructure can be used to capture login credentials, as this a fundamental part of security and has been covered on numerous occasions by multiple sources (Irongeek, 2012) (Schneier, 2011). I have also written about the overarching problems with the certificate model and how it can be bypassed by in a blog post from 2011.

 

Mluft, 2011. The Key to your Datacenter. [Online] Available at: http://www.insinuator.net/2011/07/the-key-to-your-datacenter/

Criscione, C., 2010. Blackhat 2010 – Virtually Pwned. USA: Youtube.

Wilkins, G., 2009. Vulnerability in ResourceHandler and DefaultServlet with aliases. [Online] Available at: http://jira.codehaus.org/browse/JETTY-1004

Irongeek, 2012. Using Cain to do a “Man in the Middle” attack by ARP poisoning. [Online] Available at: http://www.irongeek.com/i.php?page=videos/using-cain-to-do-a-man-in-the-middle-attack-by-arp-poisoning

Schneier, B., 2011. Schneier on Security. [Online] Available at: http://www.schneier.com/blog/archives/2011/09/man-in-the-midd_4.html

Advertisements




Attacking the hypervisor

27 11 2013

The hypervisor has the disadvantage of being potentially attacked in one of two ways, from either the network layer or from the host running on that hypervisor. The default behaviour of a hypervisor on a network is to respond to connections through standard TCP/IP, much the same as other desktop machines, devices and infrastructure. This results in the hypervisor being locatable on the network and consequently susceptible to traditional network enumeration attacks such as Nmap (nmap.org, 2012) and Nessus (Tenable, 2012). While enumeration tools are primarily used as a discovery mechanism, they are often able to extract further information about a system by analysing characteristics and information returned by the host. An example of this technique using the currently most utilised enumeration software (Nmap) would be by specifying the ‘–O’ switch, which compares the host’s packet response against a large database of software. Once this extra information about the host has been identified, additional approaches can be used to cross-examine the hosts further to identify attributes such as patch levels and service packs. Depending on the software that is found, using these approaches the attacker is then able to determine the appropriate CVE (Common Vulnerabilities and Exposures) that the host may be vulnerable to. After the vulnerabilities have been identified, the attacker is able to exploit the system using the exploit and insert a payload to further control the host and maintain access. Current examples of software that can be used to exploit systems and insert malicious payloads are Metasploit (Metasploit, 2012) and CORE Impact (Core Security, 2012). This method of enumeration and exploitation will already be familiar to security staff responsible for scanning traditional clients as it is identical.

It is the second method used to attack the hypervisor from the guest or virtual machine that is much more dangerous and an unfamiliar concept, especially for companies invested in the cloud computing or hosting servers in large datacentres.

The term virtual machine (VM) escape is the concept of breaking out of an isolated VM in order to execute malicious code on the host. There have been a number of vulnerabilities on both Type 1 and 2 hypervisors that demonstrate this concept of escape (CVE-2009-1244, CVE-2011-1751, CVE-2012-0217 (Xen, 2012), CVE-2012-3288). While the danger of ‘Type 2’ hypervisor escape is still a threat, the implications of breaking out of a guest, running on an enterprise ‘Type 1’ hypervisor such as ESXi or the Xen hypervisor would be much greater due to the environments that they are often employed in. In traditional networks, security can often be achieved through the segmentation of networks into either physical or virtual networks. This segmentation is still applicable within virtual networks; however this only offers security at the network layer, rather than this new layer of ‘guest–host’ exploitation. While this might sound like an unlikely threat, due to HA features found in VMMM such as VMware’s DRS (Distributed Resource Scheduler) the movement of machines across hypervisors is often determined by the management server rather than by a human. That is unless specified rules are created by the administration. This dynamic movement of virtual machines has the potential to result in an unpatched, publically addressable server being hosted on the same hypervisor/hardware as domain controllers and other high value target machines. This threat is certainly a cause for concern when considering mid to large size networks hosting tens to hundreds of machines within the same infrastructure separated by VLAN’s. The implications and likelihood of this attack is greatly increased when considering multi-tenant public cloud infrastructures. The topic of how hackers could potentially start to rent hosted machines on public clouds to attack other machines will be covered at a later date, but in the authors opinion this could become an actual threat that needs to be considered during a company’s risk analysis process.

There are a number of methods of assessing the security of virtual environments; one of the tools that was recently developed to assist in the evaluation of virtual environments is the VASTO project (Virtualization Assessment Toolkit) (Criscione, et al., 2012). The VASTO project is essentially a collection of Metasploit modules written to query and attack virtual environments, although mainly the VMware platform. The modules are added to the Metasploit project to leveraging an already established and robust framework.

As highlighted earlier, hypervisors are often located on the same subnet as the rest of the servers and, in some cases, the clients. This means that if an attacker is able to gain access to a network that is able to communicate with the hypervisor due to placement or incorrectly configured ACL’s (Access Control Lists), the hypervisor could be attacked directly. Shown in the following example are three simple methods that can be used to locate and query a hypervisor in order to retrieve important information such as version, build number and vulnerabilities that it is susceptible to.

For this demonstration, the author is using a laptop wired into the 192.168.20.0 network in a test environment. Shown in Figure 1, the author uses an NMAP command with the ‘-sV’ switch to scan the entire subnet to return a list of live hosts and associated services. The scan correctly identifies both of the ESXi servers located on the network.

Figure 1- Section of results from an NMAP scan “nmap –sV –T 192.168.20.0/24”

Figure 1- Section of results from an NMAP scan “nmap –sV –T 192.168.20.0/24”

As shown in figure 2 NMAP returns results showing that ESXi is installed on two IP addresses on the 192.168.20.0 subnet and has several open ports. While NMAP does identify the product and version correctly on this occasion, it is not always completely reliable in returning the exact version of the host running on the host. To do this there are a number of methods including VASTO, Nessus or OpenVAS. Using the “vmware_version” module found in VASTO (shown in Figure 2) we are able to detect the exact version of the host including build number.

Figure 2 - Section of results from VASTO vmware_version scan

Figure 2 – Section of results from VASTO vmware_version scan

This now gives the attacker the information needed to locate existing exploits against this version or even develop new exploits, depending on the value of a target. Shown in Figure 4 is a screen shot of a Nessus report generated after a scan against the IP address of the ESXi host. Nessus is an automated scanning and vulnerability assessment tool that fingerprints the host against numerous plugins in order to detect exploits that the host is vulnerable to. While the full report highlighted a number of vulnerabilities found on the host, Figure 3 shows that this particular host is vulnerable to one plugin tested – containing 3 CVE’s (CVE-2012-2448, CVE-2012-2449, CVE-2012-2450).

The number of exploits and risks associated with them is not the area being addressed in this demonstration, but rather the ability to identify the hypervisors and attack it directly. The quantity and complexity of the attacks involved in exploiting type 1 hypervisors is currently much greater than those found on type 2 implementations. As with any technology, as popularity grows and new features are added then the greater the likelihood is that easy to acquire automated attacks will exist.

Figure 3 - Section of Nessus report highlighting highly rates vulnerabilities

Figure 3 – Section of Nessus report highlighting highly rates vulnerabilities

VMware greatly increased the security of their hypervisor through the replacement of their ESX product in favour of adopting the new lightweight (smaller code footprint) ESXi, which did not include their service console within the architecture of the code (VMware, 2012). However there are still elements within the hypervisor that continue to threaten its security. One of these is the notion that ESXi is by default configured to be accessible through a browser. Clients with Port 80 and 443 access to the hosts are able to directly access the hypervisor through a browser and even use the host to download the vSphere management client. While this may be convenient, it is the author’s opinion that this ‘out of the box’ configuration lacks the fundamental security posture that should be taken against such a high value target. An attacker with the vSphere client is able to directly manage the hypervisor once a username and password have been provided. It should also be noted that all ESXi servers (by default) are configured using the ‘root’ account, meaning that the only unknown credential required to manage the host is the root password and it would be possible to ‘brute-force’ this. Furthermore there is a customised brute forcing tool in the VASTO suite called “vmware_login”, which allows automatic dictionary or ‘brute-force’ login attempts.  In addition to all of these vectors, there are also the pertinent issues of existing network security issues such as MITM’s (Man-in-the-middle attack), which could expose these credentials.

To demonstrate the prevalence of exposed hypervisors, using the online search tool ‘Shodan’ (SHODAN, 2012) the author is able to search the internet for exposed ‘ESX’ hosts. In figure 4 we are able to see that Shodan has returned 749 results fitting that description.

Figure 4 - Results of a Shodan search for host containing the term "esx"

Figure 4 – Results of a Shodan search for host containing the term “esx”

While not all of the hosts returned by Shodan are active, a large number of them are still current and allow remote connections to be made over the internet. Shown in figure 5 is a valid connection to one of the returned addresses through a web browser showing an ESXi 5 host.

Figure 5 - Connection to the IP address of one of the hosts found by Shodan

Figure 5 – Connection to the IP address of one of the hosts found by Shodan





Introduction to the hypervisor

22 11 2013

The hypervisor is arguably one of the more misunderstood concepts of virtualization for technical professionals who are more familiar with traditional methods of computing, as it can often be viewed as simply another operating system. While the hypervisor may be a form of operating system, the implications surrounding the impact of a successful exploitation against the system cannot be likened to that of a traditional network operating system. The most obvious element that distinguishes a hypervisor from a traditional operating system is the far reaching implications that vulnerabilities in the hypervisor could have upon the entire system. There are numerous implementations of vendor hypervisors which all have differing levels of vulnerabilities associated. In this post a definition of typical hypervisor implementations is given to establish a baseline of understanding before continuing into the attacks.

In its simplest form, a hypervisor is a piece of code that controls the flow of instructions between guest operating systems and the physical hardware. The hypervisor emulates the physical characteristics of the actual machine such as the processor, RAM, network cards, etc. and presents a homogeneous environment to all the guests.  There are two types of hypervisors – ‘Native’ (also known as ‘Bare Metal’ or Type 1) and ‘Hosted’ (also known as Type 2).

Native hypervisors are installed directly onto the hardware, as would be done with any traditional operating system. There are also implementations of hypervisor that come preinstalled on the host ROM. Native hypervisors benefit from having direct access to the underlying physical hardware of the host, resulting in improved performance. As these systems are not full operating systems they also have the benefit of having a smaller attack surface and are therefore considered more secure.

Hosted hypervisors are installed onto the existing operating system, eg Windows or Linux, which is responsible for communication between the hardware and the hypervisor. This type of hypervisor is less efficient in terms of performance and security and is typically used on desktops rather than servers.

While most common ‘Type 1’ hypervisors are a fraction of the size of a typical desktop operating system – such as Windows 7, the code is still an additional layer of software that is added to the total attack surface of the machine. This underlying dependence, which all hosted machines have on the hypervisor, is one of the most contested factors around virtualisation security ie the ability to compromise the hypervisor and use it to ‘escape’  to other machines hosted on that software.

The security of a Hypervisor is comparable to that of a standard operating system, when considering the size and surface attack area. Systems such as OpenBSD and TrustedBSD allow a greater level of customisation and ability to greatly reduce the features available for a particular task. This lower default functionality offered by systems often has a direct correlation to its security. Hypervisors have typically based their security and efficiency on the amount of Source lines of code (SLOC) used. The core functionally of a hypervisor is to translate and schedule the flow of instructions from the guest to the hardware, anything additional to this could be described as a non-essential feature.

One example of a security focused hypervisor is IBM’s ‘sHype’ implementation of the Xen hypervisor. The total code of this project is claimed to be around 2600 lines in length. It is reported that ESXi and KVM hypervisors were around 200,000 SLOC in 2010 (Steinberg & Kauer, 2010, p. 3), with indications that this could rise and therefore further increasing the attack surface.