Attacking additional virtualisation infrastructure

10 02 2015

I should start by stating that in almost all documents regarding live migration, High Availability (HA) and Fault tolerance configurations, it is stated that these networks should be placed on isolated networks that are not shared or accessible by unauthorised individuals. In an article by (Siebert, 2011) it specifically states to:

“Keep management and storage traffic, for instance, on a physically isolated network that’s away from the regular VM network traffic”

However, this does not diminish the argument that this information is still being moved inside the environment and should these guidelines be overlooked or the networks breached, the consequences of what can be achieved can be unintelligible to staff only familiar with traditional methods.

Live migration traffic

Live migration can be used to move machines between hosts that are located in the same blade chassis, across campuses and even over different continents, with minimal downtime (Travostino, et al., 2006). The action of preforming a live migration from one host to another is a common feature found in many of the larger VMMM such as XEN and VMware. The act of moving a machine across hosts in the VMware environment can be a manual or automated process when combined with other features such as DRS and DPM (Distributed Power Management). While explanations of the DRS feature is discussed in numerous earlier posts, DPM enables vCenter to automate the movement of machines off hosts during quiet periods of usage, allowing them to be powered down thus improving power consumption. It is my personal option that live migration is one of the more intriguing introductions of virtualisation, as the information that is located in RAM contains such important information.

Using the test environment I will demonstrate what is possible when access to the vMotion network is obtained. For this example a vMotion has been initiated specifying that VM1 on ESX1 is to be migrated – and only the host portion is required to move and not the datastore. The VM will be moved to the only other host in the cluster, ‘ESX2’. To understand how this process is vulnerable to attacks I will first breakdown the processes of the steps involved in this operation (Kutz, 2007):

  1. The request is made specifying the VM and which host it will be moved to.
  2. All of the RAM of VM1 is copied over the vMotion network to ESX2, any active changes happening during this time on ESX1 are written to a memory bitmap on ESX1.
  3. VM1 is quiesced on ESX1 and the memory contained in the bitmap is copied over the vMotion network to ESX2.
  4. VM1 is started on ESX2 and access to the VM are sent to the instance on ESX2.
  5. Remaining memory from VM1 is copied from ESX1, while memory is being read and written from VM1 on ESX1.
  6. Once successful, VM1 is unregistered on ESX1 and the task is complete.

In figure 1 we see a basic visual representation of the process in the test environment. For demonstration purposes and ease of result display, I have a separate vMotion network that uses a hub to connect the two machines. This will allow me to sniff traffic without having to perform an additional attack such as MiTM or configure port mirroring, port spanning, SPAN, RSPAN etc.

Basic visual representation of the process of a machine being migrated to another host in the test environment

Basic visual representation of the process of a machine being migrated to another host in the test environment

In the virtual machine that will be moved, I have written information into a text file but not saved it to disk, so that we can be sure it will be located in the machines memory. This information in the example is a representation of a secure document containing a username and password. This text file and content are shown in Figure 2.

An unsaved text file that has been written on the running virtual machine

An unsaved text file that has been written on the running virtual machine

I now initiate a vMotion request and starts a packet capture on the vMotion network using Wireshark. The network card is configured in promiscuous mode, to allow it to capture all of the packets being transmitted on the network. After the vMotion process has finished and the capture stopped, I am able to follow the TCP stream of the communication that took place between the two hosts and view the information in a searchable form. As is shown in Figure 3, the information transmitted in the document is viewable in clear text, although separated by some punctuation

A TPC stream of communication between the two hosts showing the notepad content

A TPC stream of communication between the two hosts showing the notepad content

Although this example used a notepad document to demonstrate the ability to sniff data, there are much greater risks to consider when addressing passive snooping attack on migration traffic. Microsoft stores LM authentication hashes for active sessions in memory (Pilkington, 2012) in all current versions of Windows. As a result, should a passive snooping attack take place while a machine is locked or logged on, the LM hash will also be viewable in the capture. Windows LM hashes are also reversible back into the user’s original password using online services such as (OS – Objectif Securite, 2012) and other hash cracking tools. Finally, and possibly of most concern, is the accessibility of encryption keys. Even when full disk encryption is used to secure data, the keys (after initial user input) are cached in memory by the operating system. On traditional hardware this is considered the safest place, as volatile memory is erased quickly once the power is taken away. This is something that usually requires physical access to the machine to exploit, unless the machine is already infected. It is now possible to sniff the encryption key during a live migration to obtain the encryption hash.

Live migration attacks are not only vulnerable to sniffing attacks. In a paper written in 2007 by (Oberheide, et al., 2007), the researchers describes a number of attacks that are possible on live migration traffic. In the paper the researchers discus three ‘classes’ of threats that can be used against live migration environments:

  • Control plane
  • Data Plane
  • Migration Module

Control plane attacks

Control plane attacks target the mechanisms that are employed to initiate and manage the migrations within the infrastructure manager. While there were no active attacks in the paper, the theory behind the attacks is still relevant and should be considered in a risk analysis exercise. The examples in the paper that (Oberheide, et al., 2007) highlights, demonstrate how successful attacks to the control plane could result in:

  • The migration of machines onto illegitimate hosts that are owned by the attacker.
  • Mass migration of a large number of machines, thus overloading the network and causing disruption to service.
  • Manipulating the resources management for hosts in a DRS style environment, so that hosts are not evenly distributed and overwhelm single resources.

Data plane attacks

Data plane attacks are threats that take place on the networks on which the migrations are situated. The passive snooping attack that was demonstrated previously is one of the attacks that are briefly mentioned in the paper. The second attack on the data plane class is described as ‘active manipulation”. As the name suggests, active manipulation is when data is changed during the migration of the machine. Although in the paper (Oberheide, et al., 2007) introduce their custom tool Xensploit for preforming this attack, it is merely the collection of existing attacks collated into one tool for ease of use. The attack works by using MiTM characteristics to intercept traffic between the two hosts and manipulating sections of the traffic (RAM) during transit. In the example of the Xensploit software in the paper (Oberheide, et al., 2007, p. 4) showed how the attack could be used to establish an SSH (Secure Shell) session to a machine configured to only allow connections from authorized sources. Using Xensploit they were able to manipulate within the object code of the SSHD process to add their key as an authorised source, thus allowing them to SSH once the new instance had completed its migration.

Migration Model attacks

Lastly, although these attacks are included in the paper, I feels that this section falls under one of my other previous topics of the hypervisor rather than on live migration. The paper briefly covers attacks that exploit vulnerabilities in the migration models – that are part of the VMM (virtual machine monitor).

Combining attacks

Gaining access to any of these attack classes should be considered a high risk to security staff, although with access to just one of the vectors, it may still not be possible for an attacker to target a specific machine depending on its physical location. If a combination of the attacks were available, the attacker would be able to leverage them to better achieve control of the environment. An example of this is in the case of data plane class attacks. These attacks are only useful if the target machine is being migrated during the period of capture. In environments where features such as DRS and DPM are not in use, it is possible for machines to stay fixed to a host for a considerable period of time. If the attacker was able to utilize an attack at the control plane they would be able to trigger the migration of the necessary machines for the attack to then take place at the data plane.

Shortly after the paper was released, VMware’s (Wu, 2008) comments on this attack:

“Although impressive, this work by no means represents any new security risk in the datacentre… Rather, it a reminder of how an already-compromised network, if left unchecked, could be used to stage additional severe attacks in any environment, virtual or physical.”

While I can appreciate what (Wu, 2008) is saying, I must disagree that these types of attacks are comparable to physical environments. Data-In-Transit on physical systems do not tend to include such sensitive information. It is also better understood in traditional systems that unsafe protocols such as email and ftp should not be used to transfer confidential information. While the previous example required physical access to the vMotion network, it is also possible to access this data remotely through misconfiguration, access to the management interface or manipulation of virtual infrastructure.

Wu, W., 2008. VMware Security & Compliance Blog. [Online] Available at: http://blogs.vmware.com/security/2008/02/keeping-your-vm.html

Siebert, E., 2011. Five VMware security breaches that should never happen. [Online] Available at: http://searchvmware.techtarget.com/tip/Five-VMware-security-breaches-that-should-never-happen

Travostino, F., Daspit, P. & Gommans, L., 2006. Seamless live migration of virtual machines over the MAN/WAN. Future Generation Computer Systems – IGrid 2005: The global lambda integrated facility, 22(8), pp. 901-907.

Kutz, A., 2007. How to obtain, configure and use VMotion and how VMotion works. [Online]
Available at: http://searchvmware.techtarget.com/tip/How-to-obtain-configure-and-use-VMotion-and-how-VMotion-works
Pilkington, M., 2012. Protecting Privileged Domain Accounts: LM Hashes — The Good, the Bad, and the Ugly. [Online]
Available at: http://computer-forensics.sans.org/blog/2012/02/29/protecting-privileged-domain-accounts-lm-hashes-the-good-the-bad-and-the-ugly

Oberheide, J., Cooke, E. & Jahanian, F., 2007. Empirical Exploitation of Live Virtual Machine Migratio. [Online]
Available at: http://www.eecs.umich.edu/techreports/cse/2007/CSE-TR-539-07.pdf

Advertisements




How Cloud Computing could improve security within small to medium businesses

21 06 2011

Small and medium businesses have historically struggled with implementing any real security strategy due to either a lack of knowledge or budget allowances. This often results in smaller organisations being left unpatched or open to attacks that would almost certainly be avoided in larger and more security conscious organisations. In a White Paper presented by Symantec in 2009, it was identified that:

“while small and midsize businesses are acutely aware of today’s security risks, a large number have yet to take even the basic steps needed to protect themselves. Further, the study shows that simple protection measures could have prevented many of the security breaches reported by these companies.”

It is often not financially viable for an organisation of around 10 to 50 employees to implement any kind of ‘defense in depth’ model, as outlined by Paloma, J (2007). This is where cloud computing could be used to alleviate the security burden off these individual companies and move it up into the cloud. As stated by Catteddu, D and Hogben, G, (2009:17)

“Put simply, all kinds of security measures are cheaper when implemented on a larger scale. Therefore the same amount of investment in security buys better protection. This includes all kinds of defensive measures such as filtering, patch management, hardening of virtual machine instances and hypervisors, etc”

Large public cloud vendors know that security (among other things) is a major reason that organisations are so hesitant about moving their data and infrastructure to the cloud. This means that cloud vendors are doing everything they can to secure these environments. The adoption of cloud computing will undoubtedly bring new challenges and areas of exploitation which have not been seen before. An example of this is ‘cloud-bursting’ which is a concept demonstrated by Kostya Kortchinsky at Blackhat in 2009. This allows code that is run from within a virtual machine to be executed on the hosting hardware and inevitably affect other machines residing on that hardware. This type of exploit would most likely be used against organisations using IaaS (Infrastructure as a Service).

Although Amazon’s EC2 service, which is a service that offers scalable computing resources in the cloud, makes use of a fully customized Xen hypervisor (2010:14), advanced firewalling capabilities and claims to provide complete separation between virtual instances running on their system. I personally feel that it is still too early on in the technology life to have proven itself as a completely ‘bulletproof’ approach yet. It is this kind of highly funded, targeted attacks that are the exact reason why a large multimillion pound conglomerate should avoid sharing hardware with others on a public cloud infrastructure in the near future.

These highly directed approaches however are far beyond the level of attacks routinely leveled at a SMB, which operate with no security practices at all. For most SMB’s, cloud computing can offer a multitude of security benefits including improved authentication and logging. One example of this increased security is with Amazon’s AWS Multi-Factor Authentication, which was reported on by Cardinal, C (2010). This multi-factor authentication, which works by requiring users to provide both a password and a number that is randomly generated by hardware token, has been notoriously hard to implement by even dedicated IT departments.

Another benefit of cloud computing is with compliance and standards. Certainly in America, compliance standards such as PCI (Payment Card Industry Data Security Standard) and HIPAA (Health Insurance Portability and Accountability Act) are now active. Here vendors can expect to see the emergence of a ‘snap-in’ security mentality in which smaller, less equipped companies are able to offload their storage and processing transactions to PCI & HIPPA approved cloud providers. This greatly reduces both time and cost.

References:

Symantec 2009, Global Study Identifies SMB ‘Security Gap’ , Symantec, accessed 16 December 2010, <http://www.symantec.com/business/solutions/article.jsp?aid=20090428_global_study_identifies_smb_security_gap>

Paloma, Jay,. (2007). Windows Server 2008 in an Organisation’s Defense in Depth Strategy. [Online]. December 2007. Available from: <http://technet.microsoft.com/en-us/library/cc512681.aspx. [Accessed: 15th December 20010]

Amazon.com Inc (2010) Amazon Web Services: Overview of security, Seattle: Amazon

Amazon.com Inc (2010) Amazon Elastic Compute Cloud (Amazon EC2), accessed 26 December 2010, <http://aws.amazon.com/ec2/>