Insecure Until Proven Updated:Analyzing AMD SEV’s Remote A�estation

Robert Buhrenrobert.buhren@sect.tu-berlin.de

Technische Universitt BerlinSecurity in Telecommunications

Christian Werlingchristian.werling@student.hpi.deHasso Pla�ner Institute, Potsdam

Jean-Pierre Seifertjpseifert@sect.tu-berlin.de

Technische Universitt BerlinSecurity in Telecommunications

ABSTRACTCloud computing is one of the most prominent technologies tohost Internet services that unfortunately leads to an increasedrisk of data the�. Customers of cloud services have to trust thecloud providers, as they control the building blocks that form thecloud. �is includes the hypervisor enabling the sharing of a sin-gle hardware platform among multiple tenants. Executing in ahigher-privileged CPU mode, the hypervisor has direct access tothe memory of virtual machines. While data at rest can be pro-tected using well-known disk encryption methods, data residing inmain memory is still threatened by a potentially malicious cloudprovider.

AMD Secure Encrypted Virtualization (SEV) claims a new levelof protection in such cloud scenarios. AMD SEV encrypts the mainmemory of virtual machines with VM-speci�c keys, thereby deny-ing the higher-privileged hypervisor access to a guest’s memory.To enable the cloud customer to verify the correct deployment ofhis virtual machine, SEV additionally introduces a remote a�es-tation protocol. �is protocol is a crucial component of the SEVtechnology that can prove that SEV protection is in place and thatthe virtual machine was not subject to manipulation.

�is paper analyzes the �rmware components that implementthe SEV remote a�estation protocol on the current AMD EpycNaples CPU series. We demonstrate that it is possible to extractcritical CPU-speci�c keys that are fundamental for the security ofthe remote a�estation protocol.

Building on the extracted keys, we propose a�acks that allowa malicious cloud provider a complete circumvention of the SEVprotection mechanisms. Although the underlying �rmware issueswere already �xed by AMD, we show that the current series ofAMD Epyc CPUs, i.e., the Naples series, does not prevent the in-stallation of previous �rmware versions. We show that the severityof our proposed a�acks is very high as no purely so�ware-basedmitigations are possible. �is e�ectively renders the SEV technol-ogy on current AMD Epyc CPUs useless when confronted with anuntrusted cloud provider.

To overcome these issues, we also propose robust changes to theSEV design that allow future generations of the SEV technology tomitigate the proposed a�acks.

KEYWORDSvirtualization, Secure Encrypted Virtualization, cloud computing,shielding systems, SEV, remote a�estation

1 INTRODUCTIONCloud computing is one of the core foundations of today’s Internetlandscape. �e manifold advantages such as on-demand resourceallocation or high availability of services have lead to a wide usageof this technology. However, outsourcing the processing of enter-prise data comes at a risk. �e technical infrastructure that formsthe cloud is owned by the cloud provider and thus under his fullcontrol. �is includes the server hardware, as well as the so�warecomponents that allow the co-location of multiple virtual machineson a single host.

�erefore security concerns impede the deployment of con�den-tial data and applications in cloud scenarios [14, 19]. �e potentialthreats range from miscon�guration of so�ware components overcloud provider admin access to foreign government access [8].

To counter these threats, the research community, as well as in-dustry, proposed new approaches to allow secure cloud computingwhen confronted with an untrusted cloud provider [18, 23, 29, 33,34, 36]. Most prominent is the recently released Secure EncryptedVirtualization technology by AMD [6]. SEV’s goals are twofold:

(a) Prove the correct deployment of virtual machines.(b) O�er virtual machine protection at runtime.

To achieve (a), a dedicated co-processor, the Platform SecurityProcessor (PSP), creates a cryptographic hash of the componentsthat form the initial virtual machine state, similar to the remotea�estation feature of a TPM.

“With this a�estation, a guest owner can ensurethat the hypervisor did not interfere with the ini-tialization of SEV before transmi�ing con�dentialinformation to the guest.” [6]

(b) is achieved by using memory encryption with virtual machine-speci�c encryption keys. �ese keys are generated upon creationof a virtual machine and are stored inside the PSP that uses itsown private memory. �is memory is not accessible from the mainprocessor, which is executing so�ware components controlled bythe cloud provider such as the hypervisor. A hardware memoryencryption unit provides transparent encryption and decryptionusing the virtual machine-speci�c key when the respective virtualmachine is scheduled.

“Even though the hypervisor level is tradition-ally ’more privileged’ than the guest level, SEVseparates these levels through cryptographic iso-lation.” [6]

Furthermore, to allow the authentication of an SEV platform,each SEV-enabled platform contains a key pair that is unique to thisplatform. �e public key is signed by AMD and, given a platform-speci�c ID, a guest owner can obtain this key from an AMD key

arX

iv:1

908.

1168

0v2

[cs

.CR

] 2

Sep

201

9

server. �is key is the basis for a remote a�estation protocol thatenables the user to verify the correct deployment of her VM, in-cluding that SEV protection is in place. Only then she will inject aguest secret, e.g., a disk encryption key, into the guest VM using asecure channel between the PSP and the guest owner. �is remotea�estation feature is a key component when using the infrastruc-ture of an untrusted cloud provider. It provides cryptographic proofthat the cloud provider uses an authentic AMD platform with SEVenabled and deployed the virtual machine according to the owner’scon�guration.

Previous research focused on the security of SEV-protected vir-tual machines at runtime [11, 15, 25] while the remote a�estationfeature of SEV has not been subject to a comprehensive analysisyet. To overcome this gap, we analyzed the �rmware componentsthat implement the remote a�estation feature1. We demonstratethat it is possible to extract the platform-speci�c private key, uponwhich the security of the remote a�estation protocol depends. �isenables the untrusted cloud provider to violate the security goalsof the SEV technology. Speci�cally, it enables the cloud provider toforge the presence of SEV altogether, intercept the communicationbetween guest owner and SEV �rmware and decrypt guest memory.

We show that it is, to the best of our knowledge, impossible toprovide a purely so�ware-based �x for these issues. �is questionsthe security capabilities of the SEV feature for the complete AMDEpyc Naples processor line. More so, it actively puts SEV customers’data at risk since SEV might a�ract con�dential data that waspreviously not hosted in the cloud.

Furthermore, we propose a new design for the remote a�estationprotocol. While the current SEV design cannot cope with �rmwaresecurity issues, the proposed design allows to revert to a trustedstate in case of previous �rmware issues. Our proposed designenables SEV customers to trust the remote platform even in casethe platform-speci�c private key was leaked.

Our contributions are:

(1) We conduct a comprehensive security analysis of the �rmwarecomponents that implement SEV’s remote a�estation pro-tocol. Our analysis reveals issues that allow to extractprivate keys that are fundamental to the security of theSEV technology. More so, the current AMD Epyc CPUsallow to install arbitrary signed �rmware versions, henceextraction is possible even on systems that use patched�rmware versions.

(2) We propose a�acks based on our �ndings that allow tofake the presence of SEV or extract encrypted VM memoryin plaintext.

(3) We show a severe design issue of the protocol, renderingit useless in case of common �rmware issues. �e severityis ampli�ed as the issue allows to mount a�acks targetingplatforms with no security issues present: Possession of anextracted key is su�cient to mount the a�acks, regardless ofwhether the key belongs to the a�acked platform or not.

1To facilitate further research we published a PSP �rmware analysis tool. �e tool canbe found at [32].

(4) Lastly, we propose robust design changes to the SEV tech-nology that allow future generations of SEV to mitigatethe impact of �rmware security issues.

Ethical considerations: We informed AMD of the �rmware is-sues found during our analysis. AMD con�rmed the presence ofthe �rmware issues and published security updates to the PSP�rmware. Similar issues had been previously reported by othersecurity researchers [21]. Furthermore, we reported the extractedkeys to AMD to allow a revocation of the corresponding certi�cates.Although security �xes are already provided by AMD, our �ndingsshow that so�ware-based mitigations are not su�cient.

We therefore refrain from publishing any speci�cdetails on the �rmware issues at this point in time.To prove the successful extraction of private keys,we provide a signature of the paper title createdwith the extracted keys at [9]. �e signature canbe veri�ed using certi�cates provided by AMD.

�e rest of the paper is structured as follows: In Section 2 wegive an overview of the SEV technology, its remote a�estation capa-bilities and the PSP. Section 3 presents the results of our �rmwareanalysis necessary to understand the a�acks. �e a�acks are moti-vated in Section 4 and described in Section 5. Section 6 discussesthe implications on the security of SEV and proposes a new designfor the remote a�estation functionality. In Section 7 we give anoverview of related work and �nally conclude in Section 8.

2 BACKGROUNDIn this section, we give an overview of basic x86 virtualizationconcepts and more speci�cally on the Secure Encrypted Virtualiza-tion technology by AMD including the Platform Security Processor(PSP). We focus on the remote a�estation feature as this is our mainsubject of analysis for the rest of the paper.

2.1 x86 Virtualization ConceptsHardware extensions to facilitate the use of virtual machines wereintroduced in 2005 by both Intel (VT-x) [30] and AMD (SVM) [1].�ese extensions distinguish between the higher-privileged hostmode and the lower-privileged guest mode. Both modes are com-prised of di�erent privilege rings, which allow separating eachmode in privileged and unprivileged execution compartments. �ehost mode controls the resources, such as memory and CPU time,of the guest mode.

A so�ware component called hypervisor executes in the hostmode, whereas the guest mode is occupied by the guest virtualmachines (VMs). In a cloud scenario, the hypervisor is suppliedby the cloud provider and therefore under full control of the cloudprovider.

Running in the higher privileged host mode, the hypervisor hasfull access to the guest VM’s memory content.

2.2 Secure Encrypted Virtualization�e Secure Encrypted Virtualization technology (SEV) was �rst pre-sented by AMD in 2016 [6]. It aims to protect a machine in thepresence of an untrusted cloud provider. �e primary responsi-bilities of SEV are runtime protection and secure initialization ofvirtual machines.

2

�roughout the rest of the document, the term platform ownerrefers to any entity that owns the SEV platform, i.e., the AMDsystem that hosts the virtual machines. �e term guest owner refersto any entity that intends to utilize the platform provided by theplatform owner. In a typical use case, the platform owner is thecloud computing provider, and the guest owner is a customer ofthe provider.

�e term SEV �rmware refers to the code running on the SEVplatform implementing the SEV API [5]. It is provided by AMDand hosted by the Platform Security Processor (PSP), in contrast tothe main x86 cores (referred to as main processor in this document),that are under full control of the platform owner.

Runtime Protection �e core of SEV’s runtime protection is amemory encryption engine embedded in the memory controllerthat encrypts the main memory using AES-128 [6]2. �e encryp-tion keys are generated by a �rmware running on a dedicatedprocessor called the Platform Security Processor (PSP). It providesan API [3] that the hypervisor, running on the main processor, mustuse in order to manage the encryption keys for SEV-enabled guests.An SEV-enabled guest controls which memory pages are passedthrough the encryption engine by its guest pagetable.

Remote A�estation Even with runtime protection in place, SEVwould be useless in the face of an untrusted cloud provider, if theplatform owner could modify the VM during deployment or fakethe presence of SEV altogether. �rough remote a�estation, theSEV �rmware implicitly provides proof of authenticity of the SEVplatform to the guest owner and explicitly a�ests a VM’s integrityto her. �is is explained in Section 2.4 and 2.5.

Platform Security Processor Introduced in 2013 [2], the Plat-form Security Processor (PSP) is a dedicated ARMv7 based co-processor built into the die of AMD CPUs providing security func-tionality. It uses its own memory and non-volatile storage and canaccess the main processor’s system memory. �e �rmware runningon the PSP is provided by AMD and integrity protected.

It also hosts a dedicated SEV �rmware [5] that implements theSEV API speci�ed in [3].

2.3 SEV: Cryptographic KeysSEV o�ers cryptographic proof that a) the remote platform is anauthentic AMD platform which supports SEV and b) that a guestwas deployed with SEV protection in place. To that end, the SEV�rmware manages several cryptographic keys that are explainedin this section.

Firmware Identity Upon initialization, the SEV �rmware run-ning on the PSP generates an ECDSA key, the Platform EndorsementKey (PEK), using a secure entropy source, see Figure 1. �e SEV�rmware uses the PEK to sign the Platform Di�e-Hellman key(PDH), which is used to negotiate a shared secret with a remoteparty, e.g., to establish the secure channel between the guest owner

2To protect the guest register state, AMD proposed an extension to SEV: SEV-EncryptedState (SEV-ES) [20].

ARKAMD Root Signing Key

ASKAMD SEV Signing Key

CEKChip Endorsement Key

PEKPlatform Endorsement Key

OCAOwner Certificate Authority

PDHPlatform Diffie-Hellman Key

derived from

signed by

Chip/Product line

Key lifetime:

Platform Ownership

Comm. session

Master Secret

KEK/KIKKey Encryption/Integrity Key

TEK/TIKTransport Encryption/Integrity Key

AMD Key Server

SEV Firmware

SEV Firmware & SEV Client

wrap

Figure 1: Cryptographic keys in SEV. A shield denotes thekey as the root of trust for the corresponding certi�catechain. Boxes show the scope of the respective keys.

and the SEV platform[3, Chapter 1.2.2].

Platform Ownership Ownership information is provided bysigning the PEK with a certi�cate authority (CA) of the cloudprovider, the owner certi�cate authority (OCA), see Figure 1. �eSEV �rmware allows generating a certi�cate signing request (CSR)of the PEK. �is allows the cloud provider to sign the PEK. �esigned PEK certi�cate is then re-imported into the SEV �rmware [3,Chapter 1.2.4].

Platform Authenticity To provide the guest owner with an au-thenticity guarantee of the platform, the PEK is also signed by theChip Endorsement Key (CEK), see Figure 1. �e CEK is an ECDSAkey which is derived from CPU-speci�c secrets stored in one-time-programmable fuses (OTP fuse) in the CPU [3, Chapter 1.2.3]. Toprove the authenticity of the CEK, it is signed by the AMD SEVSigning Key (ASK) which is in turn signed by the AMD Root Key(ARK). As the CEK is unique for each platform, the SEV API spec-i�es a command to retrieve a unique identi�er tied to a platform.While the CEK private key must remain con�dential, the signedcerti�cates of the CEK, ASK and the ARK can be obtained fromAMD [4] using the platform ID provided by the SEV �rmware. �eCEK, therefore, plays a central role in the trust model of SEV.

3

Con�dential Communication Since SEV assumes an untrustedhypervisor, con�dential communication must be ensured for twocases: First, during the initial deployment of a VM, between theguest owner and the SEV �rmware of the target platform. Second,during migration, between the SEV �rmwares of the source andtarget platform.

To this end, a client and the SEV �rmware use the Di�e-Hellmanprotocol to establish a shared secret, the master secret. A client inthis context is either a guest owner or another SEV �rmware incase of migration. Using a key derivation function (KDF) and theestablished master secret, both the client and the SEV �rmware de-rive the Key Encryption Key (KEK) and the Key Integrity Key (KIK).�ese keys are used to protect the transport keys, the so-calledTransport Encryption Key (TEK) and Transport Integrity Key (TIK).�e transport keys are encrypted using the KEK, and a MAC is gen-erated using the KIK. �is process is referred to as key wrapping [3,Chapter 2.1]. Note that the transport keys are chosen by the client,whereas KIK and KEK are derived from the master secret.

�e transport keys are then used to ensure the integrity andcon�dentiality of data exchanged between the SEV �rmware andoutside entities. �e following section explains the protocol usedto establish the secure channel.

2.4 SEV: Establish Secure ChannelIn order to establish the secure channel, both client and SEV �rmwarefollow the steps depicted in Figure 2. As the SEV API is only ac-cessible via the hypervisor, the secure channel must guaranteeauthenticity, integrity, and con�dentiality of the communication.In case of deployment, the client is the guest owner, see variant D.In case of migration, the client is the source platform and the SEV�rmware is the target platform of the migration, variant M.

In the �rst step, the hypervisor retrieves the PDH and the PEKcerti�cate together with a unique platform ID of the target platform.Using this ID, either the hypervisor, in case of migration, or theguest owner, during deployment, consult the AMD key server toobtain the CEKID certi�cate along with the ARK and ASK. �eclient is now able to authenticate the target platform by verifyingthe certi�cate chain3 as shown in Step 5. In that case, the clientwould also verify that the PEK is signed by the OCA in Step 5). InStep 6 the authenticated PDH is then used to negotiate the mastersecret using a Di�e-Hellman key exchange [3, Chapter 2.2.2]. �emaster secret is only known by the client and the target, but not bythe hypervisor. Using the master secret the key encryption keys arederived, see Section 2.3. Finally, the client generates the transportencryption keys and wraps them using KIK and KEK, Step 7. �ewrapped keys and the Di�e-Hellman share are then transferred tothe target.

Both client and the target SEV �rmware now hold the transportkeys that allow authenticated, encrypted, and integrity protectedcommunication.

3To allow the client to verify the ownership of the platform, the PEK can also be signedwith the OCA.

4. forward ARK, ASK, CEKID, PDH, PEK

2. forward PDH, PEK, platform ID

3. platform ID

4. ARK, ASK, CEKID

5. verify PDH → PEK → CEKID → ASK → ARK

6. negotiate master secret

7. generate transport keys

8. wrapped transport keys

& DH share

1. PDH, PEK, platform ID

9. forward wrapped transport keys

& DH share

2. platform ID (target)

3. ARK, ASK, CEKID

Migrate

AMD Key Server D: SEV FirmwareM: Target

D: Guest OwnerM: Source Hypervisor

Deploy

Figure 2: SEV Secure Channel. Protocol initiated by theguest owner, in case of deployment (D) and the hypervisor,in case of migration (M).

2.5 SEV: Guest DeploymentWhile Section 2.4 explained the establishment of the secure channel,this section gives an overview of the steps required to deploy aguest VM in an SEV-enabled cloud system.

Prior to the guest deployment, the platform owner has to ini-tialize the SEV platform. During initialization, the SEV �rmwarederives the platform-speci�c keys described in Section 2.3. Further-more, the �rmware establishes a chain of trust by signing the PDHwith the PEK and the PEK with the CEK:

PDH → PEK → CEK → ASK → ARK (1)

�is is also depicted in Figure 1. Optionally, a second certi�catechain is established:

PDH → PEK → OCA (2)

�ese chains allow a client to authenticate a given PDH4. A�erthese steps, the SEV �rmware transitions into the initialized state.

Before any guest VM can be deployed on the platform, the guestowner authenticates the remote SEV platform. Using the stepsshown in Figure 2, she can establish a secure channel with theremote SEV platform. �e veri�cation of the certi�cate chain, seeStep 5, ensures that the remote system is an authentic AMD system

4In case the platform was already initialized, the encrypted and integrity protectedstate is read from non-volatile storage.

4

that supports SEV.

1. deploy

Guest Owner

SEV Firmware

Platform

Guest VM

Disk ImageGuest VM

Disk Image

3. measure

 5. send  disk key 

6. forwarddisk key

trusted byguest owner

untrusted byguest owner

2. encryptmemory

Hypervisor

2. encrypt memory

 4. send  hash 

1. deploy

Figure 3: Initial deployment of a guest virtual machine inan SEV scenario.

Launching the Guest �e guest owner now prepares the guestVM to be executed by the cloud provider. �e initial guest VM is sentto the hypervisor unencrypted and therefore must not contain anycon�dential data. To ensure the con�dentiality of the guest owner’sdata, the initial guest image will usually contain an additionalencrypted disk image. In this case, the encryption key is thenlater provided using the established secure channel. Besides theVM image itself, the guest owner must also provide a policy thatde�nes restrictions on the actions the cloud provider can performon the guest VM. �ese include, e.g., the cloud provider’s abilityto migrate the guest VM to another platform or the minimum SEVAPI version that the target SEV �rmware must implement. As thememory encryption of SEV prevents traditional VM migration, theSEV �rmware provides an interface to migrate VMs to a di�erenthost, as outlined in the following section.

�e guest owner deploys the VM, including the encrypted diskimage, to the cloud provider, see Step 1 of Figure 3. �e hypervi-sor launches the guest and calls the SEV �rmware to encrypt thememory, Step 2. Next, the SEV �rmware calculates a hash of theinitial plaintext VM memory. �e hash, together with the SEV APIversion and the guest policy, is protected by the secure channel andtransferred to the guest owner, see Step 3 and 4. �e hash givesthe guest owner con�dence that the VM was deployed unmodi�edby the hypervisor. Lastly, in Steps 5 and 6, the guest owner usesthe secure channel to provide the disk encryption key to the VM.�is allows the VM to decrypt the disk and process the con�dentialdata. �e guest VM is now fully operational and protected by SEV.

2.6 SEV: Migration and SnapshotsVirtual machine migration and snapsho�ing are common tasks ina cloud computing environment. Both migration and snapsho�ingrequire the export of virtual machine memory. In case of migration,the memory is then exported to a di�erent platform, while forsnapsho�ing, the memory is saved on the same platform for a laterre-import.

As SEV encrypts virtual machine memory using ephemeral keyswhich never leave the SEV �rmware, SEV provides a special mech-anism to export memory. Additionally, the guest owner can imposerestrictions on the memory export. She can prohibit the exportusing the guest policy, and she can de�ne the minimum SEV APIversion of the target system using the API �elds of the guest pol-icy [3, Chapter 3].

Using the SEV API, the hypervisor initiates the export of the VMmemory on the source platform. �e exported memory is encrypted,and integrity protected using transport keys that are generated bythe source SEV �rmware. To that end a secure channel betweenthe source and the target SEV �rmware is established, see Figure 2variant M. �e target platform can decrypt the exported memoryand re-encrypt it using a freshly generated ephemeral memoryencryption key. �is allows the export of encrypted memory in theface of an untrusted hypervisor as only the source and the targetSEV �rmware share the transport keys that are used to encrypt theexported memory.

3 FIRMWARE ANALYSIS�e Platform Security Processor (PSP) hosts a �rmware providedby AMD. Amongst other things, this �rmware implements all SEVrelated operations carried out by the PSP. Given the trust model ofSEV where the guest owner requests services from an untrustedhypervisor, it is paramount that this �rmware is not under controlof the platform owner, but instead provisioned by a trusted entity,AMD in this case. �is section presents the results of our �rmwareanalysis on which our a�acks in Section 5 are based.

3.1 PSP Firmware StructureBy analyzing UEFI �rmware updates of AMD Epyc systems, wewere able to locate the PSP �rmware. It is comprised of severalcomponents which are stored in an undocumented area of the UEFI�rmware image. Although the layout of the UEFI �rmware residingon the SPI �ash is standardized [12], the PSP and SEV �rmware arenot part of the standardized layout. Building on published informa-tion from the Coreboot project [26] and the SEV API speci�cation[3] we were able to understand the proprietary �lesystem and iden-tify and extract all �rmware components. �e individual �rmwarecomponents are prepended with a header containing metadata. �ismetadata contains a version �eld and also determines the certi�cateused to verify the component’s integrity.

Most relevant for the a�acks presented in this work are threecomponents: �e ARK public key (see Figure 1), a component wecall PSP Operating System (PSP OS) and a component that imple-ments the SEV API, the SEV �rmware. To facilitate further research,we developed a PSP �rmware analysis tool which is publishedunder [32].

PSPTool allows to parse the proprietary �lesystem used to storethe PSP OS and SEV �rmware. It lists all �rmware componentsalongside various a�ributes. Furthermore, it is able to correlate SPIread accesses recorded with a logic analyzer with a given binary.�is allows to inspect the order in which the PSP �rmware com-ponents are loaded from �ash. A full list of features can be foundat [32].

5

ARK Public Key �is ARK is a 2048 bit RSA public key storedin a format as described in [3, Appendix B.1]. We could verify thatboth the PSP OS and the SEV �rmware are signed with the ARKprivate key. �e prepended header of each component is part ofthe signed data. �e ARK public key is also contained in the ARKcerti�cate that can be obtained from the AMD key server [4], seeSection 2.3.

PSP Operating System �e PSP OS is the only component thatcontains privileged ARM code. It executes in the privileged modeof the PSP, the SVC mode, with paging enabled. Amongst its respon-sibilities are system initialization and loading of other �rmwarecomponents. �ese components execute in the unprivileged USRmode and are thus separated from the PSP OS. �ere is always onlya single unprivileged component present in memory. To switch toa di�erent component, the currently loaded component is replaced.�e PSP OS provides a syscall interface for unprivileged compo-nents that provides, e.g., access to the cryptographic co-processor(CCP). �e CCP is a dedicated hardware component that allowso�oading various cryptographic operations. It is usable from boththe main processor and the PSP.

SEV Firmware �e SEV �rmware implements the SEV API spec-i�cation [3]. It is loaded by the PSP OS and is executed in theunprivileged USR mode. Its responsibilities include the key genera-tion steps shown in Section 2.3 as well as the policy enforcementdescribed in Step 4 of Section 2.5. It maintains a state of all SEV-enabled guest VMs as well as the platform state which includes thegenerated certi�cates and private keys. Upon initialization, the SEV�rmware will either load a previously saved state from non-volatilestorage utilizing the syscall interface provided by the PSP OS orgenerate a new state including new certi�cates. �e saved stateis encrypted, and integrity protected [3, Chapter 5.1.5]. �e gueststate includes the guest policy as well as the guest VM’s memoryencryption keys. While the PEK and PDH are generated using a“secure entropy source” [3, 2.1.1f)], the CEK is derived from “chip-unique OTP fuses” and has a lifetime of the corresponding CPU [3,2.1.3]. To retrieve the value from the OTP fuse, the SEV �rmwareissues a syscall. �e PSP OS will then retrieve 32 bytes from theCCP and relay it to the SEV �rmware. �ese 32 bytes are used asan input to a key derivation function.

3.2 PSP Boot SecurityTo be�er understand the PSP boot process, we used a logic analyzerto record accesses to the SPI �ash memory hosting the UEFI im-age. �e PSP �rmware components are stored alongside the UEFI�rmware. Speci�cally, the PSP �rmware resides in UEFI paddingvolumes (Pad File), see [12, Chapter 2.1.4]. We observed that the�rst components that are loaded from �ash are the ARK public keyand the PSP OS. We also observed a delay a�er the PSP OS is loadedand before any other a�empt to access the �ash can be observed.Our experiments have shown that a modi�ed ARK will result in nofurther �ash reads a�er the ARK is read. In case the PSP OS wasaltered, a PSP OS from a di�erent �ash location is loaded. If thisrecovery PSP OS is also altered, the system resets.

PSP ROM

On-Chip Bootloader

3. Load & verify PSP OS

1. Store fuse value in CCP

2. Load & verify ARK

One-Time-Programmable Fuse

UEFI Volume

PSP OS

ARK Public Key4. Initialize PSP

5. Load & verify SEV Firmware

SEV Firmware

6. Derive cryptographic keys

Figure 4: Boot procedure on an SEV-enabled system. A lockdenotes a non-modi�able componentwhereas a pen denotesmodi�able components. Arrows denote dependencies of ini-tialization steps.

Based on these observations and our static analysis from Section3, we inferred a boot order, as shown in Figure 4. �e Figure focuseson the security-relevant parts of the SEV technology; other stepsof the boot process are omi�ed.

In addition to the PSP OS and the SEV �rmware, a third compo-nent is responsible for bootstrapping the system. �is componentis responsible for loading the ARK certi�cate and the PSP OS from�ash, steps 2 & 3. Since it is not part of the PSP �rmware loadedfrom �ash and we conclude that it is stored in a read-only memory(ROM) of the PSP, we call it on-chip bootloader.

Our static analysis revealed that the OTP value that is used toderive the CEK, see Section 2.3, is provided by the PSP OS via asyscall. �e implementation of that syscall simply forwards a valuefrom the CCP to the calling unprivileged component. �is value isstored in a storage block of the CCP. �ese storage blocks are used bythe CCP to maintain a context for cryptographic routines [22]. Fromthat, we conclude that the on-chip bootloader is also responsiblefor storing the OTP value in the CCP storage block, see Step 1 ofFigure 4.

A�er the integrity of the PSP OS is veri�ed, the PSP OS is exe-cuted. It initializes the PSP, Step 4, which includes basic operatingsystems tasks such as initializing the pagetables and peripherals,e.g., the interrupt controller. In Step 5, it loads the SEV �rmwareand veri�es its integrity using the ARK public key. �e ARK publickey is not loaded from �ash again, but instead, the PSP OS assumesthe presence of that key at a �xed memory location. We infer thatthe PSP on-chip bootloader placed the ARK public key there a�erits integrity was veri�ed in Step 2. Lastly, upon initialization of theSEV �rmware, the �rmware either loads the platform state fromnon-volatile storage, see Section 2.2, or generates the cryptographic

6

keys, Step 6, as described in Section 2.5.

Security Issues Our experiments revealed a security issue in thesignature veri�cation mechanism of the PSP OS which allowed us toload custom components through modi�ed UEFI images. Either thea�acker has physical access to the SPI �ash chip and makes use of anSPI programmer, or the UEFI update mechanism of the motherboardvendor allows her to �ash a custom image. Depending on thevendor, there is no signature check on the UEFI image in place [28]or it can be disabled [10]. �ese tools can be used remotely, so nophysical access to the target device is required. Similar issues werepreviously published by security researchers [13]. AMD con�rmedthe existence of these issues [24] and newer versions of the PSPOS are already patched. However, our experiments showed thatthe PSP does not employ any rollback prevention mechanism, i.e.,even though a system already makes use of a patched PSP OS, itis always possible to revert to an earlier, vulnerable version. �us,as the issues exist in signed versions of the PSP OS, mitigationsrequire changes to the component verifying the signature, the on-chip bootloader, in order to prevent a vulnerable PSP OS versionfrom being loaded.

AMD con�rmed to us in personal correspondence that the on-chip bootloader is not updatable through a �rmware update. Instead,using the UEFI update mechanism, the PSP is able to update afuse con�guration that allows changing the key that is used toauthenticate the PSP �rmware components. �is could be leveragedto revoke the currently used key and thus prevent the vulnerablePSP OS version to be loaded. In this case, updated versions of thePSP OS would have to be signed with an alternative key.

But even though multiple issues of the PSP OS have been reportedto AMD in mid 2018 [13], we have not found any evidence of akey revocation. To that end, we analyzed ten di�erent binariesfrom �ve di�erent motherboard vendors. At the time of writing,the latest UEFI updates contain PSP OS versions that are signedwith the same key as the vulnerable PSP OS version. From that,we conclude that AMD has not revoked any ARK public keys inorder to suspend vulnerable PSP �rmware versions. To the best ofour knowledge, the same ARK is used across all CPUs of the AMDEpyc Naples series. �is indicates that every CPU of that series isa�ected as any PSP OS version is loaded as long as it is signed withthe ARK. Based on these results we conclude that for the AMDEpyc Naples series CPUs, an a�acker is able to:

(1) Execute custom code on the PSP using a vulnerabilityin the signature veri�cation mechanism of the PSP OS.

(2) Roll back from any PSP OS version to a vulnerable PSPOS version.

3.3 CEK ExtractionLeveraging the security issues discussed in the previous section, webuilt and deployed a patched SEV �rmware. �is �rmware allowsus to read and write arbitrary PSP memory. Using this patched SEV�rmware, we extracted the SEV state, see Section 3.1, including theCEK private key of three di�erent AMD Epyc CPUs.

We obtained the corresponding signed CEK certi�cates fromthe AMD key server, see Section 2.3, and veri�ed the extractedprivate keys by creating signatures that can be validated using the

signed CEK certi�cate. A proof-of-concept signature created withan extracted CEK can be found at [9].

Although the exact details of the CEK extraction are omi�edin this paper, we will provide security researchers with additionaldetails to reproduce our results upon request. We plan to releasethe exact details once a �xed hardware platform is available forcustomers.

As both the authenticity of the SEV platform as well as thecon�dentiality of the data protected by SEV rely on the security ofthe CEK, the CEK extraction lays the groundwork for the �rst twoa�acks in Section 5. �e motivation for our a�acks is discussed inthe following section.

4 ATTACK MOTIVATION�e SEV technology o�ers data protection of virtual machines in theface of an untrusted platform owner. �is incorporates a maliciouscloud provider or a malicious system administrator. In this section,we present two di�erent motivations for an a�acker to circumventthe security properties of SEV as presented in Section 2 and Section3.

4.1 Extract Con�dential Data�e �rst motivation regards data the� and originates from an indi-vidual targeting an SEV guest owner.

�e additional security measures provided by SEV enable compa-nies to process con�dential data in the cloud, that would otherwisenot be processed in the cloud because of its con�dentiality. �egoal of the �rst type of a�acker is to get access to this data despitethe presence of SEV.

An individual with malicious intent and su�cient permissionscould use data from a commercial guest owner to pursue traditionalfraud, e.g., with stolen credit card data.

4.2 Save Resource Overhead�e second motivation is of economic nature and originates on theorganizational level of a cloud provider.

In an Infrastructure-as-a-Service (IaaS) scenario, the cloud providercharges guest owners based on the amount of resources they allo-cate, including CPU, memory, and disk utilization.

In order to increase the overall utilization of memory, manyhypervisors employKernel same-pagemerging (KSM) [7] to increasethe memory utilization.

KSM requires the hypervisor to read the guest memory of vir-tual machines in plaintext to identify duplicate pages. Using SEVprohibits KSM, since memory pages of di�erent guest VMs are en-crypted using di�erent keys. �erefore, the memory requirementof an SEV-enabled system is increased, which results in highercosts for the cloud provider. In an IaaS scenario, it is likely thatthe cloud provider will pass those additional costs to the customerscommissioning the security features of SEV.

While a benevolent cloud provider might disable KSM to preventa�acks such as [35], and [27], this is not necessarily the case for aprovider with malicious intent.

In order to increase revenue, a malicious cloud provider couldfake the presence of SEV, while still charging additionally for SEVprotection. �e guest VM would instead be hosted traditionally

7

on a non SEV-enabled system, leveraging, e.g., KSM to reducethe memory consumption and therefore the costs for the cloudprovider.

5 ATTACKSIn this section, we propose three di�erent a�acks targeting AMDSEV. �ese are structured according to the two main a�ack motiva-tions, as described in Section 4.

5.1 Fake SEVAs presented in Section 4.2, SEV prevents the use of virtualizationfeatures like KSM, thus increasing the overall memory requirementsof a cloud setup. �is additional cost may motivate a maliciouscloud provider to fake the presence of SEV. Furthermore, by fakingthe presence of SEV, the cloud provider gains access to data whichis not accessible when protected by SEV. Using an extracted CEKprivate key as shown in Section 3.3, the cloud provider may pose asan authentic AMD SEV platform even though SEV is not enabledor even present at all.

A�ack model �e a�acker is a cloud provider running arbitraryhosting hardware, who has had access to any SEV-enabled sys-tem for one-time extraction of the CEK private key and the corre-sponding platform ID. It is not required that this speci�c systemhosts the victim’s VM. �e victim is a cloud customer expectingan SEV-enabled VM from the a�acker. �is a�ack does not imposerestrictions on the guest system in use.

Method From the point of view of a guest owner, the correctdeployment of VMs can be veri�ed through remote a�estation bythe trusted SEV �rmware. As described in Step 5 of Figure 2, theguest owner authenticates the remote platform using a certi�catechain (as seen in Figure 1) that roots in the AMD Root SigningKey (ARK). Once the remote platform is authenticated, the guestowner de�nes the transport keys, wraps them, and sends them tothe cloud provider, see Steps 7 and 8 of Figure 2.

In order to simulate the presence of SEV, the a�acker �rst gen-erates an arbitrary PEK and, if required, signs it using the OCA.Next, she uses the extracted CEK private key (see Section 3.3) tosign the PEK. She then generates a PDH which is in turn signed bythe PEK. Now, the a�acker has control over the highlighted partsof the certi�cate chains:

PDH → PEK → CEK → ASK → ARK (3)

PDH → PEK → OCA (4)

To mount the a�ack, the a�acker provides the platform ID corre-sponding to the CEK, the PEK, and PDH to the guest owner. �eguest owner will obtain the ASK, ARK, and CEK certi�cates fromthe AMD key server, see Steps 3 and 4, variant D of Figure 2. A�erveri�cation of the certi�cate chains, the guest owner deploys theguest VM, including the encrypted transport keys. As the a�ackerpossesses the private PDH key, she can decrypt the transport keysprovided by the guest owner. Now the a�acker calculates a hashof the guest VM’s memory. As shown in Figure 3 Step 4, the hashis provided to the guest owner. Due to the fact that the a�acker

controls the transport keys, she can provide the hash of the guestVM’s memory and protect it using the transport keys.

�e last step in the guest deployment phase is to provide a secret,e.g., a disk encryption key, to the guest VM, see Step 5 of Figure 3.�e guest owner protects the disk encryption key using the trans-port keys. As they are known to the a�acker, she can �rst decryptthe disk encryption key and then decrypt the encrypted disk to getaccess to the guest virtual machine’s con�dential data. To fake thepresence of SEV, the a�acker injects the encrypted disk encryptionkey into the guest virtual machine, i.e., she copies it into the guest’smemory. �e guest is now fully operational; however, althoughthe remote a�estation mechanism of SEV was successfully carriedout, the guest owner has no means to detect the absence of the SEVfeature.

�is enables a malicious cloud provider to increase the numberof guests on a single host making use of KSM, see Section 4.2. Moreso, as the runtime protection of SEV is not enabled, the a�ackercan access any data used in the guest VM.

5.2 Migration AttackAs outlined in Section 4.1, the goal of this a�ack is to extract run-time data of an SEV-enabled guest from a host system.

A�ack model �e a�acker is an individual, e.g., a system ad-ministrator of an otherwise trusted organization, with access tothe management interface of an SEV-enabled host. �e victim is acloud customer who successfully deployed a virtual machine on theSEV-enabled host of the cloud provider. We assume that there areno security issues present in the guest VM and the PSP �rmwareon the host. �e a�acker must have had access to any SEV-enabledsystem for one-time extraction of the CEK private key and to obtainthe corresponding CEK certi�cate. It is not required for the a�ackerto have access to the CEK private key belonging to the platformhosting the VM. Furthermore, he must be able to initiate a virtualmachine migration of the victim’s VM using the management inter-face of the host. Additionally, the guest policy must allow migrationof the guest. To bene�t from cloud services such as high availabilityand dynamic resource allocation, migration is required in order tohandle resource contention or failures in the host system. �us itis likely that the guest policy allows migration.

Method Similarly to the previous a�ack, the a�acker �rst needsto get hold of a valid CEK private key of any authentic SEV-enabledsystem, see Section 3.3. �e a�acker creates the two certi�catechains as described in Section 5.1, chain 3, and chain 4.

Using the SEV API commands for VM migration (see Section 2.6),the a�acker instructs the SEV �rmware to initiate the migrationof the victim’s VM using the prepared PDH, PEK, and ARK. Usingthe a�acker-controlled PDH, the SEV �rmware on the source hostwill authenticate the target platform of the migration, see Step 5of Figure 2. Since the provided certi�cates were created using avalid (extracted) CEK, the SEV �rmware will accept the PDH. �eSEV �rmware then generates the transport keys, and wraps themusing keys derived from the authenticated PDH, see Steps 7 and 8 ofFigure 2. �e memory of the VM is encrypted using the generated

8

transport keys and exported along with the wrapped transportkeys.

Instead of forwarding the keys, see Step 9 of Figure 2, the at-tacker can now unwrap the transport keys and decrypt the virtualmachine’s memory since he controls the PDH that was used toderive the keys used to encrypt the transport keys.

We emphasize that this a�ack does not require any securityissues to be present in the PSP �rmware of the source host. Byowning any CEK private key, the a�acker can impersonate a validtarget for migration. As the transport encryption keys in SEV mustbe shared with the target of a migration, the a�acker, posing as avalid migration target, can decrypt the exported memory.

5.3 Debug Override AttackSimilarly to the previous a�ack and as outlined in Section 4.1, thegoal of this a�ack is to extract run-time data of an SEV-enabledguest from a host system.

A�ack model �e a�ack model is similar to the Migration At-tack in Section 5.2, with the additional requirement that the at-tacker must be able to install UEFI updates on the system that hoststhe victim’s VM. UEFI updates can be deployed remotely withoutany physical access using the server’s update mechanism, see Sec-tion 3.2. Alternatively, they can be deployed with physical accessvia directly programming the SPI �ash located on the server’s moth-erboard using an SPI programmer.

Method �e SEV API speci�es a debug interface to assist debug-ging of SEV-protected virtual machines. �e debug interface allowsa hypervisor to read and write guest memory in plaintext, see [3,Chapter 7.1]. For example, in a QEMU/KVM scenario, QEMU o�ersthe pmemsave command to dump guest memory to a �le [31]. Inan SEV-enabled guest, that memory is encrypted and thus of nouse to debug the guest. �e SEV API debug commands enable thehypervisor to dump plaintext memory instead. �e SEV �rmwarewill only allow the use of this interface if the guest owner explicitlyenabled debugging in the guest policy as described in Section 2.5.

Using the security issues described in Section 3.2, an a�acker isable to patch the SEV �rmware so that it ignores the policy. A�erinstalling the patched �rmware, the SEV �rmware will decrypt orencrypt guest memory regardless of the guest owner’s policy. �isallows the a�acker to read and write arbitrary guest memory usingthe SEV debug interface from the SEV-enabled host.

Leveraging these security �aws, we were able to successfullyinstall such a patched version of the SEV �rmware. As opposed toprevious a�acks on SEV such as [25] and [15], this a�ack does notdepend on any services running inside the guest VM.

�is a�ack is also possible if the �rmware vulnerabilities de-scribed in Section 3.2 are �xed. Due to the missing rollback pre-vention, an a�acker can always replace the existing PSP OS with avulnerable version before installing her patched SEV �rmware.

6 DISCUSSIONIn the previous sections, we laid out how security issues in the PSP�rmware pave the way for a�acks against SEV that permanentlybreak the security properties of the SEV technology on AMD Epyc

based systems. Furthermore, we demonstrated that, although theissues have been addressed through �rmware updates, the con�-dentiality of SEV-protected systems is still at risk. �is is due to thefact that the presence of an updated �rmware cannot be veri�ed bya guest owner. A given CEK is valid throughout the lifetime of theCPU and does not depend on the �rmware version. �is sectiondiscusses possible mitigations and proposes a key generation designfor future SEV implementations.

Without vulnerabilities as the one described in Section 3, none ofour a�acks would be possible. Various methods such as static anddynamic analysis, formal veri�cation, and extensive code auditsnaturally come to mind when discussing possible mitigations. How-ever, the sheer amount of security issues that are present in relatedsystems, such as Intel ME [16, 17] and the PSP itself [13], prove thedi�culty to ensure the absence of security issues. We hence believethat the security design of SEV should incorporate the possibility of�rmware bugs. �us the proposed mitigations focus on SEV designchanges that empower a guest owner to enforce the use of both anauthentic as well as an up-to-date �rmware on the remote platform.

6.1 Current Design IssuesAs discussed in Section 3.2, the PSP allows installing any signed�rmware, including rollbacks to previous insecure versions. �isallows a�ackers to provision any AMD Epyc CPU with a vulnerablePSP OS version and mount the a�acks discussed earlier. While thecurrent on-chip bootloader is not updatable, AMD con�rmed that itis possible to revoke an ARK and enforce the use of alternative keysto verify the integrity of the PSP OS. �is mechanism can be usedto label every currently available PSP �rmware as untrusted, i.e.,e�ectively preventing an a�acker to roll back to a vulnerable PSPOS version. However, it does not allow a guest owner to verify thata PSP OS version signed with the alternative key is actually used.Furthermore, the CEK does not depend on the PSP OS version, i.e.,a CEK extracted before the revocation of the ARK key is still valida�erwards - its lifetime is the lifetime of the CPU [3, Chapter 2.1.3].�us this approach is insu�cient to mitigate our proposed a�acks.�is is also true for another SEV mechanism: SEV allows the guestowner to enforce the SEV API version implemented by the remoteplatform, see Section 2.5 Step 4. However, the SEV API check isonly enforced by the SEV �rmware. An a�acker able to manipulatethe SEV �rmware can spoof arbitrary SEV API versions.

�e validity of the CEK across �rmware versions impedes mit-igations only based on �rmware updates. To overcome this, wepropose design changes to the SEV technology which are laid outin the next section.

6.2 Proposed SEV Design Changes�e goal of our proposed design changes is to enable the guestowner to enforce the use of an authentic and up-to-date PSP �rmware.It is speci�cally not our goal to provide means to ensure the PSP�rmware has no bugs in the �rst place. We rather focus on designchanges that allow ensuring the PSP �rmware in use is still trusted.Our proposed changes aim to incur only low complexity overheadto the current SEV technology. �is enables re-use of the currentso�ware stack and minimizes the required e�ort to migrate to our

9

proposed design.

CEK Derivation �e current CEK is derived using a key deriva-tion function (KDF) that takes a 32-byte secret value which isunique per CPU and is stored in one-time-programmable (OTP)fuses (SOTP ):

CEK = KDF(SOTP )We propose to change the way the CEK is generated in order to

connect it to the PSP OS version and the SEV �rmware version. Tothat end, we introduce a two-stage secret generation procedure. In-stead of deriving the CEK directly from the SOTP , two intermediatesecrets are derived using di�erent inputs:

(1) SPSP : based on the PSP OS version (PV ) and SOTP .

SPSP = KDF(PV , SOTP )(2) SCEK : based on the SEV �rmware version (SV ) and SPSP .

SCEK = KDF(SV , SPSP )�e �nal CEK is then derived from SCEK :

CEK = KDF(SCEK )For the sake of simplicity, static, i.e., non-con�dential, inputs to theKDF were omi�ed. �e resulting CEK will not only depend on thechip-unique SOTP , but also on the current PSP OS version as wellas the SEV �rmware version. As the intermediate secrets must notbe accessible by an a�acker, our design separates the derivation ofthose secrets in di�erent PSP �rmware components.

On-ChipBootloader

PSP OS

SEV Firmware

On-ChipBootloader

SOTP

PSP OS

SEV Firmware

SPSP

SCEK

Current SEV Design Proposed SEV Design

SOTP version 

 version 

Figure 5: �e current SEV design as opposed to our proposeddesign. Dashed lines between two secrets show a derivation.Boxes show the scope of secrets and �rmware components.

In the current SEV design, the initial CEK secret, SOTP , is ac-cessible to the on-chip bootloader, the PSP OS as well as the SEV�rmware (see le� part of Figure 5).

Our new design proposes a be�er isolation, as depicted on theright hand side of Figure 5. �e SOTP is only accessible to theon-chip bootloader and is used in conjunction with the PSP OSversion to derive the SPSP . �e SPSP is accessible by the PSP OSand is given as an input to a key derivation function together withthe SEV �rmware version. �e derived SCEK is used by the SEV�rmware to �nally derive the CEK.

�e required changes in the �rmware components are discussedin the following paragraphs.

On-chip Bootloader �e current on-chip bootloader provisionsthe Cryptographic Co-Processor (CCP) with the SOTP , see Step 1 ofFigure 4. In our proposed design, the on-chip bootloader provisionsthe CCP with SPSP instead.

SPSP is derived from the SOTP and the PSP OS version using akey derivation function. �e PSP OS version is a �eld in the signedheader of the PSP OS component stored on �ash. �e currenton-chip bootloader is already required to read this header as itcontains information about the key used to verify its signature.In addition, the on-chip bootloader needs to implement the keyderivation function that derives the SPSP . It is crucial for ourenhanced design, that the original SOTP is never visible outsidethe scope of the on-chip bootloader. To that end, the hardwarecomponent implementing the access to the SOTP must preventfurther accesses a�er the �rst access.

As the on-chip bootloader is not updatable, it is paramount thatits code complexity is rather low. Any errors in this component thatare identi�ed a�er the CPU is manufactured and shipped cannotbe �xed.

While our proposed changes do increase the overall complex-ity of the on-chip bootloader, we believe they are reasonable andmanageable. Determining the PSP OS version incurs only li�leoverhead as this information is present in the PSP OS header storedon �ash. Including the PSP OS version in the SPSP simply requiresparsing one additional header �eld.

�e key derivation function does add additional complexity.However, the PSP system has access to the CCP, which o�ers thepossibility to o�oad cryptographic operations. We believe that thecomplexity of the KDF implementation can be reduced by o�oad-ing the cryptographic primitives, such as hash functions, to theCCP. In fact, the CCP is already leveraged to verify the signatureof the �rmware components. �e proposed KDF in the on-chipbootloader could make use of the CCP in a similar fashion.

Limiting the privilege of accessing the SOTP to only the on-chipbootloader e�ectively reduces the risk of leaking the SOTP as Fig-ure 5 illustrates.

PSP OS In contrast to the original SEV design, the PSP OS mustnot get access to the original SOTP . Instead, it only has access tothe intermediate secret SPSP that depends on the PSP OS version.Similarly to the proposed on-chip bootloader changes, the PSP OSuses the SEV �rmware version together with the SPSP to derivethe SCEK . �e SEV �rmware version information is present in theheader of the SEV �rmware which is parsed by the PSP OS. �eresulting SCEK now depends on the SOTP , the PSP OS version andthe SEV �rmware version. As the PSP OS runs at a higher privilegelevel (SVC mode) than the SEV �rmware (USR mode), the intermedi-ate SPSP is not accessible by the SEV �rmware. Only the SCEK isprovided to the SEV �rmware through a syscall.

SEV API To accommodate for our proposed design changes, theformat of the CEK certi�cate, see [3, Appendix C.1], must be ex-tended. In the current SEV API, version 17 at the time of writing, theCEK certi�cate contains no information about the PSP components.

10

We propose to extend the CEK certi�cate format to also include theminimum PSP OS and SEV �rmware versions. As opposed to theoriginal SEV design, there are now multiple valid CEKs for a singleCPU.

To enable the guest owner to enforce �rmware versions of theremote platform, we further propose to extend the guest policy toinclude the PSP OS version as well as the SEV �rmware version.

SEV Firmware �e current SEV �rmware derives the CEK froma secret value that is provided by the PSP OS. �is does not changein our proposed design. However, the SOTP is not exposed to theSEV �rmware. Instead, the CEK is derived from the SCEK which isaccessible via a syscall.

As there are now multiple CEKs for a single platform, the SEVAPI must support to enforce the minimum SEV �rmware versionand PSP OS version that is de�ned by the guest policy.

During the initial deployment, the guest owner will additionallyreceive the �rmware versions, Step 1 of Figure 2, and will querythe AMD key server using the platform ID together with the stated�rmware versions, see variant D, Step 3 of Figure 2. �e retrievedCEK is then used in the certi�cate chain veri�cation, as shown inStep 5 of Figure 2. �e veri�cation will only succeed if the remoteplatform hosts the stated �rmware versions.

In case of migration, the source SEV �rmware enforces the mini-mum �rmware versions when authenticating the target platform,Step 5 of Figure 2. To that end, the source SEV �rmware veri�esthat the versions speci�ed in the provided CEK certi�cate of thetarget, see variant M, Step 4 of Figure 2, are equal to or greater thanthe versions speci�ed in the guest policy.

AMDKey Server In the current SEV design, the AMD Key Serverprovides means to retrieve a CPU-speci�c CEK certi�cate for agiven platform ID, as shown in variant D, Step 3 of Figure 2. In ourenhanced design, the AMD Key Server is queried with a platformID as well as PSP OS and SEV �rmware versions.

In a similar fashion to the proposed key derivation introducedabove and based on the CPU-speci�c SOTP , the AMD Key Serverwill calculate the intermediate secret SPSP before delivering theversion-dependent SCEK to the client. It is le� to AMD how tocommunicate the revocation of certain PSP OS and SEV �rmwareversions for use with the SEV technology. One possible option isto disable the generation of CEK certi�cates for known vulnerablePSP OS and SEV �rmware versions. We believe that the on-demandcalculation of CEKs increases the required computational e�ort forthe AMD Key Server to a manageable degree.

6.3 Security Evaluation�is section discusses the advantages of our proposed design. Forthe following paragraphs, we assume that a previously releasedPSP �rmware version contained security issues which are �xedin a later version. �is is the case for the current state of the PSP�rmware [24]. Furthermore, we assume that AMD publishes in-formation about the outdated, vulnerable �rmware versions alongwith the updated version. Additionally, we assume that it is notpossible to extract the CEK from the current PSP �rmware version.

Guest Deployment �e proposed SEV design changes allow aguest owner to enforce the use of speci�c �rmware versions on theremote platform. As the CEK is now tied to the �rmware versiondeployed on the remote platform, a CEK extracted using di�erent�rmware versions is no longer valid. In the current SEV design, thesecurity of the SEV technology itself is compromised as long as abug exists in any relevant PSP �rmware component.

During the deployment of a guest VM, the guest owner authen-ticates the remote platform, see Step 5 in Figure 2. In our enhanceddesign, the guest owner retrieves the CEK certi�cate for the up-to-date �rmware, i.e., the �rmware version that includes the bug �xes.To that end, she uses the platform ID of the target platform alongwith the required PSP OS and SEV �rmware version to retrieve theCEK certi�cate from the AMD key server, variant D, Step 3 and 4of Figure 2.

While a malicious cloud provider could provide an extracted CEKfrom an outdated, vulnerable �rmware, the CEK will not matchthe CEK certi�cate served by the AMD Key Server. As the guestowner only trusts CEKs for speci�c PSP �rmware versions, shewill dismiss the proposed CEK from the malicious cloud provider.�is prevents the deployment of SEV-protected guest VMs on SEVplatforms with known security issues.

Since the Fake SEV a�ack presented in Section 5.1 relies on avalid, extracted CEK, which can only come from a vulnerable andtherefore revoked �rmware, the new design e�ectively preventsthis a�ack.

In a similar fashion, the Debug Override a�ack presented in Sec-tion 5.3 relies on the ability to alter the SEV �rmware in order topatch and abuse its debug functionality. �is is only possible witha vulnerable, i.e., revoked, �rmware, which will be dismissed bythe client.

Migration Leveraging the enhanced SEV design, the guest ownersuccessfully deployed her virtual machine on a trusted SEV platform.To ensure the virtual machine is not migrated to a platform usinga vulnerable �rmware version, the source SEV �rmware enforcesa version check on the CEK. �e a�ack discussed in Section 5.2requires the a�acker to provide an extracted CEK. In the originalSEV design, the source SEV �rmware validates the CEK solely on thefact whether it has a root of trust originating in an ARK certi�cate.In our enhanced design, the SEV �rmware also ensures that theCEK certi�cate is valid for the SEV �rmware version speci�ed in theguest policy. To that end, the CEK certi�cate contains the PSP OSversion as well as the SEV �rmware version numbers, see Section6.2. �e SEV �rmware can now verify that the version �elds ofthe provided CEK are equal to, or higher than the version numbersspeci�ed in the guest policy.

While an a�acker could still provide a valid, extracted CEK, shecannot provide a CEK with a valid version �eld assuming the spec-i�ed PSP �rmware versions do not contain known security issues.Migration of an SEV-protected virtual machine to an SEV �rmwareversion lower than speci�ed in the guest policy will not be permit-ted. �is protects against the Migration A�ack presented in Section5.2.

11

7 RELATEDWORKWork such as [18, 23, 29, 33, 34, 36] show that protecting virtual ma-chines from an untrusted hypervisor has been subject to extensiveresearch.

In [36] Zhang et al. propose the use of a higher-privileged secu-rity monitor called CloudVisor in order to protect virtual machinesin the face of a compromised hypervisor. In their a�ack model, theyassume the cloud provider himself not to be malicious and thereforeexclude physical a�acks. In their design, that is very similar to thedesign of AMD SEV, they aim to separate resource managementfrom security protection in the virtualization layer. �eir proto-type version of CloudVisor is implemented in only 5.5K lines ofcode and works with the Xen hypervisor. Zhang et al. leverage aTrusted Platform Module (TPM) in two ways: �e integrity of theCloudVisor code is ensured by the TPM and Intel Trusted ExecutionTechnology (TXT). �e second use of a TPM in CloudVisor is toprovide con�dential communication between the cloud customerand CloudVisor. It roots in the Storage Root Key (SRK) of the TPM,that has the lifetime of the platform owner and is authenticated bythe Endorsement Key (EK) of the TPM. �e EK keypair of TPMsis unique per chip, thus binding the TPM’s identity. Its privatekey must therefore never leave the TPM. �e manufacturer of theTPM provides certi�cates of the EK public key through a certi�-cate authority (CA). Given that the speci�c identities of TPMs usedfor CloudVisor are guaranteed by a third party, this enables thecloud customer to also authenticate intermediate keys signed bythe EK. �ese can then be used for con�dential communication.With AMD SEV, the CEK serves a similar purpose as the EK of aTPM: It binds the identity of an SEV-capable CPU using chip-uniqueone-time-programmable fuses.

While the remote a�estation mechanism of SEV has previouslynot been subject to research, researchers presented several a�acksagainst the runtime protection of SEV.

In 2017 Hetzelt and Buhren presented the �rst security analysisof the SEV Technology [15]. �ey proposed three a�acks on anSEV-enabled system, two of which rely on the lack of protection ofthe Virtual Machine Control Block (VMCB) and registers. �ese aremitigated in case the encrypted state extension to SEV is enabled,see Section 2.2. �e third a�ack leverages the missing integrityprotection of the guest memory, allowing the hypervisor to conducta replay a�ack. To that end, the nested pagetable was altered toenforce a pagefault on every guest page that was executed. It wasshown that the sequence of pagefaults is enough to determinethe location of a password in the target VM’s memory. �e pagecontaining the password was then replayed during an a�ackerinitiated SSH login. Using this primitive, the authors were able togain root access to a target system protected by SEV.

Morbitzer et al. leveraged the hypervisor’s control over theguest resources, namely the nested pagetables, to mount an a�ackagainst an SEV-protected VM [25]. In a similar fashion as [15] theyused page tracking to identify a guest page which is served viaa server running in the VM. As the memory is encrypted with aguest-speci�c key, the server will copy the data to an unencryptedpage, e.g., the bu�er of a virtual network card. Once this page islocated, they manipulate the nested pagetable to change the map-ping between guest physical pages to host physical pages. �e new

mapping will point to con�dential data of the VM. Instead of copy-ing the intended data, the server will instead copy the con�dentialdata into the unencrypted bu�er. �is bu�er is then readable by thea�acker. Using this method, the authors were able to fully decrypta VM with 2GB of memory.

In 2018, Israeli research �rm CTS Labs published a whitepapercalled AMDFlaws claiming to have found multiple critical vulnera-bilities in AMD processors allowing arbitrary code execution onthe PSP [21]. In the whitepaper, the researchers publish conceptualinformation about vulnerabilities in both Epyc and Ryzen PSPsas well as alleged manufacturer backdoors in both �rmware andhardware of AMD chipsets supporting Ryzen CPUs. In 2019, twomembers of CTS Labs presented insights into three vulnerabili-ties of the AMDFlaws publication [13]. In the presentation, theresearchers illuminate details about the �rmware of the PSP andthe cryptographic checks in use. Nonetheless, the researchers didnot discuss the consequences of a compromised PSP to the SEVtechnology.

8 CONCLUSIONIn this paper, we analyzed the �rmware components that imple-ment the SEV API. We identi�ed security issues in the secure bootmechanism of the PSP that hosts the SEV �rmware. �is allowedus to provide a patched version of the SEV �rmware which givesus arbitrary read and write access to the PSP’s memory. We usedthis �rmware to extract the Chip Endorsement Key (CEK) of threedi�erent AMD Epyc processors. We proposed two a�acks againstSEV-protected virtual machines using the extracted CEK as wellas an a�ack based on a patched SEV �rmware. While the patched�rmware allowed us to extract encrypted memory in plaintext, theextracted CEK allows an a�acker to impersonate the presence ofSEV altogether. Even if the targeted virtual machine is not executedon a compromised SEV platform, the migration a�ack allows ana�acker to acquire the cryptographic keys that are used to encryptthe virtual machines during migration.

�e severity of the proposed a�acks is ampli�ed due to the miss-ing rollback prevention as well as the in�nite lifetime of the CEK.We showed that an a�acker can always roll back to a vulnerablePSP �rmware to extract the CEK. Even if the PSP �rmware is up-graded to a newer version, this extracted CEK is still valid for thecorresponding CPU.

In the current design of the SEV technology, it is impossible for acloud customer to verify the integrity of the remote platform giventhe fact that a vulnerable �rmware version exists. We conclude thatthe SEV technology on AMD Epyc systems of the Naples CPU seriescannot protect virtual machines as the correct deployment cannotbe guaranteed. Given the lifetime of the CEK, it is not possible toprovide purely so�ware-based mitigations.

To overcome the issues of the current SEV technology, we pro-posed design changes to SEV that enable the cloud customer toenforce the use of a speci�c PSP �rmware on the remote platform.�is ensures the trustworthiness of the SEV technology despite PSP�rmware issues as it allows to issue so�ware-based �xes for thePSP �rmware.

12

ACKNOWLEDGMENTS�is work was supported by the Federal Ministry of Education andResearch of Germany in the framework of So�ware Campus 2.0project no. FKZ 01IS17052. Opinions, views, and conclusions arethose of the authors and do not re�ect the views of anyone else.We would like to thank the following persons for their help duringthe work on this paper: Peter Stuge, Stephan Bauroth, Nils Wisiol,Elham Amini and Heiko Lohrke.

REFERENCES[1] AMD. 2005. Secure Virtual Machine Architecture Reference Manual. Whitepaper.[2] AMD. 2013. AMD Security and Server innovation. h�ps:

//members.ue�.org/learning center/UEFI PlugFest AMD Security andServer innovation AMD March 2013.pdf Accessed: 2019-04-29.

[3] AMD. 2018. AMD Secure Encrypted Virtualization API Version 0.17. h�ps://developer.amd.com/wp-content/resources/55766.PDF Accessed: 2019-08-05.

[4] AMD. 2019. AMD CEK Certi�cate Server. h�ps://kdsintf.amd.com/cek/ Accessed:2019-04-16.

[5] AMD. 2019. SEV �rmware for Naples. h�ps://developer.amd.com/wp-content/resources/amd sev fam17h model0xh 0.17b11.zip Accessed: 2019-04-16.

[6] Kaplan AMD and Others. 2016. AMD Memory Encryption. h�ps://developer.amd.com/wordpress/media/2013/12/AMD Memory EncryptionWhitepaper v7-Public.pdf Accessed: 2019-08-05.

[7] Andrea Arcangeli, Izik Eidus, and Chris Wright. 2009. Increasing memory densityby using KSM. In Proceedings of the linux symposium. Citeseer, 19–28.

[8] Russell Brandom and Colin Lecher. 2018. House passes controversial legislationgiving the US more access to overseas data. (2018). h�ps://www.theverge.com/2018/3/22/17131004/cloud-act-congress-omnibus-passed-mlat Accessed:2019-05-14.

[9] Robert Buhren. 2019. Repository containing supplementaldata and results. (2019). h�ps://github.com/RobertBuhren/Insecure-Until-Proven-Updated-Analyzing-AMD-SEV-s-Remote-A�estationAccessed: 2019-08-01.

[10] CodeRush. 2013. Flashing modi�ed AMI AptioUEFI using AFU. h�ps://www.win-raid.com/t286f16-Guide-Deprecated-Flashing-modi�ed-AMI-Aptio-UEFI-using-AFU.html Accessed: 2019-04-18.

[11] Zhao-Hui Du, Zhiwei Ying, Zhenke Ma, Yufei Mai, Phoebe Wang, Jesse Liu, andJesse Fang. 2017. Secure Encrypted Virtualization is Unsecure. arXiv:1712.05090

[12] Uni�ed EFI. 2017. Platform Initialization (PI) Speci�cation. h�ps://ue�.org/sites/default/�les/resources/PI Spec 1 6.pdf Accessed: 2019-05-02.

[13] Uri Farkas and CTS-Labs Ido Li On. 2019. AMDFlaws �� A Technical DeepDive. h�ps://msrnd-cdn-stor.azureedge.net/bluehat/bluehatil/2019/assets/doc/�e%20AMDFlaws%20Story%20Technical%20Deep%20Dive.pdf Accessed: 2019-05-06.

[14] Keiko Hashizume, David G. Rosado, Eduardo Fernandez-Medina, and Eduardo B.Fernandez. 2013. An analysis of security issues for cloud computing. Journal ofInternet Services and Applications 4, 1 (27 Feb 2013), 5. h�ps://doi.org/10.1186/1869-0238-4-5

[15] Felicitas Hetzelt and Robert Buhren. 2017. Security Analysis of EncryptedVirtual Machines. In Proceedings of the 13th ACM SIGPLAN/SIGOPS InternationalConference on Virtual Execution Environments (VEE ’17). ACM, New York, NY,USA, 129–142. h�ps://doi.org/10.1145/3050748.3050763

[16] Intel Security Center. 2019. Intel CSME, Server Platform Services, TrustedExecution Engine and Intel Active Management Technology 2018.4 QSR Ad-visory. h�ps://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00185.html. Accessed: 2019-05-06.

[17] Intel Security Center. 2019. Intel Firmware 2018.4 QSR Advisory. h�ps://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00191.html.Accessed: 2019-05-06.

[18] S. Jin, J. Ahn, S. Cha, and J. Huh. 2011. Architectural support for secure virtualiza-tion under a vulnerable hypervisor. In 2011 44th Annual IEEE/ACM InternationalSymposium on Microarchitecture (MICRO). 272–283.

[19] Miltiadis Kandias, Nikos Virvilis, and Dimitris Gritzalis. 2013. �e Insider�reat in Cloud Computing. In Critical Information Infrastructure Security, San-dro Bologna, Bernhard Hammerli, Dimitris Gritzalis, and Stephen Wolthusen(Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 93–103.

[20] David Kaplan. 2017. Protecting VM Register State with SEV-ES.h�ps://www.amd.com/system/�les/TechDocs/Protecting%20VM%20Register%20State%20with%20SEV-ES.pdf Accessed: 2019-05-06.

[21] CTS Labs. 2018. Severe Security Advisory on AMD Processors. h�ps://safe�rmware.com/amd�aws whitepaper.pdf Accessed: 2019-05-06.

[22] �omas Lendacky and Gary Hook. 2016. ccp-dev.h. h�ps://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/crypto/

ccp/ccp-dev.h?h=v5.0.12#n402 Accessed: 2019-05-04.[23] Jonathan M. McCune, Bryan Parno, Adrian Perrig, Michael Reiter, and Hi-

roshi Isozaki. 2008. Flicker: an execution infrastructure for TCB minimiza-tion. EuroSys’08 - Proceedings of the EuroSys 2008 Conference 42, 315–328.h�ps://doi.org/10.1145/1352592.1352625

[24] Mark Papermaste. 2018. Initial AMD Technical Assessment of CTS Labs Re-search. h�ps://community.amd.com/community/amd-corporate/blog/2018/03/21/initial-amd-technical-assessment-of-cts-labs-research. Accessed: 2019-05-10.

[25] Mathias Morbitzer, Manuel Huber, Julian Horsch, and Sascha Wessel. 2018.SEVered: Subverting AMD’s Virtual Machine Encryption. In Proceedings of the11th European Workshop on Systems Security (EuroSec’18). ACM, New York, NY,USA, Article 1, 6 pages. h�ps://doi.org/10.1145/3193111.3193112

[26] Coreboot Project. 2014. PspDirectory.h. h�ps://github.com/coreboot/coreboot/blob/master/src/vendorcode/amd/pi/00660F01/Proc/Psp/PspBaseLib/PspDirectory.h Accessed: 2019-04-03.

[27] Kaveh Razavi, Ben Gras, Erik Bosman, Bart Preneel, Cristiano Giu�rida, andHerbert Bos. 2016. Flip feng shui: Hammering a needle in the so�ware stack. In25th {USENIX} Security Symposium ({USENIX} Security 16). 1–18.

[28] Supermicro. [n. d.]. Supermicro Update Manager. h�ps://www.supermicro.com/solutions/SMS SUM.cfm Accessed: 2019-04-26.

[29] Jakub Szefer and Ruby B. Lee. 2012. Architectural Support for Hypervisor-secureVirtualization. SIGPLAN Not. 47, 4 (March 2012), 437–450. h�ps://doi.org/10.1145/2248487.2151022

[30] Rich Uhlig, Gil Neiger, Dion Rodgers, Amy L Santoni, Fernando Martins, An-drew V Anderson, Steven M Benne�, Alain Kagi, Felix H Leung, and Larry Smith.2005. Intel virtualization technology. Computer 38, 5 (2005), 48–56.

[31] Stefan Weil. 2019. QEMU version 4.0.93 User Documentation. (2019). h�ps://qemu.weilnetz.de/doc/qemu-doc.html#Commands Accessed: 2019-08-01.

[32] Christian Werling and Robert Buhren. 2019. PSPTool: Display, extract, andmanipulate PSP �rmware inside UEFI images. (2019). h�ps://github.com/cwerling/psptool Accessed: 2019-08-01.

[33] Y. Wu, Y. Liu, R. Liu, H. Chen, B. Zang, and H. Guan. 2018. Comprehensive VMProtection Against Untrusted Hypervisor �rough Retro��ed AMD MemoryEncryption. In 2018 IEEE International Symposium on High Performance ComputerArchitecture (HPCA). 441–453. h�ps://doi.org/10.1109/HPCA.2018.00045

[34] Y. Xia, Y. Liu, and H. Chen. 2013. Architecture support for guest-transparent VMprotection from untrusted hypervisor and physical a�acks. In 2013 IEEE 19thInternational Symposium on High Performance Computer Architecture (HPCA).246–257. h�ps://doi.org/10.1109/HPCA.2013.6522323

[35] Yuval Yarom and Katrina Falkner. 2014. FLUSH+ RELOAD: a high resolution,low noise, L3 cache side-channel a�ack. In 23rd {USENIX} Security Symposium({USENIX} Security 14). 719–732.

[36] Fengzhe Zhang, Jin Chen, Haibo Chen, and Binyu Zang. 2011. CloudVisor:Retro��ing protection of virtual machines in multi-tenant cloud with nestedvirtualization. SOSP’11 - Proceedings of the 23rd ACM Symposium on OperatingSystems Principles, 203–216. h�ps://doi.org/10.1145/2043556.2043576

13