KVM Forum 2021: Full Schedule

Virtual Event | September 15-16, 2021
View More Details

The Sched app allows you to build your schedule but is not a substitute for your event registration. In addition, you must be registered for KVM Forum to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in Coordinated Universal Time (UTC). Please select from the drop-down menu to the right to see the schedule in your preferred timezone above "Filter by Date."

05:00 UTC

Sharing IOMMU PageTables with TDP in KVM - Lu Baolu & Zhao Yan, Intel Corporation

With assigned devices, IOMMU page tables actually hold the same mappings as TDP in KVM if IOMMU shadowed page table is not required. Sharing IOMMU page tables with TDP has the advantages of reduced memory footprint, unified page table for dirty page tracking during migration and page fault handling, probably higher performance by reducing unnecessary EPT/NPT zap, EPT/NPT pre-population, etc… This topic researches the challenges in both KVM and IOMMU subsystems and proposes an architecture to achieve this.

Speakers

Lu Baolu

Senior Software Engineer, Intel Corporation

Baolu, employed by Intel, is the Linux kernel maintainer for Intel VT-d implementation. In recent years, he focused on technologies such as thunderbolt vulnerabilities, scalable I/O virtualization and shared virtual address.

Yan Zhao

Senior Software Engineer, Intel Corporation

Yan, employed by Intel, has more than 10 years experiences in virtualization software development. In recent yeas, she focused on KVM and Linux kernel support for device direct assignment.

KVM 2021 sharing TDP IOMMU pdf

Wednesday September 15, 2021 05:00 - 05:25 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

05:00 UTC

High Performance NVMe Offloading in SPDK Using the New vfio-user Protocol - Changpeng Liu & Benjamin Walker, Intel

Now that the vfio-user protocol nears completion, the Storage Performance Development Kit is releasing code to emulate complete NVMe PCIe devices to virtual machines from a separate user-space process. This is accomplished by adding a new vfio-user “transport” to the existing SPDK NVMe-oF target. Additionally, SPDK will add a vfio-user transport for its nvme (client) driver to enable SPDK-based applications to connect to these emulated devices, acting as an alternative client to QEMU. In this session, we’ll cover how the emulated NVMe controller is implemented in SPDK and the threading model that provides high bandwidth, low latency, and linear scaling. We will also introduce the client vfio-user library which provides basic PCI abstraction APIs, then the client NVMe library built on top. We’ll wrap up by showing some use cases, performance data, and future development plans.

Speakers

Changpeng Liu

Cloud Software Engineer, Intel

Changpeng Liu is cloud software engineer in Intel and core maintainer of SPDK (Storage Performance Development Kit) open source project. His working areas include NVMe, storage IO virtualization and storage offload accelerators in IPU.

Benjamin Walker

Cloud Software Engineer, Intel

Ben Walker is a software engineer in NPG’s Storage Software Acceleration Group. He has been at Intel working on storage related projects since 2011. Prior to Intel, Ben was a software engineer working on high frequency trading infrastructure. Currently Ben is a core maintainer for... Read More →

Changpeng Liu&Ben Walker High Performance NVMe Virtualization with SPDK and vfio user pdf

Wednesday September 15, 2021 05:00 - 05:25 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

05:30 UTC

Support SDEI Virtualization and Asynchronous Page Fault for arm64 - Gavin Shan, Redhat

The asynchronous page fault has been supported on x86 for long time. It helps to improve the parallelism and performance of the guest by picking another process to execute when the requested page is not ready immediately in stage-2 page fault handler. In this way, the vCPU on which the page is requested do not have to be suspended until the requested page becomes ready. The asynchronous page fault is not supported on arm64 yet and it relies on two signals or notifications delivered from host to guest: page-not-present and page-ready. The page-not-present notification should be delivered in synchronized fashion. As NMI is not supported by arm64 naturally, so SDEI (Software Delegated Exception Interface) is leveraged to deliver the page-not-present notification synchronously. On the other hand, the page-ready is delivered from host to guest by using PPI (Private Peripheral Interrupt). The presentation will explain the design and implementation of SDEI virtualization and asynchronous page fault. Besides, the benchmarks will be presented either to show how much the guest can benefit from the features.

Speakers

Gavin Shan

Senior Software Engineer, Redhat

Gavin is currently employed by Redhat to work on KVM support for arm64. The most intrested areas spans from memory management to live migration and QEMU etc. Prior to that, Gavin has been working on Linux kernel and driver for over 10 years, used to be employed by IBM to maintain... Read More →

sdei apf for arm64 gavin pdf

Wednesday September 15, 2021 05:30 - 05:55 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

05:30 UTC

QEMU Emulated NVMe - Lessons Learned and Future Work - Klaus Jensen, Samsung Electronics

Planning the design of an emulated device in QEMU can be a daunting task and it can be hard to gain an upfront overview of how to proceed with wiring up a new device. In the context of the emulated NVMe device, this talk aims to present such an overview. The QEMU 6.0 release represents a milestone for the emulated NVMe device and add support for a number of advanced features such as multipath I/O and namespace sharing, metadata and data integrity. The implementation of these features require the use of many internal QEMU features and some were initially used incorrectly, causing frustration to developers and users alike. The focal point of this talk is the lessons learned in regard to QDev and the QEMU Object Model, asynchronous I/O, mistakes that were made, and how these are rectified. Finally, this talk will take the opportunity to suggest the idea of using the device as a thin simulator for measuring latencies in host software and drivers. The emulating (and not simulating) nature of QEMU posits several interesting challenges and members of the audience with any experience in this field are encouraged to pitch in and provide feedback on the viability of the ideas presented.

Speakers

Klaus Jensen

Staff Engineer, Samsung Electronics

Klaus Jensen is a software engineer with a background in academia. He has worked in the state-of-the-art area of High Performance Computing, avoided users as an old school conservative UNIX sysop, written a Ph.D. thesis on tape and been involved in the OpenChannel SSD community, and... Read More →

jensen qemu emulated nvme pdf

Wednesday September 15, 2021 05:30 - 05:55 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

06:00 UTC

Analysis of AMD HW-assisted vIOMMU Implementation and Performance - Suravee Suthikulpanit & Wei Huang, AMD

This talk will cover the software implementation details of AMD hardware-assisted vIOMMU (HW-vIOMMU), which includes the QEMU HW-vIOMMU device model, the extension to AMD IOMMU driver to support the new HW-vIOMMU capability in the hypervisor, and the guest I/O page table support for DMA-API in the guest kernel. The talk will also demonstrate passing-thru of a PCI device, which will be managed by the HW-vIOMMU inside the VM, and will also analyze performance data of the HW-vIOMMU in various aspects, as well as discuss future improvements.

Speakers

Suravee Suthikulpanit

Open-Source Contributor, AMD

Suravee Suthikulpanit works for AMD Server Software Group. His work mainly focus on Linux kernel and the open-source virtualization software. Within AMD, Suravee works with the hardware design and performance teams on future feature definitions. Suravee has been a regular contributor... Read More →

Wei Huang

Open-Source Contributor, AMD

Wei Huang is a member of AMD Server Software Group, with current focus on server OS and x86 virtualization. Wei has contributed to Linux kernel and various open source virtualization projects (Xen, KVM/QEMU, etc.), and presented a number of times at various technical conferences... Read More →

vIOMMU KVM Forum 2021 v4 pdf

Wednesday September 15, 2021 06:00 - 06:25 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

06:00 UTC

Debugging Secured Windows OS guest using KVM/QEMU and Windbg - Marek Kędzierski, Red Hat

In recent years Microsoft has been investing in making Windows OS as secure as possible. Features like Secure Boot or Virtualization Based Security, when enabled, make the entire system or part of that impossible to debug using system debugger Windbg. The only way of debugging a secured system is using a special hardware debugger that integrates with native debugger. During this presentation, a tool that provides similar functionality will be presented. It doesn't require any specific hardware as it uses KVM/QEMU to debug secured guests using Windbg.

Speakers

Marek Kędzierski

Senior Software Engineer, Red Hat

Software Engineer working at Red Hat in virtio-win Team.

Wednesday September 15, 2021 06:00 - 06:25 UTC
TBA

KVM Track 2

06:30 UTC

KVM Memory Cost Optimization in Alibaba Cloud - Huaitong Han, Alibaba Cloud

In the current VM life cycle, in addition to the memory used by the guest os, QEMU/KVM also uses a lot of memory, especially the KVM module, which takes up memory in proportion to the VM memory specification, especially for lightweight VMs such as Kata, etc., KVM occupies too much memory resources, which is difficult to accept. In this introduction, Huaitong will introduce the current KVM module memory usage and Alibaba Cloud's KVM module memory optimization practices.

Speakers

Huaitong Han

virtualization developer, Alibaba Cloud

Huaitong has 7 years experience in system virtualization, he is working for alibaba cloud now, and focus on virtualizaiton performace optimization and ECS feature development.

KVM memory cost optimize huaitong pdf

Wednesday September 15, 2021 06:30 - 06:55 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

06:30 UTC

Passthrough/Headless GPU Gets Ahead - Tina Zhang & Vivek Kasireddy, Intel

GPU passthrough technology guarantees users nearly full GPU performance within a virtual machine(VM), by giving a VM the direct access to GPU hardware. Since a passthrough GPU is fully controlled by guest, usually it’s non-trivial to share a passthrough GPU’s frame buffers with host-controlled devices. In the client virtualization field, there’s a trend to share guest applications’ or compositor’s frame buffers with the host display compositor, so that those guest frame buffers can be composited and posted to the display monitor. With a passthrough GPU, the guest frame buffer sharing could be very challenging, especially considering frame buffers 0-copy and synchronization between the guest applications and host display compositor. In this session, we’re going to present a solution that combines passthrough GPU with a paravirtualization display provided by virtio-gpu to achieve the frame buffers 0-copy goal as well as the synchronization. We will talk about the architecture design, the implementation challenges as well as some use cases based on the solution.

Speakers

Tina Zhang

Software Engineer, Intel

Tina is a software engineer from Intel virtualization team. She has 6 years experience in embedded system software development and 8 years experience in system virtualization and GPU virtualization. Tina was a speaker of KVM Forum 2017.

Vivek Kasireddy

Graphics Software Engineer, Intel

Vivek works on multiple components of the Graphics stack in Intel's IOTG team. His recent focus has been on enabling novel Graphics Virtualization use-cases and features. He is currently working on augmenting and improving the performance of virtio-gpu drivers in the kernel and Q... Read More →

Passthrough Headless GPU Gets Ahead pdf

Wednesday September 15, 2021 06:30 - 06:55 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

07:15 UTC

Break

Wednesday September 15, 2021 07:15 - 11:00 UTC

Breaks & Networking

10:00 UTC

BoF Sessions

Please view the Announcement section within the Accelevents event lobby for details on how to join.

Wednesday September 15, 2021 10:00 - 10:55 UTC

BoF Sessions

11:00 UTC

Mitigating Excessive Pause-Loop-Exiting in VM-Agnostic KVM - Kenta Ishiguro, Keio University

In virtualized environments, oversubscribing virtual CPUs (vCPUs) on physical CPUs (pCPUs) is common to utilize CPU resources efficiently. Unfortunately, excessive vCPU spinning, which occurs when a vCPU is waiting in a spin loop for an event from a descheduled vCPU, causes serious performance degradation. VM-agnostic KVM tries to prevent excessive vCPU spinning by rescheduling vCPUs when an excessive spin is detected by pause-loop-exiting. This talk explains how the KVM vCPU scheduler fails to avoid excessive vCPU spinning in many opportunities. Three problems have been identified: 1) scheduler mismatch, 2) lost opportunity, and 3) overboost. The first problem comes from the mismatch between the KVM vCPU scheduler and the Linux scheduler. The second and third problems come from an inefficient algorithm for choosing the next candidate vCPU to be scheduled. Simple modifications gracefully resolve the problems and the performance improves by up to 80 %. Results imply the VM-agnostic hypervisor can resolve excessive vCPU spinning more gracefully than previously believed.

Speakers

Kenta Ishiguro

Ph.D student, Keio University

Kenta Ishiguro is a Ph.d student in the department of information and computer science at Keio university, advised by Kenji Kono. He is working on KVM performance analysis related to Pause-loop-exiting. He has made two academic contributions to KVM: "Mitigating Excessive vCPU Spinning... Read More →

kvmforum 2021 kenta ishiguro pdf

Wednesday September 15, 2021 11:00 - 11:25 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

11:00 UTC

How Hard Could it be to Flip a bit? KVM PV Feature Enablement up the Virtualization Stack - Vitaly Kuznetsov, Red Hat

KVM may not be the most paravirtualized feature-rich hypervisor out there but it still implements a lot. There are 'native' KVM features as well as Hyper-V, Xen and even Vmware emulation built-in. While the implementation bits usually live in KVM itself, userspace VMMs are still need to get involved. Despite the fact that enabling a single feature may be as easy as flipping a single bit, the 'when to flip' decision may be challenging. Upstream, developers are trying their best to deliver new features to as many users as possible but different users' needs, setups, 'legacy' requirements and ease of configuring should be taken into account. The talk will use QEMU as an example VMM. Several 'Native' KVM and Hyper-V emulation features will be looked at from 'enablement' perspective: who could benefit from it, what are the software/hardware requirements, how does the feature blend with other features, namely: with live migration.

Speakers

Vitaly Kuznetsov

Principal Software Engineer, Red Hat

Vitaly works at Virtualization Engineering team at Red Hat focusing on KVM development as well as making Linux the best guest for other hypervisors. He frequently presents at FOSDEM, KVM Forum, DevConf and other technical conferences.

KVMForum2021 PV Enablement pdf

Wednesday September 15, 2021 11:00 - 11:25 UTC
TBA

KVM Track 2

11:30 UTC

Towards a More Efficient Synchronization in KVM - Wanpeng Li, Tencent Cloud

The KVM hypervisor is at the core of cloud computing, Virtual Machine based approaches to workload consolidation, as seen in IaaS cloud as well as datacenter platforms, have long had to contend with performance degradation caused by synchronization primitives inside the guest environments. These primitives can be affected by virtual CPU preemptions by the host scheduler that can introduce delays that are orders of magnitude longer than those primitives were designed for. In this presentation, Wanpeng Li will discuss lost opportunities in vCPU boost, vCPUs stacking, and RCU-Reader Preemption Problem in VMs etc. Let's have a look at their mitigations.

Speakers

Wanpeng Li

Linux Kernel Contributor, Tencent Cloud

Wanpeng Li is a 9 years experienced Linux kernel/virtualization developer who works in Tencent Cloud currently. He mainly focuses on KVM, scheduler and memory management. In KVM, he contributes a lot of features to improve performance and stability. He has worked in the IBM LTC kernel... Read More →

Towards a More Efficient Synchronization in KVM pdf

Wednesday September 15, 2021 11:30 - 11:55 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

11:30 UTC

The Traps of Using Hyper-V Features in KVM Environment - Liang Li, Alibaba

Windows guest is still widely used in public cloud, and it is irreplaceable in cloud games and cloud desktop scenarios. In order to improve its performance in the KVM virtualization environment, a series of Hyper-V related features have been gradually added. In this session, Liang will introduce the traps encountered when using Hyper-V related features in the production environment, as well as the related methods to avoid the traps. At the end of this session, he will give some suggestions for the improvements

Speakers

Liang Li

Senior Expert, Alibaba

Liang has worked in the area of system virtualization for many years. He had ever gave two talks about live migration on KVM forum in 2015 and in 2016. In KVM forum 2020, he gave a talk about GPU VM creation time optimization

The Traps of Using Hyper V Features in KVM Environmen pdf

Wednesday September 15, 2021 11:30 - 11:55 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

12:05 UTC

Break

Wednesday September 15, 2021 12:05 - 13:00 UTC

Breaks & Networking

13:00 UTC

Keynote Session: Virtio TC Bulletin - Michael S. Tsirkin, Red Hat Inc.

This talk will present new developments in the virtio TC since the last KVM forum: from charter changes, to new devices and features of existing ones.

Speakers

Michael S. Tsirkin

Distinguished Engineer, Red Hat

Michael has been with Red Hat for more than 10 years. In his role as a Distinguished Engineer he acts as a chair of the Virtio Technical Committee, overseeing the development of the virtio specification for virtual devices. He also maintains several subsystems in QEMU and Linux and... Read More →

virtio bulletin michael s tsirkin pdf

Wednesday September 15, 2021 13:00 - 13:15 UTC

Keynote Sessions

Presentation Slides Attached Yes

13:15 UTC

Keynote Session: QEMU Status Update - Alex Bennée, Linaro

It has been yet another unprecedented year of people working from home while toiling on the QEMU code base. But how can we know what has been achieved in the last year without reading the fine detail of the change logs for ourselves? Join Alex Bennée as he delves into the last 12 months of development activity and condenses the experience into an easily digestible summary. There will be data, graphs, and the all-important tables as we explore what it is we've all been doing.

Speakers

Alex Bennée

Virtualisation Tech Lead, Linaro

Alex started learning to program in the 80s in an era of classic home computers that allowed you to get down and dirty at the system level. After graduating with a degree in Chemistry he's worked on a variety of projects including Fruit Machines, Line Cards, CCTV recorders and point-to-multipoint... Read More →

QEMU Status Report pdf

Wednesday September 15, 2021 13:15 - 13:30 UTC

Keynote Sessions

Presentation Slides Attached Yes

13:35 UTC

Dirty Quota-Based VM Live Migration Auto-Converge - Manish Mishra & Shivam Kumar, Nutanix India

The current qemu auto-converge implementation works by throttling all the vcpus based on dirty-rate and network-throughput observed over an iteration. This leads to - all the vcpus getting equally penalized, multiple iterations to get to the optimal throttle value, and convergence failure when >99% throttle is required. This talk proposes a new algorithm based on individual vcpu “dirty quota” that dynamically throttles individual vcpus based on their contribution to overall memory dirtying. It does so by enforcing an upper limit(dirty-quota) on amount of pages a vcpu is allowed to dirty within a small interval of time to limit the total pages dirtied over an iteration. The advantage of this approach is that it penalizes only the write-intensive vcpus, dynamically adapts in case of varying network and dirty-rate and improves convergence by allowing more granular control over dirty rates.

Speakers

Shivam Kumar

Member of Technical Staff, Nutanix

I work for the AHV team at Nutanix where most of my work to date is around improving live migration algorithms. Other than live migration, my interests lie in systems and virtualization in general.

Manish Mishra

Member of Technical Staffd, Nutanix India

Experience of working on Esx kernel. Currently working with vmware Vmkernel team contributing to vmkcore part. Knowledge of memory management, intel 86 architecture, process management, os booting, basic filesystems, kernel debugging. Continuously learning and passanate about virtualisation... Read More →

Dirty Quota Based VM Live Migration Auto Converge pptx

Wednesday September 15, 2021 13:35 - 14:00 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

13:35 UTC

New Qemu Backup Architecture and API - Vladimir Sementsov-Ogievskiy, Virtuozzo

Qemu block-layer moves to node-oriented APIs, user gets abilities to control the whole block graph. New approach allows to insert and remove filter drivers dynamically (for example, to enable throttling or copy-on-read). New backup API goes the same way: user could use copy-before-write filter to setup fleecing scheme in separate, may combine it with backup to setup push-backup-with-fleecing or with NBD export to setup pull-backup. Simple building blocks which are combined to solve complex task. The presentation will describe new interfaces, intended usage and some internals of Qemu backup.

Speakers

Vladimir Sementsov-Ogievskiy

software developer, Virtuozzo

Vladimir works in Virtuozzo, maintaining qemu-kvm-vz package and deals in the main with Qemu block layer, developing features around backup, qcow2 and NBD. Vladimir is a co-maintainer of NBD and Block Jobs subsystems in Qemu. Some ongoing projects are: filter-based backup architecture... Read More →

kvm forum 2021 vsementsov pdf

Wednesday September 15, 2021 13:35 - 14:00 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

14:05 UTC

KVM Dirty Page Tracking - Peter Xu, Red Hat

In this talk, Peter Xu will discuss the existing solutions used in KVM for guest dirty page tracking, challenges, and wild imaginations to the future. As a start, there will be a background introduction on what is KVM dirty page tracking, why we need it, and how it works. It will cover the state of the art techniques we already use with either dirty logging or dirty rings, meanwhile discuss pros and cons for different solutions. The challenges that we face on huge VM migrations will be discussed and explained too. It turns out that maybe having a better interface is not the only thing we need to do, but also think about how to use it right.

Speakers

Peter Xu

Software Engineer, Red Hat

Peter Xu is a software engineer working for Red Hat Virtualization team. He's recently working on VM live migrations and some memory management problems of VMs or hosts.

KVM Dirty Page Tracking pdf

Wednesday September 15, 2021 14:05 - 14:30 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

14:05 UTC

Background Snapshots in QEMU: Towards Asynchronous Revert - Denis Lunev, Virtuozzo

Background snapshots is a new feature in QEMU-6.0 designed to allow making snapshots on a live VM without need for dirty page tracking. It makes possible to capture machine state at the time the snapshot operation was initiated while keeping VM running. However making snapshot loading asynchronous is not a trivial task. This talk covers some technical complications that need to be resolved: - Container for snapshot data, how to address RAM pages? - Open-coded migration handlers in QEMU, can we re-use them? - Write buffering, is writeback cache a good choice? - Making revert really live, what's good tradeoff between speed and fault-in latency? - How to split snapshot loading into stages? Is it enough to have precopy and postcopy?

Speakers

Denis Lunev

Team Lead, Virtuozzo

Denis Lunev is working in Virtuozzo around 20 years dealing with various aspects of virtualization, both in virtual machine and container worlds. Right now is working on QEMU optimizations.

kvm2021 background snapshot pdf

Wednesday September 15, 2021 14:05 - 14:30 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

14:35 UTC

Know your QEMU and KVM Test Frameworks - Thomas Huth, Red Hat

QEMU and KVM are huge projects - and so there are of course also multiple test frameworks for the various parts of the project: KVM selftests, kvm-unit-tests, qemu qtests, iotests, avocado etc. For newcomers, but also for frequent contributors to the projects, it can be quite challenging to understand and pick the right framework for the topics they are working on. This talk will give a survey over the available testing frameworks, to help with the decision of which one should be chosen for which task, along with some simple examples for adding new tests.

Speakers

Thomas Huth

Senior Software Engineer, Red Hat

I'm working as a software engineer at Red Hat in the virtualization team, focusing on KVM / QEMU on the s390x platform. I'm involved in various testing efforts of the virtualization stack, like the gitlab-CI of QEMU, the qtest framework and the kvm-unit-tests. With regards to my previous... Read More →

Huth Know your QEMU and KVM test frameworks pdf

Wednesday September 15, 2021 14:35 - 15:00 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

14:35 UTC

Lessons Learned Building a Production Memory-Overcommit Solution - Florian Schmidt & Ivan Teterevkov, Nutanix

Linux is a great hypervisor platform, but it does not yet provide a complete solution for overcommitting VM memory efficiently. We present lessons learned from implementing a self-adapting memory-overcommit solution based on QEMU/kvm and libvirt. We discuss the drawbacks of different memory backends, such as hugetlbfs and memfd, the challenges of coordinating memory ballooning and hypervisor swap via cgroups, and how out-of-the-box live-migration performance can be catastrophic without careful per-page reclamation hints. In addition, we describe how to collect VM memory stats to estimate memory needs, especially in the absence of metrics provided by balloon drivers. This includes leveraging idle page tracking to obtain insights into VM memory usage, using sampling to avoid excessive CPU overhead. Finally, we explain how to tie those input metrics and output knobs together to produce a complete solution for adaptively overcommitting VM memory efficiently in production systems.

Speakers

Florian Schmidt

Staff Engineer, Nutanix

Florian is a software engineer working on Nutanix's AHV hypervisor suite. Before that, he was one of the main developers of the Unikraft unikernel. During his Ph.D. work, he focused on network protocols and Xen virtualisation. Technical conferences he has spoken at include previous... Read More →

Ivan Teterevkov

Senior Member of Technical Staff, Nutanix

Currently working at Nutanix on memory overcommit for Acropolis Hypervisor, a KVM and QEMU based product. Previously worked on various IoT and SaaS projects and has more than ten years of software development experience in Linux.

2021 KVM Forum Memory Overcommit pdf

Wednesday September 15, 2021 14:35 - 15:00 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

15:05 UTC

Qemu Performance Regression CI - Lukáš Doktor, Red Hat Czech, s. r. o.

Motto: "An improvement is BAD in the same way a regression is." This talk builds on the "Proposal for a regular upstream performance testing" email thread on qemu-devel and describes what is being done, why it’s important and suggests a BoF session to discuss use cases and make any necessary adjustments. After a brief introduction of our upstream performance regression CI the talk is going to move towards the practical aspects useful for (mainly) qemu developers, explaining the public reports, the collected metrics, how to make sense of the issues and how to reproduce the runs.

Speakers

Lukas Doktor

Software Developer, Red Hat Czech, s. r. o.

He's part of a CI group within the virtualization team in Red Hat. Long-time Avocado/Avocado-vt maintainer with a primary focus on alternative architectures. Used to maintain an internal alt-arch CI, now trying to establish a regular performance regression CI within the same team... Read More →

ldoktor qemu perf regression ci pdf

Wednesday September 15, 2021 15:05 - 15:30 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

15:05 UTC

Support for Fast and Reliable VMM Live Upgrades in Libvirt - Soham Ghosh & Prachatos Mitra, Nutanix

Upgrading VMMs with live VMs has always been a challenging problem. The currently accepted solution is live migrating VMs outside the host and then upgrading the VMM before the VMs are migrated back. This enforces the cloud infrastructure to maintain spare resources on other nodes to accommodate the incoming VMs. Along with this it incurs huge cost on the upgrade time as well as network usage. In an effort to reduce these costs we propose a novel approach in the upgrade workflow, which can upgrade the VMM without the need to migrate VMs outside of the host. This talk focuses on how with minimal changes in the existing libvirt migration workflow we can transfer the VMs to the new upgraded qemu binary with minimal downtime while preserving external connections and device states.

Speakers

Soham Ghosh

Member of Technical Staff, Nutanix

Soham is a software engineer working for the hypervisor team in Nutanix with interests in areas of virtualization, performance and live migration. Soham completed his M.E degree in Computer Science from IISc.

Prachatos Mitra

Member of Technical Staff, Nutanix

Prachatos is a software engineer working for the hypervisor team in Nutanix with interests in areas of virtualization, operating systems and live migration. Prachatos completed his M.Tech degree in Computer Science from IISc.

VMM Live upgrade (KVM forum) pdf

VMM Live upgrade (KVM forum) pptx

Wednesday September 15, 2021 15:05 - 15:30 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

15:35 UTC

HCK-CI: Enabling CI for Windows Guest Paravirtualized Drivers - Kostiantyn Kostiuk, Daynix Computing LTD

In order to be able to accept contributions from different developers to virtio-win (http://github.com/virtio-win/kvm-guest-drivers-windows/) project, there is a need to ensure that those contributions are not breaking the ability to certify the virtio-win drivers by different members of the ecosystem. As a result, the HCK-CI test framework was created in order to enable CI for all the types of virtio-win drivers on a wide range of Windows OS versions. The framework automates setup creation (VM and network orchestration), uses HLK\HCK tools kits API in order to run Microsoft WHQL certification tests, and publishes the results in human-readable form. During the presentation, Konstantin will review the history of the project, explain the architecture of HCK-CI, demonstrate how you can deploy it in your development setup, and talk about the future of the project.

Speakers

Kostiantyn Kostiuk

Software engineer, Daynix Computing LTD

Kostiantyn is a SW engineer at Daynix. For the last several years he is working on security-based virtualization focusing on QEMUKVM related projects. Kostiantyn is lately involved in introducing CI capabilities for paravirtualized drivers based on the HCK-CI project: https://github.com/hck-ci... Read More →

HCK CI KVM Forum pdf

Wednesday September 15, 2021 15:35 - 16:00 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

15:35 UTC

Kubevirt and the Cost of Containerizing VMs - Guoqing Li, Nara Institute of Science and Technology & Dario Faggioli, SUSE

KubeVirt is an add-on for Kubernetes to manage both containers and VMs in a unified manner. With KubeVirt, however, Libvirt, QEMU and all the VM processes run in Kubernetes pods and this may introduce some overhead. For instance, resource usage accounting or limitation done by the container runtime, as well as longer (disk and network) IO paths might slow down KubeVirt VMs. For this talk, Guoqing Li, Dario Faggioli and Vasiliy Ulyanov will investigate the performance characteristics of VMs running in Kubernetes pods, using CPU, memory, disk and networking benchmarks and comparing that with plain KVM VMs. We will check the effect that tuning the configuration of the plain VMs has, for these benchmarks, and show whether it is possible to achieve something similar for VMs running with KubeVirt.

Speakers

Guoqing Li

Graduate Student, Nara Institute of Science and Technology

Guoqing Li is a graduate student at the Nara Institute of Science and Technology, Japan. He obtained his bachelor's degree in software engineering with first class honors at Chiang Mai University, Thailand. He contributed to the SaltStack open source project under Google's Summer... Read More →

Dario Faggioli

Virtualization Engineer, SUSE

Dario is a Virtualization Software Engineer at SUSE. He's been active in the Open Source virtualization space for a few years. Within the Xen-Project, he is still the maintainer of the Xen hypervisor scheduler. He also works on Linux kernel, KVM, Libvirt, and QEMU. Back during his... Read More →

KubeVirt and the Cost Of Containerizing VMs pdf

Wednesday September 15, 2021 15:35 - 16:00 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

20:00 UTC

BoF Sessions

Please view the Announcement section within the Accelevents event lobby for details on how to join.

Wednesday September 15, 2021 20:00 - 20:55 UTC

BoF Sessions

05:00 UTC

Protecting from Malicious Hypervisor Using AMD SEV-SNP - Brijesh Singh, AMD

This talk will discuss AMD SEV-SNP (Secure Nested Paging), the next generation of AMD’s x86 virtualization isolation technology. Building upon the existing AMD SEV and AMD SEV-ES features released in 2017, SEV-SNP provides additional hardware security that is designed to protect VMs from malicious hypervisors. SEV-SNP adds new memory integrity protection, new use models, and more flexibility in attestation and VM management when working with protected VMs in hostile environments. This talk will delve into the specific changes required in the KVM to support the SEV-SNP feature.

Speakers

Brijesh Singh

SMTS, Advanced Micro Devices

Brijesh Singh is a member of the Linux OS group at Advanced Micro Devices. He is responsible for enabling and enhancing support for AMD processor features in the Linux kernel. He is currently working on extending the SEV support to enable SEV-SNP (Secure Nested Paging).

SEV SNP AMD Brijesh Singh pdf

Thursday September 16, 2021 05:00 - 05:25 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

05:00 UTC

Receive Side Scaling (RSS) with eBPF in QEMU and virtio-net - Yan Vugenfirer, Daynix

eBPF is a revolutionary technology that can run sandboxed programs in the Linux kernel without changing kernel source code or loading a kernel module. Receive side scaling (RSS) is the mechanism of packet steering for multi-queue NICs optimizing multiple CPU utilization. The first usage of eBPF in QEMU is the optimization of the RSS packet steering in virtio-net. During this session, Yan will provide the motives for the RSS optimization using eBPF, review the technical solution, describe integration with libvirt, and discuss future development and additional usages of eBPF in QEMU.

Speakers

Yan Vugenfirer

CEO, Daynix

Yan is the CEO of Daynix Computing. He is an upstream maintainer of the virtio-win drivers https://github.com/virtio-win/kvm-guest-drivers-windows/. Yan has more than 20 years of kernel development and 14 years of virtualization-related development.

Receive side scaling (RSS) with eBPF in QEMU and virtio net pdf

Thursday September 16, 2021 05:00 - 05:25 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

05:30 UTC

Status Update on TDX Support - Isaku Yamahata, Intel

The Intel Trust Domain eXtensions(TDX) is to isolate VMs from VMM and other software to protect VMs from abroad range of software. It requires to enable KVM to support it. This talk will update the status of KVM TDX enablement activities from the last year. After providing an overview of the technology as introduction, the overall progress will be provided and then move on to details. newly supported features, and enhancement to KVM and then qemu change will be discussed. Changes to guest Linux kernel will be discussed. Finally the future plan will be shown as conclusion.

Speakers

Isaku Yamahata

Software Engineer, Intel

Isaku Yamahata is a Software architect in the Open Source Technology Center, Intel. His main focus is virtualization technology, network virtualization as Software Defined Networking for multiple years. Isaku is an active on Graphene LibOS and OpenStack Neutron (networking) and has... Read More →

tdx status update pdf

Thursday September 16, 2021 05:30 - 05:55 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

05:30 UTC

Towards High-availability for Virtio-fs - Jiachen Zhang & Yongji Xie, ByteDance

Virtio-fs is getting widely used, especially for VMs that are running secure containers. To increase the reliability of the services relying on Virtio-fs in large production deployments, we extend Virtio-fs implementation to support high-availability features includes virtiofsd crash recovery, virtiofsd live-upgrade, and VM live-migration when using Virtio-fs. This talk will first introduce the use cases of Virtio-fs, and why Virtio-fs high-availability features are essential. Then, the talk will detail the designs and implementations of the features. Finally, the talk will state the current limitations and future development plans of these features.

Speakers

Jiachen Zhang

Software Engineer, ByteDance

Jiachen Zhang is a Software Engineer at ByteDance, focus on storage virtualization systems.

Yongji Xie

Software Engineer, ByteDance

Yongji Xie is a software engineer at ByteDance, working on I/O virtualization topics in QEMU and Linux kernel.

virtiofs ha jiachen pdf

Thursday September 16, 2021 05:30 - 05:55 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

06:00 UTC

TDX Live Migration - Wei Wang, Intel Corp.

The Intel Trust Domain Extension (TDX) technology extends VMX and MKTME to enhance guest data security by isolating guests from host software, including VMM/hypervisor. Live migration support for such isolated guests (i.e. TDs) facilitates the deployment of TD guests in the cloud. This talk presents the QEMU/KVM design of TDX live migration and initial PoC results for the migration performance evaluation. A common framework is added to the QEMU migration to support TD guests and other similar technologies (e.g. SEV guests). For TDX live migration, the guest shared memory pages are migrated in plaintexts. The guest private memory pages, vCPU states and TD scope states are encrypted via a migration key when they are exported by KVM from the TDX module. A migration stream in the workflow has a KVM device created and the device creates shared memory between KVM and the QEMU migration thread to transport the encrypted guest states.

Speakers

Wei Wang

Senior Software Engineer, Intel Corp.

Wei is currently a software developer at Intel. He earned a Master degree from the University of Ottawa, Canada. Wei has rich experience in the virtualization field and he worked on many projects such as network virtualization, VM live migration, memory ballooning, PMU virtualization... Read More →

TDX Live Migration Wei Wang pdf

Thursday September 16, 2021 06:00 - 06:25 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

06:00 UTC

Hyperscale vDPA - Jason Wang, Red Hat

As the requirements of assigning each VM/container a vDPA instance increases, various ways of scaling vDPA to more than 10K instances per host will be introduced in this talk. The first method is to achieve this through the co-operation with technologies provided by the cpu/chipset vendor like Scalable-IOV. The second method is done via vDPA vendor specific technology such as scalable functions. The third method is to extend the virtio specification for device slicing support. In this talk, a brief overview of vDPA will be given then a in depth discussion of the above three ways will be done. In the end, a comparison of the those three technology will be presented.

Speakers

Jason Wang

Senior Principal Software Engineer, Red Hat

Co-maintainer of kernel virtio, vdpa and vhost drivers. Maintainer of Qemu networking subsystems. Author of vDPA support in Kernel.

hyper scale vDPA pdf

Thursday September 16, 2021 06:00 - 06:25 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

06:30 UTC

Unmapped Guest Memory - Yu Zhang, Intel

Historically guest memory can be seen by userspace part of VMM like QEMU in KVM architecture. This is convenient for userspace to handle services on behalf of guest. However, this ability also is seen as increasing guest attack surface and with more strict hardware memory protection features like Intel TDX introduced such ability can cause real system issue like system crash. In this talk we will present our way to address this problem that is being discussed in community.

Speakers

Yu Zhang

Virtualization Developer, Intel

Yu is a virtualization developer from Intel's virtualization team. He had 10+ years’ experiences in virtualization areas from I/O to CPU/memory virtualization, from performance tuning to security enhancements. Yu’s public presentation experience includes Xen summit/LC3 conference/Intel... Read More →

Yu Zhang KVM Forum Unmapped Guest Memory pdf

Thursday September 16, 2021 06:30 - 06:55 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

06:30 UTC

VDUSE - vDPA Device in Userspace - Yongji Xie, ByteDance

vDPA (virtio data path acceleration) device is a device that uses a datapath which complies with the virtio specifications with vendor specific control path. vDPA devices can be both physically located on the hardware or emulated by software. This talk will introduce VDUSE (vDPA device in userspace) which is a Linux driver that builds on vDPA kernel subsystem to provide a framework to implement software-emulated vDPA devices in userspace. The goal of this technology is to provide a new userspace approach for providing an unified storage/networking services for both container and VM workloads. We will discuss the design and implementation of kernel VDUSE module and give some examples to show how to emulate a vDPA device in userspace with VDUSE.

Speakers

Yongji Xie

Software Engineer, ByteDance

Yongji Xie is a software engineer at ByteDance, working on I/O virtualization topics in QEMU and Linux kernel.

YongjiXie KVM VDUSE (1) pdf

Thursday September 16, 2021 06:30 - 06:55 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

07:15 UTC

Break

Thursday September 16, 2021 07:15 - 11:00 UTC

Breaks & Networking

10:00 UTC

BoF Sessions

Please view the Announcement section within the Accelevents event lobby for details on how to join.

Thursday September 16, 2021 10:00 - 10:55 UTC

BoF Sessions

11:00 UTC

libkrun: More than a VMM, in Dynamic Library Form - Sergio Lopez Pascual, Red Hat

libkrun is a dynamic library that enables programs to easily acquire Virtualization-based process isolation capabilities. Combined with an OCI runtime like crun, enables it to seamlessly run a container inside a Virtual Machine. Combined with a small frontend, such as krunvm, it makes possible to run Lightweight Virtual Machines based on OCI images. And that's just the start of it, as new use cases for libkrun continuously emerge, such as its use as a lightweight runtime for Confidential Computing workloads. In this session, Sergio Lopez will present libkrun's main components, design choices and future developments, along with a demonstration of its current capabilities and integrations in other projects.

Speakers

Sergio Lopez Pascual

Principal Software Engineer, Red Hat

Sergio Lopez is a Principal Software Engineer working in the Virtualization team at Red Hat. He's the maintainer of the "microvm" machine type in QEMU, libkrun, krunvm, virtiofsd-rs, and co-maintainer of various rust-vmm crates. He presented previously at various iterations of DevConf.cz... Read More →

libkrun More than a VMM in Dynamic Library Form pdf

Thursday September 16, 2021 11:00 - 11:20 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

11:00 UTC

Hypervisor-less Virtio for Real-time and Safety - Maarten Koning, Wind River

There are a variety of approaches to leveraging Linux in real-time and safety-critical applications, such as those required for planes, trains, automobiles, and robots. Some approaches utilize tick-less PREEMPT_RT kernels that isolate cores for user-level processes, some introduce user-level or virtualization-based unikernels, some leverage Linux CPU hot-plug features to offload cores to auxiliary runtimes or bare metal applications, and some defer to secondary CPU clusters and run realtime or safety workloads on compute islands as found in heterogeneous SoCs. This technical presentation covers emerging “hypervisor-less virtio” technology and its resultant unifying system architecture for the sharing of resources, such as files, tty, IPC, network interfaces and others, whether between processes, kernels, cores and/or CPU clusters - with Linux as the virtio backend for all the approaches mentioned above. Although this talk goes deep but it is also broadly relevant to computer scientists and technology leaders who create, deliver, and capture embedded software value using a Linux-first approach to real-time and/or safety-based applications.

Speakers

Maarten Koning

Fellow, Wind River

Maarten joined Wind River when they acquired his DSP start-up and has since worked on real-time, virtualization, distributed and partitioned systems, safety-critical systems and development tooling. A self-described professional nerd, Maarten has a passion for enabling computers to... Read More →

kvmForumV2 (1) pdf

Thursday September 16, 2021 11:00 - 11:25 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

11:20 UTC

Don't Peek Into my Container! - Alice Frosi, Christophe de Dinechin & Sergio Lopez Pascual, Red Hat

"Confidential containers" is the application of such technologies to protect the data in containers. This matters for use cases where the "tenant" running the workloads has legal or business reasons to want the data being processed to be hidden from the infrastructure it is running on.

We will focus on the interaction between container runtimes and KVM, using Kata Containers and libkrun as two example implementations. This will expose both technical and market challenges enabling technologies
such as AMD-SEV or Intel TDX, that differ in their availability, capabilities but also in the way they perform attestation.

In this talk, we are going to explore how you can convert your containerized application into an encrypted workload using libkrun, KVM, and Kubernetes. You will learn the designed architecture in k8s, the challenges we face in deploying an attested and confidential workload by keeping the user experience agile as the usual container deployments.

We will also quickly show how Kata Containers recently added platform-level support, and how we plan to more significantly overhaul its architecture in order to deliver a solid value proposition in terms of security.

Speakers

Christophe de Dinechin

Senior Principal Software Engineer, Red Hat

Christophe de Dinechin works at Red Hat primarily on Kata Containers and its integration into OpenShift, as well as on Confidential Containers. He co-presented a talk at the KVM Forum 2021 titled "Don't peek into my container". He also has a strong interest in virtualisation, performance... Read More →

Alice Frosi

Senior Software Engineer, Red Hat

Alice Frosi is a Senior Software Engineer working in the Virtualization team at Red Hat. She is currently working in the Kubevirt project.

Sergio Lopez Pascual

Principal Software Engineer, Red Hat

Dont Peek Into my Container pdf

Thursday September 16, 2021 11:20 - 12:00 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

11:30 UTC

vdpa-blk: Unified Hardware and Software Offload for virtio-blk - Stefano Garzarella, Red Hat

vDPA (virtio Data Path Acceleration) is a novel framework designed to simplify the software stack for hardware acceleration of devices. A vDPA device provides a data path fully compliant with the virtio specification which is suitable for implementation in hardware, and a vendor-specific control path. The abstraction provided by vDPA also makes software accelerators possible, similar to existing vhost or vhost-user devices.
Leveraging the potential of vDPA, Stefano will present a new vdpa-blk software device running in the host kernel that unifies hardware and software accelerators software stack. This new device draws on the experience with io_uring passthrough presented at the last KVM Forum, with a goal of achieving high performance. Accelerators bypass QEMU and therefore preclude the use of the features provided by QEMU block layer, such as image formats (e.g qcow2), I/O throttling, snapshots, etc. To keep using these features, Stefano will discuss an idea to detect when block layer functionality is needed and automatically use it with hardware or software vdpa-blk accelerators, leveraging the device emulation provided by QEMU.

Speakers

Stefano Garzarella

Senior Software Engineer, Red Hat

Stefano is a Senior Software Engineer at Red Hat. He is working on virtualization and networking topics in QEMU and Linux kernel. He is the maintainer of Linux's VM Sockets (AF_VSOCK). Current projects cover vDPA for block devices, VIRTIO's virtqueue passthrough, and QEMU storage... Read More →

KVMForum 2021 vdpa blk Stefano Garzarella pdf

Thursday September 16, 2021 11:30 - 11:55 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

12:05 UTC

Break

Thursday September 16, 2021 12:05 - 13:00 UTC

Breaks & Networking

13:00 UTC

Keynote Session: KVM Status Update - Paolo Bonzini, Red Hat

The last year has seen a lot of new exciting contributions in KVM, from nested virtualization to optimizations and introspection features. This presentation will be a shout-out to all the developers who collaborated on the open source hypervisor that's at the heart of the cloud, as well as a peek at what's coming in 2022.

Speakers

Paolo Bonzini

Distinguished Engineer, Red Hat, Inc.

Paolo is a Distinguished Engineer at Red Hat and the upstream maintainer for both KVM and various subsystems in QEMU. As a contributor to QEMU, through the years, he has worked on various parts of the project architecture, including the threading architecture, the test frameworks... Read More →

kvmforum21 kvm pdf

Thursday September 16, 2021 13:00 - 13:15 UTC

Keynote Sessions

Presentation Slides Attached Yes

13:15 UTC

Keynote Session: Kata, Containers and KVM -- What's New? - Eric Ernst, Apple

The Kata Containers project has been active for a few years now, working to blur the lines between containers and hardware virtualization based isolation. In this talk, we’ll give a refresher on how we are leveraging virtualization today, as well as areas that are under development, and new features and use cases going forward.
Aside from this technical overview, we’ll also spend time focusing on the Kata Community — who we are, and how we work.

Speakers

Eric Ernst

Software Engineer, Apple

Eric Ernst is a software developer at Apple, where he focuses on virtualization, containers, container runtimes and Kubernetes. Eric is a contributor to Kubernetes and an architecture committee member for the Kata Containers project.

ernst KVM forum whats new with kata pdf

Thursday September 16, 2021 13:15 - 13:30 UTC

Keynote Sessions

Presentation Slides Attached Yes

13:35 UTC

Securing Linux VM boot with AMD SEV measurement - Dov Murik & Hubertus Franke, IBM Research

Booting Linux guests with AMD SEV using a kernel and initrd supplied by the VMM currently breaks the Confidential Computing promise: the binaries are supplied by the VMM which is outside the trusted domain. However, this mode of guest booting is convenient for both the platform provider and the guest owner, as usually the kernel and initrd binaries are not confidential. We introduce a way to harness SEV memory measurement and secret injection at startup to verify that the kernel and initrd supplied by the VMM are indeed approved by the guest owner, thus making this way of booting SEV guests secure for Confidential Computing workloads. The presentation will explain the boot process in the VMM and guest, the added integrity checks added in OVMF, and layouts of secret injection memory areas. We will present the current upstream status of OVMF and QEMU patches, as well as cover possible attack scenarios and mitigations.

Speakers

Dov Murik

Research Staff Member, IBM

Dov Murik is a Research Staff Member in IBM Research, working on various aspects of information and system security, and recently focusing on confidential computing. Previously he worked on malware detection, phishing prevention, and AI security. Before that he was part of IBM Trusteer... Read More →

Hubertus Franke

Distinguished Research Staff Member, IBM Research

Dr. Hubertus Franke is a Distinguished Research Staff Member at the IBM T.J.Watson Research Center since 1993. His area of current work and interests are the area of operating systems, virtualization, processor architectures, cloud runtimes and security. Some time back he has also... Read More →

securing linux vm boot with amd sev measurement pdf

Thursday September 16, 2021 13:35 - 14:00 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

13:35 UTC

rust-vmm: A Security Journey - Andreea Florescu, Amazon

rust-vmm is an open-source project that provides a set of reusable virtualization components that can be leveraged for building custom Virtual Machine Monitors (VMMs). One of the durable advantages of the rust-vmm project is the security and testability of each component. The virtualization components in rust-vmm have clear entry points, so that makes them a nice target for security testing such as fuzzing. But is it really that easy to achieve? In this talk we take a look at the multiple facets we have to consider when adding security testing. We will focus our attention on 3 aspects: * how we prepare a Rust component for security testing * how we can maintain the balance between transparency and customer trust when running fuzzing in an open source project, and * the peculiarities of fuzzing virtualization components. We will also take a look at our findings and lessons learned.

Speakers

Andreea Florescu

Software Development Engineer, Amazon

I am a software engineer @Amazon, working primarily on virtualization. I enjoy working on open source projects and dedicate most of my time to rust-vmm. I am particularly interested in software design, virtualization, security, building open source communities, and ensuring that the... Read More →

KVM Forum 2021 pdf

Thursday September 16, 2021 13:35 - 14:00 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

14:05 UTC

Encrypted Virtual Machine Images for Confidential Computing - James Bottomley, IBM & Brijesh Singh, AMD

KVM/QEMU has had the concept of encrypted qcow2 images for a while. Unfortunately the decryption is done inside the VMM which, in the current SEV and TDX paradigms , is outside of the trust zone and thus inappropriate for Confidential Computing because the machine owner must be privy to the image encryption key. We introduce a new encrypted image format, which is very similar to the current encrypted image format except that decryption is done inside the guest instead of in the VMM, thus making it suitable for Confidential Computing. This presentation will explain the image format, how it works both inside and outside of Confidential Computing hardware, and for the AMD SEV hardware, how attestation, trust and secret key release work, including a demo of the feature. Getting all this to work requires patches to tianocore/OVMF, qemu and grub, which we will describe and explain (and give the current upstream status).

Speakers

James Bottomley

DE, IBM

James Bottomley is a Distinguished Engineer at IBM Research where he works on Cloud and Container technology. He is also Linux Kernel maintainer of the SCSI subsystem. He has been a Director on the BoardJames Bottomley is a Distinguished Engineer at IBM Research where he works on... Read More →

Brijesh Singh

SMTS, Advanced Micro Devices

Thursday September 16, 2021 14:05 - 14:30 UTC
TBA

KVM Track 1

14:05 UTC

Is QEMU too Complex, and What Can we do About It? - Paolo Bonzini, Red Hat, Inc.

QEMU is considered a difficult program to work on. During this presentation I will present my own view of what makes it difficult to modify QEMU, whether it is to implement a new feature or to enable the emulation of a new machine, and suggest parts of QEMU where improvements in consistency and documentation would have a high impact on the productivity of newcomers to the project.

Speakers

Paolo Bonzini

Distinguished Engineer, Red Hat, Inc.

kvmforum21 complexity pdf

Thursday September 16, 2021 14:05 - 14:30 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

14:35 UTC

Secure Live Migration of Encrypted VMs - Tobin Feldman-Fitzthum & Dov Murik, IBM

Most Confidential Computing platforms, such as AMD SEV, encrypt guest memory and CPU state, not allowing the hypervisor to access either. This complicates live VM migration. In a non-secure setting, the hypervisor copies memory from the source node to the destination node and coordinates the CPU state of the source VM and destination VM. In a secure setting, without access to guest memory or CPU state, the hypervisor needs help from a trusted agent inside the guest to facilitate live migration. We are implementing live migration support in firmware. In this session, we will describe in detail the current and future challenges for migrating encrypted VMs. We will walk through our modified firmware and demonstrate how it can be used with QEMU and SEV VMs.

Speakers

Tobin Feldman-FItzthum

Software Engineer, T.J. Watson IBM Research Center

Tobin Feldman-Fitzthum is a Software Engineer at the T.J. Watson IBM Research Center. He works on secure virtualization and confidential computing. Tobin was a founding maintainer of the Confidential Containers CNCF Sandbox Project. He has also worked on encrypted disks and fast live... Read More →

Dov Murik

Research Staff Member, IBM

LiveMigrationSev TobinFeldmanFitzthum pdf

Thursday September 16, 2021 14:35 - 15:00 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

14:35 UTC

Live Migrating VFIO, vhost-user, and vfio-user Devices - Stefan Hajnoczi, Red Hat

While devices built into QEMU largely support live migration, this is not the case for many VFIO, vhost-user, and vfio-user devices. Questions remain about how to design interfaces for migration of these devices. This talk covers the current state and possible solutions for the missing pieces. We will explore the requirements for migrating devices and the capabilities of existing interfaces including VFIO_REGION_SUBTYPE_MIGRATION, D-Bus vmstate, and stateless vhost-user device migration. Migration must do more than just serialize and deserialize device state, it must ensure the compatibility of saved states and the guest-visible hardware interface. This talk answers two key questions about device migration: 1. Is it possible to determine if a migration destination is compatible beforehand without running a full migration? 2. Is migrating between different implementations of the same device possible in practice? A demonstration of migration with Multi-Process QEMU will show the concepts in action.

Speakers

Stefan Hajnoczi

Senior Principal Software Engineer, Red Hat

Stefan works on QEMU and Linux in Red Hat's Virtualization team with a focus on storage, VIRTIO, and tracing. Recent projects include libblkio, virtiofs, storage performance optimization for NVMe drives, and out-of-process device emulation. Stefan has been active in the QEMU community... Read More →

Live Migrating VFIO, vhost user, and vfio user Devices pdf

Thursday September 16, 2021 14:35 - 15:00 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

15:05 UTC

Securing the Hypervisor with Control-Flow Integrity - Daniele Buono, IBM

In the cloud, the Hypervisor is usually the first line of defense against attacks from malicious users. But what if the Hypervisor itself is vulnerable to attacks? What can we do to protect the host, and other VMs, against Hypervisor attacks, specifically zero-day exploits, where only generic security countermeasures can be taken? In this seminar, we present our work with the QEMU community to upstream a new security mechanism by leveraging Clang's software implementation of both backward and forward Control-Flow Integrity (CFI) for x86 systems. We show how, and why, this technique can provide an effective protection against zero-day remote execution exploits based on buffer overflows and ROP attacks, sooner and more extensively than current countermeasures such as SELinux, AppArmor, or Seccomp. We will also explain why compiler-driven CFI offers better protection than hardware-based techniques such as Intel's CET. Finally, we will discuss the few incompatibilities we encountered in QEMU's codebase, and the possibility of enabling CFI with QEMU's plugins and modules, which are currently unsupported.

Speakers

Daniele Buono

RSM, IBM

Daniele Buono is a Research Staff Member and Manager at the IBM T.J. Watson Research Center, where is currently leading the Security and Attestation for Hybrid Cloud group. He joined the Data-Centric Systems group at IBM Research in 2014, where he focused on High-Performance Computing... Read More →

Buono Securing the Hypervisor with Control Flow Integrity pdf

Thursday September 16, 2021 15:05 - 15:30 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

15:05 UTC

VFIO User - Using VFIO as the IPC Protocol in Multi-process QEMU - John Johnson & Jagannathan Raman, Oracle

This presentation is a follow on to our multi-process QEMU presentation for KVM 2019. That project described how we ran device emulation in a separate QEMU process from the one managing the guest. It was integrated into QEMU as an experimental feature in 6.0. In this presentation, we will describe how we evolved the project to use VFIO as the basis for inter-process communication. We will cover the advantages of this approach: - VFIO is an established protocol, instead of the custom protocol mp-qemu uses. It should provide fewer maintenance issues going forward. - When new features, such as live migration, are added to the VFIO client, they can more easily to added to mp-qemu. - We got to leverage the libvfio-user project for server-side VFIO message processing. - The VFIO protocol is flexible enough to be extended to non-PCI devices in the future We will also cover the current state of the project, and some future directions.

Speakers

John Johnson

Software Architect, Oracle

I've been working on virtualization technologies for a number of years, beginning with the LDOMs product at Sun Microsystems. Recently, I've been working on multi-process QEMU at Oracle, including presenting it at KVM 2019.

Jagannathan Raman

Principle Member of the Technical Staff, Oracle

Jag is a software developer focusing on virtualization for Oracle Linux.

Johnson Raman Ufimtseva Vfio user pdf

Thursday September 16, 2021 15:05 - 15:30 UTC
TBA

KVM Track 2

Presentation Slides Attached Yes

15:35 UTC

Host & Guest Tracing in Virtualization: "To sync, or not to sync?" - Stefano De Venuto, SUSE & Tzvetomir Stoyanov, VMware

Tracing is instrumental for understanding how a complex system work or for investigating issues. In virtualized environments, however, we have the complete picture only if we trace both the host and the guests (and then we combine all the traces!). This talk shows how tools like trace-cmd and KernelShark can already be used for such purpose, including having the timestamps of the events in host and guests traces synchronized However, let's assume we have a combined trace and that the tracing tools are telling us that all is sync'd. Is that really the case? And how can we be sure? Well, this talks discusses a possible "event-driven" approach to check and evaluate how accurate the synchronization turned out to be and introduces a tool, implemented on top of libkshark, that can help to do this kind of analysis automatically.

Speakers

Tzvetomir Stoyanov

Open Source Engineer, VMware

Tzvetomir Stoyanov, software engineer in the Open Source Technology Center, VMware/Bulgaria. Tzvetomir has been working with Linux and various unix-like operating systems for more than 20 years, first developing proprietary software. He joined VMware's Open Source Technology Center... Read More →

Stefano De Venuto

Intern, SUSE

Stefano De Venuto is a CS student at the University of Turin, and he is currently doing an internship at SUSE focusing on combined host and guest tracing in virtualized systems. He is also really interested in the cybersecurity field, specifically on low level stuff, and loves sk... Read More →

HostGuestTracingInVirtualizationSlides pdf

Thursday September 16, 2021 15:35 - 16:00 UTC
TBA

KVM Track 1

Presentation Slides Attached Yes

15:35 UTC

libvfio-user: Status Update - Thanos Makatos & John Levon, Nutanix

vfio-user is a device offloading protocol that allows a device to be virtualized outside the VMM, originally presented as MUSER at the KVM forum in 2019. In this presentation, we will provide a status update of libvfio-user, a library that simplifies the implementation of virtual devices that operate under the vfio-user protocol, hiding the details of the protocol implementation. libvfio-user is VMM-agnostic but is built with QEMU as its primary use case. We will provide a very brief overview of the vfio-user protocol and dive into libvfio-user’s implementation, its API, future work, and also perform a live demo of a libvfio-user device.

Speakers

Thanos Makatos

Senior Member of Technical Staff, Nutanix

I'm a software engineer at Nutanix working on storage virtualization. I'm currently working on the vfio-user protocol and libvfio-user, which allows us to use SPDK as a virtual NVMe controller outside QEMU in order to achieve high performance, low latency, and higher CPU efficien... Read More →