KernelTrap - virtualization

Quote: One Major Problem With Virtualizers

Jeremy — Mon, 14 Jul 2008 12:58:53 +0000

"One *major* problem with virtualizers is that they uniformly use an existing CPU identifier, even though they might have their own sets of bugs. This makes it much harder to work around bugs in them."

2.6.25 KVM Updates

Jeremy — Fri, 01 Feb 2008 05:19:44 +0000

Avi Kivity summarized the kvm patches bound for the 2.6.25 kernel:

"Changes include performance and scalability improvements, completion of the portability work (though no new architectures are supported with this submission), support for new hardware features, using general userspace memory for kvm (which enables swapping guest memory as well as sharing memory among guests), as well as the usual cleanups and incremental fixes."

The Kernel-based Virtual Machine project, kvm, was started in mid-2006, and has been part of the Linux kernel since the 2.6.20 release in February of 2007. The recent changes can be browsed with gitweb.

Swap Over NFS

Jeremy — Sat, 03 Nov 2007 03:07:19 +0000

"The problem with swap over network is the generic swap problem: needing memory to free memory. Normally this is solved using mempools, as can be seen in the BIO layer," explained Peter Zijlstra. "Swap over network has the problem that the network subsystem does not use fixed sized allocations, but heavily relies on kmalloc(). This makes mempools unusable."

The first fifteen patches set up a generic framework for reserving memory. Patches 16-23 actually put the framework to use on the network stack. Peter noted, "a network write back completion [involves] receiving packets, which when there is no memory, is rather hard. And even when there is memory there is no guarantee that the required packet comes in in the window that that memory buys us." He went on to explain, "the solution to this problem is found in the fact that network is to be assumed lossy. Even now, when there is no memory to receive packets the network card will have to discard packets. What we do is move this into the network stack." Patches 24-26 set up an infrastructure for swapping to a filesystem instead of a block device, which is then utilized by the final patches, "finally, convert NFS to make use of the new network and vm infrastructure to provide swap over NFS." When the usefulness of these patches were questioned, Peter noted, "There is a large corporate demand for this, which is why I'm doing this. The typical usage scenarios are: 1) cluster/blades, where having local disks is a cost issue (maintenance of failures, heat, etc) 2) virtualisation, where dumping the storage on a networked storage unit makes for trivial migration and what not.."

Virtualization Security

Jeremy — Thu, 25 Oct 2007 14:28:43 +0000

A thread on the OpenBSD -misc mailing list began by discussing whether or not XEN had been ported to OpenBSD, "is it planned at some point to release a paravirtualized xen kernel for OpenBSD 4.3 or 4.4?" Later in the discussion it was suggested that virtualization should be a priority for security reasons, "virtualization seems to have a lot of security benefits." OpenBSD creator Theo de Raadt strongly disagreed with this assertion, "you've been smoking something really mind altering, and I think you should share it." He went on to describe virtualization as "something on the shelf, [which] has all sorts of pretty colours, and you've bought it", explaining:

"x86 virtualization is about basically placing another nearly full kernel, full of new bugs, on top of a nasty x86 architecture which barely has correct page protection. Then running your operating system on the other side of this brand new pile of shit. You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes."

Later in the thread, Theo went on to note, "if the actual hardware let us do more isolation than we do today, we would actually do it in our operating system. The problem is the hardware DOES NOT actually give us more isolation abilities, therefore the VM does not actually do anything what the say they do." He then suggested that companies marketing virtualization should soften their claims to something supportable, such as, "yes, it [increases] hardware utilization, and the nasty security impact might be low".

Quote: You Are Absolutely Deluded

Jeremy — Thu, 25 Oct 2007 09:29:33 +0000

"You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes."

Virtually Debugging

Jeremy — Mon, 15 Oct 2007 17:42:20 +0000

"Incidentally i was thinking about using KVM for automated testing. Important pieces of hardware should get an in-KVM simulator/emulator, that way developers who do not own that hardware can do functionality testing too," Ingo Molnar suggested during a thread discussing a SCSI driver bug fix. Linus Torvalds was originally unimpressed by the idea:

"Using emulators to test device drivers is almost certain to be pointless. The problem with device drivers tends to be timing issues, odd hardware interactions, and lots of strange (and sometimes undocumented) behaviour and dependencies (eg things like 'you have to wait 50us after setting the reset bit until the hardware has actually reset'). These are all things that you'd generally not catch in emulation - because the emulation by necessity is only going to be a very weak picture of the real thing."

Alan Cox countered, "for some things. I do it a bit because you can use it to fake failures that are tricky to do in the real world. It won't tell you the driver works but its surprisingly good for testing for races (forcing IRQ delivery at specific points), buggy hardware you don't posses, and things like media failures and timeouts your real hardware refuses to do." Linus acquiesced conditionally, "I do agree that you likely find bugs, even if quite often it's exactly because the behaviour is something that will never happen on real hardware," then acknowledged previous debugging efforts by Alan, "but failure testing is very useful - I forget who it was who debugged some driver by taking a CD and just scratching it mercilessly to induce read errors ;)" Ingo added, "something like that wont enable 100% coverage (or even reasonable coverage for most hardware), so it's no replacement for actual hard testing, but it could push out the domain of minimally tested code quite a bit and increase the quality of the kernel."

Merge Window KVM Updates

Jeremy — Sun, 14 Oct 2007 14:35:13 +0000

Avi Kivitiy posted numerous KVM updates which Linus Torvalds merged into his mainline kernel source tree to be included in the upcoming 2.6.24 kernel. Avi summarized:

"Highlights include in-kernel pic/lapic/ioapic emulation, improved guest support, preemptibility, an improved x86 emulator, and a fair amount of cleanup.

"The changes outside drivers/kvm/ and include/linux/kvm*.h fix the CR8 mask definition (which is not otherwise used in the kernel) and expose some ioapic register definitions even if ioapic support is not compiled in. The diff is appended below."

Third Quarter FreeBSD Status Report

Jeremy — Sat, 13 Oct 2007 16:02:27 +0000

"This report covers FreeBSD related projects between July and October 2007," began the latest FreeBSD Quarterly Status Report, posted by Brad Davis. He included a summary of the recent Google Summer of Code projects noting, "lots of participants are working getting their code merged back into FreeBSD." Regarding the upcoming FreeBSD 7.0 release he noted, "the bugs in the FreeBSD HEAD branch are being shaked out and it is being prepared for the FreeBSD 7 branching. If your are curious about what's new in FreeBSD 7.0 we suggest reading Ivan Voras' excellent summary."

Among the many projects discussed in the status report was work by Marko Zec on network stack virtualization, "the network stack virtualization project aims at extending the FreeBSD kernel to maintain multiple independent instances of networking state. This allows for networking independence between jail-like environmens, each maintaining its private network interface set, IPv4 and IPv6 network and port address space, routing tables, IPSec configuration, firewalls, and more." Another project discussed was the porting of Linux KVM, "a software package that can be used to create virtual machines fully emulating x86 hardware on top of machines supporting Intel VT-x or AMD-V virtualization extensions." The report noted, "Linux KVM has been ported to FreeBSD as a loadable kernel module, using the linux-kmod-compat port (in /usr/ports/devel/) to reuse as much as possible of the original source code, plus an userspace client consisting in a modified version of qemu, that uses KVM for the execution of its guests."

Linux: 64-bit Lguest

Jeremy — Wed, 05 Sep 2007 13:45:57 +0000

"This is a formal announcement of Lguest64," Steven Rostedt said in an email posted to the Linux Kernel mailing list. He explained:

"Most are aware of the little puppies (lguest32, or simply lguest, or in some circles "rustyvisor"). But this time the puppies ate a bit too much. No more lean and mean puppies, now we got big fat lazy ones. Running on the hardware that's too lazy to do full virtualization. Yes, lguest now runs on x86_64!"

Steven went on to caution that lguest64 is still a new code base, "lguest64 is still going through a bit of growth pains, but its getting better. It's to a point that we are not that afraid to bring it to the dog show." The list of items left to do include getting SMP working for both the host and the guest, matching Rusty Russel's lguest32 feature set, and greatly optimizing the performance of the code. Steven noted that the goal is to ultimately get the 64-bit version of lguest merged into the mainline kernel.