login
Header Space

 
 

virtualization

Quote: One Major Problem With Virtualizers

July 14, 2008 - 8:58am
Submitted by Jeremy on July 14, 2008 - 8:58am.

"One *major* problem with virtualizers is that they uniformly use an existing CPU identifier, even though they might have their own sets of bugs. This makes it much harder to work around bugs in them."

— H. Peter Anvin in a July 7th, 2008 message on the Linux Kernel mailing list.

2.6.25 KVM Updates

February 1, 2008 - 1:19am
Submitted by Jeremy on February 1, 2008 - 1:19am.
Linux news

Avi Kivity summarized the kvm patches bound for the 2.6.25 kernel:

"Changes include performance and scalability improvements, completion of the portability work (though no new architectures are supported with this submission), support for new hardware features, using general userspace memory for kvm (which enables swapping guest memory as well as sharing memory among guests), as well as the usual cleanups and incremental fixes."

The Kernel-based Virtual Machine project, kvm, was started in mid-2006, and has been part of the Linux kernel since the 2.6.20 release in February of 2007. The recent changes can be browsed with gitweb.

Swap Over NFS

November 2, 2007 - 11:07pm
Submitted by Jeremy on November 2, 2007 - 11:07pm.
Linux news

"The problem with swap over network is the generic swap problem: needing memory to free memory. Normally this is solved using mempools, as can be seen in the BIO layer," explained Peter Zijlstra. "Swap over network has the problem that the network subsystem does not use fixed sized allocations, but heavily relies on kmalloc(). This makes mempools unusable."

The first fifteen patches set up a generic framework for reserving memory. Patches 16-23 actually put the framework to use on the network stack. Peter noted, "a network write back completion [involves] receiving packets, which when there is no memory, is rather hard. And even when there is memory there is no guarantee that the required packet comes in in the window that that memory buys us." He went on to explain, "the solution to this problem is found in the fact that network is to be assumed lossy. Even now, when there is no memory to receive packets the network card will have to discard packets. What we do is move this into the network stack." Patches 24-26 set up an infrastructure for swapping to a filesystem instead of a block device, which is then utilized by the final patches, "finally, convert NFS to make use of the new network and vm infrastructure to provide swap over NFS." When the usefulness of these patches were questioned, Peter noted, "There is a large corporate demand for this, which is why I'm doing this. The typical usage scenarios are: 1) cluster/blades, where having local disks is a cost issue (maintenance of failures, heat, etc) 2) virtualisation, where dumping the storage on a networked storage unit makes for trivial migration and what not.."

Virtualization Security

October 25, 2007 - 10:28am
Submitted by Jeremy on October 25, 2007 - 10:28am.
OpenBSD news

A thread on the OpenBSD -misc mailing list began by discussing whether or not XEN had been ported to OpenBSD, "is it planned at some point to release a paravirtualized xen kernel for OpenBSD 4.3 or 4.4?" Later in the discussion it was suggested that virtualization should be a priority for security reasons, "virtualization seems to have a lot of security benefits." OpenBSD creator Theo de Raadt strongly disagreed with this assertion, "you've been smoking something really mind altering, and I think you should share it." He went on to describe virtualization as "something on the shelf, [which] has all sorts of pretty colours, and you've bought it", explaining:

"x86 virtualization is about basically placing another nearly full kernel, full of new bugs, on top of a nasty x86 architecture which barely has correct page protection. Then running your operating system on the other side of this brand new pile of shit. You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes."

Later in the thread, Theo went on to note, "if the actual hardware let us do more isolation than we do today, we would actually do it in our operating system. The problem is the hardware DOES NOT actually give us more isolation abilities, therefore the VM does not actually do anything what the say they do." He then suggested that companies marketing virtualization should soften their claims to something supportable, such as, "yes, it [increases] hardware utilization, and the nasty security impact might be low".

Quote: You Are Absolutely Deluded

October 25, 2007 - 5:29am
Submitted by Jeremy on October 25, 2007 - 5:29am.

"You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes."

— Theo de Raadt in an October 24th, 2007 message on the OpenBSD -misc mailing list.

Virtually Debugging

October 15, 2007 - 1:42pm
Submitted by Jeremy on October 15, 2007 - 1:42pm.
Linux news

"Incidentally i was thinking about using KVM for automated testing. Important pieces of hardware should get an in-KVM simulator/emulator, that way developers who do not own that hardware can do functionality testing too," Ingo Molnar suggested during a thread discussing a SCSI driver bug fix. Linus Torvalds was originally unimpressed by the idea:

"Using emulators to test device drivers is almost certain to be pointless. The problem with device drivers tends to be timing issues, odd hardware interactions, and lots of strange (and sometimes undocumented) behaviour and dependencies (eg things like 'you have to wait 50us after setting the reset bit until the hardware has actually reset'). These are all things that you'd generally not catch in emulation - because the emulation by necessity is only going to be a very weak picture of the real thing."

Alan Cox countered, "for some things. I do it a bit because you can use it to fake failures that are tricky to do in the real world. It won't tell you the driver works but its surprisingly good for testing for races (forcing IRQ delivery at specific points), buggy hardware you don't posses, and things like media failures and timeouts your real hardware refuses to do." Linus acquiesced conditionally, "I do agree that you likely find bugs, even if quite often it's exactly because the behaviour is something that will never happen on real hardware," then acknowledged previous debugging efforts by Alan, "but failure testing is very useful - I forget who it was who debugged some driver by taking a CD and just scratching it mercilessly to induce read errors ;)" Ingo added, "something like that wont enable 100% coverage (or even reasonable coverage for most hardware), so it's no replacement for actual hard testing, but it could push out the domain of minimally tested code quite a bit and increase the quality of the kernel."

Merge Window KVM Updates

October 14, 2007 - 10:35am
Submitted by Jeremy on October 14, 2007 - 10:35am.
Linux news

Avi Kivitiy posted numerous KVM updates which Linus Torvalds merged into his mainline kernel source tree to be included in the upcoming 2.6.24 kernel. Avi summarized:

"Highlights include in-kernel pic/lapic/ioapic emulation, improved guest support, preemptibility, an improved x86 emulator, and a fair amount of cleanup.

"The changes outside drivers/kvm/ and include/linux/kvm*.h fix the CR8 mask definition (which is not otherwise used in the kernel) and expose some ioapic register definitions even if ioapic support is not compiled in. The diff is appended below."

Third Quarter FreeBSD Status Report

October 13, 2007 - 12:02pm
Submitted by Jeremy on October 13, 2007 - 12:02pm.
FreeBSD news

"This report covers FreeBSD related projects between July and October 2007," began the latest FreeBSD Quarterly Status Report, posted by Brad Davis. He included a summary of the recent Google Summer of Code projects noting, "lots of participants are working getting their code merged back into FreeBSD." Regarding the upcoming FreeBSD 7.0 release he noted, "the bugs in the FreeBSD HEAD branch are being shaked out and it is being prepared for the FreeBSD 7 branching. If your are curious about what's new in FreeBSD 7.0 we suggest reading Ivan Voras' excellent summary."

Among the many projects discussed in the status report was work by Marko Zec on network stack virtualization, "the network stack virtualization project aims at extending the FreeBSD kernel to maintain multiple independent instances of networking state. This allows for networking independence between jail-like environmens, each maintaining its private network interface set, IPv4 and IPv6 network and port address space, routing tables, IPSec configuration, firewalls, and more." Another project discussed was the porting of Linux KVM, "a software package that can be used to create virtual machines fully emulating x86 hardware on top of machines supporting Intel VT-x or AMD-V virtualization extensions." The report noted, "Linux KVM has been ported to FreeBSD as a loadable kernel module, using the linux-kmod-compat port (in /usr/ports/devel/) to reuse as much as possible of the original source code, plus an userspace client consisting in a modified version of qemu, that uses KVM for the execution of its guests."

Linux: 64-bit Lguest

September 5, 2007 - 9:45am
Submitted by Jeremy on September 5, 2007 - 9:45am.
Linux news

"This is a formal announcement of Lguest64," Steven Rostedt said in an email posted to the Linux Kernel mailing list. He explained:

"Most are aware of the little puppies (lguest32, or simply lguest, or in some circles "rustyvisor"). But this time the puppies ate a bit too much. No more lean and mean puppies, now we got big fat lazy ones. Running on the hardware that's too lazy to do full virtualization. Yes, lguest now runs on x86_64!"

Steven went on to caution that lguest64 is still a new code base, "lguest64 is still going through a bit of growth pains, but its getting better. It's to a point that we are not that afraid to bring it to the dog show." The list of items left to do include getting SMP working for both the host and the guest, matching Rusty Russel's lguest32 feature set, and greatly optimizing the performance of the code. Steven noted that the goal is to ultimately get the 64-bit version of lguest merged into the mainline kernel.

Linux: Xen Merged

July 20, 2007 - 2:01pm
Submitted by Jeremy on July 20, 2007 - 2:01pm.
Linux news

The Xen virtual machine monitor was recently merged into the upcoming 2.6.23 Linux kernel in a series of patches from Jeremy Fitzhardinge. The project was originally started as a research project at the University of Cambridge, and has been repeatedly discussed as a merge candidate for the mainline Linux kernel.

Xen is described in the project's FAQ as:

"Xen is a virtual machine monitor (VMM) for x86-compatible computers. Xen can securely execute multiple virtual machines, each running its own OS, on a single physical system with close-to-native performance."

Linux: lguest Merged

July 20, 2007 - 1:02pm
Submitted by Jeremy on July 20, 2007 - 1:02pm.
Linux news

Rusty Russell's lguest was recently merged into the upcoming 2.6.23 Linux kernel. The merge comment describes the project, "lguest is a simple hypervisor for Linux on Linux. Unlike kvm it doesn't need VT/SVM hardware. Unlike Xen it's simply 'modprobe and go'. Unlike both, it's 5000 lines and self-contained." The comment goes on to note:

"Performance is ok, but not great (-30% on kernel compile). But given its hackability, I expect this to improve, along with the paravirt_ops code which it supplies a complete example for. There's also a 64-bit version being worked on and other craziness.

"But most of all, lguest is awesome fun! Too much of the kernel is a big ball of hair. lguest is simple enough to dive into and hack, plus has some warts which scream 'fork me!'."

Linux: KVM Adds Support For SMP Guests

July 18, 2007 - 11:15am
Submitted by Jeremy on July 18, 2007 - 11:15am.
Linux news

A recently merged KVM patchset included support for guest SMP, various performance improvements, and suspend/resume fixes. KVM stands for Kernel-based Virtual Machine, "a full virtualization solution for Linux on x86 hardware containing virtualization extensions". In regards to the recently merged guest SMP support which will be part of the upcoming 2.6.23 kernel, Avi Kivity noted:

"Guest smp is fully operational. Kernel build on 2-way smp is 40% faster than on a up guest. Expect significant performance improvements from in-kernel apic and from further tuning."

Linux: Improved KVM Performance, Vista Support

May 6, 2007 - 9:31am
Submitted by Jeremy on May 6, 2007 - 9:31am.
Linux news

Avi Kivity [interview] announced significant performance improvements and support for running 32-bit Windows Vista as a guest within the latest release of KVM. Originally merged into the 2.6.20 mainline Linux kernel [story], KVM stands for Kernel-based Virtual Machine, "a full virtualization solution for Linux on x86 hardware containing virtualization extensions". Regarding the new release, Avi announced:

"The happy theme of today's kvm is the significant performance improvements, brought to you by a growing team of developers. I've clocked kbuild at within 25% of native. This release also introduces support for 32-bit Windows Vista."

Interview: Avi Kivity

April 23, 2007 - 9:06am
Submitted by Jeremy on April 23, 2007 - 9:06am.
Interviews

Avi Kivity is the lead developer and maintainer of the Kernel-based Virtual Machine project, better known as kvm. The project was started in mid-2006, and has been part of the Linux kernel since the 2.6.20 release in February of 2007. kvm is a full virtualization system for x86-based Linux hosts, allowing users to run isolated x86 guest operating systems in virtual machines.

Linux: KVM Paravirtualization

January 5, 2007 - 9:57pm
Submitted by Jeremy on January 5, 2007 - 9:57pm.
Linux news

A new feature that will first be availble in the upcoming 2.6.20 kernel is KVM, a Kernel-based Virtual Machine. The project's webpage describes KVM as, "a full virtualization solution for Linux on x86 hardware. It consists of a loadable kernel module (kvm.ko) and a userspace component. Using KVM, one can run multiple virtual machines running unmodified Linux or Windows images. Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, etc." The project's FAQ explains that the functionality requires "an x86 machine running a recent Linux kernel on an Intel processor with VT (virtualization technology) extensions, or an AMD processor with SVM extensions (also called AMD-V)." The userland aspect of KVM is a slighlty modified version of qemu, used to instantiate the virtual machine.

Ingo Molnar [interview] announced a new patch introducing paravirtualization support for KVM, outdating the KVM FAQ which in comparing KVM to Xen notes, "Xen supports both full virtualization and a technique called paravirtualization, which allows better performance for modified guests. kvm does not at present support paravirtualization." In describing his patch which is against the 2.6.20-rc3 + KVM trunk kernel, Ingo said it, "includes support for the hardware cr3-cache feature of Intel-VMX CPUs. (which speeds up context switches and TLB flushes)". He went on to add, "some aspects of the code are still a bit ad-hoc and incomplete, but the code is stable enough in my testing and i'd like to have some feedback." In a series of benchmarks, he found 2-task context switch performance to be improved by a factor of four, while "hackbench 1" showed twice as good performance, and "hackbench 5" showed a 30% improvement. His email goes on to detail how the paravirtualization works.

speck-geostationary