Recently I needed to install Windows XP under VirtualBox on my Fedora Core 9 mashine.

After a successfull VirtualBox installation, I got an error when trying to install Windows from .iso image:

VirtualBox can’t operate in VMX root mode. Please disable the KVM kernel extension, recompile your kernel and reboot (VERR_VMX_IN_VMX_ROOT_MODE).

The error message means your kernel has loaded kvm module which conflicts with VirtualBox.

How to fix the problem?

The first of all, become the super user.

To ensure the module is loaded execute this command in console:

#modprobe -r kvm

Output:

FATAL: Module kvm is in use.

Then execute this:

#lsmod | grep kvm

Output:

kvm_intel              41144  0
kvm                   115008  1 kvm_intel

Note: You may see ‘kvm_amd’ instead of ‘kvm_intel’.

This means the kvm module is loaded.

To unload the module execute the following:

#rmmod kvm_intel

and then

#rmmod kvm

(sequence of the commands does matter)

If all is ok no extra output will be present.
Note: You should execute

#rmmod kvm_amd

in case you’ve seen ‘kvm_amd’ on previous steps.

The kvm module is unloaded and now you can try to install Windows under your VirtualBox without “VirtualBox can’t operate…” message. However you may want to prevent the kvm module use on next system startups:

In /etc/modprobe.d/ create file named ‘nokvm’ (or with any other name you like) and put into it the only line:

install kvm /bin/true

Congrats, you’re done. Now you can reboot your mashine and enjoy using VirtualBox

I believe the described will work for Fedora Core 8 either.

More explanation about described you can find here:
http://forums.fedoraforum.org/showthread.php?t=167442
http://www.g-loaded.eu/2008/05/12/how-to-disable-ipv6-in-fedora-and-centos/