Corporate Home Open Source Home
Syndicate content
Eucalyptus
3 replies [Last post]
sukubhatt
Offline
Joined: 07/28/2010

Hi,

After a long struggle with ubuntu enterprise cloud 10.04 (where I could not get nodes to register successfully at all), I finally built eucalyptus-1.6.2 from source (after installing ubuntu-10.04-server-amd64) and am able to register the nc node BUT when I launch instances I see the instance go from pending to terminated.

nc.log shows "domain not found" error and I see the following error in /var/log/libvirt/qemu/i-47A808BE.log

libvir: QEMU error : cannot set ownership on /usr/local/eucalyptus/admin/i-47A808BE/kernel: Permission denied

The permissions are as follows:

cloud@cloudnc1:~$ ls -l /usr/local/eucalyptus/admin/i-47A808BE/
total 574744
-rwxrwxrwx 1 root root 5384437760 2010-07-29 02:49 disk
-rw------- 1 eucalyptus libvirtd 571064 2010-07-29 02:49 instance-checkpoint
-rwxrwxrwx 1 root root 3549936 2010-07-27 00:20 kernel
-rw-r--r-- 1 eucalyptus libvirtd 939 2010-07-29 02:49 libvirt.xml
-rwxrwxrwx 1 root root 7442869 2010-07-27 00:20 ramdisk
cloud@cloudnc1:~$

As you can see I have configured the 'eucalyptus' user to be in the 'libvirtd' primary group (there is no group called 'eucalyptus'). The eucalyptus root "/opt/eucalyptus" is also owned by eucalyptus:libvirtd as is /usr/local/eucalyptus - the instances location.

And of course, eucalyptus processes also run with the same user and group ownership.

Earlier I got the same error when I used the default eucalyptus:eucalyptus as user and group names.

/etc/libvirt/qemu.conf also has user=eucalyptus and group=libvirtd

And yet the files kernel, ramdisk and image are being created with root:root ownership - I don't understand how this happens when eucalyptus as well as libvirtd processes are supposed to be running as eucalyptus:libvirtd. I checked the ownership of /usr/local/eucalyptus/eucalyptus/cache where the kernel, ramdisk and image files are stored - they have eucalyptus:libvirtd ownership.

Some of the other details are as follows :-

sukumar@sukumar-laptop:~$ euca-describe-availability-zones verbose
AVAILABILITYZONE ampcloud 192.168.1.10
AVAILABILITYZONE |- vm types free / max cpu ram disk
AVAILABILITYZONE |- m1.small 0004 / 0004 1 128 2
AVAILABILITYZONE |- c1.medium 0004 / 0004 1 256 5
AVAILABILITYZONE |- m1.large 0002 / 0002 2 512 10
AVAILABILITYZONE |- m1.xlarge 0002 / 0002 2 1024 20
AVAILABILITYZONE |- c1.xlarge 0001 / 0001 4 2048 20
sukumar@sukumar-laptop:~$ euca-describe-images
IMAGE emi-012F1127 uecimage/ubuntu.9-04.x86.img.manifest.xml admin available public x86_64 machine eki-7CF5134B eri-D308149D
IMAGE eki-7CF5134B ueckernel/vmlinuz-2.6.28-11-server.manifest.xml admin available public x86_64 kernel
IMAGE eri-D308149D uecramdisk/initrd.img-2.6.28-11-server.manifest.xml admin available public x86_64 ramdisk

This is the default ubuntu-9.04-i386 image from eucalyptus store (same disk size as default).

The cloud cc is on a separate host with 2 NICs (192.168.1.10 and 192.168.10.1), I have only 1 cloud nc (192.168.10.2) on a separate subnet.

I feel there ought to be some simple permission fix for this error to go away - any suggestions?

Thanks for your help.

Sukumar.

graziano
Offline
Joined: 01/14/2010
Hello Sukumar, I suspect that

Hello Sukumar,

I suspect that the ownership is changed afterward: can you monitor an instance directory and confirm if the files in there are created with the right ownership? Do you have apparmor installed?

cheers
graziano

sukubhatt
Offline
Joined: 07/28/2010
Hi Graziano, I will check out

Hi Graziano,

I will check out whether the directories are created with the right permissions but changed later.

Apparmor is installed and running by default in Ubuntu server - do you think that is the culprit? Then I can switch off apparmor to begin with just to get eucalyptus running and launching instances.

Let me get back to you after doing the tests.

Thanks for your help.

Cheers,
Sukumar.

sukubhatt
Offline
Joined: 07/28/2010
Hi Graziano, I did what you

Hi Graziano,

I did what you asked. Here is the result :-

1. You were right about what you suspected - the files disk, kernel and ramdisk get created with eucalyptus:libvirtd ownership - but soon get changed to root:root ownership.

2. However, as you suspected apparmor to be the culprit, I switched off apparmor

cloud@cloudnc1:/usr/local/eucalyptus/admin$ sudo /etc/init.d/apparmor stop
* Unloading AppArmor profiles [ OK ]
cloud@cloudnc1:/usr/local/eucalyptus/admin$

and then again tried launching an instance while watching the /usr/local/eucalyptus/admin directory - same result as earlier - the files disk, kernel and ramdisk get created with eucalyptus:libvirtd ownership - but soon get changed to root:root ownership.

SO apparmor does not seem to be the culprit.

Also I tried running the instance using virsh from the commandline as 'eucalyptus' user

cloud@cloudnc1:/usr/local/eucalyptus/admin$ sudo chown -R eucalyptus:libvirtd i-52720928/
cloud@cloudnc1:/usr/local/eucalyptus/admin$ sudo su - eucalyptus
eucalyptus@cloudnc1:~$ cd /usr/local/eucalyptus/admin/i-52720928/
eucalyptus@cloudnc1:/usr/local/eucalyptus/admin/i-52720928$ virsh define libvirt.xml
Domain i-52720928 defined from libvirt.xml

eucalyptus@cloudnc1:/usr/local/eucalyptus/admin/i-52720928$ virsh start i-52720928
error: Failed to start domain i-52720928
error: monitor socket did not show up.: No such file or directory

eucalyptus@cloudnc1:/usr/local/eucalyptus/admin/i-52720928$

with failure for identical reasons (both with and without apparmor).

Hope this gives you a more complete picture - let me know if you need more details.

cheers,
Sukumar