cannot run instance | DEFAULT DEFAULT DEFAULT

My host system is Ubuntu Server 10.10 and the hardware does not support VT. I installed Xen and when I tried to run the instance, I have the following error in nc.log:

[Wed Feb  8 20:22:21 2012][004710][EUCADEBUG ] system_output(): [//usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/gen_libvirt_xml --ramdisk --ephemeral --basepath='/var/lib/eucalyptus/instances//admin/i-3B47074E']
[Wed Feb  8 20:22:21 2012][004710][EUCADEBUG ] doDescribeInstances() invoked
[Wed Feb  8 20:22:21 2012][004710][EUCADEBUG ] doDescribeInstances(): instanceId=i-3B47074E publicIp=0.0.0.0 privateIp=172.19.1.2 mac=D0:0D:3B:47:07:4E vlan=10 networkIndex=2
[Wed Feb  8 20:22:22 2012][004710][EUCAFATAL ] //usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/gen_libvirt_xml: Operation now in progress
[Wed Feb  8 20:22:22 2012][004710][EUCAFATAL ] Failed to create libvirt XML config for instance i-3B47074E
[Wed Feb  8 20:22:22 2012][004710][EUCADEBUG ] state change for instance i-3B47074E: Staging -> Shutoff (Extant)
[Wed Feb  8 20:22:27 2012][004710][EUCAERROR ] libvirt: Domain not found: xenUnifiedDomainLookupByName (code=42)

I also attach the full relevant log in the end. I found a little bit strange when I was verifying if Xen is working or not. And the following is my observation (i've already change the hypervisor in eucalyptus.conf from "kvm" to "xen"):

When I am in the Xen kernel, virsh list is empty:

 Id Name                 State
----------------------------------

while xm list works:

Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  1895     4     r-----   2784.5

when I check the version:

$ virsh version
 
Compiled against library: libvir 0.9.2
Using library: libvir 0.9.2
Using API: Xen 3.0.1
<strong>Cannot extract running Xen hypervisor version</strong>

However, I have already set "export VIRSH_DEFAULT_CONNECT_URI=xen:///" in the ~/.bashrc, and virsh uri gave output "xen:///".

The capabilities also indicate using xen as guest:

  <guest>
    <os_type>xen</os_type>
    <arch name='i686'>
      <wordsize>32</wordsize>
      <emulator>/usr/lib/xen/bin/qemu-dm</emulator>
      <machine>xenpv</machine>
      <domain type='xen'>
      </domain>
    </arch>
    <features>
      <pae/>
    </features>
  </guest>

So is my Xen working correctly? And what does the error "Failed to create libvirt XML config for instance", and "libvirt: Domain not found: xenUnifiedDomainLookupByName (code=42)" means??? Has someone encountered the same problem? I didn't find any relevant topic on the Internet. And wherelse should I check on my NC to detect the problem?

Many thanks!!!!

-------------Full Log-----------------

[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ] StartNetwork(): SUCCESS return from vnetStartNetwork 0
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ] StartNetwork(): done
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ] doRunInstance() invoked (id=i-3B47074E cores=1 disk=5 memory=256)
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ]                          image=emi-085C114B at http://10.13.66.13:8773/services/Walrus/mybucket/ubuntu.9-04.x86.img.manifest.xml
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ]                          krnel=eki-67E112F5 at http://10.13.66.13:8773/services/Walrus/mybucket/vmlinuz-2.6.28-11-server.manifest.xml
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ]                          rmdsk=eri-A25013D7 at http://10.13.66.13:8773/services/Walrus/mybucket/initrd.img-2.6.28-11-server.manifest.xml
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ]                          vlan=10 priMAC=D0:0D:3B:47:07:4E privIp=172.19.1.2
[Wed Feb  8 20:17:42 2012][004710][EUCADEBUG ] state change for instance i-3B47074E: Unknown -> Staging (Pending)
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ] network started for instance i-3B47074E
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ] retrieving images for instance i-3B47074E (disk limit=5120MB)...
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ] verifying cached file in /var/lib/eucalyptus/instances//eucalyptus/cache/eki-67E112F5/kernel...
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ] walrus_request(): downloading /tmp/walrus-digest-YZCnMW
[Wed Feb  8 20:17:42 2012][004710][EUCAINFO  ]                   from http://10.13.66.13:8773/services/Walrus/mybucket/vmlinuz-2.6.28-11-server.manifest.xml
[Wed Feb  8 20:17:42 2012][004710][EUCADEBUG ] walrus_request(): writing GET output to /tmp/walrus-digest-YZCnMW on 'Wed Feb  8 20:17:42 2012'
[Wed Feb  8 20:17:43 2012][004710][EUCADEBUG ] walrus_request(): wrote 3487 bytes in 0 writes
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ] walrus_request(): saved image in /tmp/walrus-digest-YZCnMW
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ] vrun(): [cp -a /var/lib/eucalyptus/instances//eucalyptus/cache/eki-67E112F5/kernel /var/lib/eucalyptus/instances//admin/i-3B47074E/kernel]
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ] verifying cached file in /var/lib/eucalyptus/instances//eucalyptus/cache/eri-A25013D7/ramdisk...
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ] walrus_request(): downloading /tmp/walrus-digest-auz1mK
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ]                   from http://10.13.66.13:8773/services/Walrus/mybucket/initrd.img-2.6.28-11-server.manifest.xml
[Wed Feb  8 20:17:43 2012][004710][EUCADEBUG ] walrus_request(): writing GET output to /tmp/walrus-digest-auz1mK on 'Wed Feb  8 20:17:43 2012'
[Wed Feb  8 20:17:43 2012][004710][EUCADEBUG ] walrus_request(): wrote 3495 bytes in 0 writes
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ] walrus_request(): saved image in /tmp/walrus-digest-auz1mK
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ] vrun(): [cp -a /var/lib/eucalyptus/instances//eucalyptus/cache/eri-A25013D7/ramdisk /var/lib/eucalyptus/instances//admin/i-3B47074E/ramdisk]
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ] walrus_request(): downloading /var/lib/eucalyptus/instances//admin/i-3B47074E/root-digest
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ]                   from http://10.13.66.13:8773/services/Walrus/mybucket/ubuntu.9-04.x86.img.manifest.xml
[Wed Feb  8 20:17:43 2012][004710][EUCADEBUG ] walrus_request(): writing GET output to /var/lib/eucalyptus/instances//admin/i-3B47074E/root-digest on 'Wed Feb  8 20:17:43 2012'
[Wed Feb  8 20:17:43 2012][004710][EUCADEBUG ] walrus_request(): wrote 5709 bytes in 0 writes
[Wed Feb  8 20:17:43 2012][004710][EUCAINFO  ] walrus_request(): saved image in /var/lib/eucalyptus/instances//admin/i-3B47074E/root-digest
[Wed Feb  8 20:17:44 2012][004710][EUCAINFO  ] downloading and preparing image into /var/lib/eucalyptus/instances//admin/i-3B47074E/root...
[Wed Feb  8 20:17:44 2012][004710][EUCAINFO  ] walrus_request(): downloading /var/lib/eucalyptus/instances//admin/i-3B47074E/root
[Wed Feb  8 20:17:44 2012][004710][EUCAINFO  ]                   from http://10.13.66.13:8773/services/Walrus/mybucket/ubuntu.9-04.x86.img.manifest.xml
[Wed Feb  8 20:17:44 2012][004710][EUCADEBUG ] walrus_request(): writing GET/GetDecryptedImage output to /var/lib/eucalyptus/instances//admin/i-3B47074E/root on 'Wed Feb  8 20:17:44 2012'
[Wed Feb  8 20:17:51 2012][004710][EUCADEBUG ] doDescribeResource() invoked
[Wed Feb  8 20:17:52 2012][004710][EUCADEBUG ] doDescribeInstances() invoked
[Wed Feb  8 20:17:52 2012][004710][EUCADEBUG ] doDescribeInstances(): instanceId=i-3B47074E publicIp=0.0.0.0 privateIp=172.19.1.2 mac=D0:0D:3B:47:07:4E vlan=10 networkIndex=2
.......
[Wed Feb  8 20:19:16 2012][004710][EUCADEBUG ] walrus_request(): wrote 1049624576 bytes in 0 writes
[Wed Feb  8 20:19:16 2012][004710][EUCAINFO  ] walrus_request(): saved image in /var/lib/eucalyptus/instances//admin/i-3B47074E/root
[Wed Feb  8 20:19:16 2012][004710][EUCAINFO  ] vrun(): [cp -a /var/lib/eucalyptus/instances//admin/i-3B47074E/root /var/lib/eucalyptus/instances//eucalyptus/cache/emi-085C114B/root]
[Wed Feb  8 20:19:24 2012][004710][EUCADEBUG ] doDescribeResource() invoked
[Wed Feb  8 20:19:25 2012][004710][EUCADEBUG ] doDescribeInstances() invoked
[Wed Feb  8 20:19:25 2012][004710][EUCADEBUG ] doDescribeInstances(): instanceId=i-3B47074E publicIp=0.0.0.0 privateIp=172.19.1.2 mac=D0:0D:3B:47:07:4E vlan=10 networkIndex=2
........
[Wed Feb  8 20:20:05 2012][004710][EUCADEBUG ] doDescribeInstances(): instanceId=i-3B47074E publicIp=0.0.0.0 privateIp=172.19.1.2 mac=D0:0D:3B:47:07:4E vlan=10 networkIndex=2
[Wed Feb  8 20:20:05 2012][004710][EUCAINFO  ] vrun(): [cp -a /var/lib/eucalyptus/instances//admin/i-3B47074E/root-digest /var/lib/eucalyptus/instances//eucalyptus/cache/emi-085C114B/root-digest]
[Wed Feb  8 20:20:05 2012][004710][EUCAINFO  ] preparing images for instance i-3B47074E...
[Wed Feb  8 20:20:05 2012][004710][EUCAINFO  ] adding key/tmp/sckey.c2nYLF to the root file system at /var/lib/eucalyptus/instances//admin/i-3B47074E/root using (//usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/add_key.pl //usr/lib/eucalyptus/euca_mountwrap)
[Wed Feb  8 20:20:05 2012][004710][EUCAINFO  ] vrun(): [//usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/add_key.pl //usr/lib/eucalyptus/euca_mountwrap 0 /var/lib/eucalyptus/instances//admin/i-3B47074E/root /tmp/sckey.c2nYLF]
[Wed Feb  8 20:20:05 2012][004710][EUCAINFO  ] vrun(): [dd bs=1M count=512 if=/dev/zero of=/var/lib/eucalyptus/instances//admin/i-3B47074E/swap 2>/dev/null]
[Wed Feb  8 20:20:15 2012][004710][EUCADEBUG ] doDescribeResource() invoked
[Wed Feb  8 20:20:15 2012][004710][EUCADEBUG ] doDescribeInstances() invoked
[Wed Feb  8 20:20:15 2012][004710][EUCADEBUG ] doDescribeInstances(): instanceId=i-3B47074E publicIp=0.0.0.0 privateIp=172.19.1.2 mac=D0:0D:3B:47:07:4E vlan=10 networkIndex=2
[Wed Feb  8 20:20:21 2012][004710][EUCAINFO  ] vrun(): [mkswap /var/lib/eucalyptus/instances//admin/i-3B47074E/swap >/dev/null]
[Wed Feb  8 20:20:24 2012][004710][EUCAINFO  ] vrun(): [dd bs=1M count=3597 if=/dev/zero of=/var/lib/eucalyptus/instances//admin/i-3B47074E/ephemeral 2>/dev/null]
[Wed Feb  8 20:20:25 2012][004710][EUCADEBUG ] doDescribeResource() invoked
......
[Wed Feb  8 20:22:13 2012][004710][EUCAINFO  ] vrun(): [mkfs.ext3 -F /var/lib/eucalyptus/instances//admin/i-3B47074E/ephemeral >/dev/null 2>&1]
[Wed Feb  8 20:22:21 2012][004710][EUCADEBUG ] doDescribeResource() invoked
[Wed Feb  8 20:22:21 2012][004710][EUCADEBUG ] system_output(): [//usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/gen_libvirt_xml --ramdisk --ephemeral --basepath='/var/lib/eucalyptus/instances//admin/i-3B47074E']
[Wed Feb  8 20:22:21 2012][004710][EUCADEBUG ] doDescribeInstances() invoked
[Wed Feb  8 20:22:21 2012][004710][EUCADEBUG ] doDescribeInstances(): instanceId=i-3B47074E publicIp=0.0.0.0 privateIp=172.19.1.2 mac=D0:0D:3B:47:07:4E vlan=10 networkIndex=2
[Wed Feb  8 20:22:22 2012][004710][EUCAFATAL ] //usr/lib/eucalyptus/euca_rootwrap //usr/share/eucalyptus/gen_libvirt_xml: Operation now in progress
[Wed Feb  8 20:22:22 2012][004710][EUCAFATAL ] Failed to create libvirt XML config for instance i-3B47074E
[Wed Feb  8 20:22:22 2012][004710][EUCADEBUG ] state change for instance i-3B47074E: Staging -> Shutoff (Extant)
[Wed Feb  8 20:22:27 2012][004710][EUCAERROR ] libvirt: Domain not found: xenUnifiedDomainLookupByName (code=42)
[Wed Feb  8 20:22:27 2012][004710][EUCAINFO  ] cleaning up state for instance i-3B47074E
[Wed Feb  8 20:22:27 2012][004710][EUCAINFO  ] vrun(): [rm -rf /var/lib/eucalyptus/instances//admin/i-3B47074E/]
[Wed Feb  8 20:22:28 2012][004710][EUCAINFO  ] stopping the network (vlan=10)
[Wed Feb  8 20:22:28 2012][004710][EUCADEBUG ] state change for instance i-3B47074E: Shutoff -> Teardown (Teardown)
[Wed Feb  8 20:22:31 2012][004710][EUCADEBUG ] doDescribeResource() invoked
[Wed Feb  8 20:22:31 2012][004710][EUCADEBUG ] doDescribeInstances() invoked

Thu, 02/09/2012 - 03:05

vivienlwt

Offline

Joined: 02/02/2012

unknown option: basepath ?

it seems that if I run manually:
//usr/share/eucalyptus/gen_libvirt_xml --ramdisk --ephemeral --basepath='/var/lib/eucalyptus/instances//admin/i-3B47074E'
I have the error message:
Unknown option: basepath
So I guess this is why it mentioned "Failed to create libvirt XML config for instance" in the log file. But what's the use of basepath? Which part went wrong? The gen_libvirt_xml or some other code that calls --basepath option?

Thu, 02/09/2012 - 03:17

more information

just notice that before when I didn't have the Xen hypervisor and got the "libvirt: internal error no supported architecture for os type 'hvm' (code=1)" errors, the log looks like this:
[Tue Feb 7 13:13:56 2012][022506][EUCADEBUG ] system_output(): [//usr/lib/eucalyptus/euca_rootwrap //usr/share/eu calyptus/gen_kvm_libvirt_xml --ramdisk --ephemeral --basepath='/var/lib/eucalyptus/instances//admin/i-4A71093B'] [Tue Feb 7 13:13:56 2012][022506][EUCAINFO ] currently running/booting: i-4A71093B [Tue Feb 7 13:13:56 2012][022506][EUCAERROR ] libvirt: internal error no supported architecture for os type 'hvm' (code=1) [Tue Feb 7 13:13:56 2012][022506][EUCAERROR ] libvirt: internal error no supported architecture for os type 'hvm' (code=1) [Tue Feb 7 13:13:56 2012][022506][EUCAERROR ] libvirt: internal error no supported architecture for os type 'hvm' (code=1) [Tue Feb 7 13:13:56 2012][022506][EUCAERROR ] libvirt: internal error no supported architecture for os type 'hvm' (code=1) [Tue Feb 7 13:13:56 2012][022506][EUCAERROR ] libvirt: internal error no supported architecture for os type 'hvm' (code=1) [Tue Feb 7 13:13:56 2012][022506][EUCAFATAL ] hypervisor failed to start domain

I notice that the code still calls gen_libvirt_xml, and there is no error. Instead, the status become currently running/booting: i-4A71093B. So what's the reason that it becomes a problem when the hypervisor now supports hvm???

Anyone has an idea?

Thu, 02/09/2012 - 06:26

temporary solution

it seems that gen_libvirt_xml does not use the option basepath while gen_kvm_libvirt_xml does, don't know why the system still add this option when using Xen hypervisor. Anyway, what I currently do is to change the code in gen_libvirt_xml to add this dummy option in order to avoid the error. Now the instance is able to run.

But since that Xen does not support euca-get-console-output, I can only go to the nc and use xm console. Then it needs me to input login. How I could know that!!!! I download the image from the Archived EMIs, and there's no indication of the default login. Seems that I have to create my own EMI. This experience is so suffering...

btw, do anyone know how to access the instance by ssh? I don't know the principle to set the private/public address in the configuration file. In the log file the publicIP is 0.0.0.0; when using euca-describe-instances it display some publicIp within the range that I configured before, but i don't think it's a valid publicIP and I cannot access it.

Fri, 02/10/2012 - 07:12

report of some weird observation

Before I was doubting if Xen is running correctly or not because xm list works but virsh list doesn't. Now I found that if you are "root" (not sudo virsh list, but sudo -s; virsh list), I can see Domain-0 in the list. What's more, if you watch to check the version, when you are user, you see:
Using API: Xen 3.0.1 Cannot extract running Xen hypervisor version
when you use "sudo virsh version", you see:
Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.14.1
when you are root, you see:
Using API: Xen 3.0.1 Running hypervisor: Xen 4.1.0

Very weird, right? Maybe it's only my special case. You know, different version of systems and software. Anyway, at least now I am sure that Xen works on my machine :D