[06:28] <davinci11> can't start instance and get this output [ FinishedVerify: Not enough resources (0 < 1: vm instances.Not enough resources (0 < 1: vm instances.]
[06:49] <nci> davinci11: if you are using euca2ools, what is the output of euca-describe-availability-zones verbose?
[06:50] <davinci11> AVAILABILITYZONE	|- vm types	free / max   cpu   ram  disk
[06:50] <davinci11> AVAILABILITYZONE	|- m1.small	0000 / 0000   1    192     2
[06:50] <davinci11> AVAILABILITYZONE	|- c1.medium	0000 / 0000   1    256     5
[06:50] <davinci11> AVAILABILITYZONE	|- m1.large	0000 / 0000   2    512    10
[06:50] <davinci11> AVAILABILITYZONE	|- m1.xlarge	0000 / 0000   2   1024    20
[06:50] <davinci11> AVAILABILITYZONE	|- c1.xlarge	0000 / 0000   4   2048    20
[06:50] <nci> so that means your compute nodes aren't registered within the cluster
[06:51] <davinci11> i tried to deregister and register nodes again
[06:51] <davinci11> but still same
[06:51] <nci> look at euca_conf --list-nodes
[06:52] <davinci11> no output
[06:52] <nci> :(
[06:52] <nci> maybe try removing the cluster and re-adding it, then adding the nodes?
[06:53] <nci> also check the keys
[06:53] <davinci11> tried as well too
[06:54] <davinci11> even remove the key and created new one
[06:55] <nci> sorry, that's about as much as I know. I got Eucalyptus working after reinstalling a few times. I remember having issues if the auto-discover didn't work during the uec install.
[06:57] <davinci11> no other way without reinstall ?? :(''
[06:59] <nci> Maybe someone with more eucalyptus experience can help you, I'm not familiar enough with it. I assume you've checked the config files to make sure the nodes are looking at the right address, and that the correct ports are open?
[07:00] <davinci11> look at nodes machine ? or cc
[07:00] <flaccid> maybe tits literal
[07:00] <flaccid> not enough resources
[07:00] <flaccid> try googling the error
[07:01] <davinci11> already tried what i found
[07:01] <nci> go to the compute nodes and check you can connect to the right port on the CC using telnet
[07:03] <davinci11> i'm using ssh to CC is that means ok?
[07:03] <nci> ssh goes over port 22, you want to check the node can connect to the CC over a different port that eucalyptus is using
[07:04] <davinci11> how to check ?
[07:05] <nci> On the front-end, ports 8443, 8773, 8774 and 9001 must be open; on the node, port 8775 must be open.
[07:06] <nci> so on the CC, try telnet [IP-address-of-node] 8775
[07:06] <nci> if the connection is accepted then the port is open and the eucalyptus compute node is ok
[07:07] <nci> then SSH to the compute node, try to telnet [IP-address-of-CC] 8773
[07:07] <nci> and check the other numbers as well.
[07:07] <davinci11> cant telnet from CC to nodes
[07:07] <davinci11> port 8775
[07:09] <nci> check if the eucalyptus process is running on the nodes - SSH to the node and run ps aux | grep euca
[07:13] <davinc1> root      1824  0.0  0.1   5108  2148 ?        Ss   12:22   0:00 apache2 -f /var/run/eucalyptus/httpd-nc.conf -D FOREGROUND
[07:13] <davinc1> 107       1913  0.0  0.2  40048  4748 ?        Sl   12:22   0:01 apache2 -f /var/run/eucalyptus/httpd-nc.conf -D FOREGROUND
[07:13] <davinc1> root      2071  0.0  0.0   2352   912 ?        Ss   12:22   0:00 avahi-publish -s cluster1 node _eucalyptus._tcp 8775 txtvers=1 protovers=1.5.0 type=node
[07:13] <davinc1> ubc1     23024  0.0  0.0   3608   868 pts/1    S+   15:09   0:00 grep --color=auto euca
[07:13] <nci> ok
[07:13] <nci> so the process is running
[07:14] <nci> try telnet localhost 8775
[07:14] <nci> on the compute node
[07:14] <davinci11> Trying 127.0.0.1...
[07:14] <davinci11> Connected to localhost.
[07:14] <davinci11> Escape character is '^]'.
[07:14] <nci> ok
[07:15] <nci> escape from that
[07:15] <nci> so I'm not 100% sure, but it seems like your CC can't connect to your NC
[07:15] <nci> which is usually a firewall issue
[07:16] <flaccid> portmap will check the port satus
[07:16] <flaccid> *status sorry
[07:18] <davinc1> Chain INPUT (policy ACCEPT)
[07:18] <davinc1> target     prot opt source               destination
[07:18] <davinc1> ACCEPT     udp  --  anywhere             anywhere            udp dpt:domain
[07:18] <davinc1> ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:domain
[07:18] <davinc1> ACCEPT     udp  --  anywhere             anywhere            udp dpt:bootps
[07:18] <davinc1> ACCEPT     all  --  192.168.122.0/24     anywhere
[07:18] <davinc1> ACCEPT     all  --  anywhere             anywhere
[07:18] <davinc1> REJECT     all  --  anywhere             anywhere            reject-with icmp-port-unreachable
[07:18] <davinc1> REJECT     all  --  anywhere             anywhere            reject-with icmp-port-unreachable
[07:18] <davinci11> iptables from CC
[07:21] <nci> I'm not sure...sorry
[08:12] <TeTeT> davinc1: maybe your cloud is not setup correctly? what's the output of euca-describe-availability-zones verbose?
[08:13] <davinci11> AVAILABILITYZONE	|- vm types	free / max   cpu   ram  disk
[08:13] <davinci11> AVAILABILITYZONE	|- m1.small	0000 / 0000   1    192     2
[08:13] <davinci11> AVAILABILITYZONE	|- c1.medium	0000 / 0000   1    256     5
[08:13] <davinci11> AVAILABILITYZONE	|- m1.large	0000 / 0000   2    512    10
[08:13] <davinci11> AVAILABILITYZONE	|- m1.xlarge	0000 / 0000   2   1024    20
[08:13] <davinci11> AVAILABILITYZONE	|- c1.xlarge	0000 / 0000   4   2048    20
[08:14] <davinci11> it work before, but now can start anymore
[08:15] <TeTeT> davinc1: you seee the 0000, that means your node controllers are not in the cloud anymore
[08:15] <TeTeT> davinc1: did you reboot the systems or something like that?
[08:15] <davinci11> yeah. both CC and NC rebooted
[08:15] <davinci11> still same
[08:18] <TeTeT> davinc1: from the front-end, do a $ sudo euca_conf --list-nodes
[08:19] <davinci11> no output
[08:19] <TeTeT> davinc1: so for whatever reasons your node controller was dropped from eucas config
[08:20] <TeTeT> davinc1: try a $ sudo euca_conf --discover-nodes
[08:20] <TeTeT> and see if the node controller is picked up again
[08:21] <TeTeT> davinc1: what Ubuntu version do you use btw?
[08:21] <davinci11> INFO: We expect all nodes to have eucalyptus installed in //var/lib/eucalyptus/keys for key synchronization.
[08:21] <davinci11> output of $ sudo euca_conf --discover-nodes
[08:22] <TeTeT> davinc1: weird, what would remove the keys from there on the front-end? I'm a bit out of ideas right now
[08:22] <davinci11> Ubuntu Enterprise Cloud (UEC) 11.04
[08:23] <TeTeT> davinci11: can you ping the node controller from the front-end? maybe $ sudo euca_conf --register-nodes <ip node controller> will work?
[08:24] <davinci11> yep both work
[08:24] <TeTeT> davinci11: try the euca-describe-availability-zones verbose again
[08:25] <davinci11> same
[08:25] <TeTeT> and euca_conf --list-nodes?
[08:26] <davinci11> still no output
[08:27] <TeTeT> davinci11: what's the oputput of $ sudo euca_conf --register-nodes <ip of node controller>?
[08:28] <davinci11> INFO: We expect all nodes to have eucalyptus installed in //var/lib/eucalyptus/keys for key synchronization.
[08:28] <davinci11> Trying rsync to sync keys with "192.168.168.20"...done.
[08:30] <TeTeT> davinci11: that looks ok, when you do a '$ sudo su - eucalyptus' on the front-end
[08:30] <TeTeT> davinc11: and then ssh to the node controller, does that work?
[08:32] <davinci11> yes is working
[08:33] <TeTeT> davinci11: looks like a bug to me then. Daviey, do you have an opinion on the above node registration problem on 11.04?
[09:04] <TeTeT> davinci11: maybe this bug is hitting you: https://bugs.launchpad.net/ubuntu/+source/eucalyptus/+bug/585108
[09:04] <TeTeT> davinci11: probably dereigstering the node and then trying to add it again will help, but I'm not sure
[09:07] <davinci11> TeTeT: I tried that way before, but still not work
[09:09] <TeTeT> davinci11: hmm, on the front-end, try a $ sudo restart eucalyptus CLEAN=1
[09:09] <TeTeT> davinci11: and then try the registering dance again
[09:13] <davinci11> still same
[17:41] <marrusl> kim0, great blog post today.  i'm sitting in a hadoop class right now.  nice timing.
[17:41] <kim0> marrusl: hehe nice
[17:41] <kim0> marrusl: is canonical supporting hadoop professionally now
[17:43] <marrusl> kim0, for premium customers I would think so yes.  but probably not in general unless it went into main.
[17:43] <kim0> got it
[17:54] <koolhead17> hi all