/srv/irclogs.ubuntu.com/2011/03/25/#ubuntu-cloud.txt

=== dendrobates is now known as dendro-afk
koolhead17morning all04:46
flaccidevening04:51
koolhead17kim0, hello05:30
koolhead17hi flaccid how have you been?05:30
flaccidbusy as05:33
koolhead17kim0, ping me once your here, got some good news :)05:35
koolhead17flaccid, hmm05:35
flaccidwhats the goss05:38
koolhead17*goss?05:39
flaccidaka gossip05:39
flaccidwhatever that news is05:39
koolhead17flaccid, :P05:40
flaccidmeh trolls. i'm going to dinner.05:55
koolhead17flaccid, lol06:01
kim0Morning folks08:02
kim0koolhead17: hey o/08:02
koolhead17hi kim0 :)08:22
=== daker_ is now known as daker
koolhead17kim0,12:00
kim0koolhead17: hey12:00
koolhead17kim0, how was it :D12:01
kim0koolhead17: very good :)12:01
koolhead17k12:03
* rwat starts to tear hair out.12:43
rwatany idea where to look for clues as to why my instances go straight from pending to terminated?12:44
rwatNC is able to run kvm12:44
rwatok they don't quite go straight. They sit at pending for a few minutes12:45
rwatthen terminate - no sign of any running12:45
rwatlike something times out12:45
=== dendro-afk is now known as dendrobates
kim0rwat: check out nc.log in /var/log/eucalyptus13:33
=== dendrobates is now known as dendro-afk
=== dendro-afk is now known as dendrobates
rwatkim0: I just went to try to launch an instance so I could tail nc.log, and I'm now getting: FinishedVerify: Not enough resources (0 < 1: vm  instances.Not enough resources (0 < 1: vm instances.14:39
rwatwhich is a bit different14:39
kim0try  maybe we could then propose a sprint in Europe. Where we14:40
kim0meet for a few days to discuss plans for 11.10.  Canonical would be open14:40
kim0to funding travel and accommodation for 2-3 x2go folks for this.14:40
kim0oops nvm that14:40
kim0rwat: try this .. euca-describe-availability-zones verbose14:41
rwatit lists the IP address of my cloud controller and some vm types all showing 0000 out of various amounts14:42
rwatlike 0000/0002 0000/0002 etc14:43
rwatso if 0000 are free then it can't find anywhere to run the image?14:43
kim0yes14:43
kim0rwat: basically I think your NC is not registered14:44
kim0the cloud thinks it's out of resources14:44
rwatok14:44
kim0try deregistering .. and reregistering14:44
rwatok14:44
kim0the nc node14:44
kim0and I am not quite sure how to do that :)14:44
rwatwoudl that be euca_conf --deregister-nodes  ?14:45
rwatderegistering gives me: SUCCESS: removed node '192.168.101.70' from '//etc/eucalyptus/eucalyptus.local.conf'14:45
rwatregistering gives : Trying rsync to sync keys with "192.168.101.70"...done.14:46
rwateuca-describe-availability-zones verbose still gives 0000/anything14:46
rwatok if I stop the node controller that goes to 0000/0000 rather than 0000/000214:47
rwatI restart the NC and it goes back to 0000/000214:48
rwatso they are talking14:48
kim0rwat: do you restart eucalyptus with CLEAN=1 parameter14:49
rwatnot so far14:49
kim0that's sometimes needed to flush the cache14:50
kim0please try that14:50
rwathow do I do that?14:50
kim0on all services14:50
kim0/etc/init.d/eucalyptus restart CLEAN=114:50
rwatok14:50
kim0and for other services too14:50
kim0google that14:50
smoserkim0, euca uses upstart14:50
smoserno ?14:50
smoserrestart eucalyptus CLEAN=114:50
kim0ew no idea :)14:50
kim0rwat: ^^14:51
kim0doesn't upstart jobs support old style too14:51
kim0dont*14:51
rwatkim0: smoser I think you can use both14:51
* kim0 nods14:51
* kim0 can't get over tab completion :)14:52
smoseri dont know that the CLEAN=1 paramas will get through the init.d wrapper. they probably will14:52
obinosmoser: yes they will15:05
obinoour init.d scripts uses cleanstart and cleanstop to clean15:05
obinonot CLEAN=115:05
obinothis was done for upstart15:05
rwatkim0: ok well now I have 0002/0002 m1.small types free15:07
kim0wow I guess that's an improvement :)15:08
rwatand now it lets me get a pending instance15:08
kim0obino: Hi o/15:08
kim0feel free to jump in and help :)15:08
obinohi kim015:08
kim0rwat: tailf nc.log and let's see what happens15:09
obinocongratz again for the cloud days session: I'm going through the session I couldn't attend :)15:09
kim0obino: awesome, thanks man .. really appreciate it15:16
kim0you presenting that is15:17
obinoanytime :)15:18
kim0:)15:18
=== dendrobates is now known as dendro-afk
rwatkim0: OK the instance goes into pending, and then gets terminated. After that m1.small only shows 0001/000215:29
rwatso it seems that not only does it not start properly, it somehow fails to reset its counter15:29
kim0what does nc.log say then15:30
rwatnothing excetp15:30
rwat[Fri Mar 25 15:30:41 2011][005969][EUCADEBUG ] doDescribeResource() invoked15:30
rwat[Fri Mar 25 15:30:41 2011][005969][EUCADEBUG ] doDescribeInstances() invoked15:30
rwatrepeated a few times15:31
kim0don't paste it here15:31
kim0use paste.ubuntu.com15:31
rwatok15:31
rwatbut there's no more than that over and over15:31
kim0eh15:31
kim0not too useful15:31
kim0rwat: try looking at other log files15:31
rwatyeah, I've been trying to wade throuhg them but there's such a huge load of them15:32
kim0rwat: is "sudo kvm-ok" happy on NC node ?15:32
rwatkim0: yes it tells me KVM acceleration can be used15:45
kim0weirdo .. then I'm fairly lost15:46
rwatyeah, well I know I am15:46
rwatwhat should normally be in nc.log?15:47
rwatahh now I'm getting a lot of errors in axis2c.log about invalid keys15:48
kim0great :)15:48
rwathttp://paste.ubuntu.com/585450/15:49
rwatI have to say that is entirely greek to me15:49
kim0rwat: did you create a new (non admin) user and are using that ?15:51
kim0that's usually helpful :)15:51
kim0that error is greek to me too15:51
rwatok I'll create a new user15:52
rwatwhat is axis2c?15:53
kim0rwat: with the new user, don't forget to euca-add-keypair for it15:53
kim0I just love java :)15:53
crazedjavajava15:54
rwatok I'm trying to start an instance with the new credentials15:57
* rwat hates java15:57
* kim0 was being sarcastic :)15:59
=== dendro-afk is now known as dendrobates
rwatkim0: and my instance just went from pending to terminated again16:07
* rwat prepares to throw cloud out of window16:07
* rwat remembers he has a pulled muscle and stops16:08
kim0rwat: hmm maybe ask in #eucalyptus16:08
TeTeTrwat: is the kvm process started on the NC at all? For me it was always useful to peek at the nc.log on the NC with grep -v doDescribe /var/log/eucalyptus/nc.log16:14
TeTeTrwat: if nothing is displayed when you do a run-instances, then the cc or clc is broken16:14
rwatTeTeT: kvm doesnt seem to start16:28
crazedcheck your bios16:28
rwatcrazed: done that - its all fine16:29
TeTeTrwat: how far does it get? Is the image transferred to the NC?16:29
crazedcan you boot a regular VM with kvm?16:29
rwatTeTeT: i don't believe it is, but then I'm not quite sure what I'm looking for16:30
rwatcrazed: I haven't tried to run a regular image with kvm, I could give that ago16:30
TeTeTrwat: do the tail -f /var/log/eucalyptus.log | grep -v doDescribe16:30
rwatTeTeT: there aren't any lines created in nc.log that don't start with doDescribe16:32
rwator don't have doDescribe in them16:32
rwatit doesn't look to me as though it gets as far as the NC16:33
TeTeTrwat: yeah, it's blocked by the CC, CLC, not sure what to do then, sorry16:34
crazeddefinitely try to launch a vm with kvm16:34
crazedalways start at the hypervisor and work your way up :)16:34
YuvarajHello Kimo16:40
kim0Yuvaraj: hey16:41
kim0Yuvaraj: o/16:41
Yuvarajcould you paste the boto url,16:42
kim0ah yeah sure16:42
kim0http://boto.cloudhackers.com/ec2_tut.html16:42
Yuvarajthere is some fault with my system, its not at all saving the history16:42
Yuvarajand missed the UCD yesteraday :(16:43
kim0Yuvaraj: read the logs :)16:43
Yuvarajdue to system fault16:43
kim0Yuvaraj: http://foss-boss.blogspot.com/2011/03/ubuntu-cloud-days-day-2-summary.html16:43
Yuvarajyes sure :)16:43
Yuvarajthank you :)16:43
kim0You're welcome16:43
SpamapSsmoser: that udev/cloud-init bug is nasty16:45
smoseryou've made some progress ?16:46
smoserreproducing ~ 4% of the time is no fun16:46
SpamapSsmoser: only in understanding how weird udev is16:46
smoserand i don't know that i've ever seen it on a reboot16:46
SpamapSThe bitwhere /dev/null can't be opened is weird too, but I think a red herring16:46
SpamapSsmoser: all those Connection Refused messages suggest that there's something wrong in the sequence of fork/listen on the workers16:48
SpamapSbut, they communicate over pipes.. so.. I'm having to look now into how a pipe even returns ECONNREFUSED16:48
SpamapSdefinitely one of those times where I wish I had ctags working properly. :-/16:50
* SpamapS realizes now that he can fix that in 10 seconds.. and does16:52
rwatcrazed: ok, my NC seems to run ordinary kvm just fine18:15
crazedhm well that's a good first start, but unfortunately my knowldege of eucalyptus is not that great. i primarily use cloudstack at this time18:17
rwatcrazed: is cloudstack ready for eral use?18:23
rwatreal18:23
crazedi'd say it is18:29
crazedbut it runs on rhel6 if you want to use kvm18:29
crazedit's got some ghettoness about it though..18:30
crazedreally i think nimbula is the most promising18:30
=== daker is now known as daker_
=== dendrobates is now known as dendro-afk
hallynkim0: not trying to dodge your bug, i just really think it's a bug in debootstrap :)20:24
hallynall right, now why won't winxp install in kvm under lucid for me?20:24
kim0yeah I agree :)20:24
kim0hallyn: switched the bug to debootstrap20:26
kim0let me know if I should do something else20:26
hallynkim0: oh, i'd already done that myself :)20:27
kim0ah ok :)20:27
zulhallyn: is that the lxc deboostrap amd64 bug?20:27
hallynkim0: well, i *was* wondering if there is some undocumented debootstrap config file20:27
hallynzul: it's the one where it doesn't install netbase20:28
zulhallyn: i think the amd64 iso images had a similar problem afaik colin fixed it20:28
zulkim0: maybe try a different mirror20:28
hallynzul: what exactly was the problem?  The mirror didn't have the package available?20:29
kim0zul: colin fixed it today ?20:29
kim0coz I just tried it a few hours back20:29
hallynzul: doh, there's a bug in the actual templates :(20:38
hallyncomes from looking at the same time at upstart jobs and writing bash20:38
SpamapSsmoser: so the udev error comes because sendmsg couldn't send on the PF_NETLINK that udev uses to talk to the kernel about device events20:44

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!