[02:37] <bigjools> roaksoax: https://code.launchpad.net/~julian-edwards/maas/missing-import-bug-1065055/+merge/129080
[02:39] <roaksoax> bigjools: can't approave it
[02:39] <roaksoax> bigjools: it will conflict with maas-region-controller
[02:41] <roaksoax> bigjools: i guess it will have to end up being shipping in maas-common and will have to add conflicts/replaces
[02:42] <roaksoax> s/shipping/shipped
[02:52] <roaksoax> ./win 8
[04:12] <bigjools> roaksoax: grar
[04:17] <bigjools> roaksoax: fixed, take a look if you are still there?
[07:57] <Daviey> roaksoax, smoser: sorry had to go
[08:16] <Daviey> allenap: maas-cli is certainly less than user friendly
[08:17] <Daviey> http://pb.daviey.com/QdsM/
[08:19] <jtv> Daviey: you want "maas-cli login" to act as "maas-cli login -h" ?
[08:20] <Daviey> jtv: i'd say so... I worked it out tho
[08:21] <Daviey> jtv: mind you, http://pb.daviey.com/jIU4/
[08:21] <Daviey> still not that clear WTF i am supposed to do.
[08:21] <jtv> Daviey: well what you have there shows the API commands you can issue directly.
[08:22] <jtv> Try "maas-cli maas node-group -h", for instance.
[08:23] <Daviey> jtv: right, but do you see how it feels like i'm using a dev tool like ipython?
[08:23] <jtv> Absolutely.  It's tough to get going with.
[08:25] <Daviey> jtv: so, deleting a node
[08:26] <Daviey> jtv: so, deleting a node
[08:26] <Daviey> I need to find the node first.. how?
[08:26] <Daviey> davewalker@maas:~$ maas-cli maas node-group list-nodes uuid
[08:26] <Daviey> Not Found
[08:27] <jtv> What's the uuid for?
[08:28] <Daviey> http://pb.daviey.com/cqtV/
[08:28] <Daviey> ah
[08:29] <Daviey> maas-cli maas node delete $system-id
[08:29] <Daviey>  maas-cli maas nodes list | grep system_id
[08:31] <jtv> Looks like we have some underdocumented methods in the API.  :(
[08:53] <rvba> jam: I was wrong, the memory was reported ok on my microservers.
[08:58] <mgz> jam: in reply to your xpath comments,
[08:59] <mgz> you're right, it's much more painful until postgres 9.2 because of the must-return-nodeset limitation
[08:59] <jam> mgz: care to mumble?
[08:59] <mgz> be a little careful comparing things to nodesets though
[08:59] <mgz> occasionally the result is suprising
[09:39] <allenap> Daviey: I've been thinking about that. maas-cli is a thin wrapper around the API, but there's an impedance mismatch there.
[09:39] <Daviey> allenap: there doesn't need to be, does there?
[09:39] <allenap> Cousin Cletus (aka the shell) just can't deal with the richness of the API.
[09:40] <Daviey> allenap: bash tab-complete?  NEVER mentioning: usage: /usr/lib/python2.7/dist-packages/maascli/__main__.py maas
[09:40] <Daviey>        [-h] COMMAND ...
[09:40] <Daviey> defaulting to providing -h if params are missing?
[09:40] <allenap> Daviey: Perhaps for 13.04 we need to come up a more task-orientated command-line tool, and leave the API tinkering in a library.
[09:41] <allenap> Daviey: Yeah, those are things we ought to have, but they're polish.
[09:41] <allenap> I think there's a fundamental problem with trying to manipulate highly structured data from the shell.
[09:41] <rvba> jtv: could you please review: https://code.launchpad.net/~rvb/maas/bug-1065406/+merge/1291
[09:41] <Daviey> allenap: polish or DESIGN? :)
[09:43] <allenap> Daviey: Polish. We want to do tab completion. Changing the error behaviour is something that has come out in practice.
[09:43] <allenap> The need to change it I mean.
[09:44] <allenap> Also, I mean: we have always wanted to do tab completion.
[09:44] <allenap> It's not a new thing.
[09:44] <Daviey> right
[09:44] <Daviey> Is longpoll still busted?
[09:44] <Daviey> not seeing the dash dynamically update?
[09:45] <rvba> This should have been fixed by now.
[09:46] <allenap> Daviey: Frankly I think shell is the wrong place to do anything other than start something else. What can you do when you ask for a listing of allocated nodes? bash can't grok that structure; we immediately need to go out to awk or something, which leads to scripts that are held together by dog lick.
[09:47] <Daviey> allenap: which is exactly what i have done..
[09:47] <allenap> Yeah :-(
[09:47] <rvba> Daviey: fixed in r127 of the packaging branch.
[09:47] <Daviey> but actually, as a application driven api.. it could be worse.
[09:47] <Daviey> rvba: I wonder if it's because i am using SOCKS
[09:48] <rvba> Should be ok.  Apache is acting a proxy to the longpoll server.
[09:49] <rvba> Daviey: could you try to access the longpoll url?  (if you get a Django 404 there is a problem because Django should not be serving that url)
[09:50] <rvba> haha :)
[09:50] <Daviey> rvba: linky for the lazy
[09:50] <Daviey> ?
[09:51] <Daviey> longpoll url
[09:51] <rvba> Just one sec.
[09:57] <rvba> Daviey: http://localhost:8080/MAAS/longpoll/?uuid=blah
[09:57] <Daviey> ta
[09:57] <rvba> You should get "Invalid request" (because the uuid is bogus) and not a reply from Django.
[09:58] <Daviey> rvba: 404,  The requested URL /MAAS/longpoll/ was not found on this server.
[09:58] <Daviey> http://192.168.9.1/MAAS/longpoll/?uuid .. right?
[09:58] <rvba> Daviey: looks like the fix from r127 is not there.
[09:58] <rvba> Yes.
[09:58] <rvba> Daviey: sudo a2enmod proxy_http && sudo service apache2 restart
[09:59] <Daviey>   Installed: 0.1+bzr1243+dfsg-0+1252+129~ppa0~quantal1
[09:59] <Daviey> Now i get, "Async frontend for None"
[10:00] <rvba> Daviey: can you try loading proxy_http?
[10:00] <Daviey> rvba: your line worked, no?
[10:00] <rvba> Yeah, but that is exactly what the fix in r127 should have done.
[10:01] <Daviey> http://pb.daviey.com/pVKb/
[10:02] <rvba> Daviey: right, so proxy_http was not enabled.
[10:03] <rvba> Daviey: can you please double check wit: cat /var/lib/dpkg/info/maas-region-controller.postinst | grep proxy_html
[10:03] <rvba> (yes, I know, the cat/grep combination is a bit stupid)
[10:04] <Daviey> rvba: I think it's covered here, http://partmaps.org/era/unix/award.html
[10:05] <rvba> heh
[10:05] <Daviey> confirmed, not in there
[10:05] <rvba> WTF
[10:06] <Daviey> the good news is IPMI discovery is working REALLLLLLY well.
[10:06] <rvba> \o/
[10:07] <rvba> Daviey: I don't understand, "a2enmod proxy_http" is in debian/maas-region-controller.postinst revision 129.
[10:08] <Daviey> No idea :/
[10:11] <rvba> Maybe roaksoax will have an idea.  Smells fishy to me.  Packaging fishy.
[10:16] <Daviey> stinks!
[10:21] <jtv> rvba: not finding that review you linked to
[10:22] <rvba> jtv: https://code.launchpad.net/~rvb/maas/bug-1065406/+merge/129131
[10:22] <jtv> That works.
[10:26] <jtv> STOP EATING MY SHOES!
[10:27] <jtv> They make sneakers much too light nowadays.  Cats can drag them out of the shoe rack.
[10:28] <rbasak> Just tried highbank on quantal and ipmi setting has completely disappeared. Nothing on enlistment and nothing on commissioning. Is this expected?
[10:29] <rbasak> http://paste.ubuntu.com/1272933/
[10:30] <rbasak> smoser, Daviey: ^^
[10:33] <mgz> jtv: clearly you need to walk around the house in clogs only, cats would struggle to make an impression on those
[10:33] <jtv> Good idea.
[10:33] <jtv> Thing is, where do I put my sneakers in the mean time?  You don't wear shoes in the house here.
[10:33] <jtv> I'm inside, the cats are outside.
[10:33] <jtv> Eating my sneakers.
[10:34] <jtv> They're new, too.  The old ones were at least a year beyond their best-by date, and as soon as I get myself a new pair, I have to visit a house that has cats prowling around.
[10:34] <jtv> Looking for shoes to pounce on.
[10:37] <mgz> maybe you could buy a very tall shoe tree. like a hat stand, but to hang shoes off
[10:37] <mgz> then the cats would at least have to climb for their prize
[10:38] <jtv> You haven't seen what these cats do.
[10:39] <jtv> I've seen them walk up bug screens like Spider-man.
[10:39] <mgz> ehehe
[10:39] <jtv> Walk, I tell you.  Two meters up, then jump up on the curtain rails.
[10:43] <jtv> rvba: the regular celeryconfig does not have the import_settings problem, just the cluster celeryconfig?
[10:44] <rvba> jtv: right: debian/maas-region-controller.maas-region-celery.upstart contains env PYTHONPATH="/usr/share/maas/" so the region celery process has access to the utility method.
[10:45] <jtv> Oh, that's used only from the upstart job
[11:03] <jam> jelmer: how's memory stuff going?
[11:08] <jelmer> jam: I was about to get lunch; I'm close to being able to push a branch up, was working on tests.
[11:08] <jam> jelmer: enjoy your lunch, just checking in to see how it was going.
[11:10] <jam> allenap: 'timeit' disagrees with you. indent=4 was only slightly slower, enforce_ascii was a huge hit.
[11:11] <mgz> allenap: I think generally having pretty printing of json isn't too bad, as most api results are small and with cli get dumped straight onto the console
[11:11] <jam> 10ms for dumps, 12ms for dumps with indent=4
[11:11] <mgz> it would be easy enough to reformat client side, but just setting a few internal calls to bypass prettiness seems okay
[11:12] <jam> ensure_ascii=False => 114ms.
[11:17] <rvba> roaksoax: I've unlinked the upstream project from bug 1065080.  I've talked to Julian about it and after some discussion, we think it would be more clean to have it fixed in the packaging (see my comment on the bug).
[11:31] <jam> jtv: yeah, the cats in our yard jump from the ground about 2 meters up to get up on the wall. I think they push off the wall a bit on the way up, but it is pretty impressive.
[11:32] <jam> mgz, allenap: I think the ensure_ascii difference is because with it set to True (default) you get back an 8-bit str that escapse all the non-ascii bytes. Vs getting back a Unicode string.
[11:32] <jtv> Love 'em.
[11:33] <jam> The only thing I can think of, is that simplejson internally ends up using a unicode string, and appends chars to it (or some other silly thing), vs using a proper buffer.
[11:35] <jam> allenap: I think I found it. "encode_basestring_ascii" has a C level accellerator, but 'encode_basestring' does not.
[11:38] <jam> So in this case, we have 1 24kb string, so it doesn't matter as much if the *encoder* object is C, it matters more if the function called to encode that string is in C.
[11:44] <rvba> rbasak: maybe you can land lp:~racb/maas/comissioning…  Looks very low-risk to me ;)
[11:51] <rbasak> rvba: thanks. Had forgotten about that!
[12:00] <Daviey> Hmm
[12:00] <Daviey> If you destroy-enviroment and juju boostrap, is it expected that it loses it's DDNS?
[12:24] <mgz> ...how do I see an oops so I know what I broke?
[12:24] <mgz> the traceback doesn't seem to get logged...
[12:25] <mgz> ..disregard that, looking in the wrong dir
[12:31] <smoser> rbasak, you still busted on that ?
[12:31] <rbasak> smoser: yes
[12:32] <rbasak> smoser: precise ephemeral works. Quantal does not.
[12:32] <Daviey> hmm.. i thought that was fixed last night?
[12:32] <rbasak> I installed from quantal this morning
[12:32] <smoser> this is commissioning?
[12:33] <rbasak> yes
[12:33] <rbasak> maas	0.1+bzr1243+dfsg-0ubuntu3
[12:33] <smoser> rbasak, can i see?
[12:34] <rbasak> smoser: sure. Same machine as last time.
[12:34] <rbasak> smoser: which bits do you need?
[12:36] <smoser> seems like bzr importer failed on maas-enlist
[12:36] <smoser> suck
[12:42] <rbasak> smoser: I'm using released quantal.
[12:42] <rbasak> smoser: just realised
[12:42] <rbasak> smoser: do I need daily quantal?
[12:42] <rbasak> Or shouldn't it matter? I don't see why it would
[12:43] <smoser> rbasak, at that point you're pulling from the archive.
[12:43] <smoser> the ephemeral fixes have primarily landed in the earlier sstages of boot and cloud-init
[12:43] <smoser> but you're getting past that.
[12:46] <mgz> amusing but non-fatal: http://pastebin.ubuntu.com/1273139/
[12:49] <mgz> wip, needs tests: lp:~gz/maas/paginate_nodes_page_1064672/
[12:53] <jam> rvba: is there a good place to look up documentation about how to set up maas with a bunch of clusters?
[12:53] <jam> I'm going to be doing some stress testing of the hardware constraints stuff, trying to set up a very large system (100,000 nodes).
[12:53] <jam> And I figure I need a DB server, N_small appservers, N_large (~25) cluster workers.
[12:54] <rvba> jam: I don't think there is documentation for that but that's precisely what I'm testing atm.
[12:54] <rvba> I'm not doing stress testing, just setting up a cluster controller on a separate machine.
[12:54] <jam> rvba: hints/docs/notes/access or whatever would be appreciated.
[12:55] <rvba> jam: we need the fix that jtv is currently landing.
[12:55] <jtv> Which one?
[12:55] <jam> jtv: import_*
[12:55] <jtv> Because one has already landed
[12:55] <jam> I believe
[12:55] <jtv> Ah
[12:55] <jtv> That's waiting for the cron job.
[12:55] <rvba> The fix for bug 1065055.
[12:56] <rbasak> smoser: I've filed bug 1065499
[12:56] <rvba> jam: I'll build a package in my ppa once this will be landed.
[12:56] <rvba> Then you only need to do 2 things:
[12:57] <rvba> Manually change the BROKER_URL setting on the region cluster celery config (s/localhost/ip.address/).
[12:57] <rvba> When you install a cluster, dpgk-reconfigure maas-cluster-controller => give the URL of the region
[12:58] <rvba> jam: that should be all.
[12:58] <jam> rvba: so the 'region' is the master?
[13:01] <rvba> jam: not sure what you mean by that.  There is a 'master' cluster controller installed with the region yes.
[13:02] <jam> so region to me sounds like a subgroup of something larger.
[13:02] <jam> but I don't think a region talks to anything else
[13:02] <jam> so it seems to be the largest unit.
[13:03] <rvba> The region controller is the MAAS server.
[13:03] <rvba> All the cluster controllers talk to the region controller.
[13:03] <jam> right.
[13:04] <jam> but if you had 'multiple regions' they wouldn't talk to eachother. correct?
[13:04] <rvba> No, not right now.
[13:05] <jam> k
[13:06] <jam> rvba: so I saw in the design that you're supposed to be able to have multiple 'appserver' processes (I'm guessing that is the 'maas' process). Is that something that has been tested?
[13:06] <rvba> jam: no, this has not been tested.  The goal was to use juju to scale horizontally like that.
[13:07] <jam> rvba: so theoretically you could use juju to configure maas appservers as well as cluster controllers?
[13:07] <jtv> jam, rvba: the import branch just landed.  Are you going to test it?
[13:07] <rvba> theoretically yes
[13:07] <rvba> jtv: I'll see if I can build a package.
[13:08] <jtv> Great.  Basically all we need to know is that installing a cluster without a region controller on the same machine doesn't break, right?
[13:08] <rvba> yes
[13:09] <rvba> Hunk #1 FAILED at 182.
[13:09] <rvba> 1 out of 1 hunk FAILED -- rejects in file contrib/preseeds_v2/enlist_userdata
[13:09] <rvba> Patch 04-fix-ipmi-enlistment.patch does not apply (enforce with -f)
[13:09] <rvba> arg
[13:09] <rvba> Looks like that change has made it upstrea.
[13:09] <rvba> upstream*
[13:14] <rvba> jtv: jam any idea how to fix this, I've removed the patch, removed the reference to it in debian/patches/series but now I'm getting: "No such file or directory: ..04-fix-ipmi-enlistment.patch". I've no idea where it's referenced.
[13:14] <rvba> It's only mentioned in the changelog file.
[13:14] <jam> rvba: in the patch queue?
[13:14] <jam> (.pc)
[13:15] <rvba> jam: where is that? find . -name "*.pc" => nothing
[13:15] <jam> rvba: it would be in the root directory, named '.pc/'
[13:16] <jtv> Leftovers in the build directory maybe?
[13:16] <jam> quilt was the name I was looking for.
[13:16] <jam> but rvba, jelmer is someone who knows deb packaging quite well.
[13:17] <rvba> jam: the root directory of the packaging branch?
[13:17] <jam> rvba: yes
[13:17] <jelmer> rvba: are you using bzr builddeb?
[13:17] <rvba> bzr bd -S -- -kemail
[13:17] <jelmer> rvba: you might have to run "bzr rm" (no arguments)
[13:18] <jelmer> otherwise bzr doesn't pick up that that file is removed, and bzr-builddeb will try to look for it while exporting the source tree
[13:18] <rvba> Thanks jelmer, that did the trick indeed.
[13:19] <rvba> Uploading to ppa.
[13:22] <rvba> jam: the ppa package will be ready soon (ppa:rvb/maas) all the packages are in http://people.canonical.com/~rvb/maas/.
[13:43] <mgz> ho hum hum.
[13:44] <jelmer> rvba: note that you can avoid the -k bit by putting something like this in ~/.devscripts:
[13:44] <jelmer> DEBSIGN_KEYID=1EEF5276
[13:45] <rvba> jelmer: I didn't know that, thanks for the tip.
[14:06] <rvba> jtv: your fix to the celeryconfig_cluster worked fine.
[14:06] <jtv> Load off my mind, thanks
[14:09] <rvba> So it txlongpoll is fine too.
[14:09] <rvba> Daviey: ^
[14:10] <Daviey> super
[14:16] <rvba> jtv: the dhcp_key generation thingy fixed the dhcp restart problem.
[14:22] <roaksoax> rvba: ack
[14:24] <rvba> roaksoax: the fixes for bugs 1065055 and 1065406 as in as of r1261.  I had to remove debian/patches/04-fix-ipmi-enlistment.patch to build the package (looks like the fixes in that patch are now upstream).
[14:25] <roaksoax> rvba: yep
[14:25] <roaksoax> rvba: i'm aware :)
[14:25] <rvba> All right :)
[14:26] <roaksoax> rvba: btw... maas-dns is only meant to be installed in the region?
[14:26] <rvba> roaksoax: yes
[14:26] <roaksoax> rvba: so that means that maas-dhcp too
[14:26] <rvba> No, the dhcp can be installed on the cluster side too.
[14:31] <roaksoax> rvba: ok cool
[14:31] <roaksoax> thanks
[14:35] <roaksoax> rvba: so maas-dns should not depend on maas-dhcp
[14:35] <roaksoax> should it?
[14:37] <rvba> roaksoax: no, I don't think it's required.
[14:38] <rvba> roaksoax: but I'm not sure we've tested that very well, maybe you can keep it for now.
[14:39] <rvba> roaksoax: the maas-dhcp package can be installed next to any cluster controller.  And there is one next to the region (it's a dependency).
[14:41] <roaksoax> smoser: your branch https://code.launchpad.net/~smoser/maas/ipmi-locate-fallback/+merge/129202  also fixes bug #1064527
[14:41] <roaksoax> right?
[14:41] <roaksoax> rvba: ok
[14:42] <smoser> no.
[14:42] <smoser> it doesn't
[14:42] <smoser> the issue in the nested kvm was it just took forever
[14:42] <smoser> as it got a false positive and then bmc-config....
[14:42] <roaksoax> rvba: cause the maas-dns package does maas set_up_dns on postinst, but IDK whether we want that
[14:42] <smoser> and that was haning
[14:42] <smoser> this increases that chance actually
[14:43] <roaksoax> smoser: right but your branch is a fallback method for IPMI detection, so in nested KVM it won't detect ipmi, would it?
[14:43] <rvba> roaksoax: yes, we want that, if you install maas-dns, you want maas to register its config in the running bind instance and that is what set_up_dns does.
[14:43] <roaksoax> rvba: ok cool
[14:46] <smoser> roaksoax, mine only increases the likelyhood of finding something
[14:46] <smoser> (but is very unlikely to false-positive)
[14:47] <roaksoax> smoser: ok
[14:47] <roaksoax> smoser: btw.. maas-enlist is in archives now so it should work
[14:54] <smoser> right.
[14:54] <smoser> we need to get the sru back
[14:55] <roaksoax> smoser: i already uploaded it
[14:55] <roaksoax> rbasak: could you please verify SRU bug #1056816
[14:56] <smoser> roaksoax, we need this bug fix in maas-enlist though. with the mac addrs.
[14:56] <smoser> we dont have that yet
[14:56] <smoser> do we?
[14:56] <roaksoax> smoser: for precise?
[14:56] <rbasak> smoser: any tips on testing from precise-proposed? Shall I just add all of it in sources.list? Or is there any pinning functionality I should be using?
[14:56] <roaksoax> smoser: it is uploaded to -proposed but needs to be accepted first
[14:57] <roaksoax> smoser: so I think the previous upload we can mark verification-done and then the new one will address the third bug
[14:57] <roaksoax> rbasak: yes just add -proposed and install and test
[14:57] <smoser> roaksoax, just ask that the -proposed be rejected
[14:57] <smoser> i think
[14:57] <smoser> and upload a new one
[14:57] <roaksoax> smoser: i asked that
[14:58] <roaksoax> smoser: they said it won't
[14:58] <roaksoax> smoser: ScottK told me to make a new upload with higher revision with everything of the previous, plus the new stuff
[14:58] <roaksoax> smoser: so I did
[14:59] <roaksoax> smoser: but SpamapS told me to make a new upload with a new revision that adds, or removes anything from the previous upload
[15:01] <smoser> ok. thats fine.
[15:02] <smoser> roaksoax, so you uploaded?
[15:02] <roaksoax> smoser: ye
[15:02] <smoser> i will then copy that version to maas-ephemeral images ppa
[15:02] <roaksoax> smoser: yes
[15:02] <smoser> and we'llget new precise dailies
[15:03] <roaksoax> cool
[15:04] <SpamapS> roaksoax: what scottk said, and what I said, is basically the same thing, isn't it?
[15:05] <SpamapS> roaksoax: I see 2 uploads. Which one should be reviewed?
[15:06] <roaksoax> SpamapS: The way I understood it is: 1. Make an upload which contains the previous fixes (in -proposed) + the new fix and increase the revision (ScottK). 2. (yours) Make a new upload with only the *new* patch because the other 2 are already in -proposed
[15:09] <roaksoax> SpamapS: so I guess 2. is the correct approach, right? So please accept the latest one uploaded, and reject the older one :) Thank you
[15:10] <SpamapS> roaksoax: use understanding #1. ScottK definitely put it more clearly than I did.
[15:11] <roaksoax> SpamapS: yeah so the latest upload is the correct one
[15:11] <SpamapS> roaksoax: lets back up. What you need to do is take the current -proposed package, modify it with a new revision, and upload that.
[15:12] <roaksoax> SpamapS: so the latest upload I've made has both ubuntu1.2 + ubuntu1.3
[15:12] <SpamapS> roaksoax: latest one does indeed look correct
[15:12] <SpamapS> http://launchpadlibrarian.net/119496240/maas-enlist_0.4-0ubuntu1.2_0.4-0ubuntu1.3.diff.gz
[15:12] <roaksoax> SpamapS: yes, that one :). Thanks for taking care of it :)
[15:13] <rbasak> So do I still need to verify current precise-proposed or wait for the new one?
[15:13] <roaksoax> rbasak: you can verify what's with current
[15:13] <rbasak> roaksoax: ?
[15:14] <roaksoax> rbasak: verify with what's in -proposed
[15:14] <smoser> and build debdiff versus last accepted.
[15:14] <roaksoax> rbasak: you don'yt need to wait
[15:14] <rbasak> ok
[15:14] <SpamapS> I'm accepting but..
[15:14] <SpamapS> ++		args="${args} --data-urlencode mac_addresses=${i}"
[15:14] <SpamapS> ++		#mac_address="$mac_address""&mac_addresses=""${i}";
[15:14] <SpamapS> commenting out?!
[15:14] <SpamapS> bad form ;)
[15:15] <smoser> roaksoax, ^ i told you to get rid of that comment before uploading :)
[15:16] <roaksoax> smoser: arg... I did... but after so many links I guess I got confused an applied the wrong patch :)
[15:16] <smoser> it works though
[15:16] <smoser> so i'm good with that.
[15:17] <SpamapS> all good
[15:18] <ppetraki> I've got a maas client that simply will not budge, precise repo, HP proliant, WOL/PXE enabled eth0, use existing dhcp. http://pastebin.ubuntu.com/1273410/
[15:18] <ppetraki> I click start node and it does nothing
[15:28] <roaksoax> ppetraki: i wouldn't be relying on precise's MAAS to work properly in what WoL is concerned
[15:29] <ppetraki> roaksoax, :(, to the ppa then
[15:30] <ppetraki> roaksoax, thanks
[15:31] <roaksoax> ppetraki: welcome :)
[15:31] <roaksoax> flacoste: so today is the last day we can upload new upstream release before a SRU.. until what rev do you want to get in?
[15:31] <roaksoax> Daviey: until when can we make the upload?
[15:32] <smoser> roaksoax, so maas-enlist ? expecting in a sru somewhere?
[15:33] <roaksoax> smoser: we just need to wait for it to be available for verification
[15:33] <smoser> i see.
[15:33] <roaksoax> smoser: its already accepted
[15:33] <smoser> https://launchpad.net/ubuntu/+source/maas-enlist/0.4-0ubuntu1.3
[15:33] <smoser> right
[15:33] <roaksoax> smoser: after verification, we wait 7 days
[15:34] <flacoste> roaksoax: so I'd like smoser fix for ipmi-locate to be in
[15:34] <flacoste> i just approved it
[15:34] <flacoste> so that would be 1264
[15:34] <flacoste> rvba, allenap: anything other fixes in-flight that could be landed shortly?
[15:34] <flacoste> jelmer: how's your memory constraint fix going? can we upload that today?
[15:35] <rvba> flacoste: all the fixes we (the red squad) wanted to check in have been landed as of r1261.
[15:36] <rvba> flacoste: Not sure if jtv will have time to finish the improvements he wanted to make to the cli.
[15:36] <rvba> Help message improvements.
[15:36] <rbasak> roaksoax: verification failed. MAC address problem. I assume this was your other fix?
[15:36] <roaksoax> rbasak: yes
[15:37] <jelmer> flacoste: Yeah, I think I can propose it for review before EOD
[15:37] <rbasak> roaksoax: it stops me verifying 0.4-0ubuntu1.2 since all highbanks have two NICs
[15:37] <roaksoax> rbasak: you can just veirfy with 1 MAC address
[15:37] <rvba> flacoste: but all our (known ;)) critical bugs have been fixed.  I tested a package I made (ppa:rvb/maas) 0.1+bzr1261+dfsg-0ubuntu4~ppa1 a few hours ago.
[15:37] <roaksoax> rbasak: maas-enlist --serverurl XYZ -i eth0 etc etc
[15:37] <roaksoax> rbasak: if you specify the interface it will only use 1 MAC
[15:38] <rbasak> roaksoax: I'm verifying by actually using MAAS to enlist. And that doesn't work.
[15:38] <flacoste> jelmer: well, it would needed to be _landed_ before EOD, so should we wait for it, or not?
[15:39] <rbasak> roaksoax: landing this in -updates will cause a regression.
[15:39] <rbasak> roaksoax: thus it should be verification-failed, no?
[15:39] <roaksoax> rbasak: that's why .3 is for
[15:39] <rbasak> OK, so I'll verify .3 then
[15:39] <rbasak> When it lands in -proposed
[15:39] <roaksoax> rbasak: cool
[15:39] <jelmer> flacoste: that should be doable too; when is the upload happening?
[15:41] <flacoste> roaksoax: what's the limit?
[15:41] <roaksoax> flacoste: Daviey's EOD
[15:41] <flacoste> roaksoax: he has one?
[15:41] <roaksoax> apparently so :)
[15:43] <roaksoax> rvba: i'll make available a new package for you to test with your fixes and some minor changes :)
[15:44] <rvba> roaksoax: all right.  I'll test the connection between the region and the cluster.
[15:51] <matsubara> roaksoax, hey, did you see the package build failure because of the ipmi-enlistment patch?
[15:51] <ppetraki> roaksoax, so should I just pin quantal working from precise or pick one of the pockets from here? http://ppa.launchpad.net/maas-maintainers
[15:52] <roaksoax> matsubara: yes: https://code.launchpad.net/~andreserl/maas/packaging_updates_bzr1263/+merge/129221
[15:53] <matsubara> roaksoax, cool. thanks!
[15:53] <roaksoax> ppetraki: the latest quantal is not yet available for precise, but you should be able to test ppa:maas-maintainers/testing
[15:53] <roaksoax> with newer maas for precise
[15:53] <roaksoax> without cobbler
[15:53] <ppetraki> roaksoax, thanks
[16:07] <jelmer> rbasak: hi?
[16:08] <flacoste> roaksoax: so prepare an upload for 1264, if jelmer lands his fix before Daviey's EOD, let's include it, otherwise 1264 is a good fallback
[16:10] <roaksoax> flacoste: ack
[16:10] <roaksoax> rvba: http://paste.ubuntu.com/1273523/
[16:10] <rbasak> jelmer: ?
[16:10] <roaksoax> rvba: i'm not particularly thrilled with that fix... but it seems the only way to do it easily
[16:11] <jelmer> rbasak: I was curious if you had some example lshw -xml output from an ARM machine
[16:11] <rbasak> jelmer: yes, one moment
[16:12] <rbasak> jelmer: https://bugs.launchpad.net/maas/+bug/1064638/comments/5
[16:12] <roaksoax> rvba: i'd much rather just set the url independently
[16:13] <rvba> roaksoax: looks like a good enough fix to me.  What do you mean independently?
[16:14] <roaksoax> rvba: see how configure_maas_default_url or configure_maas_squid_deb_proxy work
[16:14] <roaksoax> rvba: I'd much rather do it that way
[16:14] <jelmer> rbasak: thanks
[16:15] <roaksoax> rvba: unless... let me see
[16:15] <rvba> I don't really see the difference tbh.
[16:15] <roaksoax> rvba: i don't like the fact to be changing the password of rabbitmq on dpkg-reconfigure maas
[16:16] <roaksoax> rvba: cause it also means restart rabbimq
[16:16] <rvba> roaksoax: I agree.  But this is not related to that ip address thing right?
[16:16] <roaksoax> rvba: it is
[16:17] <roaksoax> rvba: because if I pass the ipaddress to that function, everything we dpkg-reconfigure we will change rabbitmq password
[16:19] <smoser> roaksoax, rbasak i started a precise ephemeral image build.
[16:19] <smoser> for daily
[16:19] <smoser> and upon that being done will do one for quantal
[16:19] <rvba> roaksoax: ah right, now I see.
[16:28] <roaksoax> rvba: ok just uploaded to experimental ppa, so it should build soon
[16:28] <roaksoax> rvba: please test clean installs and uipgrades to see if it does work correctly
[16:28] <roaksoax> rvba: in both single node, and multi nod eescneario
[16:28] <roaksoax> i;ll do the same
[16:29] <rvba> ok
[16:52] <rvba> Still not built :/
[16:53] <roaksoax> rvba: yeah they are slowtoday :/
[16:53] <roaksoax> rvba: maybe someone is rebuilding a whole bunch of packages today
[16:54] <rvba> I think the package I uploaded to my ppa a few hours ago took 40 minutes to get built.
[16:54] <flacoste> roaksoax, rvba: the lp build farm sucks since the CD move
[16:54] <flacoste> we are still missing several builders
[16:55] <Daviey> roaksoax: keep it coming
[16:55] <roaksoax> ah! that's the reasong then
[16:57] <roaksoax> Daviey: will let you know as soon as I have a package ready to upload
[16:58] <Daviey> roaksoax: cool
[17:02] <rvba> roaksoax: package is currently building.
[17:02] <roaksoax> \oi/
[17:02] <roaksoax> \o/
[17:10] <rvba> roaksoax: package published.
[17:16] <rvba> roaksoax: when doing: sudo dpkg-reconfigure maas-cluster-controller, I accidentally added an extra space at the end of the url 'http://my.ip/MAAS ' and then I got: sed: -e expression #1, char 45: unterminated `s' command
[17:16] <rvba> Probably not critical.
[17:20] <roaksoax> rvba: right, so i guess we need to get rid of whitespaces
[17:21] <rvba> yep
[17:30] <roaksoax> rvba: is everything working as expected?
[17:31] <rvba> roaksoax: yes, so far.
[17:31] <roaksoax> rvba: in my local machine cluster didn't create the celery beat stuff
[17:31] <roaksoax> but might be related to something else
[17:31] <rvba> $ ls /var/lib/maas
[17:31] <rvba> celerybeat-cluster-schedule  celerybeat-region-schedule  dhcp  dhcpd-interfaces  ephemeral  media  tftp
[17:31] <rvba> Looks ok here.
[17:31] <rvba> I see the tasks being processed.
[17:32] <roaksoax> rvba: who creates celerybeat-schedule?
[17:32] <rvba> In /var/log/maas/celery.log
[17:32] <rvba> The cluster celery worker.
[17:33] <roaksoax> doesn't it create celerybeat-cluster-schedule?
[17:33] <rvba> Which is started by the sstart-cluster-controller upstart script.
[17:33] <roaksoax> rvba: so I just purged a maas installation, and this was left running:
[17:33] <roaksoax> maas       775  0.0  1.0 330912 21124 ?        S    12:47   0:00 /usr/bin/python /usr/bin/celeryd --logfile=/var/log/maas/celery.log --loglevel=INFO --beat --schedule=/var/lib/maas/celerybeat-schedule
[17:33] <rvba> roaksoax: you're right, there is no celerybeat-schedule
[17:34] <rvba> I don't have that.
[17:35] <roaksoax> rvba: that might have been from a previous package then
[17:35] <rvba> Yeah, I think so.
[17:35] <rvba> /usr/bin/python /usr/bin/celeryd --logfile=/var/log/maas/celery.log --schedule=/var/lib/maas/celerybeat-cluster-schedule --loglevel=INFO --beat -Q b0933e01-ea42-4852-9a3e-52f7912b50d8
[17:35] <rvba> That is the cluster worker.
[17:35] <rvba> /usr/bin/python /usr/bin/celeryd --logfile=/var/log/maas/celery-region.log --loglevel=INFO --beat --schedule=/var/lib/maas/celerybeat-region-schedule -Q celery,master
[17:35] <rvba> That is the region worker.
[17:35] <roaksoax> yeah
[17:56] <rvba> roaksoax: all the cluster/region connection stuff works ok.  I cannot really test upgrades.
[17:57] <roaksoax> rvba: i just did an upgrade and things seemed to work fine
[17:57] <roaksoax> rvba: but then purge maas and all its components
[17:57] <roaksoax> rvba: and on reinstlal there was an error related to dbconfig-common
[17:57] <roaksoax> rvba: cna you try that?
[17:57] <rvba> Sure.
[18:00] <rvba> Purging a cluster controller then reinstalling works.
[18:00] <rvba> I'll try with the region controller now.
[18:03] <rvba> roaksoax: I didn't get an error.  Did you answer yes to all the questions asked by the packaging when performing the 'purge'?
[18:03] <roaksoax> rvba: yeah
[18:03] <roaksoax> rvba: it is an error with dbconfig-common
[18:03] <roaksoax> apparently
[18:03] <rvba> I didn't see any.
[18:04] <rvba> Does it break the package completely?
[18:05] <roaksoax> rvba: it does
[18:05] <rvba> My region cluster seems fully functional.
[18:05] <rvba> After re-installation that is.
[18:05] <roaksoax> rvba: cool,I guess it is just probably a race somewhere
[18:07] <rvba> roaksoax: Races are rarely easy to track down.  Do you have an idea where it could come from?
[18:08]  * rvba tries the purge/reinstall cycle again.
[18:10] <roaksoax> rvba: yeah maas-region-controller.config (in packaging) which triggers something in dbconfig-common and that's where it fails
[18:11] <rvba> Really a race, like you say, because it worked again here.
[18:12] <roaksoax> rvba: indeed
[18:14] <roaksoax> rvba: it worked for me too
[18:14] <roaksoax> rvba: i think it might appear when upgrading from precise to quantal, then purge, and then install again
[18:16] <rvba> Probably not related, but when I remove the package I see:
[18:16] <rvba> userdel: user maas is currently logged in
[18:16] <rvba> /usr/sbin/deluser: `/usr/sbin/userdel maas' returned error code 8. Exiting
[18:16] <roaksoax> rvba: yeah I already fixed that
[18:16] <roaksoax> rvba: update&upgrade
[18:16] <roaksoax> and you should not longer see that
[18:18] <rvba> Cool.
[18:21] <rvba> Indeed, I don't see the error message now.
[18:29] <adam_g_> at what point in the process of starting a node does its MAAS managed hostname become resolvable?
[18:30] <roaksoax> adam_g_: in mark's lab you are not using maas DNS i believe, so you have to wait till it is installed
[18:31] <adam_g_> roaksoax: what happens after it is installed that makes the hostname magically resolve?
[18:32] <roaksoax> adam_g_: dunno TBH... i guess it triggers dchp to tell DNS? i dind't really could investigate
[18:46] <roaksoax> rvba: yeah it is dbconfig-common for sure
[19:12]  * roaksoax lunch
[19:18] <ppetraki> ok, so I think I have an bug off the testing ppa, I can't delete a maas node, the button is literally grayed out. I tried using the maas console but hit a wall: http://pastebin.ubuntu.com/1273624/
[19:19] <ppetraki> this node had a problem provisioning, it was installed and all but the credentials didn't seem to make it, I can't login. When I do juju destroy it's able to put the other node in my two node maas back to ready but it has no idea about this node
[20:49] <globin> i get this error: http://paste2.org/p/2324832 if i try to connect to the web interface right after installing. machine is running quantal. any ideas?
[21:12] <matsubara> globin, I just got a similar issue
[21:13] <matsubara> globin, is this on a pristine environment?
[21:18] <globin> yep just created new vm
[21:19] <globin> matsubara: tried it twice even
[21:19] <roaksoax> globin: what version of MAAS are you using?
[21:21] <globin> 0.1+bzr1243+dfsg-0ubuntu3
[21:21] <roaksoax> globin: weird.. can you upgrade to latest?
[21:22] <roaksoax> globin: is rabbitmq running?
[21:26] <globin> roaksoax: 1 mom please
[21:27] <globin> roaksoax: upgrading fixed
[21:27] <roaksoax> globin: cool!
[21:27] <globin> roaksoax: tyvm, thought i'd done that
[21:28] <roaksoax> :)