[00:35] thumper-gym: hey, tracking the debug-log work. Is that branch you've got proposed the final? [00:38] o/ thumper-gym === thumper-gym is now known as thumper [01:29] hi davecheney [01:31] thumper: o/ [01:31] re that that change to peergrouper [01:31] i thought it was a fix [01:31] but it just made things more confusing [01:31] heh [01:31] rog says that the peergrouper shold return things in order [01:31] but it doesn't [01:31] even fixing the mock server to make it respect order doesn't return things in order [01:31] so I don't know what to do [01:31] if order is important to the operation of the peergropuer [01:31] then there is a problem [01:32] if its not [01:32] then we can just go back to my previous commit [01:32] s/commit/proposal [01:32] the point is, both proposals can't be right or wrong [01:34] hi all :) [01:35] o/ waigani [02:02] waigani: thumper http://paste.ubuntu.com/7215168/ [02:02] getting there, slowly [02:02] davecheney: I've been out of action sorry, getting back into it now [02:07] waigani: no worries [02:08] davecheney: fyi I put up a wip at the end of last week: https://codereview.appspot.com/84360043. I'm just reading it now to remind myself where I was at. [02:18] waigani: i can't reproduce the issue [02:18] do you have the latest gccgo and gccgo-go ? [02:21] ubuntu@winton-02:~/src/launchpad.net/juju-core/provider/local$ go test . [02:21] ok launchpad.net/juju-core/provider/local 30.707s [02:27] davecheney: oops, just saw your message hmm [02:30] i can't reproduce it now either. Though when I run make check I'm hitting nil pointer panics. - something to do with mongo being checked for [02:30] waigani: can you paste waht you see ? [02:31] waigani: do you have a /usr/bin/mongod ? [02:31] i suspect you will not [02:31] davecheney: http://pastebin.ubuntu.com/7215232/ [02:31] davecheney: ah, I'll have a look - I need to run to a standup now [02:31] bb in 30min [02:54] davecheney: I do not have /usr/bin/mongod. I am confused as I thought that mongod was a dependancy of juju-local, which I do have. [02:56] davecheney: I thought the idea was to add juju-local to install-dependencies in the Makefile and check only for juju-local as mongod would be installed as a dependency of juju-local [02:57] davecheney: though I may very easily got that wrong - thumper? [02:57] I'm busy responding [02:57] please hold [03:02] waigani: there is a bug [03:02] you need to symlink /usr/lib/..../mongod to /usr/bin [03:03] davecheney: ah okay [03:04] davecheney: if this is a bug, does it make sense for me to fix it in this branch? As it now relies on mongod being installed correctly as a dep. [03:04] s/it/we [03:06] wallyworld_: is the simplestreams a universal metadata format juju specific? [03:06] universal [03:06] hmm, how did I miss that? I'll have to read up. [03:06] from the project page on lp [03:06] Simple Streams describe streams of like items in a structural fashion. [03:06] A client provides a way to sync or act on changes in a remote stream. [03:07] It's been used to described cloud images which we just happen to make use of in Juju [03:07] ah, that helps, thanks [03:08] Does the server API rely on it? [03:08] anyway, I shouldn't distract you - I'll read up :) [03:09] davecheney, waigani: there is another problem with the tests on trusty, in that they should use the juju-mongodb mongo, and not the mongodb-server mongo [03:10] thumper: okay, I'll update my branch [03:28] thumper: your comment about cloud-init. I don't understand? [03:30] waigani: hangout? [03:30] sure [03:31] give me a sec [03:31] thumper: yes, exactly [03:33] thumper: got a link? Tried calling via hangs, no luck. [03:53] * thumper goes through axw's review comments [03:53] btw I have been through the tests now, and just had the one additional comment - expect no more pedantry :) [03:53] until you reply at least [03:54] ubuntu@winton-02:~/src/launchpad.net/juju-core/provider/openstack$ go test [03:54] OK: 53 passed, 5 skipped [03:54] PASS [03:54] ok launchpad.net/juju-core/provider/openstack 59.350s [03:54] \o/ [03:54] gentlemen, we have TWO (at least) ways of creating fake tools [03:54] everyone should feel ashamed of themselves [03:56] davecheney: only two? [03:56] axw: wondering about agent vs. entity [03:56] axw: the log files are agregated by agent [03:57] for now at least [03:57] I'll change to entity [03:57] just because I think it makes more sense [03:57] we can work out what to do later when we merge the unit agents into the machine agent [03:57] thumper: i'd be fine with N, where N was the number of providers [03:57] but it's two [03:58] thumper: thanks [03:58] thumper: that'll be "interesting" :) [03:58] davecheney: I think it would be N where N is the number of developers who have needed it :-) [03:58] thumper: well played [03:58] * davecheney runs go test ./... and goes to make lunch [04:16] * davecheney throws a chair [04:17] everywhere we use version.Current [04:17] that should be a version number, not a full number/arch/series [04:19] axw: refactoring the authentication may be bigger than I want [04:19] I'll attempt it [04:19] and see how massive it is [04:19] mk [04:19] I didn't know it was a copy [04:19] was in the work I was passed [04:19] ah right [04:26] thumper: i'm looking at https://bugs.launchpad.net/juju-core/+bug/1302935 [04:26] <_mup_> Bug #1302935: trusty local provider unit agents stuck pending

[04:26] `sudo lxc-ls` returns 'juju-precise-template' [04:26] axw: which is better `maxLines value "foo" is not a valid unsigned number` or `maxLines value "foo" is not a valid unsigned number: strconv.ParseUint: parsing "foo": invalid syntax` [04:26] tvansteenburgh: yeah... [04:27] do i need to kill that somehow? [04:27] no, it makes no difference [04:27] it is filtered out before anyone cares [04:27] What write permissions should /usr/lib/juju have? [04:27] waigani: why? [04:28] 0755 at least [04:28] thumper: I see, fair enough :) [04:28] leave it then [04:28] thumper: http://pastebin.ubuntu.com/7215512/ [04:28] thumper: hrm, not the answer i was hoping for. i deleted the trusty lock file, redeployed, and same result. i'll post the new log file [04:29] it seems the dirs can be created, no problem [04:29] tvansteenburgh: how long did you wait? [04:29] tvansteenburgh: for me to create a new template, and start it, it took over 5 minutes [04:30] * tvansteenburgh turns suddenly sheepish [04:30] thumper: you're right, now it's all started up [04:30] sorry :( [04:30] tvansteenburgh: the slow startup is just the first time the template is started [04:30] tvansteenburgh: that is fine [04:30] we should communicate this more clearly [04:31] tvansteenburgh: it will be much faster now :-) [04:31] tvansteenburgh: on my todo list is a plugin command to create the template out of band [04:31] with more feedback [04:31] so you can see what is going on [04:31] cool. unfortunately i would have never figured out the lock file thing on my own [04:31] yeah, another problem with communication I think [04:32] but now i know, so thanks for that [04:32] we should be aware that people will kill it in the middle there [04:32] i guess i killed it at some point, but didn't even remember that [04:32] axw: I'm up for ideas on how we can make the destroy environment call on Ctrl-C remove the lock file if it created one [04:33] tvansteenburgh: this is good feedback though [04:33] don't feel sheepish [04:33] thumper: which lock file is that? [04:33] axw: when we create a new lxc template for a series [04:34] axw: it can take a while [04:34] we take a fslock [04:35] thumper: is there a reason we don't just use flock? then we wouldn't have this problem, because nobody would be holding the lock [04:36] axw: I'm sure there was originally, but now I'm not so sure... [04:38] hazmat: you online? [04:40] thumper: otherwise it'll get messy if you want to handle multiple users concurrently trying to add containers in different environments [04:40] which is why it's in /var/lib/juju, rather than data-dir, I presume [04:40] yeah [04:41] can't just blow it away; you'd probably want to record the PID, and check if it's alive, etc. [04:43] meh... add it to the todo list [04:43] * thumper is refactoring http auth rubbish [04:43] * davecheney consoles thumper [04:47] wip: https://codereview.appspot.com/84360043/ [04:59] axw: refactored the auth code [05:00] axw: reusing the httpHandler now [05:00] thumper: cool, thanks [05:00] just waiting on lbox now [05:01] * axw looks forward to cutting out that particular bottleneck [05:02] takes 5-10 mins for me sometimes... [05:02] geez, how to freak out people: http://www.dailydo.co.nz/garden/glowgravel495499?utm_source=Dunedin&utm_campaign=b90cc26652-Dunedin+2014-04-07&utm_medium=email&utm_term=0_4768a78a2f-b90cc26652-280945562 [05:06] thumper: ah yeah, I forgot NewTailer starts immediately. thanks, start makes that clearer [05:11] np [05:18] https://codereview.appspot.com/84860046 [05:18] if anyone has time [05:18] davecheney: lgtm === vladk|offline is now known as vladk [05:24] thumper: ta === vladk is now known as vladk|offline === vladk|offline is now known as vladk [05:27] jam: I will be available at 10:10 for hangout, sorry === vladk is now known as vladk|offline [05:56] morning all [05:57] hmm... dimitern is here... [05:57] time to leave [05:57] o/ dimitern [05:57] hey thumper === vladk|offline is now known as vladk [07:17] hey jam1 [07:18] jam1, it seems my ssh key is missing from the bot - can't login [07:26] dimitern: hmm... I'll go check [07:26] jam1, thanks [07:28] dimitern: dimitern@kubrik is on the bot, are you sure you're connecting as the "ubuntu" user? [07:28] jam1, yes [07:29] jam1, https://pastebin.canonical.com/107836/ [07:30] dimitern: that is machine-0, I wouldn't expect you to be able to connect there, just machine 1 @ 118 [07:41] mornin' all [07:41] fwereade: got time for a quick implementation discussion? [08:41] rebooting :-) [08:49] dimitern: good morning [08:50] vladk, morning [08:51] I need to get all MACs related to node via MAAS interface, I know http://paste.ubuntu.com/7216131/, but it does not return MAC addresses [08:52] vladk, I'm working on reading the node hardware list (xml output of lshw), which contains all ethernet cards and their interface names and mac addresses [08:53] dimitern, cool, it is very convenient to know device names, instead of their MAC addresses [08:54] vladk, I'll propose it a bit later then I'm done and send you a link if you like [08:54] can I expect something like 'NetworkInfo struct' before newCloudinitConfig call? [08:56] vladk, StartInstance will gather the networkining info and return it, but will populate machine config with the necessary info so newCloudinitConfig can prepare the scripts [08:59] Why not to get that information from commissioning stage. I really need network information before cloudinit [08:59] dimitern: ^ [09:00] vladk, that's exactly where we're getting the info from [09:00] vladk, it's before cloudinit [09:02] dimitern: now, I am trying to use mgz's GetNetworksList function that uses above API call, so it becomes useless [09:02] morning all [09:02] morning mgz [09:03] vladk, we need the result of GetNetworksList to get the network details - also part of []NetworkInfo returned by StartInstance [09:04] vladk, so []NetworkInfo is populated by GetNetworksList and the new call that returns NIC details for a node [09:04] mgz, morning [09:09] dimitern, mgz: I fixed GetNetworksList, it assumes that VlanTag is string but it is a number or null in JSON, so I'm going to create a merge request [09:10] vladk, great, thanks! [09:10] do I need to create a bug in LP or card in kanban? [09:11] dimitern: ^ [09:12] vladk: a card wouldn't hurt [09:12] rogpeppe: I made it back, care to join the hangout? [09:12] morning mgz [09:13] jam1: standup hangout? [09:13] rogpeppe: our 1:1 hangout [09:13] jam1: ha! 1-1! [09:13] jam1: joining [09:14] vladk, card is fine [09:14] vladk, you could add a bug as well, but in this case i don't think it's worth it [09:32] rogpeppe, would you take a look at https://codereview.appspot.com/84430044/ please? I don't quite see what's going on there [09:49] fwereade, quick question - what should the behaviour of upgrade-juju look like with a 1.18.0 client against a 1.17.7 environment? [09:57] fwereade, also noted this potential upgrade problem - bug 1303697 [09:57] <_mup_> Bug #1303697: peer relation disappears during upgrade of juju [09:58] jamespage, the env should be upgraded to 1.18.0, does that not happen? [09:58] fwereade, without --version 1.18.0 it did not [09:59] jamespage, I *thought* we were upgrading from dev to stable without explicit args, but I may have missed something -- dimitern, can you comment? [09:59] jamespage, and that bug is weird, I will peer closer [09:59] fwereade, thanks [10:00] fwereade, the original logic was to try to upgrade to the next stable without --version [10:00] fwereade, (or current) [10:01] jamespage, did you use --upload-tools ? [10:01] dimitern, I thought so too, do you recall anyone working on that subsequently? [10:01] jam: 1:1! [10:02] dimitern, nope [10:02] fwereade, not that i know of [10:02] dimitern, I saw the same with both maas (using streams.canonical.com) and openstack (using a local mirror of streams.canonical.com) [10:02] yep, be there in 1sec [10:02] mgz: ^^ [10:02] cool [10:04] jamespage, did you try --debug to see what's going on? [10:04] dimitern, I did not [10:04] dimitern, I did hop onto the bootstrap node and looked at the log - messages about it picking 1.17.7 [10:04] jamespage, hmm... weird [10:05] fwereade: sorry, was in a call with john. looking. [10:10] jamespage, I don't suppose you have the unit agent log from lp:1303697 ? [10:10] fwereade, lemme check [10:11] c'mon internet [10:15] fwereade: i'm not sure i see what's going on either [10:15] fwereade: in the comment i made in the previous review, i was talking about address order *within* a machine, not the order of machinesw [10:19] bug 1303697 [10:19] <_mup_> Bug #1303697: peer relation disappears during upgrade of juju [10:24] jamespage, ok, I can see why the juju run could have failed [10:25] jamespage, in fact I suspect an ill-timed one could have panicked the unit agent, hence the second config-changed [10:25] jamespage, root cause remains unclear [10:25] fwereade, the call in the charm pretty much does the same thing in the config-changed hook [10:28] jamespage, yeah, but it turns out we start listening for juju run commands while the uniter is less than half initialized [10:28] jamespage, but I can't see that code path that would allow a config-changed hook to berun in those circumstances [10:30] good morning everyone [10:30] fwereade, it looks like it runs multiple times [10:30] I had to resolve it twice [10:31] fwereade, and I'm def not changing the config :-) [10:32] jamespage, config-changed runs every time the unit agent restarts [10:32] hey perrito666 [10:32] fwereade, ah [10:33] jamespage, and that juju-run could easily have bounced the agent, because the code is decidedly unsafe [10:33] fwereade, right - I see [10:34] fwereade, I'm less worried about the juju run failing and more worried that whenever I upgrade juju, keystone will hook error as its can't access the peer relation :-) [10:34] jamespage, indeed, that's why I'm stillpoking around [10:34] fwereade, OK - I'll get out of your hair then :-) [10:35] jamespage, I'd still love to see the agent log if you have it :) [10:35] fyi I've tested on openstack and local for trusty; just validating against the lastest maas [10:35] fwereade, I'll repro that for you now [10:39] * jamespage downgrades and pushed the car backup the hill [10:41] jamespage, I doubt that behaviour would vary by provider [10:41] ok [10:41] jamespage, I consider it a big deal anyway ;) [10:41] fwereade, I'm reproducing on openstack as that is quicker [10:42] jamespage, cool [10:45] fwereade, rogpeppe, mgz, jam, wallyworld, standup [10:48] rogpeppe: standup? [11:00] fwereade, unit log - http://paste.ubuntu.com/7216528/ [11:00] fwereade, machine log - http://paste.ubuntu.com/7216527/ [11:00] fwereade, and one new issue - http://paste.ubuntu.com/7216530/ [11:04] fwereade, bug 1303735 [11:04] <_mup_> Bug #1303735: private-address change to internal bridge post juju-upgrade [11:04] jamespage, thanks [11:04] fwereade, sorry :-) [11:05] jamespage, nothing to be sorry for, except on our side ;p [11:05] fwereade, I could not say whether the address change thing is a new problem [11:05] I don't spend that mich time upgrading juju environments [11:06] Plugging my iphone in appears to have locked up my computer [11:07] all except audio! [11:07] So I'll have to reboot and rejoin the hangout [11:07] sorry [11:56] fwereade, jam: the client doesn't necessarily need to read from the master === rogpeppe is now known as rogpeppe1 === rogpeppe1 is now known as rogpeppe [11:57] rogpeppe: well, we always do Strong consistency, right? [11:57] fwereade, jam: i'm giving up. connection too crappy. [12:01] hey guys, wondering if this is something i should file a bug on: http://pastebin.ubuntu.com/7216718/ [12:18] wallyworld, am now [12:18] er.. ping [12:20] re azure, does this mean the provisioning interface now knows the workload prior to machine allocation or is just fed constraints/parameters ? [12:21] could somebody please have a look here: https://codereview.appspot.com/84470053 [12:33] smoser, i just got hit by bug 1303617 [12:33] <_mup_> Bug #1303617: Latest curtin version prevents Juju from bootstrapping on MAAS [12:47] smoser, and since version 0.1.0~bzr121-0ubuntu1 is no longer in trusty I cannot downgrade to work around it [13:09] natefinch: you've got a LGTM on https://codereview.appspot.com/81980043/ [13:11] I found a reason: gomaasapi gives a panic in testing mode when GET /MAAS/api/1.0/networks/?node=xxx and no assigned network to the node [13:11] vladk, yeah that's the issue, but I don't see panics when running maas tests otherwise [13:12] fwereade, mgz, vladk, perrito666, get a list of NICs with mac addresses in maas https://codereview.appspot.com/84850045 [13:13] dimitern: because you don't call GetNetworksList before environ.startNode() [13:14] vladk, startNode shouldn't panic - if there are no networks requested for the machine it shouldn't try to get them (or if it fails it should ignore it) [13:14] rogpeppe: thanks [13:17] dimitern: tx [13:18] dimitern: it panics only in testing mode (testservice.go networksHandler() function) [13:19] vladk, take a look at TestGetNetworksList - it adds a network and then a connects a node to it [13:20] dimitern: from GetNetworksList I have (net_name, addr, mask, vlan, descr), from getInstanceNetworkInterfaces I have (mac_addr, iface_name) [13:21] dimitern: I need to extract mapping between net_name and mac_addr via MAAS API [13:22] vladk, the macs will be there if the node is connected to the network [13:23] vladk, and you can get the networks with the macs [13:23] vladk, i.e. GetNetworksList needs to return mac addresses (when set) [13:24] dimitern: what I really need i multimap(iface_name->vlan) [13:24] dimitern: GetNetworksList doesn't read mac addresses now [13:25] dimitern: I may change it, do you know MAAS API to read MAC addresses? [13:27] dimitern: as to TestGetNetworksList, this will work if I manually add a network to each instance before bootstrap.Bootstrap() or testing.AssertStartInstance() [13:27] I think that panic should be changed to error code in gomaasapi [13:28] vladk, so I can get the macs from maas maas-root network list-connected-macs vlan0 [13:29] dimitern: this will be very long way [13:31] vladk, with is equivalent to doing /api/1.0/networks/?op=list-connected-macs [13:31] vladk, s/with/which/ ...(I hope) [13:31] list_connected_macs actually [13:32] vladk, it seems MAAS does not allow us to get a list of all networks along with their mac assignments - just for a single network, so we'll need multiple API calls for each network [13:33] dimitern: long way: from GetNetworksList I get []networkNames, than for each networkName I read a listConnectMacs and search for my node in that list [13:34] dimitern: we definitely need a separate API call for all this stuff [13:36] vladk, we could have a separate api, but we don't have it now, so we can work around it with multiple calls [13:36] * dimitern needs to step out for a while [13:44] hazmat: hi, just wanted a clarification. you said in an email that you wanted upload-tools to use the jujud binary and you had hardwired it to do so. AFAICT juju actually looks for the jujud executable in the current path and uses that if found. so could you clarify where you see the problem with the current behaviour? Have I missed something? [13:46] wallyworld, sorry that wasn't clear.. i mean mentally i've hardwired myself to use upload-tools [13:46] wallyworld, indeed the tool lookup by path works fine [13:46] ah, ok :-) thanks [13:47] pinned bootstrap using exact version match should help with that for public clouds [13:53] dimitern: I take a task of changing GetNetworksList to myself [14:18] rogpeppe: what work did you do on the HA branch after I left on Friday? I forget exactly where we left it. [14:19] * rogpeppe tries to remember [14:21] natefinch: i did this: https://codereview.appspot.com/84540044/ [14:21] natefinch: (and possibly some more as well; i forget) [14:22] natefinch: over the weekend, i also tried to integrate the branches to see what might actually work [14:23] natefinch: there are a few things that we will need to do [14:23] natefinch: we need to apt-get install mongo inside EnsureMongoServer [14:23] natefinch: we need EnsureMongoServer to write the server secret files [14:23] natefinch: we need to add SystemIdentity to StateServingInfo [14:24] rogpeppe: that seems like stuff that can be in separate CLs, right? Not needed for the single-server mode we're using right now [14:24] natefinch: the state server Initiate needs to add the machine tag to the replica set entry [14:24] natefinch: yeah, although it does make EnsureMongoServer make sense. [14:24] rogpeppe: doesn't cloud-init know the intended jobs? so it can do apt-get as usual? [14:25] rogpeppe: I really want to get the MA-HA branch into trunk ASAP so we don't have to keep maintaining a huge branch [14:25] natefinch: +1 [14:25] rogpeppe: I honestly don't care if it makes sense today, as long as it doesn't break anything today [14:25] axw: i would much prefer to isolate all the mongo installation into EnsureMongoServer [14:25] axw: which means that cloud-init doesn't need to know anything at all about mongo [14:26] rogpeppe: fair enough [14:26] although, there may be trickiness around adding apt sources [14:26] axw: the apt sources *can* be added by cloud-init [14:27] axw: because they're needed for lxc too [14:27] yeah okay [14:28] rogpeppe: you had a change to make the replicaset member host separate from the dial info... is that in your version of the MA-HA branch? [14:29] axw: BTW juju ssh is broken currently if $SHELL isn't sh-compatible [14:29] axw: the fix is trivial, but i haven't got around to it yet, sorry [14:29] oy vey [14:29] what's the issue? [14:30] or you didn't get to the bottom of it? [14:30] axw: ssh proxying uses $SHELL to execute the proxy command [14:30] natefinch: did you merge my version of the MA-HA branch? [14:31] rogpeppe: doing so no [14:31] now [14:31] rogpeppe: as in, ProxyCommand is executed using $SHELL? [14:31] natefinch: yes, my version of the branch adds a MemberHostPort member to InitiateMongoParams [14:31] axw: yes [14:32] rogpeppe: is that to separate dialing address from the member address? [14:32] I was just about to ask about that [14:32] axw: yes [14:32] cool [14:33] axw: because you need to dial localhost otherwise you don't get access to mongo [14:33] yep [14:34] I spent most of Friday learning a bunch about the quirks around replica sets, localhost exception, etc. [14:34] axw: yeah, mongo replicaset stuff is like 100% quirks [14:34] hehe :) [14:37] * natefinch wonders if he's the only one who types bzr bootstrap --debug and juju install ./... [14:38] :) [14:38] I need one command that picks juju, go, or bzr based on the command and does the right thing [14:38] switch being the only tricky one between juju and bzr [14:39] dimitern, fix is uploaded. [14:40] dimitern, https://launchpad.net/ubuntu/+source/curtin/0.1.0~bzr125-0ubuntu1/+build/5886362 [14:40] and really sorry/embarrased on that. [14:42] dimitern, you just need to upgrade curtin on your maas system. the actual fix is in curtin-common. [14:44] any ppc enablers around? [14:45] there's some client agent failures in the logs ref'd in https://bugs.launchpad.net/juju-core/+bug/1303787 here that need attention [14:45] <_mup_> Bug #1303787: hook failures - nil pointer dereference

[14:47] * hazmat falls back to email [14:48] fwereade: is there any written spec on how a placement directive can look like? I see a set of tests that assert that varios values passed to machineornewcontainer yield true or false, but I am not sure how to interpret all of them [14:49] that google doc is all we have [14:49] the only ones implemented are lxc/kvm [14:49] and they all take : [14:49] perrito666, ^^ [14:50]