/srv/irclogs.ubuntu.com/2016/01/05/#juju-dev.txt

=== jillr_ is now known as jillr
=== gberginc_ is now known as gberginc
natefinchwaigani: I keep seeing issues about "lingo ls" and misread it as "lingo is" ... and expect it to have the same output as "juju is" :)03:25
waiganinatefinch: hehe03:32
mupBug #1531064 opened: lxd: cannot create multiple environments <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1531064>05:41
mupBug #1531064 changed: lxd: cannot create multiple environments <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1531064>05:44
mupBug #1531064 opened: lxd: cannot create multiple environments <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1531064>05:50
frobwarecherylj, I tried adding a container with my patch on 1.8.2 and it worked - worked in that /etc/resolv.conf was populated. Do you not see this on on 1.8.3? (Will install 1.8.3 in some spare time)08:57
voidspacedimitern: thanks!10:23
dooferladdimitern: which branches are you wanting reviews on?10:59
dooferladdimitern: the ones that are over two weeks old?11:00
dimiterndooferlad, nope, just the last one11:00
dooferladdimitern: on it11:00
dimiterndooferlad, the other two will stay like that while I tease bits of them and propose them separately11:00
dimiterndooferlad, ta!11:01
dooferladdimitern: you have a review11:09
dimiterndooferlad, awesome! thanks11:16
frobwaredimitern, take a look at https://bugs.launchpad.net/juju-core/+bug/1528217/comments/1212:04
mupBug #1528217: 1.25.2 doesn't set up DNS information with MAAS <blocker> <ci> <maas> <network> <regression> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1528217>12:04
frobwarecherylj, mgz: ^^12:04
mgz frobware: looking12:06
mgzfrobware: yeah, I get the same on 1.8.3, double space where ip should be and all12:08
frobwaremgz, so might just be a regression in 1.8.312:09
frobwaremgz, debugging some more.12:09
mgzI hae a maas checkout somewhere...12:09
mgzthere's not much in 1.8.312:15
frobwaremgz, as in not much different?12:27
mgzindeed12:27
mgzhttp://paste.ubuntu.com/1441024712:30
dimiternfrobware, looking12:52
dimiternfrobware, hmm.. that means the ConfigType: "dhcp" wasn't taking effect?12:53
dimiternfrobware, is that the result of changing static to dhcp in the InterfaceInfo ?12:58
frobwaredimitern, no. That's my same change tested against .2 and .3.13:14
frobwaredimitern, so 1.8.3, as reported, is behaving differently.13:15
dimiternfrobware, hmm - yeah, but the lack of address in /e/n/i is troubling - got the logs to compare what happened?13:16
frobwaredimitern, host and container is still up. Which would help?13:17
dimiternfrobware, the machine log of the host - looking for things like "created device ..."13:18
dimiternfrobware, both from 1.8.2 and 1.8.313:18
dimiternfrobware, logged the result of claim-sticky-ip-address should follow shortly after creating the device13:19
dimiterns/logged the/M-t/13:19
frobwaredimitern, attached to https://bugs.launchpad.net/juju-core/+bug/152821713:23
mupBug #1528217: 1.25.2 doesn't set up DNS information with MAAS <blocker> <ci> <maas> <network> <regression> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1528217>13:23
dimiternfrobware, looking13:23
frobwaredimitern, not sure the 1.8.2 log has sticky in it. hmm.13:24
dimiternfrobware, ah, sorry - those logs are not enough - machine-0 logs for both are actually better13:25
dimiternfrobware, as the device registration etc. is handled by the environ provisioner (running on the api server(s) only)13:25
frobwaredimitern, let's HO13:25
frobwaredimitern, we can do stuff live13:25
dimiternfrobware, ok, omw to standup HO13:26
mattywfwereade, ping?14:04
fwereademattyw, pong14:07
frobwaremgz, cherylj: can you take a look on your 1.8.3 installations to see if the discovered interfaces (maas-eth0) has a default gateway. I'm guessing not. Explanation in: https://bugs.launchpad.net/juju-core/+bug/1528217/comments/1514:35
mupBug #1528217: 1.25.2 doesn't set up DNS information with MAAS <blocker> <ci> <maas> <network> <regression> <juju-core:Incomplete> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1528217>14:35
mgzfrobware: was reading the maas provide code yesterday, there's a branch that checks for default gateway and, ah I see doesn't actually do anything if it's empty14:40
mgzdifference between our 18 and 19 is somewhat interesting:14:52
mgzhttp://paste.ubuntu.com/1441101914:53
mgzhttp://paste.ubuntu.com/1441102814:53
frobwaremgz, and the last pastebin was 1.9?15:03
mgzfrobware: yup15:04
frobwaremgz, as you say, interesting!15:04
frobwaredooferlad, voidspace, dimitern: http://reviews.vapour.ws/r/3451/ - PTAL now that we know why it fails on 1.8.3.15:15
voidspacefrobware: will look shortly, just picking up girl from school15:16
voidspacebrb15:16
cheryljfrobware: yes, I see that there is no default gateway for maas-eth015:17
frobwaredimitern, can we really just s/network.ConfigStatic/network.ConfigDHCP/ in the provisioner? Apart from fixing up the failing tests is there anything else to be wary of with this change?15:20
dimiternfrobware, looking at that PR15:22
dimiternfrobware, I believe it's safe to do that, but only when AddressAllocationEnabled() is false15:22
frobwaredimitern, right15:22
dimiternfrobware, well.. and pending a subsequent live tests :)15:22
frobwaredimitern, makes the tests interesting then15:23
frobwaredimitern, My sample size of 1 indicates that 1 container comes with as expected.15:23
frobwares/with/up15:23
dimiternfrobware, you've got a review btw15:26
dimiternfrobware, that's good :) - was that on 1.8.2 without any manual setting of gateways ?15:26
frobwaredimitern, it had a default gateway. I can try without now. Thanks for the review.15:27
dimiternfrobware, np - well, without GW we've seen it breaks, right?15:27
frobwaredimitern, fatally.15:28
BrunoRmarcoceppi: is there something new regarding http://review.juju.solutions/review/2398 ?15:29
dimiternfrobware, yeah.. I think at this point we only need to confirm if 1.8.3 vs 1.8.2 is broken "by default" when installed from scratch, and if it is, that settles it (and we have a simple workaround when it breaks - add GW)15:30
dooferladdimitern/voidspace: having checked out github.com/juju/charm and switched to the unsable-v6 branch I tried to run the tests and get a bunch of errors. I have installed the dependencies. Thoughts?15:30
dimiterndooferlad, tests in juju/charm or in juju/juju ?15:30
dooferladin juju/charm15:31
dimiterndooferlad, paste the errors?15:31
dooferladdimitern: http://pastebin.ubuntu.com/15:32
mgzdooferlad: slight problem...15:32
dooferladmgz: ?15:33
dimiternno ID :)15:33
dooferladhttp://pastebin.ubuntu.com/14411294/15:33
dooferladdoh!15:33
dooferladI think it may be that charm_test.go helpfully imports gopkg.in/juju/charm.v6-unstable15:34
dooferladinstead of, well, "charm"15:34
dimiterndooferlad, double-check the branch juju/charm is on is v6-unstable (commit ID 0229ee115:35
dooferladdimitern: I think it is all about the imports. I checked it out into github.com/juju/charm, not gopkg.in/juju/charm.v6-unstable15:36
dimiterndooferlad, hmm.. well gopkg.in/juju/charm.v6-unstable is backed by github.com/juju/charm @v6-unstable15:37
dooferladdimitern: yep, I just find it strange to hack in a gopkg.in directory.15:37
dimiterndooferlad, yeah, quirks of gopath15:40
dooferladdimitern: also a quirk of using package thing_test instead of package thing -- you have to import the thing you are testing instead of just being in it.15:41
cheryljfrobware: could this lack of GW somehow also lead to the leaking of IPs?15:41
cheryljor maybe other things are weird with 1.8.315:42
dimiterndooferlad, right; I've confirmed locally that even though I have e.g. github.com/juju/charm and gopkg.in/juju/charm.v6-unstable, the former's tip is an year older than the latter's tip15:43
dimiternI guess that's when we switched to gopkg.in import paths15:44
voidspacefrobware: I can't look at the diff in reviewboard as it seems to conflict15:45
voidspacewill try github15:45
cheryljfrobware: can you ping me when you get a minute?16:09
frobwarecherylj, ping16:09
frobwarecherylj, have 20 mins before the OS call16:09
natefinchcherylj: you can ignore my request to edit that bug squad list page, I just had wanted to mark my bug as fixed, but I should have known you'd be on top of it. :)16:10
cheryljfrobware: https://plus.google.com/hangouts/_/canonical.com/frobware-cherylj?authuser=016:10
dimiternfrobware, dooferlad, voidspace, OS call?16:31
voidspacedimitern: omw16:33
voidspacedimitern: frobware: dooferlad: if you get a chance please http://reviews.vapour.ws/r/3457/16:34
katconatefinch: ericsnow: forgot to mention, i have a doc. appt tomorrow morning overlapping with the standup. bumped it up a bit16:44
ericsnowkatco: oh yeah, and I have a dentist appt. today :)16:44
katcoericsnow: calendar please :)16:45
ericsnowkatco: added it yesterday :)16:45
katcoericsnow: cool ty16:45
natefinchkatco: I got nothing all week :)16:47
katconatefinch: yay :D16:47
marcoceppiBrunoR: not at the moment, there are some oddities I'm looking into16:48
marcoceppiBrunoR: you may want to join #juju to continue discussion16:49
dimiternvoidspace, looking16:56
voidspacedimitern: thanks16:57
voidspacedimitern: they're intended to be complete tests, so it might be worth reviewing apiserver/common/networkingcommon/spaces.go along with it16:58
voidspaceit wasn't really reviewed the first time round16:58
voidspacethat's *mostly* moved code though rather than new code16:59
dimiternvoidspace, reviewed17:01
voidspacedimitern: thanks17:02
voidspacedimitern: cool17:03
voidspacegood idea on the line wrapping, they're annoying17:03
dimiternvoidspace, yeah, cheers17:03
cheryljhey ericsnow, I updated bug 1521777 with the behavior needed, and a link to the changes I had done so far17:31
mupBug #1521777: Allow for upgrades to 2.0 <juju-core:In Progress by cherylj> <juju-core 1.25:Fix Committed by cherylj> <https://launchpad.net/bugs/1521777>17:31
ericsnowcherylj: thanks17:31
cheryljcouldn't update the card, I got a permission error17:31
katconatefinch: reviewed your pr17:44
frobwarecherylj, dimitern: I did a clean install of MAAS 1.8.2 and I get not default gateway for the discovered maas-eth0 interface, even when updating the Cluster controllers managed network.17:59
frobwares/and I get/and I do NOT get/18:00
dimiternfrobware, hmm this means both 1.8.2 and 1.8.3 are consistently broken then18:05
frobwaredimitern, I would have to try 1.9 to see if it is the same.18:08
dimitern.. unless the default GW is manually set18:08
dimiternfrobware, +118:08
frobwaredimitern, we have not noticed because containers were using dhcp.18:08
mgzprettty sure 1.9 is set up with the bits as before18:08
frobwaremgz, question is was the default gateway set manually?18:09
mgzperrito666: pr #3408 does not fill me with joy18:09
mgzboth requiring provider api calls in an upgrade step and putting uuids in a string field seem horrible18:10
perrito666that is the number of github or rb?18:10
mgzperrito666: my bad, review ^, pr #398618:11
perrito666oh, good because 3408 is quite nice and you where hurting my feelings18:11
perrito666mgz: well 2 things18:11
perrito6661) google does not provide any other way, because of reasones18:12
perrito6662) api calls in upgrade steps are about to become a thing, since we suddenly need to change stuff upstream to support big changes like multienv same name18:12
mgzperrito666: we don't really expect upgrade steps to fail18:14
mgzthis absolutely can18:14
perrito666that said, I accept any form of criticism and will try to deal with it :)18:14
perrito666mm, we already have steps like these for storage, definitely something we will have to deal with18:17
perrito666ftr I think that not expecting anything to fail is wishful thinking on our part18:17
mgzmeh, and I haven't got good answers on the naming that aren't major infra changes18:18
mgzperrito666: but line 65 in provider/openstack/upgrade.go is certainly a problem18:19
mgzyou're not even logging/propogating the nova error18:20
perrito666mgz: what pr is this now?18:21
mgzthe openstack sec group change18:22
perrito666mgz: line 65 is return errors.Annotatef(err, "upgrading security group name from %q to %q", group.Name, newName)18:24
mgzright18:25
perrito666aaand that err is the one that comes from calling nova?18:25
mgzprovider returns an error, the step fails part way through upgrade and we lose the provider error18:26
perrito666that error bubles up all the way to the upgrade step, I feel like I am missing something here18:27
mgzsorry, we don't lose the error, but we don't complete the step18:28
perrito666well, I assumed that if the step failed and given that the identity of the upgradestep function allowed for an error that was the way to proceed: stop and err out18:31
mgzprobably, but we don't retry steps till a new version so life seems complicated18:33
perrito666mmm, perhaps I should add some redundancy to my own step in that case18:35
mgzwell, I'm not sure, I don't think individual steps should18:35
mgzbut the framework above and below probably should18:35
perrito666you seem to be luring me into adding redundancy to upgrade steps18:36
mgzno, but I'm pointing out an issue with it given how things work at present18:36
perrito666sad, you could easily play the part of a british elf luring inocent devs into complex features :p18:37
frobwarecherylj, switch the provisioner to use DHCP again on 1.25 works based on some very limited testing. and it works without having to worry about having a default gateway. We need to decide how we should proceed from here.18:54
frobwarecherylj, (and some of that was almost a real sentence.) LOL18:55
frobwarecherylj, I can continue with the move back to DHCP and fix the unit test failures but as we've seen most of the failures have come from live usage.18:56
frobwarecherylj, I'm trying to avoid fixing the fix ... which is fixing another fix, and so on.19:00
cheryljfrobware: I'm chatting with the maas guys now about the IPs leaking for devices... I can ask them about the default behavior with 1.8.2 and the default gateway19:15
cheryljfrobware: If it's the case that 1.8.2 has the same behavior that by default, there's no default gateway, then I think we need to use the DHCP again.19:16
cheryljWe can't make users specify a gateway to be able to use juju after an upgrade.19:16
frobwarecherylj, playing devils ... OK19:16
cheryljdoes that sound reasonable?19:17
frobwarecherylj, as an end-user, sure. :)19:17
cheryljfrobware: haha, yeah :)19:17
frobwarecherylj, I think we should consider landing my change anyway.19:18
frobwarecherylj, and on top of that we switch back to DHCP. Thoughts?19:18
cheryljfrobware: yeah, that was my assumption19:18
perrito666anyone very savvy on hooks willing to spare a few moments with me?19:19
frobwarecherylj, I do get a default gw with 1.919:34
frobwarecherylj, that was noted on the back of a clean fresh 1.9 install19:35
cheryljfrobware: ok, thanks for checking19:35
cheryljfrobware: have you had the chance to try 1.8.2?19:35
frobwarecherylj, yes. 1.8.2 no default GW. ditto for 1.8.3.19:36
frobwarecherylj, my 1.9 was for completeness.19:36
cheryljah, ok19:36
frobwarecherylj, I vote for 1.9. :)19:36
cheryljhas 1.9 even been released yet?19:36
frobwarecherylj, I *think* so. I now have: 1.9.0+bzr4533-0ubuntu119:36
cheryljah, ok19:37
frobwarecherylj, I previously had MAAS Version 1.9.0 (rc4+bzr4533)19:37
frobwarecherylj, I need to EOD. Tomorrow I'll switch to DHCP, address the unit tests and perhaps we create another feature branch for testing.19:38
cheryljfrobware: ok.  And just fyi - regarding the IP addresses leaking:19:38
cheryljmpontillo> cherylj: ah, in 1.8.3 it might be different - I was thinking 1.9. in 1.9 we changed the internals of how we associate IPs with devices, so it may work better19:38
frobwarecherylj, do we know if it leaks with 1.8.2 as well?19:39
cheryljnot yet19:39
cheryljthe qa team hasn't had the chance to downgrade the MAAS19:39
cheryljfrobware: if you switch to DHCP, would that eliminate using devices to register container IPs?19:40
frobwarecherylj, need to double-check with dimiter19:40
cheryljfrobware: ok, because if the device IPs are not released when the devices are deleted, we may need to back out that change.19:42
frobwarecherylj, so the change works for 1.9, doesn't this then become a 1.8 issue?19:42
frobwarecherylj, I'm now confused. Back out which change?  Because we want IPs to get cleaned up.19:46
cheryljfrobware: technically, yes,  but we can't ship something that breaks people running on 1.819:46
cheryljfrobware: the change that registers containers as devices.19:46
cheryljfrobware: if you need to EOD, we can continue this conversation tomorrow19:46
* frobware maƱana 19:48
cheryljjog, mgz, do you know if the 1.8.3 maas you guys are using has a static IP range defined?19:50
jogcherylj, see #maas... yes19:50
katconatefinch: did you see my review?19:52
natefinchkatco: yes, was just about to ask you about it19:53
natefinchkatco: are side comments a problem?  I actually find them to be easier to read because they don't get in the way of the code19:54
katconatefinch: not a problem, i just don't think they conform to go's standard practices19:54
katconatefinch: i don't think the tooling picks them up, only top comments19:54
katconatefinch: also the comments don't conform to the standard anyway (lead with variable name)19:55
natefinchkatco: when you have multiple comments in a block like that, they get printed out enmass: https://godoc.org/github.com/natefinch/godocgo/sub#pkg-constants19:55
katconatefinch: if this weren't go, i'd say side-comments were the way to go here19:55
natefinchkatco: also, pretty sure I've seen side comments on std lib constants in similar situations.... couldn't point you at it currently.19:56
natefinchkatco: finally, the point of starting with the name of the thing is so grep will pick up the comment with what it's commenting, which will still work here, since it's on the same line19:56
katconatefinch: the two official sources i can find say to do doc comments: https://golang.org/doc/effective_go.html#commentary http://blog.golang.org/godoc-documenting-go-code20:00
natefinchkatco: hmm, can't find an example of constants with side comments in the stdlib, only struct fields.    meh.  I can do top comments, I'm not married to it20:00
katconatefinch: fwiw i agree with what you're saying; but consistency > preference20:01
natefinchkatco: yep, that's fair.20:01
katconatefinch: haven't looked; have you reviewed all of ericsnow's patches? i'm moving onto his now20:03
ericsnowkatco: sorry (forgot), I've landed the first one and the second is merging20:04
ericsnowkatco: working on the third now20:04
katcoericsnow: oh, did they already have ship-its?20:04
ericsnowkatco: yep20:04
katcoericsnow: oh cool, so no review needed?20:04
ericsnowkatco: perhaps on the remaining one of mine?20:04
katcoericsnow: sure thing20:04
katcoericsnow: listresources endpoint?20:05
natefinchkatco: yes, I reviewed all his and gave them shipits pending addressing of review comments20:08
katconatefinch: cool, gj, ty20:08
ericsnowkatco: yep20:10
ericsnowkatco, natefinch: I also found an intermittent bug in the feature branch and now have a patch up:  http://reviews.vapour.ws/r/3458/20:12
katcoericsnow: awesome! good find :)20:12
ericsnowkatco: it was annoying me :)20:13
katcoericsnow: natefinch: do you have time to hop into moonstone? i have a resources question20:17
ericsnowkatco: sure20:17
natefinchkatco: yep, one sec20:19
natefinchericsnow, katco:  Dang, should have looked a little further down.  I still don't like using stub's CheckCallNames because it tests the order of the calls, which is an implementation detail that could change, causing the test to fail erroneously.  But that's more of a comment about how the stub itself works, not this change.20:31
ericsnownatefinch: checking the order of calls is exactly the point though20:32
ericsnownatefinch: we want to ensure that the internal boundary is used in the right order20:33
natefinchericsnow: but it's not. it's testing like a dozen things that aren't actually required for the function to work correctly.20:33
natefinchericsnow: I've had to change tests that shouldn't have failed because the order of function calls changed without the actual logic changing.20:34
ericsnownatefinch: I'm not clear on how the order of calls could change without the logic changing20:35
natefinchericsnow: I forget the exact example, but it was basically two function calls that could have been done in any order, and switching them caused a test to fail.20:36
natefinchericsnow: but in this case, we don't have a requirement that the files be uploaded in the same order as they are given on the command line.... except now there's a requirement enshrined in the test that says it must be so... so if we decide we want to upload the smallest one first, or upload them alphabetically or concurrently.. this test has to change.20:37
ericsnownatefinch: correct20:37
katcoericsnow: natefinch: the function calls would have to be associative (e.g. multiplication or something)20:38
ericsnownatefinch: it will reflect the interaction with the dependencies20:38
natefinchericsnow: but it's not a requirement, and you've made it one. The only requirement is that tthe results of openresource get passed to upload20:40
ericsnownatefinch: the point of the patch is that the order *does* matter20:42
natefinchericsnow: also, it's now a requirement that we create the client before opening the first resource... why?20:42
natefinchericsnow: it makes the test care too much about very specific details of the order of calls in the implementation that it shouldn't care about.  It makes the tests fragile.20:42
ericsnownatefinch: testing the behavior of a function relative to its internal interface (dependencies) is just as important as relative to its external interface20:44
ericsnownatefinch: is your concern strictly with testing the order of interaction with dependencies?20:45
=== ericsnow is now known as ericsnow_afk
ericsnow_afkdentist20:46
natefinchericsnow_afk: well, yes.  We should be checking that what is returned from OpenResource is sent to upload for the respective resource... and if somehow the code can go back in time and do the upload first, well, good for it ;)20:48
natefinchI wish I could go back and make my reviews into a "fix it, then ship it!"21:17
natefinchkatco: btw, thanks for the review on my stuff21:18
wallyworldmenn0: hi, welcome back. quick one - this bug has been fixed in recent jujus i think, right? ie we clean up orphaned txns bug 152826121:25
mupBug #1528261: jujud won't start, log shows a lot of "cannot set addresses of machine XXX: cannot find transaction ObjectIdHex("xxxxxx"); <juju-core:Incomplete> <juju-core 1.20:Triaged> <https://launchpad.net/bugs/1528261>21:25
menn0wallyworld: hi! looking21:26
menn0wallyworld: there's a few things with that21:27
menn0wallyworld: all the unnecessary addressupdater txns have been fixed21:27
menn0wallyworld: and we do clean up the txn collection21:28
wallyworldmenn0: the issue arrose when mongo got its replicaset all screwed up21:28
menn0wallyworld: but the pancis in the ticket aren't fixed automatically in newer juju verisons21:28
wallyworldok, at least we're part way there21:28
menn0wallyworld: someone will need to shut down the state servers, get the replicaset working again and then run mgopurge to fix the txns21:29
wallyworldok, ty21:29
wallyworldniedbalski: ^^^^^^^21:30
wallyworldi'm not sure of mario's nic to tell him also21:30
niedbalskiMmike, ^^21:30
Mmikethxn!21:31
wallyworldMmike: niedbalski: so an upgrade to 1.24 or 1.25 is required plus the mgopurge21:31
Mmikehow do I upgrade when I can't get jujud started?21:32
wallyworldi think running mgopurge first will help, well hopefully?21:33
wallyworldso shut down state servers, fix replicaset, run mgopurge21:33
menn0Mmike, niedbalski, wallyworld: the upgrade to 1.24 isn't a strict requirement to get this env working again. It'll just help reduce the chance of it happening again.21:33
wallyworldyes21:33
Mmikeack21:34
Mmikewhere do I find mgopurge?21:34
menn0Mmike: I'll compile it for you... it's not easily available21:34
waiganiaxw or menn0: this needs a review when you have a moment please: http://reviews.vapour.ws/r/342221:35
wallyworldmenn0: we should consider packaging that binary with juju, like we do for plugins21:35
Mmikemenn0, thnx!21:36
Mmikemenn0, no rush, it's almost 11PM here and I'll doze off shortly21:37
menn0Mmike: ok. how shall I get it to you?21:37
menn0wallyworld: that could work.21:38
Mmikemenn0, how large is it? Email, if it's not largish, or put it to mombin somewhere and allow me to access it? (mario is the username)21:39
wallyworldmenn0: yeah, would be handy as i suspect we'll need it again21:39
menn0Mmike: it's not too big21:39
Mmikethen email could do it21:39
Mmikewallyworld, menn0, thnx, lads!21:41
MmikeI'm dozing off now.21:41
wallyworldMmike: np, menn0 did all the work :-)21:42
Mmikewallyworld, you did the connecting :) If you were in sales... :)21:42
wallyworldlol21:42
=== ericsnow_afk is now known as ericsnow
perrito666wallyworld: I might be a couple of mins late for the standup, grocery shopping22:31
wallyworldsure, np22:31
perrito666wallyworld: axw alexisb ready23:20
alexisbwallyworld, I am running late23:20
wallyworldnp23:21
wallyworldperrito666: we are here already23:21

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!