/srv/irclogs.ubuntu.com/2014/08/14/#juju-dev.txt

davecheneythumper: ping, i'm free to talk whenever you are00:30
katcowallyworld: https://github.com/juju/juju/pull/513 :)00:31
katcowallyworld: going to eat dinner, i'll check for a LGTM in a bit and hopefully land it!00:31
wallyworldkatco: awesome. for the 1.20 update, you will need to ensure you use the 1.20 branches of the relevant sub repos ie you will need to update their imports and commit the changes and then use the relevant rev shas01:00
axwo/ wallyworld01:06
axwhow was nuermberg?01:06
axwnuern nurem... equal edit distance01:07
wallyworldaxw: hey ya. good but busy. we gots a lot to do. got time for a chat?01:18
axwwallyworld: sure01:18
wallyworldsee you in standup meeting01:19
thumperdavecheney: around?01:43
davecheneyyeah01:48
davecheneythumper: sup01:48
thumperchat?01:48
davecheneythumper: sure, lets meet in the 1-1 hangout01:49
thumperkk01:49
* thumper is there waiting for davecheney01:51
wallyworldthumper: did you want to delay our 1:102:06
thumperwallyworld: yes02:06
wallyworldok, just ping me02:06
thumperwallyworld: now?02:36
wallyworldsure02:36
davecheneylucky(~/src/github.com/juju/juju) % gb ./...                                                                                                          │················02:44
davecheneyjuju/sockets/sockets.go:6:2: no buildable Go source files in /home/dfc/src/gopkg.in/natefinch/npipe.v202:45
davecheneyanyone else getting this error ?02:45
davecheneywallyworld: thumper axw https://github.com/juju/juju/pull/51402:49
davecheneyreview please02:49
davecheneythis is blocking my build02:49
wallyworlddavecheney: otp, will look soon02:50
axweh, I thought it was updated to add a file for linux02:50
* axw check02:50
axws02:50
davecheneyi ran godeps and it didn't complain02:51
davecheneyi belive I have the right rev02:51
davecheneyseriously02:51
axwdavecheney: I think you just need to merge trunk again, I think there's a new rev02:51
davecheneythis isn't the right way to fix the problem02:51
davecheneyaxw: trunk of which repo ?02:51
axwnpipe02:52
axwum02:52
axwmerge trunk of juju02:52
axwthere's a new rev of npipe in dependencies.tsv02:52
davecheneygopkg.in/natefinch/npipe.v2     git     e562d4ae5c2f838f9e7e406f7d9890d5b02467a902:53
davecheneythat is what it says02:53
davecheneyand that is the rev I have02:53
davecheneyI FUCKING HATE GOPKG>IN AND THE INTERACTION WITH GODEPS02:54
davecheneyI HATE IT SO MUCH02:54
davecheneyI MAKES ME MISERABLE02:54
* thumper chuckles 02:54
axwdavecheney: what's the problem exactly? I'm not having any problems with godeps02:55
davecheneywe have two tools that are trying to do the same thing, pin a dependency of a go package02:55
davecheneyi think it is a serious error to combine those two tools02:56
thumperdavecheney: I don't think it is too terrible, just annoying02:56
thumperone seems to pin (vaguely) the api, the other pins to commits02:57
wallyworlddavecheney: did you see all the syslog stuff being done? are you happy with it? i thought at one point you may have had an opinion on it? maybe i am misremembering?02:59
davecheneywallyworld: as in moving away from rsyslog and doing file rolling inside the application ?03:00
wallyworlddavecheney: that and there was talk of using Go's syslog, not sure if that went ahead or not03:00
davecheneydon't use that package03:00
davecheneyit's fucked03:00
davecheneywe can't remove it03:00
davecheneybut it's still fucked03:00
wallyworldyup03:00
davecheneyi also am not in favor of doing file rolling inside the applicatoin03:01
davecheneyfrom a decade of sysadmin experience03:01
wallyworldyep03:01
davecheneythat is a sure fire way to loose logs03:01
wallyworlddavecheney: can you replay to the thread on juju-dev, or email nate directly with your concerns?03:01
wallyworldreply03:01
davecheneywallyworld: will my comments be given serious consideration03:01
davecheneyi've not seen a lot of that in the past03:02
davecheneyi know that is a snotty thing to say03:02
davecheneybut i'm not going to kick up a stink unless there is actual possiblity of chance03:02
davecheneychange03:02
davecheneyotherwise i'll save everyone the discomfort of hearing me rant03:02
wallyworldthey will be taken seriously by me, and whomelese else i talk to i'll tell them :-)03:02
wallyworlddon't rant as such, just provide facts03:02
wallyworldand i'll make sure nate and whoever else takes note03:03
davecheneyhere are the facts03:03
davecheney1. the state/api/database/mongo whatever runs on linux03:03
davecheneyit will continuye to run on linux for the forseable future even if we have windows workloads03:03
davecheneywe should continuye to use rsyslog for log collection03:03
davecheneyas it decouples the application from log rolling03:03
davecheneyapplications write to stdout03:03
davecheneysomethign else takes care of where that stdout goes03:04
davecheneyif you make the application responsible for rolling it's own logs03:04
davecheneyyou'll generally find that you end up writing to a log file that has been unlinked03:04
axwI did comment on the PR about the approach to rolling being a bit broken, but I don't see any change in nate's new PR03:04
wallyworldi agree. can you put that in an email to the list? feel free to mention your previous sysadmin experience to add weight to your arguments03:05
davecheneywallyworld: http://12factor.net/logs03:05
davecheney^ this is what we shold be aiming for03:05
davecheneywallyworld: i honestly don't think i can be objective at thispoint03:05
davecheneyi've given you what I can03:05
davecheneyi don't want to take on the heartache of another loosing battle03:06
wallyworldhmmm, ok. i would prefer you provided the raw material (based on yuor experience) - we can run with it from there03:06
wallyworldie fight the battle03:06
wallyworldbut i can do it also03:06
davecheneywallyworld: which thread ?03:07
* wallyworld looks03:07
davecheneyi can't find it03:07
davecheneyif it's on juju-team03:07
davecheneyi'm not subscribed to that list03:07
davecheneyor i should say, my subscription is still pending03:07
wallyworldjuju-dev - getting rid of all-machines.log03:08
wallyworldbut we could do a new post referencing that thread03:08
axwalso: https://github.com/juju/juju/pull/375, superseded by https://github.com/juju/juju/pull/51203:09
axwwallyworld: that big test was just moved. I'd rather not change existing things too much at this point03:23
axwah you noticed :)03:23
wallyworldaxw: yeah, just made a followup comment :-)03:23
* axw tests it once more for good luck03:25
axwwallyworld: can you please review https://github.com/juju/testing/pull/28 too?03:38
wallyworldsure03:38
axwwallyworld: btw, to test with mongo 2.6 you can just do "JUJU_MONGOD=path/to/mongod go test ./..."03:41
axwwill not work in CI, though; that would require additional changes03:41
wallyworldaxw: np, thanks. there was a test CI job I started a while back03:41
axwcool03:42
wallyworldbut i never finished it03:42
TheMuemorning07:39
dimiternmorning TheMue07:41
dimiternI thought jam is here today..07:42
TheMuedimitern: IMHO he is on holiday the whole week07:52
TheMuedimitern: oh, no, just took a look into the calendar07:52
dimiternTheMue, yeah, maybe he'll pop in at some point today07:54
TheMuedimitern: yep07:56
TheMuedimitern: so comming back to my change of SupportsNetwork and its usage07:57
dimiternTheMue, how's that capability branch for the networker going?07:57
dimiternTheMue, :)07:57
TheMuedimitern: bingo07:57
waigani_axw: https://github.com/juju/juju/pull/51807:57
TheMuedimitern: the capability is currently already implemented, imho only with the wrong naming07:58
TheMuedimitern: but is this enough07:58
TheMuedimitern: and then use it where today we test if the providerType is MAAS?07:59
axwwaigani_: looking08:04
axwwaigani_:  :x a much simpler solution has just dawned on me08:06
axwwaigani_: we could have just got the bootstrap instance and used its Addresses method...08:07
dimiternTheMue, there's a new capability needed, let's call it AllowOnlySafeNetworker(machineId string) bool08:07
* axw continues looking anyway08:07
dimiternTheMue, which will return false for all providers, except local08:08
dimiternTheMue, and for local it will check internally if the machineId == bootstrapMachineId (or however it's called) and return true (i.e. do not allow unsafe networker on the bootstrap node) or false (for all other machines)08:09
TheMuedimitern: ah, already wondered. this description better matches what I’ve seen in the discussion, thanks.08:10
TheMuedimitern: as we call them mostly Supports… (reads better) I would call it SupportsUnsafeNetworker(machineId)08:11
dimiternTheMue, I'm fine with that (in this case the meaning is inverted - true for most, false for local bootstrap node)08:12
TheMuedimitern: I’m also not happy with Networker and SafeNetworker, which implies the first one is „unsafe“ ;)08:13
axwwaigani_: commented on the PR08:13
dimiternTheMue, well it is kinda unsafe :)08:14
dimiternwaigani_, the bootstrap node can have and does have more than one address usually08:14
waiganiaxw: you horrible man08:20
axwwaigani: :(08:20
axwwaigani: on the plus side, the alternative is really quick to implement :p08:21
waiganiaxw: lol, no it's a good solution - I raised an eyebrow when I came across Addresses, but assumed we could not use it for all providers08:21
waiganiaxw: will the bootstrap instance have  Id() == "0" ?08:41
axwwaigani: no08:42
axwthat's machine ID08:42
axwinstance ID you cannot predict08:42
axwwaigani: just call AllInstances() and get the one and only element out of it08:42
waiganiaxw: okay, so we only bootstrap one instance? When to the HA instances get bootstrapped?08:43
axwwaigani: bootstrap = start the first instance08:43
axwHA comes later, when you do "juju ensure-availability"08:43
waiganiright08:44
voidspac_dimitern: TheMue: just noticed your conversation08:47
voidspac_dimitern: TheMue: shouldn't "safe networking" be true for manual provider too08:47
dimiternvoidspac_, yes08:48
voidspac_dimitern: TheMue: and I agree "safe" is not a brilliant name08:48
voidspac_but I dislike burning too much energy on name bikeshedding08:48
voidspac_pick something and use it08:48
dimiternvoidspac_, for all machines with the manual provider, and only for the bootstrap node with local08:48
voidspac_yep08:48
axwdon't forget machines can be manually added to a non-manual provider environment08:49
axwin which case provider != "manual"08:49
voidspac_axw: oh, ouch08:49
axwprobably don't want to muck with their networking either08:49
voidspac_dimitern: the unit test failures are all uniter tests08:49
voidspac_dimitern: are you / is anyone looking at them?08:49
dimiternvoidspac_, which failures?08:50
voidspac_dimitern: the ones jam emailed us about yesterday08:51
voidspac_dimitern: CI failures08:51
TheMueaxw, dimitern: so manual || local bootstrap || (non-manual && manually added)?08:51
voidspac_dimitern: you said you would look at them :-)08:51
voidspac_http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1459/console08:51
dimiternaxw, good point; TheMue you should check for manually provisioned machines and not allow unsafe networker there as well08:51
voidspac_http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1458/console08:51
voidspac_http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1457/console08:51
voidspac_http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1456/console08:51
dimiternaxw, what's the best way to tell if a machine is manually provisioned by its id?08:51
dimiternvoidspac_, aw, sorry, I'm struggling with networker refactoring, will have a look before the standup08:52
axwdimitern: from what context? you can tell from the state.Machine08:52
voidspac_dimitern: I can have a look, although maybe not before standup08:52
dimiternaxw, ah, so the machine id is not special08:53
TheMueaxw: idea is to have an environ capability able to say if unsafe networkers are supported by passing the machine id08:53
axwdimitern: nope. the nonce is special08:53
dimiternaxw, we'll need to lookup the machine in state via the api to be able to tell08:53
axwdimitern: that would probably be best08:53
dimiternaxw, is the nonce constant?08:53
axwdimitern: yup08:54
axwset by the instance broker08:54
axwwell, in this case it's set by the manual provisioning code08:54
dimiternaxw, alright, thanks08:54
axwdimitern: TheMue: state.Machine.IsManual()08:54
TheMueaxw: nice08:54
* TheMue takes some notes08:55
TheMueHmm, just found „SetHasVote()“. Looks strange, would have named it „GrantVote()“.09:14
waiganiaxw: I'm hitting the same problem that I hit when I connected to the api after bootstrap: something is hanging. I've set state-server to true on tests09:21
waiganiaxw: here's what I've done: http://pastebin.ubuntu.com/8043738/09:22
axwwaigani: *shouldn't* need to do that... there should be no interaction with the state server to obtain the instances or their provider addresses09:22
waiganiaxw: I wonder if it is this: c.ConnectionEndpoint(false)09:23
axwwaigani: I have nfi what ConnectionEndpoint is09:23
mattywfwereade_, ping?09:30
mattywfwereade_, ^^ I'm going to rebase the metric branch before landing - I think most of the review was us working around what should be implemented so I'm going to cleanup the history to make it easier to understand09:31
perrito666morning09:33
perrito666fwereade_: ping me when you arearound plz09:45
=== rogpeppe1 is now known as rogpeppe
waiganiaxw: okay I got it working. Do we also only want to grab the first address of the first instance?09:59
axwwaigani: I think we should just get them all, like the existing api address caching does10:00
waiganiaxw: cool, that is how I have it now10:00
davecheneyhttps://bugs.launchpad.net/juju-core/+bug/135680610:18
mupBug #1356806: cmd/juju: juju takes 3 seconds to do nothing <juju-core:New> <https://launchpad.net/bugs/1356806>10:18
rogpeppedavecheney: it's a pity that parsing rsa keys is so slow. ISTR that it's because it does a "probably prime" check when it parses the key.10:29
rogpeppedavecheney: although i thought that issue was fixed. hmm.10:29
davecheneyrogpeppe: this is on ppc6410:34
davecheneywhere there is no asm math library10:34
davecheneyjuju is just doing something dumb10:34
davecheneyit doesn't _always_ need to read the private key10:34
rogpeppedavecheney: i agree it's dumb, but it should not take 3s to parse an RSA key on any platform10:34
davecheneyi don't know how optimised the ppc port is at the moment10:34
davecheneyone would suggest, not at all10:34
rogpeppedavecheney: even if not optimised10:34
rogpeppedavecheney: parsing an RSA key should not require many math operations10:35
rogpeppedavecheney: it's essentially just reading a couple of big numbers10:35
davecheneyrogpeppe: i think we're in agreement here10:35
davecheneyyet here we are, waiting 3 seconds for nothing to happen10:36
rogpeppedavecheney: i'd like to fix both places10:36
rogpeppedavecheney: both the Go library (to make parsing RSA keys more efficient) and juju-core (to avoid doing that when it doesn't need to)10:36
dimiternalexisb, ping10:36
dimiternvoidspac_, standup?10:46
voidspac_dimitern: I'm there...10:47
voidspac_dimitern: connection lost :-(10:58
voidspac_dimitern: overlapping subnets are not allowed in the new model - we have one network10:58
dimiternvoidspac_, yes, but where have you seen the overlapping subnets in the model?10:58
voidspac_dimitern: but isn't it possible that subnets on different nics will look like they overlap10:59
voidspac_but don't because they're on separate networks10:59
voidspac_but we can't represent that in the new model10:59
voidspac_cidr is the primary key10:59
dimiternvoidspac_, I'm not sure I understand10:59
voidspac_or should the different nics be bound to different subnets anyway11:00
dimiternvoidspac_, cidr is the pk for a subnet, yes11:00
dimiternvoidspac_, nics can be bound to one or more subnets11:00
voidspac_dimitern: two different networks can have the same subnets allocated to different machines though, right?11:00
voidspac_dimitern: so the same cidr can be duplicated on different networks11:00
dimiternvoidspac_, the confusion I think comes from the term "network"11:00
voidspac_heh11:01
dimiternvoidspac_, in juju, per the new model, all subnets are in the same (virtual/logical?) network11:01
dimiternvoidspac_, in provider-space, if networks and subnets are supported, there is a possibility to have 2 networks with overlapping subnets11:02
voidspac_dimitern: right11:02
voidspac_dimitern: how do we cope with that in the juju model?11:02
dimiternvoidspac_, but I don't think this is a valid case11:02
voidspac_dimitern: ok11:02
voidspac_dimitern: because they won't be routable to each other and so you can't use them for anything useful11:03
dimiternvoidspac_, it a weird setup, and we should not allow that, aiui11:03
dimiternvoidspac_, yes11:03
voidspac_dimitern: but we currently don't "disallow it", we pretend it can't happen11:03
dimiternvoidspac_, but good point about bringing this up11:03
voidspac_you can still have unroutable subnets, and just whichever one is added first wins11:03
voidspac_dimitern: I think in summary, networking is hard11:04
dimiternvoidspac_, another possible case is having overlapping vlan subnets with different tags11:04
voidspac_right11:04
dimitern:) oh yeah11:04
dimiternvoidspac_, you can't do this in maas I think11:04
voidspac_dimitern: TheMue:  also I spent a while talking to perrito666 yesterday - current restore does not *need* direct mongo access11:05
dimiternvoidspac_, so juju shouldn't take it either11:05
voidspac_dimitern: TheMue: it just uses it after doing the restore to update some data, and can be fixed to not do that11:05
voidspac_dimitern: right11:05
dimiternvoidspac_, great!11:05
voidspac_dimitern: fair enough11:05
voidspac_dimitern: TheMue: so that's what I've been on, I was going to swap and look at the bugs11:05
voidspac_dimitern: I have a precise VM, should I try and repro these bugs?11:05
dimiternvoidspac_, yes please, I'll do the same; hopefully different setups can lead to reproduction of at least some bugs11:07
voidspac_dimitern: cool, coffee first then I'll spin up my VM11:07
natefinchfwereade_: team leads?11:17
sinzuiwallyworld, ppc64el unittests are failing in many different ways.12:03
wallyworldoh joy12:03
sinzuiwallyworld, I am going to increase the number of retries, and also look for cruft left on the machine12:03
sinzuiI am not sure juju is really bad on ppc12:04
wallyworldsinzui: could it be new tests with ordering issues, or is it existing tests that have started failing? i'll have to look at the logs. currently in standup12:04
dimiternvoidspac_, managed to reproduce some of the ci test failures on a precise vm using rev 9b2d2106476bad0ac528256db23bad073257d4bf12:32
voidspac_dimitern: cool12:32
voidspac_dimitern: any ideas on the cause?12:35
voidspac_dimitern: I took a break and am now still updating the vm12:35
dimiternvoidspac_, actually I'm having issues with mongo now12:35
voidspac_juju won't build for me12:35
voidspac_think i need to update go12:35
dimiternvoidspac_, I installed mongodb-server from the ppa:juju/experimental12:35
voidspac_godeps completed but still build errors12:35
voidspac_ah12:35
voidspac_ls12:35
dimiternvoidspac_, you need to install mongodb-server and golang from there12:35
voidspac_dimitern: I can't just build golang from source?12:36
natefinchwhere Go comes from is certainly not the problem12:37
dimiternvoidspac_, you can ofc, but to mimic the test environment as close as possible, I'd use the ppa12:37
voidspac_ok12:37
voidspac_just added the ppa and doing an update12:37
voidspac_go is building anyway12:37
voidspac_natefinch: actually I think it was the problem (which Go I was using)12:44
voidspac_natefinch: I think we're no longer compatible with the golang in precise12:44
voidspac_which I was using before12:44
voidspac_but building go from source works fine12:45
natefinchvoidspac_: oh, yeah, sorry, I thought you were using something reasonable12:45
natefinch:D12:45
voidspac_although I now have golang from the experimental ppa as well12:45
dimiternvoidspac_, interestingly, I can run (most of) state/ tests without any mongo issues, but for agent/ or worker/uniter/ tests I get the same failures "cannot set admin password: need to login"} ("cannot set admin password: need to login"12:45
voidspac_natefinch: I'd moved it out of the way to try something a while ago and not moved it back12:45
natefinchvoidspac_: I think precise has like 1.1 or something, and we need ~1.2.112:46
voidspac_dimitern: running worker/uniter now12:46
voidspac_natefinch: right12:46
natefinchvoidspac_: newer versions of Go always work with older code.  But older versions of Go don't always work with newer code12:46
voidspac_natefinch: the actual build error was something nice and helpful like12:46
voidspac_../../../code.google.com/p/go.crypto/openpgp/packet/encrypted_key.go:8: import /home/michael/canonical/pkg/linux_amd64/code.google.com/p/go.crypto/openpgp/elgamal.a: object is [linux amd64 go1.2.2 X:none] expected [linux amd64 go1.1.2 X:none]12:47
voidspac_which looked like a version error to me...12:47
dimiterndoes anybody have any clue about that error? "cannot set admin password: need to login"12:47
voidspac_natefinch: right, in theory anyway12:47
voidspac_dimitern: axw has done a lot of work in that area recently12:47
natefinchahh yeah.... that's actually just old binaries in your pkg directory12:47
dimiternthe "need to login" part happens in several unrelated tests, but not all of them fail12:47
dimiternvoidspac_, this is actually on trunk HEAD12:48
natefinchvoidspac_: they're actually really really really careful to make sure they maintain backwards compatibility.12:48
dimiternvoidspac_, I had too many issues with rev 9b2d2106476bad0ac528256db23bad073257d4bf so decided to try trunk first12:48
voidspac_dimitern: right12:48
voidspac_*sigh*12:52
voidspac_adding upstream repo and actually pulling HEAD12:53
voidspac_now re-running12:53
voidspac_I had some failures, but I wasn't on HEAD12:53
voidspac_although I was on master, so failures still od12:53
voidspac_*odd12:53
voidspac_but different12:53
voidspac_filter_test.go:305 c.Fatalf("unexpected config event")12:54
dimiternvoidspac_, no failures with worker/uniter on HEAD; now re-running on 9b2d2106476bad0ac528256db23bad073257d4bf, after I wiped out $GOPATH/pkg/, just in case12:55
voidspac_dimitern: I get one failure on HEAD12:56
voidspac_the same as above12:56
dimiternvoidspac_, lots of failures, all of them stemming from "need to login"12:56
voidspac_dimitern: so it looks to you like that issue has been fixed12:57
voidspac_if it's on 9b2d2106476bad0ac528256db23bad073257d4bf but not on head12:57
dimiternvoidspac_, I think something's wrong with my mongo there, I want to fix this first, so I can have a reliable setup12:58
voidspac_ok12:58
voidspac_I'm trying worker/uniter with 9b2d2106476bad0ac528256db23bad073257d4bf12:59
voidspac_dimitern: I get no worker/uniter failures on 9b2d2106476bad0ac528256db23bad073257d4bf13:04
dimiternvoidspac_, hmm how about for agent/ ?13:04
voidspac_will try13:05
voidspac_dimitern: I re-ran the worker / uniter tests and got the failure I got previously13:11
voidspac_ran agent/... tests fine13:11
voidspac_no failures13:11
voidspac_but I didn't have this failure on worker/uniter before13:12
voidspac_so looks like it's flaky rather than consistent13:12
voidspac_but it's still a different failure to the one on CI13:12
voidspac_or the ones you have13:12
dimiternright13:12
voidspac_so, fark13:12
voidspac_not helpful at all13:12
hazmathmm.. recent yaml changes seem to cause some compat issues.. $ juju status -> error unmarshalling "/opt/juju/environments/ocean.jenv": YAML error: resolveTable item not yet handled: < (with <root>=DEBUG;unit=DEBUG)13:39
dimiternvoidspac_, what mongodb are you using?13:58
dimiternvoidspac_, $ `which mongod` --version ?13:58
dimiternvoidspac_, sorry, I meant $ which mongod && mongod --version14:01
perrito666natefinch: wwitzel3 standup14:01
dimiternvoidspac_, the version from the ppa is 2.2.4, but this is too old for what we're doing around state.Initialize(), more specifically, SetAdminMongoPassword uses admin.UpsertUser(), which is available in 2.4+14:02
dimiternsinzui, ping14:02
sinzuihi dimitern14:02
dimiternhey, I think I discovered a potential regression re precise + mongodb14:03
sinzui:(14:03
dimiternsinzui, but it's funny how it doesn't show on the CI tests - is the mongod there 2.2.4 from ppa:juju/experimental ?14:03
dimiternsinzui, because I can reproduce this issue at will with both HEAD of trunk and rev 9b2d2106476bad0ac528256db23bad073257d4bf (from http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1456/consoleFull)14:04
sinzuidimitern, CI uses 2.4.6 to match cloud-tools14:04
dimiternsinzui, from where?14:05
sinzuihttp://ubuntu-cloud.archive.canonical.com/ubuntu/pool/main/m/mongodb/14:05
sinzuidimitern, ^ looks like I should switch to 2.4.9 to match14:05
dimiternsinzui, how can I install mongodb-server from there? add-apt-repo ...?14:06
dimiternsinzui, is it know that we no longer support mongodb 2.2.x, even on precise?14:07
dimiternknown*14:07
hazmatsinzui, is manual provider in our functional tests?14:07
hazmati'm encountering quite a few regressions.. just trying to determine sanity14:07
sinzuihazmat, yes, deploys an upgrades to instance in aws and real metal14:08
dimiternah, found it - ppa:juju/stable14:09
sinzuisudo add-apt-repository cloud-archive:tools14:10
hazmatsinzui, hmm.. ic. okay. i'll keep filing bugs then14:10
sinzuidimitern, https://wiki.ubuntu.com/ServerTeam/CloudToolsArchive14:10
dimiternthanks sinzui14:10
sinzuidimitern, I copy packages to juju stable because many people fail to install that archive when using local precise14:10
=== bloodearnest_ is now known as bloodearnest
dimiternsinzui, right, I'm trying the same tests now with 2.4.6 from the archive14:12
dimiternsinzui, I can't reproduce any of the failures now :(14:15
hazmatsinzui, looks like manual provider add-machine but oddly not bootstrap is failing if there isn't an ubuntu user in the base image.14:17
sinzuiah14:17
dimiternsinzui, but I have a suggestion how to add debug logging to mgo, so we can see what's going on in more detail14:17
sinzuihazmat, we have a bug for a cases where ubuntu was removed from a server image and lxc fails14:18
hazmatsinzui, also wierdly bootstrap but not add-machine uses ssh-agent14:18
=== jw4 is now known as jcw4
voidspac_dimitern: /usr/bin/mongod14:20
voidspac_db version v2.4.614:20
voidspac_Thu Aug 14 17:19:47.712 git version: nogitversion14:20
voidspac_dimitern: I added the ppa and installed mongodb-server14:20
dimiternvoidspac_, right I suspected that much - it'll never work with mongodb-server 2.2.4 from ppa:juju/experimental14:20
voidspac_dimitern: but it must be preferring a newer version from elsewhere14:20
voidspac_dimitern: but that's not the same error that CI has, right?14:21
dimiternvoidspac_, with mongodb-server 2.4.6 from cloud-archive:tools - no failures14:21
voidspac_they just have uniter test failures14:21
bacjcastro: cross-team call today?14:21
voidspac_ah14:21
voidspac_dimitern: what about CI with HEAD?14:21
voidspac_I guess I can check...14:21
dimiternvoidspac_, I'll try HEAD next, just running uniter tests once more to be sure14:21
sinzuihazmat, I re added ubuntu user to the kvm/maas machine I am provisioning. And I provisioned it with manual provider14:22
voidspac_dimitern: 1470 passed!14:22
voidspac_http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1470/14:22
dimiternvoidspac_, sweet!14:22
voidspac_REVISION_ID=abfd7625309d31ecddd8fa799d64c8d4fa41977c14:22
voidspac_is that head?14:22
voidspac_I believe so14:23
hazmatsinzui, ok.. its a regression though.. used to work in 1.18.. glad to hear it works if the bug isn't triggered but ideally it could be part of the regression check.14:24
dimiternvoidspac_, so I think the failures were either bogus or they got fixes in subsequent branches14:26
voidspac_dimitern: I think so14:26
voidspac_dimitern: there are some canonistack-deploy failures14:27
voidspac_http://juju-ci.vapour.ws:8080/job/canonistack-deploy-precise-amd64-devel/14:27
voidspac_dimitern: but they're time-outs, so not sure how seriously to take those14:27
voidspac_I get that with canonistack fairly often14:27
dimiternvoidspac_, perhaps the previous failed test didn't clean up properly?14:29
voidspac_dimitern: well, the very latest one is a bootstrap failure - the two before that are timeouts14:29
voidspac_so yes, looks like the test environment is now screwed :-)14:29
dimiternvoidspac_, yes, my thoughts exactly :)14:31
dimiternsinzui, are you seeing this ^^14:31
sinzuivoidspac_, dimitern . I have seen it. I have not completed investigation. Don't panic. The tests are -devel because canonistack is not production grade. when swift gets slow for example, everything fails14:32
voidspac_sinzui: but the unit test failures have been fixed14:33
voidspac_http://juju-ci.vapour.ws:8080/job/run-unit-tests-precise-amd64/1470/14:33
voidspac_current HEAD14:34
sinzuivoidspac_, yes. I should admit that ppc64el is doing worse. I increased retesting to 4 to improve the chances to pass14:39
voidspac_sinzui: that's showing latest build passed too14:41
voidspac_Same revision.14:41
voidspac_Looks like the increased retesting did the trick...14:42
voidspac_There was a replicaset failure for that revision.14:43
Beretalexisb, sinzui - I forgot to mention this on the call14:47
Beretalexisb, sinzui - sparkiegeek has a branch up to fix an issue we found, would appreciate it getting through whatever process - https://github.com/juju/utils/pull/2214:47
Beretthat hits us quite a lot - it would be good to get it into 1.2014:48
alexisbBeret, ack14:48
Beretthanks14:48
alexisbanyone know who is the oncall reviewer today?14:48
alexisbnatefinch, ^^14:49
natefinchalexisb: tim and dave: https://docs.google.com/a/canonical.com/spreadsheets/d/1iQLLOWrjzxddm5VhYWYi0-2k3xI6wTMlpkvnVNJCYGY/edit14:51
sinzuiBeret, does that address bug 135468514:52
mupBug #1354685: installation of packages for containers should be retried in face of lock errors <cloud-installer> <landscape> <lxc> <juju-core:In Progress by adam-collard> <https://launchpad.net/bugs/1354685>14:52
sparkiegeeksinzui: yes :)14:52
natefinchalexisb: I know that's not helpful14:52
sinzuisparkiegeek, excellent...CI experiences this issue often.14:53
Beretsinzui, yes14:55
dimiternBeret, sparkiegeek, alexisb, https://github.com/juju/utils/pull/22 reviewed15:02
alexisbdimitern, you rock, thanks!15:02
sparkiegeekalexisb: hear hear!15:05
sparkiegeekdimitern: thanks, first reaction: awesome review :)15:05
hazmatsinzui, is there any trick to building 1.20 branch.. its been broken for me for a while.. here's my latest attempt.. http://paste.ubuntu.com/8046050/15:06
dimiternsparkiegeek, cheers :)15:06
natefinchhazmat: bzr branches have broken on me a few times over the past few months.  if you delete /home/kapil/src/launchpad.net/gnuflag and go get it again, it should be fine15:08
natefinch(and then rerun godeps)15:09
hazmatnatefinch, that did the trick15:09
hazmatthanks15:09
sinzuihazmat, natefinch speaks the truth. I have also changed to each branch and pulled the lastest. godeps doen't pull, so if the rev isn't in the repo, it fails15:09
hazmatsinzui, yeah.. i had been manually doing bzr pull but it kept outputting no changes.. too late to debug more now.. but good to know the workaround.15:10
dimiternsinzui, hazmat, I have a couple of nice scripts to run godeps and update automatically what's needed - I'm sending a mail to juju-dev in case you might be interested15:14
hazmatdimitern, sounds like a great merge request for a makefile ;-)15:15
natefinchsinzui, dimitern, hazmat: current godeps -u *will* fetch new revisions from the remote repo15:15
hazmatnatefinch, *should*15:15
natefinchsinzui, dimitern, hazmat: sometimes you have to do run godeps more than once15:16
dimiternnatefinch, oh really?15:16
dimiternnatefinch, yep, my script takes care of that15:16
hazmatnatefinch, didn't work for me.. and running multiple times .. with intermittent failures.. ick15:16
natefinchhazmat: one time can't possibly always work15:16
hazmatnatefinch, pull before update seems sane to me15:17
natefinchhazmat: actually... strike that15:17
natefinchhazmat: yeah, the new code does that with -u15:17
natefinchhazmat: where "new" = a month or so ago15:17
hazmathmm.. mine is pretty old (~march).. does the new version support comments15:18
* hazmat updates15:18
natefinchhazmat: that should fix the problem of not pulling down updates before trying to set the current commit15:21
rogpeppewhen i use juju switch, i see:15:25
rogpeppeERROR couldn't read the environment15:25
rogpeppethis seems to have broken some time relatively recently15:25
rogpeppeha, it should really print the actual error15:25
rogpeppeah, that's better:15:26
rogpeppeERROR couldn't read the environment: cannot parse "/home/rog/.juju/environments.yaml": YAML error: found character that cannot start any token15:26
hazmatrogpeppe, yup.. recent yaml changes15:26
ericsnowrogpeppe: when you have a minute could you take a look at https://github.com/juju/utils/pull/19?15:26
hazmatrogpeppe, i had the same issue earlier on trunk. needed to quote some strings15:26
hazmatin my jenv file15:27
=== niemeyer_ is now known as niemeyer
rogpeppehazmat: actually, it seems i accidentally had a tab at the start of my environments.yaml15:27
hazmatrogpeppe, oh. i had this one .. $ juju status -> error unmarshalling "/opt/juju/environments/ocean.jenv": YAML error: resolveTable item not yet handled: < (with <root>=DEBUG;unit=DEBUG)15:27
rogpeppehazmat: (i think it was probably me experimenting with whether yaml is fully JSON-compatible)15:27
katcorogpeppe: sorry about that; yesterday i put us on a newer version of goyaml.15:27
rogpeppekatco: it's not your fault15:28
rogpeppekatco: it's the fault of whoever printed that error without including the actual cause...15:28
katcohazmat: i received a _lot_ of those errors on goyaml head, so i used an old commit that targetted a fix we wanted. didn't look into it too much15:28
* rogpeppe does not run git blame.15:28
rogpeppekatco: oh, interesting.15:28
rogpeppekatco: we're not using gopkg.in/yaml.v1 tip?15:28
hazmatkatco, fair enough.. i'm  glad for the underscore stripping fix.. just worried about users in the field who are going to encouter this when they upgrade15:29
katcorogpeppe: we are not. lots of tests failed with hazmat's error15:29
katcohazmat: did you figure out what it was?15:29
rogpeppehazmat, katco: i'm guessing it's a quoting error on output15:29
rogpeppeperhaps it was not adding the correct quotes for strings containing "<"15:30
hazmatkatco, we're writing out jenv with    logging-config: <root>=DEBUG;unit=DEBUG15:30
rogpeppehazmat: yup15:30
hazmatkatco, but it couldn't be read without quoting it "<root>=DEBUG;unit=DEBUG"15:30
dimiternhazmat, sinzui, mail sent (Running godeps -u dependencies.tsv easily)15:31
hazmatwhat rogpeppe said ;-)15:31
katcohazmat: rogpeppe: ah i see... so we're causing our own failure with automatied output? is that going to be an issue for 1.20.415:31
katco?15:31
hazmatkatco, very likely15:31
katcohazmat: frown. so we need to fix that before i backport?15:32
natefinchgah, this is why yaml is a PITA15:32
hazmatkatco, ie. if a user has existing an envs, and upgrades, they can't use juju without manually editing their jenvs..15:32
rogpeppehazmat: interesting pyyaml also prints that string unquoted15:32
rogpeppehazmat: (actually i think it might be the same original C code base)15:32
hazmatrogpeppe, yeah.. but it parses it fine.15:32
katcohazmat: yeah that's not a good experience at all.15:32
hazmathmm.. i should verify that with trunk using godeps.. its possible i accidentally just ran with the binary produced from go get.. since it actually worked and i wanted trunk of juju.15:34
rogpeppehazmat, katco, niemeyer: yes, it's definitely a bug in gopkg.in/yaml.v1 tip15:35
rogpeppethis fails: http://paste.ubuntu.com/8046246/15:35
rogpeppeand it should not15:36
rogpeppeas it's producing output that it cannot parse15:36
katcorogpeppe: sounds like it might be a bug further back in history as well?15:37
* rogpeppe bisects15:37
katcohttps://github.com/go-yaml/yaml/commit/1418a9bc452f9cf4efa70307cafcb10743e64a56#diff-d41d8cd98f00b204e9800998ecf8427e15:37
katcois the version trunk is using (and what we're wanting to backport to 1.20)15:38
hazmatkatco, yeah.. with the pinned version in dependencies.tsv things work as expected15:38
hazmatso no issue there15:38
katcohazmat: ahh great!15:38
katcohazmat: sounds like i made the correct decision ;)15:38
hazmatindeed :-)15:38
rogpeppekatco: you definitely did15:40
rogpeppelooks like the problem was introduced with 72c33f6840f49f9ed7d1faef7562b3266640fdf415:41
rogpeppewhich is not surprising, as https://github.com/go-yaml/yaml/issues/1 shows a < being used as a special char15:41
rogpeppeonce again, yaml is too complex for its own good15:42
rogpeppehazmat, katco, niemeyer: https://github.com/go-yaml/yaml/issues/2415:47
katcorogpeppe: thank you kindly rogpeppe15:47
rogpeppekatco: np15:47
* perrito666 suddenly notices he cannot ssh from state server into agents bc so straightforward as he thought15:56
bodie_fwereade_, if/when you're available, could you comment on https://github.com/juju/juju/pull/415 ? I think it's good to go but it still has the overwrite behavior, which seems simpler to me and more like what is probably desired16:12
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1356899
sinzuihi natefinch hazmat identified a regression between 1.20 and 1.21. This isn't a blocking bug because CI didn't find it. Can you help direct an engineer to look at bug 135689916:29
mupBug #1356899: manual provider add-machine fails if no ubuntu user on machine <manual-provider> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1356899>16:29
natefinchsinzui: interesting, ok16:30
niemeyerrogpeppe: Thanks, I'll have a look16:30
gsamfiraperrito666: try adding: "ForwardAgent    yes" on your machine in ~/.ssh/config16:40
gsamfirahelps alot16:40
alexisbperrito666, I am running a few minutes late17:00
perrito666no worries, I am trying to kick my camera back into life17:00
alexisbperrito666, now I am waiting on my hangout17:04
* alexisb tries firefox17:04
mattywfwereade_, I wonder if the periodicworker should have a mechanism for working out how many times the workerFunc has been called - this could just built into whatever implements it I guess17:08
fwereade_mattyw, ehh, we'll build it when we need it17:16
fwereade_bodie_, tab opened -- and overwriting in general sgtm, I don't think I was arguing against that, I'll take a look17:16
mattywfwereade_, I *might* find it useful in the cleanup worker test I'm writing so that I know that a cleanup has been run17:18
mattywfwereade_, I'll type some things ans see how it *feels*17:19
fwereade_mattyw, sgtm17:19
fwereade_mattyw, it'll need to be goroutine-safe17:19
bodie_fwereade_, cool, thanks.  action-fail should be ready in a few minutes too17:20
bodie_fwereade_, PR 520 up17:33
=== lazyPower_ is now known as lazyPower
alexisbericsnow, team meeting?18:04
alexisbfwereade_, team meeting?18:04
mattywI'm calling it a night then folks, see you all tomorrow18:24
alexisbbye mattyw !18:24
waiganinatefinch: found your comment - usually I get notified by email. Github must not notify for closed PRs...18:31
natefinchwaigani: ahh, dang.  I'll remember that for the future18:32
sinzui:(18:45
sinzuiwe have a utopic regression.18:46
natefinch*sad trombone*18:46
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1356899 1357033
sinzuinatefinch, Can you pass this on to an engineer: https://bugs.launchpad.net/juju-core/+bug/135703318:50
mupBug #1357033: sourcesSuite.TestGetFilesToBackup consistently fails on utopic <ci> <regression> <test-failure> <utopic> <juju-core:Triaged> <https://launchpad.net/bugs/1357033>18:50
natefinchsinzui: yep18:53
alexisbwwitzel3, ready when you are for our 1x1, if you would like to meet early18:53
wwitzel3alexisb: sure :)18:58
jrwrenhow do I get relation data from outside a relation hook?19:01
jrwrenrelation-ids says error: no relation name specified19:01
jrwrenrelation-list says error: no relation id specified19:02
jrwrennevermind. consider ^^ that my rubber duck session.19:02
jrwrenrelation-ids <relation name> works fine. I typoed :(19:03
waiganinatefinch: https://github.com/juju/juju/pull/52119:06
waiganinatefinch: just reverted for now, I'll look at fixing the bug today and propose a new branch19:09
waiganinatefinch: oh and notification of your comment did come through, my mail client is just slow19:18
katcohttps://github.com/juju/juju/pull/522 ready for review20:10
katcoone of the last blockers for 1.20.4 (though i saw we found something new :(  )20:10
natefinchkatco: lgtm'd20:44
katconatefinch: ty, sir20:44
natefinchkatco:  welcome :)20:45
ericsnownatefinch: FYI, I'm planning on talking to Ian and Martin tonight about reviewboard.20:57
natefinchericsnow: awesome, thanks for spending time after hours to do so20:58
ericsnownatefinch: no worries.  I really want to get this moving. :)20:58
natefinchericsnow: awesome21:00
natefinchEOD for me21:00
=== wallyworld_ is now known as wallyworld
=== Ursinha is now known as Ursinha-afk
=== Ursinha-afk is now known as Ursinha
waiganithumper: so instance.Instance  has a Ports function, asks for a machine ID. I could call that from the bootstrap instance, pass in "0" for id.23:36
thumpernuh23:36
thumpernot what you want23:36
* thumper gets back to reviewing...23:36
waiganithumper: environ.Ports() then?23:37
thumperhang on23:37
waiganithumper: another question: should I add a new address for each port returned i.e. ["localhost:1234","locahost:5678"]23:38
* thumper ignores waigani for a minute23:38
waiganihehe23:39
thumperwaigani: if you read the comment on the environ.Ports method, it is pretty clear that it isn't what you want...23:43

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!