| thumper | twice in two days I'm happy I disabled pushing to upstream | 00:04 |
|---|---|---|
| thumper | as it has saved my arse | 00:04 |
| wallyworld | ? | 00:04 |
| wallyworld | sinzui: you finished with tim? | 00:05 |
| thumper | wallyworld: yeah, I'm sufficiently whipped | 00:06 |
| wallyworld | lol | 00:06 |
| thumper | sinzui: https://github.com/juju/juju/pull/314 | 00:06 |
| thumper | menn0: hangout now or after lunch? | 00:07 |
| menn0 | thumper: now is fine with me | 00:07 |
| thumper | menn0: https://plus.google.com/hangouts/_/canonical.com/upgrades | 00:08 |
| wallyworld | davecheney: what's the staus of the potentio mgo bug? have you managed to reproduce in a way that a bug can be filed? | 00:17 |
| wallyworld | sinzui: you got time for a quick chat? | 00:39 |
| jrwren | menn0: haven't caught you online in a week-ish. About that regex... I found it an interesting difference between go regex and perl/python. \d matches unicode digits in python. Thanks for the discussion | 00:49 |
| menn0 | jrwren: yeah, sorry. I've had a several days off over the last week and a bit | 00:50 |
| menn0 | jrwren: thanks for following up on the regex differences. good to know! | 00:50 |
| jrwren | no worries. I just meant to pick up that conversation with you again. | 00:50 |
| sinzui | wallyworld, I am available now | 00:57 |
| wallyworld | sinzui: great, wanna rejoin the hangout from before? | 00:57 |
| wallyworld | https://plus.google.com/hangouts/_/canonical.com/juju-release | 00:57 |
| davecheney | wallyworld: no update | 01:01 |
| davecheney | sorry | 01:01 |
| davecheney | other bugs to fix | 01:01 |
| davecheney | i can see there is an issue wher emongo eats the eerror | 01:01 |
| davecheney | but haven't investgated further | 01:02 |
| wallyworld | davecheney: otp, will ping you soon | 01:02 |
| davecheney | wallyworld: noted | 01:03 |
| wallyworld | davecheney: it's becoming critical we get this issue looked at upstream. is there any way you could get enough info together to file a bug? | 01:43 |
| davecheney | if strings.HasPrefix(tag, "environment-") { | 01:43 |
| davecheney | return ops, nil | 01:43 |
| davecheney | } | 01:43 |
| davecheney | ark! | 01:43 |
| davecheney | wallyworld: i'll send you what I have | 01:44 |
| davecheney | which isn't mich | 01:44 |
| davecheney | wallyworld: http://paste.ubuntu.com/7801230/ | 01:44 |
| wallyworld | davecheney: i was hoping you'd be abe to progress it to the point of filing the bug. do i need to ask thumper for some of your time? | 01:44 |
| davecheney | ^ that's it | 01:44 |
| davecheney | wallyworld: yes | 01:44 |
| davecheney | i am working on the Ci blocking windows bug | 01:44 |
| wallyworld | thumper: consider yourself asked | 01:44 |
| wallyworld | it can come after the CI blocking bug is fixed | 01:45 |
| davecheney | wallyworld: fyi, i'm wasting^h^h^^h^h^h^h spending all today setting up a windows dev environment | 01:47 |
| davecheney | going back to the mgo bug | 01:48 |
| wallyworld | oh, you have my sympathies :-( | 01:48 |
| wallyworld | davecheney: i'd rather a kick in the bollocks than having to deal with that | 01:48 |
| davecheney | what I see is when we have test failures, they are usually acompanied by the additional debugging in the patch above firing | 01:48 |
| wallyworld | ok. so seems like it would be possible to package up a bug report which illustrates or at least explains the problem | 01:49 |
| wallyworld | in enough detail that it hopefully is uncontentios to fix | 01:50 |
| davecheney | all i can show is the symptoms at the moment | 01:50 |
| davecheney | thumper: state/annotator.go insertOps | 01:53 |
| davecheney | ^ me shakes head | 01:53 |
| davecheney | wallyworld: thumper, do I even want to ask if there is windows support for bzr ? | 02:27 |
| thumper | davecheney: yes | 02:28 |
| thumper | this is | 02:28 |
| davecheney | noice | 02:28 |
| davecheney | hmm, | 02:35 |
| davecheney | that wasn't too painful | 02:35 |
| davecheney | no idea how i'm going to get a version of mongodb with ssl support | 02:35 |
| thumper | aah fuck | 02:57 |
| thumper | wallyworld: you around for a chat? | 02:58 |
| wallyworld | sure | 02:58 |
| thumper | wallyworld: https://plus.google.com/hangouts/_/gsz5xroyjpu4ty7ryxodhbqirya?authuser=1&hl=en | 02:58 |
| davecheney | wallyworld: thumper https://github.com/juju/utils/pull/10 | 03:01 |
| wallyworld | davecheney: +1, i assume you know the windows stuff :-) | 03:02 |
| davecheney | wallyworld: https://github.com/juju/juju/pull/315 | 03:06 |
| wallyworld | davecheney: LGTM | 03:07 |
| davecheney | sinzui: fix is landing for the win32 build failure | 03:10 |
| davecheney | is there a way I can tickle the CI aperatus to trigger a build ? | 03:10 |
| davecheney | omg | 03:12 |
| davecheney | if juju doesn't build from the trunk of juju/utils I will throw a shoe | 03:13 |
| davecheney | hmm | 03:14 |
| * thumper dashing out for a bit | 03:29 | |
| thumper | bbs | 03:29 |
| wallyworld | jam: hiya, no idea why this can't automatically merge, i'll do it by hand i guess, but could you please look at https://github.com/juju/errors/pull/3 | 03:40 |
| jam | wallyworld: because unless mgz set up the jobs only juju/juju is automated by the bot? | 03:40 |
| jam | wallyworld: you can't click the button because it finds a conflict between master and your branch | 03:41 |
| wallyworld | jam: but i hust pushed up my branch just then | 03:41 |
| wallyworld | so i'm not sure what the conflict is | 03:41 |
| wallyworld | i branch off tip of master, add the licence changes, pushed, and create the pr off that branch | 03:42 |
| wallyworld | i can merge by hand i guess | 03:42 |
| jam | wallyworld: so I'm having trouble finding the git syntax to copy just your fix-licenses branch into my local repository, when I figure that out, I'll let you know if I can find the merge issue | 03:45 |
| wallyworld | ok | 03:45 |
| wallyworld | jam: i'm not so worried about the merge issue for now, i can just merge by hand if needed | 03:45 |
| wallyworld | jam: this one can be merged, created the same way https://github.com/juju/errgo/pull/7 | 03:48 |
| jam | wallyworld: $ git merge --no-commit --no-ff 890f25ff011baceede953804330b590cbac89c83 | 03:52 |
| jam | CONFLICT (modify/delete): internals.go deleted in HEAD and modified in 890f25ff011baceede953804330b590cbac89c83. Version 890f25ff011baceede953804330b590cbac89c83 of internals.go left in tree. | 03:52 |
| jam | Auto-merging annotation.go | 03:52 |
| jam | Automatic merge failed; fix conflicts and then commit the result. | 03:52 |
| wallyworld | jam: ok, thanks. i'll just delete that file. not sure why it got left there when I pulled from upstream | 03:54 |
| jam | wallyworld: your 'fix-licenses' version is not from the tip of juju/errors/master | 03:54 |
| jam | wallyworld: looking at the graph it is based off of "also update the readme" | 03:54 |
| jam | which doesn't include "use deep equals" and "merge pull request from howbazaar ..." | 03:55 |
| wallyworld | oh, bollocks. could have sworn i branched off master | 03:55 |
| wallyworld | ok, thanks, will fix | 03:55 |
| jam | wallyworld: you probably branched off your *local* master | 03:55 |
| jam | you have to do | 03:55 |
| jam | git co master | 03:55 |
| jam | git pull origin master | 03:55 |
| jam | git co -b fix-licenses | 03:55 |
| wallyworld | sigh. i hate git | 03:55 |
| wallyworld | jam: fixed now, thank you | 03:58 |
| jam | wallyworld: happy to help, I got to learn a few more git commands, though probably that just means next time I need it I'll be googling again :) | 03:59 |
| wallyworld | lol, yeah i have to do that too | 03:59 |
| wallyworld | it never occurred to me i had branched off an old master | 04:00 |
| wallyworld | jam: here's a really trivial one. after this one, i need to branch the oher sub repos because 1.20 and master revs are different https://github.com/juju/schema/pull/2 | 04:03 |
| jam | wallyworld: as in you want a review? LGTM | 04:08 |
| wallyworld | jam: thanks, i really didn't need a review for that i guess | 04:08 |
| wallyworld | jam: guess who drew the short straw in having to fox the licence balls up? | 04:08 |
| wallyworld | :-( | 04:08 |
| jam | wallyworld: clearly Martin | 04:09 |
| jam | I actually haven't seen mgz/bz2/… around in the last couple of days, is he on vacation? or just swap days after the sprint? | 04:09 |
| wallyworld | just swap days | 04:10 |
| wallyworld | should be back today | 04:10 |
| jam | wallyworld: yeah, I think for the licensing fixes, as long as we know we want LGPL then it is just trivial and you can just merge them. | 04:10 |
| wallyworld | jam: btw, the sub repos are still merged by hand, hence i'm not too worried about desc, title should be self evident | 04:10 |
| tasdomas | is there a way to create a juju environment with a mongo instance that is accessible by the mongo client? | 04:18 |
| tasdomas | I've run into a strange problem where a txn works in tests, but seems to be applied only partially in an ec2 environment | 04:23 |
| jam | tasdomas: the plan is to explicitly restrict that (I think today you can technically get to it, but you have to have the right user/password) | 04:24 |
| jam | tasdomas: if you really need it, I would use SSH and port forwarding | 04:25 |
| tasdomas | jam, what do you mean by ssh and port forwarding? Forward the port for mongo? | 04:28 |
| jam | ssh $MACHINE_0 -L 37017:localhost:37017 and then you connect to your local machine at 37017 | 04:28 |
| jam | mongo localhost:37017/juju | 04:28 |
| jam | you'll need the username and password from machine-0 IIRC | 04:29 |
| wallyworld | davecheney: in utils repo, there's a zfile_windows.go that says it has been automatically generated. was that checked in my mistake do you think? | 04:29 |
| jam | 'machine-0' I believe is the username, but the password is randomly generated and in /var/lib/juju/agents/machine-0/agent.conf | 04:29 |
| tasdomas | jam, thanks | 04:30 |
| jam | tasdomas: I've had hard times getting mongo to play well with authenticated connections | 04:32 |
| jam | you *might* need to connect to the admin db first to login, and then switch | 04:32 |
| tasdomas | jam, the issue I am trying to debug is this: http://paste.ubuntu.com/7801746/ | 04:42 |
| tasdomas | a []txn.Op runs without any apparent errors, but the end result does not match what should actually have happened | 04:43 |
| jam | tasdomas: offhand I don't know what Assert: d- means | 04:44 |
| tasdomas | jam, it's txn.DocMissing | 04:45 |
| jam | tasdomas: so the Insert there has a pointer rather than a detailed struct? | 04:46 |
| jam | IIRC you can't do stuff to objects created earlier in the same transaction (I could be completely wrong) | 04:46 |
| jam | but if the first step was creating the doc, it doesn't exist for you to add the item to its set. | 04:46 |
| jam | tasdomas: but *my* mongo + transactions knowledge is pretty weak. You're better off chatting with fwereade if you and he can overlap in time. | 04:47 |
| tasdomas | jam, hm, thanks for the suggestion | 04:48 |
| jam | wallyworld: have you done any TXN stuff? | 04:48 |
| tasdomas | jam, the weird part is that those ops work in tests and that the whole transaction does not abort | 04:48 |
| wallyworld | jam: in what context? | 04:48 |
| jam | I thought it was you who commented on being able to do stuff later in a transaction to stuff earlier | 04:48 |
| wallyworld | jam: the asserts are evaluated once at the start of the txn | 04:49 |
| wallyworld | so ops that happen laster on can't have asserts that depend on previous ops in that same txn | 04:50 |
| wallyworld | kinda sucks :-( | 04:50 |
| wallyworld | but that's how mgo driver has been written | 04:50 |
| tasdomas | wallyworld, thanks | 04:50 |
| wallyworld | np | 04:50 |
| jam | tasdomas: the asserts I see there are stuff like "value ne dead" | 04:53 |
| jam | which nil != dead, right? | 04:53 |
| tasdomas | jam, I think so | 04:53 |
| jam | tasdomas: anyway, my guess is that the "if the doc doesn't exist insert it" is succeeding, but it isn't letting the "add this value to the set" work | 04:54 |
| jam | so maybe if you change the inserted doc to include the new port? | 04:54 |
| jam | I really don't know | 04:54 |
| jam | but there aren't any asserts there that would fail | 04:54 |
| jam | thus the transaction will succeed | 04:54 |
| tasdomas | jam, yeah - that's what I'm rewriting this to now | 04:54 |
| jam | tasdomas: I would have thought that you could create and then update an object, but maybe the ordering of a transaction isn't really guaranteed, and thus it could try to apply the update before the create. | 04:55 |
| jam | and thus, can't do anything | 04:55 |
| tasdomas | jam, makes sense | 04:55 |
| * thumper groans while studying code... | 04:56 | |
| jam | thumper: don't you groan for pretty much anything? :) | 04:57 |
| thumper | jam: in this particular instance I'm groaning because what I though would be simple, isn't | 04:58 |
| * thumper ungroans | 04:59 | |
| === uru is now known as urulama | ||
| davecheney | wallyworld: no, that is correct | 05:19 |
| wallyworld | ok | 05:19 |
| davecheney | it is generated then comitted | 05:19 |
| wallyworld | thanks | 05:19 |
| * thumper seems to be running around in circles | 05:28 | |
| thumper | ok, stopped running around in circles | 05:46 |
| thumper | moar tests tomorrow | 05:46 |
| thumper | menn0: I'm getting close :) | 05:46 |
| thumper | hmm if dimitern is arriving, time to leave | 05:47 |
| * thumper waves | 05:47 | |
| dimitern | :D | 05:47 |
| jam | morning dimitern | 05:49 |
| dimitern | morning jam | 05:51 |
| dimitern | (which reminds me to have some breakfast :) | 05:51 |
| === vladk|offline is now known as vladk | ||
| === vladk is now known as vladk|offline | ||
| === vladk|offline is now known as vladk | ||
| dimitern | TheMue, jam, vladk, others? review on https://github.com/juju/juju/pull/318 much appreciated - finalizing prefer-ipv6 flag implementation | 07:51 |
| === psivaa-off is now known as psivaa | ||
| * dimitern needs to step out for 1/2 h | 07:52 | |
| jam | dimitern: quid-pro-quo? https://github.com/juju/juju/pull/317 | 07:52 |
| TheMue | morning | 07:53 |
| TheMue | dimitern, jam: will take a look into both | 07:53 |
| TheMue | jam: ping | 07:57 |
| jam | TheMue: yes? | 07:57 |
| jam | morning, btw | 07:57 |
| TheMue | jam: on GH where can I see who is allowed to merge code via $$merge$$? or is everybode allowed? | 08:00 |
| TheMue | jam: otherwise I’ll merge an external provided code (like I’ve done on LP last week) ;) | 08:00 |
| jam | TheMue: AFAIK the people who are listed as part of the "juju" team (with their membership being public) are allowed to vote $$merge$$. | 08:01 |
| jam | TheMue: https://github.com/orgs/juju/members is the member list | 08:01 |
| TheMue | jam: ah, ok, thx, will take a look | 08:02 |
| jam | TheMue: so you mean if you review code, you want to check if the user can vote merge for themselves? | 08:02 |
| jam | that would probably be the list | 08:02 |
| TheMue | jam: yes, exactly. the code is reviewed. the contributor is not on the list, so I’ll merge it | 08:03 |
| TheMue | …oooOOO( After adding a card, learned from last time. *smile* ) | 08:04 |
| Egoist | Hello | 08:20 |
| Egoist | why juju execute the same hook few times one the same unit? | 08:21 |
| Egoist | on* | 08:26 |
| fwereade | Egoist, hi, depends what hook you mean | 08:27 |
| fwereade | Egoist, and when | 08:27 |
| fwereade | Egoist, https://juju.ubuntu.com/docs/authors-charm-hooks.html | 08:28 |
| === vladk is now known as vladk|offline | ||
| === vladk|offline is now known as vladk | ||
| TheMue | jam: you’ve got a review | 08:40 |
| jam | fwereade: hey, good to see you around. | 08:41 |
| jam | fwereade: I was hoping you might be able to find time to help me with cloudsigma reviews now that you're back | 08:42 |
| jam | TheMue: are you seeing user photos for github users being broken and replaced by text strings? Or is it just me? | 08:44 |
| TheMue | jam: hehe, already wanted to ask the same. it’s broken here too. already wondered. | 08:47 |
| jam | TheMue: yeah, I thought maybethey updated something so I went to upload a new image and it tells me "we can't use that image"… I stopped there. | 08:48 |
| jam | dimitern: I reviewed your patch https://github.com/juju/juju/pull/318 | 08:48 |
| TheMue | jam: funnily the organization images are shown, but not the user images | 08:49 |
| === vladk is now known as vladk|offline | ||
| jam | bodie_: ping | 08:52 |
| === vladk|offline is now known as vladk | ||
| === vladk is now known as vladk|offline | ||
| === vladk|offline is now known as vladk | ||
| dimitern | jam, thanks! | 09:02 |
| Egoist | fwereade, i mean -relation-changed, when add more unit to service | 09:03 |
| Egoist | fwereade, it'a a little strange because, hook who is done, starting again on the same unit | 09:03 |
| fwereade | Egoist, relation-changed is always specific to a remote unit | 09:04 |
| fwereade | Egoist, if there are 5 units of service S and 2 units of service T | 09:05 |
| fwereade | Egoist, each unit of T will see a -joined and a -changed for each of the 5 units of S | 09:05 |
| fwereade | Egoist, the $JUJU_REMOTE_UNIT var tells you which one you're responding to | 09:06 |
| Egoist | fwereade, yeah, but when relation-changed is finished on every related unit, it should be over, and while new unit is not added, relation changed should do nothing | 09:09 |
| Egoist | right? | 09:09 |
| fwereade | Egoist, it'll also fire whenever a remote unit writes new settings | 09:10 |
| fwereade | Egoist, if you're writing a relation-changed hook you need to look at what that remote unit has set, and you probably need to respond to it | 09:10 |
| Egoist | ok, get it :) | 09:13 |
| TheMue | ouch, looks like GH is down now | 09:13 |
| Egoist | fwereade, but it's not possible, executing relation-get every ten seconds, and it not stop :/ | 09:15 |
| === vladk is now known as vladk|offline | ||
| fwereade | Egoist, so you're seeing relation-changed firing again and again forever? | 09:16 |
| fwereade | Egoist, can you describe the situation a bit more? | 09:17 |
| fwereade | Egoist, eg just the conversation between two particular units you're having trouble with? | 09:19 |
| Egoist | fwereade, no it happen when i attach another unit to service | 09:20 |
| Egoist | this new unit firing relation-changed again and again, and it won't stop | 09:21 |
| === vladk|offline is now known as vladk | ||
| Egoist | it's hard to describe, in relation-changed i just getting data from relation and config, using this data make change on istalled software on unit, and basically that's it | 09:28 |
| Egoist | fwereade, and relation-changed return true at the end | 09:29 |
| jam | fwereade: or mattyw: as OCR, care for a reasonably trivial review: https://github.com/juju/utils/pull/13 | 09:29 |
| mattyw | jam, why has that file be renamed? | 09:30 |
| jam | mattyw: because _linux is only compiled on linux | 09:30 |
| mattyw | ^^ (not sure what other question I could ask) | 09:30 |
| jam | which isn't darwin | 09:30 |
| jam | (OS X) | 09:30 |
| mattyw | jam, ok | 09:30 |
| jam | or BSD or whatever else for that matter | 09:30 |
| fwereade | Egoist, well it will certainly fire it once for every remote unit, and it may do so more if the settings are changing -- are you seeing it again and again for the same remote unit? or cycling through all the remote units? and are the settings for the relevant units definitely not changing? | 09:31 |
| jam | but with "posix" it should compile everywhere that has Posix semantics, which should be just fine for a 'symlink' thing. | 09:31 |
| mattyw | jam, ah sorry - github wasn't drawing your description for me | 09:31 |
| mattyw | I see it now | 09:31 |
| jam | mattyw: how nice of it | 09:32 |
| jam | thanks mattyw | 09:33 |
| Egoist | fwereade, i make a little change in code, and it stopped, so maybe it was a bug in code | 09:41 |
| fwereade | Egoist, could be -- thanks for letting me know | 09:42 |
| jamespage | dimitern, around? is it possible to have a MAAS environment use an interfaces/bridge other than br0? | 09:43 |
| jamespage | for LXC/KVM instances? | 09:43 |
| dimitern | jamespage, afaik there's a setting you need to tweak for that | 09:44 |
| fwereade | jamespage, the network-bridge config setting might be what you're looking for | 09:44 |
| dimitern | jamespage, fwereade, right | 09:44 |
| jamespage | fwereade, dimitern: that sounds like the one | 09:49 |
| mattyw | davecheney, feel free to land it - or at least try to | 09:51 |
| Egoist | is there any way to get how many units is in service, but not from command line, i need that in code? | 09:53 |
| fwereade | Egoist, not really -- you should expect that the number could change at any time, but that you'll be informed by appropriate hooks running | 09:57 |
| fwereade | Egoist, what are you trying to do? | 09:58 |
| Egoist | fwereade, because in code i use relation-list, and it not alway listed all units, sometimes one is missing | 10:17 |
| fwereade | Egoist, it'll tell you about the units you're expected to know about based on what hooks have run | 10:17 |
| fwereade | Egoist, if you haven't seen the -joined hook for unit X yet, you won't see in in relation-list | 10:17 |
| fwereade | wallyworld, sorry I'm out of touch: do we have an ETA for in-environment storage? I got the impression it was nearly done, but not quite | 10:19 |
| wallyworld | fwereade: the api is done and functional and in the blobstorage repo; the refactoring work to change the juju code to store tools and charms etc is yet to be done | 10:20 |
| wallyworld | fwereade: there's ongoing debate about the charm storage side of things that needs to be sorted out also | 10:21 |
| fwereade | wallyworld, so I guess we can't yet fix it for manual/local/cloudsigma providers | 10:22 |
| wallyworld | fwereade: we can - that's on the immediate todo list. we've already started the Environ api changes and there's a wip branch to internalise the Strorage() facility | 10:22 |
| wallyworld | fwereade: significant refactoring was done at the sprint; the StateInfo() api is almost gone; replaced by a StateServerInstances() api | 10:23 |
| fwereade | wallyworld, ooh! can I direct you at https://github.com/juju/juju/pull/174/files please? just to inform them what needs to be done (or rather doesn't need to be done any more?)? | 10:24 |
| wallyworld | looking | 10:24 |
| fwereade | wallyworld, if Storage() can be internalised, I guess it's safe to return a null storage now? | 10:24 |
| wallyworld | fwereade: Storage() on ENviron will be gone. but it's still needed for now for tools and charms | 10:25 |
| wallyworld | we're not quite there yet | 10:25 |
| wallyworld | fwereade: the stuff in that pr above will be needed until we do more work to remove the need for storage. not quite there yet but making good progress | 10:28 |
| fwereade | wallyworld, ok, cool | 10:33 |
| fwereade | wallyworld, would you give them a heads up all the same please | 10:33 |
| wallyworld | fwereade: sure,who is "them". do i just talk to nate? | 10:34 |
| jam | wallyworld: "them" is Altoros who are writing the CloudSigma provider which has to use the "manual" method for storage because CloudSigma itself doesn't provide a storage solution | 10:38 |
| wallyworld | jam: ok, thanks. sadly we are "out of sync" by a few of weeks. i reckon we'll be in a good place to be able to ditch mandatory provider storage by that time | 10:39 |
| * jam clearly thinks "air quotes" are the way of the future | 10:39 | |
| wallyworld | but we need to sync with the charm store guys on charm storager so it's not entirely under out control | 10:40 |
| wallyworld | seems like a waste to commit code for only a short time but i don't hink we can hold off on the cloud sigma stuff for that long | 10:40 |
| wallyworld | natefinch: can i leave it in your hands to deal with the gojsonschema licence issue? whether it's moved to a different repo, or just the LICENSE file added, or whatever | 10:50 |
| jam1 | fwereade: maybe you remember best why gojsonschema wasn't merged into the github.com/juju namespace? Was it just evolving fast enough that they didn't want to do forced code review since they aren't directly in that group? | 10:51 |
| dimitern | jam1, standup? | 10:51 |
| jam1 | dimitern: thanks, brt | 10:52 |
| fwereade | jam1, I'm sorry, I don't immediately recall -- mgz was more directly involved in the details at that stage iirc, and if not bodie_ will be able to say when he comes one | 10:52 |
| natefinch | wallyworld: sure | 10:53 |
| wallyworld | \o/ | 10:53 |
| wallyworld | natefinch: i asked because i thought you already had a relatioship with the dev(s) concerned | 10:54 |
| natefinch | wallyworld: I'm more than happy to do it. I'm friendly with them, but otherwise no professional relationship, FYI. | 10:55 |
| natefinch | am I the only one who gets duplicate copies of rogpeppe's mailing list posts? | 10:56 |
| wallyworld | i do sometimes i think | 10:56 |
| rogpeppe | natefinch: i send another copy when i find i've sent from the wrong Sender | 10:56 |
| natefinch | rogpeppe: Oh, I see | 10:57 |
| rogpeppe | natefinch: because (i believe) juju-dev and canonical-juju will reject posts unless they're from roger.peppe@canonical.com | 10:57 |
| natefinch | rogpeppe: right, that makes sense. I never noticed they were from different senders. | 10:57 |
| rogpeppe | natefinch: unfortunately I haven't found a way to force gmail to always choose a particular sender address based on the destination address | 10:57 |
| rogpeppe | natefinch: and gmail doesn't show the sender by default, so i often forget to change it | 10:58 |
| natefinch | rogpeppe: I just have separate inboxes, so it's not a problem | 10:58 |
| natefinch | but I can understand wanting a single inbox for everything, too | 10:58 |
| natefinch | man, setting CDPATH=$GOPATH/src is frigging amazing | 10:59 |
| natefinch | from anywhere I can do cd github.com/juju/juju and it goes to the right place | 11:01 |
| dimitern | jam1, TheMue, vladk, my g+ froze and now I can't go back :/ | 11:07 |
| jam1 | morning mgz | 11:08 |
| mgz | hey jam | 11:08 |
| natefinch | int54 is evidently not a defined type | 11:10 |
| TheMue | jam: your problem seems to be in apiserver/client_test.go | 11:30 |
| jam | TheMue: actually it is state/apiserver/client/api_test.go | 11:31 |
| jam | but there may be others | 11:31 |
| TheMue | jam: gna, but ok, it’s no small change | 11:32 |
| voidspace | morning all | 11:36 |
| dimitern | so does it matter what you put between the $$ merge markers $$ ? | 11:44 |
| natefinch | dimitern: it's a regex that is $$\w$$ so, no spaces | 11:45 |
| dimitern | natefinch, right! good to know, thanks :) | 11:45 |
| perrito666 | morning | 11:45 |
| dimitern | morning perrito666, voidspace | 11:45 |
| natefinch | dimitern: we changed it at the sprint last week, for $$lotsOfFunTimes$$ | 11:46 |
| voidspace | I thought it was \s+ | 11:46 |
| dimitern | $$exactly$$ | 11:46 |
| voidspace | dimitern: perrito666: o/ morning | 11:46 |
| natefinch | voidspace: isn't \s whitespace? | 11:46 |
| mgz | I wonder if I should actually merge the change to make it a re match, it's potentially useful for less silly things | 11:47 |
| voidspace | ah, maybe \S+ then | 11:47 |
| === psivaa is now known as psivaa-lunch | ||
| katco | mgz: i have to feed my pets, but then do you want to chat about the unit tests? | 12:40 |
| mgz | sure thing | 12:42 |
| natefinch | wwitzel3: I have to move our 1 on 1 for later today | 12:54 |
| sinzui | dimitern, is bug 1261780 still an issue now that we use golanf 1.2? | 12:55 |
| _mup_ | Bug #1261780: go 1.1.2 TLS-enabled client does not accept our CACert <security> <tech-debt> <juju-core:Triaged> <https://launchpad.net/bugs/1261780> | 12:55 |
| dimitern | sinzui, i'll take a look | 13:02 |
| wwitzel3 | natefinch: np | 13:06 |
| dimitern | sinzui, it seems that should be fixed in go 1.2 | 13:17 |
| dimitern | sinzui, had to check a few forums to make sure | 13:17 |
| sinzui | thank you dimitern | 13:17 |
| sinzui | natefinch, can you help arrange a fix for bug 1342725 | 13:40 |
| _mup_ | Bug #1342725: C:/Juju/lib/juju/nonce.txt does not exist, bootstrap failed in win <ci> <regression> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1342725> | 13:40 |
| sinzui | dimitern, fwereade , jam : can I be upgrades in this channel to update the topic which is months old | 14:02 |
| sinzui | s/upgrades/upgraded AKA moderator? | 14:02 |
| dimitern | sinzui, it's done via chanserv | 14:03 |
| dimitern | sinzui, and I think we all should be able to do it | 14:04 |
| sinzui | not yet: #juju-dev :You're not a channel operato | 14:04 |
| === Ursinha is now known as Ursinha-afk | ||
| dimitern | sinzui, try /msg ChanServ TOPIC #juju-dev <text> | 14:05 |
| stokachu | we're seeing an issue where people are attempting to use juju bootstrap with kvm and it fails b/c they do have /usr/sbin in their path | 14:06 |
| stokachu | this line: command := exec.Command("kvm-ok") | 14:06 |
| stokachu | could that possibly be changed to the full path of kvm-ok? | 14:06 |
| dimitern | stokachu, please file a bug about it | 14:07 |
| sinzui | stokachu, please report that to a bug and I will target it to 1.20.2 | 14:07 |
| stokachu | i will was just curious on your thoughts | 14:07 |
| dimitern | stokachu, I think these commands like kvm-ok are supposed to be in the $PATH, and there are some checks about it in tests | 14:08 |
| natefinch | sinzui: I can help with that | 14:09 |
| stokachu | dimitern, the only test i see related is it checking if the host os is ubuntu | 14:10 |
| === ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Bugs: 1 Critical, 147 High - https://bugs.launchpad.net/juju-core/ | ||
| stokachu | sinzui, https://bugs.launchpad.net/juju-core/+bug/1342747 | 14:14 |
| _mup_ | Bug #1342747: juju bootstrap fails if kvm-ok not in path <cloud-installer> <juju-core:New> <https://launchpad.net/bugs/1342747> | 14:14 |
| sinzui | thank you stokachu | 14:15 |
| stokachu | np :D | 14:16 |
| lazyPower | I have a user in #juju asking what ports he needs to keep open in UFW to use juju. are there any other ports than the WSAPI port that he would need to open? | 14:19 |
| natefinch | lazyPower: that should be it, though he might want 22 open for SSH access | 14:20 |
| lazyPower | the wssapi port is 27017 by default right? | 14:20 |
| natefinch | lazyPower: that's the default mongo port | 14:21 |
| natefinch | lazyPower: hang on, I'll find it, I forget | 14:21 |
| lazyPower | haha yeah, i thought we used the mongodb port | 14:21 |
| * lazyPower has mongo on the brain | 14:21 | |
| natefinch | lazyPower: used to be we went straight to mongo, we've stopped doing that and it's all through the API now | 14:22 |
| dimitern | lazyPower, it's 17017 for the api and 37017 for mongo, but the latter shouldn't be needed | 14:22 |
| lazyPower | i like that we did that. It'll be easier to replace mongo if we ever do that. | 14:22 |
| dimitern | lazyPower, so 17017 + 22 should be opened | 14:23 |
| lazyPower | dimitern: thanks *hattip* | 14:24 |
| dimitern | lazyPower, depending on the provider, if httpstorage is used you need to open its port as well (there are environ settings for some of those) | 14:24 |
| dimitern | lazyPower, np | 14:24 |
| === Ursinha-afk is now known as Ursinha | ||
| === psivaa-lunch is now known as psivaa | ||
| natefinch | arg..... the fix for the windows client is going to be annoying. it's one of those things where we used to just assume everything was ubuntu, but now we have *some* windows code, and that's screwing us up. And so we have to change the signature of functions to pass in a series | 14:53 |
| perrito666 | natefinch: yup, my patch, Which I answered in the thread about that same issue does a very tiny fix and touches like 20 files | 14:56 |
| alexisb | TheMue, you keep joining and dropping :) | 15:01 |
| TheMue | alexisb: yeah, it shows me an error and says I shall retry later :( | 15:01 |
| TheMue | alexisb: and so I retry :) | 15:01 |
| alexisb | TheMue, do you have topics for today? | 15:02 |
| TheMue | alexisb: nothing special. feeling good and currently investigating in LXC and IPv6 | 15:02 |
| alexisb | TheMue, alright, I can give you 30 minutes back, I know I could use them this week :) | 15:03 |
| TheMue | alexisb: fine, then we’ll see next time again. ;) | 15:04 |
| perrito666 | natefinch: stand up? | 15:06 |
| voidspace | ok, so not such a successful test run - 22509 lines of output... | 15:07 |
| voidspace | wallyworld: ping | 15:10 |
| voidspace | natefinch: ping | 15:21 |
| alexisb | jam1, ping | 15:55 |
| alexisb | jam ping | 15:58 |
| jam1 | alexisb: pong | 16:31 |
| jam1 | I've been trying to connect to the hangout, but I keep getting "could not start video because of an error" | 16:31 |
| natefinch | jam1: is there an easy way to see who has signed the canonical CLA? | 16:32 |
| alexisb | jam1, yeah the hangouts seem to be acting all funny today | 16:33 |
| alexisb | john and I are on | 16:33 |
| alexisb | but no one else | 16:33 |
| jam1 | natefinch: there is a launchpad group, just a sec | 16:34 |
| jam1 | natefinch: https://launchpad.net/~contributor-agreement-canonical | 16:34 |
| alexisb | lol jam1 everyone keeps joining and dropping | 16:36 |
| alexisb | just John and i on | 16:36 |
| voidspace | my internet still sucks | 16:36 |
| voidspace | I'll sort it out properly on my return from Romania | 16:36 |
| voidspace | I do at least have a phone I can tether as a backup now | 16:37 |
| voidspace | I have 2.6Mbit downstream and that's the best I've had for over a month! | 16:37 |
| voidspace | alexisb: enjoy your holiday. I didn't think managers were *allowed* to go away for ten days ;-) | 16:37 |
| alexisb | :) | 16:39 |
| alexisb | katco, fyi, got asked for status on this bug: https://bugs.launchpad.net/juju-core/+bug/1319475 | 16:40 |
| _mup_ | Bug #1319475: Juju should support new signing format <ec2-provider> <goamz:New> <juju-core:Triaged by cox-katherine-e> <https://launchpad.net/bugs/1319475> | 16:40 |
| === vladk is now known as vladk|offline | ||
| alexisb | if you have any updates might be good to add a comment | 16:40 |
| katco | alexisb: right, sorry i will do so. short of it is: we're waiting on a new version of goamz from gustavo | 16:41 |
| mfoord | natefinch: ping | 16:41 |
| natefinch | mfoord: yo | 16:41 |
| niemeyer | katco, alexisb: Oh? | 16:41 |
| alexisb | katco, cool, we just want to communicate that in the bug so I have a place to pull status | 16:41 |
| katco | alexisb: will do | 16:42 |
| niemeyer | katco, alexisb: I'm not on the hook to provide a new version of goamz.. | 16:42 |
| niemeyer | katco, alexisb: .. that I know of | 16:42 |
| katco | niemeyer: yes, sorry, it's been bandied about a bit. ian sent another email and told me to shelve it for now | 16:42 |
| niemeyer | katco: Ok, so saying you're waiting on me for it is not quite correct | 16:43 |
| katco | niemeyer: i apologize. i was told to shelve this for now pending a change to goamz, and how that will occur is up in the air as i understand it. | 16:43 |
| katco | niemeyer: hopefully that makes more sense | 16:44 |
| alexisb | who owns goamz? | 16:44 |
| niemeyer | katco, alexisb: I really have no idea about the internal communication that took place around the issue. On my side I was asked to "merge a fork" and then I asked what the fork comprises and what do we want from it, which was not answered.. so I really have no intention or means of moving forward. The ball is on someone else's side at this point. | 16:45 |
| katco | alexisb: it's a canonical library, but there are several community forks with the support we need for v4 already added | 16:45 |
| alexisb | niemeyer, can you please send me the thread you are referring to so I can follow-up | 16:46 |
| alexisb | katco, thanks for the info | 16:46 |
| katco | niemeyer: i'll take an action item to have ian follow up. sorry for any confusion | 16:46 |
| niemeyer | alexisb: Done | 16:51 |
| alexisb | niemeyer, katco thank you! | 16:51 |
| niemeyer | alexisb, katco: I see there's a mail from Ian asking me a bunch of questions on the topic from yesterday too.. I'll answer that with CC | 16:54 |
| alexisb | niemeyer, thank you | 16:54 |
| katco | niemeyer: tyvm gustavo. i think the only additional context that's missing from that thread is that he asked me to shelve the analysis of the forks | 16:55 |
| katco | niemeyer: so perhaps a bit of confusion there. sorry about that. | 16:55 |
| niemeyer | katco, alexisb: No problem | 17:13 |
| mfoord | hmmm... so it's at least possible that some of these problems are due to JujuConnSuite State/BackingState issues | 17:22 |
| natefinch | fwereade: the windows bug we have currently is due to using "current OS" when we should have used "target OS". Really, it's because we used "we assume there's only one OS and that's Ubuntu" instead of "Let's actually determine what OS we're targetting" | 17:25 |
| jam1 | natefinch: that would also solve running the test suite on Mac OSX, which would probably be easier to start with | 17:29 |
| jam1 | I tried a couple of times, but for right now I just hack osversion_darwin to return "trusty" and most of the test suite passes. | 17:29 |
| voidspace | jam1: ping | 17:29 |
| jam1 | voidspace: ? | 17:29 |
| natefinch | jam1: heh, interesting | 17:29 |
| voidspace | jam1: can you explain to me why the code is correct as written and my diff makes it wrong | 17:29 |
| voidspace | jam1: http://pastebin.ubuntu.com/7804510/ | 17:29 |
| natefinch | jam1: OSX would be easier to start with if someone's willing to buy me a Macbook :) | 17:30 |
| voidspace | jam1: why do we set the password on the connection info to a hash of password and then change the password to the non-hashed version | 17:30 |
| voidspace | note that my fix causes tests to pass when I copy sessions | 17:30 |
| jam1 | voidspace: checking | 17:30 |
| voidspace | thanks | 17:30 |
| natefinch | I remember something about that hacky code where some of it is a hash and some is not. | 17:31 |
| jam1 | voidspace: so the hash of the password is because we originalyl passed in the passwords via cloud-init | 17:31 |
| jam1 | and that leaks the admin secret to everyoen | 17:31 |
| jam1 | so we leak only the hash of the password | 17:31 |
| jam1 | and on the first connect we switch back to the "real" password. | 17:31 |
| voidspace | jam1: this is in testing though | 17:31 |
| jam1 | however, since axw's patch a long time ago | 17:31 |
| jam1 | we don't do that | 17:31 |
| jam1 | voidspace: but *bootstrap* does that | 17:31 |
| jam1 | or did | 17:31 |
| jam1 | we now set up mongo via "SSH" into the machine instead of cloud-init. | 17:32 |
| voidspace | so we connect successfully with the hash (which is what we used originally) | 17:32 |
| voidspace | and then we change the password to the real one | 17:32 |
| voidspace | that was the idea | 17:32 |
| voidspace | so we leaked the hash and then changed to the real password | 17:32 |
| voidspace | but as this is in testing, my fix doesn't matter - it just changes us to use a consistent password | 17:33 |
| voidspace | which happens to be a hash | 17:33 |
| voidspace | note that in the test connecting with the hash still worked | 17:33 |
| voidspace | so something in this setup is still using the hash | 17:33 |
| jam1 | voidspace: so you probably need to look into testing/mgo.go I would think | 17:33 |
| voidspace | ok | 17:34 |
| voidspace | jam1: cool | 17:34 |
| voidspace | thanks for the help | 17:34 |
| voidspace | wel.l, that change fixed all the apiserver tests | 17:34 |
| voidspace | so it was definitely a big part of my problem | 17:34 |
| voidspace | time to go jogging and do a full test run | 17:35 |
| voidspace | it *might* be finished by the time I get back ;-) | 17:35 |
| jam1 | voidspace: "resetAdminPasswordAndFetchDBNames" looks suspicious | 17:35 |
| voidspace | I'll take a look | 17:35 |
| voidspace | thanks | 17:35 |
| jam1 | voidspace: it looks like it is doing "admin.RemoveUser("admin") as part of resetting the db, I don't quite understand how removing the admin user allows us to use the database properly afterward | 17:38 |
| jam1 | voidspace: It might be the "mongo lets us login as anyone if noone is configured" ? | 17:38 |
| natefinch | I believe that's true | 17:39 |
| jam1 | natefinch: // We try for a while because we might succeed in | 17:39 |
| jam1 | // connecting to mongo before the state has been | 17:39 |
| jam1 | // initialized and the initial password set. | 17:39 |
| jam1 | suspicious | 17:39 |
| jam1 | voidspace: so the issue would appear that we are successfully logging in (and mgo is caching) the original hashed password, but then we immediately set the password to the non hashed version, but we don't update the mgo.Session | 17:40 |
| jam1 | that sounds like an update we potentially need to do for real connections as wel | 17:41 |
| perrito666 | jam1: runs mongod with --noauth and creates a new admin iirc | 17:41 |
| jam1 | perrito666: we do that for real mongo (I think), but for the test suite? | 17:41 |
| voidspace | jam1: yep, that's the same problem we had in state.Open too which I had to fix | 17:41 |
| voidspace | jam1: making the fix in that test code causes *almost* all tests to pass with some of the watcher infrastructure doing session copying | 17:42 |
| jam1 | voidspace: JujuConnSuite.tearDownConn | 17:42 |
| perrito666 | jam1: I am reading the same tests for other stuff and it implies so, but I am not sure either | 17:42 |
| voidspace | not quite all the tests though | 17:42 |
| voidspace | jam1: I'll come back to this in a bit - but this is great | 17:42 |
| jam1 | if err := s.State.SetAdminMongoPassword(""); err != nil && serverAlive { | 17:42 |
| jam1 | voidspace: fwiw I can't actually find a place where we set the password properly to juju/testing/DefaultMongoPassword | 17:47 |
| jam1 | I see places where we appear to get rid of that password | 17:47 |
| jam1 | voidspace: it is probably the dummy environment | 17:48 |
| jam1 | as we have "juju/provider/dummy/environs.go: admin-secret: testing.defaultMongoPassword | 17:48 |
| jam1 | voidspace: and there (I believe) it is. | 17:49 |
| jam1 | juju/provider/dummy/environs.go | 17:49 |
| jam1 | line 654 | 17:49 |
| jam1 | Vif err := st.SetAdminMongoPassword(utils.UserPasswordHash(password, utils.CompatSalt)); err != nil { | 17:49 |
| jam1 | voidspace: I'm betting if you change that, you can change the testing/conn.go | 17:50 |
| jam1 | voidspace: you'll want to confirm with axw, but I'm 95% sure that "juju bootstrap" no longer uses the hash of the password in real world scenarios | 17:50 |
| voidspace | jam1: thanks | 18:06 |
| voidspace | appreciated | 18:06 |
| === vladk|offline is now known as vladk | ||
| katco | i'm having trouble getting gocheck to filter down to a single test; i'm executing "go test -gocheck.f ".*" github.com/juju/juju/cmd/juju/..." but it doesn't execute any tests. what am i doing wrong? | 18:48 |
| ericsnow | katco: doesn't gocheck require that you be in the directory where the tests are? | 18:49 |
| katco | ericsnow: maybe that's why it's not working, but i thought it was just a wrapper around go test and so you could be anywhere | 18:50 |
| ericsnow | katco: I haven't looked into it much but I ran into the same thing a while back | 18:51 |
| katco | ericsnow: the bootstrap tests are kind of slow, so i wanted to tighten up my testing cycles. let me try cding into the dir | 18:51 |
| katco | ericsnow: well, it seems that's working. ty sir :) | 18:52 |
| ericsnow | katco: np | 18:53 |
| hazmat | are there things within a state server machine agent that connect back to itself on the api? | 19:09 |
| bodie_ | https://github.com/juju/juju/pull/311 should be good to go btw | 19:14 |
| hazmat | anyone can help me diagnose a user issue.. post upgrade to 1.20.1 there env is basically broken | 19:19 |
| hazmat | http://pastebin.ubuntu.com/7804968/ | 19:19 |
| hazmat | afaics.. the underlying issue looks like some sort of timing issue .. juju starts mongodb up.. and the api worker tries to connect to it, succeeds, but does so after it triggers a timeout, which results in the api worker restarting | 19:19 |
| hazmat | although perhaps that normal, and subsequent connects are expected to succeed.. the subsequent api server is dying because it can't connect is a bit more relevant | 19:21 |
| natefinch | hazmat: I think that's normal | 19:26 |
| natefinch | bodie_: have you heard that we need a license on your go json schema code? | 19:26 |
| bodie_ | yes | 19:26 |
| natefinch | bodie_: cool | 19:26 |
| bodie_ | :) | 19:26 |
| bodie_ | thanks for checking | 19:27 |
| hazmat | natefinch, actually it does appear to that timing issue re connect from the subsequent restarts.. ie the last 100 lines of the machine-0.log is constant api server restarts and mongodb reconnects .. http://pastebin.ubuntu.com/7805073/ | 19:27 |
| === alexisb is now known as alexisb_lunch | ||
| natefinch | hazmat: is that a local provider environment? | 19:37 |
| hazmat | natefinch, it is | 19:40 |
| hazmat | natefinch, its hackedbellini's env | 19:43 |
| natefinch | hazmat, hackedbellini: can you dump the full logs? Not really sure what could cause that. | 19:46 |
| hackedbellini | natefinch: here is the log from just after the agent is restarted: http://pastebin.ubuntu.com/7804968/ | 19:47 |
| hackedbellini | it continues spamming this after that: http://pastebin.ubuntu.com/7805073/ | 19:47 |
| hackedbellini | fyi, our env (using lxc) was deployed on 1.16 (with sudo juju bootstrap at that time), we upgraded then to 1.18 and to 1.19 because of a bug (https://bugs.launchpad.net/juju-core/+bug/1325034) | 19:50 |
| _mup_ | Bug #1325034: juju upgrade-juju on 1.18.3 upgraded my agents to 1.19.2 <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1325034> | 19:50 |
| hackedbellini | today we tried to move back to stable (1.20.1) and that happened just after doing a juju upgrade-juju (both juju/juju-core packages are updated to 1.20.1 from the stable ppa) | 19:50 |
| natefinch | so this was an environment deployed with sudo juju? | 19:50 |
| natefinch | one thing I see notably absent is the log lines about initiating the mongo replicaset | 19:51 |
| hackedbellini | natefinch: yes it was deployed with sudo. But it runs with the user "juju" now | 19:52 |
| hackedbellini | I'll post the service files for you to see | 19:52 |
| hackedbellini | here: http://pastebin.ubuntu.com/7805056/ | 19:52 |
| hackedbellini | those are the service files that start juju-db and juju-agent | 19:53 |
| natefinch | hackedbellini: so that looks correct | 20:06 |
| natefinch | actually, I withdraw the comment about initiating.... forgot this was an existing environment, not bootstrap.... so used to these problems coming right at bootstrap time. | 20:11 |
| hackedbellini | natefinch: hahaha I see =P | 20:11 |
| hackedbellini | natefinch: is there any info you need, or anything you want me to try to do here? Just ask. Atm, my juju is totally broken, because of that issue, I can't even do a juju status :( | 20:12 |
| natefinch | yeah, juju status uses the API :/ | 20:12 |
| natefinch | hang on | 20:16 |
| hackedbellini | np | 20:16 |
| natefinch | hackedbellini: could you dump your whole machine-0 log for me? Or at least the stuff from just before you upgraded? | 20:28 |
| hackedbellini | natefinch: sure, just a minute =P | 20:29 |
| hackedbellini | natefinch: its sending the paste... I'll pm it to you | 20:33 |
| natefinch | cool | 20:33 |
| hackedbellini | natefinch: just sent you the log | 20:34 |
| natefinch | hackedbellini: looking | 20:35 |
| natefinch | I like the way ubuntu pastbin makes me log in to download the text I can see in my damn browser :/ | 20:38 |
| hackedbellini | natefinch: hahaha. I'll send you a download link directly from our server here so you can get it using wget if you perfer | 20:39 |
| === vladk is now known as vladk|offline | ||
| natefinch | hackedbellini: I have it, it's ok | 20:39 |
| hackedbellini | natefinch: I don't remember the exactly time I did the "juju upgrade-juju", but from the log the errorr started at 17:50 (it 14:50 here, the log is in utc =P) | 20:40 |
| hackedbellini | it was* | 20:40 |
| hackedbellini | probably I did the juju upgrade-juju a little before that | 20:41 |
| natefinch | hackedbellini: we should put in some giant ****** LOOK OUT! HERE COMES AN UPGRADE!! ****** log message | 20:42 |
| hackedbellini | natefinch: hahahaha yeah, it for sure would be very useful! :) | 20:43 |
| natefinch | instead all we get is "upgrade requested from 1.19.3-precise-amd64 to 1.20.1" | 20:44 |
| hackedbellini | natefinch: so indeed I started the upgrade at, surprisingly, 17:50:00 utc. | 20:45 |
| natefinch | hackedbellini: I saw that. Nice timing | 20:45 |
| hackedbellini | hahahah | 20:45 |
| natefinch | thumper: hackedbellini upgraded his local environment from 1.19.3 to 1.20.1 (he'd previously upgraded from 1.18.3 to 1.19 through a bug). And now his API won't start up. | 20:46 |
| natefinch | hackedbellini, thumper: so, previously, connecting to the API, we dialed "wss://192.168.99.5:17070/" and now we're dialing "wss://localhost:17070/ | 20:52 |
| natefinch | I remember we did change this code so that we always dial localhost on state machines, since they should always talk to their own APIs (otherwise they might connect to the API on another HA server) | 20:54 |
| natefinch | hackedbellini: is there anything unusual about the iptables on that machine? | 20:54 |
| hackedbellini | natefinch: could this be the issue? One thing we noticied is that there's noone listening on 17070 (netstat -nl | grep 17070 returns nothing) | 20:55 |
| hackedbellini | natefinch: what do you mean by unusual? | 20:55 |
| natefinch | hackedbellini: just wondering if maybe using the IP before was getting around some firewall issue... but if nothing's listening, obviously that a problem | 20:56 |
| natefinch | hackedbellini: I presume if you do 'ps -Al | grep jujud' you have a jujud running? | 20:58 |
| natefinch | thumper, hackedbellini: I have to run in a couple minutes, unfortunately. Previous engagement. | 20:58 |
| hackedbellini | natefinch: hrm, I don't know because I'm not the one who manages the network here (I just maintain juju at a user level without sudo privileges =P) | 20:59 |
| hackedbellini | when talking to hazmat he said something about that problem... he said that maybe it was a timing issue... that the same process (the agent) should listen and then connect to 17070, but it was not happening | 20:59 |
| hackedbellini | natefinch: not at the moment. A workmate here turned juju agent down because of the spamming on the log. But yes, when the agent is running, I see juju on ps. Actually, I can see the jujud from the lxc machines when running that atm, but they are all spamming that wss issue | 21:00 |
| natefinch | hackedbellini: I gotta run. hopefully thumper or wallyworld can help you when they fully wake up. | 21:01 |
| hackedbellini | natefinch: hahaha np. Thank you for you help so far! I'll not be here soon too (it's 18h in Brazil and soon I'll go home) so maybe I'll leave before they, like you said, fully wake up =P, but I'll be here tomorrow again at 15h utc | 21:03 |
| natefinch | hackedbellini: ok, we'll do whatever we can to help out. I'm an hour behind you, so it's 17:00 here. | 21:04 |
| hackedbellini | natefinch: thank you so much for that :) | 21:05 |
| natefinch | hackedbellini: np | 21:05 |
| * thumper is reading backlog | 21:05 | |
| thumper | hackedbellini: how do you maintain juju without sudo? it needs sudo for lxc | 21:06 |
| hackedbellini | thumper: well, when I need to run something with sudo, I ask someone with sudo powers to do that for me | 21:07 |
| hackedbellini | but usually I just "juju ssh <machine/service>" when I need to go inside an lxc | 21:07 |
| thumper | hackedbellini: ok, with the local provider the machine agent for machine 0 runs on the host with root privleges | 21:07 |
| cmars | thumper, there's also unprivileged containers, https://www.stgraber.org/2014/01/17/lxc-1-0-unprivileged-containers/ | 21:08 |
| thumper | also, for some reason I'm yet to fathom, the logs were changed to be 0600 owned by the syslog user | 21:08 |
| thumper | cmars: I know | 21:08 |
| thumper | cmars: but they weren't done in time for us to use it properly | 21:09 |
| thumper | cmars: I have plans, but no time | 21:09 |
| thumper | hackedbellini: so you are unlikely to be able to look at the logs without sudo | 21:09 |
| hackedbellini | thumper: the logs I can see with no problem | 21:10 |
| thumper | hackedbellini: so far :-) | 21:10 |
| thumper | hackedbellini: it was changed recently | 21:10 |
| hackedbellini | I can't sometimes edit a configuration file (like the agent.conf of the machine-0 =P) | 21:10 |
| thumper | hackedbellini: first thing I'd check is that the juju mongo db service is running | 21:11 |
| thumper | hackedbellini: then that the machine agent is running | 21:11 |
| thumper | hackedbellini: then check the logs of the machine agent | 21:11 |
| hackedbellini | thumper: also, about that sudo issue... we initially did the initial bootstrap using sudo (as it was a requirement at the time) | 21:11 |
| hackedbellini | now the configuration here (https://juju.ubuntu.com/docs/config-LXC.html) says that it doesn't need it anymore, that's the reason we changed the user running the agent to juju and some permissions (although some files are still owned by root) | 21:11 |
| thumper | hackedbellini: sudo is still needed for bootstrap, but not explicitly | 21:12 |
| thumper | we now ask during bootstrap | 21:12 |
| thumper | hackedbellini: yeah, if you edited the upstart script, it will fail | 21:13 |
| thumper | hackedbellini: juju expects the agents to run as root | 21:13 |
| thumper | we've talked about this before, but all hooks expect root | 21:13 |
| hackedbellini | thumper: mongodb is running. The machine agent is running (not atm because a workmate stopped it because it was spamming the 17070 port error), but it runs | 21:13 |
| hackedbellini | I have a log from a very early hour today, it contains the log from the upgrade and all the spamming after. Do you want me to pm it to you? | 21:13 |
| thumper | hackedbellini: you could pastebin it | 21:13 |
| hackedbellini | thumper: hrmmm, so this config is wrong? http://pastebin.ubuntu.com/7805056/ | 21:14 |
| thumper | hackedbellini: also, what did you change? | 21:14 |
| hackedbellini | thumper: sure, I'll pm you | 21:14 |
| thumper | hackedbellini: yes, that is wrong | 21:14 |
| thumper | hackedbellini: I recall seeing something like this before when the agent was trying to update mongo to be a replica set | 21:23 |
| thumper | hackedbellini: there was an early race condition | 21:23 |
| thumper | hackedbellini: where one part of the code thought it was done, but it wasn't | 21:23 |
| thumper | wwitzel3: did you do some of the replica set stuff? | 21:24 |
| thumper | I'm not familiar with it | 21:24 |
| hackedbellini | thumper: hrmmm, I see. Anything I can do to try to workaround that? | 21:24 |
| thumper | probably, I just don't know what it is | 21:24 |
| hackedbellini | thumper: the only person here with sudo powers is going home now... is there anything you want me to test that requires sudo? Once he is gone, I can continue the tests just tomorrow, unfortunately :( | 21:38 |
| thumper | hackedbellini: I've been discussing the issue | 21:39 |
| thumper | menn0 and I think we may know what happened | 21:39 |
| thumper | during the 1.19 cycle, HA was added | 21:39 |
| thumper | but only initializes HA when upgrading from 1.18 -> 1.20 | 21:39 |
| thumper | as upgrades from dev -> prod have never been formally supported | 21:40 |
| thumper | we *think* we may be able to trick the code into thinking it was upgrading from 1.18 | 21:40 |
| thumper | so it initializes mongo properly | 21:40 |
| * thumper quickly reads code | 21:40 | |
| hackedbellini | thumper: hrmmm, very nice!!! | 21:41 |
| hackedbellini | I asked my workmate to wait a little | 21:41 |
| thumper | hackedbellini: ok, this is a little hacky | 21:42 |
| thumper | hackedbellini: are you ready? | 21:42 |
| hackedbellini | thumper: ahhh, he just left... he said he was in a very hurry :( | 21:42 |
| hackedbellini | but tell me what it is | 21:42 |
| thumper | ok | 21:43 |
| hackedbellini | I'll see if I can do, and if I can't, I'll write it down and ask him to do it tomorrow | 21:43 |
| hackedbellini | hahaha and no problem if it | 21:43 |
| hackedbellini | it's a little hacky =P | 21:43 |
| thumper | you need to edit the machine agent conf file | 21:45 |
| thumper | hackedbellini: based on your config, it should be here: /home/juju/.juju/local/agents/machine-0/agent.conf | 21:45 |
| hackedbellini | yes, it's there indeed | 21:46 |
| thumper | hackedbellini: you need to change the line that has: upgradedToVersion: | 21:46 |
| thumper | hackedbellini: to say "1.18.4" | 21:46 |
| thumper | or something before "1.19" anyway | 21:46 |
| thumper | that way when the machine agent starts | 21:46 |
| thumper | it goes "oh, you are pre-ha, let me fix that for you" | 21:46 |
| thumper | I *think* that'll fix it | 21:47 |
| hackedbellini | thumper: hrmm, I see. And after that, restart the agent service? | 21:47 |
| thumper | right | 21:47 |
| menn0 | which needs sudo unfortunately... | 21:47 |
| hackedbellini | menn0: yes, unfortunatly :( | 21:50 |
| hackedbellini | but np, tomorrow I'll ask my workmate to do that and then tell you guys if it solved the problem =P | 21:50 |
| hackedbellini | btw, do I need to do that on all agent.conf of my lxc machines? | 21:51 |
| hackedbellini | I can't look at them right now, but probably they have upgradedToVersion: 1.19.3 too | 21:51 |
| wwitzel3 | thumper: I worked on it a little yes, right when I first started, paired with nate | 21:54 |
| * thumper is still trawling through emails | 21:56 | |
| hackedbellini | well, I'm going now too, it's 19h here in Brazil =P. Thank you guys for all the attention. Let's hope tomorrow I come back here with good news | 21:56 |
| menn0 | hackedbellini: I *think* you'll just need to update the agent conf for machine-0 (the API server) | 21:57 |
| menn0 | hackedbellini: that's where the HA initialisation runs | 21:57 |
| hackedbellini | menn0: nice. So, let's see what happens tomorrow | 21:58 |
| hackedbellini | going now. Cheers! | 21:58 |
| * davecheney woke up swinging this morning | 22:21 | |
| thumper | sinzui: you around? | 22:26 |
| thumper | davecheney: I have a task for you... | 22:26 |
| davecheney | thumper: speak | 22:26 |
| thumper | davecheney: it seems that there are many people, me included, that are unclear on the go compiler rules for conditional compilation | 22:27 |
| thumper | it seems that there are suffix rules, and special comment rules | 22:27 |
| thumper | can I get you to summarise these to the mailing list? | 22:27 |
| davecheney | thumper: http://dave.cheney.net/2013/10/12/how-to-use-conditional-compilation-with-the-go-build-tool | 22:27 |
| davecheney | done | 22:27 |
| davecheney | next | 22:27 |
| thumper | and possibly to put themin the docs directory? | 22:27 |
| thumper | nice | 22:27 |
| thumper | perhaps linking to that blog post from the hacking doc | 22:28 |
| davecheney | thumper: will post to the list | 22:28 |
| thumper | where we have a section on different targets | 22:28 |
| davecheney | thumper: ok | 22:28 |
| thumper | that may help people understand | 22:28 |
| davecheney | thumper: PR coming damn soon | 22:29 |
| davecheney | thumper: I cannot find said document | 22:39 |
| thumper | davecheney: perhaps add to the style guide? | 22:40 |
| perrito666 | davecheney: I would have gone by euro | 23:06 |
| wallyworld | sinzui: hey | 23:20 |
| perrito666 | hey gm wallyworld | 23:20 |
| wallyworld | perrito666: evening | 23:21 |
| wallyworld | sinzui: i have a question about bug 1341589 if you are around | 23:22 |
| _mup_ | Bug #1341589: Distribution tarball has licensing problems that prevent redistribution <juju-core:In Progress by wallyworld> <juju-core 1.20:In Progress by wallyworld> <https://launchpad.net/bugs/1341589> | 23:22 |
| === alexisb_lunch is now known as alexisb | ||
| voidspace | wallyworld: so I found the cause of *almost* all the remaining test failures in my watcher session copying branch | 23:37 |
| wallyworld | voidspace: hey, sorry i missed your ping last night, had fallen asleep at the keyboard | 23:37 |
| voidspace | wallyworld: unsurprisingly, JujuConnSuite is opening the state in a custom way and then changing the password | 23:37 |
| wallyworld | \o/ | 23:37 |
| voidspace | wallyworld: haha | 23:37 |
| voidspace | I thought you probably weren't around but I thought I'd try just in case | 23:38 |
| wallyworld | lol, you alomost got me | 23:38 |
| thumper | o/ voidspace | 23:38 |
| wallyworld | i woke up later to send an email | 23:38 |
| voidspace | I believe you and axw have been working on changing JujuConnSuite? | 23:38 |
| voidspace | I wonder if this code is about to go away anyway | 23:38 |
| voidspace | thumper: hi | 23:38 |
| wallyworld | not directly, but yes we do need to do futher work and it may result in that code changing significantly | 23:39 |
| voidspace | wallyworld: the conn suite follows the pattern of using the hashed password, connecting and then changing the password | 23:39 |
| voidspace | which of course screws session copying because the password is out of date | 23:39 |
| * thumper is taking kids to see a movie and will be working in town at a cafe | 23:39 | |
| voidspace | however, I believe that production code no longer follows this pattern (using the hash) anyway | 23:39 |
| * thumper doesn't want to see tinkerbell | 23:39 | |
| voidspace | thumper: have fun | 23:39 |
| voidspace | haha, I bet thumper really does... | 23:39 |
| * thumper ignores voidspace | 23:40 | |
| voidspace | wallyworld: so my belief is that I can just delete that little dance | 23:40 |
| wallyworld | voidspace: i'm not 100% across the implementation detail (yet), but it seems if we can tweak the suite set up we can try and target the root cause | 23:40 |
| thumper | voidspace: my kids are very jeleous about our marvel vans | 23:40 |
| voidspace | thumper: did you get some too? | 23:40 |
| thumper | s/our/your/ | 23:40 |
| voidspace | ah, ha | 23:40 |
| thumper | missed a y | 23:40 |
| voidspace | yeah - I still like them | 23:40 |
| voidspace | the red Iron Man ones especially | 23:40 |
| wallyworld | voidspace: i don't think it does, there's a lot of legacy in jujuconnsuite | 23:41 |
| thumper | they have gotten into the marvel movies this holidays | 23:41 |
| wallyworld | voidspace: if you push ypur latest changes, we can take a look | 23:41 |
| voidspace | wallyworld: well, the simplest fix was a one line fix to just keep the password as the hash | 23:41 |
| voidspace | wallyworld: but that's not the *right* fix | 23:41 |
| voidspace | wallyworld: I have that pushed, hang on | 23:41 |
| wallyworld | voidspace: that fix was in the test suite, right? | 23:42 |
| voidspace | thumper: ah, good stuff. I enjoy the marvel movies. | 23:42 |
| voidspace | wallyworld: right | 23:42 |
| voidspace | wallyworld: let me get you a link to that change | 23:42 |
| wallyworld | voidspace: at this stage, the right fix is what allows us to ship 1.20.2 :-) | 23:42 |
| wallyworld | we can clean up in trunk if needed | 23:42 |
| voidspace | hah... well | 23:42 |
| voidspace | jam did some digging for me and the right fix might be just as simple | 23:43 |
| voidspace | I'm about to look at that now | 23:43 |
| voidspace | it's nearly 1am but I still have jetlag :-/ | 23:43 |
| wallyworld | voidspace: ok, but get some sleep first :-) | 23:43 |
| voidspace | I didn't get it for vegas | 23:43 |
| wallyworld | voidspace: really appreciate the work on this one | 23:43 |
| voidspace | but I normally do, so I'll stay up another hour I reckon | 23:43 |
| wallyworld | ok, but don't feel obligated to :-) | 23:44 |
| voidspace | wallyworld: well, let's see how much I get actually done by Friday :-/ | 23:44 |
| voidspace | I'd like to get a chunk bitten out | 23:44 |
| voidspace | as soon as I get tests passing I can work on the actual changes | 23:44 |
| wallyworld | yup, we'll pick up with where you get to | 23:44 |
| wallyworld | if i get time today, i'll look to see what ou've done, so keep your wip branch up to date :-) | 23:45 |
| voidspace | wallyworld: see the changes to juju/testing/conn.go in this branch https://github.com/voidspace/juju/compare/copy-sessions | 23:45 |
| wallyworld | looking | 23:45 |
| voidspace | wallyworld: did you see the email I sent to juju-dev? | 23:45 |
| voidspace | that branch only has the watcher changes | 23:45 |
| voidspace | it doesn't have the transaction runner changes | 23:45 |
| wallyworld | ah not yet, still getting through backlog | 23:45 |
| voidspace | I summarised what I've done so far and gave three pastebins with the diffs | 23:46 |
| wallyworld | hopefully txn runner changes a lot easier than collection ones | 23:46 |
| voidspace | the diffs are small as the problem is getting tests to pass | 23:46 |
| wallyworld | great will read | 23:46 |
| voidspace | wallyworld: they're much more isolated, yes | 23:46 |
| wallyworld | hopefully getting tests to pass is just a small fix with password | 23:46 |
| voidspace | just three methods to change and all they do is copy the session and create a new transaction runner | 23:46 |
| wallyworld | yup, spnds right | 23:47 |
| wallyworld | sounds | 23:47 |
| voidspace | wallyworld: well, with this in place I have several test failures - down from a shitload | 23:47 |
| wallyworld | that's a start | 23:47 |
| voidspace | and that was already down from crap-tonne | 23:47 |
| voidspace | so steady progress | 23:47 |
| voidspace | yep, really a limited few failures now | 23:47 |
| wallyworld | several < shitload < fucktonne | 23:47 |
| voidspace | :-D | 23:47 |
| wallyworld | voidspace: sounds like you're well on the way to having an almost complete fix we can take and finish and run with for next week | 23:48 |
| voidspace | I think that change in conn.go is actually a no-op as the password has already been set to the hash | 23:48 |
| voidspace | the hash is actually set in the dummy provider | 23:48 |
| bigjools | sinzui: I saw you triaged https://bugs.launchpad.net/maas/+bug/1341281, but I suspect the guy has messed up his network config | 23:48 |
| voidspace | uju/provider/dummy/environs.go line 654 | 23:48 |
| _mup_ | Bug #1341281: MaaS does not report to juju that the bootstrap node is ready? <juju-core:Triaged> <juju-core 1.20:Triaged> <MAAS:Incomplete> <https://launchpad.net/bugs/1341281> | 23:48 |
| voidspace | and that code is (we believe) out of date with the actual implementation now | 23:49 |
| voidspace | wallyworld: I'll leave my hack in place and shoot a separate email to axw about that | 23:49 |
| wallyworld | voidspace: could be, i'd have to get across the implementation detail | 23:49 |
| sinzui | bigjools, I agree. I think getting more people might help close the issue. We don't want to see any more maas misadventures | 23:49 |
| voidspace | right | 23:49 |
| wallyworld | voidspace: yeah, that conn change is a no-op. also, master has diverged quite a bit from 1.20 - that conn code lives in testing in master but up a level in juju in 1.20. so we will have to backport your changes when master is finished | 23:54 |
| voidspace | right | 23:54 |
| voidspace | gah, and then other tests *depend* on the password being "dummy-secret", hard coded | 23:57 |
| voidspace | e.g. provisioner_test | 23:57 |
| wallyworld | sigh | 23:57 |
| sinzui | davecheney, et al, sorry, CI is running behind schedule because I tried to switch it over to the new scheduling rules. It wasn't a complete success. I don't think CI will see the queue revisions for another 2 hours | 23:58 |
| voidspace | wallyworld: so I'm going to kill the code that sets it to the hash instead | 23:58 |
| davecheney | sinzui: ok | 23:58 |
| voidspace | well, as well | 23:58 |
| wallyworld | voidspace: i think that's ok ottomh. let's get the tests passing and we can revisit and tweak the implementation with all known changes in place | 23:59 |
| voidspace | wallyworld: yep, I'll do another progress update tomorrow | 23:59 |
| wallyworld | \o/ thank you | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!