=== TheMue_ is now known as TheMue
wrtpfwereade_, TheMue: morning06:53
fwereade_heya wrtp06:53
TheMuewrtp, fwereade_ : moin06:53
wrtpfwereade_: i'd quite like a chat about deploying Go juju some time06:53
fwereade_wrtp, sounds good; 5 mins?06:55
wrtpfwereade_: cool06:55
fwereade_wrtp, invite out07:00
wrtpfwereade_: ah, actually i can't do a verbal chat now, 'cos i'll wake up the sleeping angel07:01
fwereade_wrtp, np, ready all the same :)07:02
fwereade_wrtp, (you mean I got dressed for nothing? :p)07:02
wrtpfwereade_: lol07:02
wrtpfwereade_: so...07:03
wrtpfwereade_: as far as i can see we've got two different problems (or maybe three)07:03
wrtpfwereade_: 1) how do we deploy an appropriate binary at development time07:03
wrtpfwereade_: 2) how do we deploy binaries that aren't for the platform we're currently on07:04
wrtp3) (?) how do we make sure that we're deploying the right version?07:04
fwereade_wrtp, yeah, I think all 3 are real, but niemeyer's plan seems to cover (3) pretty well07:05
wrtp1) is difficult unless we can assume that we're only testing on a compatible platform to the dev platform, which maybe isn't a bad assumption07:06
wrtpfwereade_: remind me of niemeyer's plan again07:06
fwereade_wrtp, I think that assumption is the right one for dev mode at least to start out07:06
fwereade_wrtp, the versioning thread I was being obtuse in yesterday07:06
fwereade_wrtp, not sure I can summarise it very well07:07
wrtpfwereade_: on the mailing list?07:07
fwereade_wrtp, yeah07:07
wrtpfwereade_: ah, i totally missed that; "Bootstrap scheme for Go port"07:08
fwereade_wrtp, but basially the initial email covers it, with the addition of the odd-number-anywhere-denotes-dev-version bit07:08
* wrtp is reading07:09
wrtpfwereade_: by "semantic versioning" he's referring to this: http://semver.org/ ?07:12
fwereade_wrtp, yes07:12
fwereade_wrtp, the odd-number dev versions break what's suggested there AFAICT, but it's pretty clear all the same07:13
wrtpfwereade_: isn't the odd-number proposal using odd numbers for the same kind of thing that "pre-release" versions are used in the original sem. ver. spec?07:24
fwereade_wrtp, it is indeed, so it's pretty clearly comprehensible07:24
wrtpfwereade_: so... why not just use prerelease versions so we'll be fully semver compatible?07:25
fwereade_wrtp, but it's still an exception that is not obvious if you just say "we're using semver"07:25
fwereade_wrtp, well, say we've released 4.0.007:25
fwereade_wrtp, for testing major upgrades of dev versions sanely we'll want to be using 5.x.x07:26
wrtpfwereade_: 5.x.x-alpha.0 ?07:27
fwereade_wrtp, but assuming 2 breaking changes in a dev cycle, we don't want to make users go 4->8 next release, it would be weird07:27
wrtpfwereade_: i don't see the problem. we can do 5.x.x-alpha.0, 5.x.x-alpha.1 etc etc07:27
wrtp5.0.0-alpha.0, 5.0.0-alpha.1 actually07:28
fwereade_wrtp, "Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API. It MAY include minor and patch level changes. Patch and minor version MUST be reset to 0 when major version is incremented."07:28
fwereade_wrtp, I think we break it one way or another whatever we do07:29
wrtpfwereade_: i'm not sure that pre-release versions need to be strictly compatible in the same way07:30
fwereade_wrtp, but if we're using the version for upgrade/compatibility logic we need to be able to have sane versions when we're testing that we can actually upgrade, don't we?07:31
fwereade_wrtp, it's really just extending the definition of pre-release07:31
fwereade_wrtp, maybe I'm wrong and the existing provisions satisfy it07:33
wrtpfwereade_: am just trying to sort it out in my head :-)07:33
fwereade_wrtp, I just recoil a little from worrying about them in the context of the upgrade logic07:33
wrtpfwereade_: i still think i don't see the problem. we can use exactly the same logic that's described on the semver page.07:37
wrtpfwereade_: but i'd suggest that in "development mode" the "pre-release" logic switches to giving pre-release a higher precedence than the normal version.07:38
wrtpfwereade_: as it is, the comparison logic with odd numbers will be weird.07:39
wrtps/as it is/as proposed/07:39
fwereade_wrtp, the core of the problem is that we can't signal a backwards-incompatible change between two differentr pre-release versions with the same major version... can we?07:39
fwereade_wrtp, how's it weird? I *think* the logic will be just the same07:40
fwereade_wrtp, we just don't release odd-numbered versions to the canonical location07:40
wrtpfwereade_: ah, i'd missed that twist07:41
wrtpfwereade_: i'm not sure why we'd need to signal a backwards-incompatible change between two different pre-release versions07:42
wrtpfwereade_: we don't care about breaking dev deployments07:43
fwereade_wrtp, yeah, but to fulfil semver we ought to07:43
fwereade_wrtp, ...right?07:44
fwereade_wrtp, I don't see it carving out an incompatibility exception for pre-release versions, but maybe I'm just blind07:44
wrtpfwereade_: i'm not sure. it doesn't talk about prerelease versions much. and why would you have a prerelease version if it wasn't to find problems and fix them *before* releasing the one that does it "right".07:45
wrtpi think if we use pre-release versions in this way, we don't need two repositories and we can just use the normal semver logic07:45
* fwereade_ thinks07:47
wrtpin fact, we don't even need a "development mode" switch, i realise, because we can just up the client version to, say 5.0.0, which is satisfied by 5.0.0-alpha1 until 5.0.0 real version is released.07:47
wrtpfwereade_: semver doesn't talk about any compatibility requirements *between* pre-release versions AFAICS07:48
fwereade_wrtp, hm, don't we need a dev repo for that all the same? otherwise we have to have a speculative version bump every time we start a new cycle07:49
wrtpfwereade_: that sounds right to me07:49
fwereade_wrtp, the speculative version bump?07:49
wrtpfwereade_: yeah07:50
fwereade_wrtp, not convinced07:50
wrtpfwereade_: but if it turns out the version bump isn't needed, we can release as a lower version.07:50
fwereade_wrtp, I think we should be shooting for releases that *are* backward compatible in general07:50
wrtpfwereade_: definitely07:50
fwereade_wrtp, that feels like it *also* breaks semver07:50
wrtphmm, yeah, probably.07:51
wrtpfwereade_: but won't we have to increment speculatively with the odd-numbering thing too?07:52
fwereade_wrtp, hmm07:52
fwereade_wrtp, yeah :(07:53
wrtpfwereade_: i don't think i mind the speculative versioning. i think it maps quite well to what we're actually doing - building a version that is planned to become a real version at some point.07:54
wrtpfwereade_: and if we decide the version is wrong, we can delete the wrongly-speculatively-versioned prerelease versions07:54
fwereade_wrtp, wait a mo, with the odd-numbered-versions we *don't* have to07:55
fwereade_wrtp, release 4.0.0; dev work starts on 4.0.1; we release 4.0.2 from latest state of 4.0.1 when we're ready07:56
wrtpfwereade_: no?07:56
fwereade_wrtp, we only have to bump the major version at the point we introduce something that actually is incompatible07:56
wrtpfwereade_: how's that different from: release 4.0.0; dev work starts on 4.0.1-alpha.0; we release 4.0.1 from latest pre-release version of 4.0.1 ?07:57
fwereade_wrtp, ok, I think you're right :)07:59
fwereade_wrtp, don't think we can avoid the separate repos though07:59
wrtpfwereade_: why's that?07:59
fwereade_wrtp, because 4.0.1-alpha.0 will appear to be a valid version for 4.0.0 to deploy on the server08:01
fwereade_wrtp, I think we may be making too much of this really -- the public repo should only have the stuff we've actually released08:01
fwereade_wrtp, so we need private repos when we're testing08:02
wrtpfwereade_: that's probably true08:02
wrtpfwereade_: so the public repo is the dev repo without the prerelease versions08:02
wrtpfwereade_: alternatively the comparison logic could be configurable to make it not consider pre-release versions.08:03
fwereade_wrtp, yeah, that just crossed my mind too08:03
fwereade_wrtp, ok, tell you what, we can get around all this by spinning up parallel universes to do dev work in and just merge the timelines later08:05
fwereade_wrtp, I'll go find a physicist ;)08:05
wrtpfwereade_: might get a few weird effects when we get conflicts tho'08:06
fwereade_wrtp, more seriously, I hadn't really been thinking about a central dev repo08:06
fwereade_wrtp, details details ;)08:06
fwereade_wrtp, as devs the only versions we care about are released ones and whatever we're working on at the moment08:07
fwereade_wrtp, there may be a preview repo somewhere for other people to test against, with the no-backward-compatibility-guarantees exception08:07
wrtpfwereade_: there's another issue if devs are sharing the same repo: you'll have to explicitly set your client version number to your "own" pre-release version, otherwise you'll get someone else's (unless yours happens to compare greater than others')08:08
fwereade_wrtp, yeah, indeed, hence my private-repo assumptions08:08
wrtpfwereade_: i think it could work ok though.08:09
wrtpfwereade_: even with a public repo08:09
wrtpfwereade_: we could accept an environment variable "JUJU_PRERELEASE" which would give the prerelease version that you're working on.08:10
wrtpe.g. JUJU-PRERELEASE=rog.008:10
fwereade_wrtp, I'd really prefer to have the repo location be the only moving part08:11
wrtpfwereade_: yeah, you're probably right.08:12
wrtpfwereade_: http://gopkgdoc.appspot.com/pkg/launchpad.net/~rogpeppe/+junk/version11:15
wrtpfwereade_: that implements the "must be at latest version to use pre-release versions" rule, as well as the rest of the semantic versioning rules as far as i could make them out.11:19
fwereade_wrtp, that seems pretty neat really11:56
fwereade_wrtp, let's see what niemeyer thinks :)11:56
wrtpfwereade_: cool. yeah, let's see.11:59
wrtpfwereade_: oh yes, i forgot, gustavo's not around today12:27
TheMuewrtp: Could you please add more doc for the structs fields? Especially the semantics of prerelease and build and how they are handled in Less().12:49
TheMuefwereade_: time for a talk about "force upgrade" for units?13:09
fwereade_TheMue, ofc :)13:09
fwereade_TheMue, I'm no expert but I'll help if I can13:09
fwereade_TheMue, re wrtp's stuff, http://semver.org/ shoudl cover most of it13:10
TheMuefwereade_: The current watcher implementation only observes if the node in ZK is created. And you wrote a comment that this isn't enough.13:10
TheMuefwereade_: Oh, after a quick look an interesting doc. Thx, will read it later.13:11
fwereade_TheMue, juju/agents/unit.py:197 is probably a good place to start looking13:12
TheMuefwereade_: Will take a look.13:12
fwereade_TheMue, basically people want to be able to upgrade broken units13:12
fwereade_TheMue, and the default implementation only does an upgrade if it's in the "running" state13:12
wrtpTheMue: what fwereade_ says.13:15
TheMuefwereade_: IC, so there already has been an initial error implementing NeedsUpgrade() and SetNeedsUpgrade() without viewing the node content13:16
wrtpTheMue: i could probably describe the correspondence of the fields with the version components as described at semver.org, i guess13:16
fwereade_TheMue, I'm not sure it was an error at the time, the --force flag is relatively recent13:16
TheMuefwereade_: Is this a newer change or already longer in code?13:16
TheMuefwereade_: Ah, ic, this is an explain (you say so?).13:17
fwereade_TheMue, sorry, cannot parse13:17
wrtpTheMue: it was a very quick coding job this morning in response to a discussion with fwereade_ - it's not fully documented yet...13:17
TheMuewrtp: I only wanted to understand those two fields better, but the doc mentioned above already looks good.13:18
wrtpTheMue: the interaction between prerelease and build doesn't really describe the interaction between those two fields, unfortunately.13:18
TheMuefwereade_: Argh, yes, hadn't the word. Should read: this is an explanation.13:18
TheMuefwereade_: So I will handle the content now too.13:19
fwereade_TheMue, cool, thanks (it was merged less than a month ago, so it was before I switched back to python, and I'm sure I remember seeing the first upgrade stuff in go while I was still on go)13:21
TheMuefwereade_: I've got to thank you for the hint. Otherwise it would have been wrong from the beginning.13:25
fwereade_TheMue, I think this is one of the biggest risks we face tbh13:25
fwereade_TheMue, and being the only person on the team with semi-current knowledge of the python makes me a little nervous, but I'll do my best ;)13:26
TheMuefwereade_: Your the official Knowledge Transfer Agent. *lol*13:27
fwereade_haha :)13:27
wrtpfwereade_: do you have a preference for what kind of provider storage we should use for juju binaries?14:01
wrtpfwereade_: perhaps we should just use S3 on EC214:02
fwereade_wrtp, not really; I know there's been quiet talk of decoupling storage provider from machine provider but that feels like a distraction at this point14:02
fwereade_wrtp, so, yeah, whatever storage we already have for the appropriate provider14:02
wrtpfwereade_: i wonder if Environ.Bootstrap should take the client version as an argument, or if the environs package should fetch the version number from somewhere on its own.14:11
wrtpfwereade_: my inclination is towards the former14:12
fwereade_wrtp, likewise, I think14:12
fwereade_wrtp, this may not be python but explicit still beats implicit in general ;)14:12
wrtpfwereade_: cool. and then i guess i'd store the location of the binaries in the state, so agents can pull them out using Environ.GetFile14:15
fwereade_wrtp, ...I *think* so but most of my cycles are still concentrated on figuring out what the hell I was oing a months ago in the followup branches I hadn't proposed yet14:17
wrtpfwereade_: :-)14:17
wrtpfwereade_: that can be educational...14:17
fwereade_wrtp, also on not being able to type, it seems14:17
wrtpfwereade_: what's the most universally available command-line tool for downloading from a web server? wget? (i realise that the code to download the binaries from S3 cannot be written in Go itself... doh!)15:28
fwereade_wrtp, ha, I guess so :)15:30
fwereade_wrtp, but really you can install what you want in the cloud-init, can't you?15:30
wrtpfwereade_: how do i get the cloud-init to install a binary from S3?15:30
fwereade_wrtp, (not that I'm advocating a separate juju-getter package or anything :p)15:31
wrtpfwereade_: sure, i could apt-get another package. but if a couple of lines of shell will do it, i think that'd be better.15:31
fwereade_wrtp, you can run arbitrary scripts, so... however you want ;p15:31
fwereade_wrtp, indeed15:32
fwereade_wrtp, wget a signed URL seems sensible15:32
wrtpfwereade_: you mean a URL containing signed content?15:32
fwereade_wrtp, I was just thinking something like juju/providers/ec2/files.py:4015:35
wrtpfwereade_: yeah, i was planning to provide that15:35
wrtpfwereade_: another question: how can i tell what tools are in a basic ubuntu release? (for instance, i'm wondering if unzip is there by default. the less apt-gets the better, i think)15:36
fwereade_wrtp, hmm, I have no idea I'm afraid :(15:36
wrtpfwereade_: guess i'll just try it and see :-)15:36
fwereade_wrtp, sounds good :)15:37
wrtpfwereade_: i *thought* the useful thing about zip vs tar is that it's supported within Go. but tar is also supported, so i think that's probably the better option.15:38
wrtpfwereade_: and tar and gzip will most definitely be part of the stock distro15:39
fwereade_wrtp, so I would assume ;)15:39
wrtpfwereade_: they darn well should be15:40
wrtpfwereade_: i guess we should probably sign the content too.15:42
fwereade_wrtp, probably sensible15:43
wrtpfwereade_: yet another thing: have you got an opinion on where we should put the binaries?15:47
fwereade_wrtp, not really :)15:47
wrtpfwereade_: /tmp then :-)15:48
wrtpfwereade_, TheMue: https://codereview.appspot.com/608104416:30
fwereade_wrtp, LGTM16:32
wrtpfwereade, TheMue: https://codereview.appspot.com/608204417:14
wrtpi should probably change it to use gocheck though, i suppose17:16
wrtpfwereade, TheMue: right, that's me for the week. see ya monday. have a great weekend!17:17
TheMuewrtp: Have a nice weekend. gocheck idea is good. I'm already off, only looking at tjose two last proposals.17:18
wrtpTheMue: one reason for not using gocheck is that it's a highly independent package. i quite like not depending on very much, and the testing package works fine here.17:20
TheMuewrtp: Yes, you don't need it. It's only that a different maintainer later may ask himself why testing here is different than for other packages.17:22
wrtpTheMue: the tomb package doesn't use gocheck, for example17:23
TheMuewrtp: It's an external package, not inside the project.17:23
wrtpTheMue: yeah. this might become external too.17:24
TheMuewrtp: But indeed, you don't need it so far.17:24
TheMuewrtp: In this case it's ok.17:24
fwereadewrtp, basically LGTM, don't worry about it until monday :)17:25
wrtpfwereade: minor versions can add features17:47
TheMueSo, last proposal done, have a nice weekend.18:24

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!