[06:53] <wrtp> fwereade_, TheMue: morning
[06:53] <fwereade_> heya wrtp
[06:53] <TheMue> wrtp, fwereade_ : moin
[06:53] <wrtp> fwereade_: i'd quite like a chat about deploying Go juju some time
[06:55] <fwereade_> wrtp, sounds good; 5 mins?
[06:55] <wrtp> fwereade_: cool
[07:00] <fwereade_> wrtp, invite out
[07:01] <wrtp> fwereade_: ah, actually i can't do a verbal chat now, 'cos i'll wake up the sleeping angel
[07:02] <fwereade_> wrtp, np, ready all the same :)
[07:02] <fwereade_> wrtp, (you mean I got dressed for nothing? :p)
[07:02] <wrtp> fwereade_: lol
[07:03] <wrtp> fwereade_: so...
[07:03] <wrtp> fwereade_: as far as i can see we've got two different problems (or maybe three)
[07:03] <wrtp> fwereade_: 1) how do we deploy an appropriate binary at development time
[07:04] <wrtp> fwereade_: 2) how do we deploy binaries that aren't for the platform we're currently on
[07:04] <wrtp> 3) (?) how do we make sure that we're deploying the right version?
[07:05] <fwereade_> wrtp, yeah, I think all 3 are real, but niemeyer's plan seems to cover (3) pretty well
[07:06] <wrtp> 1) is difficult unless we can assume that we're only testing on a compatible platform to the dev platform, which maybe isn't a bad assumption
[07:06] <wrtp> fwereade_: remind me of niemeyer's plan again
[07:06] <fwereade_> wrtp, I think that assumption is the right one for dev mode at least to start out
[07:06] <fwereade_> wrtp, the versioning thread I was being obtuse in yesterday
[07:07] <fwereade_> wrtp, not sure I can summarise it very well
[07:07] <wrtp> fwereade_: on the mailing list?
[07:07] <fwereade_> wrtp, yeah
[07:08] <wrtp> fwereade_: ah, i totally missed that; "Bootstrap scheme for Go port"
[07:08] <fwereade_> wrtp, but basially the initial email covers it, with the addition of the odd-number-anywhere-denotes-dev-version bit
[07:09]  * wrtp is reading
[07:12] <wrtp> fwereade_: by "semantic versioning" he's referring to this: http://semver.org/ ?
[07:12] <fwereade_> wrtp, yes
[07:13] <fwereade_> wrtp, the odd-number dev versions break what's suggested there AFAICT, but it's pretty clear all the same
[07:24] <wrtp> fwereade_: isn't the odd-number proposal using odd numbers for the same kind of thing that "pre-release" versions are used in the original sem. ver. spec?
[07:24] <fwereade_> wrtp, it is indeed, so it's pretty clearly comprehensible
[07:25] <wrtp> fwereade_: so... why not just use prerelease versions so we'll be fully semver compatible?
[07:25] <fwereade_> wrtp, but it's still an exception that is not obvious if you just say "we're using semver"
[07:25] <fwereade_> wrtp, well, say we've released 4.0.0
[07:26] <fwereade_> wrtp, for testing major upgrades of dev versions sanely we'll want to be using 5.x.x
[07:27] <wrtp> fwereade_: 5.x.x-alpha.0 ?
[07:27] <fwereade_> wrtp, but assuming 2 breaking changes in a dev cycle, we don't want to make users go 4->8 next release, it would be weird
[07:27] <wrtp> fwereade_: i don't see the problem. we can do 5.x.x-alpha.0, 5.x.x-alpha.1 etc etc
[07:28] <wrtp> 5.0.0-alpha.0, 5.0.0-alpha.1 actually
[07:28] <fwereade_> wrtp, "Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API. It MAY include minor and patch level changes. Patch and minor version MUST be reset to 0 when major version is incremented."
[07:29] <fwereade_> wrtp, I think we break it one way or another whatever we do
[07:30] <wrtp> fwereade_: i'm not sure that pre-release versions need to be strictly compatible in the same way
[07:31] <fwereade_> wrtp, but if we're using the version for upgrade/compatibility logic we need to be able to have sane versions when we're testing that we can actually upgrade, don't we?
[07:31] <fwereade_> wrtp, it's really just extending the definition of pre-release
[07:33] <fwereade_> wrtp, maybe I'm wrong and the existing provisions satisfy it
[07:33] <wrtp> fwereade_: am just trying to sort it out in my head :-)
[07:33] <fwereade_> wrtp, I just recoil a little from worrying about them in the context of the upgrade logic
[07:37] <wrtp> fwereade_: i still think i don't see the problem. we can use exactly the same logic that's described on the semver page.
[07:38] <wrtp> fwereade_: but i'd suggest that in "development mode" the "pre-release" logic switches to giving pre-release a higher precedence than the normal version.
[07:39] <wrtp> fwereade_: as it is, the comparison logic with odd numbers will be weird.
[07:39] <wrtp> s/as it is/as proposed/
[07:39] <fwereade_> wrtp, the core of the problem is that we can't signal a backwards-incompatible change between two differentr pre-release versions with the same major version... can we?
[07:40] <fwereade_> wrtp, how's it weird? I *think* the logic will be just the same
[07:40] <fwereade_> wrtp, we just don't release odd-numbered versions to the canonical location
[07:41] <wrtp> fwereade_: ah, i'd missed that twist
[07:42] <wrtp> fwereade_: i'm not sure why we'd need to signal a backwards-incompatible change between two different pre-release versions
[07:43] <wrtp> fwereade_: we don't care about breaking dev deployments
[07:43] <fwereade_> wrtp, yeah, but to fulfil semver we ought to
[07:44] <fwereade_> wrtp, ...right?
[07:44] <fwereade_> wrtp, I don't see it carving out an incompatibility exception for pre-release versions, but maybe I'm just blind
[07:45] <wrtp> fwereade_: i'm not sure. it doesn't talk about prerelease versions much. and why would you have a prerelease version if it wasn't to find problems and fix them *before* releasing the one that does it "right".
[07:45] <wrtp> ?
[07:45] <wrtp> i think if we use pre-release versions in this way, we don't need two repositories and we can just use the normal semver logic
[07:47]  * fwereade_ thinks
[07:47] <wrtp> in fact, we don't even need a "development mode" switch, i realise, because we can just up the client version to, say 5.0.0, which is satisfied by 5.0.0-alpha1 until 5.0.0 real version is released.
[07:48] <wrtp> fwereade_: semver doesn't talk about any compatibility requirements *between* pre-release versions AFAICS
[07:49] <fwereade_> wrtp, hm, don't we need a dev repo for that all the same? otherwise we have to have a speculative version bump every time we start a new cycle
[07:49] <wrtp> fwereade_: that sounds right to me
[07:49] <fwereade_> wrtp, the speculative version bump?
[07:50] <wrtp> fwereade_: yeah
[07:50] <fwereade_> wrtp, not convinced
[07:50] <wrtp> fwereade_: but if it turns out the version bump isn't needed, we can release as a lower version.
[07:50] <fwereade_> wrtp, I think we should be shooting for releases that *are* backward compatible in general
[07:50] <wrtp> fwereade_: definitely
[07:50] <fwereade_> wrtp, that feels like it *also* breaks semver
[07:51] <wrtp> hmm, yeah, probably.
[07:52] <wrtp> fwereade_: but won't we have to increment speculatively with the odd-numbering thing too?
[07:52] <fwereade_> wrtp, hmm
[07:53] <fwereade_> wrtp, yeah :(
[07:54] <wrtp> fwereade_: i don't think i mind the speculative versioning. i think it maps quite well to what we're actually doing - building a version that is planned to become a real version at some point.
[07:54] <wrtp> fwereade_: and if we decide the version is wrong, we can delete the wrongly-speculatively-versioned prerelease versions
[07:55] <fwereade_> wrtp, wait a mo, with the odd-numbered-versions we *don't* have to
[07:56] <fwereade_> wrtp, release 4.0.0; dev work starts on 4.0.1; we release 4.0.2 from latest state of 4.0.1 when we're ready
[07:56] <wrtp> fwereade_: no?
[07:56] <fwereade_> wrtp, we only have to bump the major version at the point we introduce something that actually is incompatible
[07:57] <wrtp> fwereade_: how's that different from: release 4.0.0; dev work starts on 4.0.1-alpha.0; we release 4.0.1 from latest pre-release version of 4.0.1 ?
[07:59] <fwereade_> wrtp, ok, I think you're right :)
[07:59] <fwereade_> wrtp, don't think we can avoid the separate repos though
[07:59] <wrtp> fwereade_: why's that?
[08:01] <fwereade_> wrtp, because 4.0.1-alpha.0 will appear to be a valid version for 4.0.0 to deploy on the server
[08:01] <fwereade_> wrtp, I think we may be making too much of this really -- the public repo should only have the stuff we've actually released
[08:02] <fwereade_> wrtp, so we need private repos when we're testing
[08:02] <wrtp> fwereade_: that's probably true
[08:02] <wrtp> fwereade_: so the public repo is the dev repo without the prerelease versions
[08:03] <wrtp> fwereade_: alternatively the comparison logic could be configurable to make it not consider pre-release versions.
[08:03] <fwereade_> wrtp, yeah, that just crossed my mind too
[08:05] <fwereade_> wrtp, ok, tell you what, we can get around all this by spinning up parallel universes to do dev work in and just merge the timelines later
[08:05] <fwereade_> wrtp, I'll go find a physicist ;)
[08:05] <wrtp> SGTM
[08:06] <wrtp> fwereade_: might get a few weird effects when we get conflicts tho'
[08:06] <fwereade_> wrtp, more seriously, I hadn't really been thinking about a central dev repo
[08:06] <fwereade_> wrtp, details details ;)
[08:07] <fwereade_> wrtp, as devs the only versions we care about are released ones and whatever we're working on at the moment
[08:07] <fwereade_> wrtp, there may be a preview repo somewhere for other people to test against, with the no-backward-compatibility-guarantees exception
[08:08] <wrtp> fwereade_: there's another issue if devs are sharing the same repo: you'll have to explicitly set your client version number to your "own" pre-release version, otherwise you'll get someone else's (unless yours happens to compare greater than others')
[08:08] <fwereade_> wrtp, yeah, indeed, hence my private-repo assumptions
[08:09] <wrtp> fwereade_: i think it could work ok though.
[08:09] <wrtp> fwereade_: even with a public repo
[08:10] <wrtp> fwereade_: we could accept an environment variable "JUJU_PRERELEASE" which would give the prerelease version that you're working on.
[08:10] <wrtp> e.g. JUJU-PRERELEASE=rog.0
[08:11] <fwereade_> wrtp, I'd really prefer to have the repo location be the only moving part
[08:12] <wrtp> fwereade_: yeah, you're probably right.
[08:39] <TheMue> re
[11:15] <wrtp> fwereade_: http://gopkgdoc.appspot.com/pkg/launchpad.net/~rogpeppe/+junk/version
[11:19] <wrtp> fwereade_: that implements the "must be at latest version to use pre-release versions" rule, as well as the rest of the semantic versioning rules as far as i could make them out.
[11:56] <fwereade_> wrtp, that seems pretty neat really
[11:56] <fwereade_> wrtp, let's see what niemeyer thinks :)
[11:59] <wrtp> fwereade_: cool. yeah, let's see.
[12:27] <wrtp> fwereade_: oh yes, i forgot, gustavo's not around today
[12:49] <TheMue> wrtp: Could you please add more doc for the structs fields? Especially the semantics of prerelease and build and how they are handled in Less().
[13:09] <TheMue> fwereade_: time for a talk about "force upgrade" for units?
[13:09] <fwereade_> TheMue, ofc :)
[13:09] <fwereade_> TheMue, I'm no expert but I'll help if I can
[13:10] <fwereade_> TheMue, re wrtp's stuff, http://semver.org/ shoudl cover most of it
[13:10] <TheMue> fwereade_: The current watcher implementation only observes if the node in ZK is created. And you wrote a comment that this isn't enough.
[13:11] <TheMue> fwereade_: Oh, after a quick look an interesting doc. Thx, will read it later.
[13:12] <fwereade_> TheMue, juju/agents/unit.py:197 is probably a good place to start looking
[13:12] <TheMue> fwereade_: Will take a look.
[13:12] <fwereade_> TheMue, basically people want to be able to upgrade broken units
[13:12] <fwereade_> TheMue, and the default implementation only does an upgrade if it's in the "running" state
[13:15] <wrtp> TheMue: what fwereade_ says.
[13:16] <TheMue> fwereade_: IC, so there already has been an initial error implementing NeedsUpgrade() and SetNeedsUpgrade() without viewing the node content
[13:16] <wrtp> TheMue: i could probably describe the correspondence of the fields with the version components as described at semver.org, i guess
[13:16] <fwereade_> TheMue, I'm not sure it was an error at the time, the --force flag is relatively recent
[13:16] <TheMue> fwereade_: Is this a newer change or already longer in code?
[13:17] <TheMue> fwereade_: Ah, ic, this is an explain (you say so?).
[13:17] <fwereade_> TheMue, sorry, cannot parse
[13:17] <wrtp> TheMue: it was a very quick coding job this morning in response to a discussion with fwereade_ - it's not fully documented yet...
[13:18] <TheMue> wrtp: I only wanted to understand those two fields better, but the doc mentioned above already looks good.
[13:18] <wrtp> TheMue: the interaction between prerelease and build doesn't really describe the interaction between those two fields, unfortunately.
[13:18] <TheMue> fwereade_: Argh, yes, hadn't the word. Should read: this is an explanation.
[13:19] <TheMue> fwereade_: So I will handle the content now too.
[13:21] <fwereade_> TheMue, cool, thanks (it was merged less than a month ago, so it was before I switched back to python, and I'm sure I remember seeing the first upgrade stuff in go while I was still on go)
[13:25] <TheMue> fwereade_: I've got to thank you for the hint. Otherwise it would have been wrong from the beginning.
[13:25] <fwereade_> TheMue, I think this is one of the biggest risks we face tbh
[13:26] <fwereade_> TheMue, and being the only person on the team with semi-current knowledge of the python makes me a little nervous, but I'll do my best ;)
[13:27] <TheMue> fwereade_: Your the official Knowledge Transfer Agent. *lol*
[13:27] <fwereade_> haha :)
[14:01] <wrtp> fwereade_: do you have a preference for what kind of provider storage we should use for juju binaries?
[14:02] <wrtp> fwereade_: perhaps we should just use S3 on EC2
[14:02] <fwereade_> wrtp, not really; I know there's been quiet talk of decoupling storage provider from machine provider but that feels like a distraction at this point
[14:02] <fwereade_> wrtp, so, yeah, whatever storage we already have for the appropriate provider
[14:11] <wrtp> fwereade_: i wonder if Environ.Bootstrap should take the client version as an argument, or if the environs package should fetch the version number from somewhere on its own.
[14:12] <wrtp> fwereade_: my inclination is towards the former
[14:12] <fwereade_> wrtp, likewise, I think
[14:12] <fwereade_> wrtp, this may not be python but explicit still beats implicit in general ;)
[14:15] <wrtp> fwereade_: cool. and then i guess i'd store the location of the binaries in the state, so agents can pull them out using Environ.GetFile
[14:17] <fwereade_> wrtp, ...I *think* so but most of my cycles are still concentrated on figuring out what the hell I was oing a months ago in the followup branches I hadn't proposed yet
[14:17] <wrtp> fwereade_: :-)
[14:17] <wrtp> fwereade_: that can be educational...
[14:17] <fwereade_> wrtp, also on not being able to type, it seems
[15:28] <wrtp> fwereade_: what's the most universally available command-line tool for downloading from a web server? wget? (i realise that the code to download the binaries from S3 cannot be written in Go itself... doh!)
[15:30] <fwereade_> wrtp, ha, I guess so :)
[15:30] <fwereade_> wrtp, but really you can install what you want in the cloud-init, can't you?
[15:30] <wrtp> fwereade_: how do i get the cloud-init to install a binary from S3?
[15:31] <fwereade_> wrtp, (not that I'm advocating a separate juju-getter package or anything :p)
[15:31] <wrtp> fwereade_: sure, i could apt-get another package. but if a couple of lines of shell will do it, i think that'd be better.
[15:31] <fwereade_> wrtp, you can run arbitrary scripts, so... however you want ;p
[15:32] <fwereade_> wrtp, indeed
[15:32] <fwereade_> wrtp, wget a signed URL seems sensible
[15:32] <wrtp> fwereade_: you mean a URL containing signed content?
[15:35] <fwereade_> wrtp, I was just thinking something like juju/providers/ec2/files.py:40
[15:35] <wrtp> fwereade_: yeah, i was planning to provide that
[15:36] <wrtp> fwereade_: another question: how can i tell what tools are in a basic ubuntu release? (for instance, i'm wondering if unzip is there by default. the less apt-gets the better, i think)
[15:36] <fwereade_> wrtp, hmm, I have no idea I'm afraid :(
[15:36] <wrtp> fwereade_: guess i'll just try it and see :-)
[15:37] <fwereade_> wrtp, sounds good :)
[15:38] <wrtp> fwereade_: i *thought* the useful thing about zip vs tar is that it's supported within Go. but tar is also supported, so i think that's probably the better option.
[15:39] <wrtp> fwereade_: and tar and gzip will most definitely be part of the stock distro
[15:39] <fwereade_> wrtp, so I would assume ;)
[15:40] <wrtp> fwereade_: they darn well should be
[15:42] <wrtp> fwereade_: i guess we should probably sign the content too.
[15:43] <fwereade_> wrtp, probably sensible
[15:47] <wrtp> fwereade_: yet another thing: have you got an opinion on where we should put the binaries?
[15:47] <fwereade_> wrtp, not really :)
[15:48] <wrtp> fwereade_: /tmp then :-)
[16:30] <wrtp> fwereade_, TheMue: https://codereview.appspot.com/6081044
[16:32] <fwereade_> wrtp, LGTM
[17:14] <wrtp> fwereade, TheMue: https://codereview.appspot.com/6082044
[17:16] <wrtp> i should probably change it to use gocheck though, i suppose
[17:17] <wrtp> fwereade, TheMue: right, that's me for the week. see ya monday. have a great weekend!
[17:18] <TheMue> wrtp: Have a nice weekend. gocheck idea is good. I'm already off, only looking at tjose two last proposals.
[17:20] <wrtp> TheMue: one reason for not using gocheck is that it's a highly independent package. i quite like not depending on very much, and the testing package works fine here.
[17:22] <TheMue> wrtp: Yes, you don't need it. It's only that a different maintainer later may ask himself why testing here is different than for other packages.
[17:23] <wrtp> TheMue: the tomb package doesn't use gocheck, for example
[17:23] <TheMue> wrtp: It's an external package, not inside the project.
[17:24] <wrtp> TheMue: yeah. this might become external too.
[17:24] <TheMue> wrtp: But indeed, you don't need it so far.
[17:24] <TheMue> wrtp: In this case it's ok.
[17:25] <fwereade> wrtp, basically LGTM, don't worry about it until monday :)
[17:47] <wrtp> fwereade: minor versions can add features
[18:24] <TheMue> So, last proposal done, have a nice weekend.