[00:36] thumper-afk, wallyworld_ CI cannot make a tarball for r2283. The error here implies Lp or goyaml project is borked. http://162.213.35.54:8080/job/prepare-new-version/960/console [00:36] ^ I cannot make sense of it [00:36] let me look === thumper-afk is now known as thumper [00:38] i blame lp [00:38] the last commit to trunk was back in nov i think [00:39] I agree [00:39] * thumper is back [00:40] I can branch that though [00:40] sinzui: maybe just retry? could be transient? [00:41] I did two reties, but I am doing a 3rd since I can see the branch [00:41] http://162.213.35.54:8080/job/prepare-new-version/961/console [00:41] this time tomb is borked [00:42] I just branched tomb as well [00:43] what! [00:43] wallyworld_, This is my rapid rebuild: http://162.213.35.54:8080/job/prepare-new-version/962/console [00:43] * sinzui finds an old-fashioned losa [00:43] wtf [00:45] I have have several rebuilds now and the failures are always a random Lp branch [00:49] wallyworld_, I am going to let CI rest. Lp is have bazaar issues. random 503s for branches that are fine [00:49] ok [00:50] sinzui: did someone ask you about a 1.17.2 release? [00:50] Yeah [00:50] would be easier if lp worked [00:50] sinzui: y u break lp? [00:51] Local deploy hasn't passed in 7 days though, and and azure is doesn't work either [00:51] sinzui: well that is fucked up [00:51] I wish I knew what was going on there [00:51] I deployed to local on trusty today :-( [00:52] it worked? [00:52] yes [00:52] * thumper think [00:52] i must retry that [00:52] * thumper tries [00:57] wallyworld_:sinzui, yes just deployed the ubuntu charm locally successfully [00:58] do addresses need to be defined up front when adding machines to state, or can they be detected at runtime by the machine agent? [01:10] thumper: does rsyslogd create /var/log/juju-${namespace} if it doesn't exist...? [01:10] I can't see a mkdir for it [01:10] axw: i believe it should [01:10] mk [01:13] thumper, can proxy values be set at runtime? [01:13] hazmat: yes [01:13] axw, do addresses need to be defined up front when adding machines to state via inject machine, or can they be detected at runtime by the machine agent? [01:13] cool [01:13] hazmat: they can be detected, but if you do that there's no guarantee about which one will be "public" [01:13] so dns-name in status may show up some interface you don't want [01:14] s/can/will/ [01:15] hazmat: I made a change just yesterday to the manual provider to record the bootstrap-host as public [01:15] for that reason [01:18] thumper: is there a reason not to change MachineConfig.LogDir, and not hard-code the /var/log/juju${namespace} in log/syslog? [01:18] axw, hmm. ic. thanks. the notion of public and private in a private cloud is a bit wonky. [01:18] axw: yes there is a reason [01:18] hazmat: indeed [01:18] axw: but on a call right now with waigani [01:19] ok [01:32] hey, did I hear that deploying to hp cloud was fixed ? [02:11] I added a jackhammer to CI. It will try harder to get the branches from Lp. r2283 is being tested now [02:20] davecheney, got a minute to help out with a golang cross compile setup? === gary_poster is now known as gary_poster|away [02:21] davecheney, also does go support aarm64? [02:22] er t242-servergroup0002s: [02:22] er.. aarch64 [02:48] hazmat: ack [02:48] golang-go does not support aarch64 [02:48] if you are using trusty [02:48] install gccgo-go [02:49] I have more info on bug 1274780. postfix cannot be setup on the unit because the unit's host name doesn't appear to be sane [02:49] <_mup_> Bug #1274780: Wordpress charm no longer works with local deploy. [02:57] davecheney, hmm [02:57] sinzui: I commented on that bug [02:57] according to the RFC, dash is a valid hostname character [02:58] A leading dash is valid? [02:58] hazmat: yeah, mwd is working on the aarch64 support in gccgo [02:58] thumper: i think it cannot be the first character [02:59] experts-exchange.com << all good [02:59] -expertsexchange.com << not good [03:01] we don't have leading dashes though do we? [03:01] ah... -local-machine-1 [03:02] the local provider grabs $USER I think [03:02] let me look [03:02] ah [03:02] $USER-local-machine-1 [03:02] namespace = fmt.Sprintf("%s-%s", os.Getenv("USER"), cfg.Name()) [03:03] sinzui: how are you running this? [03:03] why is there no USER env var? [03:04] thumper, CI runs it [03:04] sinzui: sure... how? [03:04] CI isn't running like a user would [03:04] so we need to fix CI [03:04] to behave like a user [03:04] that also means having a valid environement [03:05] thumper, This script is calling juju during the test of the env named local http://bazaar.launchpad.net/~juju-qa/juju-core/ci-cd-scripts2/view/head:/deploy_stack.py [03:06] ok, so what is the environment the script is running under? [03:07] thumper: if juju requires a $USER to be set [03:07] then it should abort [03:07] bigjools: see what I did there [03:07] davecheney: agreed [03:07] thumper: i think this might also be an issue during cloud init bootstrap [03:07] thumper, jenkins run a bash script that sets JUJU_HOME (which is not entirely honored) then destroys anything left behind by a previous test, then calls ^ that script to bootstrap deploy and status [03:08] sinzui: i'm having problems making saucy environments in HP cloud [03:08] i heard that there was an known issue [03:08] is this true ? [03:08] sinzui: ok, we can get around this by adding "USER=jenkins-ci" to the environment [03:08] I have heard that, I have not test it [03:08] in the same place you set JUJU_HOME [03:09] but I do agree with davecheney [03:09] juju should abort if it isn't set [03:09] I think it is set [03:09] jenkins@juju-juju-ci-machine-1:~$ echo $USER [03:09] jenkins [03:10] thumper, the charm only fails in local-deploy. 1.16.5 deploys it locally. So to other clouds, and the charm has not changed in a month [03:10] sinzui: but CI has changed [03:10] it works fine for me [03:11] I can tell you that the process that is running juju doesn't have $USER set [03:11] You are a developer with taited setup [03:11] that is why the name starts with a - [03:11] it may be because the environment is being overridden when it shells out [03:11] I don't know [03:11] what I do know [03:11] is that it isn't set [03:12] sinzui: I can confirm as of 12 hours ago I could not make saucy environments in HP cloud [03:12] I am forcing a built with an exported USER [03:12] is there any way we can hook into the bit that calls juju [03:12] to see what $USER is? [03:14] . o (sudo was set to pass the env, sudo is not used anymore) [03:15] thumper: how did this work before? we used $USER before too...? [03:15] davecheney, WTF, where is https://region-a.geo-1.objects.hpcloudsvc.com/v1/60502529753910/juju-dist/tools [03:16] i have no idea [03:16] axw: you may make me go look, but check blame [03:16] i was using --upload-tools so I didn't notice [03:16] I think it looked at SUDO_USER [03:17] davecheney, typo. I have got to get sleep [03:18] thumper: pretty sure it used $USER, but I'll double check [03:18] axw: well, it worked before :) [03:18] thumper: ah, it used $USER and then $SUDO_USER if $USER wasn't set... [03:18] davecheney, I see the tools are in place, but have you lost os images? [03:19] or rather, if it was root [03:19] * sinzui sees that trusty is not an option on hp [03:20] sinzui: and now there is no fallback to check on the SUDO_USER so it is empty [03:20] thumper I award you a star ✶ [03:20] Alas jenkins was so excited by the success it fell over and is reseting the tests, damn it [03:21] I think we can just fall back to using os/user [03:21] seems odd not to have $USER set though. [03:23] sinzui: we don't have trusty support anywhere [03:23] but if you can get a saucy macine [03:23] you can upgrade it [03:24] davecheney, You have a brilliant suggestion. We test trusty on aws and caonistack. azure has it too, [03:24] sinzui: to make trusty work we need two things [03:25] 1. juju has to understand trusty as a series [03:25] i'm looking for the magic constant now [03:25] 2. we have to have trusty images avaialble in simple streams [03:25] thumper: would you like me to fix this one, or are you doing it already? [03:25] i'm guessing that happens at alpah 1 [03:25] dave 2. is true alpha two [03:25] axw: fix what? [03:26] thumper: things not working if $USER not set [03:26] axw: can you? [03:26] yes, I think so [03:26] cloudinit/sshinit/configure_test.go [03:26] 74:var allSeries = [...]string{"precise", "quantal", "raring", "saucy"} [03:26] ^ we have a lot of these magic constants [03:29] * davecheney fixes [03:33] davecheney, I think the simplestreams module also defines that, then overrides it with the data from /usr/share/distro-info/ubuntu.csv [03:36] simplestreams does define trusty, i wonder where other places do not [03:36] sinzui: to clarify, suacy on HP appears broken, should I raise a bug ? [03:37] yes [03:37] sinzui: are you coming to cape town? [03:38] Yes, thumper [03:38] sinzui: ok [03:39] http://www.youtube.com/watch?v=P3-f0k620Vg [03:39] sinzui: I can't remember, do you drink alcohol? [03:39] sinzui: if so, I'll get you a drink for all the CI work :) [03:39] if not, then geez [03:39] water [03:39] thumper, Most when trying to use windows, otherwise a beer does me well [03:41] * thumper fixes two of the remaining proxy issues [03:41] man my triceps hurt today [03:41] I thought that boxing gave them a good workout [03:42] but yesterday I took my first Gracie Jiu-Jitsu class [03:42] very different muscles uses [03:42] used [03:56] axw, wallyworld_: https://code.launchpad.net/~thumper/juju-core/remove-container-special-proxy/+merge/204152 [03:57] heading out for lunch in a few minutes, will take a look when I get back [03:57] looking [04:11] * thumper EOWs and goes to pack [07:11] bac, hey, are you still there? [07:59] wallyworld_: would you mind taking a look at https://codereview.appspot.com/58140044/? [07:59] sure [08:00] i did see that one and meant to do it but forgot [08:00] nps [08:00] I forgot about it myself :) [09:58] fwereade: hi, i have a fix for the instance type selection. now selects 1024MB instance by default on canonistack (not 512MB) and constraints which cannot be satisfied eg cpu-cores=9000 are rejected https://codereview.appspot.com/58950043/ [09:58] wallyworld_, *sweet* [09:59] wallyworld_, I will try to take a look today but am uncomfortably aware of a bunch of SA prep I am behind on [09:59] i added extra tests which illustrate the behaviour, seems to ll work ok [09:59] yeah. np. i won't land till you look, can wait till next week [10:00] fwereade: this is the current debug-log after the reimplementation, review by roger and changes based on that [10:01] fwereade: we decided to filter now with regexp, and it's really nice [10:01] TheMue, I direct the honourable gentleman to my previous response [10:01] fwereade: already played with it yesterday [10:01] TheMue, cool [10:02] fwereade: eh, that link was missing https://codereview.appspot.com/58510045/ ;) [10:04] fwereade: SA means South Africa? [10:05] TheMue, yeah [10:07] fwereade: will surely be an interesting trip. but also exhausting. mine next week is shorter, only flying to munich to give talks about juju and golang. [10:51] dimitern, rogpeppe1: standup? [10:52] fwereade: 3 mins [11:12] fwereade, rogpeppe1: could it be that the Environment of state stores the name of the env, but doesn't provide an accessor? [11:18] what's provider-safe-mode do? [11:20] oh.. its the don't kill instances [11:43] fwereade or rogpeppe1: when using the local provider, nothing is written into its all-machines.log. is this a known problem or only a local problem with rsyslog? [11:49] * TheMue looks around in the channel. no one here? [11:50] fwereade: ping [11:50] rogpeppe1: ping [11:50] TheMue: we're in the standup [11:50] rogpeppe1: still? ok [11:51] rogpeppe1: thx for info, i'll be patient. ;) [12:16] hi dimitern [12:18] hey bac [12:19] bac, i commented on the bug [12:19] hi dimitern, i've just read your bug report. [12:20] i'll try to isolate the problem and update the bug if i discover anything new. thanks for looking at it. [12:21] sure [12:22] dimitern: your testing was on trusty? [12:23] bac, i'm on saucy and I've been deploying on precise [12:23] dimitern: thanks [12:33] fwereade, rogpeppe1, mgz, dimitern: Morning guys, sorry I missed the standup. Overslept. Zoë was up every hour last night :/ [12:33] natefinch, morning ;) === gary_poster|away is now known as gary_poster [13:23] * TheMue needs help with local provider [13:23] rogpeppe1: ended standup? [13:24] rogpeppe1, review poke, when you can pls [13:24] TheMue, what's the issue? [13:25] dimitern: the bootstrap environment creates its all-machines.log but the data of machine-0.log isn't written into it [13:25] dimitern: the rsyslog config looks good so far (as I can tell) [13:26] TheMue, maybe syslog is not running? [13:27] TheMue, there's a syslog-port in the env.yaml you can set (and a default is used otherwise) [13:27] dimitern: thx, will try. rsyslogd is running [13:29] dimitern, mgz, rogpeppe1, hazmat: sorry, I need to go out for a bit -- is it better for me to be late, or for us all to push the meeting back 30 mins? [13:30] please confer amongst yourselves [13:30] fwereade, i'm ok pushing it 30m [13:32] sounds good [13:35] fwereade: sgtm [13:42] natefinch, updated https://codereview.appspot.com/54680045/ [13:44] guys, any idea what's up with this error while bootstrapping on aws? [13:44] 2014-01-31 13:42:36 WARNING juju.environs open.go:258 failed to write bootstrap-verify file: cannot make S3 control bucket: A conflicting conditional operation is currently in progress against this resource. Please try again. [13:44] 2014-01-31 13:42:36 ERROR juju supercommand.go:282 provider storage is not writable [13:44] TheMue: pong [13:45] TheMue: (finally! sorry) [13:45] dimitern: looking [13:46] rogpeppe1: do you no any reason why no data is written into all-machines.log at the local provider? it is created by syslog [13:47] TheMue: all-machines.log is at a different file path in the local provider [13:47] TheMue: hence my comment on your review [13:48] rogpeppe1: yeah, sure, I know [13:48] rogpeppe1: I talk about the all-machines.log of the local provider [13:48] rogpeppe1: it is created at its right place below ~/.juju//log/ [13:48] rogpeppe1: but empty [13:49] rogpeppe1: while e.g. machine-0.log contains data (which should be aggregated then) [13:50] * TheMue is happy that at least testing with local provider is by far faster than with EC2 :) [13:51] dimitern: if the charm packaging code you put into the apiserver is basically a copy of the code in the charm package.... could we not just reference the charm package code here? [13:51] natefinch, no, fwereade wants them separate [13:52] natefinch, most of these are unexported helpers [13:52] natefinch, and exporting them is not an option [13:52] TheMue: i'm afraid i don't know why - i'd have to look into the code [13:53] TheMue: it might be something to do with the address that containers are putting into the syslog configs [13:53] rogpeppe1: ok, thx, only asked if somebody may made the same experience [13:53] TheMue: i've not looked actually [13:54] TheMue: one mo, i'll see if it happens on my machine [13:54] ahasenack, did you retry successfully? [13:54] rogpeppe1: I'll continue investigating [13:54] rogpeppe1: ta [13:55] dimitern: no [13:55] TheMue: i don't see any all-machines.log in my local env/log file [13:55] s/file/dir/ [13:55] TheMue: but that may be an old juju version [13:55] TheMue: let me try with trunk [13:55] ahasenack, if retrying doesn't help, file a bug please? [13:57] anyone remember how to fix the gobot when there's an exclusive lock that it's blocked on? [13:57] rogpeppe1, yep [13:57] dimitern: ok [13:57] rogpeppe1, kill the mongo and delete its sock file from tmp [13:57] it's been moribund since 7am this morning [13:57] dimitern: ok, thanks [13:57] rogpeppe1: I'm using one near trunk. location is $HOME/.juju/$JUJU_ENV/log/all-machines.log [13:58] ha, i wonder what happens if you name your local environment "environments" [14:06] dimitern, hangout for that meeting? [14:07] hazmat, none created, we can reuse the standup one - https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig [14:07] argh.. not allowed to join call. [14:08] fwereade, rogpeppe1, mgz, ^^ (it's in 20ish minutes) [14:08] hazmat, can you do now? i invited you [14:09] dimitern, yup that works thanks [14:09] nice empty room :-) [14:10] hazmat, it's not on for 20 more minutes [14:11] dimitern: turns out a bug about it already exists: https://bugs.launchpad.net/juju-core/+bug/1183571 [14:11] <_mup_> Bug #1183571: A conflicting conditional operation... [14:11] ahasenack, hmm it seems intermittent and region-specific [14:31] hazmat: https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig?authuser=1 [14:31] fqw [14:55] sinzui: any news on the possibility of 1.17.2 release with the lxc bridge fix and the MAAS metadata fix? [14:56] mramm, I selected r2285 this hour and am writing the release notes [14:56] mramm, Azure is very brittle and I may note that in the note for the release [14:57] If r2286 passes this hour, I will use it instead [15:01] nice [15:01] thanks [15:01] sinzui: you rock [15:02] sinzui: he, found a solution for the local debug-log [15:02] sinzui: s/he/ha [15:03] sinzui: if all-machines.log now would contain data too it would be fine [15:05] fwereade, are any of metadata.yaml, config.yaml and revision optional for a charm? [15:05] TheMue, https://bugs.launchpad.net/juju-core/+bug/1270671 says it is fix committed [15:05] <_mup_> Bug #1270671: local provider all-machines.log is empty [15:05] dimitern, pretty sure config is optional [15:05] TheMue, It is in today's release [15:05] dimitern, and in theory revision is, sort of [15:05] sinzui: fantastic [15:06] fwereade, ok, so only metadata is mandatory [15:06] dimitern, although that's something we should fix as it comes in, and Ithink the charm package handles that anyway [15:06] dimitern, yeah, missing config is fine, missing revision is sorta fine but we should fix it aswe repack [15:06] fwereade, ok [15:24] sinzui: hmm, it changed into a symlink in /var/log/..., but is still empty *sigh* [15:24] sinzui: just merged trunk [15:25] :( [15:29] sinzui: oh, stop, strange, it's only missing machine-0. machine-1 and unit-... are now added [15:30] That is odd [15:31] sinzui: and my debug-log works locally *yippieh* [15:35] rogpeppe1, btw I'm still getting godeps: cannot parse "dependencies.tsv": cannot parse "github.com/loggo/loggo\tgit\t89458b4dc99692bc24efe9c2252d7587f8dc247b\t": unknown VCS kind "git" after pulling the latest godeps [15:35] dimitern: go get -u launchpad.net/godeps [15:35] rogpeppe1, I did [15:36] rogpeppe1, and then I went in $GOPATH/src/...godeps and did bzr pull - same issue [15:36] (it said no revisions to pull) [15:36] dimitern: what revno do you get in the godeps dir? [15:36] dimitern: try rm -r the godeps dir, then go get it again [15:36] rogpeppe1, rev 10 [15:37] rogpeppe1, ha! it appears to have moved from ~rogpeppe/godeps/trunk [15:37] dimitern: ah [15:37] dimitern: rev 14 is the latest [15:38] rogpeppe1, yep, now it's fine, thanks [15:38] dimitern: cool [15:46] rogpeppe1, dimitern Do ewither of you have time to review this branch because I am releasing 1.17.2 https://codereview.appspot.com/59050044 [15:46] sinzui: looking [15:47] sinzui: LGTM [15:47] thank you rogpeppe1 [15:51] * fwereade needs to be out again for a bit, will be around again most of the evening [16:12] TheMue: you have another review [16:13] rogpeppe1: yep, thx, just answering [16:16] jamespage, hi. do you still think you'll have time for quickstart packaging? if you are going to capetown I suspect that might take some of your time [16:18] * gary_poster stepping out for lunch [17:03] TheMue, you have yet another review [17:25] gary_poster, just did the packaging work and uploaded to trusty for archive admin review [17:34] dimitern, rogpeppe1: thanks for your reviews. I added some comments and one for the whole stuff. will continue on monday. [17:38] TheMue, thanks [18:02] awesome, thanks jamespage! [18:51] hey why does -e not work in destroy-environment but it's required everywhere else? [18:52] I'm trying to figure out the proper message to tell users of quickstart [18:52] but now the -e is also dependent on the version too? [18:53] * hatch hopes it's a bug and he can leave the -e in there [18:53] hatch, it used to be.. it was abruptly changed without notice.. its been restored on trunk w/ a deprecation warning [18:54] 1.17.2 should have it [18:54] hazmat so now everything else will not have -e either eventually? [18:54] hatch, just destroy-env for hysterical raisons [18:54] its so people don't accidentally destroy something [18:55] because env to act upon can be looked in one of four ways [18:59] ok thanks for clarifying [19:02] hazmat, hatch: that was me, sorry. I wanted the environment name to be mandatory, to make it less likely you'd accidentally destroy-environment on a production environment... however, I didn't think to make -e optional, and should have. [19:05] * hatch shakes finger at natefinch [19:05] :P [19:05] :D [19:06] really, I just wanted to see if anyone was paying attention ;) [19:06] it's fine either way really, I just need to know what to tell the user in quickstart [19:06] haha [19:23] . === gary_poster is now known as gary_poster|away