/srv/irclogs.ubuntu.com/2014/01/31/#juju-dev.txt

sinzuithumper-afk, wallyworld_ CI cannot make a tarball for r2283. The error here implies Lp or goyaml project is borked. http://162.213.35.54:8080/job/prepare-new-version/960/console00:36
sinzui^ I cannot make sense of it00:36
wallyworld_let me look00:36
=== thumper-afk is now known as thumper
wallyworld_i blame lp00:38
wallyworld_the last commit to trunk was back in nov i think00:38
sinzuiI agree00:39
* thumper is back00:39
sinzuiI can branch that though00:40
wallyworld_sinzui: maybe just retry? could be transient?00:40
sinzuiI did two reties, but I am doing a 3rd since I can see the branch00:41
sinzuihttp://162.213.35.54:8080/job/prepare-new-version/961/console00:41
wallyworld_this time tomb is borked00:41
sinzuiI just branched tomb as well00:42
sinzuiwhat!00:43
sinzuiwallyworld_, This is my rapid rebuild: http://162.213.35.54:8080/job/prepare-new-version/962/console00:43
* sinzui finds an old-fashioned losa00:43
wallyworld_wtf00:43
sinzuiI have have several rebuilds now and the failures are always a random Lp branch00:45
sinzuiwallyworld_, I am going to let CI rest. Lp is have bazaar issues. random 503s for branches that are fine00:49
wallyworld_ok00:49
wallyworld_sinzui: did someone ask you about a 1.17.2 release?00:50
sinzuiYeah00:50
wallyworld_would be easier if lp worked00:50
thumpersinzui: y u break lp?00:50
sinzuiLocal deploy hasn't passed in 7 days though, and and azure is doesn't work either00:51
thumpersinzui: well that is fucked up00:51
thumperI wish I knew what was going on there00:51
thumperI deployed to local on trusty today :-(00:51
wallyworld_it worked?00:52
thumperyes00:52
* thumper think00:52
wallyworld_i must retry that00:52
* thumper tries00:52
thumperwallyworld_:sinzui, yes just deployed the ubuntu charm locally successfully00:57
hazmatdo addresses need to be defined up front when adding machines to state, or can they be detected at runtime by the machine agent?00:58
axwthumper: does rsyslogd create /var/log/juju-${namespace} if it doesn't exist...?01:10
axwI can't see a mkdir for it01:10
thumperaxw: i believe it should01:10
axwmk01:10
hazmatthumper, can proxy values be set at runtime?01:13
thumperhazmat: yes01:13
hazmataxw, do addresses need to be defined up front when adding machines to state via inject machine, or can they be detected at runtime by the machine agent?01:13
hazmatcool01:13
axwhazmat: they can be detected, but if you do that there's no guarantee about which one will be "public"01:13
axwso dns-name in status may show up some interface you don't want01:13
axws/can/will/01:14
axwhazmat: I made a change just yesterday to the manual provider to record the bootstrap-host as public01:15
axwfor that reason01:15
axwthumper: is there a reason not to change MachineConfig.LogDir, and not hard-code the /var/log/juju${namespace} in log/syslog?01:18
hazmataxw, hmm. ic. thanks. the notion of public and private in a private cloud is a bit wonky.01:18
thumperaxw: yes there is a reason01:18
axwhazmat: indeed01:18
thumperaxw: but on a call right now with waigani01:18
axwok01:19
davecheneyhey, did I hear that deploying to hp cloud was fixed ?01:32
sinzuiI added a jackhammer to CI. It will try harder to get the branches from Lp. r2283 is being tested now02:11
hazmatdavecheney, got a minute to help out with a golang cross compile setup?02:20
=== gary_poster is now known as gary_poster|away
hazmatdavecheney, also does go support aarm64?02:21
hazmater t242-servergroup0002s:02:22
hazmater.. aarch6402:22
davecheneyhazmat: ack02:48
davecheneygolang-go does not support aarch6402:48
davecheneyif you are using trusty02:48
davecheneyinstall gccgo-go02:48
sinzuiI have more info on bug 1274780. postfix cannot be setup on the unit because the unit's host name doesn't appear to be sane02:49
_mup_Bug #1274780: Wordpress charm no longer works with local deploy. <charms> <ci> <local-provider> <lxc> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1274780>02:49
hazmatdavecheney, hmm02:57
thumpersinzui: I commented on that bug02:57
thumperaccording to the RFC, dash is a valid hostname character02:57
sinzuiA leading dash is valid?02:58
davecheneyhazmat: yeah, mwd is working on the aarch64 support in gccgo02:58
davecheneythumper: i think it cannot be the first character02:58
davecheneyexperts-exchange.com << all good02:59
davecheney-expertsexchange.com << not good02:59
thumperwe don't have leading dashes though do we?03:01
thumperah... -local-machine-103:01
thumperthe local provider grabs $USER I think03:02
thumperlet me look03:02
davecheneyah03:02
davecheney$USER-local-machine-103:02
thumpernamespace = fmt.Sprintf("%s-%s", os.Getenv("USER"), cfg.Name())03:02
thumpersinzui: how are you running this?03:03
thumperwhy is there no USER env var?03:03
sinzuithumper, CI runs it03:04
thumpersinzui: sure... how?03:04
thumperCI isn't running like a user would03:04
thumperso we need to fix CI03:04
thumperto behave like a user03:04
thumperthat also means having a valid environement03:04
sinzuithumper, This script is calling juju during the test of the env named local http://bazaar.launchpad.net/~juju-qa/juju-core/ci-cd-scripts2/view/head:/deploy_stack.py03:05
thumperok, so what is the environment the script is running under?03:06
davecheneythumper: if juju requires a $USER to be set03:07
davecheneythen it should abort03:07
davecheneybigjools: see what I did there03:07
thumperdavecheney: agreed03:07
davecheneythumper: i think this might also be an issue during cloud init bootstrap03:07
sinzuithumper, jenkins run a bash script that sets JUJU_HOME (which is not entirely honored) then destroys anything left behind by a previous test, then calls ^ that script to bootstrap deploy and status03:07
davecheneysinzui: i'm having problems making saucy environments in HP cloud03:08
davecheneyi heard that there was an known issue03:08
davecheneyis this true ?03:08
thumpersinzui: ok, we can get around this by adding "USER=jenkins-ci" to the environment03:08
sinzuiI have heard that, I have not test it03:08
thumperin the same place you set JUJU_HOME03:08
thumperbut I do agree with davecheney03:09
thumperjuju should abort if it isn't set03:09
sinzuiI think it is set03:09
sinzuijenkins@juju-juju-ci-machine-1:~$ echo $USER03:09
sinzuijenkins03:09
sinzuithumper, the charm only fails in local-deploy. 1.16.5 deploys it locally. So to other clouds, and the charm has not changed in a month03:10
thumpersinzui: but CI has changed03:10
thumperit works fine for me03:10
thumperI can tell you that the process that is running juju doesn't have $USER set03:11
sinzuiYou are a developer with taited setup03:11
thumperthat is why the name starts with a -03:11
thumperit may be because the environment is being overridden when it shells out03:11
thumperI don't know03:11
thumperwhat I do know03:11
thumperis that it isn't set03:11
davecheneysinzui: I can confirm as of 12 hours ago I could not make saucy environments in HP cloud03:12
sinzuiI am forcing a built with an exported USER03:12
thumperis there any way we can hook into the bit that calls juju03:12
thumperto see what $USER is?03:12
sinzui. o (sudo was set to pass the env, sudo is not used anymore)03:14
axwthumper: how did this work before? we used $USER before too...?03:15
sinzuidavecheney, WTF, where is https://region-a.geo-1.objects.hpcloudsvc.com/v1/60502529753910/juju-dist/tools03:15
davecheneyi have no idea03:16
thumperaxw: you may make me go look, but check blame03:16
davecheneyi was using --upload-tools so I didn't notice03:16
thumperI think it looked at SUDO_USER03:16
sinzuidavecheney, typo. I have got to get sleep03:17
axwthumper: pretty sure it used $USER, but I'll double check03:18
thumperaxw: well, it worked before :)03:18
axwthumper: ah, it used $USER and then $SUDO_USER if $USER wasn't set...03:18
sinzuidavecheney, I see the tools are in place, but have you lost os images?03:18
axwor rather, if it was root03:19
* sinzui sees that trusty is not an option on hp03:19
thumpersinzui: and now there is no fallback to check on the SUDO_USER so it is empty03:20
sinzuithumper I award you a star ✶03:20
sinzuiAlas jenkins was so excited by the success it fell over and is reseting the tests, damn it03:20
axwI think we can just fall back to using os/user03:21
axwseems odd not to have $USER set though.03:21
davecheneysinzui: we don't have trusty support anywhere03:23
davecheneybut if you can get a saucy macine03:23
davecheneyyou can upgrade it03:23
sinzuidavecheney, You have a brilliant suggestion. We test trusty on aws and caonistack. azure has it too,03:24
davecheneysinzui: to make trusty work we need two things03:24
davecheney1. juju has to understand trusty as a series03:25
davecheneyi'm looking for the magic constant now03:25
davecheney2. we have to have trusty images avaialble in simple streams03:25
axwthumper: would you like me to fix this one, or are you doing it already?03:25
davecheneyi'm guessing that happens at alpah 103:25
sinzuidave 2. is true alpha two03:25
thumperaxw: fix what?03:25
axwthumper: things not working if $USER not set03:26
thumperaxw: can you?03:26
axwyes, I think so03:26
davecheneycloudinit/sshinit/configure_test.go03:26
davecheney74:var allSeries = [...]string{"precise", "quantal", "raring", "saucy"}03:26
davecheney^ we have a lot of these magic constants03:26
* davecheney fixes03:29
sinzuidavecheney, I think the simplestreams module also defines that, then overrides it with the data from /usr/share/distro-info/ubuntu.csv03:33
davecheneysimplestreams does define trusty, i wonder where other places do not03:36
davecheneysinzui: to clarify, suacy on HP appears broken, should I raise a bug ?03:36
sinzuiyes03:37
thumpersinzui: are you coming to cape town?03:37
sinzuiYes, thumper03:38
davecheneysinzui: ok03:38
sinzuihttp://www.youtube.com/watch?v=P3-f0k620Vg03:39
thumpersinzui: I can't remember, do you drink alcohol?03:39
thumpersinzui: if so, I'll get you a drink for all the CI work :)03:39
thumperif not, then geez03:39
thumperwater03:39
sinzuithumper, Most when trying to use windows, otherwise a beer does me well03:39
* thumper fixes two of the remaining proxy issues03:41
thumperman my triceps hurt today03:41
thumperI thought that boxing gave them a good workout03:41
thumperbut yesterday I took my first Gracie Jiu-Jitsu class03:42
thumpervery different muscles uses03:42
thumperused03:42
thumperaxw, wallyworld_: https://code.launchpad.net/~thumper/juju-core/remove-container-special-proxy/+merge/20415203:56
axwheading out for lunch in a few minutes, will take a look when I get back03:57
wallyworld_looking03:57
* thumper EOWs and goes to pack04:11
dimiternbac, hey, are you still there?07:11
axwwallyworld_: would you mind taking a look at https://codereview.appspot.com/58140044/?07:59
wallyworld_sure07:59
wallyworld_i did see that one and meant to do it but forgot08:00
axwnps08:00
axwI forgot about it myself :)08:00
wallyworld_fwereade: hi, i have a fix for the instance type selection. now selects 1024MB instance by default on canonistack (not 512MB) and constraints which cannot be satisfied eg cpu-cores=9000 are rejected https://codereview.appspot.com/58950043/09:58
fwereadewallyworld_, *sweet*09:58
fwereadewallyworld_, I will try to take a look today but am uncomfortably aware of a bunch of SA prep I am behind on09:59
wallyworld_i added extra tests which illustrate the behaviour, seems to ll work ok09:59
wallyworld_yeah. np. i won't land till you look, can wait till next week09:59
TheMuefwereade: this is the current debug-log after the reimplementation, review by roger and changes based on that10:00
TheMuefwereade: we decided to filter now with regexp, and it's really nice10:01
fwereadeTheMue, I direct the honourable gentleman to my previous response10:01
TheMuefwereade: already played with it yesterday10:01
fwereadeTheMue, cool10:01
TheMuefwereade: eh, that link was missing https://codereview.appspot.com/58510045/ ;)10:02
TheMuefwereade: SA means South Africa?10:04
fwereadeTheMue, yeah10:05
TheMuefwereade: will surely be an interesting trip. but also exhausting. mine next week is shorter, only flying to munich to give talks about juju and golang.10:07
fwereadedimitern, rogpeppe1: standup?10:51
rogpeppe1fwereade: 3 mins10:52
TheMuefwereade, rogpeppe1: could it be that the Environment of state stores the name of the env, but doesn't provide an accessor?11:12
hazmatwhat's provider-safe-mode do?11:18
hazmatoh.. its the  don't kill instances11:20
TheMuefwereade or rogpeppe1: when using the local provider, nothing is written into its all-machines.log. is this a known problem or only a local problem with rsyslog?11:43
* TheMue looks around in the channel. no one here?11:49
TheMuefwereade: ping11:50
TheMuerogpeppe1: ping11:50
rogpeppe1TheMue: we're in the standup11:50
TheMuerogpeppe1: still? ok11:50
TheMuerogpeppe1: thx for info, i'll be patient. ;)11:51
bachi dimitern12:16
dimiternhey bac12:18
dimiternbac, i commented on the bug12:19
bachi dimitern, i've just read your bug report.12:19
baci'll try to isolate the problem and update the bug if i discover anything new.  thanks for looking at it.12:20
dimiternsure12:21
bacdimitern: your testing was on trusty?12:22
dimiternbac, i'm on saucy and I've been deploying on precise12:23
bacdimitern: thanks12:23
natefinchfwereade, rogpeppe1, mgz, dimitern:  Morning guys, sorry I missed the standup.  Overslept.  Zoë was up every hour last night :/12:33
dimiternnatefinch, morning ;)12:33
=== gary_poster|away is now known as gary_poster
* TheMue needs help with local provider13:23
TheMuerogpeppe1: ended standup?13:23
dimiternrogpeppe1, review poke, when you can pls13:24
dimiternTheMue, what's the issue?13:24
TheMuedimitern: the bootstrap environment creates its all-machines.log but the data of machine-0.log isn't written into it13:25
TheMuedimitern: the rsyslog config looks good so far (as I can tell)13:25
dimiternTheMue, maybe syslog is not running?13:26
dimiternTheMue, there's a syslog-port in the env.yaml you can set (and a default is used otherwise)13:27
TheMuedimitern: thx, will try. rsyslogd is running13:27
fwereadedimitern, mgz, rogpeppe1, hazmat: sorry, I need to go out for a bit -- is it better for me to be late, or for us all to push the meeting back 30 mins?13:29
fwereadeplease confer amongst yourselves13:30
dimiternfwereade, i'm ok pushing it 30m13:30
hazmatsounds good13:32
rogpeppe1fwereade: sgtm13:35
dimiternnatefinch, updated https://codereview.appspot.com/54680045/13:42
ahasenackguys, any idea what's up with this error while bootstrapping on aws?13:44
ahasenack2014-01-31 13:42:36 WARNING juju.environs open.go:258 failed to write bootstrap-verify file: cannot make S3 control bucket: A conflicting conditional operation is currently in progress against this resource. Please try again.13:44
ahasenack2014-01-31 13:42:36 ERROR juju supercommand.go:282 provider storage is not writable13:44
rogpeppe1TheMue: pong13:44
rogpeppe1TheMue: (finally! sorry)13:45
natefinchdimitern: looking13:45
TheMuerogpeppe1: do you no any reason why no data is written into all-machines.log at the local provider? it is created by syslog13:46
rogpeppe1TheMue: all-machines.log is at a different file path in the local provider13:47
rogpeppe1TheMue: hence my comment on your review13:47
TheMuerogpeppe1: yeah, sure, I know13:48
TheMuerogpeppe1: I talk about the all-machines.log of the local provider13:48
TheMuerogpeppe1: it is created at its right place below ~/.juju/<env>/log/13:48
TheMuerogpeppe1: but empty13:48
TheMuerogpeppe1: while e.g. machine-0.log contains data (which should be aggregated then)13:49
* TheMue is happy that at least testing with local provider is by far faster than with EC2 :)13:50
natefinchdimitern: if the charm packaging code you put into the apiserver is basically a copy of the code in the charm package.... could we not just reference the charm package code here?13:51
dimiternnatefinch, no, fwereade wants them separate13:51
dimiternnatefinch, most of these are unexported helpers13:52
dimiternnatefinch, and exporting them is not an option13:52
rogpeppe1TheMue: i'm afraid i don't know why - i'd have to look into the code13:52
rogpeppe1TheMue: it might be something to do with the address that containers are putting into the syslog configs13:53
TheMuerogpeppe1: ok, thx, only asked if somebody may made the same experience13:53
rogpeppe1TheMue: i've not looked actually13:53
rogpeppe1TheMue: one mo, i'll see if it happens on my machine13:54
dimiternahasenack, did you retry successfully?13:54
TheMuerogpeppe1: I'll continue investigating13:54
TheMuerogpeppe1: ta13:54
ahasenackdimitern: no13:55
rogpeppe1TheMue: i don't see any all-machines.log in my local env/log file13:55
rogpeppe1s/file/dir/13:55
rogpeppe1TheMue: but that may be an old juju version13:55
rogpeppe1TheMue: let me try with trunk13:55
dimiternahasenack, if retrying doesn't help, file a bug please?13:55
rogpeppe1anyone remember how to fix the gobot when there's an exclusive lock that it's blocked on?13:57
dimiternrogpeppe1, yep13:57
ahasenackdimitern: ok13:57
dimiternrogpeppe1, kill the mongo and delete its sock file from tmp13:57
rogpeppe1it's been moribund since 7am this morning13:57
rogpeppe1dimitern: ok, thanks13:57
TheMuerogpeppe1: I'm using one near trunk. location is $HOME/.juju/$JUJU_ENV/log/all-machines.log13:57
rogpeppe1ha, i wonder what happens if you name your local environment "environments"13:58
hazmatdimitern, hangout for that meeting?14:06
dimiternhazmat, none created, we can reuse the standup one - https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig14:07
hazmatargh.. not allowed to join call.14:07
dimiternfwereade, rogpeppe1, mgz, ^^ (it's in 20ish minutes)14:08
dimiternhazmat, can you do now? i invited you14:08
hazmatdimitern, yup that works thanks14:09
hazmatnice empty room :-)14:09
dimiternhazmat, it's not on for 20 more minutes14:10
ahasenackdimitern: turns out a bug about it already exists: https://bugs.launchpad.net/juju-core/+bug/118357114:11
_mup_Bug #1183571: A conflicting conditional operation... <cmdline> <hours> <juju-core:Triaged> <https://launchpad.net/bugs/1183571>14:11
dimiternahasenack, hmm it seems intermittent and region-specific14:11
rogpeppe1hazmat: https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig?authuser=114:31
rogpeppe1fqw14:31
mrammsinzui: any news on the possibility of 1.17.2 release with the lxc bridge fix and the MAAS metadata fix?14:55
sinzuimramm, I selected r2285 this hour and am writing the release notes14:56
sinzuimramm, Azure is very brittle and I may note that in the note for the release14:56
sinzuiIf r2286 passes this hour, I will use it instead14:57
mrammnice15:01
mrammthanks15:01
mrammsinzui: you rock15:01
TheMuesinzui: he, found a solution for the local debug-log15:02
TheMuesinzui: s/he/ha15:02
TheMuesinzui: if all-machines.log now would contain data too it would be fine15:03
dimiternfwereade, are any of metadata.yaml, config.yaml and revision optional for a charm?15:05
sinzuiTheMue, https://bugs.launchpad.net/juju-core/+bug/1270671 says it is fix committed15:05
_mup_Bug #1270671: local provider all-machines.log is empty  <juju-core:Fix Committed by thumper> <https://launchpad.net/bugs/1270671>15:05
fwereadedimitern, pretty sure config is optional15:05
sinzuiTheMue, It is in today's release15:05
fwereadedimitern, and in theory revision is, sort of15:05
TheMuesinzui: fantastic15:05
dimiternfwereade, ok, so only metadata is mandatory15:06
fwereadedimitern, although that's something we should fix as it comes in, and Ithink the charm package handles that anyway15:06
fwereadedimitern, yeah, missing config is fine, missing revision is sorta fine but we should fix it aswe repack15:06
dimiternfwereade, ok15:06
TheMuesinzui: hmm, it changed into a symlink in /var/log/..., but is still empty *sigh*15:24
TheMuesinzui: just merged trunk15:24
sinzui:(15:25
TheMuesinzui: oh, stop, strange, it's only missing machine-0. machine-1 and unit-... are now added15:29
sinzuiThat is odd15:30
TheMuesinzui: and my debug-log works locally *yippieh*15:31
dimiternrogpeppe1, btw I'm still getting godeps: cannot parse "dependencies.tsv": cannot parse "github.com/loggo/loggo\tgit\t89458b4dc99692bc24efe9c2252d7587f8dc247b\t": unknown VCS kind "git" after pulling the latest godeps15:35
rogpeppe1dimitern: go get -u launchpad.net/godeps15:35
dimiternrogpeppe1, I did15:35
dimiternrogpeppe1, and then I went in $GOPATH/src/...godeps and did bzr pull - same issue15:36
dimitern(it said no revisions to pull)15:36
rogpeppe1dimitern: what revno do you get in the godeps dir?15:36
rogpeppe1dimitern: try rm -r the godeps dir, then go get it again15:36
dimiternrogpeppe1, rev 1015:36
dimiternrogpeppe1, ha! it appears to have moved from ~rogpeppe/godeps/trunk15:37
rogpeppe1dimitern: ah15:37
rogpeppe1dimitern: rev 14 is the latest15:37
dimiternrogpeppe1, yep, now it's fine, thanks15:38
rogpeppe1dimitern: cool15:38
sinzuirogpeppe1, dimitern Do ewither of you have time to review this branch because I am releasing 1.17.2 https://codereview.appspot.com/5905004415:46
rogpeppe1sinzui: looking15:46
rogpeppe1sinzui: LGTM15:47
sinzuithank you rogpeppe115:47
* fwereade needs to be out again for a bit, will be around again most of the evening15:51
rogpeppe1TheMue: you have another review16:12
TheMuerogpeppe1: yep, thx, just answering16:13
gary_posterjamespage, hi.  do you still think you'll have time for quickstart packaging?  if you are going to capetown I suspect that might take some of your time16:16
* gary_poster stepping out for lunch16:18
dimiternTheMue, you have yet another review17:03
jamespagegary_poster, just did the packaging work and uploaded to trusty for archive admin review17:25
TheMuedimitern, rogpeppe1: thanks for your reviews. I added some comments and one for the whole stuff. will continue on monday.17:34
dimiternTheMue, thanks17:38
gary_posterawesome, thanks jamespage!18:02
hatchhey why does -e not work in destroy-environment but it's required everywhere else?18:51
hatchI'm trying to figure out the proper message to tell users of quickstart18:52
hatchbut now the -e is also dependent on the version too?18:52
* hatch hopes it's a bug and he can leave the -e in there18:53
hazmathatch, it used to be.. it was abruptly changed without notice.. its been restored on trunk w/ a deprecation warning18:53
hazmat1.17.2 should have it18:54
hatchhazmat so now everything else will not have -e either eventually?18:54
hazmathatch, just destroy-env for hysterical raisons18:54
hazmatits so people don't accidentally destroy something18:54
hazmatbecause env to act upon  can be looked in one of four ways18:55
hatchok thanks for clarifying18:59
natefinchhazmat, hatch: that was me, sorry.  I wanted the environment name to be mandatory, to make it less likely you'd accidentally destroy-environment on a production environment... however, I didn't think to make -e optional, and should have.19:02
* hatch shakes finger at natefinch 19:05
hatch:P19:05
natefinch:D19:05
natefinchreally, I just wanted to see if anyone was paying attention ;)19:06
hatchit's fine either way really, I just need to know what to tell the user in quickstart19:06
hatchhaha19:06
hazmat.19:23
=== gary_poster is now known as gary_poster|away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!