thumperdavecheney: is there a way to use a standard lib function to expand "~/foo" to "/home/tim/foo" if $HOME is /home/tim ?00:07
thumperdavecheney: or is it expected that we do that manually?00:07
=== alexlist` is now known as alexlist
davecheneythumper: sorry I missed your question before01:58
davecheneyi am checking on ~01:58
davecheneythe answer might be 'it's a shell thing'01:58
davecheneybut there might be an expand function01:58
thumperI didn't find one01:59
thumperso wrote one in utils01:59
davecheneyi think that is the correct answer01:59
* thumper is also importing gocheck as gc01:59
* davecheney is trying saucy02:00
davecheneyhmm, no xmir when running under vmware ...02:15
davecheneythumper: wallyworld https://codereview.appspot.com/1081804302:20
davecheney^ trivial02:20
* wallyworld looks02:20
wallyworlddavecheney: jeez, that was a hard one02:21
davecheneythats what she said02:21
* thumper tries to work out why the test is panicing02:23
* thumper sighs...02:25
thumperpanic gone02:26
thumpernow just an error02:26
thumperwallyworld: wow, tweaking the local provider config tests took *much* longer than expected02:32
wallyworldusually does02:32
thumperwallyworld: update coming through02:33
wallyworldcool. i'm trying to debug a bootstrap issue :-(02:33
davecheneylucky(~) % juju bootstrap -v02:38
davecheney2013-07-01 02:38:11 INFO juju ec2.go:144 environs/ec2: opening environment "ap-southeast-2"02:38
davecheney2013-07-01 02:38:11 INFO juju ec2.go:254 environs/ec2: bootstrapping environment "ap-southeast-2"02:38
davecheney2013-07-01 02:38:18 INFO juju tools.go:25 environs: reading tools with major version 102:38
davecheney2013-07-01 02:38:19 INFO juju tools.go:29 environs: falling back to public bucket02:38
davecheney2013-07-01 02:38:20 ERROR juju supercommand.go:234 command failed: use of closed network connection02:38
davecheneyerror: use of closed network connection02:38
wallyworldthumper: could you do me a favour when you have time? could you pull lp:~wallyworld/juju-core/container-constraint and run the tests in ./constraints? They pass locally for me but the bot complains about TestRoundtripYaml and i'm stuffed if i know why02:48
thumperdavecheney: I've not had ap-southeast-2 actually work for me02:49
thumperdavecheney: it would start instances but then say "nope, no instances for you"02:49
davecheneyus-east-1 fails with the same error02:49
davecheneywhy doesn't the error say what failed02:49
davecheneyi'm guessing it fell off the end of the cloud image list02:49
thumperwallyworld: running the tests now02:50
wallyworldthumper: how'd you go running the container tests?03:16
thumper fail in ConstraintsSuite.TestRoundtripYaml03:21
thumper    c.Assert(cons, DeepEquals, t)03:21
thumper... obtained constraints.Value = constraints.Value{Arch:(*string)(0xf8400ccab0), Container:(*instance.ContainerType)(nil), CpuCores:(*uint64)(nil), CpuPower:(*uint64)(nil), Mem:(*uint64)(nil)} ("arch=i386")03:21
thumper... expected constraints.Value = constraints.Value{Arch:(*string)(0xf840056380), Container:(*instance.ContainerType)(0xf840056370), CpuCores:(*uint64)(0xf84006c158), CpuPower:(*uint64)(0xf84006c160), Mem:(*uint64)(0xf84006c168)} ("arch=i386 container=lxc cpu-cores=4096 cpu-power=9001 mem=18000000000M")03:21
wallyworldsigh. that's what the bot says but they pass for me03:22
thumperhmm... that's weird03:22
wallyworldand the code looks fine03:22
* thumper takes a look03:22
thumperwallyworld: yeah man, that looks weird03:26
thumperwallyworld: have you told it how to deserialize instance.ContainerType?03:27
thumperwallyworld: actually WTF?03:27
thumperwallyworld: I see the earlier test for the individual container type03:28
wallyworldthe deserialisation is done in the SetYAML03:28
wallyworldContainerType is a string, so SetYAML is not required except for the fact that it doesn't handle "" properly03:28
thumperwallyworld: ok, found it03:29
wallyworldgreat cause my eyes didn't03:29
thumperwallyworld: if you change "Mem:       uint64p(18000000000)," to something smaller, it passes03:30
thumperdon't ask me why03:30
wallyworldthat value was always there03:30
wallyworldbefore my changes03:30
wallyworldoh well, i'll change to smaller. but i wonder why it works for me03:30
wallyworldthumper: what version og go are you ruinning?03:31
wallyworldi'm on 1.103:31
wallyworldperhaps it's a 1.1 vs 1.03 thing03:31
thumpergo --version and go --help don't work03:32
wallyworldok, that may explain it03:32
* thumper nods03:32
wallyworld1.0.x vs 1.103:32
wallyworldthanks for helping with that!03:32
thumperbut why did the test pass before on the bot?03:32
* wallyworld tweaks and re-approves to land03:32
* wallyworld has no idea03:33
wallyworldthumper: actually, there wasn't a setYAML before03:33
wallyworldso parseuint64 wasn't called03:33
wallyworldmaybe the issue is in there03:34
wallyworldthumper: could you perhaps print any error in setyaml03:34
wallyworldand see what it says03:34
thumperwhere is the setyaml?03:34
wallyworldsince decoding will abort on the first error03:34
wallyworldparseUint64 is in there also03:35
thumper2013-07-01 03:36:10 ERROR juju.constraints constraints.go:199 must be a non-negative integer03:36
wallyworldwell there you go. it is in parseUint6403:36
thumperoh, turning a big int into a smaller int03:38
thumperand causing the signed  bit to flip03:38
wallyworldmaybe there is a AtoUi03:38
wallyworldi'll look03:38
thumperwallyworld: why not use ParseUint64?03:39
thumperin strconv03:39
wallyworldnope, need to call parseUint directly03:39
thumperParseUint(s string, base int, bitSize int) (n uint64, err error)03:39
wallyworldwill make the fix03:39
thumperwant me to try that here?03:39
thumperas I get the error?03:39
wallyworldok, thanks03:39
* wallyworld still wonders how it passed before03:39
thumper-if val, err := strconv.Atoi(str); err != nil || val < 0 {03:41
thumper+if val, err := strconv.ParseUint(str, 10, 64); err != nil {03:41
thumperthat change makes it pass here03:41
thumperyou may find it was calling ParseUint by default03:41
wallyworldif val, err := strconv.ParseInt(str, 10, 64); err != nil || val < 0 {03:41
wallyworldis what we'd want if we want to check for -ve ints, no?03:42
thumpersurely the ParseUint checks for netagives03:42
wallyworldok, parseuint might do that03:42
wallyworldi'll look03:42
wallyworldyes, you get an invalid syntax error03:45
wallyworldnot as nice as our error message03:45
thumperdavecheney: I need to find out the local ip address03:46
thumperdavecheney: localhost isn't good enough03:46
davecheneythumper: of what ?03:46
thumperthat's the question isn't it03:47
davecheneyof the bound interfaces03:47
thumperI'm not entirely sure...03:47
thumperso I want wlan0 or eth0...03:47
thumperbut not lxcbr003:47
davecheneystart with net.Interfaces03:47
thumperor veth*03:47
* thumper looks03:47
davecheneythen inspect the Name property03:48
davecheneyand then call .Addrs() on it03:48
davecheneyi *think* that these were all in 1.003:48
davecheneythey did get a spruce up03:48
davecheneyguess we'll cross that bridge when we get there03:49
thumperwallyworld: btw, found a weird constraints bug03:49
thumper"arch=i386 container=lxc cpu-cores=4096 cpu-power=9001 mem=18000000000M"03:49
thumperwallyworld: it finds the M suffix for mem without removing the digits03:49
thumperbecause it isn't that many megabytes03:50
davecheneythumper: possibly mem is alwasy in megabytes03:50
thumperdavecheney: actually I think you're right03:50
thumperdavecheney: arse03:56
thumperdavecheney: the Name() of them all are "ip+net"03:56
thumperdavecheney: I was hoping to get "eth0" or "wlan0" etc03:56
thumpersorry, Network()03:56
thumperwallyworld: yay, it merged04:02
wallyworldthanks thumper for the help04:03
thumperwallyworld: np04:03
thumperdavecheney: I think Interfaces is what I need04:04
* thumper away for kids activities....04:38
=== thumper is now known as thumper-afk
jamwallyworld_: /wave05:59
wallyworld_oh look at the time06:00
=== tasdomas_afk is now known as tasdomas
jtvjam: thanks for your review...  Looking closer at Environ.Instances() and the StateInfo() loop that's duplicated between providers, I am plagued by doubts about what should happen in the case without a state server.07:32
jtvIf Instances() returns success in that case, then the loop will just keep polling.  But this may be by design ("waiting for bootstrap to finish"), or  it may be an accident ("we always have an instance anyway... except maybe in tests").07:33
jtvIf Instances() returns ErrNoInstances, then StateInfo() will return ErrNoInstances which may or may not be appropriate.07:34
jtv(This is the part people always miss when they say good code needs no comments: computers need to know what to do, humans need to know intent)07:35
jtvIn the OpenStack/EC2 providers, Instances() returns no error in this particular case, so maybe the interface documentation is just not quite right in leaving this particular hole.07:37
rvbajam, mgz: Hi, would you mind updating gwacl in the landing environment?  (It should not break anything, we've just added new methods that are needed for our work on the provider)07:59
jamrvba: updating now07:59
jamrvba: says we are on r146, does that match?08:00
jamjtv: the case without a state server is pretty underdefined, given it only happens during bootstrap until the state is up.08:06
jamjtv: I would guess we are trying to loop until we see the bootstrap node08:06
jamCertainly today if you do "juju bootstrap; juju status" it polls until the state server is up.08:07
jtvAh...  OK, then we'll want to update the spec for Instance().  Thanks.08:07
wallyworld_anyone know how to do this in mongo: select * from foo where not exists (select from bar where bar.id = foo.id) and foo.acme = 'fred'08:07
jamwallyworld_: it looks like there is a $nin operator: http://docs.mongodb.org/manual/reference/operator/08:09
jamthough that requires passing the array08:10
wallyworld_jam: thanks yeah, just saw that myself08:10
wallyworld_it seems that 2 queries are needed :-(08:10
jamwallyworld_: well, you have two queries above08:10
jamit just happens that one is inlined :)08:10
wallyworld_relational dbs do it all in the db08:10
wallyworld_so faster, optimised, etc08:11
jamwallyworld_: well, the query planner can do stuff like do the "and foo.acme" before doing the "select from bar".08:11
wallyworld_and the data does not have to be serialised back to the client etc08:11
wallyworld_s/data/intermediate records08:12
jamwallyworld_: "The second part is impossible. MongoDB does not support joins or any  sort of cross querying between collections in a single query. Querying  from one collection, saving the results and then querying from the  second is your only choice unless you embed the data in the rows  themselves as I mention earlier."08:19
jamIf you can't query between collections, then you can't do what you were asking without 2 requestts.08:19
jamI thought I remembered something about no-cross-joins08:19
jamBut I also wasn't sure if it is was something that changed wrt mongo version updates.08:19
jamwallyworld_: http://stackoverflow.com/questions/12437017/group-based-cross-collection-query-in-mongodb-inc-having-clause08:20
jamseems pretty close to what you were trying to do08:20
jamit certainly was a 'user_id not in ...'08:20
wallyworld_jam: thanks, looking08:36
jamwallyworld_: well, it is saying, you can't really do that, you need to model the data differently.08:37
jamnot exactly what you were hoping for, I'm sure.08:37
wallyworld_jam: sad thing is, doing it in 2 queries crosses transaction boundaries also08:37
wallyworld_seems like a pretty bad limitation of mongo :-(08:37
wallyworld_if you want to avoid contention on document collections, you need to split them up, but then you can't join them again for queries08:40
jamwallyworld_: it sound like there is a per-db lock anyway: http://docs.mongodb.org/manual/faq/concurrency/08:42
jammultiple-readers 1 writer08:42
jamactually similar to SQLite from the sound of it.08:42
wallyworld_hmmm. if the locks are per database, i wonder why issues with doc contention were brought up in reviews08:44
wallyworld_per database seems crazy in any case08:45
jamjcastro: poke08:48
wallyworld_jam: also, doing it in two queries is "dumb" also because if the data set is large, you can't possibly expect to be able to efficiently read the entire collection into memory just to create the array for the second query08:48
wallyworld_recipe for disaster really08:49
=== tasdomas is now known as tasdomas_afk
=== tasdomas_afk is now known as tasdomas
rvbaHi there, would someone be free to review this branch: https://codereview.appspot.com/10788043/? (it has already been reviewed and approved — with a few changes — by Jeroen).09:21
jamrvba: I'm on-call today, I'll try to give it a look.09:52
rvbaThank you jam.09:52
jamrvba: you're thread got split between 2 submissions, right?09:53
rvbajam: no, it's just that every time I run 'lbox propose', a new MP is created.09:53
rvbajam: here is the original MP: https://codereview.appspot.com/10732045/09:54
jamrvba: do you happen to use a different email address for your Reitveld account than your LP one? it does generally work correctly for me on 'lbox propose'.09:59
rvbajam: my Reitveld account is setup with my .c.c email address, my lp account has 2 email addresses associated to it, one of which is the .c.c email address.10:01
jamrvba: so I don't know the Azure model of things. How are you filtering to only include the instances associated with the environment.10:03
jamIs that the "ServiceName: container" stuff?10:03
rvbajam: yes10:03
jamrvba: unrelated to your changes, a general question about Azure. Are you able to run the common tests in environs/jujutest? Or you don't have a local-only Double implementation of the provider to talk against?10:07
jamyou seem to do more of a 'push these responses into the queue, and test that you parse them out again"10:08
rvbajam: I haven't tried to run the common tests.10:08
jamrvba: the idea is to do an interface level test, to ensure that all Environ implementations have the same basic API and semantics.10:09
jamSo far most implement a local-only version, and then also let you run with a "-live" flag to run against an actual remote service10:09
rvbajam: yeah, we don't have a Double implementation.  We just "patch" the http dispatcher.10:09
rvbajam: it's a bit less clean than having a double, but it's much less work.10:10
jamrvba: and you have to rewrite the shared test suite10:11
jamthough it isn't *that* thorough at this point.10:11
jamthough you could still probably use it for live testing.10:11
rvbajam: yeah, it looks like something we could use, once we have all the methods in place, to test the provider live.10:14
jamrvba: The method we used for Openstack was after we got a couple things implemented, hook it up, and then stub out methods we don't support yet10:15
jamso you can get a decent "ratcheting" of passing tests.10:15
rvbaSounds like a good idea… we're still mostly fighting with Azure at this point, but that's something we could do once we have implemented the methods to start a node and bootstrap the env.10:18
rvbaThanks for the review jam.10:29
jamrvba: the goal is that we can test provider-specific things at that level, and then all other tests can just use DummyProvider and not worry about provider-specific differences.10:30
jamrvba: happy to review. It seems mostly about Azure specific bits, which I trust you to know more about than I10:31
jamrvba: actually, your patch was the next in the review queue anyway10:34
jamrvba: you need to set a commit message on: https://code.launchpad.net/~rvb/juju-core/az-list-instances/+merge/17196611:01
rvbaGood point, done.11:18
jamwallyworld_: if you're still around, your patch didn't pass because the regex failed to match.11:59
wallyworld_oh, let me look11:59
* wallyworld_ sighs, fixing12:00
jamwallyworld_: I can't help you too much. "This enourmous string" doesn't match "this enormous regex"12:01
wallyworld_jam: yeah i hate that test. i don't need any help. i've had the misfortune of dealing with it before :-)12:01
mrammhey hey12:40
mrammlong week of travel and now I'm back in michigan.12:41
mrammweekend of travel I guess12:41
=== wedgwood_away is now known as wedgwood
=== tasdomas is now known as tasdomas_afk
jtvDoes anyone have time for a refactoring review?  It turned out rather huge, but on the bright side, lots of code disappears!  https://codereview.appspot.com/1083004417:30
jtvAs soon as this is reviewed, I can go and disappear some more of the stuff.  :)17:30
jamjtv: if you're still here LGTM18:57
jtvThanks jam!19:24
jtvMorning thumper21:07
jtvSay, want to review some code destruction while I sleep?21:07
thumperjtv: you're up late21:07
jtvNot very.  I'm in Europe.  :)21:08
thumperjtv: rack 'em up21:08
thumpermramm: call?21:08
jtvThanks!  Got one vote already: https://codereview.appspot.com/1083004421:08
jtvnn :)21:08
mrammsure we can chat for a few, I have to leave in 20 min though21:08
thumperwallyworld_: hey22:53
thumperwallyworld_: got some time to talk?22:53
* thumper starts a hangout22:54
thumperwallyworld_: https://plus.google.com/hangouts/_/b4eba4c9a6ff830eafe2a6ab79cd49bc4c8d0579?hl=en22:54
=== wedgwood is now known as wedgwood_away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!