[12:08] <gary_poster> bac benji frankban gmb call in 2
[12:09] <bac> rt
[12:19] <gmb> benji, Thank you kindly: https://code.launchpad.net/~gmb/launchpad/ebt-logging-bug-974586/+merge/101720
[12:21] <bac> sorry i got booted.  trying to rejoin.
[12:46] <bac> gary_poster: starting hangout
[12:46]  * bac tries again
[12:50] <bac> gary_poster: g+ is having fits and won't let me join goldenhorde.  i've sent you an invite
[12:51] <gary_poster> k bac
[12:53] <gary_poster> bac, "this hangout no longer exists"
[12:53] <bac> that's a lie, a damned lie.
[12:54] <bac> i'm in it.  existing.
[12:54] <bac> argh
[12:54] <gary_poster> :-)
[12:54] <bac> gary_poster: ok, the normal hangout now let me in
[12:55] <gary_poster> bac, don't see you in https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhorde
[12:55] <bac> google needs to spend more resources making hangouts work instead of making new whitespace
[13:34] <gary_poster> heh
[13:35] <gary_poster> frankban, https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhordeoneonone whenever you are ready
[14:09] <bac> gary_poster: the newest version of juju solve the bootstrap --constraint problem
[14:09] <gary_poster> great bac
[14:31] <gary_poster> gmb, ready in 10 or less; still on with fb
[14:32] <gmb> k
[14:36] <gary_poster> gmb, https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhordeoneonone whenever ready
[14:45] <bac> gary_poster: this works:  http://pastebin.ubuntu.com/926506/   it includes at line 13 a work-around for a new issue with deploy & --constraints
[14:45] <gary_poster> cool bac will look soon
[15:08] <gary_poster> bac, so you already have everything up, or are you waiting on slave
[15:08] <gary_poster> & is the workaround "68.4"?
[15:09] <bac> gary_poster: the workaround was to do it separately as shown at line 13
[15:10] <gary_poster> oic
[15:10] <bac> gary_poster: waiting on slave
[15:10] <gary_poster> cool bac.  are you looking at 974617?  if so will move card
[15:11] <bac> gary_poster: not yet
[15:11] <gary_poster> btw frankban, when you are ready to work on a parallel testing card, please check with gmb to see if you can pair with him on it
[15:11] <frankban> sure gary_poster
[15:12] <gary_poster> thank you
[15:12] <gary_poster> bac, cool
[15:15] <bac> gary_poster: working the ppa card first
[15:15] <gmb> gary_poster, I think bug 974608 comes under "AAAAAAAAH! Sampledata!"
[15:15] <_mup_> Bug #974608: bugtask-find-similar.txt fails intemittently on paralleltest runs <paralleltest> <Launchpad itself:In Progress> < https://launchpad.net/bugs/974608 >
[15:17] <benji> gary_poster: I created a card for my incremental branch (lp:~benji/testtools/modernize-tsfr) and marked the 974622 card as blocked on my branch being reviewed (jml will do so today, UK time)
[15:18] <gary_poster> great benji thanks
[15:18] <benji> gary_poster: and now the multi-branch work lane is over limit
[15:18] <gary_poster> benji, s'ok.  it's a reasonable view on reality
[15:18] <benji> gary_poster: should I speculatively continue on the 974622 work assuming the branch survives review or do something else?
[15:19] <gary_poster> benji, +1 on speculation IMO
[15:19] <benji> k
[15:19] <gary_poster> agree?
[15:19] <benji> I guess it shouldn't be "blocked" then
[15:19] <gary_poster> yeah.  It will be blocked when you can't do anything on it
[15:20] <frankban> gmb: I am ready to pair on your nice bug, when you are
[15:21] <gmb> frankban, Ah, I think that the one I'm on now isn't suitable - it's a relatively quick fix, AFAICT. We might be better off picking one of the other cards and pairing tomorrow morning instead.
[15:21] <frankban> gmb: coll
[15:21] <frankban> hem cool
[15:21] <gary_poster> frankban, look, slack time! :-P
[15:25] <gary_poster> bac, I need to take lunch now (cat to vet).  I'll ping when back
[15:25] <bac> gary_poster: coincidence.  i was about to go eat chinese food
[15:26] <bac> (speaking of cats)
[15:26] <gmb> lol
[15:26] <gary_poster> heh
[15:26] <bac> this past trip to VN was the first time i saw a restaurant specializing in "Thịt meo".
[15:27] <frankban> I have a suggestion for everyone here who want to create a programming language: please call it in a way you can easily find info using google: a neologism with len >= 4 is strongly  appreciated
[15:27] <bac> frankban: +1
[15:29] <bac> G+ needs a 'translate' button for comments
[15:33] <gary_poster> heh
[15:33] <gary_poster> frankban, I find "golang" works ok
[15:34] <frankban> gary_poster: coll didn't know, I was using "go language"
[16:04]  * gmb -> very late lunch; back later
[16:57] <gary_poster> benji, I saw discussion in -dev with jml.  What he said about the purpose of the forwarding result makes sense (where each test needs to be isolated, not just each call), but I thought that the semaphore was handled explicitly so that would work fine.  As I thought about it more, though, I realized I can't imagine a truly sane story for global tags within a merged set of test streams
[16:58] <gary_poster> the expectations for the meaning of global tags would have to be drastically reduced
[16:58] <gary_poster> though, I'm not really sure what the use cases are for global tags anyway
[16:58] <gary_poster> stepping away; back again soon
[17:25] <benji> gary_poster: (I was eating lunch.) right, I don't think there is a general story that will work with tags without a great deal of development effort, however if we are specific about what TSFR needs to do and what it doesn't do, I think we can get something that continues to work.  The funny part is that it is less and less important to us for our worker ID tagging result wrapper.
[17:27] <gary_poster> benji, less and less important: it was only importat initially as it helped us understand what was going on, and you were using it as a template of sorts, right?
[17:27] <gary_poster> bac, lemme know when you are around and we can pair
[17:27] <bac> gary_poster: i'm here.
[17:27] <benji> right; at this point we could just abandon any work on it and go ahead with what we want
[17:28] <benji> well, not quite: I'm pretty sure as-is TSFW is broken with respect to tags, so we'll have to do *something* about it, but we can more clearly see what that something is once it bites us
[17:30] <gary_poster> benji, if we are sure of a fix, I'd be in favor of applying it.  Otherwise (for instance, the undoing of the global tags) I'm fine with putting it aside.
[17:31] <gary_poster> https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhorde
[17:31] <gary_poster> bac ^
[17:32] <benji> gary_poster: I'm not sure of the fix, but I'm relativly sure there is a problem.  I'm afraid this is one of those things that cross package boundries and will be hard to get just right.
[17:37] <benji> gary_poster: I'm trying to figure out how to move the cards around to represent the current state of the world: I intend on 1) writing an add-worker-id-tag result wrapper, 2) integrating it into ConcurrentTestSuite, 3) fixing the tag bugs I believe we will uncover in TSFR.
[17:38] <benji> 1 and 2 seem to be covered under the card for 974622 and 3 is the other card I created earlier; should they both be in Active/Coding?
[17:41] <gary_poster> benji, have you started work on 3?
[17:41] <benji> gary_poster: I thought I had, but now I see that I haven't really.  Take that for what it is. :)
[17:42] <gary_poster> benji, heh
[17:42] <benji> s/it is/you will.
[17:42] <benji> so I guess it's reasonable to move the card for 3 into the started-but-threw-it-away bucket and make a new card if we need it
[17:43] <gary_poster> ok, benji, I moved that card (#3) back to "Tasks" and changed the title to be less confident ("ThreadSafeForwardingResult doesn't handle tags as well as it could.  We suspect.")
[17:43] <benji> sounds good
[17:44] <gary_poster> benji is it also appropriate to mark 974622 s unblocked
[17:44] <gary_poster> as
[17:44] <benji> gary_poster: oh, on a different note, a nurse is going to be coming by around 2 (they are habitually late) to teach me how to change Katie's catheter dressing for when we're in TN, so that may spoil our one-on-one time
[17:44] <benji> yep, it is unblocked
[17:45] <gary_poster> benji, ok, let me know when/if you are available
[17:45] <gary_poster> and benji, cool thanks, i marked card as unblocked
[18:02] <bac> gary_poster: i now have master and slave coming up.  slave is verified to be m2.4xlarge
[18:03] <gary_poster> bac, excellent
[18:25] <benji> gary_poster: the nurse is already 25 minutes late, what say we start our call?
[18:26] <gary_poster> benji, oh ok!  I was just going to reboot.  I am in https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhordeoneonone
[19:10] <bac> gary_poster: master and slave up and ready after 65 minutes.  i've manually done work-around for bug 974584 and have forced a build
[19:10] <_mup_> Bug #974584: Semaphores cannot be created in lxc container <paralleltest> <Launchpad itself:Invalid> <lxc (Ubuntu):Fix Released by serge-hallyn> <sysvinit (Ubuntu):Confirmed> < https://launchpad.net/bugs/974584 >
[19:10] <gary_poster> yay, bac!
[19:30] <bac> gary_poster: chat?
[19:31] <gary_poster> bac, still talking :-P ready in 5 or so ok?
[19:31] <bac> ok
[19:31] <gary_poster> thx
[19:37] <gary_poster> bac, 4
[19:37] <gary_poster> bah
[19:37] <gary_poster> bac, https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhordeoneonone
[20:17] <gary_poster> benji, do you agree that the following buildbot waterfall indicates that the "teach buildbot to understand subunit in test results to properly report failure numbers in waterfall" card has a bug or problem of some sort? http://ec2-50-19-152-175.compute-1.amazonaws.com:8010/waterfall
[20:18]  * benji looks
[20:18] <gary_poster> benji, would it be worth asking bac to verify that [something or other that you describe] is in the buildbot.cfg of the master?
[20:20] <benji> gary_poster: on the first point, the "no test results" looks discouraging, as I understand it there should be a summary line there
[20:20] <gary_poster> right, so we're on the same page there
[20:21] <benji> gary_poster: on the second part, if I recall correctly, no .cfg change should be required, I just changed the bzrbuildbot Test class to report in the new way.  Checking that the bzrbuildbot/subunittest.py module exists would be a good smoke test
[20:22] <gary_poster> benji, ok cool.  bac, could you look for that on the master when you get a chance?
[20:24] <bac> benji: -rw-r--r-- 1 buildbot nogroup 3255 Apr 12 18:05 /var/lib/buildbot/masters/master/bzrbuildbot/subunittest.py
[20:24] <bac> that what you want to see?
[20:24] <benji> bac: yep, looks good
[20:25] <benji> why we're not seeing a meaningful summary is a mystery to me
[20:30] <gary_poster> benji, dragged that card back to active coding but also pulled your head off it.  We need to get it resolved but it doesn't have to be you
[20:30] <benji> k
[20:30] <gary_poster> I'll send an email to yellow inviting gmb and frankban to look at it tomorrow if they want
[20:31] <benji> my inclination, for whoever looks at it... well, I'll reply to your email with my inclination
[20:31] <gary_poster> :-) k
[20:32] <gary_poster> bac, is it appropriate for me to drag card for bug 974617 to done done?
[20:32] <_mup_> Bug #974617: test_operational_view_integration fails intermittently in parallel tests <paralleltest> <Launchpad itself:In Progress by bac> <Python PGBouncer:Triaged> < https://launchpad.net/bugs/974617 >
[20:32] <bac> yes
[20:32] <gary_poster> cool thx
[20:37] <gary_poster> bac, could you write an email to the yellow list with a description of how you successfully started up your instances today, please?
[20:37] <gary_poster> benji, I just sent the email, but who knows when it will actually make it through the mailing list.
[20:37] <bac> ok
[20:38] <gary_poster> thank you
[20:38] <bac> gary_poster: inside the container:
[20:38] <bac> buildbot@lptests:~$ bzr whoami
[20:38] <bac> Launchpad PQM <launchpad-pqm@canonical.com>
[20:38] <gary_poster> cool
[20:38] <bac> this is *before* i did anything...so not so cool
[20:38] <gary_poster> ...oh...
[20:38] <bac> even so, the celery test fails
[20:38] <bac> so that's pretty confusing
[20:38] <gary_poster> with the same error?
[20:38] <bac> yes
[20:38] <gary_poster> huh
[20:38] <gary_poster> ok
[20:39] <gary_poster> bac, I'll try now to dupe in a local lucid lxc container...
[20:39] <gary_poster> when you ran the test locally you were not in an lxc, right?
[20:43] <bac> correct
[20:46] <bac> this is odd, gary_poster
[20:46] <bac> http://paste.ubuntu.com/927024/
[20:46] <bac> this is inside an lxc container on the slave on ec2
[20:47] <bac> so the cmdline 'bzr whoami' gives results but bzrlib.config doesn't
[20:48] <bac> and if i manually set 'bzr whoami', then bzrlib.config *still* returns None, None
[20:48] <gary_poster> bac, what about the thing that calls _auto_user_id, though
[20:49] <gary_poster> bac config.username()
[20:51] <bac> gary_poster: harder to test as you have to instantiate a Config subclass object
[20:51] <gary_poster> bac, ah ok.  but bac, this fails for me on lxc
[20:51] <gary_poster> so some kind of interaction
[20:51] <bac> running the test again
[20:52] <bac> same same
[20:53] <gary_poster> bac, doing a pdb...
[20:54] <bac> well at least it is easy to reproduce...
[20:54] <gary_poster> y
[21:00] <gary_poster> bac, could you set a pdb in your local environment and see what self.file_name is in the username method when you run that test?
[21:01] <gary_poster> can do it tomorrow
[21:01] <gary_poster> EoD
[21:01] <bac> ok
[21:01] <gary_poster> I get /tmp/tmpOmHAkg/.bazaar/bazaar.conf
[21:02] <gary_poster> which looks like it is a nice test isolation thing
[21:02] <gary_poster> and then...
[21:03] <gary_poster> _get_default_mail_domain() returns None in _auto_user_id
[21:03] <gary_poster> which causes everything to fall over
[21:04] <bac> gary_poster: so it smells like an lxc isolation problem
[21:04] <gary_poster> /etc/mailname is not installed
[21:04] <gary_poster> mm
[21:04] <gary_poster> smells like an "unspoken dependency" problem to me with that
[21:04] <gary_poster> I don't know who is supposed to define /etc/mailname
[21:05] <gary_poster> but I bet it is from a package that we don't install
[21:05] <gary_poster> because we shouldn't have to
[21:05] <gary_poster> could be wrong
[21:05] <gary_poster> ok running
[21:05] <gary_poster> bye all
[21:13] <bac> setting /etc/mailname to 'example.com' allows the tests to fail differently with a TimeoutError