[14:21] <ogra_> slangasek, i wiped all calendar data from my phone and re-installed/re-created everything ... how does the TB meeting even end up in any way related to my calendar, i never added it, did someone in canonical do that ?
[19:45] <zyga> mwhudson: hey
[19:45] <zyga> mwhudson: would you mind having a look at a patch for golang
[19:45] <mwhudson> zyga: um ok but i'm not really here (sunday and all)
[19:46] <zyga> mwhudson: haha, I didn't anticipate an instant reply ;)
[19:46] <zyga> :-)
[19:46] <mwhudson> heh
[19:46] <zyga> http://paste.ubuntu.com/24159510/
[19:47] <mwhudson> zyga: seems like a good thing, although that really is a hacky workaround
[19:47] <mwhudson> zyga: you'll also need to cover the non-cgo case
[19:47] <zyga> mwhudson: where is that?
[19:47] <mwhudson> zyga: good question :)
[19:47] <zyga> mwhudson: I think that it is a hack but given the fact that we exec it cannot hurt
[19:48] <zyga> exec will replace all the threads
[19:48] <zyga> too bad the kernel doesn't say anything like that
[19:48] <zyga> EABOUTTOEXEC
[19:48] <zyga> I patched my kernel to see where this fails exactly
[19:48] <mwhudson> oh right yes
[19:49] <mwhudson> how does this manage to have bad effects other than the message?
[19:49] <zyga> and as some people anticipated it fails in kernel/fork.c:1031
[19:49] <zyga> mwhudson: not clear because the thread just doens't get created
[19:49] <zyga> something doens't happen
[19:49] <zyga> I didn't run elaborate tests yet
[19:49] <mwhudson> the non-cgo case is src/runtime/os_linux.go:newosproc
[19:49] <zyga> thanks, checking
[19:50] <mwhudson> so yeah, add the non-cgo case and come up with a good explanation of why this is a problem :)
[19:51] <mwhudson> and it looks ok
[19:52] <zyga> mwhudson: does the execing sample attached in the commit message look sufficient?
[19:52] <mwhudson> zyga: so in the bad case, the exec does not happen?
[19:52] <zyga> mwhudson: and the non-cgo case, is that in os1_linux.go?
[19:52] <zyga> mwhudson: yes
[19:52] <zyga> mwhudson: run that in while ./bug; do :; done
[19:53] <zyga> and it returns quickly with
[19:53] <zyga> runtime/cgo: pthread_create failed: Resource temporarily unavailable
[19:53] <mwhudson> zyga: uh, maybe it's os1_linux.go in older go's, it's os_linux.go in tip
[19:53] <mwhudson> zyga: ah that does sound bad enough yeah
[19:53] <zyga> OK
[19:53] <zyga> will do
[19:53] <zyga> for you it's sunday right?
[19:53] <mwhudson> yeah
[19:56] <zyga> mwhudson: I'll try to patch that part too, never touched the go compiler
[20:01] <zyga> oh, nice sched_yield is already implemented under osyield
[20:01] <zyga> I noticed after I wrote the copy :)
[20:04] <zyga> I think I got it
[20:38] <jtaylor> hm what is the context? sched_yield should not be used
[20:38] <jtaylor> on linux
[20:40] <jtaylor> ah read up, you are only using it a couple times that is probably fine
[20:40] <jtaylor> but don't use it for busy looping
[20:46] <zyga> jtaylor: TBH I'm not yet sure what is the correct solution
[20:47] <zyga> jtaylor: the case is where one thread runs clone/pthread_create while another one runs exefc
[20:47] <zyga> exec*
[21:13] <zyga> mwhudson: it worked :)
[21:13] <zyga> mwhudson: but I think sched_yield is not a good idea indeed, the number of retries is unrelible as well
[21:15] <jtaylor> don't other programs run into this? or does go just produce a larger amount of threads than everything else?
[21:16] <zyga> jtaylor: go is special here, normally you don't just create threads in a program that just execs
[21:16] <zyga> jtaylor: but in go you do
[21:16] <zyga> http://paste.ubuntu.com/24160079/
[21:16] <zyga> run this in a loopc
[21:16] <zyga> comment out the "C" import for another variant
[21:16] <zyga> while ./golang-bug; do :; done
[21:17] <zyga> I have two patches that retry pthread_create/clone up to 1000 times
[21:17] <zyga> I can still reproduce it but far far less frequently
[21:17] <zyga> while terrible I'm inclined to sleep :/
[21:18] <zyga> or patch syscall
[21:18] <zyga> to detect exec and set a flag
[21:18] <jtaylor> might be something the kernel might be interested in, give userspace the ability to detect a real EAGAIN
[21:18] <zyga> jtaylor: it is a real EAGAIN
[21:18] <jtaylor> that one seems more like the typical EINTR case not one where retrying again will result in the same result
[21:18] <zyga> jtaylor: but ... we just care about one of those
[21:19] <zyga> jtaylor: not the resource contention EAGAINs
[21:19] <jtaylor> yes but EAGAIN can be a case where trying again won't help as you hit a limit
[21:19] <zyga> jtaylor: this EAGAIN is also undocumented
[21:19] <jtaylor> the others are not real EGAINs
[21:19] <zyga> jtaylor: well I think this is up to interpretation
[21:19] <zyga> jtaylor: but I worry that given how kernel development is done this will not change (userspace sees this)
[21:19] <jtaylor> yeah, but the kernels input on how this should be handled would be valuable
[21:19] <zyga> jtaylor: and even if, now you have to roll out a fixed kernel everywhere
[21:19] <zyga> jtaylor: yes, I agree
[21:20] <zyga> jtaylor: ideally golang would not do this but I think this is somewhat unavoidable
[21:20] <zyga> jtaylor: perhaps checkinng syscall.Exec is the way actually
[21:20] <zyga> jtaylor: just set a flag (retry on EAGAIN)
[21:20] <zyga> jtaylor: and use that flag to retry
[21:23] <jtaylor> zyga: maybe I don't now go to well to really help
[21:24] <jtaylor> my alarm clocks just rang on mention of sched_yield as that occurs in lots of old numerics software I work with, written back when sched_yield was more or less equivalent to pause and not what it is today
[21:25] <jtaylor> but used here is probably a reasonable workaround until a maybe better option if found