| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| === zz_Jacky_ is now known as jackyalcine | ||
| === jackyalcine is now known as Jacky_ | ||
| infinity | pitti: Hrm. It seems like a misfeature that pkgstripfiles removes files from -doc packages. | 03:33 |
|---|---|---|
| infinity | pitti: (In the case I'm looking at, it strips all the upstream changelogs out of glibc-doc) | 03:33 |
| infinity | pitti: Opinions? I'm thinking of just making clean_upstream_changelogs() return if the package is a -doc. | 03:35 |
| === Jacky_ is now known as zz_Jacky_ | ||
| BenC | infinity: well, I am using -proposed…are the buildd's not? | 03:56 |
| infinity | BenC: They are, yes. You can see in the build logs exactly what it's doing. | 03:58 |
| infinity | BenC: And replicating it should be easy. | 03:58 |
| BenC | infinity: I ran the exact same apt-get command as in the chroot, and I didn't see the dep problems | 03:58 |
| infinity | BenC: Anyhow, haskell in proposed is deep in painful transition land, I wouldn't hold out much hope for any of it moving soon (given that many packages still need upstream fixes) | 03:58 |
| infinity | BenC: Then you didn't have the same sources.list and an up-to-date apt-cache. I can reproduce it trivially here. | 03:59 |
| BenC | infinity: I ran it from a pristine chroot with updated raring packages, using an updated cache | 03:59 |
| BenC | *up-to-date | 04:00 |
| infinity | BenC: Just reproduced here again, to make sure I wasn't crazy. | 04:05 |
| infinity | http://paste.ubuntu.com/1603451/ | 04:05 |
| infinity | Anyhow, like I said, we need to sort out that transition anyway, and this will all fall out of that. | 04:06 |
| BenC | Works for me… | 04:11 |
| infinity | Not sure what to say. :P | 04:14 |
| infinity | Your computer is special? | 04:14 |
| BenC | I meant, leaving it alone and letting it all sort itself out "works for me" :) | 04:16 |
| infinity | Well, that's kinda the only option right now anyway. | 04:16 |
| infinity | Either that, or I remove all of *ghc* *haskell* from proposed. But I see no reason to not just do the transition, it'll just take some effort (most of which needs to happen in Debian right now). | 04:17 |
| quadrispro | hi everybody | 04:24 |
| quadrispro | slangasek, bug #1110326 is done now | 04:28 |
| ubottu | bug 1110326 in portmidi (Ubuntu Quantal) "portmidi 1:200-0ubuntu1.12.04.1 is broken" [Undecided,New] https://launchpad.net/bugs/1110326 | 04:28 |
| quadrispro | cheerio | 04:33 |
| === zz_Jacky_ is now known as jackyalcine | ||
| === jackyalcine is now known as Jacky_ | ||
| === Jacky_ is now known as jackyalcine | ||
| === jackyalcine is now known as zz_Jacky_ | ||
| === zz_Jacky_ is now known as jackyalcine | ||
| === jackyalcine is now known as zz_jackyalcine | ||
| lifeless | infinity: you've climbed around eglibc before, right? wth is close defined ? | 07:15 |
| infinity | lifeless: man 2 close? | 07:23 |
| lifeless | infinity: the source, not the header :) | 07:23 |
| lifeless | infinity: I've found a stub version, and a hurd version | 07:23 |
| lifeless | infinity: I seem incapable of finding the common case version :( | 07:24 |
| sarnold | that's my usual experiences with the eglibc sources :/ | 07:24 |
| lifeless | sarnold: I'm glad I'm not alone. | 07:24 |
| mjt | it is a syscall lifeless | 07:26 |
| mjt | there's no source for it in glibc | 07:26 |
| infinity | ^ | 07:26 |
| mjt | they do some syscall magic in one go for all syscalls in a generated file | 07:26 |
| lifeless | oh; I had assumed it would still need a thunk | 07:26 |
| mjt | yeah it does | 07:27 |
| mjt | but these are auto-generated | 07:27 |
| mjt | i don't know the process. this reminds me of a file named SYSCALLS somewhere in the sources | 07:27 |
| lifeless | ./sysdeps/generic/sys/syscall.h perhaps | 07:28 |
| lifeless | mjt: thank you, tells me what I needed anyhow ;) | 07:29 |
| infinity | lifeless: Well, more likely to be in linux sysdeps (or arch-specific), but... | 07:29 |
| infinity | close - close Ci:i __libc_close __close close | 07:30 |
| infinity | Actually, sysdeps/unix/syscalls.list for the simple ones like close() | 07:31 |
| lifeless | infinity: thanks! | 07:31 |
| * lifeless is now grabbing linux source... | 07:31 | |
| sarnold | aha, thanks :) | 07:31 |
| lifeless | I'm digging into why close(1) blocks when doing dd to a block device. | 07:31 |
| infinity | lifeless: Standard answer. | 07:32 |
| lifeless | infinity: whats that ? | 07:34 |
| infinity | lifeless: "Because it hates you" was the implied answer (other answers are even less CoC friendly). :P | 07:35 |
| lifeless | infinity: ah! naturally. | 07:35 |
| infinity | lifeless: But in all seriousness, happy hunting. That's getting too close to the vfs layer for my liking. There's a reason I sit firmly on this side of syscalls. :) | 07:35 |
| lifeless | infinity: hah :). So, you'll love it when I say 'and then I need to establish behaviour with iscsi in the mix | 07:36 |
| infinity | lifeless: I suspect you'll find that, short of fundamental constraints imposed by VFS features and/or bugs, behaviour like what you're looking at may be entirely filesystem-specific. | 07:37 |
| infinity | lifeless: So you may be better off starting with iscsi and working backward. | 07:37 |
| infinity | lifeless: But ymmv, maybe blocking close is a fundamental vfs assumption. | 07:38 |
| lifeless | infinity: well, there is no fs in play | 07:38 |
| infinity | Oh, right, block device. | 07:38 |
| lifeless | infinity: I need to understand both local behaviour, and the behaviour with tgtd etc | 07:38 |
| infinity | Well, one layer lower, then. Fun. | 07:38 |
| lifeless | so that I can decide how to ensure The Right Thing happens. | 07:39 |
| infinity | Sounds like a fantastic learning experience for someone who's not me. Enjoy. :P | 07:39 |
| lifeless | infinity: it is actually. | 07:39 |
| mjt | if you want to know why dd blocks this way, do a kernel stack dump first | 07:39 |
| mjt | if strace tells you it is in close, that'll be in kernel not in glibc | 07:39 |
| infinity | (My usual response to kernel questions is "whine at a kernel engineer") | 07:40 |
| lifeless | mjt: can you do that without rebooting ? | 07:40 |
| mjt | even for a non-kernel-engineer it might be sometimes enlightening, since the stack sometimes show functions which suggest something. Somethingh like, "wait_on_alloc_pages" | 07:41 |
| lifeless | mjt: sorry, s/rebootiong/rebuilding/ | 07:41 |
| mjt | lifeless: echo t > /proc/sysrq_trigger and see your dmesg (kern.log) -- search the hung dd pid in there | 07:41 |
| mjt | (it will generate a ton of info, trace of all running tasks!) | 07:42 |
| lifeless | thank | 07:42 |
| lifeless | interestingly, while I had a blkid task caught by 'hung for > 120s' that didn't seem to catch the dd; perhaps 1G isn't enough to block that long. | 07:43 |
| mjt | and if you don't understand what's in there, -- this is the basic info any kernel engineer will ask you about, so if you include that stack trace right away it'll save you a roundtrip | 07:43 |
| mjt | wait. | 07:43 |
| mjt | you dd sleeping on close -- close of the file you just wrote? | 07:44 |
| mjt | huge file? | 07:44 |
| lifeless | mjt: block device | 07:44 |
| mjt | the same thing | 07:44 |
| mjt | is it the thing to which you just wrote a ton of gigs of data? | 07:44 |
| lifeless | specifically, dd of=/dev/sdb1 ... | 07:44 |
| lifeless | and yes. | 07:44 |
| mjt | so it is kinda expected it blocks | 07:45 |
| lifeless | close is blocking until nr_writeback hits 0 | 07:45 |
| mjt | yeah | 07:45 |
| lifeless | mjt: why? No fsync/fdatasync/sync calls. | 07:45 |
| mjt | well. this is grey area indeed. but it is the only way to know if the writes succeeds or not. | 07:46 |
| mjt | much better way to write to block devices is to use of=direct | 07:46 |
| mjt | er | 07:46 |
| mjt | oflag=direct | 07:46 |
| mjt | and a reasonable blocksize (say, bs=1M) | 07:46 |
| mjt | this way write will be smooth, it wont affect the rest of the system, it wont trash your block cache | 07:47 |
| mjt | and it will be significantly faster too | 07:47 |
| lifeless | mjt: well, yes, and on the initiating machine I will be doing that. | 07:48 |
| lifeless | mjt: but I'm mainly trying to establish all the parameters righ tnow | 07:48 |
| lifeless | mjt: the bigger picture goal is to ship a GB or so from machine A to a block device on machine B, as fast as the network will handle it, without waiting for it to hit disk - and then for machine B to validate the data itself one way or another, flush to disk and reboot. | 07:49 |
| StevenK | lifeless: nc ? | 07:50 |
| lifeless | mjt: currently using iscsi for that, but I'm suspecting a userspace process on the target is better suited | 07:50 |
| lifeless | as it can orchestrate things more directly | 07:50 |
| lifeless | StevenK: indeed | 07:50 |
| lifeless | StevenK: I inherited the current code. | 07:50 |
| lifeless | mjt: would nonblocking cause close to not block (but not arbitrarily cancel any of the IO ?) | 07:51 |
| infinity | StevenK: Heh. It never ceases to amaze me the number of useful things one can abuse a concept as simple as nc for. | 07:51 |
| StevenK | I would have thought ISCSI would be a little heavy handed, TBH. | 07:51 |
| mjt | lifeless: just try it | 07:51 |
| lifeless | StevenK: you can also use rsync with the block device patch, but it doesn't scale to Big Disks, which some folk mighy use. | 07:51 |
| StevenK | infinity: Haha. nc is wonderful, I find myself reaching for it more | 07:51 |
| mjt | lifeless: but still, direct i/o is way to go there, since it will be faster than using buffer cache | 07:52 |
| lifeless | mjt: heh - so I can; the main concern is knowing what the guarantees are, not the 'what happens when I try' | 07:52 |
| StevenK | lifeless: nc and tar? Nice, simple and fast? | 07:52 |
| lifeless | mjt: our network is faster than the target hard disk, buffer cache is a big win :) | 07:52 |
| infinity | StevenK: *nod*... It's very classic UNIX in it's simple tool that can be slotted into thousands of usecases. | 07:52 |
| lifeless | mjt: [for the whole story] | 07:52 |
| mjt | lifeless: and your app on node B will validate what? content of buffer cache? | 07:53 |
| lifeless | mjt: so, ideally O_DIRECT on the source machine avoids buffering there, but we want buffer cache on the target. | 07:53 |
| lifeless | mjt: do a sync, then O_DIRECT read of the partition I think. | 07:53 |
| mjt | and how is the size of data compared with the amount of RAM on B? | 07:54 |
| lifeless | mjt: tiny fraction | 07:54 |
| mjt | will whole thing fit in ram? | 07:54 |
| lifeless | mjt: typical cases are 500M->1G data, RAM is - well, lots more than that, I can't say more :) | 07:54 |
| mjt | speaking of guarantees, -- as i said it is a grey area already even without O_NONBLOCK | 07:54 |
| mjt | and may depend on the underlying stack, i dunno | 07:55 |
| lifeless | yeah, thus my climbing through source rather than simply asking the lazyweb : though I sure do appreciate the info you have shared. | 07:56 |
| mjt | (i didn't know close(block device) waits till writes are complete, until now) | 07:56 |
| mjt | and this is definitely in kernel. not glibc :) | 07:56 |
| mjt | lifeless: btw, how your source disk speed compared with the target disk speed? :) | 08:00 |
| lifeless | mjt: don't care, source will be cached. | 08:00 |
| mjt | with O_DIRECT? | 08:00 |
| lifeless | mjt: O_DIRECT on writes, not reads; oflag=direct | 08:01 |
| mjt | ok. count me officially confused. ;) | 08:01 |
| mjt | so it looks like the only thing you need is to stop close() from blocking. that's kernel question. | 08:02 |
| lifeless | mjt: indeed | 08:02 |
| lifeless | mjt: well, I need to get the behaviour when iscsi is in the picture pinned down. | 08:03 |
| lifeless | mjt: but thats just a matter of time :) | 08:04 |
| === zequence_ is now known as zequence | ||
| lifeless | mjt: infinity: for your interest; | 08:57 |
| lifeless | with tgtd serving out a block device, and dd to that via a regular iscsiadm connection | 08:57 |
| lifeless | oflag=direct has minimal impact on performance, either way the close() at the end waits for all data to have traversed the network. but it does not wait for it to hit disk at the far end - it buffers quite happily. | 08:58 |
| lifeless | oflag=direct does of course avoid wasting the senders buffer cache | 08:58 |
| lifeless | this is without the sync or direct flags set in tgtd | 08:59 |
| lifeless | could do that, but it would actually be counter what we want... which is to clear the network as quickly as possible - we wouldn't be able to tell when dd finished vs when teh disk finished. | 08:59 |
| lifeless | so I'll need to do something fugly on the far end to tell (or switch to nc :)) | 09:00 |
| lifeless | StevenK: ^ | 09:00 |
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| === Quintasan_ is now known as Quintasan | ||
| Quintasan | hello | 11:07 |
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| === zz_jackyalcine is now known as jackyalcine | ||
| === jackyalcine is now known as Jacky_ | ||
| tomreyn | hi | 15:57 |
| tomreyn | is it possible, as an upstream, to get access to apport submitted error reports for packages in ubuntu? if so, what's the process? | 15:58 |
| === Jacky_ is now known as zz_jackyalcine | ||
| === zz_jackyalcine is now known as jackyalcine | ||
| Elbrus | tomreyn: in principle, anybody can subscribe to a package | 16:43 |
| Elbrus | in launchpad, go to the bugs in the package | 16:43 |
| Elbrus | click link "subscribe to bug mail" | 16:44 |
| tomreyn | Elbrus: i don't think bug reports still get created for apport submissions | 17:06 |
| tomreyn | Elbrus: i'm referring to crash logs | 17:07 |
| Elbrus | tomreyn: sorry, didn't know that | 17:07 |
| Elbrus | I thought they did | 17:07 |
| tomreyn | it feels like canonical wants more information to be internal only now than in the past. | 17:08 |
| tomreyn | of course this can be just an architectural change and this information which used to be publically accessible just has not yet been made accessible again (about a year after the change took place), or i just lack the knowledge on how to access it. | 17:09 |
| penguin42 | https://errors.ubuntu.com/ has teh stats but none of the detail | 17:15 |
| stgraber | tomreyn: the new workflow is essentially: crash => upload to errors.ubuntu.com => when we notice it happens often => file a public bug report and link it | 17:17 |
| stgraber | tomreyn: so fixes will still be linked to bug reports, but we won't have bug reports with thousands of duplicates anymore | 17:17 |
| penguin42 | stgraber: I think tomreyn's original question was what happens to an upstream if they're interested in watching for bugs on their package | 17:18 |
| stgraber | tomreyn: for privacy reason the details of the crashes on errors.ubuntu.com are currently restricted to Canonical employees but I believe this is being worked on so that other people may access those provided they sign an NDA | 17:18 |
| stgraber | penguin42: upstreams should monitor Launchpad. Any frequent crasher will be forwarded there once we know for sure it's a bug. | 17:20 |
| penguin42 | stgraber: Yeh that's a bit restricting for them, but I can understand the privacy stuff to want to filter peoples coredump stuff - but backtraces without the data should be safe | 17:21 |
| tomreyn | inded, we would never sign an SDA. we would, however, be quite happy to have access to any data which can be made available without endandering the submitters' privacy. | 17:23 |
| tomreyn | and i believe this is very much automatable. | 17:23 |
| stgraber | penguin42: they are most of the time, the problem is that they aren't 100% of the time. We can't know that every single piece of software in the archive won't show user credentials in an exception for example | 17:24 |
| tomreyn | sure, it does take a bit of time, but not too much | 17:24 |
| stgraber | tomreyn: then what you want to look at is Launchpad, because that's where we forward those crashes after they've been looked at and cleaned up | 17:24 |
| tomreyn | s/SDA/NDA/ | 17:24 |
| penguin42 | stgraber: You could show purely the sequence of symbols/offsets in the backtrace with NO data | 17:25 |
| tomreyn | right. and that's appreciated. from my experience the process of reviewing crash logs could take anywhere between a day and a year when they were public, so, if the situation is the same now (but I would assume it is worse since fewer people will likely have access to these, so fewer reviewers, too), i'm not sure whether this is worth too much for upstreams. | 17:27 |
| stgraber | penguin42: true, rendering the backtrace mostly useless in the process (at least in python, which we happen to have a lot of, the actual exception text is what's most relevant, the problem is, it's also the bit that can contain any string including exposing potential private information) | 17:27 |
| tomreyn | do you have statistics on this? | 17:27 |
| tomreyn | to not be misunderstood, i do understand how the users' privacy must come first, and I actually appreciate this. | 17:29 |
| penguin42 | stgraber: Ah, I'm more of a C person, just the symbol stuff would often be useful there | 17:29 |
| * tomreyn seconds penguin42 | 17:29 | |
| stgraber | penguin42: right, C should be fairly easy to automatically forward, just retrace, send the backtrace and don't attach the core | 17:29 |
| penguin42 | stgraber: Yeh, possibly missing out the parameter data | 17:30 |
| tomreyn | stgraber: are you in a position to stifle a discussion on this internally? if so, this would be great for us and many, many other free software projects in Ubuntu. | 17:34 |
| penguin42 | I don't think stifle is the required op here | 17:35 |
| tomreyn | whoops, right, i meant trigger :) | 17:35 |
| tomreyn | sorry, not an english native speaker here | 17:35 |
| penguin42 | tomreyn: start | 17:35 |
| tomreyn | sounds good | 17:35 |
| stgraber | tomreyn: well, the right people have been highlighted above and I'm sure they'll see that discussion, so that's pretty much done. Generally, we definitely want more people to have access to that data, but we need to process step by step, the first and easiest one appears to be the NDA for those who're fine with that, and hopefully finding ways of exposing relevant data without any user data at a later point | 17:37 |
| tomreyn | It's good to know that the relevant people will be aware now how this is a strong nice-to-have for open source developers now. I can see how an NDA can seem the most logical solution for a business which deals with other businesses or organisations. I would bet, however, that signing an NDA is not an option for a large share, possibly the majority, of those who develop the software which forms Ubuntu. | 17:42 |
| tomreyn | So if Canonical wants to get a step closer to the free / open source software community again without risking anything on the other and and investing very little this could be a good way. | 17:43 |
| penguin42 | agreed | 17:43 |
| tomreyn | (and something one could blog about) | 17:43 |
| === jackyalcine is now known as Jacky_ | ||
| === Jacky_ is now known as jackyalcine | ||
| === jackyalcine is now known as Jacky_ | ||
| === Jacky_ is now known as zz_Jacky_ | ||
| === Ursinha is now known as Ursinha-afk | ||
| === zz_Jacky_ is now known as jackyalcine | ||
| === jackyalcine is now known as Jacky_ | ||
| === Jacky_ is now known as jackyalcine | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!