=== wedgwood is now known as wedgwood_away === vibhav is now known as Guest48005 === Guest48005 is now known as vibhav === jono is now known as Guest19637 [03:36] cjwatson, last time uds was in Orlando and I met you in person you were on a laptop, but watching the videos this uds, I wonder... do you use a "real fucking keyboard"? I keep hearing loud key clicks and it seems to be you. [03:44] psusi: He just types very violently. === marrusl is now known as marrusl_afk [04:14] ev: ping, have a question related to whoopsie [05:48] Good morning [05:49] Good morning [06:10] slangasek, online? [06:10] doko: heya [06:17] slangasek: AFAIK the pkexec patch in http://launchpadlibrarian.net/105173889/policykit-1_0.104-2_0.104-2ubuntu1.diff.gz should be upstreamed, too, right? [06:17] or is that Debian/Ubuntu specific somehow? [06:18] pitti: hmm, I can't think of any way that it's Debian/Ubuntu-specific. I'd say it should be upstreamed, yeah [06:23] slangasek: do you want to send that bugs.fd.o-wards with some background, or shall I file it and CC you in case there's further questions? [06:56] slangasek, stgraber: what did you do to make logind create the user cgroups properly? when I install logind and the pam module, logind errors wit "Failed to create cgroup name=systemd:/user/joe: No such file or directory", and passes that error to the PAM module which then aborts, too [06:58] (I don't see any relevant failure in strace) [07:44] good morning [07:53] Can someone illuminate how gstreamer0.10-plugins-ugly managed to get multiarched before one of its dependencies? (libcdio) From what I can tell it's never been coinstallable, and I'm wondering how such a thing could get past -proposed [07:58] the -ugly === tkamppeter_ is now known as tkamppeter [08:13] tkamppeter_: hey, cups is not cleaning up it's old conffiles, like with the introduction of cups-daemon I now have two logrotate files for cups, and cron is spamming me because of that. there's also /etc/pam.d/cups left over [08:20] tjaalton: please report a bug against cups-daemon on Debian/experimental, that's certainly valid there. [08:23] tjaalton: ah, wait, that's handled by 666367fc1d9fcdf18b71abf2254f8c12edc67f6b already, by tkamppeter [08:25] OdyX: oh, cool === slacker_1l is now known as slacker_nl [08:45] tjaalton: the problem I guess is that Ubuntu got intermediary versions [08:49] anyone know why the samba update hasn't been accepted into precise-updates? its two weeks past the date on https://launchpad.net/ubuntu/precise/+queue?queue_state=1&queue_text=samba === smb` is now known as smb [09:00] hi, can anyone tell me if there is a way to test why webapp icons are not loaded ? [09:01] it a common problem i have hit, where i put .png files on the system but just get a placeholder image on the left bar, [09:02] i have no idea why, or if this is logged some where from what i can tell the paths are all correct, and i have compared against other webapps like the bbc one but mine does not work [09:03] stgraber, slangasek: ah, we need to mount a tmpfs on /sys/fs/cgroup, so that logind can write into it [09:03] stgraber, slangasek: I guess that woudl go into /lib/init/fstab ? [09:05] pabs3: I was about to apologize for not reviewing it, then realized I uploaded it. [09:07] infinity: yeah, its uploaded but not accepted [09:43] looks like the hotkey for enabling wifi on a dell laptop I have doesn't work on raring, and it defaults to being off. is there a way to fake the key so I could enable wifi again? [09:44] hmm according to dmesg the key works.. [09:45] oh hell, now it's working :P [09:45] i'll get me coat.. [10:01] slangasek, stgraber: with http://paste.ubuntu.com/5592725/ it works OOTB, but needs a fix to udev to suppress udev-acl; we could also add /sys/fs/cgroup/ mounting to /lib/init/fstab, WDYT? [10:08] # systemd replaces udev-acl entirely, skip if active [10:08] TEST=="/sys/fs/cgroup/systemd", TAG=="uaccess", GOTO="acl_end" [10:09] pitti: that's in the latest 70-udev-acl.rules [10:09] mbiebl: yes I know, but with above dynamic mounting in logind's upstart job /sys/fs/cgroup/ appears too late [10:09] mbiebl: that's what I mean: if we mount it in /lib/init/fstab, it's early enough and that rule works [10:09] mbiebl: and if we keep the late mounting, I'll drop the TEST from that rules [10:10] couldn't you just mount the cgroups in logind itself [10:10] same thing -- udev coldplugging happens before, and coldplugged devices get tagged with both [10:11] also, systemd already mounts it, I don't want to create incompatibilities; so that patch only mounts it if it isn't already [10:28] pitti: maybe we could convince rleigh to add similar bits to mountkernfs.sh [10:28] Laney, thanks for fixing oslo-config [10:28] nps [10:28] mbiebl: systemd mounts that directory by itself, though [10:28] sure [10:29] mbiebl: so it only becomes relevant with a split, and then we need to check that systemd's mounting doesn't collide with that [10:29] im talking about the sysvinit+logind part [10:29] mbiebl: oh, mountkernfs.sh wouldn't be run by systemd I guess [10:29] correct [10:29] so if upstart does the mount in /lib/init/fstab [10:29] sysvinit in mountkernfs.sh [10:30] then everything should work indeed [10:30] and systemd itself [10:30] all should be fine [10:30] (in theory) [10:40] cjwatson: hi, I just did a sanity check on the kubuntu packageset and noticed that nepomuk-widgets is missing there even though it's on our images (libnepomukwidgets4). Does that need a manual override? [10:40] I also added plasmate to supported if you could refresh the set [10:41] it's in desktop-core, presumably bindings [10:41] I guess both that and nepomuk-widgets are functionally maintained by Kubuntu so should be overridden [10:42] it's part of the KDE SC, so it is maintained by us [10:43] s/nepomuk-widgets/nepomuk-core/ - but that's actually already overridden [10:43] no, those are 2 different things [10:43] er yes I know [10:43] I was saying that both belong in the kubuntu packageset [10:44] hm, I got the set with "./edit-acl -P kubuntu -S raring query" [10:44] and that doesn't show nepomuk-widgets [10:44] sorry I'm being confusing due to not enough coffee and unfamiliar keyboard [10:44] I understand your request and am acting on it [10:44] hehe [10:45] nepomuk-widgets moved [10:45] thanks! [10:45] and I'll refresh the autogenerated data now [10:51] "your call is important to us, please hold the line, while we familiarise ourself with a new keyboard." =))))))))))) [10:53] at least it isn't French layout :-P [10:53] \o/ azerty :p [10:54] you guys just don't know what you are missing ;-) [10:54] the ability not to type numbers? [10:54] that's what the keypad is for :p [10:55] I'd use UK layout on this US keyboard except of course it has one fewer key and it's kind of useful to be able to type \| [10:58] so french people dont use laptops ? or are french laptops extra wide to fi a numpad on ? [10:58] on an US keyboard with an azerty layout, you are missing * and µ. The latter, well, don't really used it since I finished my study. The first one is more worrying :) [10:58] *fit [10:58] ogra_: I always use "shift" [10:58] ogra_, numbers are overrated :p [10:59] and then a numpad magically slides out of the side of your laptop ? [10:59] :) === Sweetsha1k is now known as Sweetshark [11:00] ogra_: subscribe [11:02] ogra_: it's in the cd-tray ;-) [11:04] haha [11:06] m4n1sh: pong [11:15] I'm having some trouble debugging why the Ubuntu Studio ISO is not building. Was wondering if anyone could take a look, and perhaps point me in the right direction. Ever since I edited the seeds (I did some restructuring - and I'm new at this) we get these type of build logs http://people.canonical.com/~ubuntu-archive/livefs-build-logs/raring/ubuntustudio-dvd/20130306/livecd-20130306-amd64.out [11:16] the seeds are at lp:~ubuntustudio-dev/ubuntu-seeds/ubuntustudio.raring [11:19] odd. easiest way to debug it is probably going to be to attempt a matching local build [11:19] I'll have a go at that [11:20] cjwatson: hm, did the auto-refresh finish? nepomuk-widgets got added fine (and emacs24 for some reason), but plasmate not and if I did a mistake I don't see it. [11:21] cjwatson: Thanks a bunch [11:21] yofel: the data generation did, but actually putting it into LP needs another step [11:22] * cjwatson runs that [11:22] ah ok, sorry for impatient then ^^ [11:22] *for being [11:23] infinity: so, who should I ping about the samba update? [11:26] yofel: done now [11:28] cjwatson: perfect, thanks a lot! [11:29] pabs3: Someone in ubuntu-sru who isn't me, since we don't self-review our own uploads. [11:29] pabs3: But I'll work on the queue tomorrow to shrink it, so the ones left over become more obvious to others. :P [11:29] :D [11:29] infinity: cool, thanks :) [12:14] cjwatson: Do you have some nice resources on how to set up a local build system, that matches Ubuntus'? [12:14] https://lists.ubuntu.com/archives/ubuntu-devel/2011-June/033458.html [12:14] cjwatson: Ah, great! === MacSlow is now known as MacSlow|lunch === _salem is now known as salem_ [13:20] cjwatson: I suspect that /usr/share/livecd-rootfs/live-build/auto/config has something to with it. Didn't realize that needed to be adjusted in case of a restructuring of the seeds. [13:20] I've removed some seed files, and restructured [13:21] Also changed the metas [13:22] All though.. I've kept the old metas as transitional [13:22] So, I guess that was not it then.. [13:23] Forgot about the transitional metas [13:25] zequence: It'll be about task names rather than metapackages. I'll take care of it [13:25] It's trying to look up the ubuntustudio-audio-plugins task and failing [13:26] cjwatson: Aha. Ok. Yes, I removed it as a task. [13:26] Micah half-fixed it but didn't quite do it all [13:27] xnox: re consolekit> ack. search started, though it'll take a while [13:27] cjwatson: What I was trying to do was to make sure ubuntustudio-audio-plugins did not show up on the alternate installer as a task [13:27] jdstrand: awesome, thanks. [13:27] zequence: Yes, I updated tasksel the other day to match your changes [13:28] cjwatson: So, the seeds are correct then? Where is the error? [13:28] jdstrand: do we not have hadoop cluster of the unpacked sources? [13:28] zequence: livecd-rootfs 2.110 (just uploaded) should fix it [13:28] cjwatson: Thanks. [13:29] jdstrand: i recompressed all debs of all the archive in about 8 hours using tiny instances and juju. [13:29] i guess it would still take a while for grepping. [13:29] xnox: not that I'm aware of [13:30] ok. and spinning one up just for consolekit sounds a bit over-engineered =) [13:30] it has been a goal to have the ability for anyone to do this sort of grep without a local mirror, but alas, long todo list [13:31] jdstrand: well in the cloud one does have almost infinite bandwidth to a mirror on the same network. So in my case I was doing `apt-get download` well one could be doing `apt-get source` and do a grep. === MacSlow|lunch is now known as MacSlow [13:32] and then it's just a question of how many instances one has available and partitioning the job between them. [13:32] * jdstrand nods [13:33] it would probably be fun to work on (please refer to previous said long todo list :) [13:33] =)))))) [13:33] slangasek, stgraber: FYI, these systemd modifications plus my embarassing hack, plus a logind-ified polkit are in https://launchpad.net/~ubuntu-core-dev/+archive/logind [13:33] * xnox ELOOPDETECTED [13:33] hehe [13:35] yay embarassing hacks ! [13:45] didrocks: ping [13:53] xnox, are you working on the new ubiquity ? [13:54] hallo, sorry ^^ [13:57] njin: i'm hacking on ubiquity, what's up? =) [13:59] I'm writing the testcase for upgrade ubuntu step from cd , but when I run it , it requre everything (timezone, layout,ecc.) are you planning to adjust this step ' [13:59] xnox:^^ [14:00] hmmm, it require password too [14:00] * '/? [14:02] njin: yes, it does at the moment. Since you are reinstalling afresh, while preserving some data. [14:03] xnox, then is not planned any modification on these steps ? [14:05] njin: i'm not working on that part of the installer at the moment. Our preffered way to upgrade people is via upgrade-manager / do-release-upgrade. [14:05] and it's not high priority at the moment either for others on the installer team. [14:05] njin: do you have thoughts on how to improve it? [14:09] xnox: jump directly to the upgrade without asking nothing , but well, if it is not an urgent thing i can write down the testcase without problems, Thanks [14:09] njin: ok. [14:14] xnox: All that data should be easily scrapable from the installed system with minimal effort. user/password being the only sticky one, but surely in-place upgrades preserve passwd/shadow/group/gshadow anyway, or it's not much of an "upgrade"... [14:15] pitti: ah right, here I've got cgroup-lite installed so /sys/fs/cgroup is already a tmpfs as cgroup-lite mounts it as one [14:15] stgraber: ah, that explains it [14:15] pitti: if we decide to mount a tmpfs on /sys/fs/cgroup from mountall, we'd need to make sure we don't break cgroup-lite in the process [14:16] (which may try to double-mount the path) [14:16] right, we need to avoid that [14:16] * xnox we have volunteer to fix up ubiquity-ugprade. Thanks a lot infinity =))))) [14:16] stgraber: but we need to check that also if logind mounts it in the upstart job [14:16] stgraber: i. e. if logind's job starts before cgroup-lite's [14:16] stgraber: I take it it mounts it in cgroup-lite's upstart/init script? [14:16] infinity: true, but there are tweaks and bugfixes needed, apperantly we were wiping mythtv mysql database on in-place upgrade and other similar bugs. [14:17] it is fragile at the moment. [14:17] xnox: Yeah, the "fix" for the mythtv bug highlighted a massive design flaw, really. [14:17] stgraber: it's a race condition either way, of course [14:17] xnox: The assumption that none of /var is valuable, except a few whitelisted directories is bound to be very fragile. [14:18] infinity: i haven't looked into it to be honest. Where was the fix? Oh =) [14:18] xnox: Honestly, I'd be happier if we just removed the option from the installer, but others may disagree. [14:18] bdmurray wants to remove it =) [14:19] pitti: I think both should check if /sys/fs/cgroup is already mounted and only mount it if it's not. There still could be a race if both check+mount run at the exact same time, but that should be very unlikely [14:19] infinity: i'd be happy to remove it, if we have a way to preseed "keep /home partition unformatted" and or "wipe everything, but preserve /home" but it's more or less same can of warms. [14:19] s/warms/worms/ [14:19] xnox: Of course, the irony is that, for all the reasons it's an awful way to upgrade a complex desktop system, code very very similar to that may be an ideal way to upgrade phone/tablet installations. [14:19] pitti: though I'm not opposed to having this part of our default mountpoints, configured as a very small tmpfs (similar to /run/lock) and mounted by mountall (/lib/init/fstab) so that it's done before anything else [14:20] if [ -n "`grep /sys/fs/cgroup /proc/mounts`" ]; then [14:20] urgh [14:20] infinity: well at the single-image-update session we are leaning to limit our self and make the core-os image read-only, that is less fragile to update, yet still can break the rest of the user-space. [14:20] stgraber: ^ that's what /bin/cgroups-mount does [14:20] stgraber: mountpoint -q would certainly be more efficient, but it does the job [14:21] pitti: ah good, so cgroup-lite DTRT. So we could simply copy/paste that to the pre-start for logind I guess [14:21] xnox: Hrm? Running a union filesystem, then? [14:21] stgraber: see the pastebin, I do something liek that [14:21] xnox: That brings in a bunch of other bugs. [14:23] infinity: no, no unions. But the handwaving technology will teach dpkg about multiple databases of installed packages such that it all just works (tm) [14:23] ps. duct tape sold separately. [14:23] xnox: That doesn't instill confidence. But, uhm. Okay. [14:23] pitti: I remember having some weirdness happening with mountpoint disagreeing on whether something was actually a mountpoint, but /sys/fs/cgroup isn't one of those weird cases, so it's definitely cleaner than the grep [14:24] infinity: stgraber is assignee, so expect lxc technology and edubuntu logo to sneak in. [14:25] * infinity notes that having UDS double-booked with another conference on the other side of the planet was less than ideal for him. [14:25] I'll catch up next week, I guess. [14:29] say, who do you talk to about the single signon service being broken? even when I click the support link and select problems logging on, I just get a submit link that doesn't take any information and just goes straight to thanks, we'll be in touch shortly... wtf? [14:30] psusi: #canonical-isd [14:45] Laney: for the record, pyxdg cannot be in sync with Debian because don't have set_default_menu.diff, but Debian should have it [14:46] *we don't have === wedgwood_away is now known as wedgwood [14:50] * mitya57 wonders if we are freezing today [14:51] mitya57: i believe we are. standard time. [14:51] mitya57: hmm, its dropping wasn't mentioned in the merge - that's weird [14:51] I can't see why that patch would cause Ubuntu any problems though [14:52] Laney: it is mentioned in seb128's earlier changelog entry [14:52] so the "remaining changes" missed that one [14:52] yes [14:53] looks like you are right, it just adds a fallback so shouldn't cause any harm [14:53] but the menu-xdg dep is gone now [14:53] in Debian [14:55] dholbach: looks like we missed the chance to upload the new ubuntu-packaging-guide :) [14:55] apart from that, I'm ready to the freeze [14:55] ah, and I also have bug 1152007 ... [14:55] bug 1152007 in python-secretstorage (Ubuntu) "Sync python-secretstorage 0.9.0-1 (universe) from Debian unstable (main)" [Wishlist,New] https://launchpad.net/bugs/1152007 [14:56] Laney, mitya57: sorry, I tend to copy the previous merge's summary as a start point, I must have forgotten to update it with the extra diff added in between [14:57] mitya57: you have until 8pm [14:58] mitya57: I generally tend to like it when sync requests say /why/ the sync is being requested [14:58] like some compelling new features / bug fixes / ... [14:58] bdmurray: heads up: https://bugs.launchpad.net/daisy/+bug/1152206 [14:58] Launchpad bug 1152206 in Daisy "Repair overcounts" [Undecided,New] [14:59] tumbleweed: it has NEW binaries, so we missed it [14:59] mitya57: in the queue before freeze is ok (historically) [14:59] yeah, should be fine [14:59] oh, debian New queue. historically, that was OK too [14:59] but of course, the debian new queue is MASSIVE atm [15:00] tumbleweed: we can upload it to Ubuntu, and later make a Debian upload [15:00] s/later/simultaneously/ [15:00] yeah, I'd like boost1.53 =) [15:00] buy your favorite ftp-master beer :P [15:01] Laney: s/simultaneously/when I finish the Python 3 port/ [15:01] is that more of a blocker for debian than us? === chuck_ is now known as zul === kentb-out is now known as kentb [15:02] Laney: it's a blocker for "upstream" release, but yes, we can upload it to Debian if we find Andrew SB [15:02] * Laney decides to try sponsor-patch again (does it work for syncs?) [15:02] yes [15:02] * mitya57 is updating the bug description [15:04] * tumbleweed debates uploading a pypy pre-2.0 snapshot (it gained ARM JIT support) [15:08] * mitya57 updated the description [15:09] merci === marrusl_afk is now known as marrusl [15:10] Laney: thanks! [15:10] mterry: hey [15:21] psusi: keyboard> I have a new laptop that has a rather clickier keyboard than the previous one [15:21] lol [15:22] and yeah, I'm not the most delicate of typists :-) [15:25] zul, hi [15:25] mterry: we should be good for python-json-patch now [15:28] zul, I thought we were already good (I approved it yesterday) [15:30] mterry: cool thanks [15:39] Laney: thanks again, and enjoy http://anonscm.debian.org/viewvc/python-modules/packages/pyxdg/?view=log [15:39] I *do* enjoy! [15:43] barry: did you ever look at bug 1094218? the most recent comment is interesting. [15:43] bug 1094218 in lsb (Ubuntu) "lsb_release crashed with IOError in getstatusoutput(): [Errno 10] No child processes" [Medium,Confirmed] https://launchpad.net/bugs/1094218 [15:48] slangasek, cjwatson: do either of you have any idea why we'd hang while booting when efivarfs fails to mount? [15:59] bdmurray: i'll take a look now [16:10] dholbach: I've ported add-languages to Python 3, feel free to upload :) [16:10] mitya57: \o/ [16:11] mitya57, good work! :) [16:11] mitya57, I'm in the middle of 5 things right now - mind pinging somebody else to sponsor it? [16:11] * dholbach hugs mitya57 [16:15] anybody able to upload ubuntu-packaging-guide? ^^ [16:16] * mitya57 hugs dholbach back [16:16] bdrung ^^ === Tonio__ is now known as Tonio_aw [16:32] bdmurray: comments added [16:42] barry: thanks [16:42] bdmurray: let's see what they say ;) [16:45] do we not patch dch to allow specifying launchpad bugs for --closes? [16:46] or is there an alternative version available? like 'uch', heh [16:51] * tumbleweed has a shell wrapper around dch, called uch :P [17:29] hello [17:42] cyphermox: Re: NM bug, the driver is not at fault [17:42] It is a static 10G connection, doesn't support auto-negotiation, not does it report link status (if it's up, it's carrier is reported as linked) [17:42] Many other drivers also do this because the underlying hardware doesn't have link detection [17:44] BenC: we love powerpc =) would you want to push for lts hwe? or are you interested in raring only? [17:44] xnox: If raring releases next month, I'm not too concerned about back-porting :) [17:45] BenC: =) heh, touche. [17:45] xnox: precise would need lots of extras (qemu, installer, cdimage updates) to be worth it, and I'm not sure the effort is warranted === stokachu_ is now known as stokachu [17:45] sure. [17:46] xnox: but I appreciate the support === directhe` is now known as directhex [17:47] BenC: =) np, yw. [18:03] sforshee: because a missing mount point for an optional mount is ignorable, but a failed mount is an error; mountall doesn't know which virtual mounts are "important" or not, and indeed if you're booting under efi and you don't mount efivarfs, bootloader installs / upgrades won't DTRT [18:03] sforshee: what made the mount fail for you? [18:05] slangasek, it's a kernel bug in 3.9 that also got into 3.8 stable [18:05] * slangasek nods [18:05] there's a fix coming down the pipe, just wondered why the machines couldn't continue booting [18:06] sforshee: because mountall doesn't presume to know which mounts are important; consider if /proc failed to mount and it carried on booting [18:06] /proc is more important than efivarfs, but they're both important and mountall doesn't play favorites [18:06] slangasek, okay. When this happened to me I got nothing but a blank screen. Shouldn't we at least display some kind of error message? [18:08] (TBH, if you told me that we *should* play favorites, I'm not sure I have a way to do that in mountall today; but if you think efivarfs failures should be non-fatal, I could look into that) [18:08] sforshee: well, that's tricky, because the dependency chain for showing prompts to the user is virtualfs -> udev -> plymouth [18:08] we probably *should* say something on console too [18:09] slangasek, I definitely think what happens now isn't very good [18:10] I don't feel qualified to say whether or not efivarfs is critical, but it seems to me like it wouldn't be [18:10] for comparison, here's the full list of 'optional' filesystems that mountall deals with in this particular way; none of which have been reported as a problem before now: http://paste.ubuntu.com/5593851/ [18:11] I do think we want a message on console. Can you file a bug against mountall for this? [18:11] slangasek, sure [18:12] As slangasek says, there are some hidden issues if you then try to upgrade the boot loader - in particular if you're running in secure boot mode the lack of efivarfs will (currently) have the effect that a boot loader upgrade will silently install an unsigned boot loader and your *next* attempt to boot will crash and burn [18:12] cjwatson, yeah that would be a problem [18:12] I say "currently" because this is something we've agreed to change anyway, but ... [18:14] sforshee: related bug: bug #610869 [18:14] bug 610869 in mountall (Ubuntu) "mountall ignores nofail mount option" [Medium,Triaged] https://launchpad.net/bugs/610869 [18:15] (so, if we really wanted to ignore the failure, we would need to fix that bug first) [18:16] pitti: polkit forwarding to bugs.fd.o> I would be grateful if you would file that bug and CC me [18:17] slangasek, bug #1152274 [18:17] bug 1152274 in mountall (Ubuntu) "filesystem mount failures during boot halt boot with a blank screen" [Undecided,New] https://launchpad.net/bugs/1152274 [18:17] sforshee: cheers === deryck is now known as deryck[lunch] === zyga_ is now known as zyga [18:36] psusi, ping. [18:37] smoser: pong [18:37] hey. [18:37] my query today is reguarding your comments on eatmydata [18:37] do you have that work collected anywhere? [18:38] I filed a bug report... let me see [18:38] my interest is for use in cloud-images. when a stock cloud-image is booted, quite often the first thing that is done is a slew of packages installed... [18:38] bug #1126314 [18:38] bug 1126314 in debootstrap (Ubuntu) "Debootstrap is very slow. Please use eatmydata to fix this." [Undecided,New] https://launchpad.net/bugs/1126314 [18:39] and often that is orchestrated by cloud-init. if it was easy to do, cloud-init could utilize that path to make that initial install happen faster (as nothing user-data worrysome has happened at that point) [18:39] yep, that's the idea [18:39] at very least, i guess cloud-init should invoke apt-get update with --force-unsafe-io [18:39] I thought we were using dpkg --force-unsafe-io for install/bootstrap? [18:39] ah, for apt-get [18:39] we do, but that does very little [18:39] its less effective [18:39] we do for install and bootstrap. [18:40] even with --force-unsafe-io it still flushes some updates synchronously [18:40] still flushes /most/... that option gets rid of rather few syncs [18:40] hmm [18:41] BenC: ok, then with the logs you added to the bug I'll circle back and send it upstream for fixin' [18:41] psusi, did you (or is there already) a obvious/easy way to use eatmydata? [18:41] I tried to post a patch a year or two back to fix it and it was rejected by the dpkg maintainers and they said to use eatmydata instead, since that would also get rid of any syncs that happen as a result of the maintainer scripts [18:41] well, that's true, but dpkg is the main offender :P [18:42] ah. i see more data in debian bug [18:42] I guess cjwatson may weigh in on the bug [18:42] I'm not sure what you mean... I patched debootstrap to forcibly add the package to the required list so it gets unpacked right away, then set the LD_PRELOAD environment variable so it gets used during the rest of the bootstrap [18:42] cjwatson suggested adding the LD_PRELOAD to in-target for the rest of d-i to leverage it [18:43] * smoser chuckles at the idea of filing MIR for eatmydata, but that would be necessary for cloud-image usage. [18:43] I use eatmydata for all my installs since its available, it significantly reduces the time, usually takes longer to type in your credentials and stuff [18:44] also required for the debootstrap patch [18:45] to be priority:required ( or forced there by debootstrap ) it has to be in main [18:46] ok. so heres an opportunity for everyone to laugh at the ignorance of smoser. [18:47] but if i have an install and have data i care about on a different filesystem than my OS (where dpkg is writing to) [18:47] then why would i want or care for dpkg to call fsync for me. [18:47] mostly its just ensuring that 100% reproducible data gets written to its target in that case. [18:48] the idea is that if the power goes out while you are installing or upgrading, you don't corrupt your dpkg database, or end up with packages that are partially upgraded and thus, horribly broken [18:50] yeah, that makes sense. but if i've separated OS data (extremely reproducible) from non-OS data, i have increased risk of data i cannot reproduce by using eatmydata for apt. [18:51] that reminds me, I guess the next step to that MIR is to add it to a seed [18:52] I'm not following [18:52] hello! i have a question about old gnome libraries (specifically: libgnome-desktop-2-17 and libgnomeprint (and printui)). Can one safely assume they'll at least be in the archives (if not main) for a very long time? [18:52] what does apt have to do with your music and video collection? ;) [18:53] psusi, it has nothing to do with it. thats what i'm saying. [18:53] psusi: if this is for image creation why not do it on tmpfs? [18:53] i dont really care about fsync() on data that i can reproduce. [18:53] lifeless: this is for normal installation in my case, and cloud images in smoser's [18:53] outside of time recovering, i dont lose anything. [18:54] you can't readily reproduce the dpkg status database without reinstalling, and a half upgraded package can leave the system unbootable, so your average user wants to make sure those don't happen... [18:56] lifeless, interesting comment at http://glandium.org/blog/?p=1169 [18:56] " It might just be btrfs. And the best part yet is that since sync() is global, even when running pbuilder over, say, tmpfs, it doesn’t change a damn thing." [18:57] but i think that might be wrong. [18:58] dpkg doesn't yse sync()... it uses fsync() and sync_file_range() [18:58] psusi, yeah, thats what iw oudl have thought [18:58] so something else might be calling sync [18:59] it only takes on maintainer script :) [18:59] or say updating a database that calls sync [18:59] I think someone at some point a few years back suggested having it use one sync() instead of a dozen fsyncs() and the response to that was no, since sync() is global it would flush a bunch of unrelated writes [18:59] probably want to instrument it and find out where it is coming from [18:59] it's coming from dpkg ;) [18:59] what dpkg actually needs is barriers [18:59] right, but there's no user space api for them [19:00] yeah. sadface. [19:00] psusi: I thought dpkg uses fsync and sync_file_range. [19:00] psusi: so I'm saying that that blog author is on crack, or has something *else* calling sync() :) [19:01] lifeless, thats what i think too. [19:01] lifeless: or it may have been written when they were trying to use sync() in dpkg instead of fsync and sync_file_range [19:06] psusi: 2010-10, we can check that [19:06] I also found a kernel bug the other day that is hurting things too... iirc, dpkg uses sync_file_range to attempt to initiate background writeout early, then calls it later with the flag to wait until the writeout has finished... seems the kernel implementation was still doing a synchronous write without the wait flag [19:08] I think the hope was that it could unpack all of the files in a single pakcage, initiating writeout on each file, then doing one wait at the end, and this was supposed to be the best way to get both speed and safety... but with the kernel treating every individual file's non blocking sync_file_range as a blocking sync, that's got to hurt the performance considerably [19:10] let's see here, what seed should eatmydata be added to? [19:18] psusi, lifeless: or we could just declare the kernel filesystem designers to be on crack, and require data=ordered semantics for all filesystems we support in Ubuntu? :) [19:21] data=ordered *is* the default [19:22] slangasek: its metadata out of order that hurts isn't it? [19:23] psusi: data=ordered is the *default*, but all the stupid tricks that userspace software is using these days to enforce syncing (including dpkg, and firefox) is effectively a workaround for filesystems *not* using that default [19:23] lifeless: data=ordered is what ensures that the data is written before the metadata [19:23] (in ext4 terminology) [19:24] slangasek: no... filesystems do use that default, and the workarounds are to deal with the fact that there is no api to express to the kernel that this file should be synced first, *then* renamed [19:24] but this is only a default for ext4, users have the ability to change it; and xfs and btrfs don't give this guarantee [19:24] psusi: you don't *need* to express "sync, then rename" if data=ordered is in effect [19:24] which is why I'm saying we should renegotiate this contract with the kernel [19:25] yea, you do, that's why dpkg is doing so much syncing [19:25] no, it's not [19:25] dpkg is doing this *because it can't rely on the default filesystem options* [19:25] ping, pong, ping, pong [19:25] so did ext4, during the terror release, not have that default? [19:25] there are no nfilesystem options to fix it [19:25] what was with the 0 length files... [19:26] IIRC it was directory content being considered not file metadata, and not being ordered. [19:26] the zero length files happened when people were using data=writeback [19:26] so what we should do, instead of making this the userspace's problem and making everything slower as a result, is to enforce data=ordered as an architecture requirement for the OS and homedir filesystem [19:26] lifeless: correct; for a brief period, data=ordered was not the default [19:27] no slangasek, data=ordered does not fix the problem [19:27] now I would argue that it *should*, but it doesn't... [19:28] psusi: you admit that dpkg is trying to ensure that the data is written to disk before the rename. This is exactly what data=ordered gives you, with no further userspace acrobatics. [19:28] nope, it doesn't [19:29] psusi: that is the DEFINITION of data=ordered. Prove otherwise. [19:29] see the big huge arguments between ted tso and the dpkg maintainers 2 years back ;) [19:29] nothing you can paste? [19:30] I was there for the argument; I'm not going to waste my time rereading it [19:30] I'm quoting the definition of data=ordered from the manpage, the burden of proof is yours [19:31] * SpamapS wonders if the solution isn't just "cheaper SSDs" [19:31] SpamapS: that's a bandaid [19:31] slangasek: I agree that it how it should work, but the reason they added all the syncing to dpkg is because it doesn't [19:32] psusi: citation needed. [19:32] slangasek: a pretty good one :) [19:32] psusi: https://www.kernel.org/doc/Documentation/filesystems/ext4.txt [19:32] psusi: auto_da_alloc [19:33] SpamapS: except for all the cases where an SSD isn't fit for purpose, or you have so much data that the I/O itself (rather than the disk write) becomes a bottleneck [19:33] psusi: thats the thing ted added [19:33]