[00:03] <infinity> slangasek: Since we're still in a "need to rebuild the world" state anyway, I'm going to leak your cron change through.
[00:03] <slangasek> infinity: ok
[00:05] <cjwatson> that said, I believe this is going to require at least a fix to grub-installer, so I'm going to hold off on 774089 for beta-2 and try to fix it for final
[00:11] <cjwatson> just going to run ubiquity through pbuilder before uploading
[00:11] <cjwatson> will be able to upload d-i after this publisher run finishes
[00:13] <skaet> cjwatson,  goodness.  :)
[00:13]  * skaet goes to grab some dinner,  biab
[00:16] <cjwatson> seriously tired.  going for temporary caffeine boost
[00:20] <Daviey> cjwatson: The other fix for that is bed.  Maybe ask freenode to k-line you, during silly-hours.. bit like gamblers stop-limits.
[00:21] <infinity> ^
[00:21] <cjwatson> I fully intend to go to bed but I don't have a lot of option except to finish this, otherwise the whole image build pipeline is delayed
[00:22] <cjwatson> since nobody else was around who knew how to fix ubiquity and it had been left broken
[00:23] <Daviey> :(
[00:25] <infinity> One of these days, everything EXCEPT the installer will be broken.
[00:25] <infinity> And Colin will point and laugh.
[00:25] <infinity> Just wait.
[00:26] <cjwatson> I look forward to it
[00:26] <Daviey> sensible people will join in the laughter.
[00:26] <Daviey> whereas platform will be pulling their hair out.
[00:48] <cjwatson> err, hold off a moment on accepting d-i (uploaded).  The publisher still seems to be running, rather later
[00:48] <cjwatson> than usual
[00:48] <infinity> Argh.  Stupid thinko.
[00:48] <cjwatson> ?
[00:48] <infinity> cjwatson: Well, you get your wish, I had to fix jasper, so I'm switching to blkid. :P
[00:48] <ScottK> sladen: It's more than just the Ubuntu images that are affected by the font package getting bigger.
[00:48] <cjwatson> heh
[00:52] <infinity> cjwatson: http://paste.ubuntu.com/693424/ <-- Sane?
[00:53] <infinity> cjwatson: Err, ignore the typo in the changelog.
[00:53] <infinity> s/full/full path/
[00:53] <cjwatson> looks fine, but don't trust my judgement at this point
[00:53] <infinity> Fixed locally. :P
[00:53] <cjwatson> The publisher's finished now, so debian-installer is safe to accept
[00:53] <cjwatson> Dunno what took it so long.
[00:53] <infinity> cjwatson: Well, I'm mostly looking for "hey, you have obvious quoting fail" or sometihng, if I break the logic, that's my own fault.
[00:54] <cjwatson> well I'd probably do "/dev/${ROOTPART}" out of paranoia, but I can't imagine how a device node could realistically end up with a space in it
[00:55] <cjwatson> also looks like the versioning scheme for that package should be 0.60 not 0.59ubuntu1 </nitpick>
[00:55] <infinity> Oh, yeah.  Local env issue.  Changelog fixed.
[00:55] <cjwatson> and ubiquity uploaded
[00:55] <infinity> And I'll anallify the quoting.
[00:56] <infinity> Makes vim's highlighting happier anyway.
[00:58]  * infinity glares at duplicity for breaking his dupload tab-completion.
[00:58] <slangasek> d-i accepted
[00:59] <infinity> Dangit, I'll be a cycle late for jasper.
[00:59] <infinity> Oh well.  I had no plans tonight.
[00:59] <slangasek> do you want me to hold the publisher?
[00:59] <slangasek> that sounds like a yes
[00:59] <infinity> slangasek: if you want to do a manual run, sure.
[01:00] <infinity> cjwatson: ubiquity still going through local testing?
[01:00] <infinity> Oh, no, you said it was uploaded.
[01:00] <infinity> I can't read.
[01:01] <infinity> And there it is.
[01:01] <infinity> slangasek: May as well hold the publisher until both ubiquity and jasper build, I guess, then we can get on with images.
[01:01] <slangasek> infinity: yep. are you reviewing/accepting ubiquity?
[01:02] <infinity> I'm tempted to accept it blind, but I think I'll wait for a diff to look for obvious oopses.
[01:02] <infinity> Oh look, a diff.
[01:04] <infinity> That level of copy and pasting can't possibly be wrong!
[01:04]  * infinity accepts.
[01:04] <infinity> And I'll accept my own jasper upload, based on cjwatson's review.
[01:04] <cjwatson> It can't have taken four goes to get it right, then. :-)
[01:04]  * cjwatson edits reality
[01:04] <cjwatson> jasper> yeah, ack from me
[01:05] <infinity> cjwatson: When you get tired, you suffer deja vu more frequently.  You only fixed it once.
[01:06] <slangasek> infinity: poke me if you notice things are ready to publish before I do?  I'm intermittently aware of my surroundings (scrambling to get out the door, here)
[01:06] <infinity> slangasek: Sure.  I can drive the publisher, if you're on your way out.
[01:06] <infinity> Vroom, vroom.
[01:06] <infinity> I'm an excellent publisher.  Dad lets me publish in the driveway.
[01:06] <slangasek> infinity: ah, I'll gladly turn it over to you then
[01:07] <slangasek> doing some more reboot testing instead, in that case
[01:08] <infinity> Oh dear.  I've just gone full Rainman mode over here.  I just said (yes, out loud), "Uh oh, packages on the highway!"
[01:09]  * cjwatson sends infinity to fly Qantas
[01:20] <cjwatson> Right, crashing.  Don't expect me around too early tomorrow.
[01:21] <infinity> G'night. :)
[01:55] <Daviey>  /win 22
[02:12]  * infinity goes to find some food, back in an hour.
[02:19] <ScottK> http://people.canonical.com/~ubuntu-archive/testing/oneiric_probs.html seems a tad optimistic (empty).
[02:19] <ScottK> I know there's at least one problem I haven't fixed yet.
[02:30] <ScottK> ^^^ That one.
[02:31] <ScottK> Not beta2 critical (fixes armel installabliity for kalzium-dev), but since we haven't rolled dvds yet, it might be nice if someone could review.
[02:37] <charlie-tca> maybe I marked this one too low? bug 854198
[02:37] <ubot4> Launchpad bug 854198 in ubiquity (Ubuntu) "Cannot change keymap language again after double clicking on a language (affects: 1) (heat: 8)" [Medium,Triaged] https://launchpad.net/bugs/854198
[03:06] <infinity> charlie-tca: Curious bug, but probably not a beta blocker.
[03:06] <infinity> Release blocker, likely.
[03:07] <infinity> Hey neat, I'm finally about to get powerpc buildd time.
[03:09] <infinity> ScottK: Accepted.
[03:09] <ScottK> Thanks.
[03:11] <skaet> ubiquity i386 published, but amd64+armel still pending.... drat.
[03:12] <infinity> skaet: I'm waiting on ppc to build, and then publishing the lot.
[03:12] <charlie-tca> Well, I tried...
[03:12] <infinity> skaet: I' happy to turn the crank on a new set of images tonight before I go to bed.
[03:12] <skaet> infinity, fair 'nuf.   have at it.
[03:13] <skaet> :)
[03:13] <infinity> skaet: Or we can make pitti do it when he wakes up. ;)
[03:13] <skaet> infinity,  heh,  I suspect he and NCommander will need to publish them.  :)
[03:14] <infinity> Well, they publish themselves.  But post to the tracker and such, yeah.
[03:14] <skaet> point to infinity,  yes, post to tracker.
[03:22] <skaet> infinity, any images you're aware of that I shouldn't mark as rebuilding?
[03:22] <infinity> I intend to just rebuild the world.  People who think they don't want rebuilds are, at this point, wrong.
[03:22] <infinity> Given that we've touched every installer we ship. :P
[03:23] <skaet> was wondering that core might not need it, but *shrug*
[03:23] <skaet> all marked for rebuilding now.
[03:23] <infinity> server and edubuntu might want Yet Another Rebuild later, when they sort out their other bits, but building now to smoketest won't kill anyonw.
[03:23] <infinity> Nor anyone.
[03:23] <skaet> yuppers.
[03:23] <infinity> Oh, I could skip core.  But it takes about 3 minutes to build.
[03:23] <infinity> So, meh.
[03:23] <infinity> I intend to copy-and-paste lazily.
[03:24] <infinity> ScottK: Are you happy with everything under point [6] at http://pad.ubuntu-uk.org/ubuntu-release?
[03:24] <skaet> okie.   update the pad when you start,  and details of what's left when you call it a night.
[03:26]  * skaet figures probably time to call it a night...   
[03:26] <infinity> Wuss. ;)
[03:27] <infinity> (G'night)
[03:27] <skaet> :)  yup.
[03:27] <skaet> g'night.
[03:49] <slangasek> infinity: publisher is still down; did everything get published that we were waiting for, and should I unblock the publisher again?
[03:49] <micahg> is someone fixing gnome-control-center?
[03:50]  * slangasek assumes so and re-enables the cronjob
[03:50] <infinity> slangasek: I'm hand-cranking it in 2 or 3 minutes.
[03:50] <infinity> slangasek: And then going back on auto.
[03:50] <slangasek> ah, ok
[03:50]  * slangasek disables again :)
[03:50] <slangasek> and afk again for about 45min
[03:50] <infinity> Although, the cronjob is close enough to "in a few minutes". :P
[03:55] <slangasek> pff, how do we go all cycle with no one filing bugs about the dots-over-text bug *until* I diagnose the issue, open a bug report and start working on a fix and *then* the duplicate bugs come in
[03:56] <infinity> Quantum entanglement?
[03:57] <infinity> Publisher crontab back on, it more or less lines up with when I wanted it anyway.
[04:16] <pitti> Good morning
[04:16] <infinity> Morning, Martin.
[04:16] <infinity> I was going to start a respin of literally everything (except ubuntu-core, I just did that) when the next publisher run is done.
[04:16] <infinity> But then I was going to bed. :P
[04:16] <infinity> So, if you'd like to do that instead, so you can babysit it...
[04:18] <pitti> infinity: sure, can do; so everything except core, check
[04:18] <pitti> infinity: any particular package version I should check for?
[04:19] <infinity> Going to use the evil wait-on-package script?
[04:19] <infinity> Let me find something from this run. :)
[04:19] <infinity> Short memory is short.
[04:19] <pitti> infinity: not necessarily
[04:19] <pitti> infinity: just making sure that I don't trigger too early
[04:20] <infinity> Heh.  kalzium on all arches should be a good enough indicator.
[04:20] <pitti> I had cases where the package I was waiting on appeared a whole hour after I expected it
[04:20] <infinity> Or ubiquity.
[04:20] <pitti> infinity: roger; ubuntu images certainly don't block on that one, though?
[04:20] <infinity> See above. ;)
[04:20] <pitti> ah, that's a good one
[04:20] <infinity> Pretty much everything's blocking on something until this run's over.
[04:21] <infinity> (Except core, since it has no installer and no... packages)
[04:21] <pitti> infinity: alright, will do; so sleep well!
[04:21] <infinity> I doubt I'll be sleeping just yet. :)
[04:22] <infinity> But at least I don't have to babysit the World's Nastiest Shell Loop.
[04:22] <infinity> So, I call this a win.
[04:23] <infinity> Assuming you execute the blocks of doom from the Pad (minus core), I guess server-preinstalled will be up first.  I might stay up long enough to test that.
[04:24] <infinity> Since I did a bad, bad thing, and changed the preinstalled default filesystem during a freeze.
[04:24] <infinity> *cough*
[04:24] <pitti> yes, the pipelines were hard enough to put together, I don't reinvent it every time :)
[04:25] <ScottK> infinity: Looking
[04:26] <infinity> Oh, sweet.  I pilot on Wednesday.
[04:26] <infinity> "Your patch is awesome, but we're in a hard freeze for beta, muahahaha."
[04:26] <infinity> ^-- Practice.
[04:28] <micahg> infinity: 1/3 of the current queue is unseeded, so plenty to do :)
[04:28] <micahg> and 19 SRU requests (might be some overlap)
[04:29] <ScottK> infinity: It's fine.
[04:29] <infinity> ScottK: [6] is all happy?  Yay.
[04:29] <infinity> Well, except potential size issues.  But you can revisit that.
[04:29] <ScottK> infinity: Won't know about if the seed changes work until after the rebuild, but yeah.
[04:29] <ScottK> Yep.
[04:30] <infinity> Kay. Well, if you don't feel like reading backscroll, we're respinning the world as soon as this publisher run's over.
[04:30] <infinity> So, you'll have shiny things to be annoyed by in the morning.
[04:33] <ScottK> infinity: There's edubuntu and ubuntu-studio still in queue to build for powerpc, on the off chance we care.
[04:34] <ScottK> https://launchpad.net/ubuntu/oneiric/+builds?build_text=buntu&build_state=pending&arch_tag=all
[04:38]  * stgraber wonders why edubuntu-fonts is arch any and not arch all...
[04:39] <pitti> darn, gnome-control-center was FTBFS on !i386 due to arch desync
[04:39]  * pitti gives back
[04:39] <pitti> I'll build kubuntu first, so that it has some time
[04:41] <stgraber> ok, edubuntu-fonts is just a meta package... I'll poke highvoltage to get that fixed in 12.04, that source package should be dropped and edubuntu-fonts generated from our seeds and edubuntu-meta
[04:41] <stgraber> anyway, no point in uploading a new arch all package just now but will need fixing
[04:47] <pitti> infinity: would it be okay for you if I upload colord to add the missing liblcms2-dev dependency to libcolord-dev? this causes FTBFS
[04:47] <pitti> infinity: if we want to spin images now, we have to live with control-center being out of date, but it's not the end of the world; I'd just like to avoid further FTBFSes due to that
[04:52] <infinity> pitti: Go ahead.  And there's no massive rush on respinning, should just happen on your watch, so we all have stuff to test "soon".
[04:52] <pitti> infinity: right; I'll try to shuffle the queue to start with kubuntu etc.
[04:57] <pitti> ^ single targetted patch, fixes regression in .92
[05:00] <pitti> NCommander: you apparently still have a backgrounded nano process on antimony on an index.html from August 4; can you please kill this?
[05:06] <pitti> ubiquity is published, kalzium still behind
[05:12] <ScottK> kalzium only affects the Kubuntu dvd.
[05:12] <pitti> ah, ok; starting with server and kubuntu desktop then
[05:13] <ScottK> It affects edubuntu too, FYI.
[05:14] <pitti> *nod*
[05:16] <pitti> eh, seems cron.daily is broken, it returns immediately without any error message
[05:21] <slangasek> well, it helpfully sent errors in email
[05:21] <pitti> ah, there
[05:23] <pitti> ok, that's better
[05:44] <pitti> http://cdimage.ubuntu.com/kubuntu/daily/20110920.6/ posted
[05:45] <pitti> ScottK: ^ FYI
[05:45] <ScottK> Thanks.
[06:04] <pitti> http://cdimage.ubuntu.com/ubuntu-server/daily/20110920.2/ posted
[06:06] <pitti> *mumble* publisher taking over an hour these days *whine*
[06:07] <infinity> Yeah, it's gotten ridiculous. :/
[06:07] <pitti> is that because process-death-row has been running for two days now?
[06:07] <pitti> (eating IO, or what not)
[06:08] <infinity> Wait, is it still running?
[06:08] <infinity> That sounds ungood.
[06:08] <infinity> LP people might want to know about that.
[06:10]  * pitti starts some more builds, kalzium published
[06:11] <slangasek> hmm, I was going to ask whether the publisher is actually that slow or if contents-generation just overran, but contents-generation has been failing since Sep 11
[06:12] <pitti> ok, livefs builds fail, too
[06:12]  * pitti pokes for logs
[06:14] <pitti> "E: Invalid Release Signature (key id 40976EAF437D05B5)"
[06:14] <pitti> WTF?
[06:14] <infinity> Mirrors need a re-trigger, or the publisher did Something Bad?
[06:14] <pitti> lamont, infinity: ^ do you happen to know if anything changed there since yesterday? it fails right away in debootstrap
[06:15]  * slangasek files bug #854449 about the broken Contents generation
[06:15] <ubot4> Launchpad bug 854449 in launchpad "generate-contents-files.py failing on Ubuntu archive since September 11/12 (affects: 1) (heat: 6)" [Undecided,New] https://launchpad.net/bugs/854449
[06:15] <pitti> ok, then I'll just wait a bit until the publisher comes to a rest
[06:17] <infinity> I retriggered the mirrors for good measure anyway.
[06:18] <pitti> wow, publisher finished after 75 mins
[06:20] <pitti> *sigh*, does any of the images *not* fail?
[06:21] <infinity> In theory, none of them should. :P
[06:21] <infinity> But if the mirror signature issue is plaguing you, nothing will work.
[06:21] <pitti> http://cdimage.ubuntu.com/ubuntu-server/daily-preinstalled/20110920/
[06:21] <pitti> wohoo
[06:22] <pitti> (posted)
[06:22] <infinity> Successful live-based images aren't exciting if the livefs failed...
[06:22] <infinity> Since it just uses yesterday's.
[06:22] <pitti> no, I mean my alternate build attempts all silently failed
[06:22] <pitti> (will check logs in a bit)
[06:23] <pitti> are we even doing armel+ac100?
[06:23] <pitti> and imx51?
[06:23] <infinity> For mx5...
[06:23] <pitti> the tracker seems to have all these added (disabled, though)
[06:23] <infinity> And ac100.
[06:23] <infinity> For daily-preinstalled.
[06:23] <infinity> ubuntu, that is.
[06:23] <infinity> No other flavour.
[06:23] <infinity> You'll get all the right defaults if you just run the scripts. :P
[06:23] <infinity> (honest)
[06:24] <pitti> infinity: I mean for the tracker
[06:24] <pitti> Ubuntu Desktop armel+ac100 (rebuilding...)
[06:24] <pitti> ^ these
[06:24] <infinity> pitti: mx5 and ac100 should be on the tracker for ubuntu-preinstalled.
[06:24] <pitti> so we do those, ok; I think we didn't for b1
[06:24] <infinity> Yeah, that.
[06:25] <pitti> server ac100, too? because the cron job didn't build those
[06:25] <infinity> Also, that preinstalled server image looks suspiciously like it contains a current livefs...
[06:25] <infinity> Like, it may be okay!
[06:25] <infinity> pitti: No, server is only omap/omap4.
[06:25] <infinity> pitti: If the tracker lists ac100/mx5 for anything other than ubuntu desktop, it's wrong.
[06:25] <infinity> (And I thought I went over this 12 hours ago with skaet... Hrmph)
[06:26] <pitti> ok, removing alternate mx51 and server ac100 from the tracker
[06:27] <pitti> ScottK: are we doing kubuntu preinstalled and/or kubuntu-mobile preinstalled?
[06:27] <pitti> can start a build now if we want them
[06:28] <pitti> infinity: ok to accept the unity upload? it whitelists update-notifier for the systray to make slangasek get updates again
[06:29] <pitti> infinity: it was a rather unnoticed (by us) behaviour change in unity-2d after b1
[06:29] <pitti> no other changes
[06:31]  * pitti takes the bullets and accepts
[06:31] <pitti> otherwise some beta-2 users might be stuck, not ever getting updates again
[06:32] <pitti> ScottK: starting those builds, I'll just post them; let's see what sticks
[06:34] <infinity> pitti: Err.  It's a fix in the library?
[06:34] <pitti> infinity: it's just a configuration change, there's a gsettings key for the whitelist
[06:35] <infinity> Kay..
[06:35] <infinity> Was just curious how a unity upload would fix unity-2d.
[06:35] <pitti> if it doesn't make it to the images, it's not the end of the world, but as I have queued the ubuntu images last in the chain anyway, it might just make it
[06:35] <pitti> infinity: ah, it's in unity-common, which ships the gsettings schema
[06:35] <infinity> Mmkay.
[06:36] <pitti> /usr/share/glib-2.0/schemas/com.canonical.Unity.gschema.xml, systray-whitelist FYI
[06:36] <infinity> So, it turns out that there's a point where you have enough bandwidth that downloading a 600MB image is faster that zsyncing a 10% delta.
[06:36] <infinity> And I'm at that point.
[06:36] <pitti> heh, nice
[06:37] <infinity> s/that z/than z/
[06:38] <infinity> I'm actually oddly disappointed by this.
[06:46] <pitti> http://cdimage.ubuntu.com/xubuntu/daily/20110920.1/ posted
[06:47] <pitti> seems kubuntu arm isn't happy, failed
[06:47] <pitti> kubuntu-mobile, too
[06:48]  * infinity waits for logs.
[06:48] <infinity> Or, I could force an update. :P
[06:49] <pitti> kubuntu dvd build failed again on invalid release file, will try again later
[06:51] <infinity> kubuntu-mobile was the same failure (bad sig)
[06:52] <infinity> And kubuntu-preistalled.
[06:53] <infinity> So frustrating. :/
[06:53] <pitti> it has always been a gamble, but I don't remember it being that hard
[06:53] <infinity> In theory, it should never break.
[06:53] <infinity> dists/ on ftpmaster is always consistent.
[06:53] <pitti> anyway, I'll keep trying and post 20110920.257, if it ever condescends to build
[06:53] <infinity> So, the only way for it to break is for syncproxy and beyond to have issues.
[06:54] <infinity> I think you want "consents".
[06:54] <infinity> Though I like "condescends". :P
[06:55] <pitti> I actually mean it that way :) you have to address it very carefully and properly, otherwise it's grumpy and fails
[06:56] <infinity> Ahh, how I've missed awkward German English constructs. ;)
[06:56] <pitti> so is that bad grammar?
[06:56]  * pitti checks dictionary "to condescend to do sth." seems valid
[06:58]  * pitti runs anonftpsync manually and crosses fingers
[06:58] <pitti> damn you, mirror
[06:59] <infinity> pitti: That usage certainly exists (or existed?), but either it's only used in countries I never hang out, or it's archaic.
[06:59] <slangasek> I would tend to prefer 'deign'
[06:59] <infinity> pitti: (The more common meaning of "to condescend" is to talk down to someone, or treat them as an inferior)
[06:59] <pitti> infinity: ah, might be; I think I heard it first in the last song of "Cats"
[07:00]  * infinity is down with deign.
[07:00] <pitti> but apart from that, anonftpsync didn't help
[07:01] <infinity> Forcing a trigger from cocoplum might.
[07:01] <pitti> infinity: ah, the language barrier again -- what is "force a trigger from cocoplum" in shell?
[07:01] <infinity> Hahah.
[07:02] <infinity> lp_publish@cocoplum:~$ /srv/launchpad.net/codelines/current/cronscripts/publishing/distro-parts/ubuntu/finalize.d/90-trigger-mirrors
[07:02] <infinity> (Which I just did)
[07:04] <infinity> pitti: After that, when syncproxy (91.189.92.172) stops suckling on port 873 on cocoplum, you should be good to go to re-sync from antimony.
[07:04] <infinity> (At least, I think that's antimony's upstream mirror... I lose track of everything around this time of day)
[07:04] <pitti> tcp    63999      0 91.189.90.15:873        91.189.92.172:50934     ESTABLISHED -
[07:04] <pitti> wthis one, I suppose
[07:06] <infinity> Yeah, that's antimony's mirror, I'm not losing it.
[07:06] <infinity> (antimony's upstream, that is)
[07:07] <infinity> Well, for non-ports.
[07:07] <infinity> For ports, it's... ports.
[07:07] <infinity> 91.189.92.175
[07:07] <pitti> that just has a TIME_WAIT one, so is being closed?
[07:07] <infinity> And ports just finished.
[07:08] <Daviey> infinity: is that an ssh trigger?
[07:09] <infinity> It's a bit sad that I find the easiest way to remember the ANONFTPSYNC magic environment variable for config files is to "grep anonftpsync /src/cdimage.ubuntu.com/bin/*"
[07:09] <infinity> Daviey: Yeah.
[07:09] <infinity> s/src/srv/
[07:17] <Daviey> How come we are on .2?  We've had 3 spins today?
[07:18] <infinity> Mirror issues.  See above. :P
[07:18] <infinity> We've not had more than one successful spin of anything today, but it's taking some effort to get 1 of everything.
[07:18] <Daviey> ah!
[07:19] <Daviey> Hmm, we need some more duct tape.
[07:24] <infinity> Or WD-40.
[07:31] <pitti> ah, seems the cocoplum mirror push finished, running anonftpsync again
[07:33] <infinity> Alright, server-preinstalled with ext4 seems to work correctly, I think it's bedtime for me now.
[07:34] <infinity> If I can sleep. :P
[07:38] <pitti> erk, did the DC just fell off the planet?
[07:38] <infinity> I still see it.
[07:38] <infinity> Maybe it's you that no longer exist?
[07:38]  * pitti dissolves into a logic cloud
[07:38] <pitti> LP and chinstrap both don't respond right now
[07:39] <pitti> I think it wants to tell me "go have breakfast at last, dammit" or so
[07:39] <infinity> Possibly. :P
[07:39] <infinity> I never got around to breakfast today.
[07:39]  * pitti bows to the higher deity of interweb tube failure and toddles off
[07:47] <sladen> ScottK: yeah, there may be a case for splitting it into multiple binary packages to allow for selective shipping  It would be good to have some feedback if that might be necessary
[07:47] <sladen> ScottK: do you have specific examples in mind?
[07:52] <infinity> sladen: Say, is it intentional that items on my desktop recently developed nearly-unreadable grey text, instead of... Whatever it was before that I could see?
[08:03] <pitti> yay, DC back for me, and kubuntu dvd build doesn't fall over right away
[08:06] <pitti> ... and there it goes again; my provider seems to hate me today
[08:06] <pitti> infinity: would you mind starting this in screen:
[08:06] <pitti> echo kubuntu preinstalled && buildlive kubuntu daily-preinstalled && (for-project kubuntu cron.daily-preinstalled &) && echo kubuntu-mobile preinstalled && buildlive kubuntu-mobile daily-preinstalled && for-project kubuntu-mobile cron.daily-preinstalled
[08:07] <jibel> skaet, I confirm the desktop upgrade issue bug 854490
[08:07] <ubot4> Launchpad bug 854490 in cups (Ubuntu Oneiric) (and 1 other project) "update from natty to oneiric hangs on libpam0g upgrade and cups reload (affects: 1) (heat: 8)" [Critical,New] https://launchpad.net/bugs/854490
[08:07] <infinity> Sure...
[08:10] <infinity> pitti: Running.
[08:17] <pitti> infinity: thanks, I see them (can log into the DC again)
[08:36]  * infinity finally passes out.
[08:36] <infinity> See you all in the morning, or some approximation thereof.
[08:38] <pitti> infinity: sleep well!
[08:39] <infinity> Thanks.  I'll be back in ~8 hours. ;)
[08:51] <mvo> skaet: I can confirm 854490 as well and we have some idea what the problem is now
[08:51] <mvo> jibel: heh, I knew that there would be a bugreport already about the cups thing, nothing escapes your hawk eyes :-)
[08:57] <jibel> mvo, :) I'm running the upgrade with debug mode enabled now.
[08:58] <jibel> mvo, BTW there is also bug 853688 that affects upgrades
[08:58] <ubot4> Launchpad bug 853688 in gcj-4.4 (Ubuntu Oneiric) (and 1 other project) "Natty to Oneiric - failed to calculate the upgrade with gcj-4.4-jre installed (affects: 12) (dups: 5) (heat: 52)" [High,Triaged] https://launchpad.net/bugs/853688
[09:00] <mvo> jibel: thanks! I put some info into #854490, see comment #11 and #13 for the cause of the issue
[09:01] <mvo> jibel: I got one report about a dpkg failure for duplicated changelogs:
[09:01] <mvo> Unpacking libglib2.0-0:i386 (from .../libglib2.0-0_2.29.92-0ubuntu1_i386.deb)
[09:01] <mvo> +...
[09:01] <mvo> dpkg: error processing
[09:01] <mvo> +/var/cache/apt/archives/libglib2.0-0_2.29.92-0ubuntu1_i386.deb (--unpack):
[09:01] <mvo>  './usr/share/doc/libglib2.0-0/changelog.Debian.gz' is different from the same f
[09:01] <mvo> +ile on the system
[09:01] <mvo> dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
[09:01] <mvo> jibel: have you seen anything like this? iirc that was a dpkg bug that got fixed in oneiric, but I may misremember
[09:04] <jibel> mvo, nope. nothing that I remember.
[09:06] <seb128> mvo, I've seen those errors
[09:06] <seb128> mvo, it's bug #835625
[09:06] <ubot4> Launchpad bug 835625 in apt (Ubuntu Oneiric) (and 1 other project) "apt may try to unpack a foreign-arch multiarch library before the native package is at a multiarch version, prohibited by dpkg (affects: 6) (dups: 5) (heat: 54)" [High,Fix released] https://launchpad.net/bugs/835625
[09:08] <seb128> slangasek, ^ btw I'm not sure there is no bug there, we got several duplicates about this change.Debian.gz being differents
[09:15] <mvo> seb128: hrm, hrm, the problem here of course is that its the old apt calculating the ordering
[09:33] <pitti> seb128: changelog being different> that's not just from locally built/installed debs?
[09:33] <seb128> pitti, no
[09:33] <seb128> I will not pretend I understand the bug, but we got several bugs from people doing natty to oneiric dist-upgrades
[09:34] <seb128> the bug title suggest that somebody understands what the issue is though " try to unpack a foreign-arch multiarch library before the native package is at a multiarch version, prohibited by dpkg"
[09:35] <pitti> eek, kubuntu alternate amd64 is uninstallable, nevermind; I'll rebuild it
[09:35] <pitti> (http://cdimage.ubuntu.com/kubuntu/daily/20110920.6/report.html)
[09:47] <pitti> http://cdimage.ubuntu.com/kubuntu/daily-live/20110920/ posted
[10:00] <pitti> http://cdimage.ubuntu.com/kubuntu/daily-preinstalled/20110920/ posted
[10:12] <Daviey> is apport-retracer working?
[10:13] <pitti> Daviey: i386 and dup checker should, amd64 is down ATM
[10:13] <pitti> I'm working on that on the side
[10:15] <ScottK> sladen: Kubuntu ships the Ubuntu font. I assume other flavors do too.
[10:15] <ScottK> pitti: We're doing kubuntu preinstalled for omap/omap4.
[10:16] <pitti> ScottK: thanks for confirming; I posted them (mobile still building)
[10:16] <ScottK> K.
[10:17] <Daviey> pitti: thanks
[10:17] <pitti> at last! http://cdimage.ubuntu.com/kubuntu/daily/20110920.8/
[10:18] <pitti> Daviey: ^ ... and you were complaining about .2 :)
[10:18] <Daviey> pitti: I want a .42!
[10:18] <pitti> don't say that too loudly :) mirrors and publisher have been so naughty today, it might actually happen
[10:19] <Daviey> heh
[10:23] <NCommander> pitti: killed (didnt see your ping earlier)
[10:23] <pitti> NCommander: thanks
[10:24] <pitti> note to whoever accepted gtimelog: it will be uninstallable until pango1.0 is accepted as well (no big deal, though)
[10:24] <pitti> that can happen after b2
[10:27] <ogra_> shriek
[10:27] <ogra_> something changed last night in debian-cd it seems ... and that broke ac100
[10:27]  * ogra_ curses loudly
[10:29] <Daviey> ogra_: aiui infinity changed the default for arm images to ext4, could that be related?
[10:30] <ogra_> he changed a lot more (and broke a lot more)
[10:30] <ogra_> grmbl
[10:44] <cjwatson> ev: please push your release commit of lupin to bzr
[10:45] <cjwatson> (bound branches!)
[10:45] <ev> cjwatson: will do
[10:45] <ev> oh, it is bound, but I hadn't done the actual debcommit -r yet
[10:45] <ev> done now :)
[10:46] <cjwatson> ah
[10:46] <cjwatson> I have an 'ubuntu-release' script to stop me forgetting that
[10:46] <cjwatson> http://paste.ubuntu.com/693683/
[10:49] <Daviey> Interesting, never used debrelease
[10:52] <ev> cjwatson: neat, will poach
[10:52] <ev> thanks
[10:52] <ogra_> cjwatson, hmm, is it correct that bin-compress_images overwrites the .type file ? publish-release really gets confused by this for tarballs
[10:52] <cjwatson> no idea
[10:52] <cjwatson> this stuff is totally your team's problem :)
[10:52] <ogra_> infinity added code to suppress the actual gzip call for already gzipped files
[10:53] <ogra_> but the original code still rewrites .type which makes all images end up as img.gz
[10:53] <ogra_> hmm, k
[10:53]  * ogra_ will wait for infinity ... i think that behavior is wrong in general not actually arm specific
[10:57] <pitti> NCommander: are you sure?
[10:57] <pitti> cdimage  21674  0.0  0.0  10440   208 pts/2    T    Aug04   0:00 nano index.html
[10:57] <pitti> NCommander: ^ still seems alive
[10:57] <pitti> state "T" seems like Ctrl-Z'ed without bg'ing it
[10:58] <pitti> i. e. an fg in your screen session should bring it back
[11:00] <pitti> ev: do we need this lupin upload in b2? it's got no bugs attached, so I don't know about the impact
[11:00] <ev> pitti: without it, Wubi installs will always boot, but they'll never have a proper grub configuration
[11:00] <pitti> and which images does this affect? only the wubi one, or desktop images, too?
[11:01] <ev> but they'll be fixed on first update
[11:01] <ev> just Ubuntu wubi images
[11:01] <pitti> wubi isn't built/published yet, but a lot of desktop images are already
[11:01] <ev> it doesn't affect Kubuntu wubi or anything else
[11:01] <pitti> ev: ah, that's fine then
[11:01] <pitti> ev: accepted, will wait for this for the wubi image build (that still needs a bit anyway)
[11:02] <ev> okay, cool
[11:02] <ev> wubi also is missing swap. I'm working on that now, but it'll be a post-b2 thing.
[11:03] <nigelb> 49
[11:03] <nigelb> err, sorry!
[11:39] <pitti> http://cdimage.ubuntu.com/daily/20110920.4/ posted
[11:39] <ogra_> would someone mind if i ran a cron.daily-preinstalled for ac100 ? (i need to check a fix)
[11:39] <ogra_> (or could someone do it who drives the builder anyway)
[11:40] <pitti> ogra_: let me check, before lunch I still had three images building
[11:40] <pitti> ogra_: but I haven't started ubuntu preinstalled builds yet, that's still in my pipe
[11:40] <ogra_> k, i only need the cdimage run, no new livefs
[11:40] <pitti> was waiting for control-center and ubiquity
[11:41] <ogra_> should be quick
[11:41] <pitti> we need a new livefs, though?
[11:41] <ogra_> why ?
[11:42] <pitti> ogra_: did you already build on a few hours ago?
[11:42] <ogra_> i need to test a fix in debian-cd, livefs sits on the livefs builder already
[11:42] <pitti> oh, does it? not from me
[11:42] <ogra_> i dont plan to publish that
[11:42] <ogra_> no idea how old that livefs is
[11:42] <pitti> my queue is: edubuntu/ubuntu dvd, ubuntu desktop+dvd, ubuntu preinstalled
[11:43] <ogra_> hmm, i would like to have it tested before you actually do a preinstalled build
[11:43] <ogra_> the test takes 10-12min ... a full build will take hours
[11:43] <pitti> posting http://cdimage.ubuntu.com/kubuntu-mobile/daily-preinstalled/20110920/
[11:43] <pitti> ogra_: ok, then go ahead please
[11:43] <ogra_> thx
[11:43] <pitti> ogra_: and let me know when I can start the full one
[11:44]  * pitti will rebuild core and ubuntu desktop/DVD in the meantime
[11:44] <ogra_> running, should be done soon (and shouldnt interfere with anything you do anyway)
[12:05] <ogra_> pitti, thanks, done (seems i need to rm the broken img.gz files, they seem to stick from former builds)
[12:09] <pitti> ogra_: working then? can I queue the ubuntu desktop preinstalled builds?
[12:09] <ogra_> wait a sec, it seems infinity switched debian-cd to ext4 over night but forgot some places
[12:10] <ogra_> while ac100 now works the others complain about the lack of an ext4 filesystem
[12:10] <ogra_> i guess buildlive wasnt switched
[12:10] <pitti> sure, waiting for your word then
[12:11] <ogra_> yeah, as i thought, still builds ext3 rootfses
[12:11]  * ogra_ fixes
[12:15] <ogra_> ohm, he swuitched already but after the last livefs build, thats why we have no ext4
[12:15] <ogra_> pitti, all fine, was justa  race between his commit and the last buildlive ... go ahead
[12:16] <pitti> ogra_: running
[12:43] <pitti> posted http://cdimage.ubuntu.com/daily-live/20110920/
[12:45] <pitti> jibel: ^ FYI
[13:13] <stgraber> good morning
[13:14] <pitti> ev: argh, booting the i386 CD just gives an ubiquity error
[13:15] <ev> rubbish
[13:15] <pitti> ev: AttributeError: "NoneType" object has no attribute "set_online_state"
[13:15] <ev> will pull down the CD now and fix
[13:15] <pitti> ev: known, or want a bug?
[13:15] <ev> bug please
[13:20] <cjwatson> oh good, that doesn't look like it was my fault
[13:22] <pitti> ev: bug 854706
[13:22] <ubot4> Launchpad bug 854706 in ubiquity (Ubuntu) "ubiquity crashed with AttributeError in set_state(): 'NoneType' object has no attribute 'set_online_state' (affects: 1) (heat: 8)" [Undecided,New] https://launchpad.net/bugs/854706
[13:25] <pitti> seb128, ev: is it deliberate that the desktop does not have an "Install 11.10" icon any more? ubiquity is just in the launcher now
[13:25] <seb128> pitti, I don't know about this change
[13:25] <ev> thanks
[13:27] <pitti> ev: followed up; only happens when the machine is offline
[13:28] <ev> seb128: that's not purposeful. Perhaps someone broke casper?
[13:28] <ev> err pitti :)
[13:28] <seb128> pitti, ls ~/Desktop ?
[13:33] <pitti> uh, it's there
[13:33] <pitti> ubiquity-gtkui.desktop
[13:33] <pitti> why the heck isn't it displayed?
[13:33] <pitti> examples icon is there, although with dark grey text, hard to read
[13:33] <seb128> pitti, is it bug #854401
[13:33] <ubot4> Launchpad bug 854401 in nautilus (Ubuntu Oneiric) (and 2 other projects) "the text on desktop icons doesn't adapt to the background color (affects: 6) (dups: 1) (heat: 32)" [High,Confirmed] https://launchpad.net/bugs/854401
[13:34] <pitti> seb128: OnlyShowIn=GNOME;XFCE;
[13:34] <seb128> pitti, yeah, the text color issue is "known", I've pinged upstream
[13:34] <seb128> pitti, oh, need to have Unity added
[13:34] <seb128> Unity is different from GNOME since oneiric
[13:34] <pitti> ev: ^ want a bug for adding "Unity" to OnlyShowIn as well?
[13:34] <pitti> or JFDI?
[13:35] <pitti> I confirmed that this fixed it
[13:35] <pitti> weird, how did we not get that in b1
[13:35] <ev> pitti: by all means, JFDI. Working on a fix for 854706 now
[13:35] <stgraber> pitti: do we have an edubuntu build scheduled? I was a bit surprised not to get my usual e-mail from the tracker this morning :) though based on the above, if there isn't one building at the moment, I'd rather wait for these ubiquity fixes.
[13:36] <pitti> ev: I can't commit to lp:ubiquity, so I'm afraid for JFDI you have to do the change
[13:36] <pitti> stgraber: yes, it's in the pipeline
[13:36] <ev> pitti: sure thing
[13:36] <pitti> stgraber: it's building already; I won't post it, I guess
[13:37] <pitti> stgraber: (unless you want to)
[13:37] <stgraber> pitti: ok. I'll rsync it anyway and do some tests on LTSP but won't do as much testing as an actual beta2 candidate
[13:37] <pitti> stgraber: I'll ping you once it's built
[13:37] <stgraber> pitti: thanks
[13:38] <pitti> jibel: http://cdimage.ubuntu.com/dvd/20110920/ is ready for smoke-testing, but won't publish to tracker due to bug 854706
[13:38] <ubot4> Launchpad bug 854706 in ubiquity (Ubuntu Oneiric) (and 1 other project) "ubiquity crashed with AttributeError in set_state(): 'NoneType' object has no attribute 'set_online_state' (affects: 1) (heat: 10)" [High,New] https://launchpad.net/bugs/854706
[13:39] <jibel> pitti, ack.
[13:39] <pitti> ScottK: ^ I'm not sure whether this affects Kubuntu as well
[13:53] <stgraber> pitti, ev: I commited the .desktop fix
[13:53] <ev> stgraber: thanks
[13:54] <pitti> stgraber: merci
[14:13] <skaet> good morning
[14:14] <stgraber> good morning skaet
[14:14] <utlemming> jamespage: ping
[14:20]  * skaet catches up on the backscroll,  whew, busy day. 
[14:23] <jamespage> hey utlemming
[14:25] <skaet> mvo,  thanks for confirmation on bug 854490.  Any update on bug 848659?
[14:25] <ubot4> Launchpad bug 854490 in pam (Ubuntu Oneiric) (and 1 other project) "update from natty to oneiric hangs on libpam0g upgrade and cups reload (affects: 6) (dups: 1) (heat: 22)" [Critical,Confirmed] https://launchpad.net/bugs/854490
[14:25] <ubot4> Launchpad bug 848659 in update-manager (Ubuntu Oneiric) (and 1 other project) "Upgrade from natty fails with 64-bit oneiric beta cd (affects: 1) (heat: 8)" [High,New] https://launchpad.net/bugs/848659
[14:26] <utlemming> jamespage: I believe your the one that I hit up on adding the daily build of oneiric to the tracker?
[14:26] <pitti> hey skaet, good morning
[14:26] <jamespage> utlemming: hrm - sorry I can't do that but I can kick off the automated testing when required - jibel are you able to ^^
[14:27] <utlemming> jamespage: getting my names crossed....can you kick the automated testing for the 20110920 daily build?
[14:27] <skaet> hey pitti, busy day for you I see.
[14:28] <jamespage> utlemming, sure - will do
[14:28] <pitti> skaet: yeah :)
[14:28] <skaet> pitti, am having problems since last night accessing ether pad,  is it working ok for you?
[14:28] <ogra_> skaet, yeah, we are coordinating in #ubuntu-keep-pitti-busy
[14:28] <ogra_> :)
[14:28] <pitti> skaet: it often times out here, but so far I was able to keep it up with the reconnect button
[14:28] <skaet> ogra_, lol
[14:30] <skaet> ogra_,  are there things on your must fix list today for ARM?  (saw some angst on the backscroll, but not sure if its sorted now or not)
[14:32] <skaet> pitti, yeah, in again now.  was seeing crashes/fails to find last night as I left though,  so might be prudent to keep a periodic screen scrape. :/
[14:32] <ogra_> skaet, i will know once the current build finishes
[14:32] <ogra_> waiting idly for that atm
[14:32] <skaet> ogra_, gotcha.
[14:32] <pitti> skaet: current builds are ubuntu preinstalled arm, and ubuntu chinese
[14:32]  * skaet crosses fingers for ogra_ 
[14:33] <pitti> but the current dvd/chinese builds are in vain, for above bug
[14:34] <skaet> pitti, which bug?   (there are a few of them up there ;) )
[14:34] <pitti> skaet: bug 854706, see pad
[14:34] <ubot4> Launchpad bug 854706 in ubiquity (Ubuntu Oneiric) (and 1 other project) "ubiquity crashed with AttributeError in set_state(): 'NoneType' object has no attribute 'set_online_state' (affects: 2) (heat: 14)" [High,Confirmed] https://launchpad.net/bugs/854706
[14:34] <skaet> pitti,  thanks!
[14:39] <slangasek> seb128, mvo: it's not bug #835625; 835625 is about trying to unpack a Multi-Arch: same package while a Multi-Arch: none one is still installed, the bug you're seeing is about the contents of the Multi-Arch: same packages being different - which means, according to Multi-Arch, that one of the packages has wrong contents.  Maybe there's still a package manager bug, but if so it's a different bug
[14:39] <ubot4> Launchpad bug 835625 in apt (Ubuntu Oneiric) (and 1 other project) "apt may try to unpack a foreign-arch multiarch library before the native package is at a multiarch version, prohibited by dpkg (affects: 6) (dups: 5) (heat: 54)" [High,Fix released] https://launchpad.net/bugs/835625
[14:39] <skaet> pitti,  is someone actively looking into 854706?  not clear from the bug assignments...
[14:40] <jibel> skaet, ev is on it
[14:40] <seb128> slangasek, ok, I'm just saying that the changelog.Debian.gz is the same between i386 and amd64 (checked from the .debs) and that several user ran into in the issue, it seems during natty to oneiric dist-upgrades
[14:40] <jibel> skaet, <ev> pitti: by all means, JFDI. Working on a fix for 854706 now
[14:40] <seb128> slangasek, the first bug we used to track it is bug #839744 but Torsten dupped it from the other one
[14:40] <ubot4> Launchpad bug 839744 in glib2.0 (Ubuntu) "package libglib2.0-0 2.29.16-0ubuntu3 [modified: usr/share/doc/libglib2.0-0/changelog.Debian.gz] failed to install/upgrade: ErrorMessage: './usr/share/doc/libglib2.0-0/changelog.Debian.gz' is different from the same file on the system (dup-of: 835625)" [Undecided,Confirmed] https://launchpad.net/bugs/839744
[14:40] <ubot4> Launchpad bug 835625 in apt (Ubuntu Oneiric) (and 1 other project) "apt may try to unpack a foreign-arch multiarch library before the native package is at a multiarch version, prohibited by dpkg (affects: 6) (dups: 5) (heat: 54)" [High,Fix released] https://launchpad.net/bugs/835625
[14:41] <ev> just updated the bug to reflect that I'm working on it
[14:41] <ev> (done now, just testing the hell out of it)
[14:41] <skaet> thanks ev
[14:41] <slangasek> seb128: one possible explanation for this is that /usr/share/doc/libglib2.0-0/changelog.Debian.gz is a symlink on their system due to some previous upgrade issue
[14:41] <skaet> thanks jibel.
[14:46] <slangasek> seb128: also note at the top of bug #839744 that the changelog file is listed as "modified"
[14:46] <ubot4> Launchpad bug 839744 in glib2.0 (Ubuntu) "package libglib2.0-0 2.29.16-0ubuntu3 [modified: usr/share/doc/libglib2.0-0/changelog.Debian.gz] failed to install/upgrade: ErrorMessage: './usr/share/doc/libglib2.0-0/changelog.Debian.gz' is different from the same file on the system (affects: 2) (heat: 12)" [Undecided,Confirmed] https://launchpad.net/bugs/839744
[14:50] <seb128> slangasek, right, same on bug #839744
[14:50] <ubot4> Launchpad bug 839744 in glib2.0 (Ubuntu) "package libglib2.0-0 2.29.16-0ubuntu3 [modified: usr/share/doc/libglib2.0-0/changelog.Debian.gz] failed to install/upgrade: ErrorMessage: './usr/share/doc/libglib2.0-0/changelog.Debian.gz' is different from the same file on the system (affects: 3) (dups: 1) (heat: 22)" [Undecided,Incomplete] https://launchpad.net/bugs/839744
[14:58] <slangasek> seb128: I see you marked 843836 as a duplicate too, and that has nothing to do with upgrades; were you meaning to dupe it to a different bug?
[14:59] <seb128> slangasek, yes, probably a tab-by-one error, fixing it, thanks for pointing it ;-)
[15:00] <seb128> btw can be interesting for people there:
[15:00] <seb128> http://mail.gnome.org/archives/desktop-devel-list/2011-September/msg00127.html
[15:00] <seb128> it's the first draft of next GNOME cycle, https://live.gnome.org/ThreePointThree#Schedule
[15:01] <seb128> 3.4.1 would be for april 16
[15:01] <seb128> which would be nice to get in the lts if we can
[15:05] <ev> would someone remind doing peer review on http://paste.ubuntu.com/693820/ and http://paste.ubuntu.com/693821/ . It's a small change, but I'd rather prefer a second set of eyes on any change I make to pciutils :)
[15:07] <pitti> ev: http://paste.ubuntu.com/693820/ > how does that work?
[15:07] <pitti> ev: at least with dh_install, *.dirs files are usually not necessary at all
[15:07] <pitti> but even if you have the dir, don't you need to modify an *.install file or debian/rules to actually install it there?
[15:08] <ev> ah, indeed
[15:08] <ev> apols, I did that quickly and apparently without thinking clearly
[15:08] <pitti> ev: http://paste.ubuntu.com/693821/ > I think that needs to be something like "usr/sbin/biosdevname sbin/"
[15:08] <cjwatson> biosdevname to /sbin> implementation details aside, I endorse this product or service
[15:08] <pitti> ev: if upstream make install installs to /usr/sbin/
[15:08] <cjwatson> or configure --sbinidir
[15:08] <pitti> ev: or configure with a --bindir
[15:08] <cjwatson> with spelling and everything
[15:18] <skaet> cloud images (20110920) have been posted to tracker.
[15:19] <Daviey> \o/
[15:19] <skaet> :)
[15:41] <jamespage> utlemming, running now - litmus test across all regions was OK
[15:41] <utlemming> jamespage: glady to hear :)
[15:41] <jamespage> results will appear here - > https://jenkins.qa.ubuntu.com/job/oneiric-server-ec2/
[15:41] <jamespage> in about 1.5 hours
[15:44] <pitti> skaet: can you take over the coordination now? I'm in desktop meeting and will call it a day soon
[15:44] <skaet> pitti,  can do
[15:44] <pitti> skaet: right now we are mainly waiting for a new ubiquity, and then respin everything desktop-y
[15:45] <pitti> skaet: my ubuntu arm preinstalled images shouldn't take too long any more, I'll still post them when they are done
[15:45] <skaet> pitti, sounds good.   I'll let you post those last and then set up do the respin the images when ubiquity lands.
[15:55] <skaet> slangasek,  what's the outlook on bug 854490?  (would like to include it in the next set of respins)
[15:55] <ubot4> Launchpad bug 854490 in pam (Ubuntu Oneiric) (and 1 other project) "update from natty to oneiric hangs on libpam0g upgrade and cups reload (affects: 6) (dups: 1) (heat: 22)" [Critical,Confirmed] https://launchpad.net/bugs/854490
[15:56] <jibel> ev, could you look at bug 854717 before any respin.
[15:56] <ubot4> Launchpad bug 854717 in ubiquity (Ubuntu Oneiric) (and 1 other project) "Broken panel icons and dialog style during ubiquity-dm and OEM install/final user configuration (affects: 1) (heat: 6)" [Critical,Confirmed] https://launchpad.net/bugs/854717
[15:57] <slangasek> skaet: haven't had a chance to look into it yet; it's possible we can improve the pam logic, but I think something's seriously wrong with cups if it hangs like this
[15:57] <ev> jibel: will do
[16:00] <skaet> slangasek, understood.  Will ping and see where we are before the next respin.
[16:13] <pitti> slangasek: do you have an idea how to avoid it?
[16:14] <pitti> slangasek: it tries to start cups while it's libraries and daemon are being upgraded/unpacked/unconfigured
[16:14] <pitti> no amount of Depends: can rectify this, AFAICS
[16:14] <pitti> slangasek: would it be hard to apply the "ignore if it is not already running" logic that it applies to gdm to also apply to all other services?
[16:18] <slangasek> pitti: why does cups hang when things are not yet upgraded?  if /usr/sbin/cupsd fails to run, it should fail quickly... even hitting the respawn limit shouldn't take very long
[16:21] <slangasek> pitti: I'm not keen on "check if it's running, then start"; gdm is a special case because we have to use 'reload' instead of 'restart', the same logic would not be appropriate in general
[16:21] <pitti> slangasek: according to mvo's tests, it actually should fail quickly, with a symbol error
[16:21] <pitti> slangasek: oh, wow; I actually expected that to be fairly uncontroversial
[16:21] <pitti> trying to start a daemon from an unconfigured package with unconfigured dependencies is prone to fail, and not really necessary either -- it will restart itself once it's configured
[16:21] <slangasek> well, it basically means we're bypassing the invoke-rc.d policy interfaces that exist for a reason
[16:22] <slangasek> so I want to be sure I understand why we need to bypass them before doing so
[16:22] <pitti> well, we could also fix this in invoke-rc.d to not start a daemon if the package isn't configured
[16:22] <mvo> slangasek: it fails quickly, but there seems to be no respawn limit
[16:22] <pitti> but that seems harder to do
[16:23] <slangasek> it may be that we should just get rid of the "invoke-rc.d $foo stop" / "invoke-rc.d $foo start" and replace it with restart
[16:23] <slangasek> mvo: there's a default respawn limit
[16:23] <pitti> and even if we work around it in cups, we don't want to apply workarounds to the other 20 packages
[16:23] <mvo> slangasek: how big is that?
[16:23] <mvo> slangasek: thats in upstart itself, right?
[16:23] <pitti> and it would be a work around no matter what; we can't suddenly introduce the expectancy that a packagae will DTRT when not being configured
[16:23] <slangasek> mvo: "Default COUNT is 10. Default INTERVAL is 5 seconds." - so this should hang the upgrade for 50 seconds max
[16:23] <slangasek> mvo: yes
[16:23] <pitti> that only exists for essential pacakges
[16:24] <slangasek> pitti: I'm *not* expecting that the package will start when not configured
[16:24] <pitti> slangasek: asked the other way around, why does the postinst need to start services which are not running in the first place?
[16:24] <slangasek> I'm expecting that it not hang indefinitely
[16:24] <slangasek> it doesn't!
[16:24] <mvo> interessting, I'm pretty sure that I waited (way) longer then 50s
[16:24] <pitti> I thought the idea of that was to restart *running* services to make them use the new libpam
[16:24] <slangasek> it is, but that doesn't mean I agree with the hammer you've chosen :)
[16:25] <slangasek> please let me think through this horrid code for a bit
[16:25]  * mvo runs the test again with a stop-watch and more patience ;)
[16:25] <pitti> mvo: I wonder if the problem is that upstart doesn't notice that cupsd has failed, and then runs the post-start script?
[16:25] <pitti> because that tries to talk to the daemon, and might cause long timeouts
[16:27] <slangasek> invoke-rc.d restart doesn't help; this is defined to try to start the service if not already running
[16:29] <pitti> right, I think we need to ask invoke-rc.d <service> status
[16:29] <slangasek> I'm still really of the opinion that we should not have to care in the postinst about whether the service is configured yet.  invoke-rc.d is supposed to protect us from starting services that should not be started by policy, and trying to start an unconfigured service should fail gracefully
[16:29] <slangasek> and 'status' is an optional init script target
[16:29] <jibel> slangasek, mvo I waited way longer than 50s like I had time to have lunch.
[16:31]  * pitti needs to leave for an hour or two, will check in again later
[16:31] <slangasek> pitti: ^^ so we can't do a wholesale switch to 'status' without reviewing whether each one of these services implements the target
[16:31] <pitti> slangasek: I see; so as an approximation we might rather check if the package is configured then?
[16:32] <slangasek> pitti: might it not be easier to fix whatever's causing cups to hang, which we should fix anyway?
[16:32] <pitti> slangasek: it might, but who tells us that the very same problem doesn't apply to the other services it tries to restart?
[16:32] <slangasek> calling dpkg a couple dozen times isn't going to make the upgrade any faster :)
[16:33] <slangasek> pitti: the fact that this code has been in place for years and this is the first time anyone's reported a problem :)
[16:33] <pitti> as I said, we can certainly work around this in cups, but we still need to check an upgrade with the others
[16:33] <slangasek> it's not a "workaround".  'invoke-rc.d cups start' should not hang indefinitely, regardless of whether the package is configured; that it does is a bug in the cups package somewhere
[16:34] <pitti> I still think it's wrong to try and start unconfigured packages, but if you say it's too hard to fix in libpam0g, I'm ok with looking into cups here specifically
[16:35] <pitti> added a cups task
[16:36] <pitti> will look into this tomorrow morning, so hopefully we have something working for b2
[16:36] <pitti> and crossing fingers that it doesn't affect other packages
[16:37] <pitti> (cups' upstart script also hasn't changed for ages, so it's probably just bad luck that we got this particular unpack order now)
[16:37]  * slangasek nods
[16:38] <pitti> anyway, good night! see you tomororw
[16:38] <slangasek> g'night :)
[16:38]  * slangasek has a poke at cups himself
[16:42] <slangasek> pitti, mvo: at least some of the slow-down is caused by the post-start script; upstart sees cups as started, then stopped, so it waits for the post-start script to finish running before it tries to respawn
[16:43] <slangasek> oh, and this means it never hits the respawn limit
[16:44] <slangasek> because the respawn limit is defined as COUNT respawns in INTERVAL seconds, and the post-start script takes 10 seconds to run
[16:51] <mvo> slangasek: fwiw, its running on in my auto-upgrade tester (or rather its not running ;) I will have dinner and see if anything happens when I come back
[16:51] <slangasek> mvo: ok. I've reproduced the problem locally by manually hamstringing cupsd
[16:58] <slangasek> mvo, pitti: cups library interdependencies are wrong.  pam *already* only restarts services that dpkg reports as installed; new libcupsmime1 has been unpacked without deconfiguring cups, but somehow breaks the old version of cups.
[16:58] <slangasek> sounds like libcupsmime1 dropped symbols without an appropriate soname change
[16:59] <ScottK> pitti: I think it does.
[17:07] <skaet> ogra_,  how are the ARM images looking?  Is there something that goes on the must fix & respin list - or should some images be posted for further testing?
[17:08] <ogra_> skaet, still waiting
[17:08] <ogra_> we havent had successfull builds for desktop since august 30th
[17:08] <Ursinha> jesus
[17:08] <skaet> ogra_, ack.
[17:09] <ogra_> Ursinha, i doubt he can help :)
[17:09] <ogra_> unless he is a soyuz coder ;)
[17:09] <Ursinha> ogra_, heh
[17:09] <Ursinha> ogra_, what's the problem?
[17:09] <GrueMaster> I wouldn't say that.  Send him a panda.
[17:09] <Ursinha> soyuz? ugh
[17:09] <Ursinha> lol
[17:09] <ogra_> *giggle*
[17:10] <Ursinha> ogra_, you said soyuz, anything wrong with Launchpad?
[17:10] <ogra_> Ursinha, lets go to #ubuntu-arm, i dont want to spam the channel
[17:10] <Ursinha> sure
[17:10] <GrueMaster> Ursinha: Between pool skew and mirror issues, we haven't had a desktop image since Beta 1.  Unfortunately I was buried in server testing to notice.
[17:12]  * infinity sees one today, and one yesterday, at least...
[17:12] <stgraber> pitti: is 20110920 the edubuntu images you said were building this morning? (just want to check I can smoke test these)
[17:22] <jamespage> utlemming: https://jenkins.qa.ubuntu.com/job/oneiric-server-ec2/8/
[17:23] <jamespage> ec2 testing completed - only a couple of issues
[18:05] <slangasek> mvo: uhoh, I've gone full circle and now think bug #854490 is an apt bug ;)
[18:05] <ubot4> Launchpad bug 854490 in pam (Ubuntu Oneiric) (and 3 other projects) "update from natty to oneiric hangs on libpam0g upgrade and cups reload (affects: 6) (dups: 1) (heat: 24)" [Critical,Invalid] https://launchpad.net/bugs/854490
[18:06] <slangasek> because in the log, the cups package has never been deconfigured... but it's definitely broken
[18:06] <slangasek> libcups2 declares a Breaks: on old cups already
[18:08] <slangasek> hmm, but libcups2 *hasn't* been unpacked yet
[18:27] <smoser> slangasek, skaet bug 854927 seems of high to critical priority to me
[18:27] <ubot4> Launchpad bug 854927 in openssl (Ubuntu) "wget, curl can't verify certificates (affects: 1) (heat: 6)" [High,Confirmed] https://launchpad.net/bugs/854927
[18:29] <kirkland> smoser: slangasek: +1, I found this this morning in ec2
[18:30] <kirkland> smoser: slangasek: no https from the command line == bad for beta :-(
[18:30] <skaet> smoser, ack.  thanks for flagging
[18:31] <slangasek> smoser, kirkland: is this a regression vs. beta-1?
[18:31] <kirkland> slangasek: yes, this is a regression since yesterday
[18:31] <kirkland> slangasek: or thereabouts
[18:31] <smoser> it is demonstratable regression on ec2 instances.
[18:31] <slangasek> uh
[18:32] <kirkland> slangasek: i'm reproducing here on my up-to-date oneiric laptop, wget https://google.com
[18:32] <slangasek> well, we haven't changed openssl in the past 24 hours; the last update was 6 days ago?
[18:32] <kirkland> http://paste.ubuntu.com/693943/
[18:32] <kirkland> slangasek: i was thinking maybe it was blocked/queued until recently?
[18:33] <slangasek> kirkland: no, my laptop shows libssl1.0.0 was upgraded on my system 4 days ago
[18:33] <kirkland> slangasek: do you have an up-to-date oneiric?  can you wget https://google.com?
[18:33] <kirkland> slangasek: note the "https"
[18:33] <Daviey> erm
[18:33] <Daviey> I'm not sure i can reproduce this
[18:33] <slangasek> kirkland, smoser: can you give me a package delta between the working/non-working images, please?
[18:33] <kirkland> looks like wildcard certs are the problem
[18:34] <smoser> slangasek, working on it.
[18:34] <Daviey> http://pb.daviey.com/YLVR/
[18:34] <kirkland> slangasek: smoser: wget https://www.google.com works, but not google.com
[18:34] <Daviey> ah yes
[18:34] <kirkland> hmm
[18:34] <slangasek> kirkland: so did google change the certs on their side?
[18:34] <Daviey> smoser: on your bug report, www failed?
[18:34] <kirkland> okay, https://launchpad.net
[18:35] <Daviey> https://launchpad.net WFM
[18:35] <ev> jibel: I've had a look at that bug, but I'm not going to be able to fix it tonight, so I've uploaded ubiquity as is.
[18:35] <kirkland> slangasek: I'm wrong about my local laptop being affected
[18:35] <kirkland> slangasek: ec2 images, though definitely are
[18:35] <smoser> i can verify failure of www.google.com on a CanoniStack instance right now
[18:35] <kirkland> Daviey: yeah, sorry about that
[18:35] <Daviey> smoser: i cannot.
[18:35] <slangasek> kirkland: are you missing ca-certificates from the ec2 images, then?
[18:35] <utlemming> I've confirmed it outside EC2 on virtualbox VM -- same story
[18:35] <smoser> my local laptop is affected
[18:35] <ev> jibel: it appears to happen in regular ubiquity as well.  Probably something we're missing from the ubiquity-dm session or a race condition.
[18:35]  * Daviey suspects the cers are missing
[18:35] <slangasek> openssl doesn't ship any CA policy
[18:35] <Daviey> certs
[18:36] <ev> jibel: though I've only been able to reproduce it in oem mode
[18:36] <kirkland> slangasek: $ dpkg -l | grep cert
[18:36] <kirkland> ii  ca-certificates                 20110502+nmu1                           Common CA certificates
[18:36] <kirkland> ii  ssl-cert                        1.0.28                                  simple debconf wrapper for OpenSSL
[18:36] <slangasek> that all comes from the ca-certificates package
[18:36] <Daviey> ubuntu@server-972:~$ dpkg -l | grep cert
[18:36] <Daviey> ii  ca-certificates                  20110502+nmu1                           Common CA certificates
[18:37] <Daviey> smoser: I cannot reproduce this in an upgraded oneiric instance.
[18:37] <Daviey> It sounds to me like an issue with the latest cloud image builds missing someting.
[18:37] <slangasek> kirkland: strace -ff -e file wget https://launchpad.net 2>&1 | grep usr/lib/ssl ?
[18:38] <kirkland> slangasek: http://paste.ubuntu.com/693945/
[18:38] <Daviey> slangasek: http://pb.daviey.com/okGB/
[18:38] <utlemming> Daviey: I would agree -- expect my VirtualBox instance is a Beta1 Server that was installed ISO and upgraded
[18:38] <smoser> i only see failure with google.com on local system.
[18:38] <smoser> so yes, that would seem to indicate not a global issue.
[18:38] <smoser> and user failure to report that. but it does reproduce with www.google.com on ec2 instance.
[18:38] <slangasek> kirkland: you get different behavior than me or Daviey
[18:38] <slangasek> (I see what Daviey sees)
[18:39] <slangasek> kirkland: I notice that the stat size is completely off
[18:40] <kirkland> b5534824873e947df1c3e07bedfd688e  /usr/lib/ssl/certs/55a10908.0
[18:40] <kirkland> lrwxrwxrwx 1 root root 19 2011-09-20 01:34 /usr/lib/ssl/certs/55a10908.0 -> ca-certificates.crt
[18:40] <Daviey> Those who are seeing this, please can you confirm the history of the install?  daily AMI from what day?
[18:40] <Daviey> utlemming: you seeing it from iso install, dated?
[18:40]  * slangasek blinks
[18:40] <kirkland> -rw-r--r-- 1 root root 240312 2011-09-20 01:32 /usr/lib/ssl/certs/ca-certificates.crt
[18:40] <slangasek> yes, that's not normal
[18:41] <slangasek> the hash symlinks normally point to the individual certs
[18:41]  * kirkland is still getting used to Ubuntu Monospace ...  he just read "yes, that's not moral"
[18:42] <kirkland> slangasek: how are these installed/generated?  I find this puzzling:
[18:42] <kirkland> ubuntu@domU-12-31-39-04-9C-3B:~$ dpkg -S /usr/lib/ssl/certs/ca-certificates.crt
[18:42] <kirkland> dpkg-query: no path found matching pattern /usr/lib/ssl/certs/ca-certificates.crt
[18:42] <highvoltage> I thought that that is called dyslexia?
[18:43] <utlemming> ISO 2011-08-31, and upgraded. Seen on daily 2011-09-20. Checking others now
[18:43] <kirkland> Daviey: i'm running the current day's AMI
[18:43] <slangasek> kirkland: /usr/lib/ssl/certs is a symlink to /etc/ssl/certs, the contents of which are managed by the ca-certificates package
[18:44] <slangasek> ugh, mixed spaces/tabs, slap slap
[18:45] <kirkland> slangasek: in a makefile or something?
[18:45] <slangasek> no, in the postinst
[18:45] <slangasek> making it difficult to read
[18:46] <smoser> ok
[18:46] <smoser> http://paste.ubuntu.com/693953/
[18:46] <smoser> is the data i have.
[18:47] <smoser> 20110914 works, 20110918 fails
[18:47] <smoser> package diff at pastebin
[18:48]  * Daviey raises again that if we had a snapshot archive like Debian have, we could bisect this automatically.
[18:48] <slangasek> smoser: oh. nothing more recent than 20110914?
[18:49] <smoser> i'll try bisect, slangasek
[18:49] <slangasek> ok, yes, this is a regression in openssl; I can recreate the issue by running c_rehash manually
[18:49] <kirkland> openssl *has* changed in smoser's list
[18:49] <smoser> i can try a 15 and an 18
[18:49] <smoser> err.. 15 and 17
[18:49] <slangasek> (which wouldn't get run on an upgrade because there was nothing in need of rehashing)
[18:51] <mvo> slangasek: I can have a close look into the cups issue tomorrow morning (in ~ +12h). but I think its best to workaround in libpam0g or cups for now
[18:52] <utlemming> singe 16 was broken due to a armel build failing...
[18:52] <slangasek> mvo: I'm not sure how to sanely work around it in either place.  Maybe the new libcupsmime1 should also say it Breaks: cups?
[18:52] <Daviey> utlemming: arm failure breaks the supported arches?
[18:53] <Daviey> slangasek: Are you driving the ssl issue?
[18:53] <smoser> ok. 20110915 pass, 20110917 fail
[18:53] <smoser> getting diff of those
[18:53] <slangasek> Daviey: yeah, I can; I'm close to a workaround via ca-certificates
[18:53] <Daviey> rocking
[18:54] <slangasek> cjwatson: ^^ when you're around, maybe you can comment on the behavior change in c_rehash suddenly preferring ca-certificates.crt as a target and blowing up
[18:56] <smoser> http://paste.ubuntu.com/693959/
[18:56] <slangasek> Daviey, smoser, kirkland: do we need to fix this for upgrades from these images, or do we only care about fixing it on new images?
[18:57] <mvo> slangasek: yeah, I can muck around a bit and see if I can get it to work
[18:57] <slangasek> mvo: ok, thanks
[18:57] <Daviey> slangasek: Not many people use oneiric ec2 images, so the support of them can be weak... however, this also hits people who installed the daily iso for the last 6 days, on all flavours?
[18:57] <Daviey> Desktop aswell?
[18:58] <smoser> slangasek, well i would hope we'd support someone upgrading from beta-1.
[18:58] <smoser> admittedly there are not likely a lot of people who would care though.
[18:58] <slangasek> smoser: this doesn't affect people upgrading from beta-1
[18:58] <Daviey> smoser: upgrades are not the issue, it's fresh installs
[18:58] <slangasek> it *only* affects people who installed in the window since the last openssl upgrade
[18:59] <smoser> can those people fix themselves ?
[18:59] <smoser> i would think it would be ok if there were a documented work around for those 6 days of install
[19:00] <slangasek> I'm not sure where to document a workaround... or whether it's worth the effort, considering we're talking about people installing from random dailies
[19:01] <smoser> well, for hte cloud-images, i think its ok to ignore it.
[19:01] <smoser> but really, if someone did a install of the isos, and their ssl is busted, and the "fix" is reinstall, that sucks.
[19:01] <utlemming> if you're installing from dailys, lets face it, you like pain. I would suggest we put it in the bug.
[19:02] <slangasek> lp:ubuntu/ca-certificates contains a tentative fix; I'm afk for a few minutes, can people put that change through its paces?
[19:03] <slangasek> to fix an existing install, it should be sufficient to run update-ca-certificates --fresh with the new package
[19:05] <Daviey> bah, might it just be easier to include that in the postinst if people are upgrading from that package?  It can be dropped when openssl is next uploaded after this.
[19:06] <Daviey> FWIW, i don't care that much for the 6 day window.
[19:06] <Daviey> People should expect some manual work-arounds pre-release.
[19:07] <smoser> i tihnk its < 6 day window too
[19:07] <smoser> 20110915 daily build PASS
[19:07] <smoser> 20110917 daily build FAIL
[19:08] <smoser> so worst case its 20110916 -> 20110921 window.
[19:08] <Daviey> meh
[19:10] <smoser> so current desktop isos would be affected?
[19:10] <smoser> and server isos ? or am i missing something.
[19:11] <Daviey> smoser: It should be all flavours, surely.
[19:11]  * utlemming confirms fix
[19:13] <smoser> utlemming, did you build a binary ?
[19:13] <utlemming> smoser: yes, through pbuilder
[19:14] <utlemming> installed it and then ran "update-ca-certificates --fresh"
[19:16] <smoser> and that fixes issue ?
[19:16] <utlemming> yes, I verified it on two instances and I'm doing a couple more for good measure
[19:22] <slangasek> Daviey: yeah, we can probably call that from the postinst; but it's a heuristic, and I'm always wary of heuristics when trying to land an urgent fix
[19:25] <skaet> Daviey, can you review the latest nova upload in the pending queue, and determine if you want it accepted and included in the next server build?
[19:26] <utlemming> smoser, slangasek: I tested the fix on 6 instances that had the problem. After updating and running update-ca-certificates, the problem goes away
[19:27] <Daviey> sure
[19:27]  * Daviey lols at seeing "skype (source)" in the queue.
[19:28] <zul> Daviey:  it contains some fixes needed for juju as well
[19:28] <Daviey> zul: ack
[19:29] <Daviey> can squid be accepted please?
[19:32] <skaet> Daviey, yeah looks straight forward.  accepted
[19:33] <Daviey> ta
[19:33] <Daviey> zul: nova-2011.3~rc~20110920.r1192/debian/nova_sudoers seems to have introduced a tab vs space issue?
[19:34] <zul> yeah it should be fine
[19:37] <skaet> doko,  why is there a binutils in queue - bug number I'm spotting is bug: 690194, which doesn't look like release blocker.  Am I misunderstanding something?
[19:37] <Daviey> zul: didn't we want to add the flag True for force_dhcp_release ?
[19:38] <Daviey> (as in, the whole reason we got dnsmasq-utils into beta2?)
[19:38] <zul> Daviey: crappers..are you sure?
[19:38] <Daviey> zul: can you confirm, perhaps i missed something?
[19:39] <skaet> doko, and bug 778292,  which is down as medium
[19:39] <ubot4> Launchpad bug 778292 in gcc-4.6 (Ubuntu Oneiric) (and 5 other projects) "undefined reference to `pow' when building with -flto (affects: 4) (heat: 24)" [Medium,Invalid] https://launchpad.net/bugs/778292
[19:39] <zul> daviey: that branch only contained the debian/control and the sudoers change
[19:40] <zul> perhaps i missed something
[19:40] <Daviey> zul: Yeah, the merge from Vish didn't include the conf file change.
[19:40] <Daviey> (it defaults to False.)
[19:40] <Daviey> we want True
[19:41] <sbeattie> slangasek: behavior change in c_rehash is likely due to the patch that fixed http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=628780
[19:41] <ubot4> Debian bug 628780 in openssl "Wrong hash link to cacert.org.pem and wron certificat hash handling at all" [Important,Fixed]
[19:41] <zul> Daviey: ok gimme a sec
[19:41] <slangasek> sbeattie: aha, thanks for the pointer
[19:41] <Daviey> zul: fancy resolving the tab vs space?
[19:41] <slangasek> ca-certificates uploaded; can someone review please?
[19:41] <slangasek> infinity: ^^ ?
[19:42] <infinity> slangasek: On it.
[19:42] <slangasek> skaet: ^^ that's the fix for bug #854927; ca-certificates is on all images, so a respin seems to be in order
[19:42] <ubot4> Launchpad bug 854927 in openssl (Ubuntu Oneiric) (and 3 other projects) "c_rehash creating bogus links to ca-certificates.crt (affects: 2) (heat: 10)" [High,Triaged] https://launchpad.net/bugs/854927
[19:42] <infinity> Well, on it when it lands...
[19:43] <Daviey> zul: is the lack of not declaring a --bridge an issue?
[19:43] <skaet> slangasek,  yup.
[19:43] <slangasek> (since this is a major problem for cloud images and they can't release beta2 with it)
[19:43] <zul> Daviey: no
[19:43] <Daviey> zul: grep for, 'WARNING! ACHTUNG! Setting the bridge to br100 '
[19:43] <Daviey> ok
[19:44] <doko> skaet, it's not marked as a blocker. it's a regression fix for our ld --as-needed default
[19:46] <infinity> slangasek / skaet: Any qualms about letting in that new light-themes?  It's not particularly RC, but the broken desktop text is certainly something that will make Beta screenshots awful. :P
[19:46] <infinity> (And I assume we're rebuilding anyway for other reasons)
[19:46] <skaet> infinity,  I was wondering about that too.   ok, by me.
[19:47] <zul> Daviey: ok uploading
[19:47] <skaet> infinity, and yes we're rebuilding.  see: ubiquity fix, and ca-certificates
[19:48] <infinity> +nmu1ubuntu1 ... That version needs more icky.
[19:49] <infinity> I, err... Uhm.
[19:49] <infinity> slangasek: lolwut?
[19:49] <slangasek> ?
[19:49] <infinity> +       if [ "$2" = 20110502+nmu1 ]; then
[19:49] <infinity> +         update-ca-certificates --fresh
[19:49] <infinity> +       else
[19:49] <infinity> +         update-ca-certificates --fresh
[19:49] <infinity> +       fi
[19:49] <slangasek> sih
[19:49] <slangasek> sigh
[19:49] <slangasek> thanks :)
[19:50] <slangasek> infinity: please reject, I'm reuploading with -v
[19:51] <infinity> Rejectified.
[19:52] <infinity> The info box really needs to not say "OK: <package>" on rejects.  I keep having little heart attacks.
[19:52]  * skaet makes note of rejectified,   plans to use it with twirling images.
[19:53] <infinity> skaet: Rejectificated is also useful.  It all depends on how hard you press the button.
[19:55] <slangasek> ca-certificates reuploaded
[19:55] <slangasek> should I stall the publisher for a few minutes?
[19:55] <Laney> FYI I got some kind of libproxy working but I think it is probably too much too hard at this time
[19:55] <infinity> slangasek: With how long it's been taking, I would.
[19:56] <slangasek> done
[19:56] <Daviey> slangasek: Are we planning a respin in the next 2 hours?
[19:56] <Daviey> s/respin/twirl
[19:57] <infinity> slangasek: Also, congrats on missing the upload queue by 5 seconds.
[19:57] <slangasek> heh
[19:57]  * infinity goes for a smoke for the next 3 minutes.
[19:58] <ScottK> Sigh.  Kubuntu desktop just grew by 5MB.
[20:00] <ScottK> Trimmed.
[20:00] <skaet> ScottK.  okie.  adding it to the respin list.
[20:01] <doko> o kdesvn: kdesvn-kio-plugins libsvnqt6
[20:01] <doko>    [Reverse-Depends: kdesdk, kdesvn-kio-plugins]
[20:01] <doko> ScottK, ^^^ still on your list?
[20:02] <infinity> slangasek: Accepted.
[20:02] <slangasek> ta
[20:02] <ScottK> doko: It's an artifact of kdesdk FTBFS on powerpc which I've got no hardware to work on.  michahg said he might take a shot at it on the porter box, but it needs someone with hardware.
[20:03] <ScottK> The powerpc hardware I had access to died.
[20:03] <doko> ahh, ok
[20:03] <ScottK> It's probably a reasonably tractable FTBFS is someone with hardware would take alook.
[20:04] <infinity> ScottK: I could give you remote access to my firewall, if you promise to be a Good Boy.
[20:04] <ScottK> Does it have a oneiric chroot that you could install the build-deps for kdesdk into?
[20:04] <infinity> slangasek: Please don't publish until ubiquity's built all over.
[20:04] <slangasek> ack
[20:05] <infinity> Which should actually be really soon...
[20:05] <infinity> ScottK: It could do.  I have the technology.
[20:06] <Daviey> skaet: can the topmost nova be accepted, and the prior one be rejected please?
[20:06] <skaet> Daviey,  go ahead, you know them better than me. :)
[20:06] <Daviey> skaet: I'm not an archive-admin. :)
[20:07] <Daviey> I assume ~ubuntu-release was an archive admin.. seems not.
[20:08] <Laney> that reminds me
[20:08]  * Laney repings
[20:09] <skaet> Daviey, hmm. thought that ubuntu-release had the perms.   going in now.
[20:10] <skaet> topmost: 2011.3~rc~20110920.r1192-0ubuntu2 - accepted.
[20:10] <Daviey> ta
[20:10] <Laney> I guess nobody reads ubuntu-archive@, eh?
[20:10] <Daviey> Laney: you mean the most noisest list ever?
[20:10] <skaet> 2011.3~rc~20110920.r1192-0ubuntu1 - rejected
[20:11] <Daviey> ta
[20:11] <Laney> that's why He invented procmail
[20:11] <Daviey> Laney: do you filter out SRU acceptance crap?
[20:11] <ScottK> skaet: There's an open bug asking to get ubuntu-release access to the unapproved page in LP.
[20:11] <Daviey> Laney: things like, https://lists.ubuntu.com/archives/ubuntu-archive/2011-September/043772.html really don't interest me.
[20:12] <skaet> ScottK, if you can find the bug number at some point,  I'll add it to the list of ones I'm discussing with the launchpad team.
[20:12] <ScottK> Sure.
[20:13] <Laney> Daviey: I'm not subscribed to that list, but I have some heavy duty filters for other ones
[20:13] <Laney> on the order of Launchpad? => Blah/bulk
[20:14] <Laney> launchpad gives you headers, but they can only go so far
[20:14] <Daviey> Laney: well yes, we all do that.. but what i am saying is that ~ubuntu-archive isn't the most interesting of reads, at the best of times.
[20:15] <Daviey> The only time i look at it, is when someone mentions something that makes me look.
[20:15] <Laney> sometimes people do need to mail the archive administrators though, and for that there is no other list
[20:17] <infinity> Meh.  We need to replace all our old arm buildds with more Pandas.  This ubiquity build is taking longer than I expected.
[20:18] <infinity> (And just when we have enough Pandas, maybe we'll get real server hardware!)
[20:19] <ScottK> skaet: Bug #648611
[20:19] <ubot4> Launchpad bug 648611 in launchpad "Queue admin permissions need pocket and status granularity (affects: 1) (heat: 1)" [High,Triaged] https://launchpad.net/bugs/648611
[20:20] <slangasek> infinity: is there an eta for that build?  You want me to keep waiting?
[20:20] <skaet> Thanks ScottK,  added it to my tracking
[20:20] <slangasek> ah, it's on dh_builddeb, that's promising
[20:22] <infinity> slangasek: That's deceptive.  It's d-i, it builddebs about 4000 times.
[20:22] <slangasek> ah, heh.
[20:22] <slangasek> so, no ETA?
[20:22] <infinity> Not sure.  I could look at an old build log and see where we are. :P
[20:23] <infinity> Can't respin armel without it, though (like, not at all, since it causes skew with oem-config)
[20:33] <skaet> slangasek, infinity, while we're waiting for the arm builder, could you check the pad and confirm I've got the state of what needs to be rebuilt accurate?
[20:34] <skaet> http://pad.ubuntu-uk.org/ubuntu-release
[20:36] <GrueMaster> I would also suggest rebuilding kubuntu for arm (unless they don't want to release).  They will need the ubiquity fixes and the jasper-initramfs fixes.
[20:37] <infinity> skaet: I'm not sure what the archive skew bit's meant to be about.  Was that just Oliver having a bit of a fit earlier? :P
[20:37] <GrueMaster> But not my call.
[20:38] <infinity> skaet: It can probably be boiled down to "everything but ubuntu-core needs rebuilding due to ubiquity and ca-certs"
[20:38] <skaet> infinity,  yes, wasn't sure if there were problems, since he was waiting for a rebuild.
[20:38] <skaet> infinity,  yeah, that's what its looking like.
[20:38] <GrueMaster> And netinstall.  We don't need it respun.
[20:38] <infinity> skaet: Yeah, I saw him freaking out in backscroll.  Most of the issues this morning were actually mirror issues, but many other scapegoats were found. ;)
[20:39] <infinity> GrueMaster: netinstall == d-i upload, not an image.
[20:39] <skaet> GrueMaster, netinstall?  or you mean netboot?
[20:39] <infinity> skaet: Same thing.
[20:39] <skaet> coolio. :)
[20:40] <infinity> netinst and netboot are just ways of saying "those random little pxe and mini iso images that d-i spits out".
[20:40] <GrueMaster> what he said.
[20:40] <GrueMaster> I look at what iso.qa lists for Arm.  Everything but core and netboot. to respin.
[20:41] <infinity> Dear buildd with the unpronouncable hostname, please build ubiquity faster.  No love, Adam.
[20:41] <skaet> LOL
[20:41] <GrueMaster> Do the kubuntu guys have someone to test their images?  If not, I will need to plan for it (after my regular testing).
[20:42] <skaet> ScottK ^^
[20:42] <ScottK> Not for omap/omap4, afaik.
[20:42] <ScottK> I'll check though.
[20:44] <infinity> slangasek: Ooo, we're in the home stretch.
[20:45] <slangasek> does that mean it'll be done before the next regularly scheduled publish? :)
[20:45] <infinity> Yeah, yeah.
[20:45] <infinity> I may have SLIGHTLY miscalculated that based on not realising that it was building on a babbage...
[20:46] <infinity> Still armel images take the longest to build, it sort of makes sense to get the archive in order for them first, and then queue up the world.
[20:46] <infinity> Ish.
[20:46] <ScottK> Got one maybe for omap.
[20:47] <ScottK> He's checking to see the status of the system.
[20:49] <skaet> infinity,  just talked to rbelem,  kubuntu mobile images aren't going to be published with beta 2.   Removing them from the ISO tracker.
[20:50] <infinity> skaet: This doesn't hurt my feelings.
[20:50] <slangasek> skaet: I don't see anything missing from the pad, but I'm not sure that I would either
[20:52] <skaet> slangasek,  thanks.
[20:52] <infinity> Well, honestly, if you just run pitti's rebuild blocks of doom, minus the two "ubuntu-core" bits at the beginning of the preinstalled block, you should be good to go. :P
[20:52] <infinity> Assuming nothing fails.
[20:52] <slangasek> ubiquity armel accepted, publisher now running
[20:52] <infinity> \o/
[20:52] <slangasek> well yes, I'm not sure if those are "pitti's rebuild blocks of doom" or if they're modified :)
[20:52] <skaet> :)
[20:52] <infinity> If they're modified, he'll be grumpy.
[20:53] <infinity> He took a while writing those pipelines. :P
[20:53] <slangasek> mine were better, less duplication of code :)
[20:53] <skaet> mine had timestamps, so I could gather stats.  ;)
[20:55] <infinity> Kids, kids.
[20:55] <infinity> You can each bring me a beer.
[20:55] <infinity> And then someone can write a parser for etc/crontab and have a proper script on antimony to rebuild the world. :P
[20:56] <slangasek> that would be slow compared to the current scripts
[20:56] <slangasek> the cronjobs wouldn't interleave
[20:57] <infinity> Well, I implied a "good" parser.
[20:58] <infinity> One smart enough to know how to do what the current mess does, but adapt.
[20:58] <infinity> Also, as sarcastic as this suggestion is, with crontab being stuffed as full as it is these days, this does get worse every release.
[20:58] <infinity> I remember when "rebuild the world" meant "Ubuntu alternate and live for three arches".
[20:59] <infinity> And we whined about how long THAT took.
[20:59] <slangasek> we could, hmm, stop letting it get more stuffed?
[21:00] <infinity> I like this idea.
[21:00] <infinity> We could drop everything but Xubuntu.
[21:00] <infinity> All in favour?
[21:00] <skaet> heh, yeah,  there are all those arm images ;)
[21:01]  * skaet just wants to not build what doesn't get tested, as a starter. 
[21:08] <skaet> Daviey, https://launchpad.net/ubuntu/+source/nova/2011.3~rc~20110920.r1192-0ubuntu2/+build/2796534
[21:09] <Daviey> skaet: hmm.
[21:09] <Daviey> So it's Dep-wait.
[21:09] <skaet> its Dependency wait on vernadsky - retry?
[21:09] <Daviey> Please don't assume it wasn't build tested.
[21:09] <infinity> skaet: No point retrying.
[21:09] <Daviey> The reason it's now depwait, is that nova has been promoted to main since the last build
[21:10] <Daviey> The dep it is waiting on is still universe, and i've already chased the MIR.
[21:10] <infinity> ^
[21:10] <infinity> skaet: It'll retry itself when that MIR is sorted.
[21:12] <Daviey> infinity: jdstrand is trying to grok the complex package, it was only recently handed over to him from the previous person undertaking the MIR.
[21:12] <infinity> Daviey: Seems a bit odd that you got nova promoted before it's build-deps though. :P
[21:12] <skaet> Daviey wasn't assuming it wasn't build tested. ;)  just wondering if temporary glitch.  universe -> main would explain it.
[21:12] <Daviey> We either pre-promote it, and make doko shout at us - or just hold out for jdstrand to finish. :)
[21:12] <Daviey> no pressure jdstrand :P
[21:12] <infinity> Please don't pre-promote it.
[21:13]  * doko is wondering why people wait with MIR's until the last  minute ...
[21:14] <Daviey> doko: Is that a general assumption, or specific to this one?
[21:15] <Daviey> MIR opened 2011-06-23 fwiw.
[21:16] <doko> Daviey, people very often cry for pre-promotion, and don't realize that the MIR is incomplete on their side. yes, for some this seems to be business as usual
[21:17] <Daviey> I'm sure i have been, and will be in the future guilty of that.
[21:18]  * Daviey pre-apologises
[21:20] <highvoltage> it's impossible to pre-apologise.
[21:23] <slangasek> nah; it would be impossible to pre-metalogize, but that's not a word :)
[21:24] <slangasek> publisher claims to be done
[21:25] <infinity> Thanks to the magic of language evolution, it is now.
[21:25] <slangasek> skaet: ^^ are you driving the respins?
[21:25] <infinity> Probably better if one of us does, she has Big Plans tonight.
[21:25] <skaet> infinity,  can you do the honors.  Would rather you're able to watch it.
[21:25] <infinity> I'll flip a coin for it.
[21:26] <skaet> or slangasek.
[21:26] <skaet> :)
[21:26] <infinity> slangasek: My coin says it's your turn.  But it could be wrong.
[21:26] <slangasek> heh
[21:27] <slangasek> infinity: would appreciate it if you could do it, I was intending to be off-line doing flicker-free boot debugging
[21:27] <infinity> I'm happy to, if you and Patty have plans tonight.  And no, making plans hastily in the next 5 minutes doesn't count. ;)
[21:27] <infinity> Oh, or if you're working on other work crap.  Sure.  I'll respin.
[21:28] <skaet> infinity,  I'll go ahead and mark the iso tracker that the respins have started.
[21:28] <slangasek> infinity: thanks
[21:28] <infinity> Alright.  Any last words from anyone?
[21:29] <slangasek> klatuu verata oneiricto
[21:29] <infinity> *snicker*
[21:29] <stgraber> yeah! might have something to test for Edubuntu tonight then (for some value of tonight I guess)
[21:30] <infinity> Hrm.  Going to skip wubi and chinese-edition from the loop, unless anyone has arguments against that?
[21:32] <infinity> And, they're off.
[21:34] <infinity> And our first round of livefs builds fail.
[21:34] <infinity> Huzzah.
[21:37] <infinity> Oh, FFS.
[21:37] <infinity> E: Invalid Release signature (key id 40976EAF437D05B5)
[21:37] <infinity> *headdesk*
[21:38] <slangasek> which means we need to stab the mirroring?
[21:38] <infinity> slangasek: Wait... How did you run the publisher?
[21:38] <slangasek> copying the command in lp_publish's crontab
[21:38] <infinity> slangasek: That shouldn't be a mirror at all... livefs builds (unless broken), should be pulling from ftpmaster.internal.
[21:38] <infinity> Which should NEVER be broken.
[21:38] <slangasek> /srv/launchpad.net/codelines/current/cronscripts/publish-ftpmaster.py -d ubuntu -q
[21:39] <infinity> LPCONFIG=ftpmaster-publish
[21:39] <infinity> ?
[21:39] <slangasek> oh bah
[21:40] <slangasek> this setting of env vars at the top of crontabs, it is not a thing of good
[21:40] <slangasek> running again
[21:43]  * infinity wonders how this broken signature situation affects alternates, if at all...
[21:48]  * jdstrand doesn't feel any added pressure. I'm getting to all these this week and actively working on them now
[21:50] <infinity> jdstrand: Would you feel more pressure if Daviey repeatedly poked your forehead while you worked?
[21:50]  * jdstrand sorta feels that way already :P
[21:51] <jdstrand> seriously though, I got all this this week, and am working on them this week
[21:52] <infinity> The correct answer is "every time someone bugs me about something, it loses an hour in my queue".
[21:52] <infinity> Which, given the expense of context switches, is likely true.
[21:52] <infinity> So, shoo.
[21:52] <infinity> Stop being bugged.
[21:52] <Laney> ScottK: is there any precedent for bouncing wishlist FFes to backports?
[21:54] <slangasek> infinity: ok, publisher done again
[21:54] <infinity> slangasek: With 38% more keysigning?
[21:54] <slangasek> maybe?
[21:54] <infinity> slangasek: Or, releasesigning, rather.
[21:55]  * infinity asks a livefs builder.
[21:55] <slangasek> that's an annoying failure mode.
[21:56] <infinity> It's an annoying variable to have to set.  We used to have per-machine configs that got symlinked on LP rollout.
[21:56] <infinity> Apparently, that was effort, so now it's all in variables in crontabs.
[21:56] <infinity> And I weep.
[21:57] <infinity> Okay, no failures yet, we'll call it good.
[21:58] <infinity> On the plus(?) side, debian-cd seems to completely ignore sigs for alternates (this could really be seen as a bug), and just blissfully downloads unverifiable packages and re-signs them with the cdimage key.
[21:58] <infinity> I guess if we don't have a trusted path to our own internal archive, we have bigger problems, but that still feels broken. :P
[22:04] <infinity> It probably doesn't help that someone has LPCONFIG=ftpmaster in lp_publish's bashrc...
[22:04]  * infinity fixes.
[22:05] <infinity> slangasek: ^-- That should make it less error-prone, I hope. :P
[22:09] <skaet> infinity, publisher crontab re-enabled?
[22:11] <infinity> Maaaaybe.
[22:11] <infinity> (apparently, yes)
[22:12] <slangasek> infinity: heh, ok
[22:13] <skaet> :)
[22:37] <skaet> infinity,  am going to need to leave now for a while.
[22:38] <infinity> skaet: Mmkay.
[22:38]  * skaet passes baton to infinity to keep an eye on those images and bugs. 
[22:38] <skaet> :)
[22:38] <skaet> Thanks!
[23:38] <sC> are the ubuntu studio images being respun again?  the beta images are showing a line through them in the QA website
[23:38] <sC> argh
[23:38] <ScottL> ah, better
[23:42] <infinity> ScottL: Everything's being respun, yeah.
[23:42] <charlie-tca> yes, all images are being respun
[23:46] <ScottL> oh, okay...thank you