| === jbicha is now known as Guest32183 | ||
| === Guest32183 is now known as jbicha_ | ||
| === jbicha_ is now known as jbicha | ||
| === jbicha is now known as Guest48606 | ||
| === Guest48606 is now known as jbicha_ | ||
| seb128 | infinity, cjwatson, wgrant: hey, do you know what are the disk space limitations on ppa builders? is there any way we could have libreoffice somewhat build on machines that have > 40G for free disk space? | 09:41 |
|---|---|---|
| wgrant | seb128: no, there's no way to do that. how can it possibly need suxh a ridiculous volune? | 09:44 |
| seb128 | it's libreoffice, don't ask... | 09:44 |
| seb128 | c++ objects are not small | 09:44 |
| seb128 | webkit for example with a 5Mb source uses some 15GB of disk space to build | 09:45 |
| wgrant | sure, but it still seems implausible | 09:48 |
| wgrant | regardless, not something we can support. you would probably have to talk to IS | 09:49 |
| infinity | IS can't do much about it either, other than upgrade the world. | 09:52 |
| infinity | This isn't exactly new. | 09:52 |
| seb128 | well, hardware improve, you could think that 50G of disk space is not a lot by today's standards | 09:55 |
| seb128 | there is no way we will make libreoffice smaller with our one maintainer | 09:56 |
| wgrant | seb128: we have multiple guests on a single host, and lots of old hardware that we are stuck with | 09:56 |
| infinity | seb128: It's not, but it's a matter of them either swapping hard drives in a bunch of old computers (which they don't want to do), or wait for the cloudified builder. | 09:56 |
| seb128 | Sweetshark suggested turning off tests on the buildders to workaround the space issue :/ | 09:56 |
| infinity | seb128: Turning off tests for PPA builds is fine, as long as he doesn't do it for archive uploads. :P | 09:56 |
| seb128 | so archive builders don't have that space issue | 09:57 |
| infinity | I thought he was already mangling his PPA builds anyway. | 09:57 |
| seb128 | I guess the ppa wouldn't either if it was devirtualized, right? | 09:57 |
| infinity | No, archive buildds are fine in this regard. | 09:57 |
| seb128 | he is mangling them | 09:57 |
| seb128 | but that's becoming ridiculous | 09:57 |
| seb128 | and that makes ppa builds different from archive builds | 09:57 |
| seb128 | so not as good as a testbed for archive uploads as they could be | 09:58 |
| seb128 | we had bugs in the mangled part before which meant archive builds failed | 09:58 |
| infinity | I remember, I fixed that once. :/ | 09:59 |
| * infinity wonders why britney hasn't britnized in 5 hours... | 10:04 | |
| ogra_ | "britnized" ? | 10:05 |
| ogra_ | oh my | 10:05 |
| infinity | ogra_: Britnified? | 10:07 |
| ogra_ | i dont know, i kind of get a picture of a bold and suddenly chubby girl on drugs in my head if you say that | 10:08 |
| infinity | cjwatson: Has the autopkgtest integration caused britney to asplode, or...? | 10:08 |
| cjwatson | Unlikely, since it's not committed yet. | 10:10 |
| infinity | So process 15176 doesn't relate? | 10:10 |
| cjwatson | That's me trying to pdb things into existence - it shouldn't interact | 10:10 |
| infinity | Anyhoo, britney hasn't output anything useful in ~5h, I was just quickly finger-pointing without cause. :) | 10:10 |
| cjwatson | I started that process more like 20 hours ago | 10:11 |
| cjwatson | Well, 19 | 10:11 |
| cjwatson | infinity: proposed-migration/log/2013-06-12/10:03:43.log shows a crash | 10:11 |
| cjwatson | I'll need considerably more coffee if you want me to debug that :) | 10:12 |
| * infinity shoves a coffee packet in the CD slot, and clicks 'brew'. | 10:13 | |
| === doko_ is now known as doko | ||
| wgrant | seb128: Do we know how much space libreoffice actually needs nowadays? | 10:23 |
| seb128 | wgrant, I need confirmation from Sweetshark, the number he gave first were putting the build around 30GB ... but that's the disk space used at the end of the build, it seems to spike over that during the build (some objects are copied and then replaced by symlinks) | 10:29 |
| seb128 | I asked him to join the channel when he reads my ping | 10:29 |
| Sweetshark | seb128: joined, but I have to go to lunch, I have an appointment. Ill read the backlog though. | 10:32 |
| seb128 | Sweetshark, ok, wgrant was asking for an estimate for the space you need for the build | 10:32 |
| seb128 | Sweetshark, your number seems to be in range from 30GB to 100GB ... the second one seems a bit high, Laney managed to build in memory recently using 32GB iirc | 10:33 |
| wgrant | 100GB!? | 10:35 |
| Sweetshark | seb128: so, I would estimate the peak of the work directory to be in the 30-35GB range. The 100GB figure was the whole pbuilder-root (to take into account the big set of deps that LO has). | 10:35 |
| * wgrant wanders off to sob quietly | 10:35 | |
| Sweetshark | 110GB that is. | 10:35 |
| czajkowski | God lord 110GB! | 10:35 |
| seb128 | wgrant, I think 30GB is a better estimate, 100GB was a du on a full pbuilder with the libreoffice build-depends installed (which seems to be like half of the archive) | 10:36 |
| Laney | the one I managed to do in RAM was an arch-dep build only | 10:36 |
| seb128 | wgrant, make it 40GB to be safe | 10:36 |
| infinity | seb128: 100ish sounds more accurate, then. | 10:36 |
| Laney | arch+indep ran out of space | 10:36 |
| doko | cjwatson, http://people.canonical.com/~ubuntu-archive/proposed-migration/ seems to be outdated | 10:36 |
| infinity | seb128: The PPAs don't magically install build-deps on another disk, after all. | 10:37 |
| infinity | doko: Yeah, it's crashing. | 10:37 |
| doko | ouch | 10:37 |
| seb128 | infinity, yeah, I guess if you count system + builddir | 10:38 |
| infinity | If libreoffice and its build-deps keep growing, the archive buildds won't be able to build it soon. :/ | 10:39 |
| infinity | They're easier to justify upgrading but, still, that's insane. | 10:39 |
| seb128 | well, we are not creating the situation | 10:39 |
| seb128 | but yeah, it sucks | 10:39 |
| infinity | We're not helpless pawns, either. | 10:39 |
| seb128 | not sure what else to do though :/ | 10:39 |
| infinity | I suspect things could be done, patches could be submitted. | 10:40 |
| seb128 | right | 10:40 |
| wgrant | What takes up all the space? | 10:40 |
| wgrant | I actually cannot comprehend it. | 10:40 |
| wgrant | It's insane. | 10:40 |
| seb128 | Sweetshark, ^ | 10:40 |
| doko | Sweetshark, split build! | 10:40 |
| Sweetshark | infinity: https://bugs.freedesktop.org/show_bug.cgi?id=60924 and followups might make split builds easier in the long run, but that still takes time ... | 10:40 |
| ubot2` | Freedesktop bug 60924 in Libreoffice "move libraries to autoinstallation in scp2" [Normal,New] | 10:40 |
| infinity | Probably the most obvious (but painfully time-intensive) win would be splitting the upstream source. A lot. | 10:40 |
| infinity | wgrant: The fundamental problem is that it's a bunch of libraries with a bunch of build-deps and a bunch of binaries with a buch more build-deps, all in the same build tree. | 10:41 |
| infinity | wgrant: Imagine if all of KDE built in one go, instead of the hundreds of packages it currently is. | 10:41 |
| wgrant | what, like 4 years ago? :) | 10:42 |
| infinity | A bit longer than that. :P | 10:42 |
| infinity | 4 years ago, it was in nice little related bundles, I guess. | 10:42 |
| infinity | kdegames, etc. | 10:42 |
| infinity | But libreoffice is kinda like kdelibs + kderuntime + koffice + kitchensink | 10:43 |
| Sweetshark | doko: also note the irony of the thing causing us to run out of space is running the subsequenttests. and with a split build, that would need to be reconsidered anyway: subsequenttests can detect an error in any of the split packages, but can only run with all of them build. | 10:43 |
| Sweetshark | doko: so that would need reevaluation than anyway. | 10:44 |
| === Termana is now known as Guest58875 | ||
| infinity | Sweetshark: Turn them into runtime tests instead of build-time tests, and they can be tested agaisnt the binary packages. | 10:44 |
| infinity | Sweetshark: (For debian/ubuntu, that would be autopkgtest, but others have similar post-install test frameworks) | 10:45 |
| Sweetshark | infinity: I already did this cycle: http://skyfromme.wordpress.com/2013/03/19/autopkgtests-for-adults/ | 10:45 |
| infinity | Is it safe to click that link? | 10:46 |
| * infinity is suspicious. | 10:46 | |
| Sweetshark | infinity: here is the catch: they were also too big for the default autopkgtest images ;> | 10:46 |
| Sweetshark | infinity: its sfw | 10:46 |
| * Sweetshark gotta run | 10:47 | |
| xnox | Sweetshark: talk to unity/dx people, I think it should be fine to run on top of utah and or otto. With later being quite fast and caching/keeping dependencies pre-installed. | 11:13 |
| xnox | Sweetshark: looking at lp:auto-package-testing it does support resizing the disk image and thus you can have bigger disk for autopkgtest in jenkins. | 11:17 |
| cjwatson | Sorry for the delay in proposed-migration coming back; I'm still attacking it with gdb | 12:15 |
| cjwatson | Er, pdb | 12:15 |
| ogra_ | cjwatson, i assume it will still take a while ? (if so i'll disable the touch cronjob since i need the two last uploads in the next build) | 13:25 |
| cjwatson | When would it fire? | 13:26 |
| ogra_ | 6 min according to my clock | 13:26 |
| cjwatson | Oh, there's no way you'll get new packages by then anyway - the publisher still has to run | 13:26 |
| cjwatson | Even if I fixed it right now | 13:26 |
| ogra_ | i'm fine doing a manual build later ... i just dont want it to run automatically and block for 90min | 13:26 |
| ogra_ | ok | 13:27 |
| ogra_ | no prob :) | 13:27 |
| cjwatson | It's something complicated in the undo logic for rolling back failed hints, I think | 13:27 |
| ogra_ | yeah, no worries | 13:28 |
| cjwatson | Ah, I think I might see the problem ... | 14:00 |
| cjwatson | I bet undoing changes in reverse order would improve things | 14:00 |
| Sweetshark | xnox: well, yeah. I already resized the autopkgtest image locally -- it works. Otherwise I wouldnt have blogged "it works" ;). | 14:02 |
| xnox | Sweetshark: yeah there is a -S size option in prepare-testbed in that branch. Maybe talk to qa or pitti about bumping the size for libreoffice or bumping the size in general. | 14:04 |
| cjwatson | argh argh argh. reverse the undo list and I get a different crash | 14:13 |
| jibel | Sweetshark, xnox I increased the size to 12GB. Will it be enough for LO? | 14:19 |
| xnox | jibel: doesn't it need something like 40GB? | 14:20 |
| jibel | xnox, if the test builds the package, probably. | 14:23 |
| jibel | xnox, currently it crashes because it downloads 2GB of dependencies which uses 5G of disk space after installation, let see how far it goes with the bump. | 14:28 |
| chrisccoulson | can someone please approve that flash upload? ^^ | 14:40 |
| cjwatson | Phew, I think I've fixed proposed-migration | 14:55 |
| cjwatson | Running now | 14:55 |
| ogra_ | \o/ | 14:56 |
| cjwatson | There we go. Next publisher run should do it | 14:59 |
| === Sweetsha1k is now known as Sweeshark | ||
| === mmrazik is now known as mmrazik|afk | ||
| === jbicha is now known as Guest15484 | ||
| === Guest15484 is now known as jbicha_ | ||
| === james_ is now known as Guest33807 | ||
| infinity | jbicha_: ^-- Do you really mean it this time? | 17:05 |
| jbicha_ | infinity: yes :) | 17:05 |
| infinity | chrisccoulson: You might want to kick the -0precise1 habit in favor of -0.12.04.1 before we wrap around the alphabet. ;) | 17:10 |
| === rtg_ is now known as rtg-afk | ||
| chrisccoulson | infinity, won't support for flash have ended by the time we roll around? ;) | 17:30 |
| chrisccoulson | i can't remember when they announced the 5 years of security updates for the current version | 17:31 |
| infinity | chrisccoulson: One can hope so. It's more about breaking the habit in general. :) | 17:31 |
| === TheLordO- is now known as TheLordOfTime|EC | ||
| === TheLordOfTime|EC is now known as LordOfTime|EC2 | ||
| bdmurray | cjwatson: with the phased-updates work how would you imagine sru-release working since it should set the phased update percentage to something low? | 17:38 |
| cjwatson | yes, I was expecting it would crank the PUP on the copy in -updates down to something low | 17:39 |
| cjwatson | however I wonder if that's possible without waiting for the copy to actually take place ... | 17:40 |
| infinity | Not if it's an override. | 17:40 |
| infinity | The copy job needs to happen, then you can override in the queue before it actually publishes. | 17:40 |
| infinity | Well, not in the "queue", per se, since it has a pending publishing record at that point, but you know what I mean. | 17:41 |
| bdmurray | so perform the copy then use changeoverride to set the PUP low? | 17:43 |
| cjwatson | right, the problem is copies are async and you have to wait; right now an expected wait of a little over 30 seconds +/- 30 seconds | 17:44 |
| cjwatson | I was wondering if copyPackage(phased_update_percentage=blah) would be sane | 17:45 |
| infinity | Overrides are inherited on copy, why don't we just set it in proposed, then copy? | 17:45 |
| cjwatson | I guess that would work too | 17:45 |
| cjwatson | Does it copy pending override changes? | 17:46 |
| infinity | Oh, probably not. | 17:46 |
| infinity | I meant set it earlier. | 17:46 |
| cjwatson | Try it on dogfood I guess | 17:46 |
| infinity | But I guess that's undesirable. | 17:46 |
| cjwatson | It might, I just don't believe any particular property about copies until I've seen it | 17:46 |
| infinity | We want it 100% in proposed, but something < 1 in updates, I guess? | 17:46 |
| bdmurray | Right | 17:46 |
| cjwatson | Yeah, that's the thing, dropping the PUP in -proposed might inhibit the testing we get | 17:47 |
| infinity | How long are these phase cycles meant to be in general, anyway? | 17:47 |
| infinity | I get antsy when the cool kids get a google app a day or two before me, a week would be lame. :P | 17:47 |
| bdmurray | Also regarding "trying it" - I don't have the proper permissions to use changeoverride iirc | 17:48 |
| cjwatson | You could on dogfood | 17:48 |
| cjwatson | It has independent team membership | 17:48 |
| bdmurray | Is dogfood staging? | 17:49 |
| cjwatson | No | 17:49 |
| infinity | No, it's dogfood. | 17:49 |
| cjwatson | However, if sru-release is going to need to change overrides, you'd need those permissions on prod | 17:49 |
| cjwatson | So let's see if you have them already | 17:49 |
| bdmurray | When we were testing checkbox last week I didn't. | 17:50 |
| infinity | Yay, more permissions and more potential for subtle archive breakage. :/ | 17:50 |
| cjwatson | Looks like you need launchpad.Append on the archive, and it's not per-series. | 17:50 |
| cjwatson | Seems to me that ought to be tied into the per-series permissions for other things. | 17:50 |
| infinity | Perhaps, except that as long as only ubuntu-archive had the permission, no one cared. | 17:51 |
| cjwatson | Anyway, by way of testing, you're in ~ubuntu-archive on dogfood now. | 17:51 |
| infinity | Honestly, expanding that one scares me a bit in general. Bad override mangling can cause very hard-to-debug behaviours. | 17:51 |
| cjwatson | Well, let me put it a different way. If ~ubuntu-sru stop being able to use sru-release, that would be bad. | 17:52 |
| cjwatson | I agree override mangling can be a problem, but it's a problem even today, realistically ... | 17:52 |
| infinity | Can we not just have a default PUP, and ask AAs to twiddle if your package is a unique snowflake? | 17:52 |
| cjwatson | Default just for copies into -updates? | 17:53 |
| cjwatson | It's all a bit magic. | 17:53 |
| infinity | It's absolutely a problem today, I agree, I just fixed a bunch of overrides in post-release pockets this morning. | 17:53 |
| infinity | I'd just prefer to limit the number of fingers in that pie. | 17:53 |
| cjwatson | copyPackage(phased_update_percentage=) would allow sru-release to control this without opening overrides in general. | 17:53 |
| infinity | Would it? Wouldn't that still require them to have the permission? | 17:54 |
| cjwatson | Don't see why. | 17:54 |
| infinity | Well, the copy already does. I suppose the other bit could skip a permission check, sure. | 17:54 |
| infinity | That might be the path of least scary, then. | 17:54 |
| cjwatson | The permission check is on the BPPH.changeOverride interface, but the copier has to create a new BPPH anyway and it could just set it up in the desired state to start with. | 17:55 |
| infinity | That's a fair point. | 17:55 |
| infinity | I like that proposed solution, then. | 17:55 |
| infinity | It means ubuntu-sru can't alter the PUP post-release, but that's probably not a big deal. | 17:56 |
| cjwatson | That was all going to be done by a cron job anyway, IIRC | 17:56 |
| cjwatson | I guess we'll see whether the LP guys think it's too magic ... | 17:56 |
| cjwatson | Or too specialised | 17:56 |
| infinity | Sure, I meant if someone felt they messed it up or needed to force a new value. | 17:56 |
| infinity | But asking an AA to intervene there seems fine. | 17:56 |
| bdmurray | if that means incrementing and setting to 0 for a cronjob, then yes I'm working on that | 17:57 |
| === jbicha is now known as Guest12501 | ||
| === Guest12501 is now known as jbicha_ | ||
| ScottK | At least if it was 0 in proposed we'd get fewer complaints about regression in proposed. | 18:33 |
| === rtg-afk is now known as rtg_ | ||
| bdmurray | infinity / cjwatson: so uing change-override with -l dogfood should be okay then? | 19:55 |
| infinity | bdmurray: Should do. Given that you don't have access to mess up the primary archive anyway. ;) | 19:56 |
| bdmurray | I'm getting a httplib2.CertificateHostnameMismatch: Server presented certificate that does not match host api.dogfood.launchpad.net: error | 19:57 |
| stgraber | haha, yeah, classic problem, the certificate is for *.launchpad.net which doesn't cover *.*.launchpad.net and so covers dogfood.launchpad.net but not api.dogfood.launchpad.net | 19:59 |
| bdmurray | great | 19:59 |
| stgraber | so maybe there's an environment variable or option you can pass to bypass the SSL cert check, otherwise I'm affraid you may have to patch your local copy of lazr to skip the check... | 20:00 |
| bdmurray | did I say great already? ;-) | 20:01 |
| stgraber | infinity: can you add me to ~ubuntu-archive on dogfood looks like the DB import is a bit old | 20:02 |
| stgraber | (and the TB doesn't own that one for some reason) | 20:03 |
| infinity | stgraber: Done. | 20:04 |
| stgraber | bdmurray: LP_DISABLE_SSL_CERTIFICATE_VALIDATION=True change-override ... | 20:04 |
| stgraber | infinity: thanks | 20:04 |
| stgraber | bdmurray: obviously, that's usually a very bad idea to set that env variable so make sure that it's not persistent :) | 20:05 |
| bdmurray | stgraber: right of course. thanks! | 20:11 |
| bdmurray | Hrm, now I'm getting a "Cannot change overrides in suite 'quantal'" error | 20:22 |
| infinity | bdmurray: Because you can't, it's stable. | 20:29 |
| infinity | bdmurray: Did you mean quantal-updates, perhaps? | 20:29 |
| bdmurray | infinity: ah yeah, I'd used -s quantal no -s quantal-updates | 20:32 |
| wgrant | cjwatson: Copies don't preserve PUP, do they? | 23:47 |
| wgrant | I'm pretty sure there's a comment in copyBinaries saying that there's no point | 23:47 |
| cjwatson | You may be right | 23:47 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!