| nacc | rbasak: well that's nice to see -- latest importer picked up the unapplied where it was, and is imported all the applied, so far successfully, so i can do the php7.0 merge | 00:06 |
|---|---|---|
| rbasak | Nice! | 00:12 |
| === Spydar007 is now known as Guest92796 | ||
| === cpaelzer_ is now known as cpaelzer | ||
| === JanC is now known as Guest5365 | ||
| === JanC_ is now known as JanC | ||
| === klebers_ is now known as klebers | ||
| === _salem is now known as salem_ | ||
| nacc | rbasak: ah i see i had to revert HEAD -- we need an indevelopment feature of git to allow us to skip known empty changes | 15:31 |
| nacc | rbasak: but that would end up skipping (potentially) changes that are now empty becuase they can be dropped | 15:32 |
| nacc | rbasak: which we don't want | 15:32 |
| rbasak | I see, OK. | 15:33 |
| nacc | rbasak: i put a comment in the comment with a link to a patch for git that might let us specify it more clearly | 15:34 |
| nacc | rbasak: already empty commits vs. commits that have become empty | 15:34 |
| nacc | rbasak: but not yet existent, so i reverted from `usd merge` and fixed the wiki | 15:35 |
| === salem_ is now known as _salem | ||
| juliank | arges: I saw that you accepted apt 1.2.21 and 1.3.6 for xenial/yakkety, but not 1.4.1~17.04.1 for zesty. Any specific reason for that? | 16:05 |
| juliank | It's a bit odd to have the SRUs for older releases in -proposed without the one for the newer release | 16:06 |
| LocutusOfBorg | Noskcaj, FYI I'll sync dhelp as soon as it is picked up | 16:31 |
| LocutusOfBorg | after only 9 years seems that LP: #205308 is fixed | 16:31 |
| ubottu | Launchpad bug 205308 in dhelp (Ubuntu) "dhelp postinst error on gutsy->hardy upgrade" [High,Fix released] https://launchpad.net/bugs/205308 | 16:31 |
| LocutusOfBorg | cjwatson, ^^ :) | 16:31 |
| xnox | juliank, we have more stuff to do for apt =)))))) we shall talk ;-) in a second. | 16:31 |
| juliank | xnox: More stuff to do sounds scary | 16:46 |
| juliank | My veggies are in the oven now, so I do have some time :) | 16:48 |
| xnox | juliank, hehehehe =) i had steamed veggies, quicker ;-) | 16:48 |
| xnox | juliank, so. we spoke to sysadmins and we have a huge spike at 6am..6:30am UTC from all the pre-xenial machines. | 16:49 |
| juliank | Right. That's expected | 16:49 |
| xnox | juliank, switching xenial+ machines to that window will decrease the baseload, and will spike the window even more. Which will take us over our bandwidth limit in the key datacentres. | 16:50 |
| xnox | juliank, but we had a brainstorm session and this is what we established. | 16:50 |
| xnox | juliank, currently we have 1 timer which triggers/does 3 things as a monolithic block: apt update; apt upgrade --download-only; apt upgrade. | 16:51 |
| xnox | the first two things, ideally should happen universally spread across the 24h window. | 16:51 |
| xnox | the end users (server/cloud sysadmins) care a lot about the precise time of upgrades application. And they do not care when the bandwidth is used up. | 16:51 |
| juliank | Well, yes, that's not really new :) | 16:51 |
| juliank | What about memory usage during update, though? | 16:52 |
| xnox | thus my current proposal is to decouple unattended upgrades application; from the update & download-only | 16:52 |
| juliank | slangasek said his machine went really slow. | 16:52 |
| juliank | at the 1800 update due to memory spikes | 16:52 |
| slangasek | juliank: xnox speaks for me | 16:52 |
| xnox | juliank, i did not consider memory usage at all. And my expectations is that one should not run ubuntu if one cannot do apt update. | 16:53 |
| slangasek | juliank: fwiw I think it was ultimately the unattended-upgrades part (auto-installation of security updates) that broke me rather than the list updates | 16:53 |
| xnox | slangasek, plus i suspect slangasek has like a gazzilion of repositories enabled, unlike our core user base, which is actually hitting security pocket only. | 16:53 |
| slangasek | which I hadn't considered when I talked to you | 16:53 |
| juliank | OK | 16:53 |
| slangasek | juliank: also, fundamentally, 1) I've upgraded the RAM in my machine since then :P, and 2) I've been larted by our IS team ;-) | 16:54 |
| juliank | What is "larted"? | 16:54 |
| xnox | juliank, currently i'm writting up a back report with the decoupling plan, and will work on updating the units and/or configs; in apt and/or unattended upgrades. Such that the following behaviour happens for the long running instances: | 16:54 |
| xnox | run apt update smeared across 24h; if unattended-upgrades are enabled, execute --download-only straight after; and make unattanded-upgrades scheduled at the 6am..7am window. | 16:55 |
| xnox | on boot/resume: if we missed the cycle, execute on boot the whole lot, as enabled. | 16:55 |
| juliank | That's what I'd have seen as the optimal solution last week ... | 16:55 |
| xnox | juliank, this kind of means we are putting the sru on hold =/ | 16:56 |
| xnox | juliank, if you want the other fixes in, i can redo the sru's without the timer change. | 16:56 |
| xnox | juliank, unless you really want to DDoS canonical datacentres and see an angry elmo =) | 16:56 |
| juliank | xnox: Nah | 16:56 |
| juliank | xnox: But then we basically just have to revert the change in apt, and set APT::Periodic::Unattended-Upgrade to 0 in unattended-upgrades, and add a timer for that | 16:58 |
| xnox | juliank, yes. | 16:58 |
| juliank | Although maybe we want 0:00 as time + random delay of 24 hours instead of two start + 12 hour delay | 16:58 |
| xnox | juliank, and the unattended-upgrade timer will be in the unattended-upgrade package. | 16:58 |
| juliank | Yep, sure | 16:59 |
| xnox | juliank, i think two times a day is good. | 16:59 |
| xnox | juliank, because on ubuntu it has other effects apart from unattaned upgrades. | 16:59 |
| xnox | there is MOTD updates; and other messenging that is triggered by apt update. | 16:59 |
| juliank | You gotta realize that there is a timestamp file in the apt script itself, so you could run it every minute, it only does something if 24 hours have passed since the last time | 17:00 |
| juliank | :D | 17:00 |
| xnox | slangasek, i am also affected!!!!!!! i've now realised why i have random alerts texted to me on telegram whenever my database/webserver are down. And it was all weird to me, as to why they are restarting, when i didn't touch it. | 17:00 |
| xnox | juliank, can we get rid of that, at least in ubuntu, and rely on the systemd timer alone? | 17:01 |
| xnox | (persistent systemd timer alone) | 17:01 |
| xnox | juliank, doesn't like anacron keep persistent timestamps like that as well? | 17:01 |
| juliank | Well, the script has multiple distinct timers | 17:01 |
| xnox | /o\ | 17:01 |
| juliank | APT::Periodic::AutocleanInterval | 17:01 |
| juliank | APT::Periodic::CleanInterval | 17:01 |
| juliank | APT::Periodic::Update-Package-Lists | 17:02 |
| juliank | APT::Periodic::Download-Upgradeable-Packages | 17:02 |
| juliank | APT::Periodic::BackupArchiveInterval | 17:02 |
| juliank | all in the unit of days | 17:02 |
| xnox | juliank, so apt can be a cron replacement?! =) | 17:02 |
| juliank | Maybe it makes sense to split this into 5 systemd timers, but that's kind of weird | 17:03 |
| juliank | The two times a day should be enough to make it run often enough | 17:04 |
| xnox | juliank, not 5. Update-Package-Lists & Download-Upgradeable-Packages should be one timer triggered thing. And the rest could be the other. | 17:04 |
| xnox | because you do want to download things, straight after update, if enabled to do so. | 17:05 |
| juliank | Well, you say that. | 17:05 |
| xnox | (e.g. if unattended upgrades are enabled, or download-upgradable-packages is enabled) | 17:05 |
| xnox | i totally want to support download Update-Package-Lists only. | 17:05 |
| juliank | xnox: Now, I'd really like to have the same logic in Debian, I don't really want to have distinct Ubuntu branches again :) | 17:08 |
| juliank | So, I'd like keep the current setting of 6 to 7, and install a drop-in unit from the Ubuntu vendor directory. | 17:09 |
| xnox | juliank, are unattended upgrades enabled by default in debian? | 17:09 |
| juliank | Does that make sense? | 17:09 |
| juliank | xnox: I hope not. | 17:09 |
| xnox | juliank, drop-in: yes, that is fine, the ubuntu drop-in can go into /lib/systemd/system/apt-update-whatever-the-name-is.timer.d/ubuntu.conf | 17:09 |
| juliank | But we actually should have at least fixed the race conditions now, so that's something :) | 17:09 |
| xnox | [Timer] | 17:09 |
| xnox | OnCalendar= | 17:09 |
| xnox | OnCalendar=<ubuntu setting> | 17:10 |
| juliank | Yep | 17:10 |
| juliank | + Adjustments for RandomizedDelay | 17:11 |
| juliank | xnox: I also see u-u already has a service | 17:14 |
| juliank | So adding a timer should be easy | 17:14 |
| juliank | Ah, but that's just for shutdown | 17:15 |
| xnox | juliank, i think it would be more elaborate yes. | 17:17 |
| xnox | juliank, i was thinking to override apt settings when calling apt timers, and in ubuntu at least have something with -oAPT::DO-THIS-ONLY | 17:17 |
| xnox | and -oAPT::DO_THAT_ONLY | 17:17 |
| xnox | or some such, or more verbose scripts that determine what happens when. | 17:18 |
| xnox | i'm currently writting the plan up, and will be working on a proposal of patches to apt and/or unattended-upgrades packages | 17:18 |
| juliank | xnox: The script does not take -o options, that's very ... involved. | 17:20 |
| juliank | You could use config files, but I think it's pointless. | 17:21 |
| juliank | What you want as a default according to your description is APT::Periodic::Update-Package-Lists=1, APT::Periodic::Download-Upgradeable-Packages=1 and APT::Periodic::Unattended-Upgrade=0 | 17:22 |
| slangasek | juliank: larted> from LART, luser attitude readjustment tool ;) | 17:22 |
| juliank | and then just run unattended-upgrades from the unattended-upgrades-daily service or whatever | 17:22 |
| juliank | Although, clean and autoclean would be weird in the apt job | 17:23 |
| juliank | Can we just split up the apt job in two? | 17:23 |
| xnox | juliank, clean and autoclean should be in the unattended-ugprade now? | 17:23 |
| xnox | juliank, we can do that too. | 17:23 |
| juliank | clean and autoclean run after unattended-upgrades | 17:23 |
| juliank | if configured | 17:24 |
| juliank | So if you'd move only unattended-upgrades out, you'd end up with update, download, clean if configured, and your files would be gone again | 17:24 |
| juliank | This way we can have apt-daily and apt-daily-dangerous | 17:25 |
| juliank | :D | 17:25 |
| juliank | (OK, maybe a better name for the second one) | 17:25 |
| juliank | I guess we can specify After betweens them, so *if* both are started at the same time, the upgrade&clean job runs later | 17:28 |
| xnox | juliank, yeah, we will need to do the after/before orderings right, for the onboot case. | 17:29 |
| xnox | juliank, so could you please checkout and comment on: | 17:29 |
| xnox | https://bugs.launchpad.net/ubuntu/artful/+source/unattended-upgrades/+bug/1686470 | 17:29 |
| ubottu | Launchpad bug 1686470 in unattended-upgrades (Ubuntu Artful) "Apt updates that are uniformally spread across all timezones, with predictable application windows" [High,Triaged] | 17:29 |
| juliank | That sounds right | 17:30 |
| juliank | If we just split up the apt job in two, we can just keep the changes localized to apt though, and not change unattended-upgrades at all :D | 17:31 |
| xnox | yeah \o/ no need to do versioned breaks and depends | 17:32 |
| xnox | juliank, u-u service is for shutdown, i believe. not for on-boot stuff. | 17:33 |
| juliank | xnox: yeah | 17:35 |
| juliank | xnox: Another advantage of that scheme is that I can probably sell that the Debian release team :D | 17:36 |
| juliank | "Oh, we discovered that this might cause unwanted spikes in memory bandwidth if a lot of machines update at the same time, let's just do this all day, and special case upgrades" | 17:37 |
| xnox | juliank, yes. | 17:39 |
| xnox | juliank, "memory bandwidth" you mean "network bandwidth" right? | 17:40 |
| juliank | yes | 17:40 |
| juliank | odd mistake | 17:40 |
| xnox | juliank, and not can, but does take out a DC center we can see spikes to 10Gbps, vs the baseload of 4Gbps and some of our pipes are limited to 10Gbps | 17:41 |
| xnox | at which point we have protections to limit the bandwidth, such that #is can ssh into those boxes at least; and end users get timeouts doing apt update. | 17:42 |
| juliank | xnox: The diff is stupid, but https://github.com/Debian/apt/compare/master...julian-klode:lp1686470?expand=1 should be it on the script side | 17:44 |
| juliank | + install rules and a new timer and service | 17:44 |
| xnox | destructive-daily | 17:45 |
| * xnox giggles | 17:45 | |
| juliank | xnox: Well, find a better name! | 17:46 |
| xnox | apt-download-updates, apt-install-upgrades | 17:48 |
| xnox | ... by definition installing things, breaks toys. | 17:48 |
| xnox | juliank, however we should probably not change the original timer name though =/ because $reasons | 17:48 |
| juliank | xnox: Yep. Maybe I should also keep the script together and just put the parts in an if/else, that's a bit easier to look at and build. | 17:52 |
| juliank | Then we can just pass upgrade as $1 to run upgrade, update to update, and if nothing passed it does all, or something like that | 17:53 |
| juliank | Has the benefit of being completely easy to review | 17:53 |
| juliank | just do a diff -w | 17:53 |
| juliank | :D | 17:53 |
| juliank | https://paste.ubuntu.com/24461931/ | 18:00 |
| juliank | xnox: With the additional timer and service: https://paste.ubuntu.com/24461953/ | 18:06 |
| juliank | Oh, also needs an adjustment to rules | 18:06 |
| juliank | that's a -w git, the unreadable version is at https://github.com/Debian/apt/compare/master...julian-klode:lp1686470?diff=split&expand=1&name=lp1686470 | 18:07 |
| juliank | Exciting times! | 18:09 |
| juliank | I guess I'll just put this in a PPA quickly for zesty and artful to play with | 18:10 |
| juliank | But first, my own system | 18:11 |
| juliank | xnox: Stuff seems ready to go (built successfully, and I installed it :D), I could upload the branch to artful to give it a test drive. | 18:44 |
| xnox | juliank, artful would be nice; many of us run artful things. | 18:45 |
| juliank | xnox: One question, though: Does it make sense to have the u-u/clean stuff not depend on network-online, as I did, or should both depend on that? | 18:49 |
| juliank | I wonder if u-u works without network | 18:49 |
| xnox | juliank, u-u/clean should not need network, correct. | 18:50 |
| xnox | and i do not care about msft-fonts-package that does wget in postinst =) | 18:50 |
| juliank | It's theoretical anyway. | 18:50 |
| juliank | The network-online target is a boot thing | 18:51 |
| juliank | and apt-daily-upgrade has after=apt-daily which has after=network-online | 18:51 |
| juliank | (and wants=network-online) | 18:51 |
| xnox | juliank, possibly both, however the chain should be enough | 18:54 |
| juliank | yeah | 18:54 |
| juliank | xnox: Currently writing an email to deity@l.d.o and debian-release, do you want to be CCed on that? | 19:02 |
| juliank | (and anyone else) | 19:03 |
| juliank | Mostly collecting additional feedback | 19:03 |
| xnox | juliank, sure | 19:08 |
| juliank | xnox: which email address? | 19:09 |
| xnox | juliank, xnox@ anything - ubuntu.com, debian.org, spi-inc.org, surgut.co.uk...... | 19:11 |
| xnox | as you wish | 19:11 |
| * xnox wonders surely everyones email addresses are predictable =) | 19:11 | |
| juliank | xnox: Some people have preferences :) | 19:13 |
| juliank | Some people even commit with different emails when working vs non-working | 19:13 |
| xnox | some people have too much time on their hands =) | 19:13 |
| juliank | xnox: I do this for changelog entries, signing the ubuntu ones with juliank@ubuntu.com and the Debian ones with jak@debian.org | 19:14 |
| juliank | The commits all use my @debian.org, though | 19:15 |
| juliank | git should really allow me to set up per-branch emails | 19:15 |
| juliank | Author: jak@debian.org Committer: juliank@ubuntu.com | 19:15 |
| * juliank wishes jak@ubuntu.com were an alias for juliank@ubuntu.com | 19:16 | |
| juliank | That account is not even used: https://launchpad.net/~jak | 19:17 |
| juliank | (and how did John Knottenbelt became "jak", where does the a come from ...) | 19:17 |
| Pici | middle name? | 19:17 |
| juliank | Probably from the email address? | 19:18 |
| juliank | anyway, that was just as unfortunate as jak being blocked on freenode and oftc | 19:18 |
| sarnold | juliank? | 19:18 |
| juliank | sarnold: yes? | 19:19 |
| sarnold | juliank: what do you mean 'blocked on oftc'? | 19:19 |
| juliank | well, registered by another user | 19:19 |
| juliank | Imagine if I had a single nick everywhere | 19:20 |
| juliank | sarnold: I almost bought jak.fun recently | 19:20 |
| * juliank would have been jak@jak.fun | 19:20 | |
| sarnold | juliank: 'jak' is unused forever; if you want it, I'll drop it for you | 19:21 |
| juliank | I think it's a bit late to change nicks now :) | 19:21 |
| sarnold | juliank: 'jak' is unused on freenode for 36 weeks too, they may be willing to drop it here, too | 19:21 |
| juliank | sarnold: Maybe on OFTC, but I'm not sure if that does not confuse people who are both in Debian & Ubuntu :) | 19:24 |
| juliank | That said, highlighting me is not a problem - I react to both :D | 19:24 |
| juliank | OK, I think I can go relax now, and hope I did not break any artfuls | 19:47 |
| nacc | smoser: for orig tarball imports -- what should we do if debian (or ubuntu) switches, for the same upstream, from .gz to .bz2? we correctly don't see the bz2 in `pristine-tar list` but we can't import it, because the same upstream version has already been imported. this happened for src:acl between 2.2.51-1 and 2.2.51-2 | 20:48 |
| nacc | infinity: slangasek: --^ if you have an opinion/advice on ideal behavior. Basically the importer we have no will use the 'first' orig tarball for a given upstream version as uploaded. But we don't verify if we see a second (different named) tarball for the same upstream that the contents match or anytyhing (currently) | 20:53 |
| jbicha | Ubuntu won't allow you to use two different tarballs for the same upstream version | 20:54 |
| nacc | jbicha: yes, but debian did | 20:55 |
| nacc | well "different" in that they have different names | 20:55 |
| infinity | jbicha: We sure will. | 20:55 |
| nacc | their contents might very well be the same | 20:56 |
| nacc | oh and actually, i found a case where what you said wasn't true, jbicha :) | 20:56 |
| nacc | i'd have to go look it up, but it does confuse LP a bit :) | 20:56 |
| infinity | jbicha: As long as the filename changes, you can change taballs all you want. People often take advantage of that "feature" to use a cleaner/better upstream tarball by switching file compression. | 20:56 |
| infinity | nacc: So, I'm not sure how the importer should deal with this, but when people switch compression, they almost always do it intentionally to force a new tarball that *does* have different contents, so you certainly can't assume that it's still the same. | 20:58 |
| infinity | (I'm not saying this practice is sane or rational, but it's certainly not uncommon) | 20:58 |
| nacc | infinity: yeah, that's what i was thinking -- i'm not sure gbp-import-orig can handle this, which means i'll need to rethink that part | 21:01 |
| infinity | nacc: The fundamental problem here is that, while VCS workflows (and dpkg/Debian versioning Policy) make a big fuss about upstream version, and tracking an upstream series/branch, the archives (dak and soyuz) literally don't care. upstream version is meaningless to archive software, all that matters is file consistency. So, you can't upload foo_1.2.3.orig.tar.gz twice with different contents, but change the filename and all bets are off. | 21:03 |
| infinity | nacc: How you represent that eventuality back in the VCS world where "upstream version" actually means something, I'm not sure. You might just need a different branch. 1.2.3^xz or something. | 21:04 |
| nacc | infinity: yep, understood -- i just want, if possible, to be able to reproduce a given build's files from our git tree. | 21:04 |
| nacc | infinity: yeah, i'm thinking we'd need to do something like that | 21:04 |
| infinity | (Bonus points for finding a char that's legal in git refs but illegal in dpkg versions) | 21:04 |
| nacc | infinity: i think we can also do something with the version tag | 21:04 |
| nacc | *upstream tag, rather | 21:04 |
| nacc | i'll need to play with it a bit to make everything happy | 21:04 |
| nacc | infinity: thanks for the feedback! | 21:04 |
| hloeung | juliank, xnox, slangasek: thanks for all your work on LP#1686470 | 22:55 |
| juliank | hloeung: Let's see tomorrow if that actually works or if I messed up :) | 23:01 |
| hloeung | heh | 23:01 |
| juliank | We should build test cases for the daily script, actually | 23:05 |
| juliank | Well, unit tests | 23:05 |
| juliank | Just gotta mock various tools like apt-get and u-u | 23:06 |
| Unit193 | I mock a lot of tools | 23:07 |
| sarnold | :) | 23:07 |
| Unit193 | sarnold: I presume there's no update that you know of to 716535? | 23:11 |
| sarnold | Unit193: correct | 23:12 |
| sarnold | Unit193: or, rather, none that I've heard | 23:12 |
| sarnold | oh I see that's explicit in your question. sigh. :) | 23:12 |
| Unit193 | Yep, you seem to "know everything" for the sec team, and easy to talk to. | 23:13 |
| * sarnold blushes | 23:13 | |
| nacc | mdeslaur: just an fyi, i'll be sru'ing php7.0 7.0.18 shortly (hopefully today, or tmrw) | 23:14 |
| nacc | infinity: ah, it's actually quite a bit easier (for our case), we just need to make the tag names unique, so we can append the extension | 23:18 |
| nacc | infinity: that's the only part that fails for `gbp import-orig` | 23:18 |
| nacc | infinity: in any case, thank you again for the insight! | 23:23 |
| mdeslaur | nacc: ack, thanks | 23:38 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!