[15:01]  * slangasek waves
[15:01] <juliank> o/
[15:01] <fginther> o/
[15:01] <aleks_bogdanov> o/
[15:02] <sil2100> o/
[15:02] <slangasek> #startmeeting
[15:02] <meetingology> Meeting started Thu Sep 13 15:02:06 2018 UTC.  The chair is slangasek. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
[15:02] <meetingology> Available commands: action commands idea info link nick
[15:02] <slangasek> [TOPIC] Lightning round
=== meetingology changed the topic of #ubuntu-meeting to: Lightning round
[15:02] <slangasek> $ echo $(shuf -e slangasek bdmurray xnox tdaitx doko sil2100 rbalint infinity cyphermox mwhudson rcj philroche Odd_Bloke fginther juliank cshep aleks_bogdanov tobikoch)
[15:02] <slangasek> fginther aleks_bogdanov tdaitx cshep bdmurray xnox philroche infinity cyphermox doko mwhudson sil2100 Odd_Bloke slangasek juliank rbalint rcj tobikoch
[15:02] <slangasek> fginther: hello!
[15:02] <fginther> * Out for three days on holiday
[15:02] <fginther> * Created a new test image for a partner
[15:02] <fginther> * Cloud image build system vanguarding
[15:02] <fginther> * done
[15:03] <rbalint> o/
[15:03] <aleks_bogdanov> * working on simple streams exporters
[15:03] <aleks_bogdanov> * getting to learn all the systems and processes
[15:03] <aleks_bogdanov> * done
[15:03] <tdaitx> Short week: last Friday, working 6h/day this week
[15:03] <tdaitx> * fixed openjdk atk bridge error (LP: #1788250) a while ago
[15:03] <tdaitx>   - provided package to the security team, released on Bionic yesterday
[15:03] <tdaitx> * updated openjdk-lts sru exception, created ffe bug report (LP: #1791716)
[15:03] <tdaitx> * provided debdiff for new openjdk-11/openjdk-lts to doko
[15:03] <ubottu> Launchpad bug 1788250 in openjdk-8 (Ubuntu) "AWT applications fail with java.awt.AWTError: Assistive Technology not found: org.GNOME.Accessibility.AtkWrapper" [Undecided,Confirmed] https://launchpad.net/bugs/1788250
[15:03] <ubottu> Launchpad bug 1791716 in java-common (Ubuntu) "FFe: sync java packages from Debian to fix OpenJDK 10/11 issues in Cosmic" [Undecided,Incomplete] https://launchpad.net/bugs/1791716
[15:03] <tdaitx> * fixed orphanedold-dkms files, tested on Xenial, Bionic, and Cosmic, SRU for Xenial/Bionic and patch for Cosmic (LP: #1791959)
[15:04] <ubottu> Launchpad bug 1791959 in initramfs-tools (Ubuntu) "[SRU] remove orphaned initrd old-dkms files in /boot" [Medium,Confirmed] https://launchpad.net/bugs/1791959
[15:04] <tdaitx> * so far unable to cause sync to hang in order to reproduce LP: #1667512
[15:04] <ubottu> Launchpad bug 1667512 in initramfs-tools (Ubuntu) "update-initramfs hangs on upgrade, dpkg unusable, unbootable system" [Medium,Fix released] https://launchpad.net/bugs/1667512
[15:04] <tdaitx> (done)
[15:04] <slangasek> cshep:
[15:05] <cshep> working on updating image classes
[15:05] <cshep> that's pretty much it. done
[15:05] <bdmurray> committed a fix for an OOPS encounted when submitting crashes to daisy
[15:05] <bdmurray>   https://oops.canonical.com/oops/?oopsid=OOPS-fed140a72ccf553f7f43898fbc0a8f4c
[15:05] <bdmurray> submitted RT re daisy and apport code updates for the production Error Tracker
[15:05] <bdmurray> modified upgrade-charm hooks in Error Tracker charms to use proper directories
[15:05] <bdmurray> fixed cheetah issue w/ logrotation jobs for daisy and errors app servers
[15:05] <bdmurray> sorted out issues with staging version of the Error Tracker
[15:05] <bdmurray> submitted RT regarding staging errors frontends access to prod cassandra
[15:05] <bdmurray> updated Error Tracker main page filtering to show only deb crashes and add a
[15:05] <bdmurray> figured out how to and imported a snap crash into the staging version of the Error Tracker
[15:05] <bdmurray>  selection for 'all other binary packages'
[15:05] <bdmurray> updated Bionic release notes to include info re dropping of powerpc arch
[15:05] <bdmurray> investigation into apport SRU regressions
[15:06] <bdmurray> overrode increased rate of crashes for open-vm-tools after discussion w/ cpaelzar
[15:06] <bdmurray> SRU verification of libc6 upgrade change (LP: #1787649)
[15:06] <ubottu> Launchpad bug 1787649 in ubuntu-release-upgrader (Ubuntu Bionic) "ubuntu-release-upgrader crashed with SystemError: E:Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages." [High,Fix committed] https://launchpad.net/bugs/1787649
[15:06] <bdmurray> review of didier's apport MP which fixes LP: #1791324
[15:06] <ubottu> Launchpad bug 1791324 in apport (Ubuntu Bionic) "/usr/share/apport/apport-gtk:KeyError:/usr/share/apport/apport-gtk@598:run_argv:run_crashes:run_crash" [Medium,New] https://launchpad.net/bugs/1791324
[15:06] <bdmurray> irc discussion with tdaitx re LP: #1791959 and old-dkms files
[15:06] <ubottu> Launchpad bug 1791959 in initramfs-tools (Ubuntu) "[SRU] remove orphaned initrd old-dkms files in /boot" [Medium,Confirmed] https://launchpad.net/bugs/1791959
[15:06] <bdmurray> ordered a dragonboard and parts
[15:06] <bdmurray> ✔  done
[15:06] <bdmurray> xnox:
[15:06] <xnox> * smc-tools new upstream release LP: #1689782
[15:06] <xnox> * PATH is broken, fixing everywhere:
[15:06] <xnox>   LP: #1792004 LP: #1614080
[15:06] <xnox>   https://github.com/systemd/systemd/pull/10073
[15:06] <xnox> * openssh-server socket activation patches
[15:06] <ubottu> Launchpad bug 1689782 in smc-tools (Ubuntu) "[18.10 FEAT] New package: smc-tools >=1.1.0" [Wishlist,Fix released] https://launchpad.net/bugs/1689782
[15:06] <xnox>   - done, reviewed with debian maintainers & foundations
[15:06] <ubottu> Launchpad bug 1792004 in systemd (Ubuntu) "built-in PATH seems to have sbin and bin out of order; and inconsistent" [Undecided,New] https://launchpad.net/bugs/1792004
[15:06] <xnox>   - will clean-up more and post to systemd-devel review
[15:06] <ubottu> Launchpad bug 1614080 in bash (Ubuntu) "PATH contains dot when PATH is unset before running bash" [Medium,Fix released] https://launchpad.net/bugs/1614080
[15:06] <xnox>   - also will post for review to openssh-portable
[15:06] <xnox> * debugging finarld incomplete shutdown initrd generations
[15:07] <xnox> done
[15:07] <philroche> * Cloud image modification for partner clouds
[15:07] <philroche> * Helping onboard new team members
[15:07] <philroche> * Preparing for team sprint next week
[15:07] <philroche> (done)
[15:07] <philroche> cyphermox:
[15:08] <cyphermox> - MIR: gpsd
[15:08] <cyphermox> - SRU verifications for gnu-efi, shim 13 in trusty, shim 15 in bionic
[15:08] <cyphermox> - fixing netplan autopkgtests prior to upload
[15:09] <cyphermox> - optional-addresses upstream systemd work
[15:09] <cyphermox> - merged mwhudson's optional-addresses code for netplan
[15:09] <cyphermox> - plymouth fix for tty deactivation (LP: #1767918) and SRU to bionic
[15:09] <cyphermox> - fwupdate: drop uefi artefacts
[15:09] <cyphermox> - investigating missing MokSBStateRT in efivars
[15:09] <ubottu> Launchpad bug 1767918 in plymouth (Ubuntu Bionic) "Login password from GDM is shown in plain text on the VT1 console" [High,Triaged] https://launchpad.net/bugs/1767918
[15:09] <cyphermox> - SRU syslinux with patch for gnu-efi 3.0.8 support in bionic
[15:09] <cyphermox> - proposed-migration: looked into python-pkginfo
[15:09] <cyphermox> (done)
[15:09] <doko> - at the GNU Cauldron in Manchester from Thu to Mon
[15:09] <doko> - working on toolchain SRUs with sil2001
[15:09] <doko> - review all open MIRs
[15:09] <doko> - archive work, toolchain updates
[15:09] <doko> (done)
[15:09] <doko> ohh, and started a cosmic test rebuild
[15:10] <doko> http://people.canonical.com/~doko/ftbfs-report/test-rebuild-20180911-cosmic.html
[15:10] <juliank> +1
[15:11] <slangasek> xnox: do you suppose we could get a central writeup afterwards of how you expect PATH to work, so we a) know what would be a regression, and b) can make sure there are no holes in it?
[15:11] <bdmurray> sil2100:
[15:11] <sil2100> - Candidate reviews
[15:11] <sil2100> - Kernel SRU reviews
[15:11] <sil2100> - Regular SRU reviews and releases
[15:11] <sil2100> - FFe reviews
[15:11] <sil2100> - raspi3 u-i classic:
[15:11] <sil2100>   * Adding some tweaks to ubuntu-image for easier kernel/initrd preparation
[15:11] <sil2100>   * Figuring out the correct uboot.env for raspi3
[15:11] <Odd_Bloke> slangasek: You want to understand the PATH forward?
[15:11] <sil2100>   * Add some more changes to the gadget tree and livecd-rootfs to get proper boot contents
[15:11] <sil2100>   * Multiple test image builds - final local image finally booting into the system on the rootfs
[15:11] <sil2100> - Tweaks to the work-in-progress britney branch for SRU ADT regression handling, test suite additions
[15:11] <xnox> Odd_Bloke, ahahhahahhaha
[15:11] <sil2100> - Travel preparations
[15:11] <xnox> slangasek, possibly.
[15:11] <sil2100> (done)
[15:11] <Odd_Bloke> * Visiting a partner last week, so playing catch-up this week
[15:11] <Odd_Bloke> * Double-checked that microcode is correctly installed in bare metal images we provide to partners
[15:11] <Odd_Bloke> * Met with our new folks for the first time!
[15:11] <Odd_Bloke> * Continued work on streamlining the delivery of our images to GKE
[15:11] <xnox> slangasek, but i would argue we should drop /usr/local and then people will stop listening
[15:11] <Odd_Bloke> (done)
[15:12] <Odd_Bloke> slangasek:
[15:12] <xnox> slangasek, because e.g. unattended-upgrades.service should not have /usr/local in PATH
[15:13] <doko> tdaitx: fyi: the above test rebuild is done using openjdk-11 as the default
[15:13] <tdaitx> doko: ack, thanks for that!
[15:14] <rbalint> xnox, why shouldn't u-u have /usr/local/bin ?
[15:14] <tdaitx> doko: just for curiosity sake, does the rebuild respect the build dependebcy order at all?
[15:14] <slangasek>  * SRU processing, there are some good shims and grubs running around
[15:14] <slangasek>  * Support for nvidia modules in GKE
[15:14] <slangasek>  * Preparing for keyring cabal activities during the sprint
[15:14] <slangasek>  * proposed-migration wrangling
[15:14] <slangasek>  * upcoming:
[15:14] <slangasek>   * swapping tomorrow for sprint travel
[15:14] <slangasek>   * travelling to sprint
[15:14] <slangasek> (done)
[15:14] <juliank> * submitted systemd-networkd state documentation PR (https://github.com/systemd/systemd/pull/10033)
[15:14] <juliank> * debugged packagekit -crashing- restarting when upgrading itself (bug 1790613) - hello, maintainer scripts!
[15:15] <ubottu> bug 1790613 in packagekit (Ubuntu Bionic) "Regression: packagekit crashes updating itself to 1.1.9-1ubuntu2.18.04.1" [High,Triaged] https://launchpad.net/bugs/1790613
[15:15] <juliank> * fixed ipxe FTBFS
[15:15] <juliank> * fixed ipxe booting grub via shim in qemu (bug 1789319)
[15:15] <ubottu> bug 1789319 in MAAS "Unable to load shimx64.efi using iPXE over UEFI" [Undecided,Triaged] https://launchpad.net/bugs/1789319
[15:15] <juliank> * some apt upstream discussions
[15:15] <juliank> * reviewed an update-manager merge proposal from rbalint (https://code.launchpad.net/~rbalint/update-manager/retry-after-failed-transaction/+merge/354648)
[15:15] <juliank> * submitted PR for networkd-dispatcher /etc overrides (https://gitlab.com/craftyguy/networkd-dispatcher/merge_requests/40)
[15:15] <juliank> * merged newt for automake 1.16 (after a friendly request from LocutusOfBorg, who merged automake); reported some bugs in debian
[15:15] <juliank> (done)
[15:15] <rbalint> (short week)
[15:15] <slangasek> rbalint:
[15:15] <rbalint> * triaging unattended-upgrades regressions
[15:15] <rbalint> * gardening update-manager bugs
[15:15] <rbalint> * fixing LP: #1317164 and LP: #1791931 which have > 100 duplicates and lot of reported crashes
[15:15] <ubottu> Launchpad bug 1317164 in update-manager (Ubuntu) "Show reason of TransactionFailed and let the user try again instead of crashing" [High,In progress] https://launchpad.net/bugs/1317164
[15:15] <ubottu> Launchpad bug 1791931 in update-manager (Ubuntu) "Update-manager crashes in _show_transaction due to packages being already removed" [High,In progress] https://launchpad.net/bugs/1791931
[15:15] <rbalint> * renaming WSL apps to proudly include LTS in the name
[15:15] <rbalint> (done)
[15:15] <doko> tdaitx: no. why would it be necessary?
[15:16] <slangasek> xnox: I'm asking you for documentation, not a polemic
[15:16] <slangasek> rcj:
[15:17] <tdaitx> doko: hmm, I don't see maven, ant, or gradle there, is that expected?
[15:17] <rcj> * Debugging and reported a missing dependency in linux-firmware (bug #1792189)
[15:17] <ubottu> bug 1792189 in linux-firmware (Ubuntu) "linux-firmware does not depend on initramfs-tools" [Undecided,New] https://launchpad.net/bugs/1792189
[15:17] <doko> tdaitx: still running ... blame the libreoffice/chromium/firefox test builds for the delay ...
[15:17] <rcj> * cloud image development and fixes
[15:17] <rcj> (done)
[15:17] <tobikoch> * Learning about image building.
[15:17] <tobikoch> * Debugging small problems related to that.
[15:17] <tobikoch> (happy end)
[15:18] <slangasek> tdaitx: it rebuilds using the main archive for build-dependencies, so in principle everything there should be buildable against itself now without bootstrapping
[15:18] <slangasek> and anything that isn't is a failure
[15:18] <xnox> rbalint, because /usr/local/bin/python3 breaks distro upgrades.
[15:18] <slangasek> xnox: just create a systemd unit that watches for /usr/local/bin/python3 with inotify and deletes it
[15:18] <tdaitx> slangasek: ack, thanks
[15:18] <xnox> slangasek, excellent idea!
[15:19] <rbalint> xnox, u-u does not really run during distro upgrades
[15:19] <xnox> rbalint, /any/ package upgrade can fail... we have bugs...
[15:19] <slangasek> any other questions on status?
[15:22] <slangasek> [TOPIC] Bugs
=== meetingology changed the topic of #ubuntu-meeting to: Bugs
[15:23] <slangasek> bdmurray: hello
[15:23] <bdmurray> bug 1754777
[15:23] <ubottu> bug 1754777 in casper (Ubuntu) "bionic casper nfsboot not reaching desktop env, failure to mount various kernel filesystems and /tmp" [High,Confirmed] https://launchpad.net/bugs/1754777
[15:24] <slangasek> bdmurray: I don't think casper nfs support is a priority for us
[15:25] <cyphermox> people still use nfs?
[15:25] <cyphermox> :D
[15:26] <slangasek> doko: btw is that rebuild test going to show up at http://qa.ubuntuwire.org/ftbfs/rebuilds/ soon?
[15:26] <bdmurray> Could somebody comment on the bug since it has a lot of people affected?
[15:26] <slangasek> who are these lots of people that are affected?
[15:27] <slangasek> if you're doing casper+nfs, you're doing something custom
[15:27] <cyphermox> oh, actually
[15:27] <cyphermox> could it be some kind of edubuntu thing maybe?
[15:27] <slangasek> there's no edubuntu in 18.04
[15:27] <cyphermox> no, but people do upgrade
[15:27] <cyphermox> I wonder if it's casper used outside of for install purposes.
[15:28] <doko> slangasek: no, using my modified ftbfs-report. and you wanted to review my changes and propose a package where to file these changes ...
[15:28] <cyphermox> (not that it particularly changes whether we support nfs+casper really)
[15:28] <doko> and only sending the annonucement once main has finished to build
[15:28] <gaughen> I have a bug I wanted to ask about before we move on, slangasek
[15:30] <slangasek> doko: afaik you still need to push changes to lp:~wgrant/lp-ftbfs-report/production
[15:30] <rcj> suspenseful
[15:30] <gaughen> https://bugs.launchpad.net/vaultlocker/+bug/1780332
[15:31] <ubottu> Launchpad bug 1780332 in cryptsetup (Ubuntu) "vaultlocker does not ensure that udev is triggered to create /dev/disk/by-uuid/<uuid-in-luks-header> symlink and fails" [Undecided,New]
[15:31] <slangasek> doko: so you need wgrant to review rather than me
[15:32] <doko> slangasek: we should add these scripts to some distro repo
[15:33] <slangasek> doko: it should have its own repo, but you still need to negotiate that with wgrant to change which branch he's deploying from
[15:33] <doko> well, next week. he got the same emails that you got
[15:36] <doko> sil2100: sru ping?
[15:36] <xnox> i'll comment on the bug
[15:36] <slangasek> xnox: ta :)
[15:36] <slangasek> bdmurray: other bugs?
[15:37] <bdmurray> How about a crash? https://errors.ubuntu.com/problem/ef53414acdfe092569388629a5c446880d0bcba1
[15:37] <slangasek> are crashes bugs?
[15:37] <rbalint> slangasek, they can become ones :-)
[15:38] <slangasek> systemd-journald crashing
[15:38] <slangasek> in fsync()
[15:39] <slangasek> am I blind that I can't find here what signal it's crashing with?
[15:39] <bdmurray> 6
[15:39] <sil2100> doko: I just need the SRU verification steps mentioned on the tracking bugs and then it should be good to go
[15:39] <cyphermox> slangasek: top of the screen, in the breadcrumbs thing
[15:40] <cyphermox> ah, bdmurray already answered
[15:40] <slangasek> oh, that's the 6? ok
[15:40] <slangasek> so it's a SIGABRT
[15:40] <juliank> similar to https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1773148?
[15:40] <ubottu> Launchpad bug 1773148 in systemd (Ubuntu Cosmic) "/lib/systemd/systemd-journald:6:fsync:fsync_directory_of_file:journal_file_rotate:do_rotate:server_rotate" [Undecided,New]
[15:41] <juliank> both crash in fsync.c:27 with SIGABORT
[15:41] <slangasek> rbalint, xnox: ^^ so should we be concerned about this, or does systemd-journald recover gracefully and this is a non-event?
[15:41] <slangasek> since SIGABRT likely means journald itself thought there was a problem
[15:41] <juliank> Probably systemd is aborting the syscall in fsync because the fsync hangs or something?
[15:42] <juliank> same bug: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1750855 it seems
[15:42] <ubottu> Error: launchpad bug 1750855 not found
[15:43] <bdmurray> juliank: yeah, that was my thought
[15:44] <slangasek> right, so I'm not sure that's user-affecting
[15:44] <slangasek> or fixable in systemd
[15:45] <juliank> Does systemd send SIGABRT to unresponsive services or something?
[15:45] <slangasek> it will, yes
[15:45] <slangasek> there's a keepalive on journald and I think logind
[15:45] <slangasek> so a hanging fsync() could just be tripping the watchdog
[15:46] <xnox> slangasek, juliank - systemd sends SIGABRT when internal timer times-out.
[15:46] <slangasek> right
[15:46] <xnox> because Watchdog= thing is set
[15:46] <xnox> actually
[15:46] <slangasek> we did change watchdog on some of the processes, didn't we?
[15:46] <juliank> ok
[15:46] <xnox> i thought we agreed to drop watchdog stanza on journald
[15:46] <xnox> slangasek, yeah, that's what i thought.
[15:46] <slangasek> well, I'm sure I opined that we should drop it
[15:46] <xnox> maybe we only talked about it in portland
[15:46] <slangasek> I don't know if it was agreed :)
[15:47] <slangasek> xnox: so should we take this into the backlog?
[15:47] <xnox> yeah
[15:47] <slangasek> ok, doing
[15:47] <bdmurray> that's it for me then
[15:48] <xnox> i wonder if we can have a graceful watchdog, which instead of killing things, would log that "i would have killed it"
[15:48] <slangasek> I'll take via bug #1750855
[15:48] <ubottu> Error: Launchpad bug 1750855 could not be found
[15:48] <slangasek> [TOPIC] AOB
=== meetingology changed the topic of #ubuntu-meeting to: AOB
[15:48] <slangasek> anything else?
[15:48] <bdmurray> then should we mark the ET created bug as a dupe of the private crash?
[15:48] <slangasek> probably
[15:49] <bdmurray> doing so
[15:49] <slangasek> going once (AOB)
[15:49] <gaughen> sold!
[15:50] <slangasek> ok
[15:50] <slangasek> #endmeeting
=== meetingology changed the topic of #ubuntu-meeting to: Ubuntu Meeting Grounds: Please leave swords by the door | Calendar/Scheduled meetings: http://fridge.ubuntu.com/calendars | Logs: https://wiki.ubuntu.com/MeetingLogs | Meetingology documentation: https://wiki.ubuntu.com/meetingology | <wxl> be nice
[15:50] <meetingology> Meeting ended Thu Sep 13 15:50:05 2018 UTC.
[15:50] <meetingology> Minutes:        http://ubottu.com/meetingology/logs/ubuntu-meeting/2018/ubuntu-meeting.2018-09-13-15.02.moin.txt
[15:50] <slangasek> thanks, all :)