=== _salem is now known as salem_
=== gavinlin is now known as Guest34349
=== salem_ is now known as _salem
[07:19] <pitti> bdrung_work: I suppose you could use python3-apport or a tool of your own to construct a .crash file with the data you have
[07:20] <pitti> bdrung_work: it doesn't require ProcMaps and other bits, but it does require the exact ExecutablePath
[09:12] <bdrung_work> pitti, thanks. i will have a look at python3-apport. how does apport get the ExecutablePath?
[09:13] <pitti> bdrung_work: readlink /proc/<pid>/exe
[09:13] <pitti> bdrung_work: there's some extra magic that it does for interpreters though
[09:14] <pitti> bdrung_work: i. e. if ExePath is /bin/sh, /usr/bin/python and so on, it dissects teh command line instead to find the "real" program (i. e. the script)
[09:14] <pitti> if that's not an use case for you, you can ignore that
[09:14] <pitti> you won't need it for retracing indeed, this is just for filing correctly targetted bug reports
[09:14] <bdrung_work> interpreter are not our issue
[09:17] <bdrung_work> pitti, which file format is the apport crash file?
[09:18] <pitti> bdrung_work: essentially RFC-822, see http://bazaar.launchpad.net/~apport-hackers/apport/trunk/view/head:/doc/data-format.tex
[09:19] <pitti> bdrung_work: if you can use python3, it's easier to use apport.Report(), this is more or less a dictionary with a write() method to spit out the .crash file format
[09:20] <bdrung_work> pitti, is there a rendered version of that tex file?
[09:20] <pitti> I was meaning to ship the PDF in apport, but apparently I missed to package it
[09:21] <pitti> bdrung_work: http://piware.de/tmp/data-format.pdf
[09:21] <bdrung_work> i can (and prefer) to use python3 for retracing, but the host (where the crash happens) should be kept simple. we use this script (or at least something similar): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=857300
[09:21] <ubottu> Debian bug 857300 in makedumpfile "makedumpfile: Please add a dump-core script" [Wishlist,Open]
[09:22] <pitti> bdrung_work: look at the various /usr/share/aport/ scripts like gcc_ice_hook or kernel_crashdump for examples how to use the api to generate .crash files
[09:24] <bdrung_work> pitti, what exactly happens when the kernel crashes (regarding the workflow with apport)?
[09:24] <bdrung_work> we have the contrain that the servers do not have a disk
[09:25] <pitti> bdrung_work: it calls /usr/share/apport/apport like in /proc/sys/kernel/core_pattern, see man core(5)
[09:25] <pitti> and feeds the core dump to its stdin
[09:26] <pitti> and apport then creates an apport.Report(), sets the report['CoreDump'] to stdin, and adds a few other fields from /proc and its CLI arguments (pid and signal number mostly)
[09:45] <bdrung_work> pitti, /usr/share/apport/apport looks interesting. the missing bits for our use case is support for directly uploading the coredump
[09:46] <bdrung_work> since the whole system runs on tmpfs
[09:46] <pitti> mount a tmpfs on /var/crash? :-)
[09:47] <bdrung_work> that needs ram too.
[09:47] <pitti> processing a core dump needs a lot of ram
[09:48] <pitti> with pipelining it directly to a remote server it might work better, but you probably want to do the .crash file assembly on the server then
[09:48] <pitti> (still need to compress the core dump in the pipeline)
[09:48] <bdrung_work> i know. that is why set /proc/sys/kernel/core_pattern to pipe to a core-dump script that pipes stdin into a compression into a busybox ftpput
[09:49] <bdrung_work> i would want to have the .crash metadata in a separate file
[09:49] <bdrung_work> then upload the metadata first and then pipe the compressed dump
[09:49] <pitti> ah, so replace CoreDump: in the local .crash file with some identifier that relates it to the ftpput, so that you can match it up on the server?
[09:50] <pitti> or CoreDumpID:
[09:50] <bdrung_work> yes. thinking of CoreDumpFile: foo-20170310.lz4
[09:50] <bdrung_work> the file will be in the same place
[09:51] <pitti> bdrung_work: I suggest using gzip, then you don't need to uncompress/recompress on the server
[09:51] <bdrung_work> gzip is too slow
[09:52] <bdrung_work> and the retrace would happen on a different host
[09:52] <bdrung_work> can gdb work with a gzipped crash dump?
[09:52] <pitti> no
[09:53] <pitti> it mmaps, so apport-retrace always unpacks it first
[09:53] <acheronuk> odd http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#rocs
[09:53] <acheronuk> virtual memory exhausted: Cannot allocate memory
[09:53] <acheronuk> on amd64
[09:54] <acheronuk> and on ppc64el
[09:54] <acheronuk> dotgrammar.cpp
[09:54] <acheronuk> c++: internal compiler error: Killed (program cc1plus)
[09:55] <acheronuk> only on the autopkgtest rebuilds. builds fine normally
[09:57] <pitti> probably needs to be added to bigtests
[09:58] <Laney> take the opportunity to figure out how to not have to rebuild the whole thing each time
[09:58] <acheronuk> pitti: to me? that allocates extra resources?
[09:58] <pitti> yes, that needs to be added to Ubuntu's CI config
[09:59] <pitti> but yes, if you can't build that with 1 GB of RAM, that's a bit evil :)
[10:01] <acheronuk> hmmm. longstanding issue it seems: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=730170
[10:01] <ubottu> Debian bug 730170 in src:rocs "rocs: FTBFS: virtual memory exhausted in DotGrammar.cpp" [Important,Open]
[10:16] <acheronuk> Laney: well as as far as I can see, the build at test time is not really avoidable. So could that maybe added to 'bigtests' as Martin suggested?
[10:30] <Laney> acheronuk: The autopkgtests are really meant to test against the already built packages that are in the archive, and here I bet you're testing against the thing you just built.
[10:30] <Laney> Perhaps you can build the tests against the system libraries, or perhaps you can ship them in a binary package like some GNOME stuff does. That would likely require some upstream work though.
[10:45] <acheronuk> Laney: yes, it would require upstream changes that are likely beyond me/us to work out and submit. I get what you are saying, but this is an interesting but not core/crucial KDE app, so time/benefit there means is not going to happen I think
[10:49] <Laney> acheronuk: Ok, but basically all of the KDE applications have the same problem AFAICS
[10:49] <Laney> In addition to not telling you if the packages in the distro are broken, they are a serious strain on the infrastructure
[10:51] <acheronuk> Laney: yeah, quite aware of that :/
[10:53] <acheronuk> gotta go. thanks
[10:59] <Laney> acheronuk: I put it in big_packages for you
[10:59] <Laney> Some attention to the wider problems would be appreciated
=== hikiko is now known as hikiko|bbl
=== _salem is now known as salem_
[12:06] <bdrung_work> pitti, do you collect just the dependencies of the package or do you also collect recommends/suggests?
[12:08] <pitti> bdrung_work: backends/packaging-apt-dpkg.py get_dependencies() → Pre-Depends:, Depends:, Recommends:
[12:08] <pitti> bdrung_work: normally that doesn't matter that much for retracing though, as it prefers ProcMaps
[12:08] <pitti> that will skip irrelevant packages, and more importantly, also catch plugins or other dynamically loaded stuff
[12:09] <pitti> but it falls back to Dependencies: if there is no ProcMaps field (for your case)
[12:09] <bdrung_work> ah, cool. and how does it map from ProcMaps to the package with the debug information?
[12:10] <pitti> bdrung_work: apport-retrace downloads Contents.gz for that
[12:10] <pitti> and from there, try to install -dbg, or -dbgsym
[12:10] <pitti> (but you need both the package and the symbols)
[12:11] <bdrung_work> so with contents you can map the file to the package
[12:11] <bdrung_work> how do you map the package to the -dbg or -dbgsym package?
[12:11] <pitti> with string concatenation :)
[12:12] <pitti> and then asking apt.Cache() if that exists
[12:12] <bdrung_work> heuristics :D
[12:12] <pitti> bdrung_work: http://bazaar.launchpad.net/~apport-hackers/apport/trunk/view/head:/backends/packaging-apt-dpkg.py#L672
[12:13] <bdrung_work> I am collecting some more information now: http://paste.ubuntu.com/24151344/
[12:13] <bdrung_work> thanks for the documentation of these fields.
[12:13] <pitti> yeah, these days we migth actually use the debug links that are part of -dbgsym packages, but this was all written way before that existed
[12:14] <pitti> bdrung_work: btw, apport itself doesn't collect the expensive stuff (Package:, Dependencies: etc.) at crash time
[12:14] <pitti> only after you actually agree to send a report
[12:14] <pitti> just the stuff that's absolutely required to collect while the process is core dumping (/proc and the core dump)
[12:14] <pitti> but for automatic submission you need to do it, I agree
[12:14] <bdrung_work> how do you make sure that the packages haven't been updated in the meantime?
[12:14] <pitti> it compares time stamps
[12:15] <pitti> but since the popup is triggered via inotify, that rarely happens
[12:15] <bdrung_work> which timestamps?
[12:15] <pitti> of the executable file
[12:17] <bdrung_work> and what about the libraries?
[12:17] <pitti> no timestamp for those
[12:18] <bdrung_work> would you notice changes for them?
[12:18] <pitti> (it has never been a practical problem)
[12:18] <pitti> due to the inotify popups
[12:21] <mdeslaur> could someone please push gnutls28 and pcsc-lite out of proposed, please? The failing tests are unrelated.
=== hikiko|bbl is now known as hikiko
[12:41] <Laney> mdeslaur: I retried them with --all-proposed, think that'll work
[12:42] <mdeslaur> Laney: thanks
[12:43] <Laney> (any uploader can do that, BTW)
[12:43] <Laney> mmm, not for pcscada apparently
[12:45] <mdeslaur> Laney: what tool do you use?
[12:46] <Laney> mdeslaur: retry-autopkgtest-regressions with --all-proposed from lp:ubuntu-archive-tools
[12:47] <mdeslaur> Laney: cool, thanks
[14:34] <jbicha> tseliot: hi, I'm pinging again about bug 1559576, a very serious bug for Ubuntu GNOME that looks like it has a very simple fix
[14:34] <ubottu> bug 1559576 in gdm3 (Ubuntu Xenial) "Ubuntu GNOME boots to black screen when using proprietary Nvidia drivers" [Critical,Triaged] https://launchpad.net/bugs/1559576
=== salem_ is now known as _salem
[15:24] <tseliot> jbicha: yes, sorry but I've been extremely busy with other stuff
[15:26] <jbicha> tseliot: ok
=== balkamos_ is now known as balkamos
[15:52] <bdmurray> powersj: Have you identified a patch which fixes bug 349913?
[15:52] <ubottu> bug 349913 in openssh (Ubuntu) "sftp: cannot enter umlauts like ä, ö, ü" [Medium,Fix committed] https://launchpad.net/bugs/349913
[15:53] <powersj> bdmurray: not yet
[15:54] <bdmurray> powersj: but you thought it was worth adding a Trusty task?
[15:55] <powersj> bdmurray: sorry if I did this the wrong way, I wanted to work on the bug to fix in trusty, and wanted to assign to to myself, however since it was marked fix committed I thought adding trusty and assigning it to me would be the way to track that
[15:56] <powersj> if I found the backport to be too large or crazy then could note that, otherwise work on an SRU
[15:56] <bdmurray> powersj: no, adding a task if you going to assign yourself makes sense to me.
[15:56] <bdmurray> powersj: there, its been approved
[15:57] <powersj> bdmurray: thank you!
[16:10] <cjwatson> powersj: I've suggested some likely commits in the bug, but personally I would consider this too invasive for backporting.
[16:11] <cjwatson> Especially since it did go wrong.
[16:11] <powersj> cjwatson: thanks for looking at it! I will happily take your advice :)
[16:20] <barry> why did the recent chromium browser upgrade *again* forget all of my passwords?  this is the second or third time during zesty that it's done that.  I'm glad i haven't upgraded my other machine yet or i'd have to do password resets for every site i generate random passwords for.  now i have to pick them out of my other browser instance, but some of my passwords are loooongggg
[16:36] <jackpot51> Hey guys - does anyone have a lead on Gnome Software not installing dependencies for .deb packages?
[16:41] <jbicha> jackpot51: did you file a bug yet?
[16:45] <jackpot51> I think https://bugs.launchpad.net/bugs/1573206
[16:45] <ubottu> Launchpad bug 1573408 in gnome-software (Ubuntu Xenial) "duplicate for #1573206 GNOME Software does not install third-party .deb packages" [High,Fix released]
[16:46] <jackpot51> https://bugs.launchpad.net/ubuntu/+source/gnome-software/+bug/1573408 says fix released, am I missing something?
[16:48] <ubottu> Launchpad bug 1573408 in gnome-software (Ubuntu Xenial) "GNOME Software does not install third-party .deb packages" [High,Fix released]
[16:48] <nacc> jackpot51: fix released refers to the development release at hte time
[16:48] <jackpot51> Oh, hope to see it in 17.04 then
[16:50] <nacc> jackpot51: oh but i see that bug also has the 16.04 task as FR
[16:52] <nacc> ugh, that bug is hard to read
[16:55] <ginggs> fixed in https://launchpad.net/ubuntu/+source/gnome-software/3.20.1+git20160426.1.a976144-ubuntu-xenial-0ubuntu1
[16:56] <jackpot51> Thanks ginggs
[16:58] <nacc> ginggs: confused, though, people kept saying they saw it after that was released?
[16:59] <ginggs> maybe that release didn't really fix it, or it was subsequently broken again
[17:00] <nacc> ginggs: ack, it's very unclear from the bug itself
[17:00]  * nacc is glad not to really work on desktop stuff :)
[17:11] <dobey> barry: ugh, really?
[17:11] <dobey> barry: i don't see how it would forget passwords, since it uses gnome-keyring as the storage
[17:11] <tyhicks> mwhudson: hi - the docker-in-lxd test is broken with recent lxd uploads
[17:12] <tyhicks> mwhudson: you seem to be tending to that test so I wasn't sure if fixing it was something that you're already working on
[17:18] <tyhicks> stgraber: it looks like lxd-bridge unit went away recently - what's the correct way to apply configuration that is manually written to /etc/default/lxd-bridge? (asking because the docker-in-lxd docker autopkgtest is stopping lxd-bridge, writing to that file, and then starting lxd-net)
[17:19] <tyhicks> (I meant "and then starting lxd-bridge")
[17:20] <jackpot51> nacc ginggs: I will test it again with a clean install
[17:22] <nacc> tyhicks: iirc, that is now stored in the lxd database
[17:22] <nacc> tyhicks: and there is a script to migrate, but fresh users don't have any /etc/default/lxd-bridge
[17:22] <nacc> tyhicks: i believe this changed with `lxc network`
[17:23] <jackpot51> ginggs nacc: Clean 16.10 install, Chrome installation fails silently with the Software Center
[17:24] <tyhicks> nacc: ah, I'm starting to remember that now
[17:24] <tyhicks> nacc: thanks!
[17:24] <nacc> jackpot51: there does appear to be a version in yakkety-proposed that might be worth testing
[17:26] <jackpot51> nacc: Ok, I will update, purge the half-installed google-chrome-stable, and retry
[17:26] <stgraber> tyhicks: yeah, on LXD 2.3 or higher (whichever advertises the "network" extension), you should use the "lxc network ..." commands instead.
[17:27] <stgraber> tyhicks: we have code that deals with that in the autopkgtest package and also deals with the new storage API from 2.9+
[17:27] <stgraber> tyhicks: let me get you a link
[17:27] <stgraber> tyhicks: that's the LXD test setup logic in autopkgtest: //paste.ubuntu.com/24152739/
[17:27] <stgraber> *http://paste.ubuntu.com/24152739/
[17:28] <nacc> jackpot51: and the i'd check the chagnelog of the yakkety-proposed one to see if it's also got a bug referring to ti
[17:28] <stgraber> tyhicks: in the autopkgtest case, it doesn't bounce lxd-bridge because it knows that it's not going to be running at the time (clean setup) and so just relies on the socket activation to bring up the unit afterwards
[17:29] <tyhicks> stgraber (cc mwhudson): thanks but I now see what's going on with this docker.io autopkgtest...
[17:29] <tyhicks> stgraber (cc mwhudson): it already knows how to deal with `lxc network` but it doesn't know how to compare software versions
[17:29] <jackpot51> nacc: Works with updates, just not with the default ISO
[17:30] <stgraber> tyhicks: ah, it shouldn't compare versions, it really should check for the LXD extensions instead
[17:30] <nacc> jackpot51: yes, that's sort of expected -- that is with -updates (and not -proposed0?
[17:30] <tyhicks> stgraber (cc mwhudson): it thinks that version 2.10 is less than 2.2 and is going back to the old way of configuring the bridge
[17:30] <nacc> jackpot51: as the we don't respin the ISO; were you using a daily?
[17:30] <stgraber> tyhicks: haha, funny :)
[17:30] <tyhicks> yeah :)
[17:30] <stgraber> I think there was a similar but in Juju a while back
[17:31] <tyhicks> the extension check is the fix - thanks!
[17:36] <jackpot51> nacc: Complicated - I work for System76 so we do OEM install and keep the install image up to date. ISO failed, but our install doesn't
[17:38] <nacc> jackpot51: got it!
[17:38] <nacc> jackpot51: does your install image use -updates?
[17:38] <nacc> jackpot51: the 16.10 ISO (release ISO) i think never gets updated
[17:38] <jackpot51> nacc: Yes, the computers will ship with updates
[17:39] <nacc> jackpot51: but the 16.10 daily would have updates applied (i think?)
[17:39] <jackpot51> Right
[17:40] <nacc> jackpot51: so it's not surprising (at least to me) that there is a bug int he release version that's later fixed
[17:40] <nacc> jackpot51: same would be true of 16.04.0; but 16.04.1 (and i think 16.04.2) have the fix
[17:49] <jbicha> jackpot51: if you have a good test case, please file a new bug
[17:49] <jbicha> if the problem is with Google Chrome, I think I heard that there might be a problem there but I don't think there's a bug yet
[17:50] <jbicha> but the original xenial "can't install a .deb" bug was fixed
[17:51] <jgrimm> nacc, fwiw i have a debdiff up for the tc man page bug you pointed me at if you'd want to sponsor it. https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1668813
[17:51] <ubottu> Launchpad bug 1668813 in iproute2 (Ubuntu Xenial) "The tc man page references tc-index man page but tc-index man page does not exist" [Low,In progress]
[17:51] <jgrimm> not urgent at all
[17:52] <nacc> jgrimm: ack
[17:52] <nacc> jgrimm: i believe your version is incorrect?
[17:52] <nacc> jgrimm: should be 4.3.0-1ubuntu3.1 ?
[17:53] <jgrimm> bah
[17:53] <jgrimm> nacc, i'll fix
[17:53] <nacc> jgrimm: and, unfortunately, we'll need to fix yakkety too :/
[17:53] <nacc> otherwise current upgrades who get the fix from xeniall will not update in yakkety
[17:53] <nacc> well, i guess that's not 'fatal', but it's unexpected
[17:53] <nacc> (x and y have the same version)
[17:53] <jgrimm> nacc, agreed, and trivial
[17:53] <jgrimm> to do
[17:54] <nacc> jgrimm: yep, should be easy -- will need a bit of version love :)
[17:54] <nacc> 4.3.0-1ubuntu3.16.{04,10}.1 i think
[17:56] <nacc> jgrimm: also, would you mind using `dep3changelog` ?
[17:56] <nacc> jgrimm: that way "Phil Sutter" gets attribution in the changelog :)
[17:56] <jgrimm> nacc, apologies for not thinking through the versions, was juggling multiple things yesterday
[17:56] <nacc> jgrimm: not a problem! nothing to apologize for
[17:57] <jgrimm> ah, dep3changelog is new to me. cool, i was just adding based on looking a debian docs
[17:57] <jgrimm> but \o/ for tooling
[17:58] <nacc> jgrimm: yep! :)
[18:29] <barry> dobey: yep, definitely.  maybe it's gnome-keyring that got updated and somehow zeroed out its db.  or maybe it just hates me
[18:31] <dobey> barry: could be maybe the back-end in chromium got flipped to the native thing
[18:31] <dobey> or something weird
[18:31]  * barry votes for something weird
[18:32] <barry> dobey: yeah, it's happened at least twice during zesty
[18:32] <dobey> hmm
[18:32] <dobey> someone should write an autopkgtest :)
[18:32] <barry> ;)
[20:27] <chiluk> barry is there a way to get launchpad mail for all bugs that contain a specific tag?  in my case sts-sponsor?
[20:27] <chiluk> barry I only picked on you since you were last to touch the channel.
[20:27]  * barry quit:
[20:27] <chiluk> lol.
[20:28] <barry> chiluk: i'm not sure, but there may be an option in your bugmail
[20:28] <chiluk> i'll go ask in launchpad
[20:28] <barry> yeah
[20:28] <barry> probably best
=== zul is now known as zulVacation
[21:21] <xnox> infinity, libc postinst hangs upgrades to yakkety when doing trusty->xenial->yakkety without rebooting
[21:22] <xnox> apperantly telinit doesn't handle well, when telinit u causes one to reexec upstart into systemd
[21:23] <xnox> i thought you might find it funny =) i have killed hung telinit, and continue with upgrades.
[21:23] <xnox> it is a lot of fun.
[22:52] <jgrimm> nacc, i'd think it would be 1:4.3.0-1ubuntu4 for yakkety, and 1:4.3.0-1ubuntu3.1 for xenial?
[22:53] <jgrimm> sorry, just getting back to this finally today
[23:03] <mun24> how to make ssh connection using an existing tcp/ip connection
[23:09] <nacc> jgrimm: fwiw, ubuntu4 would not be typical for a released version
[23:09] <nacc> jgrimm: my suggestionearlier was off of https://wiki.ubuntu.com/SecurityTeam/UpdatePreparation#Update_the_packaging
[23:10]  * jgrimm goes off to read. thanks
[23:13] <jgrimm> nacc, ok i see. invoking the >1 release with common version. thanks.  indeed then would be 1:4.3.0-1ubuntu3.16.10.1 and 1:4.3.0-1ubuntu3.16.04.1
[23:17] <nacc> jgrimm: yep
[23:17] <jgrimm> i was looking for a page like that yesterday, thanks for the pointer
[23:17] <nacc> jgrimm: from a review perspective, as well, it means i don't really need to check if ubuntu4 is already used or published or whatever in the devel release
[23:17] <nacc> jgrimm: as those versions are so distinct
[23:18] <jgrimm> indeed, i can totally see that
[23:18] <nacc> jgrimm: that's on my list of things to broadcast wider in :)
[23:18] <jgrimm> good cheatsheet!!
[23:18] <nacc> that page, that is
[23:18] <jgrimm> indeed, and dep3changelog was cute
[23:18] <jgrimm> i'll definitely use that again
[23:19] <nacc> yeah, it simplifies one step (IMO) and makes it repeatable
[23:19] <jgrimm> +1
[23:26] <jgrimm> nacc, y and x debdiffs attached to bug fwiw
[23:26] <nacc> jgrimm: thanks, i'll probably review them early next week
[23:26] <jgrimm> +1 no urgency at all
[23:40] <Unit193> cjwatson: Have you seen https://launchpad.net/bugs/1668093?  It'd likely be pretty useful considering https://launchpad.net/bugs/1670745 (https://bugzilla.mindrot.org/show_bug.cgi?id=2692)
[23:40] <ubottu> Launchpad bug 1668093 in openssh (Ubuntu Yakkety) "ssh-keygen -H corrupts already hashed entries" [High,Triaged]
[23:40] <ubottu> Launchpad bug 1670745 in openssh (Ubuntu) "ssh-keyscan : bad host signature when using port option" [Undecided,New]
[23:40] <ubottu> bugzilla.mindrot.org bug 2692 in ssh-keyscan "Hash does not include the port" [Minor,Resolved: fixed]
[23:44] <cjwatson> Unit193: I replied to it ...
[23:45] <Unit193> Well dang, I missed that part.  Wasn't subbed. >_<
[23:45] <Unit193> Sorry for the ping.
[23:46] <cjwatson> np - might be worth me cherry-picking the fix for bug 1670745 too though, from the sounds of it
[23:46] <ubottu> bug 1670745 in openssh (Ubuntu) "ssh-keyscan : bad host signature when using port option" [Undecided,New] https://launchpad.net/bugs/1670745
[23:47] <Unit193> Yeah I just saw 8a2834454c73dfc1eb96453c0e97690595f3f4c2 too, that'd be greatly useful, though if the first is fixed at least there's a workaround.