[01:52] <callmepk> good morning
[02:17] <duflu> Morning callmepk 
[02:17] <kenvandine> good morning callmepk and duflu 
[02:17] <kenvandine> and jamesh 
[02:17] <duflu> Hi kenvandine. How goes?
[02:17] <callmepk> morning duflu kenvandine jamesh 
[02:17] <jamesh> hi callmepk, duflu 
[02:18] <duflu> Morning jamesh 
[06:04] <oSoMoN> good morning desktoppers
[06:25] <didrocks> good morning
[06:25] <oSoMoN> salut didrocks 
[06:26] <didrocks> salut oSoMoN, ça va, pas terminé trop tard hier soir ?
[06:33] <duflu> Morning oSoMoN and didrocks 
[06:34] <oSoMoN> didrocks, un peu après 21h, comme prévu, ça va
[06:34] <oSoMoN> hey duflu 
[06:37] <didrocks> hey duflu 
[06:52] <ricotz> good morning
[06:52] <ricotz> hey oSoMoN didrocks duflu 
[06:53] <didrocks> hey ricotz 
[06:53] <duflu> Hi ricotz 
[06:57] <seb128> gooood morning desktopers!
[06:57] <seb128> hey ricotz didrocks duflu, how is it going?
[06:57] <duflu> Hi seb128. Going OK. You?
[06:58] <seb128> I'm alright thanks!
[06:58] <didrocks> hey seb128, ça va, et toi ?
[06:59] <seb128> trying to understand why the new installer snap is segfaulting in swrast_dri.so :/
[07:00] <seb128> didrocks, ça va :)
[07:00] <seb128> it works when using GALLIUM_DRIVER=softpipe
[07:01] <seb128> something wrong with the driver in the classic snap env :/
[07:02] <duflu> seb128, sounds like a VM? That driver shouldn't be used on real hardware
[07:04] <seb128> duflu, the snap forces LIBGL_ALWAYS_SOFTWARE=1 , unsure why Ken did that but I assume it was to workaround driver issues in the snap env
[07:04] <duflu> :(
[07:04] <seb128> duflu, it's not the issue though, if I unset that it segfault in iris_dri.so
[07:05] <seb128> it's staging those from a deb and copying the deb files over fixes the segfault
[07:05] <duflu> seb128, kind of good. If you can get a stack trace with symbols (or without) I can try and triage the crash
[07:05] <seb128> so it's something snapcraft is doing
[07:05] <seb128> it's setting an rpath
[07:05] <seb128> which I think is creating the issue
[07:05] <duflu> Although if it's just snap-mixing-versions-of-mesa then it's not a Mesa bug
[07:05] <seb128> but if I patchelf --remove-rpath it's not enough to resolve the issue
[07:06] <seb128> duflu, well, at this point I'm puzzled
[07:06] <seb128> it's not a bug in the code otherwise the deb .so copied over would segfault the same way
[07:06] <seb128> it's something snapcraft is doing to it
[07:06] <duflu> seb128, anything like a crash file or oops link?
[07:06] <seb128> I wonder if setting the rpath could lead to a bug in some way
[07:07] <seb128> duflu, no, that's inside a classic snap :/
[07:07] <seb128> rogram received signal SIGSEGV, Segmentation fault.
[07:07] <seb128> 0x00007ffff69af93d in ?? ()
[07:07] <seb128>    from /snap/ubuntu-desktop-installer/x1/usr/lib/x86_64-linux-gnu/dri/iris_dri.so
[07:07] <seb128> (that's trying to start glxinfo)
[07:08] <seb128> I rebuilt the deb witgh nostrip to get the debug info
[07:08] <duflu> Ideally that file should not exist because we want the OS to only have a single set of Mesa drivers. Otherwise they will crash each other
[07:08] <seb128> which is how I discovered that the deb build doesn't segfault...
[07:08] <jibel> Good morning
[07:08] <duflu> Hi jibel 
[07:08] <seb128> jibel, hey
[07:08] <seb128> duflu, what do you mean 'crash each other'?
[07:08] <seb128> maybe that's the issue there
[07:09] <duflu> seb128, I mean the slightest version difference means ABI changes and crashes
[07:09] <seb128> though than seeing from the snap the idea I think is that only the snap ones are seen
[07:09] <seb128> duflu, https://paste.ubuntu.com/p/mJGDPctxy2/
[07:09] <duflu> But good news: The libGL ABI is super stable for decades. You shouldn't need to ship your own
[07:11] <seb128> it's a bit weird that ld-linux-x86-64.so.2 is used from the system though, unsure if that could lead to this issue...
[07:12] <duflu> seb128, that's the dynamic loader. I would expect you don't get the option of replacing that one
[07:12] <seb128> k, so not my issue
[07:13] <marcustomlinson> morning desktoppers
[07:13] <seb128> marcustomlinson, hey, how are you?
[07:13] <didrocks> hey marcustomlinson 
[07:14] <duflu> Hi marcustomlinson 
[07:14] <marcustomlinson> seb128: hey, tired but alright thanks
[07:14] <marcustomlinson> hey didrocks and duflu
[07:15] <jibel> hi duflu seb128 didrocks marcustomlinson 
[07:15] <seb128> marcustomlinson, :-(
[07:16] <marcustomlinson> hey jibel
[07:22] <seb128> duflu, ah, I did success with debug symbols by copying over the ddebs files to the snap env
[07:22] <seb128> duflu, https://paste.ubuntu.com/p/WpJYNqwFwC/
[07:23] <duflu> seb128, good? But if there's any version mismatch with the system then the stack would still be bogus. So make sure it's an identical Mesa build
[07:23] <seb128> duflu, they are, the snap just prime the focal deb which is the same build as the ddeb
[07:24] <seb128> duflu, the only difference is that snapcraft sets an rpath on the binaries due to how classic snaps work (so it loads libraries from the snap env and not the system ones)
[07:24] <duflu> Hmm duplicate static variables could still be a problem there but unlikely
[07:25] <seb128> how would that happen?
[07:25] <seb128> I wonder if patchelf could damage the binary in some way :/
[07:26] <seb128> $ objdump -x /tmp/lib/x86_64-linux-gnu/dri//swrast_dri.so |grep RPATH
[07:26] <seb128>   RPATH                $ORIGIN/..:/snap/core20/current/lib/x86_64-linux-gnu
[07:26] <duflu> Yeah patchelf sounds like it's designed to break things in a controlled way
[07:29] <seb128> ok, so I think it's it and I'm not the first one to hit that
[07:29] <seb128> https://github.com/MirServer/egmde-snap/blob/master/snap/snapcraft.yaml#L117
[07:29] <seb128> '      - no-patchelf # Otherwise snapcraft may strip the build ID and cause the driver to crash'
[07:29] <seb128> for whatever reason remove that build ID which snapcraft is doing is making the driver crash
[07:31] <duflu> seb128, ah yes. "build_id" in the function names of the crash
[07:32] <seb128> just spent a day to get there, tricky problem to get at the bottom of
[07:32] <seb128> marcustomlinson, ^ finally found the real issue
[07:35] <seb128> and https://forum.snapcraft.io/t/caveats-for-no-patchelf-in-a-classic-snap/ has the details
[07:36] <seb128> 'As part of the shuffling, in some scenarios, patchelf may end up shuffling the GNU build id note section, that normally will live in the “note” segment (i.e. PT_NOTE) into another segment (not PT_NOTE). This may cause issues for applications that assume they are in the note segment, such as mesa (https://github.com/intel/external-mesa/blob/master/src/util/build_id.c#L75)'
[07:41] <seb128> duflu, ^ so that was it, thanks for proposing to help :)
[07:57] <duflu> seb128, GL APIs and ABIs are very stable long term. Maybe the most stable ones around. So those libraries ideally shouldn't be needed in any snaps
[07:58] <duflu> Pretty much what worked in the 1990s still does now
[07:59] <duflu> I mean shouldn't be duplicated from the system ones
[08:00] <seb128> duflu, thanks for the hint, I will check if we can do without those
[08:04] <Laney> moin
[08:13] <oSoMoN> good morning ricotz, seb128, marcustomlinson, jibel, Laney 
[08:13] <oSoMoN> seb128, well done on getting that snap to work, it was a nasty issue
[08:14] <seb128> hey Laney, how are you?
[08:15] <seb128> oSoMoN, thanks! and indeed, I feel better now, spent most of the day yesterday on it and I kept me busy thinking in the evening/night on what it could be
[08:15] <seb128> I'm glad I figured it out now, feeling better!
[08:15] <duflu> Hi Laney
[08:16] <didrocks> hey Laney 
[08:16] <jibel> salut oSoMoN 
[08:25] <Laney> hey oSoMoN seb128 duflu didrocks jibel 
[08:25] <Laney> doing good, was out this morning potting up some sunflowers and peas
[08:25] <Laney> more to do later
[08:25] <Laney> seb128: well done on getting to the end of that investigation
[08:26]  * duflu high fives Laney after fertilising some flowers at lunch
[08:28] <Laney> \o\
[08:33] <marcustomlinson> hey oSoMoN and Laney
[08:34] <Laney> duflu: I guess there's a decent amount of interesting things that will grow for you in winter
[08:34] <Laney> yo marcustomlinson 
[08:35] <duflu> Laney, I usually only try to stop the tropicals from dying. They don't like anything near freezing, despite that being rare
[08:36] <Laney> nod
[09:09] <seb128> Laney, thanks :-)
[09:10] <seb128> wifi router went unhappy for some reason, hangout was still working but most of the other things timeout in the back
[09:10] <seb128> I reconnected and it's all working, weird
[09:25] <jibel> Good morning Laney 
[10:00] <Laney> \o
[10:00] <Laney> seb128: https://people.canonical.com/~platform/desktop/versions/stats/current/ looks a bit odd, those subdirs
[10:02] <seb128> Laney, what did I do this time :p
[10:03] <Laney> touched a computer, that was your first mistake
[10:03] <seb128> that's right!
[10:13] <seb128> Laney, fixed, thanks for reporting! I was basically doing 'ln -sf $(date) current', works fine the first time but when 'current' exists it consider it as a the target dir, needed to add 'n' to the arguments
[10:17] <Laney> nice one!
[10:17] <seb128> thanks for the reviews!
[10:17] <seb128> I will check the grafana side later :)
[13:06] <seb128> jibel, I would appreciate a review on https://code.launchpad.net/~ubuntu-bugcontrol/ubuntu-qa-tools/+git/ubuntu-qa-tools/+merge/402641 before the long weekend, I think we are still not getting daily current iso due to that one
[13:07] <seb128> jibel, also the dl-ubuntu-test-iso snap needs a rebuild to pick an updated distro-info-data
[13:07] <seb128> it's a bit annoying how distro-info just errors out after new ubuntu series are created
[14:01] <jibel> seb128, okay, I'll have a look
[14:02] <jibel> sil2100, Laney would you have time to review https://code.launchpad.net/~jibel/livecd-rootfs/+git/livecd-rootfs/+merge/402585 ?
[14:05] <seb128> jibel, thanks
[14:21] <Laney> jibel: sure, that looks fine™ to me, what could possibly go wrong?
[14:26] <hellsworth> good morning desktopers
[14:27] <Laney> wait a second
[14:27] <Laney> jibel: have you checked this in a ppa?
[14:27] <Laney> hey hellsworth 
[14:29] <jibel> Laney, no, I just ran the germinate command to see if it generated the expected seed. I am not sure if there could be some side effects on other flavours or images actually
[14:29] <seb128> hey heather, how are you?
[14:29] <jibel> Good morning hellsworth 
[14:30] <hellsworth> hi folks
[14:30] <hellsworth> i'm good thanks, how are you seb?
[14:33] <Laney> jibel: I think your problem is not this, but the lack of Task: headers in the archive, the package is in the live seed so it should get Task: ubuntu-live, which add_task() is meant to pick up
[14:33] <Laney> laney@nightingale> GET http://archive.ubuntu.com/ubuntu/dists/focal-updates/main/binary-amd64/Packages.xz | xzcat | grep-dctrl -sPackage,Version,Task -FPackage -X realmd
[14:33] <Laney> Package: realmd
[14:33] <Laney> Version: 0.16.3-3
[14:33] <Laney> laney@nightingale>                                                                                              ~
[14:37] <seb128> Laney, do you remember when the current +1 rotation started?
[14:37] <seb128> hellsworth, I'm alright thanks!
[14:37] <Laney> around start of cycle or a week before
[14:37] <Laney> ish
[14:38] <seb128> hellsworth, btw you did one week but have another to do in theory which I think we didn't plan, unsure when the current rotation is ending though
[14:38] <hellsworth> i would love to!!!
[14:38] <seb128> you could do one next week or the week after?
[14:38] <seb128> unsure how busy you are with other things
[14:38] <hellsworth> next week i move and my life will be kind of crazy. but the following week would be good
[14:39] <Laney> I need to put another one in too
[14:39] <seb128> k, let me check with Matthieu if that's still in this rotation period
[14:39] <seb128> I never know where to check, if there is a place
[14:39] <Laney> there's a calendar but it doesn't actually list the rotation periods
[14:39] <Laney> would be a good suggestion to add those as events on there
[14:40] <hellsworth> seb: next week is bad, the week after is good, the week after is bad, any future weeks are good
[14:40] <seb128> k, let's try for the week after next
[14:40] <seb128> thanks!
[14:40] <hellsworth> awesome!
[14:42] <seb128> oh also, waht's the status of the glimpse-editor snap build fixes? ;)
[14:42] <seb128> that's still having a backlog on the trello USN rebuilds!
[14:42] <seb128> and the cherrytree one is also on there
[14:42] <jibel> Laney, when you look at the build log, it's ignored because it is not in the germinated seed
[14:44] <jibel> Laney, updates is not considered by germinate and for germinate the package doesn't exist
[14:45] <hellsworth> seb128: the glimpse-editor fixes requires some back and forth with upstream but one of my 3 PRs to fix the builds were merged yesterday. I expect the others to be merged today or soon
[14:45] <hellsworth> cherrytree is building so I'll go look at why it's still there
[14:47] <Laney> jibel: I see it, but I don't think this germinate output is used for seeding debs, that is using apt-cache inside the chroot looking at the indexes that it got from the Ubuntu archive
[14:47] <Laney> we can try a test build to be sure if you want
[14:50] <jibel> sure, how do we do that?
[14:52] <jibel> isn't the live task generated from the list of germinated packages?
[14:52] <Laney> yes, but that's done at the archive level by lp:ubuntu-archive-tools
[14:52] <Laney> and then expressed as a Task header in the Packages file
[14:53] <Laney> testing> I'll upload to a PPA, that's the first step
[14:53] <Laney> then I have to remember how to use the API to make livefs builds
[14:55] <seb128> hellsworth, thanks! cherrytree seems it's arm64 being outdated, so either that fails to build and needs fixing or the binaries on that arch need to be deleted from the store
[14:55] <hellsworth> i will look at that today!
[14:55] <seb128> hellsworth, oh, and i386 but those should probably be deleted
[14:55] <seb128> thx
[14:56] <seb128> it would clear a stack for trello cards
[14:56] <hellsworth> oh i know what happened. fixing it real quick
[14:56] <seb128> great!
[15:01] <Laney> jibel: it is building in https://launchpad.net/~laney/+archive/ubuntu/livefs
[15:02] <Laney> once I re-figure out how to make the livefs build I will share the knowledge with you
[15:35] <jibel> Laney, thanks!
[16:25] <hellsworth> seb the cherrytree snaps in the store are updated now (no more i386 either). and those trello cards will automatically close, right?
[16:26] <hellsworth> seb128: ^
[16:27] <seb128> hellsworth, lol, I was about to write that that my script listed them to close and that it's done now, thanks for fixing!
[16:28] <hellsworth> ha great!
[16:29] <seb128> we still have a few that didn't build for a while but we are getting there
[16:34] <hellsworth> indeed!
[16:51] <Laney> jibel: https://launchpad.net/~laney/+livefs/ubuntu/focal/ubuntu/+build/275004 https://paste.ubuntu.com/p/9BZ9JPsJPw/
[16:52] <Laney> (whoops, ignore 8)
[17:06] <jibel> seb128, your MP has been merged and built https://code.launchpad.net/~canonical-platform-qa-jenkins/+archive/ubuntu/canonical-platform-qa-jenkins-prod/+build/21539931
[17:06] <jibel> seb128, I'll deploy it on the server as soon as it's published
[17:27] <jibel> seb128, and done
[17:52] <seb128> jibel, thanks!