| mborzecki | morning | 06:22 |
|---|---|---|
| mvo | mborzecki: good morning | 06:23 |
| mvo | mborzecki: I cherry-picked 8575 to 2.44 | 06:37 |
| mvo | mborzecki: that is needed, right? we will need a new 2.44.4 with a fix for syscalls with base: core20 | 06:37 |
| mborzecki | mvo: thanks! | 06:38 |
| mborzecki | mvo: yes, it fixes the problems with libcrypto dependency being injected on centos | 06:38 |
| mborzecki | mvo: i'll need to add a similar change in downstream packaging for fedora/epel | 06:39 |
| mvo | ta | 06:39 |
| zyga | Good morning | 06:40 |
| zyga | How are you guys? | 06:41 |
| zyga | I will start late but I will look at opensuse a little today | 06:41 |
| zyga | I’ll fix our tests there and update the packaging /release | 06:41 |
| mvo | zyga: hey, I'm good, thanks! | 06:42 |
| mborzecki | qemu 5.0 is out, supports emulating tpm for arm | 06:45 |
| zyga | Groovy | 06:45 |
| zyga | Can we snap it? :-) | 06:46 |
| mborzecki | mvo: hmm 2.44 travis unit test job runs with different go version? | 06:52 |
| mborzecki | ah, it's go master | 06:53 |
| mvo | mborzecki: yeah, just removed that | 06:56 |
| mvo | mborzecki: I want to see the spread run | 06:56 |
| mborzecki | i'm out for a bit, and yay it's raining! | 06:57 |
| pstolowski | morning | 07:04 |
| mvo | hey pstolowski - good morning | 07:04 |
| === rawr is now known as grumble | ||
| juergh | mvo, zyga can one of you help me with that snapd download, please? | 07:43 |
| zyga | I will try | 08:01 |
| zyga | Which version do you need? | 08:01 |
| zyga | juergh: ^ ? | 08:22 |
| ogra | did xdg-open change recently ? | 08:29 |
| ogra | error: error running snapctl: Unknown command `user-open'. Please specify one command of: get, is-connected, restart, services, set, set-health, start, stop or unset | 08:29 |
| zyga | ogra: hmm, weird | 08:33 |
| zyga | ogra: perhaps a bug | 08:33 |
| ogra | i'm not yet sure, might be that the app changes ... the snapctl got me hooked though | 08:34 |
| ogra | *changed | 08:34 |
| * zyga breakfast | 08:37 | |
| juergh | zyga, snapd 2.43.3 | 08:42 |
| zyga | ok | 08:42 |
| zyga | give me a moment please | 08:42 |
| zyga | juergh: which architecture do you need? | 08:51 |
| juergh | zyga, amd64 | 08:52 |
| zyga | juergh: hmm, but wait, 2.43.3 is latest stable | 08:52 |
| zyga | ah | 08:52 |
| zyga | sorry 44 | 08:52 |
| * zyga needs coffee | 08:52 | |
| * zyga looks at fixing the opensuse build issue now | 09:59 | |
| tpreston | Hi, is there a reason why snapd /etc/profile.d/snapd.sh *appends* snap_bin_path to PATH, rather than prepends? | 10:01 |
| tpreston | https://forum.snapcraft.io/t/should-snap-bin-be-prepended-to-path/4506 | 10:01 |
| zyga | tpreston: it's a more conservative choice, I suspect | 10:01 |
| tpreston | I have ctags installed at a system level (because of a package dependency), but I'd prefer to use snapd universal-ctags | 10:01 |
| tpreston | zyga: I guess so, maybe I should alias ctags then | 10:02 |
| doko | how can I make how can I make file:///usr/share/doc/python3-doc/html/index.html work with the chromium snap? | 10:04 |
| zyga | doko: apart from serving it over http, I guess you cannot | 10:06 |
| zyga | doko: maybe copy it to ~ ? | 10:07 |
| doko | zyga: broken by design to access anything in /usr/share/doc ? | 10:09 |
| zyga | doko: not broken by design | 10:09 |
| zyga | doko: it's just not designed to access arbitrary host locations | 10:09 |
| zyga | doko: there are two elements at play, the view of the filesystem and the sandbox | 10:10 |
| doko | well, copying everything from /usr/share/doc to ~ is not viable | 10:10 |
| zyga | doko: chromium and your system see different filesystem | 10:10 |
| zyga | doko: and while /usr/share/doc is probably safe and could be allowed, it's not a general property | 10:10 |
| doko | is there a bug report about that? | 10:10 |
| zyga | doko: before you file one, do you have an idea of how it should work? | 10:11 |
| zyga | doko: I mean, I share the sentiment of it feeling broken | 10:11 |
| zyga | doko: but what could we do? | 10:11 |
| doko | well, for me it then not using the snap anymore | 10:13 |
| ogra | have a "host-documentation" interface that lallows reading /usr/share/doc content ? | 10:13 |
| zyga | doko: to partially answer my own question, at some point the document portal could be integrated into chromium, so that it could open a file | 10:13 |
| zyga | but IIRC the portal does not handle directories yet | 10:13 |
| zyga | doko: while I agee it's a regression I think using chromium to browse debian-style local documentation is fortunately niche | 10:14 |
| zyga | ogra: if you do that then the snap won't be able to see the /usr/share/doc from the base, I guess that's acceptable but rather obscure | 10:15 |
| zyga | ogra: but I think that's a good idea | 10:15 |
| ogra | yeah, i doubt the base will actually have content in /usr/shar/doc apart from copyright files | 10:17 |
| zyga | doko: I would strongly encourage you to file a bug or open a forum thread, I think ogra's idea is viable and well worth discussing | 10:17 |
| ogra | unless the building has changed massively (we used to remove everything there in the expectation that nobody would every want to use docs inside a base) | 10:17 |
| zyga | ogra: yes, I think we are lucky in a sense | 10:18 |
| zyga | if it was something other than that directory, it's probably not something that we could rely on | 10:18 |
| ogra | yeah | 10:18 |
| ogra | btw, using a portal would be horrible i fear, every link inside that index.html would pop up a new portal window | 10:20 |
| doko | zyga: where to file a bug? | 10:20 |
| zyga | ogra: I think there's a plan to open an entire directory as a portal | 10:21 |
| ogra | ah | 10:21 |
| zyga | doko: please file the bug on snapd on launchpad | 10:21 |
| doko | https://bugs.launchpad.net/snapd/+bug/1875860 | 10:24 |
| mup | Bug #1875860: local documentation is not accessible from the chromium snap <regression> <snapd:New> <https://launchpad.net/bugs/1875860> | 10:24 |
| zyga | ogra: I added a comment explaining your idea | 10:26 |
| zyga | doko: thank you! | 10:26 |
| ogra | :) | 10:26 |
| ackk | hi, are these error coming from snapd itself https://launchpadlibrarian.net/477374648/DpkgTerminalLog.txt ? | 11:04 |
| zyga | yes | 11:08 |
| zyga | ackk: udev in containers | 11:08 |
| zyga | ackk: it's a known topic that's on the backlog | 11:08 |
| ackk | zyga, what's the issue there? | 11:10 |
| zyga | ackk: snapd tries to reconfigure udev but udev is not meant to be used in containers (we pull it as a debian dependency) and it fails | 11:10 |
| ackk | zyga, for context, https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1875741 is the bug for that trace. basically a bionic->focal dist-upgrade with maas transitioning from deb to snap broke | 11:11 |
| mup | Bug #1875741: package maas 2.6.2-7841-ga10625be3-0ubuntu1~18.04.1 failed to install/upgrade: new maas package pre-installation script subprocess returned error exit status 1 <amd64> <apport-package> <focal> <third-party-packages> <uec-images> <maas (Ubuntu):New> <https://launchpad.net/bugs/1875741> | 11:11 |
| zyga | ackk: on subsequent runs snapd things everything is okay and no longer tries | 11:11 |
| ackk | zyga, so trying once again should fix it? | 11:11 |
| zyga | ackk: it's a topic we know about since Fra but we haven't managed to discuss at depth with Security | 11:11 |
| zyga | ackk: yes | 11:11 |
| zyga | ackk: there are some disagreements and we frankly didn't have time to sit down and discuss this | 11:11 |
| ackk | zyga, did anything change recently? I did test maas deb->snap transition in containers, never seen this issue | 11:12 |
| zyga | not that I know of | 11:12 |
| ackk | zyga, won't that break anyone dist-upgrading a container with snapd installed to focal? or is some kind of race that might happen? | 11:13 |
| zyga | ackk: it probably will | 11:14 |
| zyga | ackk: just a fire more distant than fires close up | 11:14 |
| sparkiegeek | zyga: got a bug number for it? | 11:14 |
| zyga | sparkiegeek: yes, let me find it | 11:14 |
| zyga | sparkiegeek: I tried addressing it before | 11:14 |
| zyga | one sec | 11:14 |
| ackk | zyga, sure, not implying anything, just trying to understand if there's something on the maas package side we could do, which it seems ther's not? | 11:15 |
| zyga | https://github.com/snapcore/snapd/pull/8219 | 11:17 |
| mup | PR #8219: interfaces: use udev backend if udev socket exists <Security-High> <Created by zyga> <Closed by zyga> <https://github.com/snapcore/snapd/pull/8219> | 11:17 |
| zyga | this is my idea and the bug is https://bugs.launchpad.net/snapd/+bug/1865503 | 11:17 |
| mup | Bug #1865503: snapd (deb) fails to install snapd (snap) inside LXD on a buildd based image <core20> <snapd:In Progress by zyga> <https://launchpad.net/bugs/1865503> | 11:17 |
| zyga | the discussion there is interesting as well | 11:17 |
| zyga | my idea is that we should not depend on udev, not pull it in for containers, not use any udev when there's no real udev or real /dev | 11:17 |
| zyga | sparkiegeek, ackk: if you have opinions I'd love to learn from you as well | 11:19 |
| sparkiegeek | zyga: no educated opinion here :| | 11:20 |
| ackk | zyga, same here... | 11:20 |
| mup | PR snapd#8578 opened: interfaces: add host-docs interface <Created by zyga> <https://github.com/snapcore/snapd/pull/8578> | 11:31 |
| mborzecki | pstolowski: does your PR now work on centos 8 with the fix in master? | 11:58 |
| pstolowski | mborzecki: which PR? | 12:19 |
| mborzecki | pstolowski: don't remember :P one that was blocked by selinux basically, or was there none? | 12:19 |
| pstolowski | mborzecki: master was blocked | 12:20 |
| mborzecki | ahh cool | 12:20 |
| zyga | doko: https://github.com/snapcore/snapd/pull/8578 | 12:22 |
| mup | PR #8578: interfaces: add host-docs interface <Created by zyga> <https://github.com/snapcore/snapd/pull/8578> | 12:22 |
| zyga | doko: it even has a test: https://github.com/snapcore/snapd/pull/8578/files#diff-aea859a8dd781e56f1c21b809b953d95R13 | 12:22 |
| ogra | oh, look someone found the mvo backdorr in ubuntu core !!! once you create an mvo account you can never delete it anymore !! https://forum.snapcraft.io/t/login-and-delete-user-created-with-v2-create-user/17019 | 12:31 |
| mborzecki | all devices belong to mvo ;) | 12:33 |
| ogra | yeah ! | 12:33 |
| mvo | ogra: haha | 12:33 |
| ogra | i really wonder what makes people use some random accounts found on the internet for such stuff :) | 12:34 |
| mvo | it's baffling! | 12:34 |
| zyga | mvo: lol | 12:37 |
| zyga | mvo: I wonder if the user found this is our tests or if there is some documentation example | 12:53 |
| mvo | probably | 12:54 |
| zyga | I could use a review for rather simple https://github.com/snapcore/snapd/pull/8565 | 13:48 |
| mup | PR #8565: osutil: expand FileLock to support shared locks and more <Created by zyga> <https://github.com/snapcore/snapd/pull/8565> | 13:48 |
| zyga | ok, i have my opensuse install under control now | 14:02 |
| pstolowski | +1 | 14:05 |
| zyga | thanks! | 14:05 |
| zyga | ijohnson: do you have a moment ^ it's pretty simple | 14:09 |
| zyga | ijohnson: it would unblock a host of other PRs | 14:09 |
| ijohnson | zyga: will look in a bit busy atm | 14:11 |
| zyga | sure | 14:11 |
| mborzecki | mvo: pstolowski: do you think we can land https://github.com/snapcore/snapd/pull/8536 to unblock the rest? with any tweaks in the followups (there's a bunch of PRs stack on top of this one) | 14:17 |
| mup | PR #8536: store,asserts,many: support the new action fetch-assertions <Bulk assert refresh :scroll::scroll::scroll:> <Created by pedronis> <https://github.com/snapcore/snapd/pull/8536> | 14:17 |
| mborzecki | hm pedronis isn't around | 14:17 |
| zyga | brb | 14:18 |
| zyga | tea | 14:18 |
| pstolowski | mborzecki: best to ask him actually... don't know if there is any risk | 14:19 |
| mup | PR snapd#8536 closed: store,asserts,many: support the new action fetch-assertions <Bulk assert refresh :scroll::scroll::scroll:> <Created by pedronis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8536> | 14:29 |
| mvo | mborzecki: I think it makes sense, done | 14:29 |
| mborzecki | thanks! | 14:29 |
| zyga | jdstrand: https://github.com/snapcore/snapd/pull/8578/files#diff-aea859a8dd781e56f1c21b809b953d95R13 | 14:29 |
| mup | PR #8578: interfaces: add host-docs interface <Created by zyga> <https://github.com/snapcore/snapd/pull/8578> | 14:29 |
| jdstrand | zyga: cool thanks. I have it on my list to review, but note I'm sprinting/etc | 14:30 |
| zyga | no rush :) | 14:31 |
| ijohnson | cmatsuoka: I hit the infinite loop of failed to transient mount unit for ubuntu-seed you had the other day | 14:57 |
| cmatsuoka | ijohnson: it's quite infrequent, but it's something we'll have to fix | 15:13 |
| ijohnson | cmatsuoka: did the issue go away when you rebooted ? | 15:13 |
| * ijohnson would like to debug but is afraid that when rebooting the issue goes away | 15:14 | |
| cmatsuoka | ijohnson: yes, when I booted the same image again it worked as expected | 15:14 |
| cmatsuoka | looks racy | 15:14 |
| ijohnson | mmm yeah | 15:14 |
| ijohnson | ah actually I can consistently reproduce this now on my image | 15:17 |
| * ijohnson debugs | 15:17 | |
| cmatsuoka | oh that's good, I couldn't reproduce it | 15:18 |
| * cachio lunch | 15:23 | |
| ijohnson | cmatsuoka: I video recorded the boot and caught this in the first few seconds of the log | 15:25 |
| ijohnson | https://usercontent.irccloud-cdn.com/file/84WtzfpC/loop%20uc20%20edge.png | 15:25 |
| ijohnson | the last line `systemd-fsck CP437 Invalid argument` is suspicious | 15:26 |
| cmatsuoka | oh interesting | 15:26 |
| cmatsuoka | yes, very suspicious | 15:26 |
| ijohnson | https://usercontent.irccloud-cdn.com/file/uYCPMgpF/loop%20uc20%20later.png | 15:27 |
| ijohnson | cmatsuoka: then this is also suspicious where run-mnt-ubuntu\x2dseed.mount fails because wrong fs type, bad option, etc. | 15:28 |
| cmatsuoka | ijohnson: maybe you could inspect the image file? (it could be a good idea to make a copy first in case you end up fixing the problem) | 15:29 |
| cmatsuoka | ijohnson: and check if the partition contents are there or the filesystem seems right | 15:30 |
| ijohnson | cmatsuoka: yes I am doing that now | 15:30 |
| ijohnson | the partition was mounted fine with kpartx + mount of the loopback device | 15:30 |
| cmatsuoka | does it look good? | 15:31 |
| ijohnson | yeah it has all the expected files and such | 15:31 |
| cmatsuoka | hmm | 15:31 |
| ijohnson | should I try to run fsck on the partition in the .img file ? | 15:31 |
| cmatsuoka | yeah, let's see what happens | 15:32 |
| ijohnson | huh fsck.etx4 says ubuntu-seed is actually a vfat partition? | 15:34 |
| ijohnson | https://www.irccloud.com/pastebin/C2v5Dpoz/ | 15:34 |
| cmatsuoka | mm, yes, seed is vfat | 15:35 |
| ijohnson | really? I thought seed was ext4 on amd64 ? | 15:35 |
| cmatsuoka | we must EFI boot it | 15:35 |
| ijohnson | ah okay so I was just wrong about that then | 15:36 |
| cmatsuoka | seed is vfat, the rest is ext4 | 15:36 |
| ijohnson | ok, fsck.vfat doesn't complain about the partition at all then | 15:36 |
| ijohnson | cmatsuoka: anything else I should check about the partition you think? | 15:36 |
| cmatsuoka | did it complain about code pages or something like that? | 15:37 |
| cmatsuoka | or maybe systemd-fsck does something different? | 15:37 |
| cmatsuoka | does it still fail if you boot the image again? | 15:38 |
| ijohnson | cmatsuoka: yes it reliably fails every time I boot now | 15:42 |
| ijohnson | cmatsuoka: all I got from fsck.vfat output was this: | 15:42 |
| ijohnson | https://www.irccloud.com/pastebin/nuIWXTi1/ | 15:43 |
| cmatsuoka | hmm | 15:43 |
| ogra | xnox, when you disabled the ability to disaable console-conf in ubuntu-image, did you make sure it still works for core18 builds ? | 15:44 |
| ogra | (someone just said it is completely gone, we have customers usign that option to build their images) | 15:44 |
| cmatsuoka | ijohnson: so the fs is good but for some reason it's not being mounted | 15:46 |
| ijohnson | yes | 15:47 |
| ijohnson | I can mount it fine though when the image is not booting | 15:47 |
| ijohnson | actually you know what, since this is still in install mode I can change the kernel cmdline since we never make it out of the initrd | 15:47 |
| ijohnson | so unsealing doesn't affect anything at this point | 15:47 |
| cmatsuoka | yes, you're right, let's inspect it while it's happening | 15:48 |
| ogra | xnox, ah, nevermind ... it still works but the --help info about it is gone | 15:48 |
| ijohnson | well damn now it boots fine | 15:49 |
| cmatsuoka | did you make a copy of the original image? | 15:50 |
| ijohnson | yes I did | 15:50 |
| ijohnson | let me make a copy of that original image and try again | 15:50 |
| ijohnson | ah now the copy of the original broken image works too :-/ | 15:51 |
| cmatsuoka | oh noes | 15:51 |
| ijohnson | so this definitely seems like a race | 15:52 |
| cmatsuoka | /o\ | 15:54 |
| ijohnson | well on the other hand, I notice that in the log of the failed boot that goes into this loop there's this message too: | 15:57 |
| ijohnson | [ 1.342368] FAT-fs (vda2): IO charset iso8859-1 not found | 15:57 |
| ijohnson | and that message doesn't appear in a good boot | 15:57 |
| ijohnson | so perhaps it's a race with some kernel module being loaded? | 15:57 |
| cmatsuoka | that sounds plausible | 15:58 |
| cmatsuoka | but then if it's desperately retrying, at some point it should work | 15:58 |
| cmatsuoka | ijohnson: https://askubuntu.com/questions/372445/fat-fs-io-charset-iso8859-1-not-found-error-while-mounting-fat-drives | 15:59 |
| ijohnson | cmatsuoka: what's so weird about this issue, is that it finishes the fsck check on /dev/vda2 in the loop case | 16:01 |
| ijohnson | ah but it wouldn't be loading the kernel module from that partition at this point, it would be loading it from the initrd in the kernel | 16:01 |
| ijohnson | so perhaps that kernel module in the initrd is corrupted somehow? | 16:02 |
| cmatsuoka | but then why it would work in the next boot? | 16:02 |
| ijohnson | well tbh I didn't quite make a perfect backup of the original image that was always reproducing, what I did was this: | 16:03 |
| ijohnson | 1. tried booting same img over and over again always reproducing infinite loop | 16:04 |
| ijohnson | 2. tried mounting the img file with kpartx and ran fsck.ext4 on it as explained | 16:04 |
| ijohnson | 3. unmounted with kpartx and made a backup copy | 16:04 |
| ijohnson | 4. re-mounted it and tried with fsck.vfat | 16:04 |
| ijohnson | 5. booted it and it didn't work first time | 16:04 |
| ijohnson | 6. rebooted it and changed kernel cmdline in grub menu, now it works every time | 16:04 |
| cmatsuoka | well, it still didn't boot in step 5 | 16:05 |
| ijohnson | yes but somehow changing the kernel cmdline made it boot fine | 16:06 |
| cmatsuoka | that's weird | 16:06 |
| ijohnson | I set the kernel cmdline to this: `console=tty1 console=ttyS0 rd.systemd.journald.forward_to_console=1 systemd.journald.forward_to_console=1 rd.systemd.debug_shell=1 dangerous panic=-1` | 16:06 |
| ijohnson | let me try re-creating this image fresh the way I did originally | 16:07 |
| cmatsuoka | I'll break for lunch and hopefully have a good idea while eating | 16:08 |
| cmatsuoka | because right now I don't have any | 16:08 |
| ijohnson | yeah just doing same thing again to build the image didn't reproduce it | 16:09 |
| * ijohnson runs it in a loop | 16:09 | |
| ijohnson | hey I reproduced it again! | 16:11 |
| * ijohnson saves that img | 16:11 | |
| zyga | you guys are having some fun :) | 16:14 |
| ijohnson | uc20 is too much fun | 16:14 |
| ijohnson | alright so every few time it hits that loop and fails, I see the IO charset iso8859-1 not found message, but when it does boot, I don't see that message, so it definitely is related | 16:17 |
| xnox | ogra_: i didn't change anything in ubuntu-image, and do not do any ubuntu-image development there. | 16:26 |
| xnox | ogra_: open an issue against ubuntu-image repository, with any bugs. | 16:26 |
| thresh | hi. I see my snapcraft push nightlies/vlc_*.snap --release beta, has succeeded in my CI, and I also see it under "Latest revisions for amd64" list. However it's not in the beta channel. | 16:27 |
| thresh | why would that happen, on the store? | 16:27 |
| zyga | thresh: this is more of a snapcraft question but perhaps snapcraft devs are around as well | 16:28 |
| thresh | thanks zyga, I'll complain in the related channel too :-) | 16:29 |
| zyga | thresh: I don't use snapcraft that often but I usually separately upload and release | 16:31 |
| zyga | cachio: are tumbleweed images updated periodically somehow or are we running a snapshot until something happens explicitly? | 16:47 |
| cachio | zyga, failed yesterday | 16:50 |
| zyga | sorry, I don't understand | 16:50 |
| cachio | let me try to fix it | 16:51 |
| zyga | cachio: can you please answer my question :-) | 16:51 |
| cachio | I mean | 16:51 |
| zyga | I' just want to understand what the process is | 16:51 |
| zyga | you can tell me what failed and about fixing it as well | 16:52 |
| zyga | but that's not what I was asking about | 16:52 |
| cachio | the process is: | 16:52 |
| cachio | the image is updated to a tamporal image | 16:52 |
| cachio | then the snapd tests are executed | 16:52 |
| cachio | if tests pass, then we turn that temporal image in a test image | 16:53 |
| cachio | otherwise the image is discarded | 16:53 |
| cachio | yesterday it failed running tests | 16:53 |
| cachio | failed preparing the the snapd teest suite | 16:54 |
| zyga | is there any notification when that happens? | 16:54 |
| cachio | no | 16:54 |
| zyga | that's not great, can we try go get some | 16:54 |
| cachio | it is in the brnaches board | 16:54 |
| cachio | ans it happens every monday | 16:54 |
| zyga | what is "branches board"? | 16:55 |
| cachio | https://travis-ci.org/github/snapcore/spread-cron/branches | 16:55 |
| cachio | https://travis-ci.org/github/snapcore/spread-cron/branches | 16:55 |
| cachio | zyga, this is the last log https://travis-ci.org/github/snapcore/spread-cron/builds/680087171#L2749 | 16:55 |
| cachio | sometimes I manually run the jobs to update the images | 16:56 |
| cachio | zyga, I see this Package 'python-docutils' not found. | 16:57 |
| zyga | I see, I think this package just got removed, it's a python 2 package | 16:58 |
| cachio | zyga, also 'pkg-config' not found in package names. | 16:59 |
| zyga | cachio: why are we installing python-docutil? | 17:00 |
| zyga | I don't see it in anything related to opensuse | 17:00 |
| zyga | cachio: yes, that's also been removed and replaced by something else | 17:00 |
| === ijohnson is now known as ijohnson|lunch | ||
| cachio | zyga, this is the snapd test suite which fails | 17:01 |
| zyga | cachio: python-docutils is only listed in arch | 17:01 |
| zyga | cachio: why are we installing python-docutils, do you know? | 17:01 |
| cachio | zyga, don't know why it is installed | 17:01 |
| cachio | I guess a test need it | 17:01 |
| ogra | xnox, oh, sorry, GH tricked me ... you were just a reviewer on https://github.com/CanonicalLtd/ubuntu-image/pull/186 | 17:01 |
| mup | PR CanonicalLtd/ubuntu-image#186: Some options are unsupported for UC20 builds <Created by sil2100> <Merged by sil2100> <https://github.com/CanonicalLtd/ubuntu-image/pull/186> | 17:01 |
| zyga | cachio: do you know which part of the code is installing it? I certainly don't see it in pkgdb.sh | 17:01 |
| zyga | I see it in opensuse dependencies but that's not managed by pkgdb.sh | 17:02 |
| zyga | cachio: ok, I see the problem now | 17:04 |
| cachio | packaging/opensuse-15.0/snapd.spec: | 17:04 |
| zyga | cachio: I think for sid, tumbleweed and arch we should not stop | 17:04 |
| zyga | cachio: if we stop and use some ancient version because tests don't pass | 17:04 |
| zyga | that's pretty much because snapd is in a way broken | 17:04 |
| zyga | the world has moved on | 17:04 |
| zyga | staying on an old image without alerting the team could be a problem later | 17:05 |
| zyga | in a sense those systems are all unstable because they never stop | 17:05 |
| cachio | zyga, for arch is not a problem because we update in every run the image | 17:05 |
| zyga | cachio: regardless of how we update | 17:05 |
| cachio | zyga, but I could remove snapd tests validation for tumbleweed and sid | 17:06 |
| cachio | it is very easy | 17:06 |
| zyga | cachio: yeah, I think we should do that | 17:06 |
| zyga | cachio: we can have them in unstable systems | 17:06 |
| cachio | zyga, yes | 17:06 |
| zyga | cachio: in reality, the package is gone from the archive | 17:06 |
| cachio | totally agree | 17:06 |
| zyga | cachio: not updating a stale image is not buying us anything | 17:06 |
| zyga | cachio: the package cannot be built | 17:06 |
| zyga | cachio: CI is not functioning as CI :) | 17:06 |
| zyga | cachio: cool, I'll send a patch to fix our packaging | 17:07 |
| cachio | zyga, nice, thanks! | 17:07 |
| zyga | cachio: but please let's figure out how to make some notifications flash on everyone's systems | 17:07 |
| zyga | cachio: so that next time we know earlier | 17:07 |
| cachio | zyga, sure, I'll configure to send emails | 17:07 |
| zyga | super, thank you :) | 17:07 |
| cachio | zyga, yaw | 17:07 |
| zyga | cachio: https://github.com/snapcore/snapd/pull/8579 | 17:17 |
| mup | PR #8579: packaging: stop depending on python-docutils <Created by zyga> <https://github.com/snapcore/snapd/pull/8579> | 17:17 |
| mup | PR snapd#8579 opened: packaging: stop depending on python-docutils <Created by zyga> <https://github.com/snapcore/snapd/pull/8579> | 17:17 |
| * zyga goes to make coffee ahead of the meeting | 17:22 | |
| cachio | zyga, thanks | 17:28 |
| cachio | lets test it whith the new image once it is ready | 17:28 |
| zyga | sure | 17:31 |
| kyrofa | If someone runs `snap revert`, will it run the refresh hooks? | 17:49 |
| === ijohnson|lunch is now known as ijohnson | ||
| ijohnson | kyrofa: afaik it won't | 17:49 |
| kyrofa | ijohnson, any hooks? Any way the snap to which is being reverted can know that a revert happened? | 17:50 |
| ijohnson | no, I think that's by design that the snap doesn't know it got reverted | 17:50 |
| kyrofa | Fair enough, thanks ijohnson | 17:51 |
| ijohnson | cmatsuoka: ah I think I understand the problem now | 17:53 |
| ijohnson | cmatsuoka: what's happening is this sequence of events: | 17:53 |
| ijohnson | 1. there is a systemd unit for running fsck on /dev/disk/by-label/ubuntu-seed that just always runs, so it happens to run first | 17:54 |
| ijohnson | 2. snap-bootstrap via "the-tool" runs, says to mount /run/mnt/ubuntu-seed _with transient unit_ (that's important) | 17:55 |
| ijohnson | 3. that fails because systemd-modules-load has not run, or finished running yet | 17:55 |
| ijohnson | 4. so we see IO charset iso8859-1 not found from the kernel because nls_iso8859_1 was not loaded | 17:55 |
| ijohnson | 5. systemd-modules-load eventually runs/finishes running and does load nls_iso8859_1: systemd-modules-load[238]: Inserted module 'nls_iso8859_1' | 17:56 |
| ijohnson | 6. the-tool keeps running snap-bootstrap, which says to mount /run/mnt/ubuntu-seed, however because of how the-tool tries to do the mount, it just requests a transient unit for the mount | 17:56 |
| ijohnson | 7. the transient unit was already started/created by systemd so systemd doesn't try again and just complains thusly: the-tool[380]: Failed to start transient mount unit: Unit run-mnt-ubuntu\x2dseed.mount already exists. | 17:57 |
| ijohnson | 8. thus we get stuck in an infinite loop where the-tool never exits because the-tool doesn't run systemd-mount with set -e, it explicitly runs systemd-mount with set +e first, then turns set -e back on after running systemd-mount | 17:58 |
| cmatsuoka | that makes perfect sense | 17:58 |
| ijohnson | cmatsuoka: so I have a few thoughts on how to fix this, 1) make the-tool depend on systemd-modules-load, 2) make the-tool eventually give up (because since it never gives up systemd never allows a debug shell to be started either) | 17:59 |
| ijohnson | 3) don't have a unit which runs fsck always on ubuntu-seed, instead have the-tool dynamically do that when snap-bootstrap requests something to be mounted | 17:59 |
| ijohnson | wdyt? | 17:59 |
| cmatsuoka | 2 looks like a good thing to have in case we run into problems again | 18:01 |
| cmatsuoka | 1 + 2 maybe? | 18:02 |
| ijohnson | yes actually I requested a very similar thing when mborzecki's original PR to ubuntu-core-initramfs was proposed, but I confused myself and said it wasn't a problem haha | 18:02 |
| ijohnson | (for 2 that is) | 18:02 |
| mborzecki | hahah | 18:02 |
| cmatsuoka | well I'm very happy now that this mystery is solved | 18:03 |
| mborzecki | it does make sense, i mean there's not gooing to me more than some limited number of mounts | 18:03 |
| cachio | zyga, hey, I also see this error https://paste.ubuntu.com/p/gcw9dTrRGm/ | 18:04 |
| ijohnson | alright I will propose a PR to ubuntu-core-initramfs to fix this, but I was going to file a LP bug first to summarize all this, but seems LP just died, so I will have to do that later | 18:07 |
| * ijohnson goes back to console-conf on core20 hacking | 18:07 | |
| zyga | ack | 18:17 |
| zyga | cachio: please release the image anyway, I will send another change to address that | 18:19 |
| cachio | zyga, ok | 18:19 |
| mup | PR snapd#8579 closed: packaging: stop depending on python-docutils <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/8579> | 18:26 |
| cachio | mvo, hi, any idea about this ppa ? https://paste.ubuntu.com/p/ttF5CHt56X/ | 18:35 |
| ijohnson | cachio: launchpad is/was down for a bit | 18:41 |
| cachio | ijohnson, ah, thanks | 18:42 |
| mvo | cachio: looks like a LP issue | 18:44 |
| cachio | mvo, yes | 18:46 |
| cachio | thanks | 18:46 |
| mup | PR snapd#8580 opened: data/completion: add `snap` command completion for zsh <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/8580> | 19:12 |
| mborzecki | any zsh users here? | 19:13 |
| mup | PR snapcraft#3095 closed: plugins: break out rosdep resolve parsing for external use <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/3095> | 19:34 |
| mup | PR snapcraft#3098 opened: build providers: bootstrap with dirmngr <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/3098> | 19:34 |
| cmatsuoka | ijohnson: just hit the problem again here, and in the next boot it's gone | 19:45 |
| ijohnson | cmatsuoka: yes I'm not sure what changed, but something seems to make it more likely to happen recently as I never saw it before yesterday | 20:05 |
| cmatsuoka | and it seems that I'm having some dns issues with the store | 20:06 |
| ijohnson | there was a big outage with launchpad a while ago that maybe propagated to the store? I dunno store seems a little slow but otherwise operational on my end | 20:10 |
| cmatsuoka | right now I can't resolve snapcraft.io | 20:12 |
| Lukewh | seems ok here cmatsuoka where are you based? | 20:13 |
| cmatsuoka | I'm in brazil | 20:13 |
| cmatsuoka | oh, and now it's back | 20:13 |
| cmatsuoka | a few days ago there were some routing problems from brazil to the rest of the world so it can be something weird on my side again | 20:15 |
| mup | PR snapcraft#3099 opened: repo: always refresh cache when build packages are required <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/3099> | 20:16 |
| cmatsuoka | oh, it seems that the ipv4 addresses are resolving correctly but AAAA is not for some reason | 20:17 |
| cmatsuoka | anyway, let's just use ipv4 | 20:18 |
| zyga | I will fix the damn pulseaudio test tomorrow | 20:22 |
| zyga | now that other stuff is not red anymore | 20:22 |
| mup | PR snapd#8565 closed: osutil: expand FileLock to support shared locks and more <Created by zyga> <Merged by anonymouse64> <https://github.com/snapcore/snapd/pull/8565> | 20:23 |
| zyga | \o/ | 20:24 |
| zyga | thank you Ian :) | 20:24 |
| cjwatson | ijohnson: it was a Canonical network issue, not specifically LP, so there'll have been various bits of fallout | 20:27 |
| ijohnson | yaw zyga | 20:27 |
| ijohnson | cjwatson: yeah that's kinda what I expected | 20:28 |
| cjwatson | (GS2 I think) | 20:28 |
| zyga | ijohnson: https://github.com/snapcore/snapd/pull/8566/files is next on the r-a-a pipe | 20:28 |
| zyga | but it has probably-poor names | 20:28 |
| mup | PR #8566: cmd/cmdutil: add run inhibition operations <Created by zyga> <https://github.com/snapcore/snapd/pull/8566> | 20:28 |
| zyga | I was split between lock naming | 20:28 |
| zyga | and totally not lock naming | 20:29 |
| ijohnson | mmm zyga does that need samuele review for names? | 20:29 |
| zyga | ijohnson: for sure but we can do the technical review | 20:29 |
| ijohnson | ack will try to get to it today, but probably that one will have to wait for tomorrow | 20:29 |
| zyga | ijohnson: or maybe better this: | 20:30 |
| zyga | https://github.com/snapcore/snapd/pull/8571 | 20:30 |
| zyga | this is self-sufficient | 20:30 |
| mup | PR #8571: overlord/snapstate: warn of refresh/postpone events <Created by zyga> <https://github.com/snapcore/snapd/pull/8571> | 20:30 |
| zyga | and useful today | 20:30 |
| zyga | and actually tiny apart from tests :) | 20:30 |
| zyga | just two print-like calls | 20:30 |
| zyga | but I should EOD | 20:31 |
| zyga | or I'll start sending more things | 20:31 |
| * zyga waves | 20:32 | |
| ijohnson | haha | 20:32 |
| ijohnson | have a good night zyga | 20:32 |
| zyga | one last thing: https://github.com/snapcore/snapd/pull/8572#event-3283181589 | 20:32 |
| zyga | I like this line | 20:32 |
| mup | PR #8572: packaging: add the inhibit directory <Created by zyga> <https://github.com/snapcore/snapd/pull/8572> | 20:32 |
| zyga | "via automation" | 20:32 |
| zyga | meh | 20:43 |
| zyga | I fixed the stupid test | 20:43 |
| zyga | ijohnson: https://github.com/snapcore/snapd/pull/8581 draft for now | 20:58 |
| mup | PR #8581: tests: port pulseaudio test to session-tool <Created by zyga> <https://github.com/snapcore/snapd/pull/8581> | 20:58 |
| mup | PR snapd#8581 opened: tests: port pulseaudio test to session-tool <Created by zyga> <https://github.com/snapcore/snapd/pull/8581> | 20:59 |
| zyga | so, session-tool on 16.04 - it needs to start doing upstart things, I guess | 21:09 |
| ijohnson | sad | 21:11 |
| ijohnson | that's unfortunate | 21:11 |
| * ijohnson EODs early | 21:12 | |
| mup | Issue core20#48 closed: python3 from base causes segfault for confined snaps on armhf <question> <Created by sergiusens> <Closed by xnox> <https://github.com/snapcore/core20/issue/48> | 22:35 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
