[05:37] good morning === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk [06:34] morning [06:34] starting a bit later today, had to take my wife's car to the shop [06:37] PR core18#31 opened: hooks: move 003-writable-paths.chroot to static/etc/system-image === chihchun_afk is now known as chihchun [06:43] hey mborzecki - good morning [06:44] mvo: hey [06:44] any fires today? [06:44] PR core18#32 opened: writable-paths: add needed paths for cloud-init [06:46] mborzecki: no fires, I'm a little bit frustrated with core18 in gce, spread tests does not work there yet but they work fine in qemu and I'm scratching my head why [06:46] mborzecki: but no fires afaict [06:47] mvo: hmm that's interesting, anything in particular that fails int he tests? or does it not boot at all? [06:47] mborzecki: it boots and cachio pasted me the console output which looks fine (for some reason I don't have enough access in the gce console to see it myself). so I think it does not get network [06:48] mvo: do the new images use netplan or sth? [06:48] mborzecki: yeah, it is, it *should* be the same config as the core16 [06:48] mvo: interface naming then? [06:48] mborzecki: my current straw is that cloud-init is not working yet on core18 and maybe it is doing some magic here [06:49] mborzecki: its a pretty generic config, one sec, let me find it [06:49] mborzecki: https://github.com/snapcore/core18/blob/master/static/etc/netplan/00-snapd-config.yaml [06:50] mvo: looks pretty straightforward [06:50] mborzecki: yeah, I need to learn if I can get a OOB shell with gce somehow [06:50] mvo: there is a dhcp client in the image? [06:50] mborzecki: systemd-networkd [06:51] mborzecki: I don't think there another [06:51] mvo: ok, that should be fine, networkd does it's own dhcp [06:52] mborzecki: yeah [06:52] mvo: hm you could try and ship your own config file for networkd in case netplan doesn't do it's thing [06:53] mborzecki: thats an interessting idea [06:59] heh, got another update to shellcheck, now it complains about not quoting variables when sourcing files, iirc this wasn't strictly needed [06:59] mborzecki: heh [06:59] mborzecki: nice find on your latest shellcheck PR in the framebuffer test [07:00] mvo: yeah, looking into it right now [07:00] mborzecki: shell is just not the best language, really good we run the checker now === chihchun is now known as chihchun_afk [07:00] mborzecki: I wish there was something like "-o strict" or somesuch that would at least protect against all the basic mistakes. anyway [07:01] mvo: shperl [07:02] mborzecki: heh === pstolowski|afk is now known as pstolowski [07:06] mborzecki: btw, gustavo added some comments to github.com/go-check/check https://github.com/go-check/check/pull/100 [07:06] PR go-check/check#100: many: return kr/pretty:Diff when Equals/DeepEquals check fails [07:07] mborzecki: the first question about "if result { clear error message" is from your diff, this is a test failing, right? I wonder if we should test this more explicitly, wdyt? [07:07] mvo: ah interesting, i can look into this later [07:08] mborzecki: thank you, I will look at his other points [07:08] morning [07:09] hey pstolowski === chihchun_afk is now known as chihchun [07:11] mvo: ok, i think i remember now, there may not be an explicit test for this, but the clearing was added there to prevent the Not() checker from raising false positives [07:12] mvo: a vague recollection, that if you did say, Not(DeepEqual()) and the Not checker left the error message intact it would still come up as an error in the tests [07:13] zyga: how's your SLED work? [07:13] Stuck [07:13] Look at that twitter post [07:16] zyga: boo :( [07:18] zyga: slightly surprised those are still behind paywall [07:22] well, those are like RHEL [07:22] but... I have a license [07:22] something is wrong on the website [07:23] https://forum.snapcraft.io/t/nvidia-cuda-on-ubuntu-core/292/24 heh, nice pattern from nvdiia [07:24] ioctl(12, _IOC(_IOC_READ|_IOC_WRITE, 0x41, 0x2, 0x18), 0xbeb00fc0) = 0 this does some magic [07:24] and works outside of snap [07:24] but inside it fails, the driver apparently trues to do another little poke ioctl(12, _IOC(_IOC_READ|_IOC_WRITE, 0x41, 0x3, 0x10), 0xbee73f28) = -1 EINVAL [07:25] in either case, the device accessed is open("/dev/nvhost-as-gpu", O_RDWR|O_DSYNC) = 12 [07:27] PR core18#31 closed: hooks: move 003-writable-paths.chroot to static/etc/system-image [07:47] I'm looking into opensuse releases [07:47] something is not working well on LEAP [07:47] and this is blocking a high-priority snap apparently === chihchun is now known as chihchun_afk === chihchun_afk is now known as chihchun [08:15] zyga: when a device is not in device group, then will open() fail? or would other operations, eg. ioctl() fail but open() will succeed? [08:15] s/group/cgroup/ [08:15] I believe open would fail, yes [08:16] hmm, i'm thinking about that nvidia on ubuntu core on TK1 problem, the snap is accessing some special devices there, eg. /dev/nvhost-as-gpu and ioctl() fails, but open() works [08:17] zyga: we don't seem to add any of those devices, but it's likely that the board is runnign some old(ish) kernel, maybe without device cgroup even [08:38] mborzecki: we need to further differentiate leap and tumbleweed [08:38] :/ [08:38] 42.3 cannot use any apparmor [08:38] leap 15 and tumbleweed can [08:38] zyga: what's missing there? [08:39] apparmor userspace is at 2.10 [08:39] that's far too old [08:39] it cannot even parse our snap-confine profile [08:40] well, we don't have much choice then [08:41] I will check leap 15 to be sure [08:41] and then patch this in the distro [08:41] and then send the patches to master [09:06] hm the framebuffer test was probably not executing on ubuntu-* because there's no /dev/fb* node [09:07] then on others, it was just an echo due to missing " [09:07] yeah [09:07] and looks like write to a frame buffer is hitting a region outside of available memory, hence 'file too large' error [09:17] I'm building 2.33 opensuse packages here https://build.opensuse.org/package/show/home:zyga:branches:system:snappy/snapd [09:17] I will be patching that as I get builds and test them on my machines [09:18] but now I need to go and pick up a package from downtown, I will commit my apparmor work and continue on the way there [09:37] hmm [09:37] tests fail on opensuse [09:37] failed test in the state somewhere https://www.irccloud.com/pastebin/Aeu5glwF/ [09:37] fun fact, ubuntu-{16.04,18.04,-core-16} do not have /dev/fb so the apparmor rules are not really tested anywhere [09:39] :D [09:39] fun fact, fun facts aren't usually fun [09:39] barrels of fun ;) [09:40] Son_Goku: fun fact: ketchup was once medicine [09:40] Chipaca, really? [09:41] well, sold as such [09:41] that's actually an interesting fact, if that's true [09:41] (but then so was cocaine) [09:41] well, _that_ I knew [09:41] Chipaca: so was mercury [09:42] also fun fact: tin (can) openers were invented 48 years after food was being canned [09:42] in the interim people just stockpiled the stuff* [09:42] * may not be true [09:42] Chipaca: hi, I'm working on exposing publisher, am I confused? we don't seem to have a test that checks the details of unmarshaling of search results [09:43] (the last thing using details.go) [09:43] pedronis: it's probably covered indirectly, but probably no unit tests [09:44] another fun fact, our 16.04 image has minix module loaded even before the test using that module starts [09:44] wut [09:44] Chipaca: I mean store_test.go (I know we don't have details_test.go), I don't see the indirect bit either but maybe I'm confused [09:44] mvo, do you know when 2.33.1 is going to be tagged? [09:44] pedronis: 1 sec [09:45] along with xfs, ntfs, hfs, qnx4 and so on :? [09:45] qnx4?! [09:45] mborzecki: something is trying really hard to mount a filesystem [09:45] maybe the gce initrd? [09:46] Chipaca: no clue [09:46] well, fwiw the rmmod-or-not part of the test did not work before shellchecks and it would always rmmod it [09:48] pedronis: it seems you're correct, there are individual tests for specific aspects of details, but nothing whole hog [09:48] Chipaca: I suppose I should add something, maybe its own PR tough [09:48] kjackal: that's a lot of yous [09:49] pedronis: ok [09:49] I should go do some work on my back an' stuff [09:49] * Chipaca very comfy tho [09:50] Chipaca: probably your info branch removed a details one that can reuse [09:50] *that I can reuse in some form [09:50] pedronis: I like your optimism :-) [09:50] heh, I do think we had some tests (but with /details apparently) [09:51] * pedronis is hopeful [09:52] Son_Goku: last fun fact for a while: colgate beef lasagne didn't sell well [09:52] * Chipaca ~> physio [09:52] I'm going to go with "eww" for that [09:53] a solid choice [09:53] * Chipaca ~> really afk now [10:16] zyga, sil2100: a quick look at #32 (in core18) would be great [10:17] Ack [10:17] mvo: on it now [10:18] sil2100: thank you [10:20] Ah, already saw it in the morning, didn't manage to +1 it then [10:21] PR core18#32 closed: writable-paths: add needed paths for cloud-init [10:21] * mvo hugs sil2100 [10:38] zyga: are new interface PRs waiting on 2.33 going out before landing? just checking not waiting for anything from me [10:38] * zyga is in the underground [10:39] They are waiting for jdstrand [10:39] He is off this week [10:40] I think 33 is branched already so there is nothing blocking master [10:50] zyga: the PR has your and Jamie's +1s but the tests have not yet run successfully, looks like there was provisioning error [10:56] I’ll look into it soon [10:56] Returning home now [10:58] zyga: do you remember if firmware loading is done directly in the kernel now or still through a udev helper? [11:00] PR snapcraft#2160 closed: many: refactor snapcraft.yaml loading out of load_config [11:07] zyga: pinged you in the forums, there's an interesting case regarding firmware loading and interaction with mount namespace [11:10] mborzecki: replied there [11:11] mborzecki: as for firmware loading, it AFAIK changed from userspace to kernel but I could be wrong and it could be backwards [11:11] mborzecki: given that it fails I suspect it's userspace and the firmware cannot be found [11:11] idk, haven't touched that really since 2.6.x days [11:11] mborzecki: also on core vs classic? [11:11] zyga: the topic is about core [11:11] mborzecki: on core, unless you use bases, you are using the same filesystem as before [11:12] there's a mount namespace but it is largely the same [11:12] zyga: confinement-options: classic devmode [11:13] that's ... weird [11:13] classic devmode is meaningless [11:13] does it work outside of a snap [11:13] ? [11:13] zyga: the thing works outside of snap [11:13] strace it [11:13] zyga: and if you start the app outside of snap, then it will work inside too [11:13] zyga: there's starce logs attached 2-3 messages up [11:14] ah, let me see then [11:16] mborzecki: well [11:16] confinement options is what is *supporteD* [11:17] not what is used [11:17] mborzecki: so this could be a strict snap [11:17] but I see there's no apparmor at all [11:17] zyga: ther's no aa [11:17] so it's somewhat meaningless [11:18] some funky errors at the end of snap_directly_log.txt https://www.irccloud.com/pastebin/qOLUGDob/ [11:18] note that it closes -1 [11:18] probably missing error checking [11:20] mborzecki: is there /dev/nvidiactl inside the cgroup? [11:20] (device cgroup) [11:21] zyga: yes, we add it, we're adding nvidiactl, nvidia-uvm, nvidia-modeset and then some nvidia devices [11:22] 1811 open("/dev/shm/cuda_injection_path_shm", O_RDWR|O_NOFOLLOW|O_CLOEXEC) = -1 ENOENT (No such file or directory) [11:22] this looks related [11:23] is it there on the outside? [11:23] zyga: don't know, i don't have access to this host [11:23] zyga: it's also funny taht if you run the app outside once, it will work inside the snap too [11:23] until reboot, right? [11:24] please ask for a strace -f, after reboot, of the app running _outside_ === pstolowski is now known as pstolowski|lunch [11:24] then for a similar strace but on the inside, also after reboot (where it will fail) [11:24] if we have that we can investigate [11:25] * zyga is about 10-20 minutes away from home [11:25] cannot wait to get back really [11:25] it's so damn hot today [11:25] my son's computer broke and I had to get a part that was faulty [11:26] he needs it for classes today [11:26] niemeyer: hey, good morning. could you give me permission so that I can see "jun191112-575930" in GCE? I would love to see the terminal and ideally login if possible (there is a problem with getting the network working right now on core18 so an oob login would be great to figure out what is going on) [11:26] mvo: ouch, still fighting that? [11:26] mvo: did you look at diffing /etc between core18 and core? [11:26] zyga: yeah [11:26] zyga: btw. notice that there's a log in dmesg that getting firmware failed, given that it's kepler, iirc those already required a blob to work [11:26] it may be something that was added to current system [11:26] zyga: I poked around a bit, added some more cloud-init bits [11:27] mvo: and something in /etc/modules.d or modprobe.d? [11:27] mborzecki: it would be good to ask some nvidia people how this is supposed to work [11:28] it's a bit silly that when you want to use the hardware then each app actually goes to load firmware into the hardware [11:28] zyga: i bet many people in open source had the same thought :) [11:28] zyga: i think it's the first use [11:28] zyga: not afaict [11:28] m [11:28] zyga: it might be something silly [11:28] zyga: so you try to do something with the hardware for the first time and the driver goes to load the firmware [11:28] zyga: but without access to the machine its hard to diagnose :/ [11:28] yeah [11:29] * mvo gets lunch instead [11:31] mvo: netplan has [11:31] mvo: netplan has two patterns, one matches en* and the other matches eth* [11:31] maybe on a newer core you get veth or something silly like this [11:31] becuase newer systemd [11:31] and then stuff falls apart [11:31] suggestion [11:32] as a hack change /etc/netplan/00-snapd-config.yaml [11:32] and mach on * [11:32] fingers crossed [11:33] * zyga walks home, cannot wait to have A/C next season [11:36] zyga: there's 19C in oslo right now, ever considered moving? [11:37] oslo with 19C must be melting [11:38] Hahah [11:38] Well, I’m not *that* crazy [11:38] zyga: [citation needed] [11:44] pedronis: shall we land #5314 once it's green? [11:44] PR #5314: many: rename snap.Info.Name() to snap.Info.InstanceName(), leave parallel-install TODOs [11:46] PR snapd#5334 closed: tests: show executed tests on current system when a test fails [11:47] Chipaca: hehe :) [11:47] * zyga is home now [11:47] geez, outside is not fun [11:48] mborzecki: you should ask mvo if there is something high prio from him that could conflict [11:48] mvo: is there? [11:49] i can help resolving conflicts if needed [11:49] PR snapd#5344 opened: store: have a basic test about the unmarshalling of /search results [11:49] Chipaca: ^ , not beautiful but better than nothing [11:50] will look in a bit [11:50] currently reviewing shellcheck [11:52] Chipaca: thank you! [11:58] mvo: any luck with that etherent change? [12:04] mborzecki: don't thank me until the fat lady sings [12:04] I ask Questions [12:07] Chipaca: right, i might as well cleanup the \- now that i'm doing the changes anyway [12:08] Chipaca: looks like it was copy pasted pretty much everywhere :P [12:08] mborzecki: I understand it's easier to think of it as a pure shellchecking change, but yeah [12:08] I'd have to read the docs on too many things to find out if that \- actually matches what we want it to match [12:09] mborzecki: there's also a bunch of the PWD ones [12:19] mvo: Heya, yeah, can definitely do it [12:20] zyga, hey [12:20] hey [12:20] interfaces content if failing sporadically https://api.travis-ci.org/v3/job/393533314/log.txt [12:20] checking [12:20] did you see that one? [12:21] cachio: yes, [12:21] it's not related to content really [12:21] it's related to mount exploiding [12:21] - Mount snap "test-snapd-content-slot" (2) ([start var-lib-snapd-snap-test\x2dsnapd\x2dcontent\x2dslot-2.mount] failed with exit status 1: Job for var-lib-snapd-snap-test\x2dsnapd\x2dcontent\x2dslot-2.mount failed. [12:21] See "systemctl status "var-lib-snapd-snap-test\\x2dsnapd\\x2dcontent\\x2dslot-2.mount"" and "journalctl -xe" for details. [12:22] Jun 18 09:29:29 arch snapd[12112]: Jun 18 09:29:28 arch kernel: print_req_error: I/O error, dev loop3, sector 0 [12:22] there's a thread about it [12:22] zyga, ah, ok, good to know [12:22] tx [12:22] https://forum.snapcraft.io/t/unexplained-mount-failure-protocol-error-what-we-know-so-far/5682 [12:23] mborzecki: sorry, I just merged something that probably needs a remerge for your PR [12:23] PR snapd#5344 closed: store: have a basic test about the unmarshalling of /search results [12:24] Chipaca: I got a +1 from zyga, happy to tweak in a follow up though if you find something that merits changes [12:34] pedronis: LGTM [12:34] pedronis: also thank you for updating the json [12:34] Chipaca: which reminds me, we can remove MockDetailsJSON [12:34] but something of another PR [12:35] s/of/for/ [12:36] pedronis: there's a lot of unused stuff, according to 'unused' [12:37] pedronis: I can propose a removal pr in a bit :) [12:37] thx === pstolowski|lunch is now known as pstolowski [12:40] Hello. I submitted two pull requests last August, and got 0 feedback. Could someone please have a look at them? [12:40] https://github.com/snapcore/snapd-xdg-open/pulls?q=author%3ACyberShadow [12:40] CyberShadow: hello [12:41] I think that repository is dead now, let me see [12:41] we should archive it [12:41] we re-wrote that code significantly since [12:41] I found the software useful for use outside of snaps. How do you solve that problem now? [12:42] let me show you [12:43] https://github.com/snapcore/snapd/tree/master/userd [12:43] https://github.com/snapcore/snapd/blob/master/userd/launcher.go this is what the old code did [12:43] PR snapd#5345 opened: store: drop unused: channel map types, and details fixture [12:44] pedronis: ^ +0 -115 :) [12:44] CyberShadow: this runs on the outside and sits on the bus [12:44] CyberShadow: the one that runs on the inside ... [12:45] CyberShadow: is here https://github.com/snapcore/snapd/blob/master/cmd/snapctl/main.go#L50 [12:46] and more completely here: https://github.com/snapcore/snapd/blob/master/xdgopenproxy/xdgopenproxy.go [12:46] cachio: is there a way to log into a gce instance when there is no ssh/network ? some magic via serial console etc? [12:46] I see, thanks! So you consolidated everything into one daemon rewritten in Go. [12:47] CyberShadow: one codebase, there's more than one process [12:47] I think I'll maintain my fork of the old snapd-xdg-open then, it's still useful with Firejail/Bubblewrap. [12:47] CyberShadow: I will close your PRs now, sorry [12:47] Right, that hasn't changed. [12:47] No problem. Consider adding a note that the repository is EOL. [12:47] mvo, you should to send all the info to the console, then we can check that [12:48] mvo, let me research a bit about other ways to do it [12:48] zyga: github allows repos to be archived now, no clue though who can mark that one as such [12:48] mborzecki: yeah, I know [12:48] mborzecki: I will ask during the standup [12:53] CyberShadow: if the two bits were still go but separate, would you contribute to them? [12:53] CyberShadow: or is the hacky aspect of the previous incarnation part of their appeal :) [12:54] kyrofa: https://forum.snapcraft.io/t/update-all-python-snaps-not-working-with-classic-confinement-even-with-cleanbuild/5971 [12:55] CyberShadow: I ask because the new things support stuff the old ones do not, like apps autostart [12:55] so there might be value in them for you as well [13:02] Chipaca: Sure, language doesn't matter [13:02] Sorry, I'm not sure how autostart fits in with the Firejail/Bubblewrap flow... [13:03] CyberShadow: I don't know either :) [13:04] CyberShadow: how would an app go around setting up autostart in that world? [13:04] It doesn't, it's something that would break encapsulation :) [13:05] The user would need to do that manually. They would need to create the container manually too, so it's not unexpected. [13:11] CyberShadow: I'll raise separating it in the standup [13:11] CyberShadow: is it the inside-the-container thing that you use, or the outside one, or both? [13:12] Both [13:12] k [13:12] Here is the user-facing documentation: https://wiki.archlinux.org/index.php/Bubblewrap#Opening_URLs_from_wrapped_applications [13:13] Just to clarify, I think everyone involved would be quite happy with maintaining a fork, so don't feel obligated to support this use case :) [13:14] PR snapd#5345 closed: store: drop unused: channel map types, and details fixture [13:14] CyberShadow: yep yep [13:14] CyberShadow: the old one was attractively simple so I get that === chihchun is now known as chihchun_afk [13:31] CyberShadow, I suggest renaming the application, but yeah, go for it [13:32] Good idea :) [13:32] CyberShadow, talk to the guys in #atomic about hosting the program under their banner [13:32] that way it sits adjacent to bubblewrap ;) [13:39] True, though it looks like they have their own higher-level snapd equivalent to use with Flatpak (which uses Bubblewrap): https://github.com/flatpak/xdg-desktop-portal [13:40] CyberShadow, well, that's why I suggested projectatomic and not flatpak org ;) [13:40] Ah, right [13:47] CyberShadow: the conclusion in the standup is that most of the new features are very snap-specific in their approach, and maybe it's not that useful outside of this [13:47] CyberShadow: so fork away, with our blessing [13:47] Thanks! [14:00] mvo: I'll be a couple of minutes late [14:01] ah, the core18 meeting is still in 29 minutes [14:01] I thought it was back to back [14:02] ah, same, was confused [14:21] uh [14:21] I ran out of disk space [14:21] sigh [14:21] * zyga moves stuff around [14:24] * ogra_ hands zyga a paperbasket [14:24] * zyga throws lumps of lead and cement inside [14:24] *thump* [14:24] look, the bucket is now a pipe [14:30] niemeyer: the compute.instances.reset permission for image debug would be great [14:34] downloading chrome [14:39] mvo: will landing #5314 conflict with any stuff that you have open atm? [14:39] PR #5314: many: rename snap.Info.Name() to snap.Info.InstanceName(), leave parallel-install TODOs [15:06] mborzecki: its fine, I will deal with any conflicts [15:07] mvo: (done, for our record) [15:18] niemeyer: thank you (also for the record ;) [15:18] I must say that the google meet thing has better video quality [15:18] or mvo changed his camera to some high-def one [15:18] it looked much sharper than before [15:21] I will break for dinner now, back in one hour when I also have my disk space free [15:58] mborzecki, pstolowski: hey, I got a question about the new snapd.seeded service and I think I need help looking at this. the issue is https://pastebin.canonical.com/p/thJBrCX4wr/ i.e. it looks like this starts too early for some reason. do you think one of you have some minutes (tomorrow is fine too) to look into it? [16:29] mvo: sure, i can look at it tomorrow [16:38] niemeyer: hi there, are you joining? [16:39] sergiusens: Yep, sorry === pstolowski is now known as pstolowski|afk [16:46] mvo: is 2.33 in general a go? [16:47] or are there any issues warranting a .1 [16:57] zyga: no issues but we will most likely do a .1 to fix some autopkgtests in the distro [16:57] only autopkgtests? [16:57] zyga: but no real issues, all tests related [16:57] ok [16:58] good, I'll make sure we can release to opensuse tomorrw [16:58] zyga: thank you [16:58] and I need to buy a disk or a NAS maybe [16:58] zyga: if you want we can coordinate with a .1 [16:58] sure, that's fine [16:58] changing the tarsal is all we need I suspect [16:58] though [16:58] we can merge my packaging fixes for .1 too [17:13] cachio: interessting datapoint - I just got a econnreset test failure where we got a 503 from fastly [17:14] hmm, any gas recommendations [17:14] NAS [17:14]