| zyga | o/ | 05:14 |
|---|---|---|
| pstolowski | mornings | 07:02 |
| zyga | Hey Paweł | 07:32 |
| zyga | Quite a quiet | 07:32 |
| zyga | Morning :-) | 07:32 |
| zyga | Making coffee, I somehow could not wake normally | 07:33 |
| pedronis | seems there's a new gccgo version in bionic and we are hitting a bug that is fixed only in newer dh-golang | 07:34 |
| pedronis | gccgo tests have started failing on 18.04 | 07:35 |
| pedronis | mmh | 07:42 |
| zyga | pedronis: oh, do you have a bug reference? | 07:48 |
| zyga | shall we disable 18.04 for the weekend so that we can keep pushing things? | 07:49 |
| pedronis | we get this: https://paste.ubuntu.com/p/GVvz4qq4hY/ | 07:49 |
| pedronis | there's a debian bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=907263 | 07:49 |
| zyga | pedronis: perhaps we can pin the previous version of the package if it is still in the archive | 07:49 |
| zyga | thank you, looking | 07:50 |
| pedronis | that mentions dh-golang 1.35 to have the fix, but bionic has still only 1.34 | 07:50 |
| zyga | or we can add a ppa with dh-golang 1.35 perhaps | 07:50 |
| wgrant | Have you reported the critical SRU regression? | 07:51 |
| pedronis | not yet | 07:52 |
| pedronis | it took me a while to understand it was not new code | 07:52 |
| pedronis | but new package | 07:52 |
| pedronis | version | 07:52 |
| wgrant | doko: ^^ | 07:53 |
| joelkraehemann | hi all | 07:53 |
| joelkraehemann | Do all snap packages need manual review or is it only because the warning showing? | 07:54 |
| zyga | joelkraehemann: hello, what warning is that? | 07:55 |
| zyga | joelkraehemann: normally snap package do not require a manual review | 07:55 |
| joelkraehemann | zyga: great to know. I was using experimental layouts but for now I pass the environment variable $AGS_SNAP_PATH | 07:56 |
| joelkraehemann | ... and no more layouts | 07:56 |
| zyga | joelkraehemann: yeah that _may_ trigger a manual review for now, I'm trying to get that to be enabled by default | 07:56 |
| joelkraehemann | what do you think, when is it enabled by default? | 07:57 |
| zyga | I'd like to do that for the 2.36 release | 07:57 |
| zyga | there's a PR for that | 07:57 |
| zyga | some people are going to be off today so I don't know if it will land yet | 07:58 |
| zyga | probably monday | 07:58 |
| joelkraehemann | nice | 07:58 |
| zyga | pedronis, wgrant: so shall we file a bug for this and wait? | 07:58 |
| wgrant | zyga: File a bug, tag it with regression-update, and ping the uploader and/or release team on IRC | 08:00 |
| wgrant | s/release/SRU/ | 08:00 |
| zyga | kk | 08:00 |
| wgrant | https://wiki.ubuntu.com/StableReleaseUpdates#Regressions | 08:00 |
| zyga | thank you! | 08:01 |
| Chipaca | moin moin | 08:04 |
| Chipaca | mwhudson: o/ | 08:05 |
| zyga | pedronis: so gccgo is the faulty package? | 08:05 |
| * zyga tries to phrase the bug report | 08:05 | |
| Chipaca | mwhudson: if we have 1.10 in 16.04 etc, that still wouldn't pull gccgo equivalent in, right? | 08:05 |
| pedronis | zyga: https://bugs.launchpad.net/ubuntu/+source/dh-golang/+bug/1794936 | 08:05 |
| mup | Bug #1794936: new gccgo version update () is triggering dh-golang bug <dh-golang (Ubuntu):New> <gcc-defaults (Ubuntu):New> <https://launchpad.net/bugs/1794936> | 08:05 |
| zyga | ah, thanks, | 08:06 |
| mborzecki | morning | 08:06 |
| pedronis | zyga: it's not a bug on gccgo but things will fail with dh-golang unless we also get a new dh-golang | 08:07 |
| zyga | pedronis, wgrant: I added the regression-update tag to your bug | 08:07 |
| mborzecki | starting later than usual, had to run an errand | 08:07 |
| Chipaca | mwhudson: maybe I should ask this question on the bug | 08:07 |
| pedronis | zyga: I tried to mention both packages, but gcc packages are a maze, not sure I picked the right one | 08:07 |
| zyga | yeah, I was lost at gcc-defaults, dh-golang is easier | 08:08 |
| wgrant | Retargeted at gcc-8 | 08:08 |
| wgrant | Which was the SRU that broke things | 08:08 |
| Chipaca | pedronis: should we skip gccgo tests on 18.04 until this is sorted? | 08:11 |
| mup | PR snapd#5868 closed: tests/main/snap-env: extend to cover parallel installations of snaps <Parallel installs> <Simple> <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/5868> | 08:11 |
| mup | PR snapd#5872 closed: tests/main/parallel-install-local: rename from *-sideload, extend to run snaps <Parallel installs> <Simple> <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/5872> | 08:11 |
| pedronis | Chipaca: I suppose | 08:12 |
| mborzecki | zyga: can you take a look at https://github.com/snapcore/snapd/pull/5870 ? | 08:12 |
| mup | PR #5870: tests/main/parallel-install-services: add spread test for snaps with services <Parallel installs> <Simple> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5870> | 08:12 |
| pedronis | is michael off today? | 08:13 |
| zyga | pedronis: I think so | 08:13 |
| zyga | mborzecki: yes | 08:13 |
| === KaitoDaumoto is now known as Guest13744 | ||
| zyga | pedronis: I've asked him in private | 08:14 |
| zyga | pedronis: confirmed | 08:14 |
| pedronis | Chipaca: a 2nd review of #5824 would be appreciated | 08:14 |
| mup | PR #5824: fetch the device store assertion together and in the context of interpreting snap-declarations <Created by pedronis> <https://github.com/snapcore/snapd/pull/5824> | 08:14 |
| mborzecki | zyga: thanks! | 08:16 |
| ogra | ppisati, your kernel patch works great ! | 08:18 |
| mup | PR snapd#5880 opened: interfaces/repo: two helper methods for houtplug <Created by stolowski> <https://github.com/snapcore/snapd/pull/5880> | 08:20 |
| Chipaca | pedronis: should the assertion fetchers get a context at some point? | 08:24 |
| Chipaca | to be cancelable i mean | 08:24 |
| pedronis | maybe, at some point | 08:24 |
| Chipaca | pedronis: maybe as part of the move to bulk | 08:25 |
| Chipaca | tbh this code with functions getting functions that get passed functions is not super easy to follow :-) | 08:26 |
| pedronis | Chipaca: we have a bigger problem, in this sense that we fetch assertions from sync places | 08:26 |
| mup | PR snapd#5870 closed: tests/main/parallel-install-services: add spread test for snaps with services <Parallel installs> <Simple> <Created by bboozzoo> <Merged by zyga> <https://github.com/snapcore/snapd/pull/5870> | 08:26 |
| pedronis | (for good reasons but it will not end well at some point) | 08:26 |
| Chipaca | mhmm | 08:27 |
| Chipaca | not that I'd spotted it, but yes it sounds like trouble | 08:27 |
| pedronis | it's totally preexistent | 08:27 |
| pedronis | and it's a potential issues with hundreds of snaps | 08:28 |
| * Chipaca hopes popey isn't listening | 08:28 | |
| popey | *ears prick up* | 08:28 |
| pedronis | anyway we really have a problem that is either sync or changes | 08:28 |
| pedronis | we don't have an inbetween to do things | 08:29 |
| popey | what's up? | 08:29 |
| Chipaca | popey: talking about things that can't ever happen, ever ever | 08:30 |
| Chipaca | popey: unless somebody had hundreds of snaps | 08:30 |
| popey | alan@hal:~$ snap list | wc -l | 08:31 |
| popey | 234 | 08:31 |
| Chipaca | pedronis: which sync things do this? | 08:31 |
| popey | Who would do such a thing? | 08:31 |
| pedronis | Chipaca: snap refresh | 08:31 |
| Chipaca | popey: IKR | 08:31 |
| Chipaca | pedronis: ah, it does it as part of pre-changes work? | 08:31 |
| pedronis | yes | 08:31 |
| pedronis | since forever | 08:32 |
| Chipaca | shouldn't be too hard to move it into a change then | 08:32 |
| pedronis | it is | 08:32 |
| Chipaca | he says, like an idiot | 08:32 |
| * Chipaca grins | 08:32 | |
| pedronis | remember why we moved pre-flight checks out of changes | 08:32 |
| pedronis | it would undo that | 08:32 |
| Chipaca | TBH, I don't remember why, but I'm sure it's a good reason | 08:32 |
| Chipaca | (and I can re-remember it by reading, probably) | 08:33 |
| Chipaca | anyway, this isn't landing PRs | 08:33 |
| pedronis | to be fair it was specifically about downloads | 08:33 |
| pedronis | so maybe there's a why | 08:33 |
| pedronis | but is not a simple change | 08:33 |
| pedronis | s/why/way/ | 08:33 |
| ogra | hmm ... does the prepare_device hook of a gadget snap only run at the end after all snaps have been processed ? ... when i set a hostname from that hook and have avahi installed, the name resolution only works after a reboot | 08:34 |
| pedronis | ogra: it was not defined exactly | 08:35 |
| ogra | ah | 08:35 |
| pedronis | it might have been changed recently tough | 08:35 |
| pedronis | I need to go look | 08:35 |
| ogra | should the install hook run earlier ? | 08:35 |
| pedronis | which install hook? | 08:36 |
| pedronis | of the gadget | 08:36 |
| pedronis | ? | 08:36 |
| ogra | yeah | 08:36 |
| ogra | i see the gadget beinmg processed before the app snaps | 08:36 |
| pedronis | yes | 08:36 |
| ogra | so perhaps thats the better choice | 08:36 |
| ogra | ok | 08:36 |
| pedronis | it's core kernel gadget apps | 08:36 |
| pedronis | that is well defined | 08:37 |
| ogra | somehow the name tricked me that prepare_device runs before anything else :) | 08:37 |
| ogra | will use install then | 08:37 |
| pedronis | ogra: with 2.35 or so prepare-device is strictly after seeding | 08:43 |
| ogra | yeah, that explains the behaviour | 08:43 |
| pedronis | Chipaca: zyga: do you remember whether we have anywhere bash code that make choices based on comparing snapd version in the spread tests? | 08:44 |
| * zyga thinks | 08:45 | |
| zyga | apart from debian control scripts I don't think we do | 08:45 |
| Chipaca | pedronis: we have tests that fail if the version is not what's expected, but I don't think that's what you mean | 08:46 |
| pedronis | I would like to do something in upgrade/basic only if the old snapd is newer than 2.35 | 08:46 |
| Chipaca | it actually might be the prepare bit of the test tbf | 08:46 |
| pedronis | but writing might be a pain | 08:47 |
| pedronis | the issue really affects only debian in practical terms (the relevant change is actually older than 2.35) | 08:47 |
| Chipaca | pedronis: i thought we had a compare-versions in there but can't find it | 08:49 |
| pedronis | thought the same | 08:49 |
| pedronis | but not clear how to write portably except with python or something | 08:49 |
| pedronis | (we can't assume dpkg anymore) | 08:49 |
| Chipaca | pedronis: 'snap version --greater-than "$otherversion"'? | 08:54 |
| Chipaca | maybe python is easier :-) | 08:54 |
| pedronis | by definition the old snap will not have that | 08:55 |
| pedronis | anyway I might just do something blunter | 08:55 |
| pedronis | Chipaca: thx for the review | 08:56 |
| Chipaca | np | 08:56 |
| mup | PR snapd#5881 opened: tests: disable gccgo tests on 18.04 for now, until dh-golang vs gccgo are fixed <Created by pedronis> <https://github.com/snapcore/snapd/pull/5881> | 09:24 |
| pedronis | Chipaca: zyga: ^ | 09:24 |
| zyga | ack | 09:27 |
| Chipaca | pedronis: Error executing google:ubuntu-16.04-64:tests/main/interfaces-accounts-service | 09:50 |
| Chipaca | Error: GDBus.Error:org.freedesktop.DBus.Error.InvalidArgs: Type of message, '(ssssa{ss})', does not match expected type '(sssa{sv}a{ss})' | 09:50 |
| pstolowski | Chipaca: looks like the elusive error we see very sporadically on spread tests | 09:52 |
| Chipaca | mhmm | 09:53 |
| Chipaca | pstolowski: go away you're sick | 09:53 |
| pstolowski | (i saw it a couple of times over last few months) | 09:53 |
| zyga | travis is slooow | 09:56 |
| Chipaca | wow, I hadn't realise we were doing ~1k commits per release | 09:59 |
| Chipaca | sounds like a lot | 09:59 |
| === scroll is now known as hfjvjffju | ||
| pstolowski | zyga: commented on #5867 | 10:01 |
| mup | PR #5867: many: enable layouts by default <Created by zyga> <https://github.com/snapcore/snapd/pull/5867> | 10:01 |
| * Chipaca takes a break | 10:01 | |
| zyga | pstolowski: thanks | 10:05 |
| * zyga -> walk | 10:13 | |
| doko | snappy is using golang? | 10:27 |
| zyga | doko: yes | 10:34 |
| doko | zyga: please ask mwhudson about that | 10:49 |
| Chipaca | doko: about what? | 10:55 |
| mup | PR snapd#5881 closed: tests: disable gccgo tests on 18.04 for now, until dh-golang vs gccgo is fixed <Created by pedronis> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/5881> | 10:56 |
| Chipaca | woohoo | 10:57 |
| * Chipaca merges master | 10:57 | |
| pstolowski | yess | 10:59 |
| Chipaca | travis now experiencing stampeding herd of snapd PRs | 11:00 |
| ogra | pedronis, (not sure you are the right person, but i'll ask anyway) is there a way for me to obtain the model name from a hook in the gadget (i would like to set the default hostanme to the model name on frst boot) ... without having snapd-control ? | 11:45 |
| ogra | (would "snapctl known model" work without extra interface ?) | 11:46 |
| mup | PR snapcraft#2302 closed: requirements.txt: stop using pymacaroons-pynacl <Created by kyrofa> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2302> | 11:51 |
| mup | PR snapcraft#2303 closed: [legacy] requirements.txt: stop using pymacaroons-pynacl <Created by kyrofa> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2303> | 11:51 |
| pedronis | ogra: not atm | 11:55 |
| pedronis | there are some argument that we could have snapctl ack and snapctl known at least for gadgets but they are not there atm | 11:55 |
| ogra | hmm, ok | 11:56 |
| ogra | thats sad ... since it means i'll need one gadget per image type | 11:56 |
| ogra | thanks anyway ! | 11:57 |
| stgraber | sergiusens: so in theory we have the new logic in place in both edge and stable snaps now, the next upgrade will still be bad for you (because it's running the old stop logic) but hopefully you'll be good after that | 12:13 |
| sergiusens | stgraber: nice, can I see the PR that implements this? Just for curiosity | 12:14 |
| sergiusens | I am glad I raised it then too | 12:14 |
| stgraber | sergiusens: https://github.com/lxc/lxd-pkg-snap/commit/82950207244f445ad9c59ab22967ab20c002bd25 | 12:14 |
| pstolowski | hmmm, snapcraft tells me this: The linker version '2.23' used by the base 'core' is incompatible with files in this snap: | 12:31 |
| pstolowski | kyrofa: hey, any idea ^? i'm not doing anything with bases in the snap i'm building. snapcraft installed from snap, i presume it's the base of the snapcraft? | 12:33 |
| sergiusens | stgraber: looks much more robust | 12:34 |
| sergiusens | pstolowski: it means you are probably not building on 16.04 | 12:35 |
| mup | PR snapd#5882 opened: many: support and consider store friend-stores when checking device scope constraints <Created by pedronis> <https://github.com/snapcore/snapd/pull/5882> | 12:35 |
| stgraber | sergiusens: yep and the type field really shouldn't be translated :) | 12:37 |
| pstolowski | sergiusens: indeed, i'm on 18.04 | 12:37 |
| jdstrand | popey, Wimpress: hi! can one of you comment on https://forum.snapcraft.io/t/requesting-approval-for-goreleasers-classic-snap/6571/9? | 12:47 |
| zyga | jdstrand: hey | 12:47 |
| zyga | jdstrand: gentle ping for https://github.com/snapcore/snapd/pull/5395 | 12:47 |
| mup | PR #5395: interfaces: generalize writable mimic profile <Created by zyga> <https://github.com/snapcore/snapd/pull/5395> | 12:47 |
| jdstrand | zyga: I know, sorry | 12:47 |
| zyga | sure, I understand | 12:48 |
| zyga | Chipaca: standup? | 13:01 |
| Chipaca | omw | 13:01 |
| sergiusens | jdstrand: I commented | 13:28 |
| sergiusens | jdstrand: fwiw, it was elopio and myself who were involved in that | 13:29 |
| mup | PR snapd#5824 closed: many: fetch the device store assertion together and in the context of interpreting snap-declarations <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/5824> | 13:31 |
| jdstrand | sergiusens: ah, thanks :) | 14:06 |
| jdstrand | Wimpress, popey: sergiusens responded to https://forum.snapcraft.io/t/requesting-approval-for-goreleasers-classic-snap/6571/12, but can one of you vet the publisher? | 14:07 |
| popey | done | 14:10 |
| kenvandine | zyga: found a snapd bug in the cosmic live session https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1794953 | 14:16 |
| mup | Bug #1794953: GNOME Calculator snap has wrong theme in live session <rls-cc-tracking> <gnome-calculator (Ubuntu):Invalid by ken-vandine> <snapd (Ubuntu):New> <gnome-calculator (Ubuntu Cosmic):Invalid by ken-vandine> <snapd (Ubuntu Cosmic):New> <https://launchpad.net/bugs/1794953> | 14:16 |
| kenvandine | zyga: permission denied errors mounting the content interface | 14:16 |
| Chipaca | https://pastebin.ubuntu.com/p/nYmtwkYWXp/ :-D | 14:19 |
| Chipaca | niemeyer: new category, or should I put them into "other"? ^ | 14:20 |
| * niemeyer look | 14:20 | |
| niemeyer | s | 14:20 |
| Chipaca | we still have space for a few more categories, fwiw | 14:20 |
| zyga | kenvandine: hey | 14:20 |
| niemeyer | Chipaca: That seems to fit nicely into a "Snapshots" one | 14:20 |
| Chipaca | yerp | 14:20 |
| niemeyer | Chipaca: Nice tests, btw | 14:20 |
| zyga | kenvandine: interesting, can you add the apparmor denials please? | 14:21 |
| Chipaca | niemeyer: :) | 14:21 |
| kenvandine | sure | 14:21 |
| zyga | I need coffee, brb | 14:22 |
| mup | PR snapd#5833 closed: asserts,interfaces/policy: add support for on-store/on-brand/on-model plug/slot rule constraints <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/5833> | 14:24 |
| kenvandine | zyga: added to the bug | 14:24 |
| kenvandine | zyga of particular interest is the content interface for gnome-3-26-1604 must be mounting just fine | 14:25 |
| kenvandine | it's only gtk-common-themes | 14:25 |
| zyga | re! | 14:29 |
| zyga | man | 14:29 |
| zyga | I put the powdered coffee beans into the mug | 14:29 |
| zyga | Sep 28 14:08:52 ubuntu kernel: [ 68.310240] audit: type=1400 audit(1538143732.840:235): apparmor="DENIED" operation="open" profile="snap-update-ns.gnome-calculator" name="/upper/snap/" pid=6535 comm="3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 | 14:30 |
| zyga | that's the thing | 14:30 |
| zyga | well, I can look at that next week | 14:30 |
| kenvandine | zyga: thanks, that bug is a cosmic release blocker | 14:31 |
| kenvandine | zyga: just FYI | 14:31 |
| zyga | yeah, no pressure ;) | 14:31 |
| zyga | it's not an LTS, right? ;-) | 14:31 |
| kenvandine | lol | 14:31 |
| kenvandine | willcooke: ^^ FYI zyga's on the case | 14:32 |
| willcooke | thanks chaps | 14:32 |
| zyga | I assigned it to myself | 14:32 |
| zyga | well | 14:33 |
| zyga | kenvandine: while I have you | 14:33 |
| zyga | do you have the session open? | 14:33 |
| kenvandine | yes | 14:33 |
| zyga | can you poke at one file please | 14:33 |
| zyga | let me give you the details | 14:33 |
| zyga | can you go to /var/lib/snapd/apparmor/snap-confine | 14:35 |
| zyga | and tell me what you have there | 14:35 |
| zyga | there _should_ be a file called "overlay-root" there | 14:35 |
| kenvandine | yes | 14:35 |
| zyga | kenvandine: can you also share /proc/self/mountinfo from the live session | 14:35 |
| zyga | and if the file is there, pastebin the file please | 14:35 |
| kenvandine | http://paste.ubuntu.com/p/PhsbzsCzZ8/ | 14:38 |
| kenvandine | zyga: and just to confirm, there is just one file in snap-confine, overlay-root | 14:40 |
| zyga | hmmm | 14:40 |
| zyga | that's very cool | 14:40 |
| zyga | but ... | 14:40 |
| zyga | something new in the kernel apparently | 14:40 |
| zyga | what's in the overlay-root file? | 14:40 |
| zyga | look at the mount table you pasted | 14:40 |
| zyga | the Wally is /cow | 14:40 |
| zyga | where's Wally? | 14:40 |
| zyga | is there a /cow directory? | 14:41 |
| kenvandine | overlay-root has "/upper/{,**/}" r, | 14:41 |
| kenvandine | there is no /cow | 14:41 |
| zyga | 29 1 0:24 / / rw,relatime shared:1 - overlay /cow rw,lowerdir=//filesystem.squashfs,upperdir=/cow/upper,workdir=/cow/work | 14:42 |
| zyga | ! | 14:42 |
| zyga | right? | 14:42 |
| zyga | so what's /cow? | 14:42 |
| zyga | I mooost know | 14:42 |
| kenvandine | no idea :) | 14:42 |
| zyga | a sample denial is: | 14:43 |
| zyga | Sep 28 14:08:52 ubuntu kernel: [ 68.309885] audit: type=1400 audit(1538143732.840:232): apparmor="DENIED" operation="open" profile="snap-update-ns.gnome-calculator" name="/upper/snap/" pid=6535 comm="3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 | 14:43 |
| ogra | it makes milk usually :P | 14:43 |
| zyga | but the pattern you pasted should cover that | 14:43 |
| ogra | (shake it and you have cream !) | 14:43 |
| kenvandine | :) | 14:43 |
| zyga | kenvandine: can you look at /var/lib/snapd/apparmor/profiles and look for /upper there? | 14:43 |
| kenvandine | zyga: and the mount for gnome-3-26-1604 is working fine | 14:43 |
| zyga | in the snap-update-ns.gnome-calculator profile? | 14:43 |
| zyga | jdstrand: ^ FYI, not sure if you know about any changes to overlayfs between 18.04 and 18.10 | 14:44 |
| zyga | kenvandine: please collect all those log files in the bug report | 14:45 |
| kenvandine | zyga: there is no occurrences of upper in the profile | 14:45 |
| zyga | maybe as files for easier access | 14:45 |
| zyga | oh! | 14:45 |
| zyga | you mean that snap-update-ns.gnome-calculator has no /upper in it? | 14:45 |
| kenvandine | right | 14:45 |
| jdstrand | zyga: I do not. this is on the livecd? | 14:45 |
| zyga | yep | 14:45 |
| kenvandine | zyga: yes, live session | 14:45 |
| zyga | kenvandine: that's the bug perhaps | 14:45 |
| kenvandine | works fine installed | 14:45 |
| jdstrand | well, someone needs to adjust the policy for that. it me, it will be a little while, but someone can submit a pr and I'd look at it | 14:46 |
| zyga | kenvandine: please add the file you looked at to the bug report | 14:46 |
| jdstrand | s/it me/if me/ | 14:46 |
| zyga | jdstrand: not sure if there's something to adjust yet | 14:46 |
| zyga | it looks more magic than before | 14:46 |
| Chipaca | niemeyer: should I add SnapID to snapshots before, together with, or after landing the commands? | 14:46 |
| zyga | jdstrand: anyway, I just wanted to check if you were aware of any changes to overlayfs | 14:47 |
| zyga | kenvandine: I'll break for dinner now, ok? | 14:47 |
| kenvandine | zyga: ok | 14:47 |
| kenvandine | i'll attach stuff to the bug | 14:47 |
| Chipaca | niemeyer: there's already an after-landing tasklet of blocking epochs, and restoring configs, both of which aren't done | 14:47 |
| Chipaca | could be part of that | 14:47 |
| niemeyer | Chipaca: Sounds fine either way.. as long as we don't release in the wild before the disk format is stable | 14:48 |
| niemeyer | Chipaca: We can add an experimental flag for that, if necessary | 14:48 |
| Chipaca | niemeyer: I can mark the PR blocked and just not land it until the followup is ready as well | 14:49 |
| Chipaca | mmm | 14:49 |
| Chipaca | dunno | 14:49 |
| Chipaca | niemeyer: I'll do something sensible. Thanks. | 14:49 |
| niemeyer | Chipaca: Thanks for that :) | 14:51 |
| * Chipaca takes a break | 14:58 | |
| ogra | ogra@localhost:~$ snap interfaces | 15:04 |
| ogra | error: no interfaces found | 15:04 |
| ogra | GRRRRRR !!!! | 15:04 |
| Chipaca | ogra: eh, you didn't need them anyway | 15:04 |
| * Chipaca really tries to take that break now | 15:04 | |
| ogra | Chipaca, well, you guys make setting a hostname from a gadget hook really really hard ... i switched from prepare_device to the install hook of the gadget (because prepare_device only runs after the device hads been prepared which makes it completely useless for that case) ... and that got me into that state | 15:06 |
| ogra | seems snapd simply stopped seeding the system completely now | 15:06 |
| ogra | ogra@localhost:~$ snap list | 15:08 |
| ogra | No snaps are installed yet. Try 'snap install hello-world'. | 15:08 |
| ogra | :( | 15:08 |
| pedronis | what does snap changes says? and snap tasks for the ones there | 15:10 |
| ogra | ogra@localhost:~$ snap changes | 15:10 |
| ogra | ID Status Spawn Ready Summary | 15:10 |
| ogra | 1 Error today at 14:52 UTC today at 14:54 UTC Initialize system state | 15:10 |
| ogra | 2 Error today at 14:53 UTC today at 14:53 UTC Initialize device | 15:10 |
| ogra | 3 Error today at 15:01 UTC today at 15:02 UTC Initialize system state | 15:10 |
| ogra | 4 Error today at 15:01 UTC today at 15:02 UTC Initialize device | 15:10 |
| ogra | 5 Error today at 15:09 UTC today at 15:10 UTC Initialize system state | 15:10 |
| ogra | Error today at 15:08 UTC today at 15:09 UTC Run install hook of "pi-kiosk" snap if present | 15:10 |
| ogra | pedronis, that install hook calls "/usr/bin/hostnamectl set-hostname ..." the gadget.yaml has the right connect statement and the snapcraft.yaml allows the hostname-control plug for the install hook | 15:12 |
| pedronis | ? | 15:12 |
| pedronis | connect is called later | 15:12 |
| ogra | ugh | 15:13 |
| pedronis | you really need an auto-connect for that | 15:13 |
| pedronis | connections doesn't make a lot of sense for those kind of interfaces | 15:13 |
| ogra | any chance the configure hook would be late enough ? | 15:13 |
| zyga | re | 15:13 |
| ogra | i really just need to set a hostname before avahi comes up for the first time | 15:13 |
| pedronis | I would need to check but unlikely | 15:13 |
| ogra | sigh | 15:13 |
| pedronis | we set up connections after everything else | 15:14 |
| pedronis | anyway they are not a good fit for this | 15:14 |
| ogra | we're making this really hard | 15:14 |
| pedronis | this is really more a auto-connect case | 15:14 |
| pedronis | which would work | 15:14 |
| ogra | yeah, but i dont really want to wait for someone to set up autoconnect for the gadget ... | 15:15 |
| Chipaca | ogra: the avahi service could wait, couldn't it? | 15:15 |
| ogra | Chipaca, how ? (without me modifying the avahi snap) | 15:16 |
| Chipaca | ah | 15:16 |
| Chipaca | picky, picky | 15:16 |
| ogra | i'm trying to exercise a customer usse case here of simply creating an appliance image from mostly exissting snaps | 15:16 |
| Chipaca | right | 15:16 |
| Chipaca | and having the gadget set the hostname does sound reasonable | 15:17 |
| ogra | yeah | 15:17 |
| ogra | well, and it works from prepare_device ... but thats too late to work with avahi ... there will be client images that will try to auto-conect to that server appliance | 15:17 |
| Chipaca | maybe the bug is that we start things before prepare device is done? | 15:18 |
| ogra | right, i wouldnt expect prepare_device to be run as last thing after all apps have been processed | 15:18 |
| ogra | doesnt really sound like "prepare" anymore | 15:19 |
| ogra | rather like "postprocess" | 15:19 |
| Chipaca | OTOH, one of the things that we start might be network manager | 15:19 |
| Chipaca | without which prepare-device wouldn't | 15:19 |
| Chipaca | it's worms all the way down i tell you | 15:20 |
| * Chipaca runs off to some place with less worms | 15:20 | |
| * Chipaca sends beer | 15:20 | |
| ogra | heh, well ... all in all trying to build a simple appliance makes me hop from one roadblock to another atm | 15:20 |
| ogra | the worst thing is that i do not get a snap id for sideloaded gadgets so even to test if something works i have to do a full turnaround through the store | 15:22 |
| ogra | ... even for one line changes | 15:23 |
| * ogra moves back to the prepare_device hook and adds reboot interface support to it ... | 15:24 | |
| ogra | i'll just force a reboot after setting the hostname ... even though thats the worst UX i can imagine | 15:24 |
| ogra | (not to mention that the firt boot on a pi3 takes 15-20 min just for sseeding the snaps) | 15:25 |
| ogra | *first | 15:25 |
| ogra | oh ... wait | 15:28 |
| ogra | pedronis, on a device with an unofficial (non canonical) image, is the prepare_device hook called over and over ? i.e. would the above get me a reboot loop ? | 15:29 |
| pedronis | ogra: yes, the prepare-device hook can be called multiple times | 15:31 |
| ogra | oh my | 15:31 |
| ogra | so no way out for me unless i add gross hacks ... which is exactly what i didnt want for this specific image :( | 15:31 |
| ogra | sad ... | 15:32 |
| pedronis | ogra: the doc says as much, https://forum.snapcraft.io/t/the-gadget-snap/696 | 15:32 |
| pedronis | see the first paras of prepare-device hook | 15:33 |
| ogra | yeah, i remember reading that somewhere | 15:33 |
| ogra | ah, well, i can work around that in the hook i guess | 15:37 |
| ogra | oh sigh ... so the shutdown interface only allows dbus connecttions ... not calling the reboot command directly | 15:42 |
| pedronis | ogra: why do you need to reboot? | 15:42 |
| ogra | pedronis, i need avahi restarted after the hostname has been set | 15:42 |
| pedronis | reboot is a big hammer to restart avahi | 15:43 |
| pedronis | sounds more like you need to restart avahi | 15:43 |
| ogra | since i dont have snapd-control (and dont want it) to restart avahi from the gadget hook, thats the other option | 15:43 |
| ogra | hahahaha | 15:43 |
| pedronis | that doesn't make a lot of sense to me | 15:43 |
| ogra | yes, "just restarting avahi" or "setting the hostname before avahi starts" would be the right things | 15:44 |
| pedronis | I mean it's not a reasonable practice, to reboot to restart one thing | 15:44 |
| ogra | but the current design doesnt allow either | 15:44 |
| pedronis | if your gadget as the right auto-connect it would work | 15:44 |
| pedronis | there was an idea to make connections in gadget.yaml more like auto connect | 15:45 |
| pedronis | but it was punted | 15:45 |
| ogra | yes, but only via requesting that from the store | 15:45 |
| ogra | imagine i'm a developer that evaluates core for his company | 15:45 |
| ogra | this is the role i'm currently trying to play | 15:45 |
| pedronis | I understand but also unclear that setting hostname is so relevant | 15:45 |
| ogra | exercising the existing stuff we have | 15:46 |
| pedronis | anyway it seems you should make a case in the forum to make connections in gadget more like auto-connections | 15:46 |
| pedronis | the part that was punted there is deciding what was the limit of that | 15:46 |
| pedronis | general auto-connect behavior was deemed too much | 15:46 |
| ogra | the thing i'm building is a two image system ... one server applance that runs a digital signage tool and a bunch of leaf systems running from a second image that will find their sever via an mdns request | 15:47 |
| pedronis | also you could probably use an interface hooks | 15:47 |
| pedronis | maybe | 15:47 |
| pedronis | that runs once the interface you need is *connected* | 15:47 |
| pedronis | bit strange but better than rebooting I suppose | 15:47 |
| ogra | the expectation is that you just shove in pre-flashed SD cards into a bunch of RPi's and have your store running the ads ... with a web mgmt UI on the server system | 15:48 |
| ogra | so both images should set up themselves fully automatic | 15:49 |
| ogra | and sadly avahi on the server side is a must | 15:49 |
| pedronis | can't you invoke the equivalent of avahi-set-host-name | 15:51 |
| pedronis | through avahi-control? | 15:52 |
| ogra | hmm... and build an additional snap just for that ? | 15:52 |
| pedronis | ? | 15:52 |
| pedronis | from the gadget | 15:52 |
| pedronis | as I said you can probably use interface hooks to run things from the gadget | 15:53 |
| pedronis | once you have the interfaces you need | 15:53 |
| pedronis | from connections | 15:53 |
| ogra | hmm | 15:53 |
| pedronis | if I understand how they should work | 15:53 |
| pedronis | they = interface hooks | 15:53 |
| ogra | the prob is: i dont think avahi-set-hostname is persistent ... | 15:54 |
| pedronis | you do both | 15:54 |
| pedronis | set the hostname through systemd | 15:54 |
| pedronis | but also for avahi | 15:54 |
| pedronis | so you don't have to restart the latter | 15:54 |
| pedronis | or at least that's the idea | 15:54 |
| ogra | well, thinking of it a -config snap that runs a service might not be the worst idea anyway ... and leaving all of this out of the gadget | 15:55 |
| ogra | i might find more things i want to set and i would really like to re-use the gadget for the client images too | 15:55 |
| ogra | (and setting hostanmes on the client image isnt needed at all) | 15:56 |
| ogra | i'll tinker with that | 15:56 |
| * zyga iterates on interesting mount issues | 16:31 | |
| zyga | next week I'll split 50% for mount bugs and for new features | 16:32 |
| zyga | not to starve either | 16:32 |
| zyga | but no more distractions | 16:32 |
| * zyga dives into 3.10 kernel | 16:42 | |
| ogra | pedronis, FYI: i actually found that i can use a default setting from gadget.yaml for the avahi snap to set the hostname in parallel (this is only for the first boot anyway and a reboot wouldnt have been that bad given seeding the 4 app snaps the image ships already takes 15-20 min, so a reboot only adds a minute in max) ... but i guess in general thats an area for improvement | 16:50 |
| mup | PR snapd#5883 opened: tests,cmd/snap-update-ns: add test showing mount update bug <Created by zyga> <https://github.com/snapcore/snapd/pull/5883> | 17:02 |
| jdstrand | zyga: approved 5395 | 17:19 |
| zyga | jdstrand: awesome, thank you :) | 17:19 |
| zyga | jdstrand: I'm exploring a pair of bugs highlighted by that PR above | 17:20 |
| zyga | but I think I'm on it and it's all good :) | 17:20 |
| mup | PR snapd#5395 closed: interfaces: generalize writable mimic profile <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/5395> | 17:21 |
| * zyga is super happy and grateful! | 17:21 | |
| mup | PR snapd#5884 opened: overlord/snapshotstate: store the SnapID in snapshot, block restore if changed <Created by chipaca> <https://github.com/snapcore/snapd/pull/5884> | 18:21 |
| * Chipaca ~> dinner | 18:21 | |
| zyga | enjoy Chipaca :) | 18:34 |
| mup | PR snapd#5885 opened: Adding DPDK interface for DPDK Snap <Created by wililupy> <https://github.com/snapcore/snapd/pull/5885> | 18:48 |
| smoser | have other people seen slow download times for snaps ? | 18:49 |
| smoser | in some testing we have, the 'snap install lxd' which is the first snap so it gets the core snap sometimes took 15 minutes or more to download. | 18:49 |
| smoser | where the link was not the problem | 18:50 |
| smoser | i realize the fastly cdn is where the data comes from, but i would expect higher speeds than we're consistently seeing. | 18:50 |
| zyga | smoser: no idea | 18:52 |
| Chipaca | smoser: that is very slow. Can we set debugging to see where it's getting the things from? | 19:44 |
| smoser | Chipaca: well, it doesnt always happen. but last night when it did it was from fastly. i think the ip was in 104. or 103. range | 19:46 |
| smoser | thats from memory though | 19:46 |
| smoser | ranges at https://api.fastly.com/public-ip-list | 19:46 |
| smoser | http://paste.ubuntu.com/p/vCMMCvK3ZZ/ | 19:47 |
| smoser | som emory was wrong. | 19:47 |
| smoser | 151.101194.217 | 19:47 |
| Chipaca | smoser: I don't know if that's enough to know enough | 19:49 |
| smoser | what else would you want ? | 19:49 |
| smoser | other than my uplink being saturated at that point. | 19:50 |
| Chipaca | smoser: if you can set, in the environment where this happens, SNAPD_DEBUG=1 and SNAPD_DEBUG_HTTP=3 (e.g. as in https://forum.snapcraft.io/t/http-response-503-when-installing-anbox/6823/4) and then, when it happens, you can access the snapd logs (as in journalctl -u snapd), that would help | 19:50 |
| smoser | but i was anot aware of other issues at that time. | 19:50 |
| smoser | i think its pretty clear that it was just downloading stuff slowly. | 19:51 |
| smoser | what other explanation could you have? | 19:51 |
| smoser | uless snapd is debug showing route information (mtr like output) | 19:51 |
| Chipaca | smoser: was it slow because you were getting RST packets thrown at you by a hostile ISP? Because you were getting network timeouts? Because you connected to the wrong fastly? Because the store was in distress? I don't know | 19:52 |
| smoser | but what information would snapd debug bring to the table ? | 19:53 |
| Chipaca | smoser: if the download was being retried, there'd be a log line for each retry / continuation | 19:53 |
| smoser | i dont think it would likely give you any of those things. | 19:53 |
| smoser | it wasnt being retried | 19:53 |
| Chipaca | smoser: it also logs what fastly it downloads from | 19:53 |
| smoser | i provided the ip of the fastly there. | 19:53 |
| Chipaca | oh, I thought that was your ip | 19:54 |
| smoser | bah. i typod' it. but 151.101.194.217 | 19:54 |
| smoser | nice | 19:55 |
| smoser | i hit the same IP as ejfinneran did there. | 19:55 |
| Chipaca | smoser: and where was the server located? | 19:55 |
| Chipaca | or what was its ip :) | 19:55 |
| Chipaca | i mean, the one you were running snapd on | 19:55 |
| smoser | here. in detroit area | 19:55 |
| Chipaca | from where I stand that sounds close enough to scottsdale to be reasonable | 19:56 |
| Chipaca | noise][: is there anybody with bandwith to check on a slow download thing? | 19:57 |
| Chipaca | noise][: i know people are running ragged right now | 19:57 |
| smoser | maybe reasonable... but i'd surely expect a CDN to have an east coast and west coast US mirror | 19:57 |
| smoser | and for arizona to hit the west coast | 19:57 |
| smoser | and detroit to be east | 19:57 |
| smoser | if there were not others in the mix | 19:57 |
| smoser | :) | 19:57 |
| Chipaca | smoser: bah, i used a geolocator that seems broken (I give it 151.101.194.217 and it rewrites it to 50.63.136.217 for whatever reason) | 19:58 |
| noise][ | Chipaca: I'd guess not on a friday afternoon after a crazy week :) | 20:03 |
| noise][ | ID'ing the IP and having snapd logs is a good start | 20:04 |
| noise][ | and then traceroutes/mtr to the IP helps too | 20:04 |
| noise][ | often the case is just some issue between user's network and the CDN PoP | 20:04 |
| Chipaca | noise][: thanks | 20:06 |
| noise][ | smoser: feel free to file a snapstore bug with as many details as you can, but these tend to be hard to repro w/o a machine in the same location | 20:07 |
| smoser | of course | 20:07 |
| smoser | we did seem to be having some pain yesterday , both on our jenkins and local systems | 20:07 |
| Chipaca | smoser: for the record, yes it should be a lot quicker than that | 20:07 |
| smoser | but... having not collected debug information it is all in the realm of non-useful information | 20:07 |
| Chipaca | I get better download speeds on my ADSL :-) | 20:08 |
| smoser | Chipaca: fastly does seem to want to send me to mirrors on the west coast | 20:10 |
| Chipaca | smoser: detroit isn't that far west, is it | 20:10 |
| smoser | ~600 miles to new york city. ~2500 to san fran | 20:12 |
| smoser | but i dont knwo where they have data centers | 20:13 |
| Chipaca | https://www.fastly.com/network-map says they probably have something closer | 20:14 |
| smoser | oh well | 20:20 |
| mup | PR snapcraft#2301 closed: tests: move most tests to spread and reorder travis.yaml <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2301> | 23:10 |
| mup | PR snapcraft#2304 opened: project loader: remove remote parts support for bases <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2304> | 23:16 |
| k1412 | Hello everyone, I'm just beginning with Snapd. I wanted use a custom netns configuration but I not understand how I must use that with network-control. Have someone a little expain about that ? Thanks. | 23:21 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!