/srv/irclogs.ubuntu.com/2018/03/14/#snappy.txt

Chipacanot finished, will carry on tomorrow00:08
Chipacaoff to zzz :-)00:08
mupPR snapd#4835 opened: tests: add bionic system to google backend <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/4835>00:54
=== thresh_ is now known as thresh
zyga-ubuntuGood morning05:27
mborzeckimorning06:07
zyga-ubunture!07:15
zyga-ubuntukids are handled, wife at work, dog is sleeping (again) and I can finally get to work :)07:15
mborzeckizyga-ubuntu: hey07:19
zyga-ubuntuhey :-)07:19
mborzeckizyga-ubuntu: so the kernel bug with dangling symlinks is real, quite a lot of these came up in your pr07:21
zyga-ubuntuyes, it's real07:21
zyga-ubuntunow it's unclear if it is related to the issues we saw07:21
zyga-ubuntuor if it is just a harmless issue07:21
mborzeckizyga-ubuntu: does it also appear in debian sid, or is this part of apparmor ubuntu specific?07:22
zyga-ubuntuI don't know07:22
zyga-ubuntumy gut feeling is that it's not specific to ubuntu07:23
zyga-ubuntuoops, I managed to get an appointment with a doctor at 1107:25
zyga-ubuntugood morning mvo07:36
mvozyga-ubuntu: hey! good morning07:36
mborzeckimvo: hey07:39
mvohey mborzecki07:39
mvotime for 2.32~rc2!07:39
mvozyga-ubuntu: sorry for being a pain, what is the (rough) status of the layout fix we need for 2.32?07:43
zyga-ubuntuno change since last week, I know what the problem is and I worked on the first of the two problems (symlink problem) but put this on hold earlier this week07:44
zyga-ubuntuI'm adding unit tests07:44
zyga-ubuntuthough I think I will be changing that code a little as well07:44
mvozyga-ubuntu: thanks!07:44
zyga-ubuntuso the status is: known bugs, in progress07:44
mvozyga-ubuntu: could we disable parts of the functionality to get there quicker? or is that a) stupid b) not-needed c) all of the above?07:44
zyga-ubuntuit's not worth it, we need the experimental feature flag anyway07:45
mvozyga-ubuntu: aha, ok07:45
zyga-ubuntuthe symlink bug is "don't make symlinks if one is in place and has good value"07:45
zyga-ubuntuthe writability leak problem is "don't write over things that are not tmpfs"07:45
mborzeckihm the opensuse kernel that we use has apparmor disabled apparently07:46
mborzeckiuname shows 4.15.7-x86_64-linode102, linode specific kernel?07:46
zyga-ubuntuyes, apparently07:47
mvoanyone working on the vet failures on 18.04? if not I will do now07:47
mborzeckidamn, kernel-default-4.4.27-2.1.x86_64 this is installed, but it's running a different kernel, pfff07:48
mborzeckimvo: i can take a look07:48
mvomborzecki: its about the vet failures in the test log of 483507:49
zyga-ubuntuthere are some people who want to contribute to the gentoo overlay07:50
zyga-ubuntuhttps://github.com/zyga/gentoo-snappy/issues07:50
zyga-ubuntusome PRs and issues07:50
mupPR snapd#4803 closed: snap-confine, snap-seccomp: utilize new seccomp logging features - 2.32 <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4803>07:50
mupPR snapd#4831 closed: tests: force profile re-generation via system-key <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4831>07:50
mupPR snapd#4828 closed: many: support holding refreshes by setting refresh.hold (2.32) <Critical> <Created by pedronis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4828>07:51
mvozyga-ubuntu: how safe is 4830?07:52
mvozyga-ubuntu: I am inclinded to merge07:53
zyga-ubuntuI don't know of any issues07:53
mvozyga-ubuntu: this runs in edge for some time, right?07:53
zyga-ubuntuyes07:53
zyga-ubuntuI think it's safe07:53
zyga-ubuntualso on reverts07:53
zyga-ubuntusince reverting brings the reverted snap-confine07:53
zyga-ubuntuand its own profile and startup behavior07:53
zyga-ubuntuthe only potential downside is if we made a mistake in the apparmor profiles07:54
mvook07:54
zyga-ubuntubut nobody complained so far07:54
mupPR snapd#4830 closed: many: generate and use per-snap snap-update-ns profile (2.32) <Created by zyga> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/4830>07:54
zyga-ubuntumvo: hmmmm07:55
zyga-ubuntuhttps://github.com/snapcore/snapd/pull/476507:55
mupPR #4765: interfaces: harden snap-update-ns profile <Created by zyga> <https://github.com/snapcore/snapd/pull/4765>07:55
zyga-ubuntuactually, I'm drunk07:55
zyga-ubuntusince that PR you just merged is just the split of the profiles07:55
zyga-ubuntuthis is the hardening07:55
zyga-ubuntuand this is the part when I meant that maybe we missed something07:56
zyga-ubuntuthe first PR is safe07:56
zyga-ubuntumvo: this PR is the interesting one that jamie wanted to see.07:56
zyga-ubuntuman, I thought we merged it already07:56
zyga-ubuntuand is full of policy07:57
zyga-ubuntumvo: can you review it?07:57
mvozyga-ubuntu: yes, in a bit, let me go over the essential bits for 2.32 first07:58
zyga-ubuntumvo: I think this is also for 2.32 but I understand if you lack it07:58
zyga-ubuntuNACK it07:58
zyga-ubuntu(silly spellchecker on macOS)07:58
kalikianamoin moin08:00
zyga-ubuntuhey kalikiana08:01
mupPR snapd#4836 opened: tests: force profile re-generation via system-key <Created by mvo5> <https://github.com/snapcore/snapd/pull/4836>08:03
=== pstolowski|afk is now known as pstolowski
pstolowskimorning!08:06
mborzeckikalikiana: pstolowski: morning guys08:06
mborzeckiwoow, we use testable examples in snap/info_test.go, never seen it being used before08:07
mvomborzecki: we want 4826 - right?08:07
mborzeckimvo: for 2.32?08:07
mvomborzecki: I mean we want it for 2.3208:07
mvomborzecki: yes08:07
mvopstolowski: good morning!08:07
mborzeckimvo: yes, it'd be nice to include it, it's a bugfix :)08:08
mvopedronis: 4825 looks almost ready, just one question from gustavo left afaict?08:08
mvomborzecki: cool, lets see if we can find a second reviewer08:08
mvozyga-ubuntu: do we know how the feature flag will look like? environment key?08:15
zyga-ubuntuCore setting08:16
zyga-ubuntusnap set core experimental.layouts true08:16
mborzeckihmm vet does not handle struct methods defined in export_test.go files08:16
mvozyga-ubuntu: do you have code for this or shall I add some? asking because I think with that in place ~rc2 would be ready08:19
mvozyga-ubuntu: I can work on it08:20
mborzeckihttps://github.com/golang/go/issues/2370108:29
zyga-ubuntumvo: I don't have that yet08:33
zyga-ubuntuso if you want to pick it up, thanks please do08:33
mvomborzecki: hm, this reads as if the issue should be fixed? do we just need a golang update in bionic?08:33
mborzeckigo version go1.10 linux/amd64 locally and i can still see this08:34
mvozyga-ubuntu: yeah, I want 2.32~rc2 today if layout have bugs thats ok, we need one more upload for 2.32(-final) anyway08:34
zyga-ubuntuack08:34
mvomborzecki: hm, does that have https://go-review.googlesource.com/c/go/+/92215 ?08:34
mborzeckimvo: yes, the commit is in 1.10 branch08:36
mvo:(08:37
mvook08:37
mvomborzecki: if you have a fix for the other vet issues, please push08:37
mupPR snapd#4837 opened: many: go vet cleanups <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/4837>08:41
mborzeckimvo: ^^08:41
mvomborzecki: \o/08:41
mborzeckias for the bug that's supposed to be resolved in 1.10, i can still reproduce it with the sample that's provided in the original ticket on both 1.10 from golang.org download and 1.10 from arch packages08:42
mvomborzecki: ok, so we need to reopen upstream? and/or find a local workaround :/08:43
mvoand/or disable vet on go1.10 (which would suck)08:44
mborzeckii have a sort of a workaround but it's labor intensive :)08:45
mborzeckibasically changing the struct methods to simple funcs08:45
mborzeckiand update all call sites08:46
mvomborzecki: hm, that sounds slightly nasty08:47
Chipacamoin moin08:50
* Chipaca realises he still has time for more coffee before official start-of-day08:51
mborzeckimvo: it isn't that bad https://paste.ubuntu.com/p/ZPgmsKpM6s/08:52
mborzeckimvo: i can push that too08:53
mvomborzecki: yeah, that looks reasonable08:53
mvomborzecki: unfortunate but reasonable08:53
mvoChipaca: hey! good morning!08:53
mborzeckiChipaca: morning, more coffee yes, that's a splendid idea :P08:54
om26erelopio: Hi! I see you did initial snap packaging for keybase client. Are there any blockers on the way, anyway I could help get that finished ?08:54
* Chipaca hugs elopio 08:55
* Chipaca hugs his coffee, also08:56
mborzeckimvo: pushed the workarounds08:58
mborzeckineed to run an errand, bb in ~1.5h09:02
* zyga-ubuntu -> doctor09:27
zyga-ubuntuttyl09:27
mupPR snapd#4838 opened: snapstate: put layout feature behind feature flag <Created by mvo5> <https://github.com/snapcore/snapd/pull/4838>09:40
zyga-ubuntumvo: commented, thank you for writing that09:57
* zyga-ubuntu waits at the doctors09:57
zyga-ubuntuThis will take a while (hopefully not too long)09:58
mvozyga-ubuntu: thank you10:04
mvozyga-ubuntu: heh, spread tests> probably, I guess the spread run will be very unhappy!10:08
mvozyga-ubuntu: good catch10:08
Chipacamvo: can you remind me what advise-snap's supposed to do without --command?10:08
mvoChipaca: advice on snap name10:09
Chipacamvo: what's the apt equivalent of that one?10:09
mvoChipaca: so that "apt install spotify" could say something nice like "The spotify snap is available at version 2.0, see snap info spotify for additional versions"10:09
Chipacaah10:09
Chipacaa'ight10:09
mvoChipaca: its a bit under speced right now10:09
mvoChipaca: we also probably want mispell there10:09
mvo(which should be trivial)10:09
Chipacaspeling is trivial indeed10:11
Chipacawe really struggle with 'temporarily'10:17
Chipacaso far i've found temporarely and temporariy10:17
mvoChipaca: *cough* s/we/I/10:23
zyga-ubuntumvo: I will miss standup. I need to do an X-ray now :-(10:24
zyga-ubuntuNot sure how soon I will be back today10:25
zyga-ubuntuI can work on reviews from my phone10:25
mvozyga-ubuntu: uhh, good luck10:26
pstolowskiyay, found a bug in go-udev10:27
zyga-ubuntupstolowski: that was quick10:27
pstolowskizyga-ubuntu: it's a small project and just a few functions really. and fix is trival thankfully10:30
Chipacahuh, there's a new rpi10:37
zyga-ubuntuYep10:38
zyga-ubuntuAnd it has a heat spreader on the chip10:38
zyga-ubuntuAnd does PoE with a hat10:38
zyga-ubuntuThat latter thing makes it really interesting but there is no information about the availability of the hat yet10:38
ChipacaI didn't know Poe used a hat10:39
* Chipaca imagines ravens with trilbies10:39
Chipacaanyway, bbiab10:40
zyga-ubuntuChipaca it is a very stylish poe10:51
* zyga-ubuntu waits for X-ray10:51
mupPR snapd#4839 opened: errtracker: respect the /etc/whoopsie configuration <Created by mvo5> <https://github.com/snapcore/snapd/pull/4839>10:55
zyga-ubuntumvo: reviewed11:02
abeatomvo, it looks like the root reason for the bug we chatted around yesterday is device cgroups: https://forum.snapcraft.io/t/race-condition-between-snapd-and-nm-when-providing-permissions/4451/311:03
abeatozyga-ubuntu, ^^11:03
zyga-ubuntuHey11:03
abeatozyga-ubuntu, o/11:03
zyga-ubuntuI’m AFK so sorry if I don’t respond quickly11:03
zyga-ubuntuLet me read the forum thread11:04
abeatothanks11:04
zyga-ubuntuSo a quick question11:04
zyga-ubuntuWhat created the additional device nodes11:04
abeatozyga-ubuntu, a script that needs to set some GPIO to power on the device. That takes a few seconds11:05
zyga-ubuntuIt feels like we are missing an udev rule that adds tags such devices11:05
zyga-ubuntuIs it udev tagged by any part of network manager interface?11:05
abeatothe udev rule is fine, it tags /dev/tty*11:05
abeatoit is in the ppp interface, used by NM11:06
zyga-ubuntuCan you please paste the udev profile generates by snapd for this snap11:06
abeatosure11:06
zyga-ubuntuThanks11:06
* zyga-ubuntu only has a phone with him11:06
abeatozyga-ubuntu, https://pastebin.canonical.com/p/jGBrPdkZZq/11:07
zyga-ubuntuAww11:08
zyga-ubuntuMy token is at home11:08
zyga-ubuntuIs it sensitive?11:08
abeatozyga-ubuntu, nah, not really: https://paste.ubuntu.com/p/sGWRcj7Ts4/11:08
abeatowho creates the device cgroup, snap-confine?11:09
zyga-ubuntuYes11:09
abeatoit would make sense that it adds only the present devices11:10
abeatoand if a new one appears, apparmor rules are fine thanks to udev, but not the device cgroup11:10
abeatowhich is not updated11:10
mborzeckire11:11
abeato(well, not thanks to udev, it is just that apparmor has a tty*)11:11
zyga-ubuntuRe11:13
zyga-ubuntuWell it should be updated11:13
zyga-ubuntuLook at those rules11:13
zyga-ubuntuWe run a tool to change the device cgroup as devices come and go11:13
zyga-ubuntuUnless we are hit by a bug11:14
zyga-ubuntuWe renamed that tool now11:14
abeatohm then a bug must be11:15
abeatowhich is that tool?11:15
zyga-ubuntuCan you confirm it exists in your system with the given path you see in the udev rule11:15
zyga-ubuntuRead the rule please11:15
zyga-ubuntuIt is raining11:15
zyga-ubuntuAnd typing on a phone is not easy11:15
abeatosure :)11:16
abeatook, /usr/lib/snapd/snappy-app-dev11:16
zyga-ubuntuYes11:19
abeatozyga-ubuntu, mvo is it possible that core in candidate has been recently upgraded? now I cannot reproduce the issue anymore, although it is a race so I am not 100% sure11:19
mvoabeato: candidate got updated on monday11:20
mvoabeato: from 2.31.1 to 2.31.211:20
abeatohm...11:20
mvoabeato: http://people.canonical.com/~mvo/core-changes/html/candidate/2.31.1r4110_2.31.2r4206.html11:20
* abeato trying to reproduce again11:21
zyga-ubuntuabeato: do you see this on core or on classic11:23
abeatocore11:23
zyga-ubuntuI see11:23
zyga-ubuntuHmm hmm11:23
zyga-ubuntuWell11:23
zyga-ubuntuIf it shows up please tell us11:23
abeatoyup11:23
zyga-ubuntuTo the best of our knowledge the system supports devices that are added after startup11:24
zyga-ubuntuOne possible thing to check for would be a race during cgroup construction11:24
zyga-ubuntuThat may indeed be buggy11:24
zyga-ubuntuI would have to look at the code to confirm this11:24
zyga-ubuntuPerhaps the tool should be grabbing the per-snap lock11:25
zyga-ubuntuAs that would probably rule out the race11:25
abeatoI can control when the device is created, will tell you if I can reproduce that way in 1min11:25
zyga-ubuntuSo to be clear11:25
zyga-ubuntuIf the udev event fires at the same moment as the first process of that snap11:26
zyga-ubuntuWe may have an issue in that specific case11:26
zyga-ubuntuOk11:26
zyga-ubuntuI have my X-ray but results won’t be available till 4PM, I’m heading home11:26
abeatozyga-ubuntu, definitely I can reproduce (ping me when you are back again)11:30
zyga-ubuntuDo can you reproduce that and capture udev logs as things happen?11:31
abeatoyes11:31
zyga-ubuntuPlease add all findings to the forum thread11:31
abeatowill do, thanks11:32
zyga-ubuntuAnd can you look at the resulting cgroup and ensure the devices are not there11:32
zyga-ubuntuI wonder if we can make the small helper tool log something11:32
abeatothat's how I checked that I reproduced it :)11:32
abeatothe new devices are not in devices.list11:33
abeato(and I got the EPERM too)11:33
zyga-ubuntuOk11:33
zyga-ubuntuWhich device is that? The serial port?11:34
abeato /dev/ttyACM0 from a modem11:34
mborzeckiabeato: is there any indication in dmesg that the port may go away and appear again?11:35
zyga-ubuntuThe rule looks ol11:35
zyga-ubuntuCan you check one more thing11:35
zyga-ubuntuRun the tool as udev would11:35
abeatomborzecki, it appears and it is kept there11:35
zyga-ubuntuPass the right tings (cannot tell you from the top of my head)11:36
zyga-ubuntuAnd see if it gets added11:36
abeatosure, let me try that11:36
abeatozyga-ubuntu, https://paste.ubuntu.com/p/hZFQ5XS62g/ :/11:39
zyga-ubuntuWoah11:40
zyga-ubuntuThat looks like a bug11:40
zyga-ubuntuIs the tool anywhere in the filesystem?11:40
abeato /lib/udev/snappy-app-dev11:40
abeatobad path11:40
jjohansenzyga-ubuntu: just a quick update before I bail, I have a patch thats is a wip, I should have something for you to test tomorrow (today?) sometime11:40
abeatozyga-ubuntu, so there is a bad path in the udev rule, that's it11:41
zyga-ubuntuThat is great news jjohansen. Thank you for getting this so quickly! I can test the patch when you have it11:41
zyga-ubuntuMvo: we need a small fix for what abeato just found11:42
zyga-ubuntuabeato: which version is that?11:42
abeatozyga-ubuntu, 420911:42
zyga-ubuntuAnd in snap —version parlance?11:43
mvozyga-ubuntu: oh, what fix is needed?11:43
zyga-ubuntujjohansen: is there any indication that the bug may have other side effects?11:43
abeatozyga-ubuntu, https://paste.ubuntu.com/p/tWkMJRP84Y/11:43
zyga-ubuntumvo: udev rules use a wrong path to the helper tool11:43
zyga-ubuntuWoah11:44
abeatomvo, /usr/lib/snapd/snappy-app-dev -> /lib/udev/snappy-app-dev11:44
zyga-ubuntuabeato your snapd is wonky11:44
zyga-ubuntuCan you reproduced that on vanilla 2.31.211:44
zyga-ubuntuI will be home soon11:44
zyga-ubuntu~40 minutes maybe11:44
mvoabeato: this is slightly strange, the snapd version is pretty out of sync with the snap version11:45
zyga-ubuntu(That not super soon though)11:45
abeatozyga-ubuntu, mvo well, it is the core from candidate, no hacks I promish11:45
zyga-ubuntuThat is cery ul ujęły11:45
zyga-ubuntuUnlikely11:45
zyga-ubuntuDid you push a locally built snapd over?11:46
mvoabeato: when I grep snappy-app-dev in master I get only a compat symlink hit, I'm a bit puzzled why its looking in /usr/lib/snapd/snappy-app-dev, is that in the udev rule? or where do you see it?11:46
zyga-ubuntuIt is in the udev rule but it looks like the snapd there is old or at least based on an old dirty build11:47
abeatozyga-ubuntu, hm, wait, I remember I pushed one snapd and mounted it on boot to measure a performance issue11:48
abeatolet me remove that11:48
zyga-ubuntuThank you!11:48
mupPR snapd#4840 opened: many: add `core.problem-reports.disabled` option <Created by mvo5> <https://github.com/snapcore/snapd/pull/4840>11:56
mborzeckiheh, go vet is flaky in 1.10 and even more so in master, this is what I get with master atm: https://paste.ubuntu.com/p/B2vc9F3CGr/ using a test case https://github.com/golang/go/issues/23701#issuecomment-36318722411:56
mvomwhudson: hm, do you have an opinion about go vet in bionic? we have some problems with 1.10 -^11:57
mvomborzecki: maybe the default of 1.10 is a bit too aggressive :/11:57
mborzeckithere will be an update to 1.10.1 though?11:58
mborzeckii think i'd prefer a slightly broken version if that helps from getting stuck with 1.9 for the next 5 years11:58
mvomborzecki: I guess, worst case is that we disable vet on 1.10 for now11:59
mborzeckimvo: the workarounds are there, we can revert them once we know it's fixed again11:59
abeatozyga-ubuntu, mvo after reverting the snapd version I see the right path in /etc/udev/rules.d/70-snap.network-manager.rules, so this was my fault. Sorry for the noise11:59
mvomborzecki: aha, ok. I thought you mentioned there are additional errors, am I outdated :) ?12:01
mvoabeato: no worries, thanks for reporting it, better this way than if we let a bug escape :)12:01
abeatoat least I've learned a bit about device cgroups ;)12:02
mborzeckimvo: i mean i tried go master, built it and was able to reproduced the problem using a test case posted in the original ticket, got a nice backtrace from go internals :P at least 1.10 did not panic12:02
zyga-ubuntuabeato: thank you for confirming that!12:04
zyga-ubuntumvo: alarm is over :)12:04
abeatozyga-ubuntu, thank you in fact12:05
zyga-ubuntuand I'm back12:08
zyga-ubunturesuming work on fixes12:08
mvomborzecki: ok12:08
mvozyga-ubuntu: thank you for handling it!12:08
mupPR snapcraft#1983 closed: snap: update revision of patchelf to use <Created by sergiusens> <Closed by sergiusens> <https://github.com/snapcore/snapcraft/pull/1983>12:26
mupPR snapd#4841 opened: snap/pack, cmd/snap: add `snap pack --check-skeleton` <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/4841>12:54
zyga-ubuntuChipaca: would you mind if I bother you with a symlinkat review later today?13:39
pstolowskiniemeyer: that's the project I was referring to: https://github.com/pilebones/go-udev13:39
niemeyerpstolowski: Yeah, that's one I looked into as well, given my browsing history13:39
niemeyerpstolowski: Looking for others now13:39
Chipacazyga-ubuntu: yes, I'd be very upset with myself if you asking me for a review bothered me13:40
Chipacaniemeyer: wrt the 'halp' pr (#4809) your comments on 'snap pack' seem to imply you want a change of behaviour of snap pack13:43
mupPR #4809: cmd/snap: tweak and polish help strings <Created by chipaca> <https://github.com/snapcore/snapd/pull/4809>13:43
niemeyerChipaca: Hmm.. it wasn't intentional at least13:43
niemeyerChipaca: What did I miss?13:43
Chipacai won't be working on that pr until this evening so there's no rush, but13:43
Chipacaniemeyer: 'snap pack' already defaults both arguments to '.'13:43
niemeyerChipaca: Huh, ok13:43
niemeyerChipaca: So what's the delta with try?13:44
niemeyerChipaca: That it doens't look for "prime"?13:44
Chipacaniemeyer: essentially yes, but also it only sets it to . if there's a meta/snap.yaml in .13:45
Chipacaand it only sets it to prime if there's a snapcraft.yaml in one of the usual places13:45
niemeyerChipaca: Okay, I indeed missed those details.. so +1 on your suggestion13:45
Chipaca(curiously it doesn't look for meta/snap.yaml in the prime directory, but that's a tweak)13:45
niemeyerChipaca: I think we should ignore snapcraft.yaml and just look for meta/snap.yaml as you suggest13:46
Chipacaniemeyer: it's still slightly awkard that it'll happily drop a snap inside the snap (and there are snaps with snaps in them in the store, so it's a real problem)13:46
niemeyer(in prime, in .)13:46
niemeyerChipaca: Is there a way to make snap pack ignore those?13:46
* niemeyer --help13:47
Chipacaniemeyer: not currently; snapcraft has an ignore list, but not snap pack13:47
ChipacaOTOH most people will be using it via snapcraft so it'll be fine :-)13:48
Chipacaanyway, glad that's cleared up, i'll tweak the text and the behaviour (but possibly in a followup)13:48
Chipacathis evening though; now to the cache13:48
Chipacahm, maybe, maybe target should default to the parent dir of snap-dir13:49
mborzeckiChipaca: or soem /tmp location13:49
Chipacamborzecki: it might not be the best example, but dpkg-buildpackage plops the debs in ..13:50
niemeyerChipaca: Yep, -e works13:50
niemeyerChipaca: Although it's completely undocumented apparently13:50
niemeyerChipaca: -e <filename>13:50
niemeyerChipaca: I suggest using that for any *.snap in the root directory13:50
Chipacaniemeyer: fair 'nuff13:50
niemeyerChipaca: I've seen actual snaps shipping the prior revision of their own snaps13:51
Chipacaniemeyer: IKR13:51
niemeyerChipaca: Built with snapcraft13:51
niemeyerSo it looks like a common issue13:51
niemeyerIn the rare circumstances where people do want to ship snaps, they can just put it in a subdir13:52
Chipacaniemeyer: https://i.imgflip.com/26d5u0.jpg13:53
niemeyerYeah :)13:53
* Chipaca forgets the whole conversation and moves to the hash cache stash13:53
cachiomvo, I have tried the upgrade test issue with the last bionic image in google and I see the same problem13:57
cachioall the snaps appear as broken13:57
cachiomvo, any idea?13:58
cachioI could get a debug session with the error13:58
Chipacamy / has inode number 214:03
Chipacahow14:03
* Chipaca doesn't really care and gets back to work14:03
mvocachio: yeah, happy to get into  a debug session14:04
mvocachio: what did you run to trigger it?14:04
cachiomvo, https://github.com/snapcore/snapd/pull/483514:05
mupPR #4835: tests: add bionic system to google backend <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/4835>14:05
cachiothis branch14:05
cachiomvo, but using this image14:05
cachioubuntu-os-cloud-devel/daily-ubuntu-1804-bionic-v2018030714:05
cachiowhich is the last one14:05
cachiomvo, nad run spread -debug google:ubuntu-18.04-64:tests/upgrade/basic14:05
mborzeckimvo: I'll close #4597 if you don't mind14:31
mupPR #4597: snapstate: allow core config via $core <Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/4597>14:31
mvomborzecki: ok, hope some of the code will still be useful for your work in this area14:39
mborzeckimvo: yes, i'll pull in your commit and work on top of it14:40
mupPR snapd#4597 closed: snapstate: allow core config via $core <Blocked> <Created by mvo5> <Closed by bboozzoo> <https://github.com/snapcore/snapd/pull/4597>14:42
mvomborzecki: \o/14:43
mupPR snapd#4842 opened: interfaces: make hash of apparmor profile for snap-confine in core part of the system-key <Blocked> <Created by mvo5> <https://github.com/snapcore/snapd/pull/4842>14:44
mvo^- this is to test the theory that the system-key needs extending to get rid of the errors we saw with 2.32 and the out-of-sync of the profiles, please ignore until green14:44
zyga-ubuntumvo: interesting14:44
mvozyga-ubuntu: yeah, it usually this is covered by the core rev but not in our tests14:45
mvozyga-ubuntu: lets see if its green or not14:45
kyrofaHey Chipaca, if/when you're able, can I get a sanity check review from you on a snapcraft PR? We're adding the ability to call functions from scriptlets which are then handled by the running Snapcraft instance, and we're doing it via fifos: https://github.com/snapcore/snapcraft/pull/200215:02
mupPR snapcraft#2002: pluginhandler: add builtin functions to scriptlets <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2002>15:03
Chipacakyrofa: fifos, as in fifo(7)?15:03
kyrofaYessir15:04
kyrofaRelevant parts of the PR are python (https://github.com/snapcore/snapcraft/pull/2002/files#diff-313e8dc0aa8d02f56e6ec918c25ddd93R74) and shell (https://github.com/snapcore/snapcraft/pull/2002/files#diff-8b247ab4b649deb9ee488e09837eba39R30)15:05
mupPR snapcraft#2002: pluginhandler: add builtin functions to scriptlets <Created by kyrofa> <https://github.com/snapcore/snapcraft/pull/2002>15:05
kyrofaWonder if it would be simpler with a more snapctl-like approach15:19
Chipacakyrofa: what does the 'status' variable get used for?15:23
kyrofaChipaca, to block15:23
kyrofaUntil the function has been handled15:23
kyrofaOtherwise once the "call" fifo has been read, the scriptlet goes on its merry way15:23
Chipacakyrofa: does this work?15:25
kyrofaChipaca, in my testing anyway, yeah. Do you see issues?15:26
Chipacakyrofa: only things that seem strange but they might be connected outside of my limited window into the code15:26
Chipacalike, cwd of the popen is not tempdir15:26
Chipacabut it finds the fifos15:27
Chipaca?15:27
kyrofaThe fifos are an absolute path, handed to the script run by the popen15:27
Chipacaah :-)15:28
Chipacahmmm15:31
Chipacakyrofa: another question: why nonblocking?15:33
kyrofaChipaca, on the Python side? So that the scriptlet can actually exit and Snapcraft can move on even if they don't utilize the fifos15:34
kyrofaOtherwise Snapcraft will block checking for a function call, and the scriptlet will exit in the meantime15:34
Chipacakyrofa: i'm having trouble hitting that failure mode15:41
cachioPharaoh_Atem, hey15:41
cachioPharaoh_Atem, do you haave time to continue with the dbus test?15:42
Pharaoh_Atemcachio: I will in ~20 minutes15:42
cachioPharaoh_Atem, great, thanks15:42
cachioPharaoh_Atem, I'll prepare the environment in that case15:43
Chipacakyrofa: granted my code is a butchering of yours just to test it (mostly because my experience with fifos is that they are finickity)15:43
kyrofaChipaca, that's exactly the kind of info I was hoping you'd have! This approach does not give you warm and fuzzies, then?15:45
Chipacakyrofa: it can work, and when it does it's brilliant15:45
Chipacakyrofa: and most of my bad experiences have been around things deadlockinig, which the nonblocking thing should avoid? I think15:45
Chipacaif i understand it correctly at least15:45
kyrofaIndeed. But your hacking is confusing, let me try removing the non-blocking and see if I can get it to fail like I expect, hold on15:46
Chipacabut, with it nonblocking you do end up spinning quite a bit15:46
kyrofaYeah totally15:46
kyrofaIt's not beautiful15:46
kyrofaYeah Chipaca remove ` | os.O_NONBLOCK` in `_NonBlockingFifo` and then run a YAML that looks like this: https://pastebin.ubuntu.com/p/V2tZ7GfzBn/15:50
kyrofaWhereas, if you add a `snapcraft_build` call to the bottom of the `build` scriptlet, things start working again since you wrote to the fifo to unblock snapcraft15:50
* zyga-ubuntu goes to pick up xray results, ttyl15:54
* Chipaca plays with this15:55
cachioPharaoh_Atem, env ready16:02
Chipacakyrofa: I'm not seeing it getting blocked16:03
kyrofaHuh. I can't explain that16:03
Chipacakyrofa: https://pastebin.ubuntu.com/p/xpPwjyvKQS/16:03
Chipacathat's with O_NONBLOCK commented out16:04
Chipacakyrofa: but it might not be picking up my changes16:06
kyrofaChipaca, how do you have it installed?16:06
kyrofavenv?16:06
Chipacakyrofa: i don't, i'm running it from the git repo + venv16:07
Chipacai don't think i installed it to the venv16:07
* Chipaca checks16:07
Chipacadangit16:07
Chipacayes it's there16:07
kyrofaAnd it's installed into the venv with --editable ?16:07
Chipacathere we go16:07
kyrofaOh phew16:08
Chipacakyrofa: somehow doing 'pip install -r requirements.txt .' (as per HACKING) also installed snapcraft itself16:08
Chipacai guess that's the . there :-)16:08
kyrofaIndeed. Although below is the guide for actually _developing_ using that method, which requires --editable16:09
kyrofaAlthough even that falls on its face sometimes16:09
kyrofaEvery once in a while I need to reinstall it and I never know why16:09
Chipacakyrofa: is there a way to tell it to do a clean and then a build?16:10
kyrofasnapcraft clean && snapcraft build16:10
Chipacaor to jfd a build?16:10
Chipacaeh, fair enough16:10
pstolowskiChipaca: thanks for staying vigiliant with serial-port interface regexp... completely overlooked what you spotted!16:16
Chipacapstolowski: no16:17
Chipacapstolowski: np*16:17
Chipaca:-)16:17
Chipacakyrofa: i need to go for a bit, but i'll carry on looking when ig et back16:17
* Chipaca afk16:17
pstolowskiChipaca: it looked very innocent :}16:18
kyrofaChipaca, of course, I appreciate your time!16:18
cachioPharaoh_Atem, there?16:42
Chipacapstolowski: IKR, I wouldn't've noticed if the regexp hadn't been blatently wrong16:53
Chipacakyrofa: *blush*16:53
popeyWhen do we plan to have overlay support landed?17:10
jjohansenzyga-ubuntu: no, no indication that the bug has other side effects (well it will break LXD checkpoint/restore). The symlink isn't getting properly updated on profile replacement, to point to the new blob.17:11
=== pstolowski is now known as pstolowski|afk
Chipacakyrofa: does this code need to run on non-linux?17:22
kyrofaChipaca, I don't believe so, no17:23
Chipacakyrofa: my suggestion about O_RDRW is non-portable, you see :-)17:29
kyrofaChipaca, are FIFOs portable?17:29
Chipacakyrofa: not to cp/m, dos, nor derivatives17:30
Chipaca:-)17:30
Chipacakyrofa: but to unices yes17:31
* kalikiana heading out o/17:48
Chipacaniemeyer: question about the approach to the cache18:01
Chipacaniemeyer: currrently asserts.SnapFileSHA3_384 calls osutil.FileDigest to get the digest and then asserts.EncodeDigest to format it (prepend sha3-384: etc)18:03
niemeyerChipaca: Ok18:04
Chipacaniemeyer: writing osutil.GetPathInfo, I can either make it receive a hasher (the caller would then wrap and pass in SnapFileSHA3_384), or i could move SnapFileSHA3_384 and EncodeDigest to osutil, or I could move GetPathInfo to asserts (or elsewhere)18:04
Chipacaniemeyer: the disadvantage of the hasher is that it makes it awkward to add more 'expensive' stuff, if everything is passed in18:05
Chipacabut otherwise it seems fine18:05
Chipacaniemeyer: so the question is, which approach do you think is best?18:07
niemeyerChipaca: Yeah, I think it should compute internally to make it convenient.. I'm just wondering if it's a bit of a weak design in that if we introduce further hashes, we'll probably want to read the file only once to do them all, instead of asking N hashing functions to each compute it again18:07
niemeyerChipaca: So perhaps neither? Where is the real sha3 algo living?18:08
Chipacaniemeyer: which is the 'real sha3'18:08
niemeyerChipaca: The thing that computes the sha3 digest given bytes18:08
* niemeyer digs in18:08
Chipacaniemeyer: osutil.FileDigest takes a generic crypto.Hash18:09
niemeyerChipaca: That's good.. we can cook a crypto.Hash which is actually multiple hashes easily, I suppose18:09
niemeyerChipaca: So the answer to my question above is golang.org/x/crypto/sha318:10
Chipacafwiw I think I need a thing that takes an io.Reader, not bytes18:11
Chipacabut that's minor18:11
Chipaca(otoh it's easy to multiplex readers to do multiple hashes)18:11
niemeyerChipaca: I suggest leaving asserts alone for now and just doing the low-level thing inside osutil, leveraging crypto/sha3 directly18:12
niemeyerChipaca: The cache can keep bytes too18:12
niemeyerChipaca: DeriveSideInfo can return the actual bytes too, unencoded18:13
mupPR snapd#4843 opened: interfaces/builtin: let MM change qmi device attributes <Created by alfonsosanchezbeato> <https://github.com/snapcore/snapd/pull/4843>18:14
niemeyerChipaca: FWIW, if we find a need to reuse EncodeDigest later, it should probably live inside strutil18:14
Chipacaniemeyer: I _think_ i understood what you meant :-)18:14
Chipacaniemeyer: basically make SnapFileSHA3_384 call osutil.GetPathInfo instead of osutil.FileDigest to get the digest18:15
Chipacai think that'll work, because i think everything uses that to get the hash18:15
Chipacai'll double check and report back if it's otherwise18:16
niemeyerChipaca: I don't think that will work18:16
Chipacaniemeyer: then i didn't understand what you meant18:16
niemeyerChipaca: Well, or maybe it will, if we reuse the StateCache type there as well18:16
Chipacaah, because it doesn't have the state :-/18:17
niemeyerChipaca: The original suggestion was to *return* the digest, but I guess your solution is cleaner actually18:17
Chipacaheh18:17
Chipacaniemeyer: the problem is what is 'the digest'18:17
Chipacaall these functions return 'the digest'18:17
niemeyerChipaca: Right.. but we can pass in the state as a StateCache interface as well18:17
Chipacait's just … different digests18:17
Chipaca:-)18:17
niemeyerChipaca: DeriveFooBar doesn't return any digests today18:18
Chipacaniemeyer: i never mentioned DeriveFooBar18:18
Chipaca;-)18:18
niemeyerChipaca: I did, both in the call and above18:18
niemeyer"Chipaca: DeriveSideInfo can return the actual bytes too, unencoded"18:18
niemeyerChipaca: This is the actual function that performs the digest inside firstboot18:19
Chipacaniemeyer: yes, but i'm not sure what the bytes would be useful for18:20
niemeyerChipaca: To cache them18:20
niemeyerChipaca: But... you're right18:20
Chipacai hate being right before understanding the thing18:21
Chipaca:-)18:21
Chipacaprobably a bit late in the day for me18:21
niemeyerChipaca: It's cleaner to pass the StateCache in as an interface, and call GetPathInfo inside it18:21
niemeyerChipaca: Perhaps osutil.ComputePathInfo instead18:22
niemeyerChipaca: So callers realize this is not a trivial op18:22
Chipacaniemeyer: HoldOnToYourTrowsersWereGoingToGetPathInfo18:22
Chipaca:-p18:22
niemeyerEven better :P18:22
Chipacayeah the name was a this-thing-needs-a-name token one, not the actual thing18:23
ChipacaComputePathInfo it is18:23
Chipacaand, eod for a while (probably be back later to play with help strings)18:24
niemeyerChipaca: Enjoy18:25
mupPR snapd#4837 closed: many: go vet cleanups <Created by bboozzoo> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/4837>20:04
cachioniemeyer, could you please take a look to this one? https://github.com/snapcore/spread/pull/5220:15
mupPR spread#52: Add storage option for google <Created by sergiocazzolato> <https://github.com/snapcore/spread/pull/52>20:15
cachioit is making fail the ubuntu-16.04-32 on google20:17
Chipacaniemeyer: ping21:02
mupIssue snapcraft#1659 closed: Implement support for supported bases <Created by sergiusens> <Closed by sergiusens> <https://github.com/snapcore/snapcraft/issue/1659>21:21
mupPR snapcraft#1993 closed: core: initial support for bases <Created by sergiusens> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/1993>21:21
niemeyerChipaca: Hyea21:23
Chipacaniemeyer: noodling around with this stuff, it looks like I should give PathInfos a Rename21:23
Chipacaniemeyer: at which point I'm not sure PathInfo is the right name for the beast21:23
* Chipaca goes back to noodlin' and lets niemeyer worry about names21:24
niemeyerChipaca: I'm not sure about what other name might be appropriate though.. we're effectively associating a bag of knowledge with a particular path21:24
Chipacaniemeyer: HashedFile?21:27
Chipacaeh, probably not or it'll grow Open() and stuff21:27
Chipacaand I guess Rename is actually a method of the hash cache, not the path info21:28
Chipacahrmph21:28
Chipaca(the rename thing is because in quite a few places (at least two so far :-) ) we hash just before moving the file_21:29
Chipacathe download with deltas path is really interesting21:29
Chipacaapply delta hashes the composed file to make sure it's ok21:29
Chipacaand if so it's treated like a complete partial21:30
Chipacaso it's hashed to make sure it's ok21:30
mupPR snapcraft#2003 opened: patches: make the ctypes patch more robust and add armhf arch triplet <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2003>21:30
Chipacaand that's in store, before hashing it to make sure it's ok21:31
Chipaca:-)21:31
* cachio afk21:40
niemeyerChipaca: Renaming the path info doesn't sound too bad21:55
Chipacaniemeyer: yeah, i was originally thinking of having it do the actual rename21:55
Chipacabut it'll just rename the cache and be happy21:55
niemeyerChipaca: The association is still with the path, and it's not just a hash.. we want mtime, size, etc, as well21:55
niemeyerChipaca: Ah, agreed.. it should act just on the cache21:55
mupPR snapcraft#2004 opened: errros: add a specific error when running commnds from plugins <bug> <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2004>21:57
cachioniemeyer, hey23:23
niemeyercachio: Heya23:23
cachioniemeyer, when you have 5 minutes, could you please take a look to https://github.com/snapcore/spread/pull/52 ?23:24
mupPR spread#52: Add storage option for google <Created by sergiocazzolato> <https://github.com/snapcore/spread/pull/52>23:24
cachioniemeyer, I have 1 PR waiting for this change23:24
niemeyerACk23:25
cachioniemeyer, tx23:25
cachioniemeyer, something else23:28
cachioniemeyer, I see https://paste.ubuntu.com/p/2Wd5dQX6WS/ after a reboot in debian 9 in google23:28
cachioit is like spread tries to connect and for some reason pam is closing the session for the root23:29
cachiothe sshd config is the same we have in linode23:29
cachiodid you see that before?23:29
mwhudsonum23:44
mwhudsonsubiquity is failing to start in a live cd with "cannot create user data directory"23:44
mwhudsonwhat can possibly have changed since this worked yesterday23:44
* cachio eod23:49

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!