/srv/irclogs.ubuntu.com/2018/08/17/#snappy.txt

solrachello00:31
mborzeckimorning05:05
mupPR snapd#5664 closed: interfaces: workaround for activated services and newer DBus <Created by jdstrand> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5664>05:30
mborzeckimvo: morning05:36
mvohey mborzecki05:37
mborzeckimvo: regarding experimental.<flag>= we seem to have a problem there05:37
mvomborzecki: uh, ok, tell me more05:37
mborzeckimvo: it's actually quite silly https://paste.ubuntu.com/p/tVGVZjstcm/05:37
mvomborzecki: ohhh, i see05:38
mvomborzecki: its because of the bool i guess05:39
mborzeckimvo: yeah, should be a quick fix05:39
mvomborzecki: funny, so it accepts null as false05:39
mvomborzecki: but not ""05:39
mvomborzecki: thanks05:39
mborzeckimvo: null == nothing to unmarshal, so a bool stays in its default value05:39
mvomborzecki: aha, nice05:40
mvomborzecki: wasn't aware of this05:40
mborzeckimvo: even funnier, the corecfg code unmarshals to a string and allows ""05:41
mupPR snapd#5675 opened: overlord/snapstate: improve feature flag validation <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5675>06:23
mborzeckimvo: ^^06:23
mborzeckimvo: updated #5671 too06:31
mupPR #5671: tests: basic test for parallel installs from the store <Parallel installs> <Simple> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5671>06:31
mvomborzecki: yay, thank you06:43
zygagood morning07:00
zygaI'm not really here, just checking which office to go to handle the car paperwork07:00
mvozyga: good morning07:01
mborzeckimvo: that initialiazation is for the case when there is nothing to unmarshal or null (cause json, null is untyped :))07:01
mvomborzecki: aha, ok07:01
mborzeckizyga: hey, so you're off-really-off today or just regular off?07:01
zygamborzecki: I'm swapping for the weekend07:02
mborzeckizyga: if the former we can chat about s-c on monday07:02
zygamborzecki: but after I handle the paperwork I will return07:02
zygamborzecki: and we can chat07:02
zygaor we can chat straight away now since I'm here07:02
mborzeckihaha so the 'regular off' ;)07:02
zygahaha, so that's what you meant by "regular off" :D07:02
zygaI guess that's only fair :D07:03
mborzeckioff-but-not-off07:03
=== pstolowski|afk is now known as pstolowski
pstolowskimorning07:09
mborzeckipstolowski: heyah07:23
Chipacamoin moin07:31
Chipacais the lxd snap busted?07:32
mborzeckigoogle:ubuntu-16.04-32:tests/main/lxd seems to be failing07:32
Chipacayeah07:32
Chipacasame here07:32
mborzeckiChipaca: pedronis: hellos07:32
Chipacamborzecki: hi07:32
mvohey pstolowski and Chipaca07:32
Chipacamvo: o/07:33
mvoChipaca: I added some comments to the dump-db PR, I think a slightly more generic format for the output would be nice. I'm thinking about the field spearator, \ff is used right now07:34
Chipacamvo: yep, and wotsisname said it'd be fine07:34
mvoChipaca: do you think ":" is reasonable? or shall we go with something else?07:34
Chipacamvo: an emoji would be frouned on, i  guess07:35
Chipaca: is reasonable07:36
mvoChipaca: heh, ok07:36
pedronismvo: I'm looking again at the changes in  device_asserts.go and now I'm very confused07:40
mvopedronis: can I help fix that somehow?07:41
pedronismvo: this should go away no:   https://github.com/snapcore/snapd/blob/master/asserts/device_asserts.go#L248 ?07:41
mvopedronis: yes, sorry, that was a oversight, let me kill it (with fire)07:42
pedronismvo: why is this here and not one level up:  https://github.com/snapcore/snapd/blob/master/asserts/device_asserts.go#L163 ?07:42
pedronismvo:  the new branch needs a similar check for gadget, no?07:42
pedronismvo: checkModel means check the "model" header07:43
mvopedronis: indeed, let me fix that too07:43
mvopedronis: plus some gadget track error tests are missing (which of course would have found the issue)07:44
pedronismvo: yea,  I found these because I had the nagging feeling that something was missing in the new PR, it was too short :)07:45
pedronisand so I went back to see what we did for kernel07:45
mvopedronis: thanks for noticing! I will generalize it a bit, I get the feeling that this will come again07:46
mvopedronis: should I split the PR up?07:46
pedronismvo: as your prefer07:46
mvopedronis: ok, I think I do that then07:47
mvopedronis: do you have an opinion about "Gagdget() SnapWithTrack" vs "Gadget() string and GadgetTrack() string" ?07:47
pedronismvo: I think I prefer the latter until we can do something outside of asserts07:52
mupPR snapd#5669 closed: asserts,image: support gadget tracks in the model assertion <Core18> <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/5669>07:53
mvopedronis: ok, thats fine, its easy enough to fix later especially if/when we get support for this in snap install07:55
mupPR snapd#5676 opened: asserts: add support for gadget tracks in the model assertion <Created by mvo5> <https://github.com/snapcore/snapd/pull/5676>08:03
mvopedronis: the first part -^08:03
pedronismvo: reviewed08:10
mvopedronis: yay, thank you08:12
mupPR snapd#5654 closed: cmd/snap-confine: establish snap directory mappings for parallel instances <Parallel installs> <Created by bboozzoo> <Closed by bboozzoo> <https://github.com/snapcore/snapd/pull/5654>08:15
mupPR snapd#5677 opened:  image: add support for "gadget=track" <Created by mvo5> <https://github.com/snapcore/snapd/pull/5677>08:16
mupPR snapd#5678 opened: snapstate: add support for gadget tracks in model assertion <Created by mvo5> <https://github.com/snapcore/snapd/pull/5678>08:16
pedronismvo: this is are all for 2.35, right?08:17
mvopedronis: correct08:18
mvopedronis: I added tags now08:18
Chipacaso08:19
Chipacaselftest is failing in lxd in 16.04-3208:20
mvoChipaca: uh, what is the error?08:21
mvoChipaca: I mean, what part of the selftest fails?08:21
Chipacaerror: cannot start snapd: cannot mount squashfs image using "squashfs": mount: /tmp/selftest-mountpoint-487148902: mount failed: Unknown error -108:21
Chipacalxd fails to start the first time with an error, and it restarts and doesn't print the logs leading up to the error either, which is suspicious08:22
ChipacaAug 17 09:12:18 autopkgtest lxd.daemon[18103]: ==> Setting up persistent shmounts path08:22
ChipacaAug 17 09:12:18 autopkgtest lxd.daemon[18103]: ====> Making LXD shmounts use the persistent path08:22
ChipacaAug 17 09:12:18 autopkgtest lxd.daemon[18103]: ln: failed to create symbolic link '/var/snap/lxd/common/lxd/shmounts': No such file or directory08:22
zygaohhh drat, I need to remove the lxd quirk in 1808:23
zygaor maybe I did08:23
zygathe lxd quirk is applied on all classic systems08:24
zyga_hmmm_08:24
* Chipaca goes for more coffee08:24
mborzeckiChipaca: root@my-ubuntu:~# systemd-detect-virt --help09:03
mborzeckibash: /usr/bin/systemd-detect-virt: Numerical result out of range09:03
mborzeckiChipaca: that's inside lxc container09:03
Chipacamborzecki: why does that start with bash:?09:04
Chipacamborzecki: i mean, that's a bash error?09:04
mborzeckiChipaca: heh, beats me, no clu09:04
Chipacayes09:04
Chipacamborzecki: it's an error from bash09:05
Chipacabecause09:05
Chipaca*EXECVE RETURNED THAT*09:05
mborzeckiChipaca: root@my-ubuntu:~# /usr/bin/systemd-detect-virt --container09:05
mborzeckibash: /usr/bin/systemd-detect-virt: Numerical result out of range09:05
mborzeckiomg09:05
Chipacaexecve("/usr/bin/systemd-detect-virt", ["/usr/bin/systemd-detect-virt"], [/* 12 vars */]) = -1 ERANGE (Numerical result out of range)09:05
mborzeckiChipaca: so if that fails, useFuse() => false, mount is done with -t squashfs which fails09:05
mborzeckiroot@my-ubuntu:~# mount -t squashfs $PWD/data.squashfs /mnt/09:05
mborzeckimount: /mnt/: mount failed: Unknown error -109:06
mborzeckiChipaca: like this ^^09:06
ogrado you have squashfuse installed ?09:06
Chipacamborzecki: it gets more interesting09:06
Chipacamborzecki: do a getcap of systemd-detect-virt09:06
mborzeckiChipaca: Failed to get capabilities of file `/usr/bin/systemd-detect-virt' (Numerical result out of range) ?09:07
Chipacamborzecki: yes09:08
Chipacastgraber: in 16.04 i386 (only), installing lxd from stable and launching an unprivileged container results in weirdness: /usr/bin/systemd-detect-virt fails to execve, returning ERANGE09:43
Chipacasparkieg`: is that a typo for a german war on spas09:50
=== sparkieg` is now known as sparkiegeek
sparkiegeekChipaca: heh, glitch in the matrix, combined with a non-friendly unique-naming scheme in my IRC client :)09:50
Chipacasparkiegeek: you could've gone with 'yes'09:51
mborzeckiguys, how abou we disable tests/main/lxd on *-32 until this is resolved?09:54
pedronisChipaca: ah, interesting, I was wondering if systemd-detect-virt coul fail when ways that weren't just I'm not a container09:54
pedroniss/when/in/09:55
pedronisChipaca: I might have even asked mvo at some point to put more defensive code around it09:55
Chipacapedronis: I doubt it's systemd-detect-virt itself09:55
Chipacapedronis: it never gets to have a say in the matter09:55
Chipacapedronis: (the execve call fails)09:56
pedronisChipaca: ok, but our code anyway assumes it means we are not virtualized ?09:56
Chipacayes, yes it does09:56
pedronisthat was more my point09:56
pedronisanyway09:56
Chipacawe should probably bail there instead of assuming tbf09:57
mborzeckimaybe we should bubble the error up09:57
mborzeckiotherwise it's rather cryptic while anything fails at this stage09:57
Chipacamborzecki: dunno, stgraber is often up really early09:59
mborzeckiChipaca: i can open a PR and we can close it if a solution is found soon(ish)10:00
Chipacawhat's VGAuthService10:01
Chipacanm10:02
Chipacamborzecki: sure10:02
mborzeckidamn, that test has a whitelist of systems10:07
Chipacafun fact: byobu-config will lock up the whole everything10:10
mupPR snapd#5679 opened: tests/main/lxd: run ubuntu-16.04 only on 64 bit variant <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5679>10:10
* Chipaca was looking to see if any other binaries failed to exec in the same way10:11
mborzeckiChipaca: anything else failed?10:18
Chipacamborzecki: my patience10:18
mborzeckiheh ;)10:18
ChipacaI should probably step away from the forum for a bit10:20
mborzeckianyone feels like looking at https://github.com/snapcore/snapd/pull/5614 ?10:24
mupPR #5614: interfaces: parallel instances support, extend unit tests <Parallel installs> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5614>10:24
mupPR snapd#5680 opened: [RFC] hotplug: handling of simple add/remove scenario <Blocked> <Hotplug> <Created by stolowski> <https://github.com/snapcore/snapd/pull/5680>10:25
pstolowskiuh, what10:25
pstolowski(about byobu-config)10:26
Chipacapstolowski: inside a snapped lxd, inside kvm, inside spread, running 'byoby-config --help </dev/null >/dev/null' locks the whole thing up10:28
Chipacabyobu*10:28
pstolowskifinny10:29
pstolowski*funny10:29
Chipacaviry finny. hilirius, ivin10:29
pstolowskijust checked the dictionary in case the word finny exists and means something. not in my dict ;)10:30
Chipacapstolowski: 'abounding in fishes', fwiw10:30
Chipacapstolowski: http://www.dict.org/bin/Dict?Form=Dict1&Query=finny&Strategy=*&Database=*&submit=Submit+10:31
pstolowskiaha10:32
Chipacasergiusens: poing10:32
Chipacawhy do people sell things they call "Ubuntu" with just random crap running as the kernel10:51
Chipaca>:-(10:51
ograwell, its a massive improvement ... the last openvz servers i saw (when deugging something similar together with zyga) was 2.x or early 3.x10:52
ograsurprisingly openvz finally supports 4.x kernels :)10:53
Chipacaogra: that's why snapd ships a test squashfs :-)11:00
Chipacaogra: https://github.com/snapcore/snapd/blob/master/selftest/squashfs.go#L5511:00
niemeyerChipaca: So looking at snapshotstate, the last missing point is the last id name11:01
niemeyerChipaca: "last-snapshot-set-id"?  It's a mouthful, but has precedence in other options11:01
Chipacaniemeyer: what do you mean?11:01
niemeyerChipaca: The "snapshots.last-id" thing, and the comment from me and pedronis in the PR11:02
Chipacaah11:02
Chipacasnapshots.last-set-id is what it is now11:02
Chipacawhich seems alright to me, if we need to add more info about snapshots it won't be out of place in there11:04
Chipacadunno11:04
Chipacaniemeyer: I think both approaches are fine (I mean: snapshots.last-set-id is fine, and a toplevel last-snapshot-set-id is fine; anything more structured and I'm going to call YAGNI on it)11:07
Chipacaexactly which names are best, I don't know11:07
niemeyerChipaca: I think pedronis had a point about "snapshots" generally being a map of actual snapshots per other cases11:08
niemeyerChipaca: And we indeed have the last-foo-bar-id case already in other places11:08
niemeyerlast-refresh, last-refresh-hints, last-change-id, last-task-id11:09
niemeyer"ubuntu-core-transition-last-retry-time"11:09
niemeyer/o\11:10
mvodid we figure out more about the lxd issue btw?11:12
Chipacamvo: <Chipaca> stgraber: in 16.04 i386 (only), installing lxd from stable and launching an unprivileged container results in weirdness: /usr/bin/systemd-detect-virt fails to execve, returning ERANGE11:12
mvoChipaca: heh, woah, ERANGE11:12
Chipacamvo: getcap of the file _also_ fails with ERANGE11:12
Chipacamvo: so we're about to learn something about _something_11:13
mvoChipaca: what a surprising error11:13
mvoChipaca: yeah, its amazing11:13
niemeyerChipaca: I've +1d assuming that's tuned per agreement.. someone else needs a final +1 too11:13
niemeyerpedronis ^11:13
Chipacaman, i'm shaking11:14
Chipacawhoa11:14
Chipacaok11:14
Chipacaniemeyer: thanks11:14
pedronisChipaca: mvo: yes, ERANGE usually makes me think of math libraries11:14
ChipacaI didn't expect the emotional response from myself ¯\_(ツ)_/¯ good thing i'm going on holidays next wednesday :-)11:14
mvoChipaca: uhhhh, snapshot going in?11:14
=== pstolowski is now known as pstolowski|lunch
Chipacamvo: got a +1 from niemeyer11:15
mvopedronis: heh, exactly11:15
Chipacaooooh, somebody's jealous :-p11:15
ograChipaca, dmesg -H means he needs to understand that he is in a pager ... i'd have suppressed the -H11:15
Chipacaogra: maybe TERM isn't set or something stoopid11:15
ograor that, yeah11:16
pedronismvo: sorry for being annoying about context,  but is really mostly meant to have  a place talk back to itself or connected places talk between unrelated or user layers11:16
pedronisI mean it's Value feature11:17
pedronisniemeyer: yes either me or mvo need to do a 2nd pass of snapshotstate11:18
pedronisany consistent reason why all newish PR seems to be red ?11:20
Chipacapedronis: lxd11:20
Chipacapedronis: 16.04-32 lxd errors11:20
pedronisthe ERANGE issue ?11:21
Chipacapedronis: yes11:21
ChipacaEDERANGED11:21
pedronisfun :(11:21
ChipacaERANGE is not even listed as a return value for execve11:21
Chipaca:-)11:21
Chipaca(but it's probably something to do with the xattrs)11:22
Chipacaif I had to guess, I'd guess that11:22
Chipacabecause systemd-detect-virt is one of the very few files in 16.04 that uses caps (via xattrs)11:22
Chipacain fact, i should look at the other ones, d'oh11:22
* Chipaca does that11:22
Chipacapedronis: dunno if you noticed but mvo wasn't online when you were apologising to him11:24
pedronisno11:24
Chipacapedronis: maybe you just wanted to get it off your chest :-D11:24
mborzeckihttps://github.com/snapcore/snapd/pull/5679 shall we pull the trigger?11:25
pedronismaybe I didn't complete his nick11:25
mupPR #5679: tests/main/lxd: run ubuntu-16.04 only on 64 bit variant <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5679>11:25
ogra"The linux beginners course with ogra and Chipaca"11:28
ograsession 1 ...11:28
ogra:)11:28
Chipacaogra: "If you survive with both your kidneys, [...]"11:29
ogralol11:29
Chipacaogra: as an aside, what on earth have they done to that poor "Ubuntu" that dmesg doesn't work11:30
ogragood question11:31
ogralol11:32
ogra"no entries"11:32
ograprobably he run no kernel at all !!!!11:32
Chipacaogra: it's secretly just running WSL11:33
ChipacaYESS11:38
Chipacait's the capabilities11:38
Chipacamtr fails in the exact same way11:38
Chipacaas does traceroute6.iputils11:39
sergiusensChipaca: what's up?11:40
mvoChipaca: should I merge 5679 or is the solution so close that its not worth adding the workaround?11:41
Chipacamvo: NFI about the solution -- merge away11:41
Chipacasergiusens: WHY DIDN'T I GIVE MYSELF MORE CONTEXT WITH THE PING :-(11:41
Chipacasergiusens: now I have _no_ idea what it was about11:41
Chipacait was, like, six context switches ago11:41
mupPR snapd#5679 closed: tests/main/lxd: run ubuntu-16.04 only on 64 bit variant <Created by bboozzoo> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5679>11:42
Chipacasergiusens: I hope I'll remember and ping you again11:42
Chipacaoh!11:42
Chipacasergiusens: i remembered :-D11:42
Chipacasergiusens: were you aware of 'snap watch --last=auto-refresh?'11:42
Chipacasergiusens: coming in 2.3511:43
sergiusensChipaca: yes we are https://github.com/snapcore/snapcraft/blob/master/snapcraft/internal/build_providers/_snap.py#L26311:45
sergiusensbut I think I might want to disable refreshes for a lot longer to not get killed mid build :-)11:45
Chipacasergiusens: no no11:45
Chipacasergiusens: the question mark at the end of the change type11:45
Chipacasergiusens: means "no error if none found"11:46
Chipacasergiusens: or from --help, “A question mark at the end of the type means to do nothing (instead of returning an error) if no change of the given type is found.”11:46
sergiusensChipaca: oh, then I can get rid of the suppress. I must say though that that syntax is hard to spot given you phrased the sentence as a question :-)11:46
Chipacasergiusens: mbuahaha11:47
Chipacasergiusens: (sorry)11:47
sergiusensand I did an improper quote match11:47
Chipacatut tut11:47
* sergiusens needs to put his glasses on11:47
Chipacasergiusens: anyway, 2.35+, so you probably can't use it yet11:47
sergiusensChipaca: still good to know!11:47
Chipacasergiusens: but, with our conversation abour --check-skeleton  from the other day i thought i'd call it out to you11:47
sergiusensall is good :-)11:48
Chipacaogra: wasn't there a file in proc to tweak the kernel log level? we could ask this person to try that11:51
ograChipaca, not sure if in /proc ... there is a sysctl setting you can apply though11:52
ogramvo, this is an interesting one https://forum.snapcraft.io/t/set-system-proxy-from-custom-snap-service/692611:52
Chipacaogra: sysctl is just writing to /proc/sys/<thing>11:52
Chipacaogra: :-)11:52
ograah, indeed11:52
ograso yeah, there is one, i just dont know the node then ;)11:53
Chipacaogra: but yes better to present it with sysctl11:53
ograshould be something about log_level11:53
Chipacaogra: sys.kernel.printk ?11:54
ograChipaca, /proc/sys/kernel/printk11:54
ograyeah11:54
Chipacaheh11:54
ogra$ cat /proc/sys/kernel/printk11:55
ogra441711:55
Chipacaogra: I did not point them to https://i.imgur.com/Pfr9dj0.jpg !  I think I deserve a cookie.12:04
* ogra hands Chipaca a well deserved cookie :D12:05
ograi wonder if he paid for that carp ...12:06
Chipacaogra: at lunch time? are you mad?!?12:06
ogra*crap12:06
Chipaca:-)12:06
ograhahaha12:06
Chipacaogra: 8GB of ram? you betcha it was paid for12:06
Chipacaalthough given they said it was Ubuntu and it wasn't, maybe it's "8GB" of "RAM"12:06
Chipaca(actually just a big swap file on a 1GB netbook)12:07
ograhaha12:07
mborzeckizyga: snap-update-ns is looking good, did a change, it's actually surprisingly simple12:12
zygawoot, that's great12:12
mborzeckizyga: you know, i might have screwed something up there too :P12:14
zygaI'll check next week :)12:14
jdstrandzyga: hey, fyi, responded to https://github.com/snapcore/snapd/pull/564412:15
mupPR #5644: interfaces: add audio-playback/audio-record and make pulseaudio manually connect <Created by jdstrand> <https://github.com/snapcore/snapd/pull/5644>12:15
zygajdstrand: thank you12:15
zygajdstrand: I'm swapping today, doing office move and legal paperwork12:15
mborzeckizyga: https://paste.ubuntu.com/p/YZ7w7RKw5d/ mountinfo (does not include $SNAP_USER_DATA yet)12:15
jdstrandzyga: np. I suspect you'll just agree with me and ack12:15
zygaI'll look quickly now :)12:16
jdstrandzyga: but by all means, exercise your day off :)12:16
mupPR snapd#5675 closed: overlord/snapstate: improve feature flag validation <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/5675>12:16
jdstrandChipaca: fyi, I think that hostnamectl issue will be resolved if the PR merges from trunk12:16
jdstrandChipaca: and hi :)12:17
Chipacajdstrand: I was assuming as much :-) hi12:17
Chipacajdstrand: excitement now is about lxd on 16.04-32 being unable to execve files that use capabilities12:18
* Chipaca ~> lunch12:19
=== pstolowski|lunch is now known as pstolowski
ograpedronis, Chipaca https://imgur.com/a/ZFNu5pV ... finally able to reproduce ... capturing logs now12:23
zygajdstrand: +112:23
jdstrandzyga: thanks :)12:24
zygamarked as such on the PR12:24
mvoogra: aha, you reproduced the shutdown hang?12:26
mborzeckidamn, the trackpoint left click in my x220 is starting to fail :(12:28
ogramvo, yeah12:29
ogramvo, pedronis Chipaca https://pastebin.canonical.com/p/DGKBDMzQ2r/ logs (filtered out binder and anbox audit messages since they ake it unreadable)12:30
ogra*make12:30
pedronisinternal shutdown seems correct12:33
pedronisso it would some handshake with systemd problem12:33
pedronisseems we get a sigterm but don't do the right thing:  snapd.service: State 'stop-sigterm' timed out. Killing.12:34
* mvo wonders if anything has chnaged here12:36
pedronismvo: well it might have been like since a while12:37
pedronismvo: it's related to the waiting we do on reboot and signal unhandling12:38
ogranote this is all edge plus a devmode daemon (anbox) though i see the daemon being killed several lines before the snapd timeout shows up12:38
ogra(also not sure if a misbehaving snap could actually make snapd not stop)12:38
pedronismvo: we also added the watchdog12:38
mvopedronis: aha, thats a good one12:39
mborzeckipedronis: Aug 17 12:19:55 localhost.localdomain snapd[1005]: daemon.go:577: Waiting for system reboot ?12:39
pedronismborzecki: ?12:39
mborzeckiisn't snapd waiting in a long sleep here?12:39
pedronisyes12:39
pedronisas I said signal handling is not quite right over shutdown12:39
mborzeckiso any signals are not really handled12:39
pedronisyes12:39
pedroniswhat I'm trying to say12:39
pedronisnot sure it's related to the timeout though12:39
pedronisif it's really much later12:39
pedronismvo: we need to call Reset or Stop for the signal handling we setup in main.go, not sure exactly where12:41
ograif you want to repro: create a qemu VM with an image from edge ... leave it off over night so there is a new core ... make sure to start it only after core has updated in the store... boot it and watch it to do an auto-refresh with that hang12:42
ograi have never seen it when doing a manual refresh12:42
mborzeckipedronis: i'd assume it's related, then things start to make sense, term get queued, if systemd gets a request to restart the process it would make sense to ignore it since the system is going down anyway, systemd timeouts waiting for snapd to exit, snapd would timout waiting for reset :)12:44
pedroniswell, not from the logs  it seems systemd then kills snapd12:45
pedronisso snapd doesn't timeout12:46
mvoyeah, I'm puzzled12:46
pedronisogra timeout is systemd saying something about snapd again12:46
pedronisbut maybe I'm confused12:46
mvoif snapd does not handle sigterm correct I should be able to simulate this by simply kill -TERM $(pidof snapd)12:46
mvoand that exist normally12:46
pedronismvo: this is about reboot mode12:46
pedronisnot normal running snapd12:46
ograpedronis, i'm seeing exactly whats in the imgur png12:47
pedroniswe are inside daemon Stop at that point12:47
pedronismvo ^12:47
mvopedronis: ok12:47
pedronisI mean,  I know what we need to do about sigterm (except exactly how to place it)12:47
pedronisnot sure it fixes what ogra sees12:47
mvopedronis: that makes sense, we are in stop at this, so yeah12:48
pedronisah, it says A Stop Job so yes, it's related12:49
Chipacapedronis: sorry to sidetrack you, but did you see Gustavo's comment about last-snapshot-set-id?12:49
pedronisyes12:49
Chipacapedronis: +1?12:49
pedronisI agree with that12:49
Chipacapedronis: k12:49
pedronismvo: something like this perhaps (untested):  https://pastebin.ubuntu.com/p/rYWJ8PR5HH/12:56
mvopedronis: yeah, that looks sensible12:58
mupPR snapd#5318 closed: interfaces/builtin: add new cuda-support interface <Created by anonymouse64> <Closed by jdstrand> <https://github.com/snapcore/snapd/pull/5318>13:13
pedronismvo: I can try to cut a PR from that pastebin on monday I suppose unless you want to13:42
mvopedronis: I can look at this while waiting for travis to biuld my gadget-track PRs13:44
pedronisok13:44
mvopedronis: this mostly needs tests, right? the feature itself looks reasonalbe13:44
pedronisyes, some kind of tests (not sure it's easy to test)13:45
mvopedronis: yeah, it looks tricky, maybe I can manage some sort of integration test at least, I will poke a bit at it13:46
pedronisa 2nd review for #5676 would be great13:47
mupPR #5676: asserts: add support for gadget tracks in the model assertion <Core18> <Created by mvo5> <https://github.com/snapcore/snapd/pull/5676>13:47
mvoyeah, was about to ask for this - should be straightforward (now)13:48
=== cory_fu_ is now known as cory_fu
Chipacastgraber: ping?13:55
stgraberChipaca: pong13:56
Chipacastgraber: I don't know if you saw that we're having issues with lxd since the release yesterday?13:56
stgraberChipaca: I just read your comment about systemd-detect-virt on i386, will take a look13:57
Chipacastgraber: it's any executable that uses capabilities13:58
Chipacastgraber: systemd-detect-virt, mtr, and traceroute6.iputils13:59
Chipacastgraber: but, on 64 bits, snapcraft is also having trouble with snapd dying, that goes away with switching to 3.013:59
stgraberChipaca: what kernel are you on?14:00
Chipacastgraber: e.g., on travis, «apt install snapd; snap install snapcraft --classic; snap install lxd; mkdir project; cd project; snapcraft init; snapcraft cleanbuild» fails14:00
Chipacanot sure what travis is using14:00
ChipacaI'm on 4.4.0-131-generic; mborzecki is on something newer I think14:00
Chipacaand the spread runs on 16.04-32 are on fresh cloud images, whatever's shipped there14:01
Chipaca(I can track it down if it's important)14:01
mborzeckii was on whatever we used in spread instances14:01
Chipacaoh wait my machine is 4.4.0-131-generic but the test was run in kvm14:02
ChipacaI'd have to check what was there :-)14:02
* Chipaca checks14:02
mborzeckii'll start the test and check what's used in gce14:03
Chipacastgraber: 4.4.0-133-generic14:04
Chipacarunning on a 4.4.0-131-generic14:04
stgraberChipaca: reproduced the issue14:07
Chipacastgraber: should we be installing lxd from candidate in our integration suite, to catch this family of errors before they hit stable?14:08
stgraberChipaca: that may be useful, yeah, this is definitely related to fscaps in this case so our own tests didn't trip on that14:09
Chipacastgraber: also mborzecki noticed that a privileged container didn't have this issue14:09
stgraberyeah, that part would be expected14:10
Chipacaok14:10
Chipacastgraber: does it only failing on 32 bit intel make sense to you?14:10
Chipaca(this particular failure i mean -- the snapcraft one i am yet to dig into)14:11
stgraberChipaca: no, got it failing the same way on amd6414:12
Chipacathat's very strange14:13
mborzeckithat's good news, right?14:13
Chipacamborzecki: how are our tests working on amd64?14:13
mborzeckiChipaca: they are 'passing'14:13
Chipacamborzecki: that's my point: if the bug is there, they shouldn't14:15
Chipacamborzecki: I'll dig14:15
mborzeckistgraber: Chipaca: got a debug shell on i386, do you want to check anything?14:15
stgrabertracked it down and working on a fix now14:15
mborzeckistgraber: what's the issue?14:15
stgrabertl&dr is your kernel doesn't support unprivileged file capabilities, yet it lets us write an xattr that uses that new v3 fscap format14:16
stgraberbut then blows up when reading the file14:16
mborzeckistgraber: heh, nice14:16
Chipacastgraber: so it's a bug in the image itself?14:16
Chipacas/bug/regression caused by a change/14:17
stgraberChipaca: it's a combination of things, LXD 3.4 introduced logic to remap file caps rather than just strip them and unsquashfs-tools was fixed yesterday to not drop xattrs in 16.0414:18
stgraberChipaca: so even in our candidate and edge channels, everything was good until the last snap rebuild yesterday which picked up the fixed unsquashfs14:19
stgraberand most of our manual testing is done on 4.15 kernels which do support the v3 caps or on 4.4 systems with -proposed enabled that again do support v3 caps (next kernel SRU has a backport of the feature)14:19
ChipacaMAGIC14:19
Chipacasigh14:20
ChipacaI'll push a pr that adds --candidate to the lxd integration test14:20
stgraberand even though we do have snap tests that use the broken kernels, our test image doesn't use file caps (it's just a tiny busybox image)14:20
Chipacastgraber: we woudln't've caught it if we hadn't just happened to be using one of the three binaries that have caps in xenial14:21
stgraberChipaca: yeah, I expect that detect-virt is what's going to break most users so trying to rush a fix now14:22
Chipacastgraber: as soon as the fix is in candidate let me know, and I'll push a pr that turns the test back on and fetches lxd from candidate14:29
Chipacamborzecki: I'm guessing amd64 just happened to get a newer kernel or something14:32
Chipacadunno14:32
mborzeckiChipaca: ubuntu-16.04-32 uses 4.4.0-131-generic14:35
Chipacamborzecki: I got 133 here, but ok14:35
Chipacaand indeed if I rebooted I'd have 133 as well14:36
Chipacamborzecki: mirror sync14:36
mborzeckiChipaca: ubuntu-16.04-64 image users different kernel indeed, it's 4.13.0-1019-gcp14:43
Chipacahaha, hehe. Ha ha ha ha, he he he he14:44
Chipacamborzecki: ok14:44
pedronisbig difference14:44
Chipacaso there was a little tiny window of hitting the bug, and we *nailed* it14:44
Chipaca:-)14:44
mborzeckiflawless victory14:45
Chipacanext question: should we make sure -32 and -64 are running similarish kernels?14:45
mborzeckiheh :)14:45
mborzeckicachio__: ^^ ?14:45
cachio__mborzecki, yes14:47
cachio__mborzecki, I think the problem with the function14:48
cachio__is that "if apt-cache policy "linux-image-extra-$(uname -r)" >/dev/null 2>&1; then" is going always through the then14:50
cachio__with match or without14:50
mborzeckicachio__: yeah, something to figure out14:52
mborzecki the current pr will effectively switch the kernel to something else14:52
mborzeckicachio__: i have 4.13.0-1019-gcp and the proposed code wants to install linux-image-extra-4.13.0-31-generic, this will pull in the non-gcp kernel14:53
cachio__mborzecki, yes, but that linux-image-extra-4.13.0-31-generic works well with the current kernel14:54
cachio__mborzecki, at least the tests pass14:54
cachio__so, what should we use when there is no linux-image-extra pkg for the current kernel?14:55
mborzeckicachio__: i see that there's linux-module-extra for the newer kernels, and linux-image-extra seems to be gone14:58
mborzeckicachio__: the real issue is that apt-cache policy <nonexistent-package> returns 0 :(14:58
cachio__mborzecki, yes14:58
mborzeckidamn, and so does apt list14:59
cachio__let me try installing linux-modules-extra-4.15.0-1017-gcp and running the tests14:59
pedronisneed to look at output it seems15:00
mborzeckicachio__: can you replace apt-cache policy with apt-cache show?15:00
cachio__mborzecki, that works15:02
mborzeckicachio__: something like this perhaps https://paste.ubuntu.com/p/QVSbPQcqWC/15:05
cachio__mborzecki, yes, makes sense15:06
Chipacaniemeyer: in spread, is there a reason why I can't say “systems: [ubuntu-*, -ubuntu-14*]”?15:07
cachio__mborzecki, I pushed that15:08
cachio__I'll test this with the new gce images to see if it works15:08
mupPR snapd#5681 opened: overlord: handle sigterm during shutdown better <Created by mvo5> <https://github.com/snapcore/snapd/pull/5681>15:14
mvopedronis: ^- this is your earlier pastebin with tests, hope the tests make (some) sense, its a bit tricky15:14
mvoogra: ^- this should hopefully fix the reboot timeout15:14
mvoogra: I mean the wait15:14
mvoa second review for 5676 and 5677 would be great, should be easy15:15
ogramvo, awesome, will happily test once it landed15:25
t1mpis there a way to speed up snap building if I already have all the .deb dependencies installed in the system?15:26
ogramvo, btw, kyle told me it is likly he had the SD mounted when dd'ing to it ... (yay, nautilus) ... you really need to promote godd more ;)15:27
t1mpI have a full pipeline on Jenkins now, which runs inside a Docker image that has all the dependencies. Still, the snapcraft step takes a long time because it is re-downloading all the deb files15:27
mupPR snapd#5676 closed: asserts: add support for gadget tracks in the model assertion <Core18> <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5676>15:28
pedronismvo: #5678 has conflicts now, merging master into it would be nice either way15:28
mupPR #5678: snapstate: add support for gadget tracks in model assertion <Core18> <Created by mvo5> <https://github.com/snapcore/snapd/pull/5678>15:28
mvopedronis: yeah, I updated it15:30
mvopedronis: and also the snapstate one15:30
mvopedronis: when you say "before adding the close probably it would have worked anyway" in the review of 5681 - do you mean that without the close the test I added would behave the same? or am I misunderstanding?15:31
mvopedronis: I will add the nil checks, I like that15:31
pedronismvo: no, I mean the code would not have explode passing nil in, but with the close it does15:31
mvopedronis: aha, got it - will fix it15:32
niemeyerChipaca: The logic is simpler.. you start with the fixed list for the file at hand, and can either add or drop.. it's not sequential15:32
mvopedronis: I just need the close for my test, I could do it differently but this looked most natural (next to sending sigkill itself in the test which does not work)15:32
pedronismvo: the close is ok, but I think making sigCh optional is also natural15:32
niemeyerChipaca: At least from my vague memories.. it's been a while15:33
mvopedronis: yeah, we are in agreement :)15:33
mvopedronis: updated, thanks for the feedback15:34
pedronismvo: I would have put the if around the whole bit using sigCh,  Stop supporting nil works but is not super clear from the docs why it should15:37
mvopedronis: indeed15:38
Chipacamvo: any idea what failing to reset devices.list means?15:47
mvoChipaca: do you have more context15:49
pedroniswhat is devices.list15:50
mvoChipaca: is that an lxc error? related to the lxd devices cgroup?15:51
mvoChipaca: or to our deivices cgroup?15:51
mupPR snapd#5660 closed: wayland: add extra sockets that are used by older toolkits (e.g. gtk3) <Created by gerboland> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/5660>15:55
plarssergiusens: how often do you normally do a release to stable of the snapcraft snap? Do you spend any time in beta/candidate? it looks like the current set of snaps is stable==candidate==beta, so I'm guessing you just do testing in edge?15:55
plarssergiusens: the reason I ask, is we'd like to do some testing of our snaps with upcoming releases of snapcraft, trying to figure out which channel would be the best to pull from15:56
sergiusensplars: when we tag, we put the snap in beta once it passes all our automated tests, then we do internal testing and if it all works we move it to candidate and make an ANN for a call for testing16:02
plarssergiusens: ok cool, how long do you normally give it in candidate? sounds like that's where we should aim for16:09
Chipacamvo: I'm afraid we moved on, but yes related to lxd (not sure if it's lxd or snapd printing that)16:19
Chipacapedronis: are you around?16:29
noise][he EOW'd a bit ago16:31
Chipacaah well16:32
Chipacapedronis: it was to avoid having to reflash a device to undo a serial assertion, but it'd only save us ~5 minutes :-)16:33
Chipacanoise][: thanks16:33
=== pstolowski is now known as pstolowski|afk
t1mpis there a way to cache a snapcraft pull when I run it in fresh docker images on Jenkins?16:44
t1mpthat stage always takes 10 minutes of installing new deb packages16:44
t1mp:%s/fresh docker images/fresh docker containers/16:45
sergiusensplars: 3 days, but we can negotiate on that with something reasonable (like a week)16:54
sergiusenss/reasonable/expectable/16:55
kyrofat1mp, two thoughts: 1) If we're talking build-packages, generate a docker image with them already installed and use that, should speed things up. 2) if we're talking stage-packages, snapcraft caches them in ~/.cache/snapcraft/, you can preserve that between runs16:59
kyrofaConsider that each of those steps makes the build less "clean"17:00
pedronisChipaca: having dinner17:01
t1mpkyrofa: thanks. That's useful info. I'm snapping a python app, so I don't really need to build anything.17:02
t1mpkyrofa: maybe it would be nice to have a 'snapcraft create-cache' command that downloads the .debs. I can include that in my Dockerfile since it doesn't change often.17:03
t1mpbasically I only need the 'dump' plugin to copy some files that are already generated in previous steps (using PyInstaller)17:04
t1mpbut setting up the stage takes 20 minutes :(17:04
kyrofat1mp, I don't quite understand. If you're only using the dump plugin, why is snapcraft fetching a bunch of stuff?17:05
plarssergiusens: I don't have any concerns about that. 3 days sounds reasonable. I don't expect we would normally have any problem17:06
t1mpkyrofa: it has dependencies, see https://pastebin.ubuntu.com/p/tbvGycDvvt/17:10
kyrofaAh, the remote part is probably taking a chunk of time17:11
t1mpyes, and it is repeating every time I make a small change somewhere17:12
t1mpI really only need to build the snap before publishing a new version, but for now I'm building it for each commit to make sure I don't break it. And to test that the snap building works properly17:12
kyrofat1mp, every time you make a small change you fire up a clean docker container?17:12
kyrofaAh, commits, okay that's fair17:13
t1mpyes, Jenkins does17:13
t1mpfor each push17:13
kyrofat1mp, if you run `snapcraft define desktop-gtk3` you'll see that it's mostly stage-packages as well17:15
kyrofat1mp, stage-packages are not installed on the host, which means creating a new docker image with them installed won't help you17:15
kyrofat1mp, but you can try preserving that cache between runs (point (2)) so they don't need to be fetched again17:15
sergiusenskyrofa, t1mp: you can mount the cache directory into the container17:15
kyrofasergiusens, you're stealing my thunder17:16
sergiusensexactly that17:16
t1mpI realized that :) I meant to have an image that includes ~/.cache/snapcraft. That would be kind of ugly though. Probably better to mount it.17:16
kyrofaIndeed17:16
sergiusensyeah, that (thunder) too :-)17:16
kyrofa:D17:16
t1mpsergiusens, kyrofa: right :)17:16
t1mpthat caches only the deb files right? They still need to be installed17:17
kyrofat1mp, they're just unpacked into the snap, yeah17:17
kyrofaShould be quick compared to fetching them17:17
t1mpyeah pull takes 3 minutes now https://pastebin.ubuntu.com/p/PJqfM6ndQW/17:19
t1mp(the total time went down from 20 to 5.. I think the server was overloaded before)17:19
t1mperr.. to 8 min, not 5. :)17:19
t1mpI'll look into keeping the cache. On Monday :)17:20
kyrofat1mp, we'll be here!17:20
t1mpgreat, thanks :)17:20
kyrofaHave a lovely weekend17:20
t1mpyou too17:21
stgraberChipaca, sergiusens: we have a fix (https://github.com/lxc/lxd/pull/4943), should land upstream in ~30min, then in snap another 30min or so later18:14
mupPR lxc/lxd#4943: shared/idmap: test fcaps support <Created by stgraber> <https://github.com/lxc/lxd/pull/4943>18:14
sergiusensthanks for the update18:14
Chipacastgraber: so I could push the PR about now?19:31
stgraberChipaca: yeah, we should have the fix in candidate in the next 10min or so19:32
stgraberChipaca: once Jenkins is happy on our side, I'll promote to stable19:32
Chipacastgraber: the pr I'd push would bring it back for i386, but also pull from candidate19:32
Chipacastgraber: (is candidate the right place to pull from?)19:33
stgraberChipaca: the issue isn't arch-specific though so you shouldn't make anything specific to i38619:34
stgraberyeah, we only use edge, candidate and stable19:34
Chipacastgraber: the difference is our i386 images have 4.4.x, whereas amd64 has 4.13.x19:34
Chipacastgraber: so the issue only manifested on i38619:34
stgraberChipaca: looks like we're seconds away from having the fix in candidate now19:35
stgraberthat snap takes so long to build...19:35
stgraber52min for that build19:36
Chipacastgraber: via snapcraft?19:36
stgraberChipaca: yeah on LP, but it's not a simple snap :)19:40
mupPR snapd#5682 opened: tests/main/lxd: pull lxd from candidate; reënable i386 <Created by chipaca> <https://github.com/snapcore/snapd/pull/5682>19:41
Chipacastgraber: tadaa19:42
stgrabersergiusens, Chipaca: promoting to stable now19:52
Chipacawoop19:52
kyrofaThank you, stgraber20:19

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!