[00:00] <nacc> kyrofa: of course they do :)
[00:00] <kyrofa> nacc, they're essentially dynamically-created channels. Perfect for creating a snap of "this" pull request
[00:01] <kyrofa> nacc, they're ephemeral, though, and expire after a month
[00:01] <nacc> kyrofa: ah i see
[00:01] <kyrofa> (unless published to again)
[00:01] <nacc> kyrofa: interesting, will read up on it tmrw
[00:01] <nacc> kyrofa: snaps feel like such a moving target  sometimes :)
[00:01] <kyrofa> nacc, yeah I feel ya
[00:02] <Chipaca> oooh, i think i know how to make the keygen tests quicker
[00:02] <kyrofa> Chipaca, what the heck man, were you asleep and awoke with a genius idea?
[00:02] <Chipaca> … don't we all?
[00:02] <kyrofa> Yes :P
[00:04] <Chipaca> kyrofa: also, manchester explosion has me woke
[00:04]  * kyrofa googles manchester explosion
[00:05] <kyrofa> Dang
[00:06] <kyrofa> Chipaca, you're not close, I hope
[00:06] <Chipaca> nor is any of my family as far as i know
[00:07] <kyrofa> Good
[00:12] <Chipaca> need to check with samuele wrt sanity, but this makes the key generation step of the tests ~20× faster
[01:55] <mup> PR snapd#3383 opened: Fix spread flaky tests linode <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/3383>
[02:50] <mup> PR snapcraft#1327 opened: tests: small updates for manual kernel tests <Created by elopio> <https://github.com/snapcore/snapcraft/pull/1327>
[04:58] <elfgoh> I had a question regarding sanitisation of snap interfaces: https://forum.snapcraft.io/t/sanitisation-of-snaps-requested-interfaces/739
[05:43] <mup> PR snapd#3382 closed: daemon,overlord/auth: store from model assertion wins <Created by pedronis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3382>
[05:49] <mup> PR snapd#3379 closed: cmd/snap,tests: show the snap id if available in snap info <Created by pedronis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3379>
[06:16] <mup> PR snapd#3226 closed:  snap-repair: add `snap-repair run`  <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/3226>
[06:37] <zyga> good morning
[06:38] <zyga> mvo: hey
[06:38] <zyga> mvo: just to give you some more context for my message yesterday
[06:38] <zyga> mvo: gustavo reviewed the way we convey meta-data for interfaces and wanted to make some changes
[06:38] <zyga> mvo: we had a quick brainstorm session and the bottom line is that the current rest API has to be reverted to what it was before meta-data layer was merged
[06:39] <zyga> mvo: and the meta-data must move to a new endpoint "interface" (vs "interfaces" currently)
[06:39] <zyga> mvo: I will work on this and the CE blocker today
[06:39] <zyga> mvo: when do you plan to release?
[07:11] <morphis> zyga: morning!
[07:14] <zyga> morphis: good morning
[07:15] <pstolowski> morning
[07:21] <palasso> Hello, I'm not following snap development closely. I'm wondering on why the OS snap ubuntu-core was renamed to core?
[07:30] <morphis> mvo, zyga, pedronis, Pharaoh_Atem, niemeyer: can you guys look into https://github.com/snapcore/snapd/pull/3222 again today? Would like to get that merged soon
[07:30] <mup> PR snapd#3222: many: fix test cases to work with different DistroLibExecDir <Created by morphis> <https://github.com/snapcore/snapd/pull/3222>
[07:32] <pstolowski> hello palasso! this was mainly to avoid confusion about cross-distro support
[07:32] <palasso> ty pstolowski for the answer
[07:50] <morphis> can it be that Linode/spread is pretty unreliable these days? Trying to get a green light on a PR which touches nothing which is tested anywhere but running into a lot of timeout issues: https://github.com/snapcore/snapd/pull/3371
[07:50] <mup> PR snapd#3371: packaging: import packaging bits for opensuse <Created by morphis> <https://github.com/snapcore/snapd/pull/3371>
[08:04] <mvo> morphis: yeah, looks like it, maybe worth poking the store people about: "DEBUG: The retry loop for https://assertions.ubuntu.com/v1/assertions/snap-declaration/16/99T7MUlRhtI3U0QFgl5mXXESAiSwt776?max-format=2 finished after 4 retries, elapsed time=40.008154662s, status: Get https://assertions.ubuntu.com/v1/assertions/snap-declaration/16/99T7MUlRhtI3U0QFgl5mXXESAiSwt776?max-format=2: net/http: request canceled while waiting for connection (Client
[08:04] <mvo> .Timeout exceeded while awaiting headers)"
[08:05] <mvo> morphis: 40s and 4 retries is a pretty long wait
[08:05] <fgimenez> hi morphis in that build you had a lot of connectivity problems, seems that between linode instances and the store, at least two "dial tcp 162.213.33.92:443: getsockopt: no route to host" (162.213.33.92 is public.apps.ubuntu.com), "dial tcp 162.213.33.92:443: i/o timeout", timeouts awaiting for headers from assertions.ubuntu.com, the tests/main/completion error seems to be related to connectivity too, maybe the store was down or all the instances
[08:05] <fgimenez> involved (more than one had errors) lose connection
[08:11] <morphis> yeah
[08:11] <morphis> is pretty nasty and it looks like I am getting similar problems with all my PRs
[08:12] <morphis> mvo: what is a good place to poke on the store people?
[08:14] <cjwatson> kyrofa: not yet; at the time when we were adding publish-to-track support to LP, branches were only defined for stable, and automatic jobs shouldn't be publishing directly to stable.  Some of the assumptions here may have changed since though
[08:15] <zyga> re
[08:17] <zyga> geez, firefox ate 3GB of ram on 4 tabs open
[08:36] <pedronis> my 2.27 PRs need reviews
[08:39] <Chipaca> pedronis: hiya
[08:46] <mup> PR snapd#3384 opened: tests: use pollinate to seed the rng <Created by fgimenez> <https://github.com/snapcore/snapd/pull/3384>
[08:46] <fgimenez> hi Chipaca, could you please take a look at ^^^? never use pollinate before, seems to be working fine with a plain call
[08:51] <Chipaca> fgimenez: yup
[08:52] <Chipaca> fgimenez: i also want to check with samuele about what happens if we use a gpg in testing that clamps keys to 1024 bits
[08:55] <fgimenez> Chipaca: great thanks!
[08:59] <pedronis> Chipaca: where? we check that the keys we get have the expected length
[08:59] <Chipaca> pedronis: we do? where?
[09:00] <Chipaca> snap create-key seemed to be happy with them
[09:00] <Chipaca> pedronis: good morning!
[09:01] <pedronis> Chipaca: well, when you try to sign with it, it won't (maybe that test doesn't use the key it makes, but as a general strategy across all tests doesn't work)
[09:02] <pedronis> Chipaca: here,  https://github.com/snapcore/snapd/blob/master/asserts/crypto.go#L367
[09:04]  * Chipaca pokes at his internet
[09:13] <Chipaca> pedronis: sadface
[09:13] <Chipaca> pedronis: maybe i should override gpg and then un-override it when tests break :-)
[09:14] <pedronis> Chipaca: override it with what anyway ?
[09:16] <Chipaca> pedronis: sed and more gpg :-)
[09:16] <pedronis> Chipaca: anyway we do have  a snap-sign test
[09:16] <morphis> zyga: step 1, CI passes on https://github.com/snapcore/snapd/pull/3365 :-)
[09:16] <mup> PR snapd#3365: tests,packaging: add package build support for Fedora for our spread setup <Created by morphis> <https://github.com/snapcore/snapd/pull/3365>
[09:16] <Chipaca> pedronis: yeah, that's the one i was poking around in last night
[09:16] <pedronis> Chipaca: which funnily I never seen fail
[09:17] <Chipaca> pedronis: oh, i did
[09:17] <Chipaca> pedronis: I'd point you at https://s3.amazonaws.com/archive.travis-ci.org/jobs/234990914/log.txt?X-Amz-Expires=30&X-Amz-Date=20170523T012508Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJRYRXRSVGNKPKO5A/20170523/us-east-1/s3/aws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=2f9a55bec2d1b61a8c23f3b46d7c955155fc8e2f719cb66e788743bc7181c36a but travis has already been "helpful"
[09:17] <pedronis> Chipaca: anyway what you could do is merge create-key into snap-sign
[09:17] <pedronis> so you need less overall entropy
[09:17] <Chipaca> pedronis: the entropy doesn't seem to be the problem afaict
[09:17] <pedronis> Chipaca: what is the problem then?
[09:18] <Chipaca> pedronis: at least, every time i've looked we've had ~3k of entropy
[09:18] <pedronis> doesn't compute
[09:18] <pedronis> why would create-key take forever if not that
[09:18] <Chipaca> pedronis: I'm guessing here but we might be getting penalized
[09:18] <Chipaca> for heavy cpu use
[09:19] <pedronis> anyway my point stand
[09:19] <Chipaca> yep
[09:19] <pedronis> we would use less CPU if we create less keys :)
[09:19] <Chipaca> don't get me wrong, we might also be running out of entropy
[09:19] <Chipaca> and i think we need to add that to the debug output
[09:20] <Chipaca> so we can see
[09:20] <Chipaca> (also to the regular output! but that's going to be harder)
[09:20] <pedronis> Chipaca: so mergeing  snap-sign create-key  and the key completion test into one
[09:22] <ogra_> [   12.818573] F2FS-fs (sda): Can't find valid F2FS filesystem in 1th superblock
[09:22] <ogra_> [   12.829425] F2FS-fs (sda): Can't find valid F2FS filesystem in 2th superblock
[09:22] <ogra_> hmpf
[09:22] <ogra_> i wonder if we cant quieten f2fs on a kernel level
[09:23] <Chipaca> augh
[09:23] <Chipaca> who writes that code :-(
[09:23] <Chipaca> who writes "%dth" and thinks that's a good idea
[09:23] <ogra_> someone who doesnt know about english numbering ?
[09:24] <Chipaca> but then there's like 5 other people that looked at that code and thought "fine whatever" and let it pass :-/
[09:24] <ogra_> yeah
[09:24] <Chipaca> anyway
[09:24] <Chipaca> always too easy to criticise other projects
[09:24]  * Chipaca shuts up
[09:25] <ogra_> well, i'm sure whatever code there is probes all possible filesystems ... but only f2fs prints that stuff
[09:26] <ogra_> apw, do you think we could quieten such f2fs messages ?
[09:27] <apw> ogra_, why do you care about two lines in your dmesg ?
[09:27] <Chipaca> apw: users (actually, device developers) see them and freak out
[09:27] <apw> ogra_, and indeed what is triggering them from userspace
[09:27] <ogra_> apw, dunno, i'm german :P
[09:27] <apw> Chipaca, i think device developers need to develop thicker skin :)
[09:28] <ogra_> apw, snapd checks all attached non rootfs devices for possible snap assertions on boot ... so it mounts them once ...
[09:28]  * apw sticks several fingers in each of his ears ...
[09:29] <ogra_> apw, we used to include ramdisks in that... when we did that i have about 30 of these messages ... now i get them for the one atttached usb device only ... but still only f2fs (which we never used) prints that stuff
[09:29]  * Chipaca wonders how apw types with several fingers in several ears
[09:29] <ogra_> s/i have/i had/
[09:29] <apw> Chipaca, with my feet, the only way
[09:29] <Chipaca> pedal keyboards ftw
[09:29] <apw> ogra_, surely we do not try all the possible filesystem types though
[09:29] <apw> ogra_, that way lies _vast_ attack surface
[09:30] <ogra_> apw, well, we just call mount i guess ... the rest is kernel
[09:30] <ogra_> (or internal logic of mount)
[09:30] <apw> ogra_, they kernel does not guess you filesystem type, that would be mount
[09:30] <Chipaca> ogra_: do we explicitly load f2fs?
[09:30] <ogra_> Chipaca, nope
[09:31] <apw> ogra_, so what is on that stick which triggers that error
[09:31] <ogra_> Chipaca, i think it is compiled in
[09:31] <Chipaca> ogra_: at least here it's a module
[09:31] <ogra_> apw, a vfat fs
[09:31] <Chipaca> dunno on other arches etc
[09:31] <apw> ogra_, ie what does blkid say on it
[09:31] <ogra_> Chnot in xenial
[09:32] <ogra_> Chipaca, ^
[09:32] <apw> as one would expect mount to ask what it is and then mount that specific type
[09:32] <Chipaca> ogra_: chtotally in xenial
[09:32] <ogra_> ogra@pi3:~$ lsmod|grep f2fs
[09:32] <ogra_> ogra@pi3:~$ grep f2fs /proc/filesystems
[09:32] <ogra_> 	f2fs
[09:32] <ogra_> ogra@pi3:~$
[09:33] <ogra_> ogra@pi3:~$ sudo blkid /dev/sda1
[09:33] <ogra_> ogra@pi3:~$
[09:33] <ogra_> bah
[09:33] <ogra_> /dev/sda1: UUID="72AD-5403" TYPE="vfat" PARTUUID="7524a7a6-01"
[09:33] <ogra_> silly IRC
[09:33] <Chipaca> ogra_: try: modinfo f2fs
[09:33] <apw> heh, so why the heck is f2fs being used against it
[09:33] <ogra_> apw, dunno, why was it being used on all the ramdisks before
[09:34] <apw> ogra_, can you manually try mounting it with mount /dev/sda2 /mnt
[09:34] <ogra_> this isnt new or anything ...
[09:34] <apw> and with mount -t vfat /dev/sda2 /mnt
[09:34] <apw> sort of thing
[09:34] <apw> and see if the latter is quieter
[09:34]  * apw would suggest we should be using blkid and only attempting to mount if it is one of a very very limited set of formats, ones we are willing to work hard to keep security good on
[09:35] <apw> else i could format it as some utterly nasty format and you will mount it, w
[09:35] <apw> which is just asking to be attacked
[09:35] <apw> i would vote for one format if i was involved, perhaps vfat
[09:36] <ogra_> apw, neirther mount nr mount -t vfat print anything
[09:36] <apw> ogra_, nothing in dmesg you mean ?
[09:36] <ogra_> right
[09:36] <apw> then you have a new mystery, who the heck is triggering that
[09:36] <apw> are you using some magical go helper
[09:37] <ogra_> pedronis, mvo, ^^^ does snapd rotate over filesystems when mounting a device to lok for assertions ?
[09:37] <ogra_> *look
[09:39] <pedronis> no clue, that's really for mvo
[09:42] <zyga> re
[09:42] <zyga> morphis: that's great :)
[09:42] <zyga> I need to catch up with PRs but first I need to prepare two important things for 2.27 for mvo
[09:43] <ogra_> looks like this is the mount code https://github.com/snapcore/snapd/blob/master/cmd/snap/cmd_auto_import.go
[09:43] <morphis> zyga: do that
[09:43] <ogra_> cmd := exec.Command("mount", "-o", "ro", "--make-private", deviceName, tmpMountTarget)
[09:43] <ogra_> nothing fancy regarding filesystems
[09:47] <zyga> hmm hmm
[09:47] <zyga> those are two distinct mount syscalls AFAIK
[09:49] <ogra_> because of the --make-private ??
[09:49] <zyga> yes
[09:49] <zyga> first you do mount -o ro deviceName tmpMountTarget
[09:49] <zyga> then mount --make-private
[09:49] <zyga> it's not atomic
[09:49] <zyga> (mount-the-command does that internally)
[09:50] <ogra_>  mount(8)  does  not  read  fstab(5)  when a --make-* operation is requested.  All necessary information has to be specified on the command
[09:50] <ogra_>               line.
[09:50] <ogra_> hmm
[09:50] <Chipaca> how do you un-duplicate a bug?
[09:50] <ogra_> Chipaca, iirc somewhere in the target bug
[09:50] <Chipaca> ah
[09:55] <mvo> ogra_: I'm not sure what rotate over filesystems means, but it does look at block devices for assertions, yes
[09:56] <mvo> apw: aha, just finished reading backlog - sure, we can make this smarter to only mount ext{2,3,4} and vfat or maybe even only vfat
[09:58] <zyga> mvo: we may even do more, register a GUID
[09:58] <zyga> mvo: that is unique to snapd
[09:58] <mup> PR snapd#3385 opened: cmd: add stub new snap-repair command and add timer <Created by mvo5> <https://github.com/snapcore/snapd/pull/3385>
[09:58] <zyga> mvo: (not partition ID, but partition *type* ID)
[09:59] <zyga> mvo: then you can make a stick, with any filesystem, and snapd will recognize it
[09:59] <apw> zyga, you don't want it to support "any" anything, if any device will mount and consume it on b
[10:00] <apw> zyga, boot.  that is just asking for people to exploit every possible filesystme bug
[10:00] <zyga> (you can still add extra requirements, but the guid makes it distinct even before we mount it)
[10:00] <mvo> zyga: maybe, but OTOH anything that makes it harder for people to use this feature is not ideal IMO, I like the simplicify of the current: grab random-usb-stick, copy file, done
[10:00] <zyga> apw: I agree
[10:00] <zyga> mvo: it all depends on tooling
[10:00] <apw> easy is the antithisys of secure
[10:01] <mvo> zyga: well, yes, but its still at least one extra step: find the right tool for your $OS, download, run
[10:01] <zyga> mvo: more like "make disk image" website
[10:01] <mvo> apw: you think even limiting to vfat would be an issue?
[10:01] <zyga> mvo: but I agree
[10:02] <apw> mvo, i would cirtainly recommend limititing it to something vfat might be that thing
[10:02] <mvo> apw, zyga: once I finished with my current branch I will do that then
[10:02] <apw> mvo, though i would ask the security team for a recomendation if anything
[10:02] <mvo> apw: good point
[10:09] <mup> PR snapd#3386 opened: interfaces, osutil: move flock code from interfaces/mount to osutil <Created by mvo5> <https://github.com/snapcore/snapd/pull/3386>
[10:12] <mup> PR snapd#3387 opened: cmd: auto import assertions only from vfat file systems <Created by mvo5> <https://github.com/snapcore/snapd/pull/3387>
[10:13] <zyga> mvo: thank you for moving the flock code
[10:15] <mvo> zyga: thanks for writing it :)
[10:17] <zyga> mvo: what is the `wrong prefix` test from?
[10:18] <zyga> ah
[10:18] <zyga> I misread
[10:18] <zyga> that's just a comment
[10:27] <ogra_> mvo, s/rotate/loop/ :)
[10:27] <ogra_> apw, still though, why do the fs drivers print such stuff at all ... this is similar to the ext2/3 messages you get when mounting an ext4 fs, that also spills a lot scary messages
[10:28] <ogra_> i.e.
[10:28] <ogra_> [    6.720679] EXT4-fs (mmcblk0p2): couldn't mount as ext3 due to feature incompatibilities
[10:28] <ogra_> [    6.729695] EXT4-fs (mmcblk0p2): couldn't mount as ext2 due to feature incompatibilities
[10:28] <ogra_> [    6.755923] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[10:28] <pstolowski> guys, i've a longer power outage today. i'm using my cellphone as a hotspot, but going for a lunch now and will be offline for ~1h
[10:28] <apw> ogra_, so that you can diagnose why a mount fails, as all you get in user space is EINVAL
[10:29] <ogra_> hmm
[10:29] <apw> they are not scarey they are informative and useful if a mount you intended to make fails
[10:29] <ogra_> they are scary for someone not understanding them
[10:29] <apw> someone not understanding them is unlikely to be able to find them
[10:30]  * ogra_ had enough bugs from users that got confused by these messages over the years
[10:30] <apw> that is why they are not on the screen, but in a hidden kernel buffer
[10:30] <apw> yes users will always find somethign to be confused about, but in this context it is unlikely a user can even see this buffer
[10:31] <ogra_> dmesg shows it prominently as does syslog and the kern.log file
[10:31] <ogra_> so if an illiterate user has a prob he looks at it and reacts with "OMG filesystem errors" ...
[10:31] <apw> few illiterate users know about any of those logs, or how to find them
[10:32] <ogra_> well
[10:32] <apw> and a semi-literate user will always be scared by everything
[10:32] <apw> and if you don't record them, their literate friend they go and ask
[10:32] <apw> will have literally nothing to work with
[10:32] <ogra_> heh, k
[10:32] <apw> computer equipement should have "no user servicable parts" on the front :)
[10:33] <ogra_> lol
[10:33] <apw> we log a lot more of that kind of thing than necessary because we are lazy in usersapce and
[10:33] <apw> try things because we know the kernel will fail to do things which are silly
[10:33] <apw> instead of only asking it to do things we want to do
[10:34]  * apw goes back to punching docker repeatedly wishing it _would_ log something useful
[10:37] <Chipaca> apw: punch harder?
[10:37] <apw> if only i could punch it hard enough
[10:38] <Chipaca> maybe a hydraulic punch
[10:45] <mup> PR snapcraft#1328 opened: qmake plugin: set default qt version <Created by tim-sueberkrueb> <https://github.com/snapcore/snapcraft/pull/1328>
[10:52] <pedronis> mvo: made a comment about snap-reapir branch, my explanation yesterday might have been confused, I still owe to update the forum entry
[10:52] <pedronis> s/confused/confusing/
[10:54] <Chipaca> pedronis: I requested changes on one of your PRs, so it all evens out
[10:54] <pedronis> Chipaca: seems we have not tests for snap info directory fwiw
[10:54] <Chipaca> pedronis: there are
[10:54] <Chipaca> pedronis: just not with --verbose
[10:55] <pedronis> so we have no ---verbose tests?
[10:56] <Chipaca> certainly none for ---verbose :-p  and only a rather bare local check for --verbose
[10:56] <Chipaca> wait
[10:56] <Chipaca> taht might be yours
[10:56] <Chipaca> :-)
[10:56] <Chipaca> so, no, no tests for --verbose
[10:56] <pedronis> Chipaca: I added it
[10:57] <Chipaca> pedronis: yeah
[10:57] <Chipaca> pedronis: any reason not to also show the sha3 for remote snaps in info?
[10:58] <pedronis> Chipaca: we don't have that info around
[10:58] <pedronis> ah remote ones
[10:58] <Chipaca> pedronis: it's in DownloadInfo
[10:58] <pedronis> but not locals?
[10:58] <Chipaca> pedronis: ¯\_(ツ)_/¯
[10:59] <pedronis> because that needs thinking, if we want local too
[10:59] <Chipaca> it's strange that we don't store it in state
[10:59] <Chipaca> i guess it's somewhere in the asserts db?
[10:59] <pedronis> it's in the assert db
[10:59] <Chipaca> thinking is hard
[10:59] <Chipaca> let's have lunch
[11:00] <Chipaca> pedronis: to be clear, i'm not saying this is in any way a blocker to the pr you have up
[11:00] <Chipaca> but every time i look at snap info i want to make it better :)
[11:00] <Chipaca> and then i remember making it better starts by looking at goyaml
[11:01] <pedronis> Chipaca: anyway that's not a snap info problem, we need to think how to put the ash back into api.go
[11:01] <pedronis> then info is easy
[11:01] <Chipaca> yup
[11:01] <pedronis> there's remote, local unasserted, and local asserted
[11:02] <pedronis> to consider
[11:11] <_28Kb> why there's no php7 server snap?
[11:14] <Chipaca> _28Kb: what's a php7 server?
[11:15] <_28Kb> like one in XAMPP
[11:17] <Chipaca> _28Kb: I don't think there's a reason
[11:18] <mup> PR snapcraft#1329 opened: tour: use https for source urls <Created by tim-sueberkrueb> <https://github.com/snapcore/snapcraft/pull/1329>
[11:19] <Chipaca> mvo: have you seen the travis error on snapd#3374?
[11:19] <mup> PR snapd#3374: partition: add directory sync to the save uboot.env file code <Created by mvo5> <https://github.com/snapcore/snapd/pull/3374>
[11:19] <_28Kb> i need apache, php and mysql for my ubuntu core..
[11:20] <Chipaca> mvo: (hint: it goes “can't load package: package github.com/mvo5/uboot-go/uenv: [...]”)
[11:20] <_28Kb> i got only this nextcloud option offered
[11:24] <Chipaca> _28Kb: out of curiosity, why do you need a snap with these things?
[11:24] <_28Kb> i got snappy core, so i need snaps i guess
[11:25] <Chipaca> _28Kb: _for what_?
[11:25] <Chipaca> _28Kb: that is, you don't need a server, nobody needs a server. Servers are for building services with. What service are you building?
[11:26] <_28Kb> service which i could access from outside the box
[11:26] <Chipaca> _28Kb: and this is for a single box, your box, not for building a bunch of boxes that use this?
[11:26] <_28Kb> through websocket for example
[11:27] <_28Kb> i can't build one.. bunch is for distant future :)
[11:28] <mup> PR snapd#3377 closed: asserts: simplify and adjust repair assertion definition <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/3377>
[11:28] <pstolowski> Chipaca, pedronis, zyga, mvo is any of you working on retry for deltas? if not, I'll do it today
[11:29] <Chipaca> _28Kb: it should be fairly straightforward to build a snap like that; nobody's done it because they haven't felt the need i guess? all the needed bits are there
[11:29] <pedronis> pstolowski: retry in which sense?
[11:29] <pedronis> but no
[11:29] <_28Kb> ok, i see
[11:31] <Chipaca> pstolowski: i can confirm that I am not working on retry for deltas
[11:32] <Chipaca> in fact, I'm going to start working on lunch.
[11:33] <pstolowski> pedronis, ah, nvm... I thought we don't retry deltas after a quick look yesterday when we failed in the tests on travis, but looking again now and we actually do retry
[12:05] <mup> PR snapd#3380 closed: cmd/snap,tests: show the sha3-384 of the snap for snap info --verbose SNAP-FILE <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/3380>
[12:07] <pedronis> Chipaca: about info and sha3-384, proably leaving it out for unasserted local snaps would be ok or desired, so that would make it relatively doable
[12:12] <Chipaca> pedronis: well, snap info _could_ cheat and go look at the .snap in that case :-)
[12:13] <pedronis> Chipaca: yes, it could but not as I said it could be argued that it doesn't make sense
[12:13] <pedronis> s/but not/but/
[12:14] <mup> PR snapd#3388 opened: osutil: add non-blocking flock <Created by mvo5> <https://github.com/snapcore/snapd/pull/3388>
[12:16] <mup> PR snapd#3384 closed: tests: use pollinate to seed the rng <Created by fgimenez> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3384>
[12:17] <mup> PR snapd#3378 closed: tests: fixes for executions using the staging store <Created by fgimenez> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3378>
[12:39] <ogra_> mvo, hmm, your uboot change failed all tests ... something still looks for github.com/mvo5/uboot-go/uenv somewhere
[12:39] <mvo> ogra_: checking
[12:53] <mup> PR snapd#3389 opened: overlord/snapstate: have an explicit code path last-refresh unset/zero => immediatey refresh try <Created by pedronis> <https://github.com/snapcore/snapd/pull/3389>
[12:55] <pedronis> mvo: small PR of something I mentioned/we discussed ^  (also forum https://forum.snapcraft.io/t/refresh-schedule-via-core-config/434/9 )
[12:55] <mvo> zyga: any progress on the work for seccomp constants resolving (the things we talked about yesterday) btw?
[12:56] <mup> PR core#47 closed: keep version Makefile in sync with version-script <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/core/pull/47>
[12:57] <mvo> pedronis: very nice, thanks a lot
[13:03] <morphis> niemeyer: can you snapshot spread-61 as opensuse-42.2-64 when you have a minute? :-)
[13:13] <dragly> Is there anything apart from the "camera" plug I need to request for camera access? I'm currently running a Qt app on a computer without a camera and get the following error:
[13:13] <dragly> defaultServiceProvider::requestService(): no service found for - "org.qt-project.qt.camera"
[13:35] <Chipaca> fgimenez: how much were you giggling while adding "pollinate"
[13:36]  * Chipaca just realised it might be funny in Spain
[13:39] <fgimenez> Chipaca: :D yes something you wouldn't hear from your grandmother
[13:41] <pedronis> niemeyer: this is the PR  I mentioned:  https://github.com/snapcore/snapd/pull/3375  (also I pointed to it from aliases topic)
[13:41] <mup> PR snapd#3375: snapstate,many: implement snap install --unaliased <Created by pedronis> <https://github.com/snapcore/snapd/pull/3375>
[13:42] <niemeyer> pedronis: Thanks for the pointer
[13:42] <Chipaca> one more review wanted for snapd#3342 please
[13:42] <mup> PR snapd#3342: many: refactor in preparation for 'snap start' <Created by chipaca> <https://github.com/snapcore/snapd/pull/3342>
[13:42] <niemeyer> Chipaca: Will look into it too
[13:43] <Chipaca> niemeyer: taw
[13:53] <niemeyer> It's sad we're still blocked from using our neat state patching mechanism...
[13:55] <niemeyer> It'd be nice to be able to drag comments to different lines before the review has been delivered
[13:58] <Chipaca> niemeyer: as opposed to cut, delete, recomment?
[13:58] <niemeyer> Chipaca: Yeah
[13:58] <Chipaca> (like a caveman)
[13:58] <niemeyer> Exactly :)
[13:58] <Chipaca> niemeyer: drop wossisname a tweet
[14:00] <niemeyer> Chipaca: Who's that?
[14:00] <Chipaca> niemeyer: back when pyconar and pyconbr worked together one of the guests we brought was this github guy
[14:01] <Chipaca> (i don't even know if he's still at github :-) )
[14:01] <niemeyer> Chipaca: The account on Twitter seems dead
[14:02] <Chipaca> ah well
[14:04] <mup> PR snapd#3389 closed: overlord/snapstate: have an explicit code path last-refresh unset/zero => immediately refresh try <Created by pedronis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3389>
[14:07] <pedronis> mvo: thanks for merging ^   it also cuts down snapstate tests to <1s (go test displayed time) here
[14:07] <niemeyer> Chipaca: Reviewed
[14:07] <niemeyer> pedronis: Too
[14:07] <pedronis> niemeyer: thanks, saw that, trying to add the tests John asked for
[14:07] <mvo> pedronis: neat
[14:09] <Chipaca> niemeyer: thanks!
[14:09] <mup> PR snapd#3342 closed: many: refactor in preparation for 'snap start' <Created by chipaca> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/3342>
[14:09] <Chipaca> mvo: “snap log -p” ftw
[14:12] <Chipaca> mvo: also https://github.com/golang/go/issues/6123
[14:12] <Chipaca> (which didn't get anywhere, but that's neither here nor there)
[14:13] <mvo> Chipaca: ok, if everyone is happy with TryLock I will rename it
[14:13] <Chipaca> having said all this, remember: don't trust me with names :-)
[14:13] <Chipaca> niemeyer: defunkt!
[14:13] <mvo> niemeyer: are you ok with FLock.{Lock,TryLock} for a blocking and non-blocking file lock ?
[14:14] <fgimenez> mvo: before i forget this is the error on the docker test execution https://travis-ci.org/snapcore/spread-cron/builds/235061721#L452 (only failed on amd64)
[14:14] <niemeyer> mvo: As in, introducing such a type with those methods?  Didn't we have flocking somewhere already?
[14:15] <mvo> niemeyer: yeah, we had blocking flock so far, we are brainstorming names for the non-blocking variant of this
[14:15] <mup> PR snapcraft#1330 opened: storeapi: log download retries <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/1330>
[14:16] <mvo> fgimenez: interessting and strange, especially if it only failed on a single arch
[14:16] <niemeyer> mvo: TryLock sounds nice
[14:17] <fgimenez> mvo: i'm trying to reproduce locally, will let you know how it goes
[14:17] <niemeyer> mvo: perhaps FileLock rather than FLock
[14:17] <mvo> niemeyer: great, I will make this happen, thanks for your input!
[14:18] <niemeyer> mvo: Thanks for working on it.. are we flocking something new?
[14:19] <mvo> niemeyer: yes, snap-repair
[14:19] <mvo> niemeyer: we want only a single snap-repair running, this is the prereq for this
[14:20] <niemeyer> mvo: Cool.. should be safe enough, assuming the snap-repair process itself doesn't get frozen for whatever reason :)
[14:21] <mvo> niemeyer: yeah, that would be bad(tm)
[14:23] <kyrofa> cjwatson, indeed, I only just discovered that branches only exist on stable
[14:23] <Chipaca> mvo: should probably grow into something lockfileish if it's a concern
[14:25] <Chipaca> as in, write the pid when locking, and then other lockers get to kill you if you take too long
[14:28] <mvo> Chipaca: yeah
[14:29]  * Chipaca gives reviews in bites
[14:29] <pedronis> Chipaca: pushed some more tests to snapd#3375
[14:29] <mup> PR snapd#3375: snapstate,many: implement snap install --unaliased <Created by pedronis> <https://github.com/snapcore/snapd/pull/3375>
[14:30] <Chipaca> pedronis: yep, looking now
[14:35] <mup> PR core-build#11 closed: remove cruft from the writable-paths <Created by mvo5> <https://github.com/snapcore/core-build/pull/11>
[14:36] <mup> PR core-build#11 opened: remove cruft from the writable-paths <Created by mvo5> <https://github.com/snapcore/core-build/pull/11>
[14:41] <Chipaca> mvo: boooo to returning errno :-)
[14:42] <Chipaca> (yes sure it knows its an errno so .Error() gives you "resource temporarily unavailable", but it's still 0xb)
[14:42] <mup> PR snapd#3390 opened: tests: remove additional setup for docker on core <Created by fgimenez> <https://github.com/snapcore/snapd/pull/3390>
[14:48]  * ogra_ is dizzy ... u-boot all around me ... 
[14:50]  * zyga has food poisoning; sorry guys, will be back some other time today
[15:02] <Chipaca> zyga: I ate tomato sauce that was out all night, and _you_get food poisoning?
[15:02] <Chipaca> zyga: something is wrong
[15:03] <morphis> niemeyer: you saw my message about labeling a spread node for opensuse?
[15:03] <morphis> mvo: can you merge https://github.com/snapcore/snapd/pull/3357 now that CI passes?
[15:03] <mup> PR snapd#3357: tests/lib: abstract build dependency installation a bit more <Created by morphis> <https://github.com/snapcore/snapd/pull/3357>
[15:11] <Chipaca> niemeyer, pedronis: bug #1692866 might be something you guys can follow and have useful input on (i don't feel i can given it's juju which i know next to nothing about)
[15:11] <mup> Bug #1692866: /snap/bin not in path for juju run/juju ssh <juju:Incomplete> <snapd (Ubuntu):New> <https://launchpad.net/bugs/1692866>
[15:12] <niemeyer> morphis: I haven't seen it
[15:12] <morphis> niemeyer: ok, here it comes again: need spread-61 tagged as opensuse-42.2-64
[15:12] <niemeyer> morphis: You mean a new image snapshot out of it, right?
[15:13] <morphis> niemeyer: yes
[15:13] <morphis> PR for spread-images is coming now
[15:13] <niemeyer> morphis: Cool, thanks. Will do that first thing after lunch.
[15:15] <morphis> niemeyer: thanks!
[15:16] <ogra_> Chipaca, you ate tomato sauce that went clubbing ? doesnt all that dancing turn it into "salsa" anyway ?
[15:16] <Chipaca> ogra_: you're saying this as if it were a bad thing?
[15:16] <ogra_> nah, not a bad thing at all
[15:17] <ogra_> In file included from include/common.h:22:0,
[15:17] <ogra_>                  from lib/asm-offsets.c:15:
[15:17] <ogra_> include/errno.h:12:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘extern’
[15:17] <ogra_>  extern int errno;
[15:17] <ogra_> *that* is a bad thing :(
[15:21] <Chipaca> ogra_: I agree. Get your cat off your keyboard.
[15:30] <ryebot> zyga: are you available to discuss that "cannot change profile for the next exec call: No such file or directory" bug from yesterday?
[15:30] <ryebot> I attempted the suggested fix (no classic installation), but it's still happening
[15:31] <ryebot> I also checked /var/lib/snapd/seccomp/profiles/snap.etcd.etcd (http://paste.ubuntu.com/24634459/) to see if it was indeed not being installed classically, and it seems to check out. (I assume I'm looking for the absence of @unrestricted at the top?)
[15:33] <ryebot> ah, I see he's out sick
[15:34] <ryebot> ^ Anyone else want to take a swing at it? :)
[15:45] <Chipaca> no, but i'll take a swing at a cuppa tea
[15:45] <ogra_> such a "danceish" day ...
[15:46] <ogra_> swing and salsa ...
[15:53] <ralsina> I have two snaps in my account I really don't care about anymore (gatertest and gatedtest) since they were used for OLS development. However, neither one has the trashcan icon to delete it. Can any of you people delete them?
[15:54] <ralsina> They are https://dashboard.snapcraft.io/dev/snaps/6115/ and https://dashboard.snapcraft.io/dev/snaps/6114/
[15:55] <ogra_> GRRRRR
[15:56]  * ogra_ curses u-boot ... 
[15:56] <ogra_> finally got 2017.01 building for the hummingborat ... now the SPL doesnt init it ... :(
[16:06] <Facu> ralsina, I don't know if that's possible after they got releases...
[16:06] <ralsina> Facu: hmmm ok, that's very annoying
[16:07] <roadmr> make them private, then nobody will be able to see them at least
[16:07] <Facu> ralsina, mmm... I have this package, with a release to edge, and I can delete it! https://dashboard.snapcraft.io/dev/snaps/7466/
[16:08] <niemeyer> morphis: The image is way too large at the moment, at 2.8GB
[16:08] <Facu> this one I can not delete: https://dashboard.snapcraft.io/dev/snaps/7544/
[16:08] <ralsina> Facu: can't unpublish it either, or can't see how to, even when the help says I can "unpublish at any time"
[16:09] <sergiusens> ralsina: have you tried `snapcraft close <snap-name> <channel>`?
[16:10] <Facu> ralsina, that what sergiusens says
[16:10] <ralsina> sergiusens: nevwer thought I needed to use the command line to make something go away on a website
[16:10] <Facu> ralsina, I unpublished everything in https://dashboard.snapcraft.io/dev/snaps/7544/ and still can not delete it
[16:10] <Facu> nessita, may you know what makes a package "undeleteable"?
[16:10] <ralsina> "Your account lacks permission to close channels for this snap."
[16:11] <Chipaca> Facu: somebody installing it, is what i was told way back
[16:11] <Facu> Chipaca, "in the process of installing it"? mmm
[16:11] <ralsina> oh, well, who cares
[16:11] <Chipaca> ralsina: but since you're here
[16:11] <Chipaca> ralsina: (hola!)
[16:11] <morphis> niemeyer: hm
[16:11] <ralsina> I'll rename them zzzzzfoo and zzzzzbar since they are in the middle of two snaps I actually care about :-P
[16:11] <ralsina> hola Chipaca!
[16:11] <Chipaca> ralsina: what was that about you not being able to publish to edge from travis?
[16:12] <morphis> niemeyer: you killed the instance?
[16:12] <morphis> ah no you didn't
[16:12] <ralsina> Chipaca: not travis, build.snapcraft.io
[16:12] <niemeyer> morphis: Yeah, still there
[16:12] <ralsina>  Chipaca: it builds every commit, and then doesn't actually have a knob to make it release to edge
[16:12] <Facu> ralsina, build.snapcraft.io pushes to edge every 'fades' release
[16:13] <morphis> niemeyer: hm, looks like I forgot to do the cleanup on the new install I spawned up this afternoon after my previous one got removed
[16:13] <ralsina> Facu: how?
[16:13] <Facu> ralsina, don't remember doing anything special
[16:13] <ralsina> Facu: the only things the UI lets you configure is the repo and the package name
[16:13] <ryebot> I updated the relevant bug; if anyone could assist me with finding a fix or workaround, that would be awesome. https://bugs.launchpad.net/snappy/+bug/1687079
[16:13] <mup> Bug #1687079: cannot change profile for the next exec call: No such file or directory <Snappy:New> <https://launchpad.net/bugs/1687079>
[16:14] <Facu> ralsina, I had something in edge before setting up build.s.io
[16:14] <Facu> ralsina, does your package? maybe that's a detail?
[16:36] <mup> PR snapd#3387 closed: cmd: auto import assertions only from ext4,vfat file systems <Created by mvo5> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/3387>
[16:50] <mup> PR snapd#3391 opened: tests: reboot after upgrading to snapd on the -proposed pocket <Created by fgimenez> <https://github.com/snapcore/snapd/pull/3391>
[16:54] <morphis> niemeyer: running out of time, will ping you tomorrow again
[16:54] <niemeyer> morphis: Sounds good, thanks!
[17:01] <ralsina> Facu: yeah, had stuff in all channels, really
[17:20] <pedronis> seems timeouts are really trying to stop me from landing snapd#3375 :(
[17:20] <mup> PR snapd#3375: snapstate,many: implement snap install --unaliased <Created by pedronis> <https://github.com/snapcore/snapd/pull/3375>
[17:37] <mup> PR snapd#3386 closed: interfaces, osutil: move flock code from interfaces/mount to osutil <Created by mvo5> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3386>
[17:39] <mup> PR snapd#3357 closed: tests/lib: abstract build dependency installation a bit more <Created by morphis> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/3357>
[18:13] <cachio> niemeyer, after review many errors in travis, I think basd on the errors that are causing the fails perhaps we could adopt a different strategy to deal with flaky tests
[18:14] <niemeyer> cachio: That's a bit too general to correlate.. what's the issue, what's the first approach, what's the different approach, etc? :)
[18:14] <niemeyer> cachio: Sounds like a forum topic.. it'll quickly go out of hand here
[18:14] <cachio> niemeyer, have you ever read this post https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html ?
[18:14] <cachio> niemeyer, agree with that, I'll move out to the forum
[18:15] <niemeyer> cachio: We need data about what's going on.. not a blog post :)
[18:15] <cachio> niemeyer, of course, i am caollecting data
[18:15] <niemeyer> cachio: I suggest starting small.. take one single test that you'd like to fix and explain what is happening and how you'd like to fix it
[18:15] <niemeyer> cachio: Rather than trying to solve the world at once
[18:16] <cachio> niemeyer, i am doing that, but i see that some tests are failing because of external dependencies such as internet connection
[18:17] <cachio> niemeyer, just to clarify, I am not saying that we dont have to fix tests
[18:18] <cachio> niemeyer, I'll start a forum thread, better :)
[18:18] <niemeyer> cachio: Yeah, I get it.. I'm just suggesting to start with small but solid incremental steps
[18:18] <nacc> sigh, anyone else getting proxy errors for LP to the store?
[18:18] <nacc> i'll wait a bit and retry, but just wondering if it's a known issue
[18:18] <niemeyer> cachio: Thanks!
[18:25]  * zyga feels terrible but returned to the office for his laptop
[18:48] <mup> PR snapcraft#1331 opened: integrations: use the snapcore/snapcraft docker image in travis <Created by filibtester> <https://github.com/snapcore/snapcraft/pull/1331>
[18:49] <pedronis> well for sure we have now a category of tests where prepare usually takes less than the timeout but sometimes more
[18:51] <mup> PR snapd#3375 closed: snapstate,many: implement snap install --unaliased <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/3375>
[18:54] <niemeyer> pedronis: fgimenez has a branch bumping the timeout.. did we merge this?
[18:57] <pedronis> niemeyer: it was bumped to 20mins, but I still saw prepare kill-timeout failures
[18:57] <pedronis> today
[19:04] <pedronis> anyway now that I have green run I don't see anythin taking more than 1000s,  so it's either load or network when they take that really long time
[19:20] <niemeyer> pedronis: 20 minutes preparing?  That ought to be something frozen
[19:30] <cachio> niemeyer, do you know the configuration needed to run tests in fedore?
[19:30] <cachio> fedora?
[19:32] <niemeyer> cachio: I wouldn't worry too much about anything that is not already enabled in Linode on master
[19:32] <cachio> niemeyer, ok
[20:01] <mup> PR snapd#3392 opened: interfaces: add minimal seccomp resolver to avoid backward compatiblity issues <Created by mvo5> <https://github.com/snapcore/snapd/pull/3392>
[20:04] <mup> Bug #1693016 opened: snap install conjure-up fails if juju snap installed <Snappy:New> <https://launchpad.net/bugs/1693016>
[20:28] <kyrofa> store... proxy errors....
[20:33] <mup> PR snapcraft#1332 opened: cli: provide a whoami command <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/1332>
[20:37] <kyrofa> noise][, I'm having trouble installing the snap I just released into a branch. `snapcraft status` shows it's there. Is this a cache issue?
[20:37] <kyrofa> It's been about ten minutes now
[20:44] <noise][> kyrofa: not a cache issue, but let me see if someone can help investigate
[20:46] <kyrofa> noise][, alright, thanks. Snapcraft show me http://pastebin.ubuntu.com/24637022/ , but when I install I get http://pastebin.ubuntu.com/24637027/
[20:46] <kyrofa> Note that pr-283 works fine, however
[20:47] <Facu> hola kyrofa
[20:47] <kyrofa> Hey Facu :)
[20:52] <noise][> kyrofa: refresh into the branch works
[20:53] <kyrofa> noise][, weird... how can that be?
[20:53] <kyrofa> noise][, but install fails?
[20:53] <noise][> different endpoints, might be a bug in details (used for install), but if so clearly we are missing a big test!
[20:57] <Facu> noise][, you say that a refresh from the branch worked, while installing from that branch didn't?
[20:58] <noise][> yes: https://pastebin.ubuntu.com/
[20:59] <noise][> and although the output doesn't show it, it did change to the proper revision
[21:05] <kyrofa> noise][, wrong link, I think :P
[21:05] <noise][> woops
[21:06] <noise][> https://pastebin.ubuntu.com/24637178/
[21:06] <Facu> kyrofa, would you please tell me the snap_id of that package?
[21:07] <kyrofa> Facu, njObIbGQEaVx1H4nyWxchk1i8opy4h54
[21:07] <Facu> kyrofa, thanks!
[21:10] <mup> Bug #1693037 opened: Can’t start a snap app <Snappy:New> <https://launchpad.net/bugs/1693037>
[21:11] <Facu> kyrofa, it's strange, indeed
[21:13] <kyrofa> Facu, yeah, weird. Any ideas?
[21:14] <Facu> kyrofa, not yet, and I need to run now :(, but I'll grab this until I figure it out, can I get to you tomorrow about this?
[21:15] <Facu> (if somebody didn't get to you with this before)
[21:19] <thomi> kyrofa: I'll take a look after I'm out of this call
[21:19] <kyrofa> thomi, alright I appreciate it, thank you.
[21:19] <thomi> np
[21:56] <thomi> kyrofa: OK, I'm out of the call. Just to make sure I understand the problem, you have some branches that you can refresh into, but you can't install from?
[21:56] <kyrofa> thomi, one, yes. The other branches work fine
[21:56] <thomi> it's the 283 branch that you can't install from?
[21:56] <kyrofa> 284
[21:57] <kyrofa> 283 works
[21:57] <thomi> ahh ok. Thanks.
[21:57] <thomi> I'll investigate. When's your EOD, and how should I let you know what I find if it's late?
[22:10] <thomi> wgrant: So the branch kyrofa is looking for doesn't exist in snaprevs: https://pastebin.canonical.com/188993/ I guess this may be an SCA integration issue?
[22:11] <kyrofa> thomi, in about an hour or so, but this is standing in the way of some stuff so I'll be checking back periodically-- feel free to just ping me here
[22:11] <thomi> kyrofa: ok, thanks
[22:12] <wgrant> thomi: Yeah, I think it's a combination of SCA and snapd bugs.
[22:12] <wgrant> thomi: Bret's refresh worked because snapd doesn't always parse out the branch
[22:13] <thomi> but you think it's not actually getting a revision from that branch?
[22:13] <thomi> I cna't see hwo it could, if we don't have that branch in our db
[22:13] <thomi> ugh. typoing
[22:13] <thomi> wow...
[22:14] <wgrant> Well, there could easily be a bug in snapdevicegw that caused this
[22:14] <wgrant> But the code in this case is pretty clear, and looking at snapd HTTP requests the client is doing some weird stuff.
[22:14] <wgrant> Trying to work out exactly what
[22:15] <wgrant> Hm. I just told it to refresh to "lalala", it made a query for "lalala/stable" which is correct, got an empty response as expected, but then said it upgraded
[22:16] <wgrant> And before I told it to refresh to "stable/1234" and it asked for edge.
[22:18] <kyrofa> Yikes
[22:18] <wgrant> Oh, no, that latter one was a typo.
[22:19] <wgrant> But it still says the snap was refreshed even if the server returns nothing at all for the channel.
[22:19] <wgrant> So Bret's test doesn't reveal anything, really. The only real issue here is that the branch never got pushed for some reason.
[22:20] <thomi> wgrant: want me to file a snapd bug for the poor UX around refreshing, while you investigate the push issue?
[22:20] <wgrant> thomi: Sounds reasonable.
[22:20] <wgrant> The really confusing thing is that it prints the version and says it was refreshed, even if nothing changed.
[22:21] <wgrant> And also even if the server doesn't give anything to suggest that the requested channel actually exists.
[22:21] <kyrofa> wgrant although note that snapcraft thinks it does: http://pastebin.ubuntu.com/24637022/ .
[22:21] <Chipaca> what's the ux around refreshing that's bad?
[22:22] <Chipaca> and what is the weird stuff you see snapd doing?
[22:22] <wgrant> kyrofa: Right, somehow it didn't make it from dashboard to the services that serve devices.
[22:22] <wgrant> Chipaca: wgrant@lamuella:~$ sudo snap refresh core --channel=totally/bogus
[22:22] <wgrant> core (totally/bogus) 16-2.26.3+git204.1bc8375 from 'canonical' refreshed
[22:22] <kyrofa> wgrant, ah, right
[22:22] <Chipaca> huh
[22:22] <thomi> wgrant: what's worrying is what snapd shows if you do 'snap info core' after that
[22:22] <Chipaca> yeap, that's a bug
[22:23] <thomi> I'm filing the bug now...
[22:23] <wgrant> May 24 08:22:27 lamuella /usr/lib/snapd/snapd[12917]: logger.go:75: DEBUG: < "HTTP/1.1 200 OK\r\nContent-Length: 40\r\nContent-Type: application/json\r\nDate: Tue, 23 May 2017 22:22:27 GMT\r\nServer: gunicorn/19.7.1\r\nStrict-Transport-Security: max-age=2592000\r\nX-Request-Id: e8ab92e3-449f-4531-9cf1-60933ff2f0b1\r\nX-Vcs-Revision: 51cfdf0\r\n\r\n{\"_embedded\":{\"clickindex:package\":[]}}\n"
[22:23] <wgrant> So the server correctly returned nothing.
[22:23] <Chipaca> thomi: so it changes the tracking channel, indeed
[22:23] <Chipaca> :-(
[22:23] <Chipaca> thomi: wgrant: nice catch
[22:24] <Chipaca> hmmmm
[22:24] <Chipaca> wait
[22:24] <wgrant> I don't think this can be a server-side change.
[22:24] <Chipaca> why is the store not saying something like "404"?
[22:24] <wgrant> But let me check.
[22:25] <wgrant> Chipaca: It's a metadata request.
[22:25] <wgrant> It takes multiple snaps, so can't 404.
[22:25] <Chipaca> right
[22:25] <wgrant> Unless it's meant to, that'd be weird and a bug in the new services.
[22:25] <wgrant> Well, a bug in the API
[22:25] <wgrant> And a bug in the new services for not emulating that API bug
[22:25] <Chipaca> :-)
[22:26] <Chipaca> I didn't remember metadata requests started with the _embedded/clickindex:package thing, thought that was just search
[22:26] <thomi> https://bugs.launchpad.net/snapd/+bug/1693042 filed
[22:26] <mup> Bug #1693042: snapd will refresh to a channel that does not exist <snapd:New> <https://launchpad.net/bugs/1693042>
[22:26] <thomi> sadly it's everything that CPI used to return.
[22:26] <Chipaca> so, about the message, that's the correct message for when you switch to a channel that has the same snap revision as you already have
[22:27] <Chipaca> ie if you then do "snap refresh --channel=<whatever you were in before>" it'll say refreshed
[22:27] <thomi> Chipaca: but you agree that it's the wrong message for the case where the channel you're switching to doesn't exist?
[22:27] <thomi> yeah, that makes sense to me
[22:27] <Chipaca> thomi: the bug is that it switches successfully to a channel that doesn't exist, indeed
[22:27] <thomi> I must remember to refresh my core back to a channel that exists :D
[22:29] <wgrant> Chipaca, thomi: Confirmed that the same thing happens on CPI, so it's not a regression.
[22:30] <thomi> Good.
[22:30] <wgrant> There is a slight behaviour change in that CPI didn't include _embedded/clickindex:package at all if it was empty, but snapd doesn't care about that.
[22:31] <thomi> oh, are you sure? I could imagine perhaps snapd would fail to parse the json of that was missing, resulting in a different error path
[22:31] <wgrant> I just tested.
[22:31] <thomi> ahh ok
[22:31] <wgrant> Overriding SNAPPY_FORCE_CPI_URL=http://cpisnap.ols.internal/api/v1/
[22:32] <Chipaca> so, here's one thing i don't understand
[22:32] <thomi> wgrant: ahh
[22:32] <Chipaca> the metadata response is supposed to be
[22:32] <Chipaca> a json document with 'snaps' and 'fields'
[22:32] <Chipaca> e.g. something that starts {\"snaps\":[{\"snap_id\":\"....
[22:33] <Chipaca> and that's what you get when you refresh to 'beta'
[22:34] <Chipaca> actually, give me a bit
[22:34] <wgrant> You might think that
[22:34]  * Chipaca 's not too bright right now
[22:34] <wgrant> snapd probably strips the extra wrapping pretty early.
[22:34] <wgrant> But checking.
[22:35] <thomi> it's parsed in store.go
[22:35] <wgrant> type searchResults struct {
[22:35] <wgrant>         Payload struct {
[22:35] <wgrant>                 Packages []snapDetails `json:"clickindex:package"`
[22:35] <Chipaca> the log is from the wire, directly
[22:35] <wgrant>         } `json:"_embedded"`
[22:35] <wgrant> }
[22:35] <Chipaca> which is what i'm looking at
[22:35] <wgrant> That's used for metadata as well
[22:35] <wgrant> Chipaca: The *request* looks like what you pasted
[22:35] <wgrant> The response is not.
[22:35] <Chipaca> ah, sorry then
[22:36] <Chipaca> i should redo this without the macaroon; it makes it very hard to read the logs
[22:38] <Chipaca> does the same thing happen when refreshing more than one snap i wonder?
[22:38] <Chipaca> (yes it's a different codepath)
[22:38] <Chipaca> (shut up)
[22:39] <wgrant> Chipaca: It's not possible to override the channel for a multi-snap refresh, but I imagine so.
[22:39] <Chipaca> ah, of course it won't happen
[22:39] <Chipaca> because you can't set channel in the other codepath
[22:39] <Chipaca> (that's why the single snap codepath exists and is different)
[22:39] <wgrant> If they were already on an invalid channel they'd erroneously think they were up to date, though.
[22:40] <Chipaca> wgrant: they'd think that because we told them so, you mean? :-)
[22:40] <wgrant> Chipaca: Well, ish.
[22:40] <wgrant> Chipaca: Long term we probably want the server to be able to report "hey, this channel doesn't exist any more, you should probably warn your user"
[22:40] <wgrant> Which makes the distinction between "there is no update" and "there is nothing at all" important.
[22:41] <wgrant> Even on non-switching refresh
[22:41] <Chipaca> ah that reminds me
[22:41] <wgrant> And bulk refresh
[22:41] <Chipaca> niemeyer: is there anywhere i can read about the reasoning behind not having snapd boot a system of a branch when the branch closes?
[22:42] <niemeyer> Chipaca: Probably not in the depth you'd like to hear about, but I'm happy to provide the proper rationale here or in the forum
[22:43] <Chipaca> niemeyer: and perhaps more importantly, a branch that closes can't ever be re-used, right?
[22:43] <niemeyer> Chipaca: It can, and that's one side of the argument.. we have existing behavior in that regard
[22:44] <niemeyer> Chipaca: Think beta closing
[22:44] <Chipaca> well.. beta has a defined meaning around risk
[22:44] <Chipaca> hotfix, a lot less so
[22:45] <Chipaca> and my hotfix could well be my neighbour's nightmare
[22:45] <niemeyer> Chipaca: Interestingly and neatly, the behavior is exactly the same in both cases as far as channels are concerned
[22:46] <niemeyer> Chipaca: beta falls back to candidate once closed.. the branch falls back to its underlying risk level
[22:46] <Chipaca> niemeyer: I can see the mechanics at our end being the same and that being nice and consistent, but I fear the two things are going to be used very differently and that's going to get users into trouble
[22:47] <niemeyer> Chipaca: In my mind it's quite the opposite.. It sounds terrible to fiddle with configuration behind the administrator's back
[22:47] <Chipaca> basically the scenario where today there's a stable/hotfix for problem A, that's incorporated into stable and hotfix is closed, and then that makes problem B happen, so a hotfix for that is put out that reverts
[22:48] <Chipaca> i'm _just_ talking about letting branches be reused
[22:48] <kyrofa> thomi, wgrant so the refresh versus install test wasn't real. The real issue with my branch not showing up is a communications breakdown between the dashboard and the backend?
[22:48] <niemeyer> Chipaca: Okay, I don't understand the case you just described
[22:48] <Chipaca> (it's bad in both cases, the reverting-to-underlying and the non-reverting-to-underlying)
[22:48] <Chipaca> (for different but similar reasons)
[22:48] <Chipaca> aww
[22:48] <thomi> kyrofa: that's a reasonable summary, yes
[22:49] <niemeyer> Chipaca: If stable/fix-terrible-problem is closed because stable has the fix, it will fallback to stable.. end of story?
[22:49] <Chipaca> niemeyer: i need a way to just dump my brain state somewhere for you to look at
[22:49] <Chipaca> writing is too much work
[22:49] <niemeyer> Chipaca: :D
[22:50] <Chipaca> niemeyer: the way i imagine branches being used is to address urgent problems with particular users (or classes of users)
[22:50] <niemeyer> Chipaca: Yeah, that's one reasonable use case
[22:50] <kyrofa> thomi, if I release to another branch, will that interfere with your debugging?
[22:50] <Chipaca> e.g. you put out a stable, a week later it's discovered it's quietly corrupting rpi3s that have a particular wifi card
[22:51] <Chipaca> you can't bump roll back the release for x reason
[22:51] <Chipaca> you can't tell them to jump to beta for some other reason
[22:51] <Chipaca> so you put out a hotfix for them
[22:51] <thomi> kyrofa: should be fine
[22:51] <Chipaca> niemeyer: then you fix the underlying problem (or you think you do!) and close the hotfix branch after the next release to stable
[22:52] <niemeyer> Okay, so far with you
[22:52] <Chipaca> niemeyer: so now the users that are on hotfix are tracking stable again
[22:52] <niemeyer> Yeah, woohay
[22:52] <niemeyer> Bug fixed, people are back into the normal line
[22:53] <Chipaca> niemeyer: but now you find out that what you thought was a fix from the above problem is now making printers on people that have a particular bluetooth card on rpi 2s print nasty messages
[22:53] <Chipaca> niemeyer: so, you put out a hotfix for those users
[22:53] <Chipaca> the hotfix is basically undoing what you thought was a fix (but not quite, because of other unrelated changes)
[22:54] <Chipaca> niemeyer: with me so far?
[22:54] <niemeyer> Okay.. the problem there is that the branch name was reused, which is a very bad idea if you are convey two opposite meanings to the same content
[22:54] <niemeyer> Even snapd aside, people might end up installing the wrong thing because they've read someone's blog post abou tit
[22:55] <niemeyer> So, IMO a case of "doctor, it hurts!"
[22:57] <Chipaca> niemeyer: so how would this play out, in your view? what's the bit that went wrong?
[22:58] <niemeyer> Chipaca: Reusing the branch name for something that is not intended to be a real sequence..
[22:58] <Chipaca> niemeyer: right, how is that being explained/communicated/enforced?
[22:58] <kyrofa> thomi, alright, done. I'm now unblocked
[22:59] <niemeyer> Chipaca: This is just like any other channel.. when we put two sequential things into a channel, we need to have in mind that people might install the first, the second, or both in succession..
[22:59] <thomi> kyrofa: \o/
[22:59] <Chipaca> thomi, wgrant, spotted where we mess up i think (still digging though)
[22:59] <niemeyer> Chipaca: and again, this is not specific to branches.. every channel behaves like that
[22:59] <wgrant> kyrofa: Releasing to another branch would have actually run into the same problem, but we've fixed both now.
[22:59] <wgrant> kyrofa: Your missing branches are no longer missing.
[23:00] <wgrant> kyrofa: We're investigating how this happened.
[23:00] <kyrofa> wgrant, thanks for that
[23:00] <Chipaca> niemeyer: ok, i need to think about this some more. you're probably right, but something has me uneasy
[23:00] <niemeyer> Chipaca: Thanks for trying to find out what it is.. sometimes we do find curious edge cases with those conversations
[23:10] <noise][> fwiw, i'm in total agreement with niemeyer here on the branch behavior
[23:11] <noise][> publishers can absolutely screw their users if they repurpose a branch name for different cases, so they shouldn't do that, and we should make it clear in the docs
[23:11] <noise][> but there's absolutely cases where you'd want to re-open a closed branch
[23:26] <wgrant> kyrofa: We're still investigating how nextcloud got into that bad state, but it looks like it might be a race in dashboard.snapcraft.io's automatic review process. The new store infrastructure noticed the data inconsistency and refused to accept the channel updates. We'll get alerted if it happens again, and we have a button to fix it immediately.