[00:53] PR snapcraft#2143 opened: lifecycle: don't clean priming area if the snap is being tried [02:47] PR core-build#11 closed: remove cruft from the writable-paths [02:47] PR core-build#22 closed: unit testing for sync_dir() [02:47] PR core-build#26 closed: move most of the customization into the core snap build [02:48] PR core-build#11 opened: remove cruft from the writable-paths [02:48] PR core-build#22 opened: unit testing for sync_dir() [02:48] PR core-build#26 opened: move most of the customization into the core snap build === chihchun_afk is now known as chihchun [03:10] PR core#38 closed: Add another pi-config option [03:11] PR core#38 opened: Add another pi-config option [03:11] PR core#83 opened: move most of the ubuntu-core config deb into the snap snap build === pbek_ is now known as pbek [04:46] PR snapd#5190 opened: tests: new parameter for the journalctl rate limit [05:05] morning [05:45] PR snapd#5182 closed: snapstate/hooks: reorder autoconnect and reconnect hooks [06:00] mborzecki: hey, if you don't mind I look at the spread test for the symlink PR [06:01] 5189 looks like an easy win btw [06:01] mvo: hey, go ahead, had that in my queue but i'm ok looking at something else [06:02] mborzecki: thanks, it pretty trivial, I will push something in a sec. got a bit of ocd sometimes, sorry [06:03] PR snapd#5189 closed: interfaces/many: miscellaneous updates for default, desktop, desktop-legacy, system-observe, hardware-observe, opengl and gpg-keys [06:03] mborzecki: \o/ [06:07] pstolowski|afk: when you are around, can you please remind me what the problematic PR on master is that we need to look into reverting (for now). I would love to spend some time on 2.33 today [06:08] mvo: i can try to look into exposing appstreamid (or common-id as it was named in the forum) [06:08] mborzecki: yeah, that sounds great [06:09] mborzecki: I think we also need to ensure the roadmap is up-to-date after the last sprint, this was not updated yet, was it? [06:11] mvo: no, i don't think so === chihchun is now known as chihchun_afk [06:19] mborzecki: ok, I will do that in a bit then [06:26] mvo: could you approve rev 2 of test-snapd-eds? It's the same as rev 1 but built for i386 === chihchun_afk is now known as chihchun [06:27] jamesh: sure, let me do this now [06:27] thank you [06:32] mvo: echo tests/main/snap-core-symlinks/task.yaml >> tests/unit/spread-shellcheck/must [06:34] 'error: cannot pack "/home/gopath/src/github.com/snapcore/snapd/tests/lib/snaps/snap-hooks": unable to copy /home/gopath/src/github.com/snapcore/snapd/tests/lib/snaps/snap-hooks/meta/hooks/install to /tmp/snappy-build-416974275/meta/hooks/install: no space left on device' [06:34] https://api.travis-ci.org/v3/job/382511824/log.txt current master build [06:41] mborzecki: shellcheck> aha, right, let me add this [06:41] mborzecki: oh? master broken? [06:42] mvo: some tests failed in prepare or eexcute due to no space left, looks weird, restarted it for now and we'll see if it repeats [06:42] mvo: we seem to clean up snappy-build though [06:42] is there anything we keep in /tmp during the tests? [06:45] mborzecki: I can't think what could fill up /tmp - long shoot - maybe we hit the bug that we discussed yesteday where the state grows? === pstolowski|afk is now known as pstolowski [07:14] mornings [07:14] mvo: i'll prepare it for you in a moment [07:14] hey pstolowski [07:14] pstolowski: thank you! [07:18] mvo: we want to target master with this revert right? [07:20] pstolowski: yes [07:20] ack [07:20] pstolowski: and once that is in I will branch 2.33 [07:20] yep [07:39] pstolowski: hey [07:39] o/ [07:55] mvo: there's a conflict in #5134 i can push a quick fix if you're busy [07:55] PR #5134: Shrink image generated with snap prepare [07:58] mvo: pushed :) [08:00] mborzecki: heh, thank you [08:10] PR snapd#5191 opened: snapstate: revert the addition of "reconnect" task [08:10] mvo: ^ [08:11] pstolowski: nice [08:11] mvo: no, it's depressing :(. but good we found it early enough [08:12] pstolowski: indeed, this won't affect anything that was using stable, right? so its fine, we push it back one release :) and it removes the pressure to fix it ASAP we can step back and look at it carefully [08:13] mvo: right [08:15] * mvo hugs mborzecki for finding another real bugs with the spread-shellcheck [08:32] bug #1772844 is interesting [08:32] Bug #1772844: snapd didn't initialize all the seeded snaps

[08:32] pstolowski: might this be related to the issue you're working on? [08:33] in this one there aren't a bunch of tasks, but snapd gets stuck in 'Automatically connect eligible plugs and slots of snap "gnome-calculator"' [08:34] Chipaca: yes.. but it's on stable on autoconnect. so far we assumed it's only reconnect in edge [08:34] pstolowski: that's the most 'no' yes I've seen in a while [08:37] Chipaca: in other words, yes, most likely same problem. we thought it wouldn't happen on autoconnect [08:38] pstolowski: dangit [08:38] pstolowski: this is Bad [08:38] pedronis: ^ [08:38] pstolowski: I leave you to update that bug appropriately then [08:38] I didn't say, it couldn't happen [08:38] just that is more rare [08:39] but maybe it's actually a different bug that what I think [08:39] * Chipaca puts the bug down and slowly backs away [08:39] that's seeding? [08:40] then it's related but a different bug [08:41] Chipaca: pstolowski: seeding it's always serial [08:42] pedronis: even when one snap has content interfaces and such? [08:43] Chipaca: we don't support that [08:43] but you should do the right thing [08:43] otherwise you are on your own [08:44] pedronis: what's 'that' in this context? [08:44] snapd's performance with a lot of tasks is very very bad :-( [08:45] Chipaca: prepare-image doesn't do the right thing atm [08:45] but this is not prepare image [08:45] our to-be-fixed conflict logic will fix seeding case i think with no special case for seeding? [08:45] pstolowski: no, seeding is special [08:45] if you mess up the order of the snaps [08:46] you are on your own [08:46] unless we teach seeding to sort them for you [08:46] the seed.yaml is available in https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1772844/comments/6 ordering looks vaguely sane [08:46] Bug #1772844: snapd didn't initialize all the seeded snaps

[08:46] except themes [08:46] if it's a default provider [08:46] no idea about that [08:47] aha, good point, let me look [08:47] as I said, serializing is not really a win here [08:47] it makes thing harder not simpler [08:47] but seeding is serial since forever [08:48] pedronis: fwiw if it's serialised wouldn't there be no Done tasks after the Do tasks? [08:48] Chipaca: ? [08:48] pedronis: ha! you are right, its a content interface [08:48] well, the question is what are the default providers listed in gnome-calculator [08:49] pedronis: https://pastebin.ubuntu.com/p/wfqwkFn7WC/ [08:49] ^ plugs from gnome-calculator [08:49] ok, so themes [08:49] themes [08:49] need to come before [08:49] the apps [08:49] or it will never work [08:50] nice one [08:50] I'll tell them about it in the bug [08:50] I mean, nice that we can suggest a workaround so easily [08:50] pedronis: could we detect and warn about this? [08:50] we can but in general we need to think what we want to do about this [08:50] the fact they don't use prepare-image is also an issue [08:51] we could teach prepare-image to do the right thing [08:51] (as I said it doesn't) [08:51] Chipaca: what else is missing for snapshots? I wonder when to branch 2.33 [08:51] PR snapd#5191 closed: snapstate: revert the addition of "reconnect" task [08:51] mvo: daemon and cmd [08:52] Chipaca: ok [08:52] prepare-image doesn't know about default-providers or bases atm [08:52] I think [08:52] mvo: is that ^ right ? [08:52] mvo: 3 files changed, 462 insertions(+), 3 deletions(-) [08:52] pedronis: I will soon need to teach it at least about bases for the core18 image work [08:53] mvo: (without tests) [08:53] pedronis: I think so too [08:54] Chipaca: thats not too bad, I'm sitting on the fence a bit right now if I branch now and merge these bits into release/2.33 or wait until they arrive [08:54] mvo: I don't think snapshots, as they stand, are useful to users [08:54] mvo: there's a ways to go still [08:54] Chipaca: the upside of branching now is that a) ocd b) more time to test if 2.33 builds everywhere etc [08:54] mvo: this is the very very basic functionality [08:54] mvo: but a user could easily mess up their snaps with it as it stands [08:55] Chipaca: ok, cool, I think I will branch in this case and start with getting things test-clean in cosmic etc [08:55] mvo: (also the most user-friendly aspect of snapshots is when they're automagic) [08:55] Chipaca: if they are actively dangerous should we hide them behind a feature flag? like we do for layouts? [08:55] mvo: I think hiding the commands is good enough [08:56] Chipaca: even better, less work. thank you! [08:56] if we want to talk about them I'd present them as a 'feature preview' [08:57] there's a list of caveats written down somewhere :-) [08:59] mvo: did you see my comment about symlinking snapd in core18? [08:59] Chipaca: no, let me look [09:00] Chipaca: in the forum? [09:09] pstolowski: is there a way to unstick my snapd from the megatasks bug? [09:12] Chipaca: I think for seeding we need a stable sort that sorts bases + then default providers + then the rest, and finds and errors on cycles with the default providers [09:12] Chipaca: you need to get onto new edge somehow (which is in fact an older edge image that mvo reverted to yesterday) [09:12] my state.json is 25MB and each task takes 5 seconds to do :-( [09:13] when are we moving to an actual database [09:13] (this is not a puny machine) [09:13] well given our todos, not this cycle [09:13] sounds a goal for 2020 [09:14] pstolowski, Chipaca I triggered a new core sync so soon there should be a core with the reverted PR [09:14] pstolowski: each night we get a new edge so I think this is the best option (fast-track the edge with the reverted pr) [09:15] Chipaca: also probably easier with split out snapd and epochs [09:15] pedronis: buaah [09:16] mvo: right. but i wonder how hard is it to get out of the 'broken' state and regain control [09:17] Chipaca: btw what snaps were involved in your broken change? [09:17] pedronis: core, vlc [09:17] ok [09:17] pedronis: if i manually update them one by one should it fix itself? [09:18] it will stop you to do that, no? [09:18] pending change and stuff [09:18] join #ubuntu [09:18] pedronis: i abort the auto-refresh one every time it starts [09:19] Chipaca: snap refresh vlc then could help [09:19] because all my cores get all hot and bothered [09:19] if you sneak it in with the abort [09:23] pstolowski: the fact that the refresh exploding is "core vlc" still makes me think my theory seems reasonable about what is the bug [09:25] all these prepare-hook-blah tasks are killing it :-( [09:26] ? [09:27] pedronis: i'm still unlucky in reproducing it in unit test. it's probably a problem with setup/mock somewhere [09:27] pstolowski: your 2nd snap need an interface that autoconnects [09:27] to be clear [09:27] hmm [09:28] with core [09:28] like network I suppose [09:32] pedronis: i mean https://pastebin.ubuntu.com/p/8Yg5jq5wGk/ [09:32] pedronis: this is me trying to manually update core on its own [09:33] Chipaca: it dies even on its own? [09:33] pedronis: well, it doesn't finish, and seems to be in a loop [09:33] or just take forever? [09:33] that is stranger [09:34] pedronis: note how e.g. «Run hook prepare-slot-home of snap "core"» is repeated many times [09:34] pedronis: with different task numbers [09:34] that is expected [09:34] one for each plug [09:34] of home [09:34] I just got tired and aborted it, but even abort is taking ages [09:35] I mean, it might be that even the correct behavior [09:35] is too heavy [09:35] (and the fans are spinning away like mad) [09:35] load average is > 2 of just snapd [09:35] Chipaca: it's going to create connect and hook tasks for each plug to core into the system [09:36] at >2s per task, that's insane [09:36] because your db is big ? [09:36] because of the previous error [09:36] ? [09:36] pedronis: it might be [09:36] or in general [09:36] well, they're never quick [09:36] but the >2s is because my state is too big [09:37] (i'm not going to call it a db :-) ) [09:37] Chipaca: compile a snapd with lower prune times and run it? [09:37] i have 22 installed snaps of different complexity [09:37] we shouldn't create hook tasks at all if not needed, they will be no-op for 99% of snaps i suppose [09:39] 22 installed snaps -> 92 entries in /data/conns [09:39] -> 184 tasks [09:39] at 2s per task, that's 5 minutes on just this [09:39] even more [09:40] 6 [09:40] Chipaca: you need to prune your state [09:40] see suggestion above [09:40] but, even at 200ms per task, that's 30 seconds on just this [09:40] that's _still_ too much [09:41] this is reverted [09:41] but yes, it might be that we need to think [09:41] what reconnect really needs to do [09:41] to be sane [09:42] there are also some O(N²) that could be pared down, but that's for later [09:42] at least reading the code :-) [09:44] pstolowski: reconnect could do the connect only if the snap being installed has hooks on that connection [09:44] but we should probably fix the bigger issue first (infinite retries) [09:45] pedronis: yes, we shouldn't create hook tasks if not needed [09:45] ok, with the pruned state it got through the changes ok [09:46] pstolowski: pedronis: note there's also the thing of regenerating apparmor every time instead of just once at the end, which zyga was going to look at at some point [09:46] (this one is visible in the long time the interfaces-many test takes) [09:46] pedronis: another improvement would be to run reconnect/autoconnect only once if two connected snaps are refreshed in one operation [09:47] That thing may be a but larger but yeah. [09:47] zyga: ohai [09:47] zyga: i thought you were basking in the sun somewhere [09:47] Hi [09:47] I’m going home already [09:48] Also no longer ill [09:48] Long way home [09:51] zyga: https://www.youtube.com/watch?v=qmWC5dGVvH4 [09:54] PR snapd#5192 opened: testutil: import check.v1 differently to workaround gccgo error === chihchun is now known as chihchun_afk [10:08] pedronis: can you think of anything i could have missed in this managers test - https://pastebin.ubuntu.com/p/fDZMGN8Wmt/ ? all tasks done, 2 connections exist as expected at the end, no conflict/retry reported; only unexpected think is the very last check of 'Done' status on entire change (it's still Doing) [10:16] Chipaca: a new core in edge with the reverted reconnect is available now [10:17] installing it [10:17] pedronis: pstolowski: about the tasks taking too long: note that i pruned the state back very aggressively to get unstuck, and the first refresh after that was reasonably fast, but the second one after that is already too slow (taking >30s for the 'connecting stuff' phase) [10:21] pstolowski: well, if it is still doing, then is not done [10:22] pstolowski: do you have a branch I can play with , with this and still the reconnect code ? [10:25] pstolowski: lunch here, let me know [10:34] pedronis: i've pushed it to https://github.com/stolowski/snapd/tree/fix-reconnect-conflictcheck [10:34] pedronis: yes, the change is not done, but all tasks in the state are done (including auto-connect and reconnect) === chihchun_afk is now known as chihchun [10:40] 2018-05-23T10:38:28Z INFO no handler for task "reconnect", task ignored [10:40] i think that may be the first time i see it in action [10:40] turned out to be useful [10:54] pstolowski: is there already a different change in that branch? [10:55] pedronis: yes i changed the logic to check all conflicts upfront [10:55] shouldn't make a difference, but if you want just the test, it's managers_test.go [11:03] pstolowski: it doesn't pass if run together with other tests [11:03] afaict [11:03] ok, that's possible, i was mostly running it alone, let me check [11:04] it gets stuck [11:04] ineed i see it [11:05] it doesn't print anything either though :/ [11:08] pstolowski: I know why it doesn't fail btw [11:08] which is probably unrelated to the other issue [11:08] with all the tests [11:09] pstolowski: the mocking of the response from the server for core has the wrong snap type [11:09] so the ordering doesn't happen [11:09] ahh [11:10] without the forced order the current code is fine [11:10] where is that exactly? [11:10] what? [11:10] that=? [11:12] the mocking; what makes it correct/incorrect? [11:12] that we default to app [11:12] it needs to grow some smarts about type [11:12] uhm, not familiar with that, do we havy any example in any other test? [11:13] no [11:13] by definition [11:13] if we had it would do the right thing [11:13] I'm talking about fillHit to be clear [11:14] let me see [11:15] it's probably easyish [11:19] pstolowski: something like this: https://pastebin.ubuntu.com/p/3HRNvjZMg7/ (haven't checked if it creates other problems with the other tests though) [11:19] it also makes the other test fail [11:19] pedronis: thanks a lot! i'll play with it [11:19] there is still some other problem with that the test and the others though [11:20] mvo: rewrote spread-shellcheck in python in pyyaml, on my desktop the old shell version needed 3:18 to run through 'spread.yaml tests', the new one does the same in 29s, checking the same number of files [11:20] pstolowski: I skipped the new one with that change (@TYPE@), seems it doesn't break other tests [11:25] pedronis: and the new test now behaves completely different (and fails on connections check). now many tests are still not started [11:25] pstolowski: it should print RETRY [11:25] I suppose [11:25] at least once [11:26] pedronis: it didn't. and the tasks are waiting for restart [11:28] and i see we mock restarting in tests.. trying [11:29] ah [11:29] yea, because core [11:30] Hi everyone, what is the proper way to "simulate" a snap refresh? I would like to test the snap refreshing/upgrading before releasing new revisions. [11:30] yes!! [11:30] pedronis: getting RETRY now [11:30] good [11:31] pedronis: thank you very much! [11:37] Wimpress: yup, my session just died when i snap installed something (again) [11:39] mborzecki: nice! [11:39] pstolowski: let me know if you want to discuss a fix [11:40] pedronis: thanks. i need to figure out what makes the test hang when running entire suite, then will work on the fix [11:41] ok [11:49] * cachio afk [11:51] PR snapd#5193 opened: spread-shellcheck: port to python [11:52] PR snapd#5185 closed: tests: shellchecks part 2 [11:53] mvo: do you remember why we used a task adding more tasks and didn't compute bases/default-providers when constructing the change? [12:07] pedronis: I don't remember exactly, I think it was related to the fact that when there are multiple changes that needed the same baseit was difficult to ensure the right ordering and also what to do if one of the changes gets aborted/fails but other changes need the base in question [12:08] mvo: I see but in practice those changes need to conflict (the bit we are missing) so we lose that [12:08] there can be at most one change doing things with one of the bases [12:19] mvo: the context , is me thinking what we can simplify about conflicts etc [12:20] pedronis: right, this is certainly an interessting area to simplify things [12:23] mvo: I'm also thinking this because we need this kind of static detection for prepare-image [12:23] "static" [12:27] * mvo nods [12:29] PR snapd#5192 closed: testutil: import check.v1 differently to workaround gccgo error [12:30] mvo: shall we land #5134 ? [12:30] PR #5134: Shrink image generated with snap prepare [12:35] mborzecki: yes [12:36] PR snapd#5134 closed: Shrink image generated with snap prepare [13:00] so remember how i came here the other day trying to fix the "devmode" off of firefox? [13:00] i did [13:01] and now it's running 60.0-2 [13:01] but the fonts exploded somewhat [13:01] http://paste.anarc.at/snaps/snap-2018.05.23-08.32.25.png [13:01] anyone has a clue wf is going on there? [13:01] no problem in chromium, terminals or emacs [13:04] anarcat: try closing firefox, `rm ~/snap/firefox/current/.last_revision`, then start firefox again (it should take a few moments to launch because it recaches everything) [13:05] diddledan: no go [13:07] hmm [13:07] * ogra_ wonders what you mean by "fix the "devmode" off of firefox" [13:08] ogra_: for some reason, snap wouldn't update firefox recently - it was stuck on 59... there was a "devmode" flag on the "snap info" output... uninstall firefox and reinstall fixed that problem [13:08] but now fonts are all fubar'd [13:08] ah [13:08] thats why [13:08] don't use --devmode [13:09] well you must have installed it with --devmode at some point [13:09] then it wont update ... since thats "developer mode" ... [13:09] well thanks [13:09]