/srv/irclogs.ubuntu.com/2019/04/18/#snappy.txt

=== chihchun is now known as chihchun_afk
=== ErichEickmeyer is now known as Eickmeyer
=== tedg_ is now known as tedg
=== bluesabre_ is now known as bluesabre
=== coreycb_ is now known as coreycb
=== bashfulrobot_ is now known as bashfulrobot
=== chihchun_afk is now known as chihchun
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
=== ShibaInu is now known as Shibe
mborzeckimorning06:20
zygaHey :-)06:38
zygaI still owe you that review06:38
mborzeckizyga: hey :) haha06:40
mborzeckizyga: heh love how unpredictable our spread runs are06:44
mvohey mborzecki and zyga06:48
mborzeckimvo: hey06:48
zygaHey mvo06:50
pedronisseems no mup06:53
pedronishi06:53
pedronismvo: hi, https://github.com/snapcore/snapd/pull/674706:53
mborzeckimup_: ping06:53
pedronismvo: read the description as well, hope it makes sense06:54
mvopedronis: yeah, thank you!06:54
mvopedronis: I'm reading it now (but just started only in 3rd file or so)06:54
pedronismvo: TBH, not surprisingly tests were most of the work06:55
* mvo nods06:55
=== chihchun is now known as chihchun_afk
pstolowskimorning07:13
pedronismvo: I answered the comment, could you reply?07:19
mborzeckizyga: interesting find, on centos, when snapd runs runuser, it somehow triggers keyctl()07:20
=== chihchun_afk is now known as chihchun
mvopedronis: yes, that makes sense07:22
mvopedronis: its more a warning for the future, that makes sense now07:22
pedronismvo: ok, I'll change it, and rewrap a couple of doc comments07:23
pedroniswhich I noticed are bit too long one liners atm07:23
pedronisdone07:28
* mvo nods07:29
pedronis3 of my PRs need a 2nd review,  one probably is best to wait of the prereq to land07:29
* pedronis is going back to do some reviews too07:29
zygamborzecki: keyctl, that's interesting07:33
mborzeckizyga: mhm, found that they adjusted the core policy around rhel 6.5 release, to account for logrotate calling runuser too07:33
zygaI will be in the office in 2007:36
mvozyga: quick question - does 6717 also addresses the bug 1819318 ? the htop implicit slots/plugs?07:38
zygamvo: one moment, let me look07:41
pedronismvo: yes, I don't think I need to review closely 674107:41
zygamvo: no, it is not related07:42
mvozyga: ok, trying to understand the remaining blockers07:44
mvozyga: thanks, I understand now I think07:45
mvozyga: so the fix is to install the snapd snap automatically when the first snap gets installed, thats fine07:45
zygaI would say the condition is:07:46
pedronismvo: ah, that bug, yes07:46
zygaNo core snap, no snapd snap, any app snap: install snapd07:46
zygaRemember that systems in the field are broken07:46
mvozyga: yes07:49
mvozyga: I think we can do that right after 6741 landed07:50
mborzeckizyga: probably interesting for you too: https://github.com/snapcore/snapd/pull/674807:51
mborzeckizyga: still, makes me wonder how / got devpts_t label07:52
mvopedronis: just to double check - for the remodel use-case I just add code to DeviceCtx(.., task, ...) that gets it from the task (and hang it on the right task in devicestate.Remodel() (?)07:56
zygamborzecki: what does -i do?07:57
mborzeckizyga: --interpret07:57
mborzeckizyga:  tehre's an example in PR description07:58
mborzecki(and a commit message too)07:58
zygaback now08:04
pedronismvo: not quite like that08:06
pedronismvo: where do you need the info again?08:06
mvopedronis: I think I have ideas now, I can pastebin in 5-10min what I have. I need the info in the mount-snap handler in snapstate08:07
pedronismvo: I still think we should set the model on the change08:07
mvopedronis: my idea was to define "ModelForTask" (similar to ModelFromTask) that adds a new remodelDeviceContext08:07
mvopedronis: aha, ok08:08
pedronismvo: you should not need new helper08:08
pedroniss08:08
pedronisI did it wrong if you do08:08
pedronisyou need more code in devicestate08:08
mvopedronis: I was thinking I need a new "remodelDeviceContext" struct and then need to add it as data to task/change, yes?08:09
zygaokay08:09
* zyga checks backlog first08:09
zygathen reviews08:09
pedronismvo: yes, you need a remodelDeviceContext correct08:10
pedronismvo: then you need to teach  devicestate.DeviceCtx that if task is passed in08:10
pedronisand it's change has  new-model data attr,  to make a remodelDeviceContext08:10
mvopedronis: cool, so I add remodelDeviceContext and change Remodel() to return a change and in the handlers will get the remodel from the change (via DeviceCtx)08:11
pedronismvo: because Install etc also might check model08:11
pedronisyou also want to make remodelDeviceContext by hand inside Remodel08:11
pedronisand pass to Install Update etc08:11
mvopedronis: right08:11
pedronisas I explained in the PR description its a simple change08:12
pedronisthere's an example of what should happen to install for example08:12
pedroniswe need to do that only for snapstate functions that we plan to use in the remodel08:12
* mvo nods08:12
pedronismvo: this change will need some kind of unit test though08:12
=== chihchun is now known as chihchun_afk
pedronismvo: when there will be a temporary store as well this changes will also help08:14
pedronistogether with the changes I will do around the use of snapstate.Store08:14
mvopedronis: indeed08:14
mvopedronis: for my current PR (allow different kernel) I *may* not need to change Install() as we do the checks late in mount-snap if we can install the kenrel or not. I can still change the signtuare or keep it for the initial PR, what do you prefer?08:15
pedronismvo: as you prefer, if you don't do the change please add a TODO at least08:16
pedronisthough08:16
mvopedronis: thanks, will do - I dive deeper now08:16
pedronis(I might have forgotten everything after my vacation ;) )08:16
mvopedronis: heh :)08:17
pedronismvo: I forgot about ifacestate (because it's using directly  devicestate)08:19
pedronismvo: it's actually clear that except some form for tests, devicestate.Device/SetDevice/Model/Serial should go away08:20
pedronisas public methods08:20
zygahttps://bugs.launchpad.net/ubuntu/+source/snapd/+bug/182529808:21
pedronismvo: daemon etc can use the methods on the DeviceManager itself08:22
pedronismvo: to be clear it's not a blocker for your work, just shows there is a bit more to do08:22
mvopedronis: ok, I think we need to add this to a card somewhere08:22
pedronisand that some aspects of the old devicestate api is dangerous now08:23
pedronisbecause what is the relevant model is not so a fixed concept08:23
* pstolowski doh @ get_journal_log08:26
pedronismvo: don't know if noticed but seems we need to increase the timeout on debian-sid-64:tests/main/sbuild08:33
pedronisI have seen it fail timing out quite often08:34
pedronishttps://api.travis-ci.org/v3/job/521593407/log.txt08:34
pedronis(maybe is just a symptom, anyway908:35
=== chihchun_afk is now known as chihchun
pedronisChipaca: degville: some wondering from me that you might consider in my comments to #6669, also Chipaca that one is something for you to review when you have a bit of time08:59
Chipacapedronis: I wasjust about to say08:59
Chipacapedronis: 'expiration' is a date08:59
Chipacanot a duration08:59
pedronisyes, not a fan08:59
Chipacaand obviously it makes no sense for this to be a date08:59
pedronisno09:00
Chipacaautomatic-snapshots.keep ?09:01
Chipacathen it could be a duration or "no"09:01
pedronisthat is too terse09:01
pedronisfor me09:01
pedronisChipaca: we are trying to get rid of automatic-snapshots09:02
pedronisit needs to be snapshots.something09:02
pedronisalso09:02
Chipacaright09:02
zygasnapshot.retention09:02
pedronisauto-saves-retention=2h or no09:03
pedroniswould seem ok (a bit clunky but ok)09:03
pedronisI don't want to just have retention because it doesn't apply09:03
pedronisto other snapshots09:03
zygammm09:03
Chipacasnapshots.auto-saves-retention seems a'ight09:03
zygasnapshots.automatic.retention?09:04
Chipacaooh09:04
Chipacaschnazzy09:04
pedronisnesting09:04
pedronislosting09:04
zygalosting? :)09:04
Chipacasnapshots.automatic.on-removal=no09:04
Chipaca:-)09:04
pedronisI'm 50/50 whether I want more nesting or not09:04
zygaflip a coin09:05
zygaI take tails09:05
pedronisnot today09:05
* Chipaca watches it land on its edge09:05
pedronisno coin flipping before holidays09:05
Chipacashouldn't've used a pound coin09:05
pedronishard rule09:05
zyga"god does not throw a coin" said someone, somewhere09:05
Chipacazyga: *play dice09:06
zygaI know :D09:06
zygain absence of dies you use dimes ;)09:06
Chipacazyga: clearly einstein was a lousy DM09:06
pedronisChipaca: do we have anything already with two levels?09:06
Chipacayes09:08
Chipacapedronis: service.rsyslog.disable09:08
Chipacapedronis: and ssh, same09:09
degvillepedronis: Chipaca: ...ah, I've only just seen this discussion :) for some reason my IRC session was stuck a few pages up. Actually, my nick had disconnected.09:22
Chipacadegville: the internets. because just simple networks were too reliable.09:24
pedronisChipaca: pstolowski: let's go with snapshots.automatic.retention09:24
Chipaca\o/09:25
zygawoot09:25
* zyga helped name something :)09:25
pstolowskipedronis: sounds good09:26
pstolowskiand gives room for more options under "automatic"09:26
degvillesounds good!09:27
zygamborzecki: looking at the volumes PR again09:32
mborzeckizyga: great, thanks!09:32
=== chihchun is now known as chihchun_afk
zygaChipaca: if the string value is relevant then simply explain that09:41
zygaChipaca: my point was that it is a magic string09:41
zygaso some context is useufl09:41
zyga*useful09:41
Chipaca# magic09:41
Chipacazyga: noted09:42
Chipacawait was your comment on the 'snap download --cohort' test done in the create-cohort pr09:42
zygayes09:43
zygaI didn't notice there are two09:43
Chipaca:-)09:43
=== sparkieg` is now known as sparkiegeek
ChipacaI'm off to do some exercise again, bbiab10:15
* Chipaca really goes now10:21
pstolowskifun fact. the semantics of (some) network errors in Go are different between 16.04 & 18.04 - at least in the case i'm interested wrt to auto-refresh issue :/10:38
pstolowskithat's the little unexpected dragon hiding there after all10:43
zygamborzecki: sent partial review on gadget validation10:46
zygapstolowski: https://github.com/snapcore/snapd/pull/6749 one for you, mainly to check if you agree with the rationale10:48
zygapstolowski: and if you are in review mood, maybe also https://github.com/snapcore/snapd/pull/671710:49
pstolowskizyga: yeah i remember about that one, sorry i didn't get to it yet. it needs full focus10:49
zygapstolowski: no apologies needed10:49
zygathe watch is telling me to get up but I sprained my ankle yesterday10:50
zygaeh10:50
=== chihchun_afk is now known as chihchun
=== amurray` is now known as amurray
mborzeckizyga: hm the offset thing is inability of current structure to distinguish a case when there's explicit offset:0 and when no offset was set at all, i have a patch for that in another branch, maybe i should include it in the PR10:57
zygaplease10:57
mborzeckizyga: fwiw it's kind of ugly :/ things become pointers and so on10:57
zygayes, unavoidably so10:58
mborzeckiwish we could do better but not with Go10:58
zygaor start to use magic -1 ;)10:58
mborzeckihaha :P10:58
zygahow would we do better?10:58
mborzeckizyga: we'd need to have a none like thing10:59
zygathat's *exactly* a pointer :)10:59
mborzeckiwell, not exactly, but i won't start that without a glass of beer :P11:00
zygamborzecki: value vs reference but yeah11:01
zygaactually beer sounds good regardlesss ;)11:01
pstolowskizyga: +1 with one remark11:16
zygapstolowski: thanks11:17
zygajust to clarify here before I commit to the code: they are optional in the sense that we don't die on absence of a cookie11:17
zygaunless you think that should change11:17
pstolowskizyga: no, that is fine; we need to have a clue if that happens, and snapctl should give us one (but that shouldn't happen anymore)11:20
* pstolowski lunch11:22
pedroniszyga: I still dislike the solution in 6117, as did originally11:44
zygahmm?11:44
zygaprobably wrong PR number11:45
pedronis671711:45
zygaah11:45
zygabecause of the public data piece (Scoped)?11:45
pedronisyes11:45
zygaI can make it private and write a PublicDeepEquals11:45
zygait's really only public because of unit tests11:45
pedronisthat doesn't sound better11:46
zygahmm11:46
zygaso is it the data piece of the access to the data piece that is problematic in your eyes?11:46
pedroniszyga: the data, we use Plug/SlotInfo too much as standalone things11:47
pedronisalso it makes public a semantics detail11:47
pedronisof which nothing later should care about afaik11:47
zygahmm, not sure I follow, the standalone aspect is that they are too standlone or?11:47
pedronisI still prefer some kind of private maps on Info11:48
zygacounterpoint: maps take more memory, this is literally a bit of information (well, a word because go)11:48
zygapedronis: please add a comment to the PR, I can rework it to use private maps if you strongly prefer that11:50
zygathough personally I think this is okay, I don't see the risk of storing it this way (I only dislike that it is public)11:51
pedronisstrictly speaking we don't even need to attach the maps to the struct11:52
zygainteresting11:52
zygaI think the calling order is a bit unfortunate but with some shuffling I think  you are right11:52
zygaperhaps I remember wrong but AFAIR implicit hooks are added in a separate pass at a later stage so we'd have to return the maps to that stage to proceed11:53
pedronisinfoFromSnapYamlWithSideInfo11:53
pedronisis internal11:53
pedronisso we can change it's signature11:54
zygayeah11:54
zygaI will also explore the idea to load the hooks first11:54
pedronisto return a map or take one11:54
zygathat would be easier, as the maps would be totally internal11:54
pedronisthat might be painful I fear11:54
pedronisanyway all that code is a bit strange (evolved very organically)11:55
zygaI agree :)11:55
pedronisand under weird constraints (some we cannot get rid of though)11:55
zygaI'm happy to rework it, probably tomorrow because today I need to break for a doctor visit11:55
zygait would be nice to rework to something that uses canonical yaml11:56
zygaand a series of passes that transform the input yaml11:56
zygaand yaml is used freely here, it's just the yaml* types that we have there11:56
zygaload yaml, (processing passes), single analysis pass that reads "canonical form"11:56
pedroniswe are still trying to fix a bug, not rework the all thing over 3 months11:57
zygayes :)11:57
zygaI'm only making a point on how it could be cleaned up from the organic growth11:57
=== chihchun is now known as chihchun_afk
zygawe could merge as-is and iterate11:57
zygathe bug would be fixed for 2.3911:57
pedroniswere to we check that a plug and a slot cannot have the same name?11:59
pedronis*where11:59
pedronis*where do we11:59
zygapedronis: we have a checker for that11:59
* zyga looks11:59
zygaohh11:59
zygamaybe we only do in repo11:59
zygalet's check11:59
pedroniswe shouldn't do it only that late11:59
pedronisfwiw11:59
pedronisif that's the case11:59
zygaplugsSlotsUniqueNames12:00
zygafortunately we do validate early12:00
=== cachio_ is now known as cachio
cachiozyga, hey, I found a problem in the test core16-provided-by-core and I think it is realted to ns12:03
zygacachio: tell me12:04
cachiozyga, if you run spread -debug -repeat 2 google:ubuntu-core-16-64:tests/main/core16-provided-by-core12:04
cachiozyga, it fails the second execution12:04
cachiobacause it is not falling back to core12:04
pedroniszyga: it's in validate though, so we do it after everything else12:04
pedronisanyway that's fine12:05
cachioI was debugging this and the problem is about how we reset12:05
zygacachio: how do we reset?12:05
zygacachio: (please tell me all you know)12:05
cachiozyga, the reset.sh uninstall all the snaps12:06
cachiobut core, kernel. etc12:06
cachioand then12:06
cachiodiscard all mount namespaces and active mount profiles12:06
zygaactive mount profiles?12:06
cachiozyga, https://paste.ubuntu.com/p/8tW5M7Bcgy/12:07
cachiois it doing that12:07
zygaah, the comment is confusing, I understand12:07
cachiowhen I run this it makes the test fail12:08
cachioin fact the test is failing master with the same error12:08
cachiodepending of the tests which run before12:08
zygahow does the test fail? do you understand the problem and know what do do about it or are sharing information to get support in fixing it?12:09
cachiozyga, I don't really understand why it is happening12:09
cachioI took the look to the code and I think it is failing in snap-confine-invocation.c12:10
zygahow is it failing?12:10
cachioI mean there is where it should fall back to core and it is skipping that part12:11
cachiobut not sure12:11
cachioperhaps you do :)12:11
zygano, I don't12:11
zygathat part is never skipped12:12
zygaI bet that the problem is indeed in how we reset12:12
pedroniszyga: I'm loooking at it myself, I don't think reworking it from the current state should be a lot of work (last famous words)12:12
zygaand that if we look at the before / after state (before 1st run, after reset on 1st run) you will see the difference12:12
zygapedronis: +112:12
zygapedronis: do push if you have a patch, my remark was really only about that it is fixing the bug as-is and it's green; it's certainly something we can rework12:13
cachiozyga, ok, I'll add some debug to the code to see if that helps12:13
zygacachio: please do, my recommendation is really to start checking for leftovers12:13
zygaif the code fails on 2nd run reliably12:14
zygait is clearly because the environment is no longer the same12:14
zygaright?12:14
cachiozyga, I suppose that12:15
cachioI already compared that12:15
zygawhat state did you compare and what did you find?12:15
cachioand I couldnt detect the diff12:15
cachioeverything inside /run/snapd/ns/12:16
zygathat's not sufficient, the code looks at /snap/*12:16
cachiook12:16
zygamy advice is to compare: /snap/* , /var/lib/snapd/* and /run/snapd/ns12:16
cachiozyga, I'll compare that too12:16
zygaI'm sure that whatever you will surely find will fix not only this test12:16
zygabut many others12:17
cachiozyga, great12:17
zygacachio: you can also compare /proc/self/mountinfo12:17
zygato see if we are leaking something12:17
zygaperhaps also look inside existing mount namespaces, among the set we are not discarding12:17
zygagood luck12:17
cachiozyga, nice12:17
cachiozyga, thanks, I'll continue with this12:18
zygathank you!12:18
pedroniszyga: I have the rework, to you want me to push it separately or on the PR ?12:50
diddledanergh. I wish the build.snapcraft.io would tell me why it thinks the yaml is invalid!!12:53
diddledanhow can I fix it if it won't tell me what's wrong?!12:55
zygapedronis: push it directly there please13:14
pedroniszyga: in meetings, but will do, it seems to work nicely, but it was quick hacking so names etc can maybe be tweaked13:15
zygasure13:15
zygathank you!13:15
=== ricab is now known as ricab|lunch
cachiozyga, well I found the problem13:56
cachiozyga, well, almost13:56
cachioafter reset13:56
cachiozyga, https://paste.ubuntu.com/p/S25ntst2xQ/13:56
cachiocore16 is not installed but it remains in /snap13:57
cachiozyga, https://paste.ubuntu.com/p/NqdzDvjKVg/13:57
cachioI am trygin a fix now13:59
=== tinwood_ is now known as tinwood
=== chihchun_afk is now known as chihchun
pedroniszyga: I pushed it14:27
=== chihchun is now known as chihchun_afk
mborzeckizyga: pushed updates to cross structure validation PR14:32
zygapedronis, mborzecki: ack14:33
zygapedronis: I opened trivial https://github.com/snapcore/snapd/pull/675114:35
zygait's the whole feature sans the tests14:36
zygaat +16,-3 it's the smallest feature branch I can recall :)14:36
=== ricab|lunch is now known as ricab
zygaoff-topic: being in snap world where releases happen often and when they are ready I find the current craze around 19.04 release oddly quaint :)14:47
zyganon the less, time to update14:48
* Chipaca wanders off15:00
cachiozyga, found the issue15:03
cachiohttps://paste.ubuntu.com/p/CwPtSXqcdV/15:03
zygawhere is that code? in the test?15:03
cachiowe need to trim the variable when there is just 1 base snap to remove15:03
zygaaah15:03
zygaheh15:03
cachioI added logs to reset.sh15:03
zygaI see15:03
cachiohehehe15:03
zygahow was that not broken?15:03
zygaas in, not breaking the execution before15:03
zygais the error ignored?15:04
zygacachio: in any case, super nice find, thank you for chasing this! :)15:04
cachioset -e -x15:04
cachioI'll fix it15:04
cachiozyga, and also I'll add a new pr to print the tree for the state and compare it with the initial one15:05
cachiothanks to that I found the problem15:05
zygacachio: please coordinate with Chipaca on that work, he's started this under the invariants theme15:05
cachiozyga, yes15:06
pedronispstolowski: I added some comments to your timings PRs15:19
pstolowskipedronis: ty15:19
pstolowskipedronis: btw are you off tomorrow?15:19
pedronisyes15:19
pstolowskiand then on vacation?15:19
pedronisyes, back on April 3015:21
pstolowskiok15:22
zygamborzecki: 2nd pass on 668815:26
cachiozyga, this is the fix #675315:54
zygamvo: https://github.com/snapcore/snapd/pull/6753 needs a 2nd review15:55
* zyga EODs15:55
zygattyl everyone15:55
cachiozyga, thanks for the review15:55
cachiozyga, #6744 updated16:03
cachiothanls16:04
* cachio lunch16:04
=== pstolowski is now known as pstolowski|afk
mvozyga: thank you16:17
mvoa second review for 6720 would be great16:26
mvocachio: I will release 2.39~pre1 to beta hopefully later tonight, its already building in the ppa16:39
mvocachio: can we move the 2.38.1 snapd snap to candidate? or is there more validation required?17:31
cachiomvo, my validation is done for 2.38.117:44
cachioI was waiting for cert17:44
cachiolet me check17:44
mvocachio: ta17:44
om26erHi! Is there someone I could ping here to get that done quickly https://forum.snapcraft.io/t/please-transfer-flatc/1098019:31
om26er@popey ^^19:32
roadmrom26er: hi I can help19:43
om26erroadmr thanks for verification https://github.com/google/flatbuffers/pull/5293#issuecomment-48465491419:44
roadmrom26er: a github conversation is super helpful because it lets me verify the recipient's acceptance and identity in addition to verifying your email in github matches your snap store developer email :)19:45
om26erroadmr heh, true. I added a comment on the forum :-)19:47
roadmrdone, om26er ✅19:52
om26erroadmr super! thank you. So what would be the next step here, should I delete the autobuild entry I had created in https://build.snapcraft.io/user/om26er ?19:53
roadmrom26er: hm, not very sure about what happens with build.snapcraft.io but I think it would be sane for you to delete it and let upstream set up auto-builds19:54
zygaChipaca: https://eng.uber.com/optimizing-m3/ <- interesting, also in terms of our state engine and memory usage19:55
om26erroadmr ok, so if he setups the autobuild, will I be a "collaborator" by default or would he need to add me again ?19:56
roadmr        om26er you're already a collaborator, this is set up at the snap store level and is done automatically when a snap is transferred19:57
roadmrom26er: as a collab, in principle you could own the auto-build on build.snapcraft.io (you have access to the upstream source, and can publish/release revisions for the snap)19:57
roadmrom26er: so in principle, what you have *should* work, but because the ownership changed, maybe the authentication credentials will go stale and you might need to reauthenticate or something19:58
om26erroadmr ok, thanks a lot for the help :-)20:00
roadmrnp, happy to help!20:01

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!