[00:24] sergiusens: sorry I forgot to put my test results at https://code.launchpad.net/~sergiusens/goget-ubuntu-touch/beBetter/+merge/261804 [00:24] sergiusens: but yeah, it's a +1 for me as well [00:25] Chipaca is a review machine === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk === chihchun_afk is now known as chihchun [07:00] good morning [07:11] good morning [08:36] Chipaca: hey, I think you had a idea how to make https://code.launchpad.net/~mvo/snappy/snappy-gettext nicer (?) i.e. how to get a import so that we can use G() or something similar short, or am I misremembering(?) [08:58] sergiusens: Hey - do you have any update on the review process for the ODROIDC OEM snap? It seems like it's stuck and i got no reply to my latest comments. [08:59] Good morning all; happy Eat Your Vegetables Day! [09:05] mvo: hey [09:05] mvo: yes [09:05] mvo: but only if you promise it'll only export one function (and ideally not something as collisiony as G(), but maybe i'm overthinking it) :) [09:06] Chipaca: we need two :) G() (or L() i don't care) and NG() (ngettext) [09:07] mvo: fair enough [09:07] i guess G and LG don't clash with anything right now [09:07] mvo: very easy to do, tbh; just import it with a . as we do with gocheck in the tests [09:09] Chipaca: heh, sure! great suggestion, thanks! [09:09] Chipaca: its one of these ideas thats obvious *once* you had it [09:10] \o/ [09:36] tyhicks: love your work on the CAP_NET_ADMIN thing, thank you! [09:49] longsleep: dholbach might be able to help you with the review related questions [09:51] rsalveti, hum... how=? [09:51] rsalveti, so mzanetti just asked me an interesting question ... how will phone devs develop system components on the phone once we have no dpkg support at all anymore ? [09:51] dholbach: store related review questions, thought you had review permission as well [09:52] rsalveti, yes, I do - I just wasn't sure if we had a process figured out for reviewing OEM snaps already [09:52] hm, right, that I don't know [09:52] dholbach: well i received some comments, but now the process seems to be stuck as nobody did answer lately :) [09:53] ogra_: I fail to related developing system components and dpkg here [09:53] *relate [09:53] "Supported architectures:Architecture independent" is likely wrong [09:53] rsalveti, today you build your stuff in a silo and then install the debs [09:53] let me take another look at the comments [09:53] dholbach: the only request i got was to rename it to odroid-community - i renamed it in the store only and i am not sure if that is the correct way of action [09:53] or re-build the packages locally [09:54] ogra_: right, if an app I'd imagine it would just be a simple snap, if part of core, I'd imagine you'd need to rebuild/reinstall the framework [09:54] and then install them ... [09:55] rsalveti, if the phone bits including the UI are in the core image ? [09:55] (which is how i understood the planned setup) [09:55] rsalveti, so if I want to test a small patch on unity I need to rebuild the framework? [09:55] ok, the severity of that really depends on what the framework is :D [09:55] i dont think it will be a framework ... unless rsalveti knows something new ... [09:55] for personal you would indeed just follow the current dev steps you do for phone [09:55] dpkg will be available [09:55] ah, "Architecture independent" might actually be right [09:56] rsalveti, so we will have special developer images that ship dpkg ? [09:56] once we move away from personal and get just core + snaps, then we'd need to find the right spot for your piece [09:56] i thought the masterplan was to completely get rid of it [09:56] can you elaborate on the "move away from personal" thing? [09:56] to me it looks like a dev changing core would have to re-build the whole image every time [09:57] ogra_: that is the masterplan, but I don't think that will be removed from personal [09:57] ok [09:57] as i understood some desktop discussions the plan was to only have dpkg fenced inside an lxc container we ship by default [09:57] mzanetti: so personal is a shortcut atm, since transforming everything into a snap (framework and apps), is a lot of work [09:58] jdstrand, beuno, sergiusens: regarding the odroidc oem snap decision: are you aware of anything else that needs to be done? is "arch independent" correct in this case? [09:58] so instead of doing that now, we created another base image that includes everything [09:58] (so you wouldnt have it available on the actual core system) [09:58] but if we keep it i guess thats fine then [09:58] longsleep, ^ I just went ahead and pinged some other folks who should know more [09:58] moving away from personal would mean start using the core + snaps (framework and apps) [09:58] longsleep, sorry for not being of more help here [09:58] rsalveti, you mean converting unity into a snap itself? [09:58] mzanetti: yup [09:59] dholbach: ok great thanks - no worries i am not really in any hurry as long as there is no way to handle the device part with the store. [09:59] mzanetti: how that will be done is not clear yet [09:59] rsalveti, ok. thanks for clarifying. [10:03] rsalveti, but personnal is still a snappy image and ro, so you can't dpkg things there [10:07] seb128: right, but that restriction already exists on phone images [10:07] the use case is for people already remounting things with rw [10:07] rsalveti, you can turn the phone rw by touching a file and rebooting, is that going to work on snappy? [10:08] seb128: that is still an open question (if we're going to offer something similar), but I'd imagine we don't want to do that [10:08] the logic is currently inside the initrd [10:08] not sure if for personal this is something you might want to offer [10:09] rsalveti, but you can mount -o remount rw? [10:09] seb128: if you have sudo access, sure [10:09] seb128, sure, why not [10:10] seb128, the initial question was just "what do we do if dpkg is gone" .... in which case you would have to install binary tarballs or some such [10:10] oh ok [10:10] mvo_: when tarmac complains about "there are additional revisions yadda yadda", it's the timestamp of the top-approval it's complaining about [10:10] well the snappy image is still built from debs [10:10] so no reason dpkg is gone [10:10] but since we will keep it for now even though the masterplan is to have it not in the images, we are fine [10:11] do we plan to stop using debs to build the images? [10:11] long term it will be gone, but by that time the desktop/phone will have switched to frameworks and snaps [10:11] no [10:11] not necessarily, but there is no need for dpkg to be installed [10:11] but we will likely remove dpkg at the end of the build [10:11] if we're not using it [10:11] (and a lot more) [10:12] i would imagine out core image to have only a third of the current size once we are done [10:12] if not even less [10:12] right [10:12] there is a card to investigate that [10:12] https://trello.com/c/JdwIFPwn/57-investigate-if-we-can-remove-apt-dpkg-from-the-image [10:12] yeah [10:12] rsalveti, right, it's just difficult to convince a deb system to remove dpkg :p [10:12] rm :) [10:12] well then no need to debs anymore [10:13] you can also cp [10:13] sure [10:13] perhaps we'll do that one day [10:13] but our infra is still built around packages today [10:13] right [10:13] so why not use them [10:14] on an embedded system where you are only interested in getting enough OS to boot, you dont really want the bloat [10:15] sure [10:15] the core image i envision will only have dash, systemd and a few snapp tools installed (and whatever is needed to make these run) [10:15] beuno: you around? [10:17] Chipaca: mvo_: I know sergiusens was having this issue the other day, but do you guys know why we cna't use --install=docker when creating images? looking at bug https://bugs.launchpad.net/snappy/+bug/1465879 [10:17] Ubuntu bug 1465879 in Snappy "docker framework does not install via ubuntu-device-flash" [High,Confirmed] [10:18] rsalveti: check snappy-devel [10:18] rsalveti: or, https://bugs.launchpad.net/snappy/+bug/1464486 [10:18] Ubuntu bug 1464486 in goget-ubuntu-touch (Ubuntu) "frameworks that install policies cannot be preinstalled" [Undecided,New] [10:19] rsalveti: we should confirm whether it happens with trunk snappy, as it's merged there [10:20] ah! maybe it's still needing u-d-f work [10:23] * rsalveti checks [10:24] rsalveti, Chipaca: it may just need a rebuild of u-d-f against current snappy (shared libs ftw :/) [10:25] guess also backporting https://code.launchpad.net/~sergiusens/snappy/policyRoot/+merge/261802 ? [10:25] or just udf would be enough? (the rebuild) [10:33] Hello, I think there is a mistake in https://developer.ubuntu.com/en/snappy/guides/porting/ under "Snappy enablement basics and the device tarball." [10:33] it says: "system: directory shipping kernel modules that are only needed after the rootfs has been mounted;" [10:33] that is correct [10:34] what would you expect it to be ? [10:34] shouldn't it say modules that are needed before mounting rootfs? [10:34] no, these have to go into the initrd [10:34] ah, right [10:34] (though i'd always recommend that if you use a custom kernel you simply compile that stuff in, it is usually just something like ext4 support and the like) [10:43] The following packages have unmet dependencies: [10:43] ubuntu-snappy : Depends: ubuntu-snappy-cli (= 1.1.2-1+510~ubuntu15.10.1) but it is not going to be installed [10:43] E: Unable to correct problems, you have held broken packages. [10:43] ... from this mornings image build ... [10:43] did anyone look into that yet ? [10:43] (armhf and i386 images failed it seems) [10:45] rsalveti: it should work with the latest build already [10:46] rsalveti: or a rebuild of u-d-f I mean [10:46] sergiusens: morning :-) [10:46] we can check with Built-Using tags [10:46] good morning :-) [10:46] sergiusens: cool, guess we just need to upload a new udf and copy it over to our ppas [10:47] ogra_: hm, that version is in our ppa [10:48] https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/wily/ubuntu-core-system-image [10:48] sergiusens: care to upload a new version once https://code.launchpad.net/~sergiusens/goget-ubuntu-touch/installYaml/+merge/261865 gets merged? [10:48] amd64 built fine [10:48] tarmac is currently complaining about it [10:48] probably a timing error where the arch all side had not migtared [10:48] ogra_: yeah [10:48] i guess someone should just trigger a new build [10:48] rsalveti: yeah, gofmt, no sure how it happened, but it happened [10:49] rsalveti: maybe due to the files I just rebased but did not edit (I have goimports run as a PreWrite or whatever it was called in vim) [10:49] hm, but amd64 is failing quite frequently [10:49] https://launchpadlibrarian.net/209210238/buildlog_ubuntu-wily-amd64.ubuntu-snappy_1.1.2-1%2B511~ubuntu15.10.1_BUILDING.txt.gz [10:49] FAIL: touch_test.go:30: HTestSuite.TestUpdateTimestamp [10:49] mvo_: seems those unstable tests showing up again [10:50] this time for amd64 [10:50] rsalveti: heh, time based test failures ftw [10:50] * ogra_ triggers a new image [10:50] ogra_: amd64 is still ftbfs [10:50] sure, but that shouldnt make armhf and i386 fail [10:50] we we only care about armhf and amd64 atm [10:50] that package comes from the PPA ... [10:51] no proposed-migration there [10:51] so one of the main archs will fail [10:51] (or do we have it) [10:52] damn ! [10:52] * ogra_ read arm64 [10:52] there is another build in progress [10:52] someone triggered it [10:52] silly silly silly naming !!!!! [10:52] haha, /me does that all the time [10:52] well, i triggered one ... might be me [10:52] :-) [10:54] armd64 [10:54] well, at least they fail fast :) [10:55] (i386 is already done) [10:55] ricmm, on compiler level the naming is even worse ... [10:55] aargh64 [10:55] err [10:55] aarch64 [10:58] oh [10:58] amd64 succeeded [11:39] fgimenez: elopio where should I add an acceptance test for https://trello.com/c/PJE5oYAF/84-disabling-updates-and-auto-update-when-using-sideloaded-kernel-snaps ? [11:39] can any of you two help me get started? [11:39] sergiusens, oh, has that landed ? [11:39] * ogra_ needs to not forget to re-enable autopilot [11:39] ogra_: in process of [11:40] k, thanks [11:40] ogra_: got stuck in the pipeline === soee_ is now known as soee [11:41] sergiusens, currently we are adding them in _integration_tests/tests, see https://code.launchpad.net/~fgimenez/snappy/go-functional-tests/+merge/261748 [11:43] sergiusens, i'll push changes shortly to allow adding more than one test file, all of them are now in snappy_test.go [11:45] fgimenez: this doesn't drive image creation though, right? [11:47] sergiusens, yes, calling it with "go run _integration_tests/main.go" should do it all, compile debs, create image and call adt-run on it [11:49] fgimenez: nice, I need to create the image in a different way though [11:50] fgimenez: hmm, maybe for E2E but I can tamper with the image to set what I need [11:50] sergiusens, yep, it's not very configurable atm :) [11:53] fgimenez: should I go with the half test? [11:53] I would like to have E2E eventually though [11:56] sergiusens, how could this be setup? [11:57] Chipaca, hola [11:58] beuno: hola! [11:59] beuno: in CPI, the description is described as “Full description of the app's functionality.”. Is that one of the things entered via the online form, or is it extracted from the package? [12:00] Chipaca, yes. [12:00] Chipaca, extracted from the package, can be edited in the form [12:00] beuno: i'm confused a little, then [12:01] beuno: where is it extracted from? [12:01] beuno: "snappy build" loads it with the description from the readme [12:02] beuno: that is, it sets the manifest's description entry to the readme [12:02] to the second+ line of the readme [12:02] Chipaca, it parses it from the json [12:02] to all the non-blank lines after the first non-blank line in the readme [12:02] beuno: from the json manifest, yes? [12:03] beuno: can we make it not editable? [12:03] beuno: same place you get all the other metadata from? [12:03] Chipaca: for when time happens https://code.launchpad.net/~sergiusens/goget-ubuntu-touch/personalCmd/+merge/262174 [12:03] beuno: because i'm confused, therefore, by the results from the store which seem to prepend the title to the description in the description [12:04] sergiusens, no, because then you need to re-upload a 800mb snap to fix a typo in the description [12:04] Chipaca, same place, es [12:05] Chipaca, I think it might merge 2 fields into the description [12:05] jayteeuk knows the details [12:05] beuno: also the title is not the title, it's the name? [12:05] seriously confused :) [12:05] It wasn't me! [12:05] beuno: right, but then the data is inconsistent as well [12:06] JamesTait: ehlo :) [12:06] sergiusens: holy, there are at least 5 branches waiting to be merged in goget [12:06] if you GET 'https://search.apps.ubuntu.com/api/v1/search?fields=title%2Cdescription&q=hello-world' [12:06] ubuntu-touch [12:06] beuno: or in other words, the local data is useless [12:06] Chipaca, 501 Syntax: EHLO hostname [12:06] 😉 [12:06] sergiusens: can we have an upload after merging https://code.launchpad.net/~sergiusens/goget-ubuntu-touch/installYaml/+merge/261865 ? [12:06] JamesTait: ehlo chipaca [12:06] to split the personal stuff in another upload [12:07] rsalveti: yeah, didn't I says yes already :-P ? [12:07] sergiusens: just wanted to double confirm :P [12:07] JamesTait: if you GET the above uri [12:07] sergiusens, correct [12:07] JamesTait: and look at hello-world.canonical [12:07] rsalveti: split is new though... so one sec I'll set to needs review [12:07] sergiusens, the idea always was to take whatever the store told you instead of what the package says [12:07] JamesTait: or any of the others really [12:07] beuno: so why do all this packaging format if we don't need it? [12:08] sergiusens, initial seeding of the information [12:08] let's just remove icon. description, title, readme.md from the packaging [12:08] JamesTait: it seems to be setting title to ... something? and description to something else, neither of which match the title and description in the click manifest as far as i can tell [12:08] beuno: we can seed with an external file [12:08] no need for it to be part of the packaging if it's going to be useless data [12:08] mvo_: ^ [12:08] Chipaca, without having clicked on those links, let me first put pad.lv/1303354 out there. [12:09] Now let me click on the links and see what else I can add. [12:10] Chipaca, so, "look at hello-world.canonical" where? [12:10] JamesTait: in particular, the title is ... dunno what; the description is the \n-glomming of title and description [12:11] JamesTait: http://pastebin.ubuntu.com/11730186/ [12:12] beuno: sergiusens: easy way to fix that is to "fix" the package when edited [12:12] Chipaca, right, so the latter part of that is addressed by the bug I pasted above. [12:13] Chipaca, do you have a devportal link to that package? [12:13] JamesTait: i'll believe you, but i have no idea what the "app summary" or the "tagline" is, in this context [12:14] Chipaca, ew ew ew [12:14] no! [12:14] ew [12:14] JamesTait: id 1999 <- is that good enough? [12:14] no touching the binaries! [12:15] beuno: you'd rather confuse the user by letting them download things that when downloaded do not describe themselves as the things they chose to download did? [12:15] Chipaca, indeed it is. ☺ [12:15] Chipaca, or, you download the store's json when downloading the app and use that to display [12:15] beuno: you don't have to edit the binary, of course; you can also modify it on the fly when the user downloads it :-p [12:16] * beuno dims his laptop's screen to the minimum [12:16] The "Sorry, can't hear you!" approach. [12:18] beuno: that makes the client a lot harder, especially wrt uniformity of behaviour wrt sideloaded apps; it also n-uplicates information (we'd have the package.yaml, the click manifest, and the store manifest) [12:18] beuno: the data needs to be consistent though [12:18] I think it's bad design [12:18] beuno: and moves work to the client, which is underpowered and numerous [12:18] Chipaca, bear in mind that these models evolved from the old software centre ones, which were based on .deb packages. So tagline is the one-line description and app-summary is the full description. [12:19] JamesTait: http://www.brainlesstales.com/images/2013/Jul/bear-in-mind.jpg [12:19] Chipaca, in snap package terms, that maps to... hang on, checking. ☺ [12:21] deb-control(5) does not call it a tagline nor an app summary, either [12:21] Chipaca, so we make people upload 800mb to fix a typo or set a promo for their app? [12:21] short description and long description [12:21] beuno: we can also split the packaging if it's useless data as you said [12:22] the "metadata" doesn't need to be there [12:22] sergiusens: how is that different from downloading the store json? [12:22] beuno: but keep in mind the no caching rule, the store would be spammed by thousands of clients [12:22] Chipaca: oh, I'm talking about "on upload" [12:23] sergiusens: i'm talking about not understanding how that would be different for the client :) [12:23] Chipaca: the initial seeding problem [12:23] sergiusens: you mean the server would assemble the package from these bits? [12:23] Chipaca: oh, if the store stays this way the client gets more logic, and the store gets more load [12:24] ok, i'm going to move on for now, but this is a problem and needs addressing at some point [12:24] Chipaca: we'd download two bits; the store json (and icon) and the actual package (which would be a list of hashed files) [12:24] * Chipaca comes back [12:24] sergiusens: k [12:25] Chipaca: but let's break the lie about package.yaml and readme.md if it is just to "seed" [12:29] sergiusens: i didn't quite follow you there === chihchun is now known as chihchun_afk [12:29] beuno: tbh i'm starting to wonder whether having the click and snappy stores conjoined is still a good idea [12:29] we seem to be making things harder for ourselves as we go [12:30] Chipaca, I asked if I could double the team size and they counter-offered with taking away half [12:31] beuno: just in case it sounded that way, i'm not saying you or your team messed up; i do believe we've at each step collectively taken the shortest path to get places [12:31] as we should [12:32] Chipaca, :) [12:32] Chipaca, so what would you propose? [12:32] splitting away from parsing the json? [12:32] beuno: i'd propose coffee [12:32] :) [12:32] Chipaca, SOLD! [12:32] I like proposals like that. [12:33] beuno: wrt the store, is it the case that it and the cpi is getting riddled with click-vs-snappy complications? or is it still relatively unified? [12:34] JamesTait: ^ might be for you actually [12:34] * Chipaca puts on the kettle [12:35] Chipaca, I haven't seen keeping that backwards compatibility force us to make compromises [12:35] but JamesTait and nessita would have a more boots-on-the-ground answer [12:35] I'm a little out of touch with the big picture, tbh, but unless things have changed drastically the differences between handling a click package and a snap package are miniscule. === chihchun_afk is now known as chihchun [12:36] on the client side it is a little fiddly, and that's without trying to make the remote-vs-local thing go away [12:36] Like, we parse a yaml file instead of a json file, and we set the is_snap flag to True. [12:36] Chipaca, the phone went through the same remote-vs-local phase, FWIW [12:36] .... [12:36] JamesTait: you parse a yaml file? [12:36] JamesTait, we still parse the json [12:36] * May be simplifying slightly. [12:37] and snap packages copy over the yaml to json :) [12:37] “copy” [12:37] “yaml” [12:37] Chipaca, so, for the most part, we can't split the stores [12:37] because most of our deliverables apply to both [12:37] it sounds like the store is doing the least work, and we're all muddled on the client [12:38] which is not terrible, as long as we can resolve the local-vs-remote without further muddling (and which might explain why we didn't really jump at the download-another-json idea) [12:38] Chipaca, so I think the idea of downloadind a metadata blob from the store is a recurring thing [12:38] the flat namespace is a good example [12:39] another one will be the apparmor profile [12:39] which we'd like to split out and have the store attach to apps at some point [12:40] my guess is we'll end up downlading and storing a separate blob of metadata anyway [12:40] hello apt lists [12:41] :P [12:41] ogra_, you mean, parsing out of a file on one path instead of another? :) [12:42] well, i thought the scope of snappy was to get right of index files [12:42] also, if we're re-implementing apt, lets bring back dependencies! I think that's what makes it fun [12:42] *rid [12:42] beuno: i am not opposed to downloading and storing separate metadata*; i'm opposed to having three different metadata sources :) [12:43] ogra_, index files *with all apps in the archive* [12:43] ogra_, not the ones you have :) [12:43] Chipaca, SO PICKY! :) [12:43] Chipaca, agreed [12:43] * i'd quesiton whether it needs to be separate on disk, or could be on-the-fly glommed into an ar (to make debugging easier maybe) [12:44] +1 on having one source of information [12:44] Chipaca, so my naive mental model here is that you download the json blob from the store on download, refresh it every now and then, always query that for any metadata [12:44] fwiw, I dislike the idea of having metadata that is not reflected in the package. mostly on philosophical grounds right now (can not be version controlled for example and packages are no longer self-contained). but thats just me [12:45] plus its something we need to authenticated seperately, ideally having a gpg signature for the blob too [12:45] mvo_: that's why I asked for the store not to allow edits [12:45] beuno: "refresh it every now and then" is problematic [12:45] sergiusens: I like that better than the alternatives [12:45] beuno: otherwise, with you [12:45] beuno: mark said no caching [12:45] Chipaca, I'm sure cron accepts "now and then" as a parameter [12:45] especially since when we have delta downloads it won't matter if someone uploads a new version [12:45] mvo_: problem is delta uploads :) [12:46] sergiusens, I don't think he meant that. We've discussed downloading a metadata blob separate from the package many times [12:46] mvo_, it won't matter to the user [12:46] Chipaca: is it ;) we could make that work easily via snappy upload [12:46] mvo_, it'll matter to the developer uploading 800mb [12:46] beuno: right, but caching on the client he said no; ask lool ;-) [12:46] mvo_, and spamming 100k users with a no-op download to s/downlod/download [12:46] beuno: if we sign hashes.yaml thats a solvable problem [12:47] beuno: we spam even more users if we cron a apt-get update like snappy run [12:47] sergiusens, I'll bring it up, I think it will have been a different context [12:47] don't get me wrong I'm not totally opposed but I dislike it [12:47] i think there is a workable way [12:47] and I think it will come back and hunt is in bad ways [12:47] mvo_, I wouldn't cron, I'd update the list whenever you talk to the store, snappy update, etc [12:48] there will always be metadata outside of the package though [12:48] like reviews [12:48] right [12:48] and, again, we already have the flat namespace [12:49] all the metadata that the client might require while offline needs ot be in the package [12:49] well [12:49] that's the issue here [12:49] we disagree on that [12:49] ummm [12:50] maybe :) [12:50] lool: there is metadata you don't have when you're offline [12:50] Chipaca: yeah, like reviews, that's fine [12:50] yep, and all those don't need to live with the package [12:50] but you need things like the icon, the translated name [12:50] on disc [12:51] however, the package when the user creates it is a series of metadata chunks, and the binary itself [12:51] that's then sliced and diced into a .snap file [12:51] and signed [12:51] webdm ad click scope see a lot of problems with the current model fwiw [12:51] so you have a file that is, conceptually at least, a series of blobs and their signatures [12:52] s/see a lot of problems/haven't implemented separate metadata storage [12:52] when you download the package, the blobs get put in certain places, and you get the original package back again [12:52] maybe it's just confusing to have a general discussion on metadata and we only need to list the actual data and use cases [12:52] currently, the store treats the package itself as a single blob [12:52] however that does not need to stay that way [12:53] the store could treat it as a series of blob,sign pairs [12:53] and stream those [12:53] and then on unpack we'd put the metadata blob wherever, and would be none the wise [12:53] right [12:53] indeed we could [12:53] while on the server the metadata blob was stored in a database, instead of in disc, and editing happens right there [12:53] there is metadata that the store will provide that the package won't, like UUID [12:54] so the file you give the store might not match exact what is served back [12:54] and the overall signature also [12:55] but that gives an on-disc and on-the-wire format that are the same, and an in-store format that lets us edit metadata [12:55] * beuno nods [12:55] and it's all *almost* there [12:55] beuno: it's fine for the store to provide more data; it's bad for it to modify provided data [12:55] we've just not been thinking about it in these terms [12:55] sergiusens: why is it bad? [12:55] sergiusens, it isn't, the user is! [12:56] gosh this coffee is strong :) [12:56] and I don't want to force the user to re-upload 800mb to fix a typo, or even generate a new file and upload that [12:56] there's a nice web ui for them to do that [12:56] Chipaca: the packaging layout is meant to be navigateable so the user can browse what's installed easily, if I look at package.yaml it will be all wrong [12:56] sergiusens: you weren't following me maybe [12:57] sergiusens: or maybe i wasn't clear [12:57] Chipaca: if we delete package.yaml (or most of it) then fine [12:57] and please lets get rid of readme.md [12:57] sergiusens: the on-disc (and on-the-wire) format of the snap package has all the metadata in one place; call it the package.yaml [12:57] Chipaca: that's _$version [12:58] which we talked about [12:58] sergiusens: the store streams out the package.yaml from the database, and the binary blob from wherever; the client just gets a stream with package.yaml in there [12:58] Chipaca: ok, and does that need constant updating? [12:59] sergiusens: that is a separate discussion i have no horse in [12:59] Chipaca, sergiusens, another example of out-of-package metadata is release [12:59] sergiusens: it could be, it could not be [12:59] sergiusens: also, this might make _$version not necessary [13:00] but we are talking something that is not jfdi-level of change [13:00] and one that is convenient, because we can re-target the same binary to newer releases without changing it [13:00] * JamesTait heads off for lunch [13:01] beuno: I know of all these extras, I just don't like the sources being mangled with [13:01] beuno: while we're at it, if in the store a binary is a stream of (data,signature) pairs, you could store and stream deltas instead of whole packages [13:01] beuno: if the source is the store and the only reason we have a package.yaml is to seed, then it shouldn't be there at all [13:02] Chipaca, not sure I follow, deltas is still being pondered by mvo, on where to slice [13:02] per file, binary deltas, etc [13:06] beuno: i'm not sure we're talking of the same level of deltas; what i mean is you could xdelta v1 and v2 and only store/stream the xdelta between them (looking at the binary blob alone) [13:06] but that's probably exactly what mvo is looking at :) [13:06] and it's even more pie-in-the-sky than the rest of the above [13:06] sergiusens: i'm not sure i understand your "seeding" argument [13:08] Chipaca: that was martin's initial argument, package.yaml is in the package just to seed the initial store configuration. [13:08] Chipaca: and that it shouldn't be relied upon [13:09] sergiusens: but that's not where we finished, is it? [13:09] Chipaca: if we stream the files as you say, this may be not relevant but is considered "tampering" with the packaging [13:09] Chipaca: changing uploaded files is weird, but then again it's not as the same user is changing them from a webform [13:10] unless they do package signing themselves... [13:11] sergiusens: we could build and sign the binary blob (and by binary blob here i mean everything-user-provided-but-meta/) with the user's key, and sign the meta blob with our key [13:11] sergiusens: because meta is “ours” [13:11] that's the whole point [13:12] well, it's one of the whole points :-p [13:12] beuno, Chipaca , JamesTait: correct that the store treats snaps and clicks 95% equally. And yes, for now we only support .json files for the package metadata on upload [13:12] Chipaca: your idea makes me happy; well anything that gets us out of N sources of information makes me happy [13:13] sergiusens: note my idea is 95%* reinterpreting what we already have [13:13] * some statistics copied from nessita [13:13] lol [13:13] and then following up those reinterpretations, but nothing "new" really there [13:14] anyway, somebody should write this down before we forgot we agreed on something [13:14] Chipaca: that's why we have architects; let's leave that to th people about to sprint :-P [13:14] * beuno hits control + p > A4 > Print [13:14] i'll send an email [13:15] beuno: +1 :) [13:15] Chipaca: I sent a short one to rsalveti to start discussing during the architecture meetings [13:15] Chipaca: before I thought the conversation would stop short (as it did most of the time) [13:16] * sergiusens now feels hungry after what just happened [13:22] sergiusens, hey, I saw that you have goget changes up for review for personnal, any idea when that should land? [13:29] seb128: in a bit [13:29] seb128: but the images I create go into a boot loop (kvm) [13:29] seb128: did you have a succesful build yet? [13:29] sergiusens, yeah, I don't understand [13:29] grub has 4 entries [13:29] the first one which is snappy boot and goes back to grub [13:30] the "ubuntu" one fails to boot/hang in the middle of init jobs [13:30] I tried to boot on upstart, it stops on what seems like cloud-init's job having issues with the fs being ro [13:30] exceptions about creating a dir [13:31] sergiusens, yeah, I got it to work on amd64, doesn't work on i386 though (it's still giving me the partitions error, even with the 10G) [13:33] seb128: there shouldn't be differences there [13:34] seb128: also try sudo losetup -d /dev/loop[0-9] before starting [13:34] sergiusens, thanks [13:34] sergiusens, anyway the amd64 issue is having those boot issues, I'm trying to have a look but didn't find anything useful so far [13:34] did you? [13:35] seb128: no, this was late last night [13:37] hey JamesTait - I have a question :) [13:37] JamesTait, if I use the app store APIs, can I filter for certain snap types? [13:37] dholbach, "certain snap types"? [13:38] JamesTait, like a gadget snap, a framework snap, etc [13:39] https://developer.ubuntu.com/en/snappy/guides/package-metadata/ → 'type' [13:39] dholbach, I don't know if we have that metadata available. The analogue in click packages would be content: app|scope I think. [13:40] JamesTait: we do [13:40] JamesTait, can I filter for snaps right now, as opposed to clicks? [13:41] JamesTait: dholbach https://search.apps.ubuntu.com/api/v1/package/docker "content" in there [13:41] as an example [13:41] JamesTait, we do have it, in the same field [13:41] allowed types for far are application, framework [13:41] (for snaps) [13:41] nessita: and oem [13:41] nessita, right, I was just trying to find one. ☺ [13:42] here's an oem one https://search.apps.ubuntu.com/api/v1/package/beagleblack [13:42] dholbach: to filter pass X-Ubuntu-Framework: ubuntu-core-15.04-dev1 (when can we rid ourselves from this? :-P) [13:43] dholbach, https://search.apps.ubuntu.com/api/v1/search?q=content:"framework" [13:43] dholbach: and also X-Ubuntu-Release: [15.04-core|rolling-core|rolling-personal] [13:44] sergiusens, what do you mean with getting rid of the framework header? [13:44] JamesTait: can we do that with release and framework as well? [13:44] nessita: as in ubuntu-core-15.04-dev1 [13:44] nessita: it's a meta question, I know it was set back in due to some apparmor issues as well ;-) === rickspencer3_ is now known as rickspencer3 [13:45] sergiusens, yes, you can. [13:45] kyrofa: do you have time to meet today? [13:45] JamesTait: neat, which is more efficient? as query param or http header? [13:46] rsalveti: https://launchpad.net/ubuntu/+source/goget-ubuntu-touch/0.24-0ubuntu1 (why it is in no pocket I don't know) [13:47] sergiusens, they're slightly different. As an HTTP header makes it a filter, so the index will only return results that depend *only* on frameworks that are in your list. [13:47] sergiusens: in a vortex :-) [13:47] sergiusens: thanks [13:48] sergiusens, if you send "X-Ubuntu-Frameworks: ubuntu-core-15.04", then packages that declare 'framework: ["ubuntu-core-15.04", "docker-1.0"]' won't be returned. [13:48] sergiusens, whereas just putting it in the query string will return them regardless. [13:49] JamesTait: so http header is all or nothing? You make me ask questiong about our code base now :-P [13:51] sergiusens, HTTP header filters out stuff you can't install (due to missing frameworks). So phones don't see packages targered at core, for example. [13:51] Yet, anyway. [13:53] JamesTait: but it's an || and not and && right? [13:54] JamesTait: as in you have to satisfy at least one of the declared frameworks === chihchun is now known as chihchun_afk [13:56] sergiusens, you have to satisfy all of the frameworks that the package declares. It's neither || nor && because you might have a bunch of other frameworks installed that make no difference to the package. As long as what's in the package metadata is a subset of what you send in the header, you'll see the package. [13:58] sergiusens, there's an example in https://wiki.ubuntu.com/AppStore/Interfaces/ClickPackageIndex#Frameworks [14:03] fgimenez: meeting [14:04] elopio, omw [14:07] JamesTait: shouldn't this http://paste.ubuntu.com/11730647/ only return docker? [14:09] sergiusens, it should return packages where framework is one of: ["docker"] ["docker", "ubuntu-core-15.04-dev1"] ["ubuntu-core-15.04-dev1"] [14:13] JamesTait: ah, so that was an || in my mind :-P [14:27] sergiusens, adding a 'img' flag would help? something like 'go run _integration-tests/main.go -img /path/to/img' [14:29] fgimenez: well, I need to test the u-d-f output too [14:29] fgimenez: stdout should warn about --device [14:33] rsalveti: we need an ubuntu-snappy release and to rebuild u-d-f after that to allow --install docker :-/ [14:34] sergiusens: sure [14:35] sergiusens: anything waiting to be merged in ubuntu-snappy or can we just release current trunk? [14:36] @activereviews [14:36] sergiusens: No such command! [14:36] @activereview [14:36] sergiusens: No such command! [14:36] @help [14:36] "list" To see the available commands ; "help cmd" for specific command help [14:36] @list [14:36] The available commands are: ['bug', 'critical', 'help', 'last', 'list', 'more', 'ping', 'reviewlist', 'seen'] [14:36] @reviewlist [14:36] https://code.launchpad.net/~mvo/snappy/snappy-gettext/+merge/262202 | No reviews (less than a day old) [14:36] https://code.launchpad.net/~mvo/snappy/snappy-console/+merge/262061 | Approve: 1 (less than a day old) [14:36] https://code.launchpad.net/~fgimenez/snappy/go-functional-tests/+merge/261748 | No reviews (5 days old) [14:36] https://code.launchpad.net/~mvo/snappy/snappy-verify/+merge/261718 | No reviews (5 days old) [14:36] https://code.launchpad.net/~mvo/snappy/snappy-improve-developer-mode-detection/+merge/261646 | No reviews (6 days old) [14:37] @tellsergiuenstostopplaying [14:37] ogra_: No such command! [14:37] ogra_: I was tied to lp speak [14:37] heh [14:37] rsalveti: no, nothing in queue [14:39] let me release that then [14:46] sergiusens, sorry to disappear; other people needing attention, school run, I need a clone. ☺ [14:47] sergiusens, so a query string like ?q=framework:ubuntu-sdk-15.04-dev1,framework:docker would give you an || [14:56] fgimenez: maybe update _integration-tests/README with the go run comment [14:56] sergiusens: https://launchpad.net/ubuntu/+source/ubuntu-snappy/1.2-0ubuntu1 [14:57] rsalveti: thanks [15:01] Also, sergiusens, by default query-string search terms are analysed and do prefix matching, so ?q=framework:docker will also match packages with framework: ["docker-1.3"]. Wrapping the term in quotes should prevent that and make it a literal phrase search, but that doesn't seem to be the case any more. I'll need to dig a bit to work out why. [15:01] JamesTait: it does literals for package names at least as nessita showed me [15:02] JamesTait: so I want headers and not query strings for this and that last comment settles it [15:03] sergiusens, yes you do [15:04] nessita: err, what [15:04] * JamesTait grabs a drink [15:05] nessita: in any case I know what I mean :-) [15:05] sergiusens, I found issues on the iso build which explain the hang, fixing them in livecd-rootfs now [15:05] seb128: oh nice, any reason why the image is so big? [15:06] sergiusens, define "so big"? [15:06] it's 2.5G [15:06] compressed ?!? [15:07] ogra_, well, the ubuntu desktop iso is likge 1G [15:08] ogra_: 2.5 uncompressed [15:08] and the snappy image is x2 because of the a-b partitions [15:08] so seems about right to me? [15:08] ah, uncompressed [15:09] that sounds rather sane [15:09] sergiusens, sure thx! elopio suggested changing the way we build images depending on the type of test being executed [15:09] jdstrand: tyhicks thanks for help y'day, i made good progress....last thing i'm struggling with is a script that is part of the snap that launches the server [15:09] i'm getting ubuntu-core-launcher:/apps/mir/snap1/bin/server.real pidof: Operation not permitted [15:09] sergiusens, elopio we still need to define how to group tests tough [15:09] but i've already add /bin/pidof to may aa file, which took care of the denial [15:10] any ideas? [15:11] http://bazaar.launchpad.net/~kgunn72/mir/snappy-packaging-with-secprofile/view/head:/server [15:11] that's the script ^ [15:11] is it b/c it's outside of the mir binary itself ? [15:12] kgunn: add u or U (I forget which one is the more correct one) [15:13] kgunn: /bin/pidof Umrix, [15:13] sergiusens: thanks! i'll try that [15:13] sergiusens: i had added everything but that i think :) [15:14] kgunn: I have that a plenty; uU is for run unconfined [15:14] one cleans up the env while the other doesn't [15:17] Ux scrubs the environment [15:17] tyhicks: what means "scrubs" [15:17] * kgunn grunts like caveman [15:18] kgunn: it attempts to remove any risky environment variables [15:18] kgunn, audio or it didnt happen ! [15:19] :) [15:19] :) [15:19] I don't like the idea of snaps being able to do unconfined transitions while calling out to external binaries [15:20] I'm not sure if that's what jdstrand has been recommending for situations like this or not [15:21] using Ux will get you unblocked for now but we may end up wanting to do something more secure [15:21] tyhicks: no problem....i'm game to be a guinea pig [15:21] i am getting a little heat to get mir in the store tho :) [15:21] mterry: my work on the debian stuff https://github.com/mvo5/gettext/tree/debian [15:22] tyhicks: do you consider using u on pidof ok for store use ? [15:22] Hey folks, can someone explain the strategy how snappy can use the full disk / resizing automatically on first boot or something? [15:22] kgunn: well, jdstrand is the one that has been defining those boundaries so I'll defer to him [15:23] mvo_, I don't have a branch for mine -- sorry we collided, I tried to ping you but I think I got a stale copy of your irc client (mvo__ with two underscores if I recall) [15:23] mvo_, we used the same source and binary names, so at least we won't get two copies [15:24] mvo_, I fixed the test issue though by copying example files into the obj-* dir [15:25] mvo_: yep, i'm confused :) can you elaborate on what you want for LP: #1466124 ? [15:25] Launchpad bug 1466124 in system-image (Ubuntu) "Please provide a way to get the progress in used from a plugin" [Undecided,New] https://launchpad.net/bugs/1466124 [15:25] mterry: oh, nice! I thanks for fixing the tests. lets just merge it together [15:25] barry: meh, I figured I did not express myself very well [15:25] mvo_, just add [15:25] override_dh_auto_test: [15:25] # copy data files in [15:25] cp -r examples/*/ obj-*/src/github.com/gosexy/gettext/examples/ [15:25] dh_auto_test [15:25] mvo_, to yours [15:25] mvo_: s'okay :) [15:25] barry: once sec, I'm in a meeting right now, won't help with being consistent [15:25] eh or understandable or anything really [15:26] mterry: \o/ [15:26] mvo_, and then maybe switch your deletion of those files to dh_auto_install rather than dh_auto_build [15:27] sergiusens: your 'U's and 'u's are not necessarily recommended [15:27] kgunn: can you paste the full denial? [15:28] kgunn: can you use 'rmix' (ie, don't use 'U') and try again, showing me the denial? [15:28] kgunn: is this something that can run in a kvm ubuntu-core image? [15:30] note that pidoff isn't as innocent as it may seem, from the pidof(8) man page: [15:30] "pidof is actually the same program as killall5" [15:30] yes [15:30] so I'd prefer ix so we can then use signal mediation [15:31] agreed [15:31] tedg: hi, so what exactly is it you need? the uri of the source package? [15:31] * jdstrand notes this is a framework, but frameworks are privileged and we should stop using demo policy and do real policy for these things [15:32] mvo_, This is what I did. Get from binary packages to the list of dev packages associated with them: http://paste.ubuntu.com/11731018/ [15:32] mvo_, The list is hardcoded in that example. [15:32] tedg: thanks, let me have a look [15:37] tedg: http://paste.ubuntu.com/11731045/ is slightly shorter but yeah, srcpkg stuff is not the strength of python-apt [15:37] tedg: this examle needs updating in the api docs :/ it does not reflect the latest features of python-apt. thanks for finding that [15:39] mvo_, The other example that is really bad is this one, as it is camel case and the functions aren't: https://apt.alioth.debian.org/python-apt-doc/library/apt_pkg.html#apt_pkg.PackageRecords.lookup [15:39] mvo_, That's where I came up with the stuff that you deleted :-) [15:41] mterry: hm, if your package has this already fixed, we could simply use your version? [15:42] tedg: uh, indeed, that needs fixing [15:42] barry: ok, so … let me try again. we "hook" into the upgrade (the apply hook) with our custom upgrader. when that is run we currently check the options if the user requested to use machine-readable output [15:43] mvo_, other differences I see: I add misc:Built-Using, and I do actually ship the examples folder (since go seems to like to ship the _test.go files, and they are needed to make it work [15:43] barry: our branch set this in the global config object [15:43] mvo_, but I'm not convinced the examples folder should be shipped, so yours might be doing the right thing there [15:43] mterry: right, lets keep yours if its actually more complete, or is there a downside? [15:43] mvo_, just the examples folder [15:44] mvo_, and no source tree [15:44] mvo_, in vcs that is [15:44] barry: but it seems like with the current 3.0 I can not get the information if the user has requested a machine-readable output [15:44] jdstrand: hey, sorry, was on a HO... [15:44] yeah so i ran last night with [15:44] mvo_, how would we stop one of our uploads? [15:44] /bin/pidof mrix, [15:44] I'm not used to cancelling an upload [15:45] and the error was ubuntu-core-launcher:/apps/mir/snap1/bin/server.real pidof: Operation not permitted [15:45] mterry: ok, no vcs is ok, mine is just a single commit so far. and if yours does not need further changes I'm in favour of keep it :) [15:45] mvo_, alright [15:45] kgunn: what is the apparmor denial? grep DEN /var/log/syslog [15:45] mvo_: oh, do you just mean that the hook doesn't know whether --progress=json was provided on the cli? [15:45] jdstrand: that's just it....there's not one [15:45] mterry: we can just reject my upload, I can ask the archive admins to do that [15:45] barry: yeah, exactly that [15:46] barry: so it might be as simple as setting it as a transient config in the global config object [15:46] barry: thats what I did in the fork we are currently using [15:46] mvo_: ah, okay, that should be easy to expose in the global config [15:46] yeah [15:46] kgunn: is there a seccomp denial? [15:46] jdstrand: nope [15:47] mvo_: thanks, i'll update the bug description. is it worth holding up 3.0.1 into wily to actually do a quick 3.0.2, or can i target that at 3.1? [15:47] kgunn: did you disable kernel rate limiting? [15:47] jdstrand: nope, but i can try that now [15:47] barry: its not blocking us, I just always spit out json to stdout now [15:48] mvo_: cool, thanks [15:48] barry: which is bad but its only snappy thats driving it right now, so its not too terrible [15:48] mvo_: ack [15:48] thanks! and sorry that it was so confused [15:49] no worries! [15:50] mvo_: thanks. description updated. please sanity check [15:51] barry: yes, thats it [15:53] mvo_: \o/ [15:59] jdstrand: so diabled kernel rate limiting (with sudo sysctl -w kernel.printk_ratelimit=0) [16:00] run via systemctl start mir-blah.service [16:00] the syslog just shows [16:01] "systemd started system compositor" [16:01] no seccomp or aa denial error [16:01] but...system compositor doesn't appear in process list [16:01] hack it to spit out more info ? [16:02] (the systemd unit i mean) [16:03] kgunn: does the service try to drop privs and then regain them? [16:06] jdstrand: i don't think so, the system compositors run as root.... [16:07] mterry: ^ any privlegde changes ? [16:07] kgunn: try using @unrestricted as the seccomp policy [16:07] kgunn, I don't think so [16:07] kgunn, is it waiting for agetty? [16:08] kgunn, or did the server shell script bail for some reason? [16:09] mterry: sorry to bother you this is all about getting sec policy for mir correct (as fmwk) to get in store.... [16:09] kgunn, right (pidof stuff still?) [16:09] worked through all the aa & seccomp errors, now stuck on pidof in the launching script [16:10] brb [16:22] ogra_: do you want to top-approve? https://code.launchpad.net/~zyga/snappy-hub/fix-1464275/+merge/261833 [16:24] jdstrand: ok, so running with bin/pidof & bin/sleep :q [16:24] elopio, done [16:24] oops ignore the :q [16:25] pidof and sleep with mrux, and syslog shows both "Opertation not permitted" just like before [16:26] note, that was without the kernel rate liimiting denial disabled [17:08] ogra_: sergiusens_: mvo_: one interesting problem: http://cdimage.ubuntu.com/ubuntu-core/vivid/daily-preinstalled/pending/ [17:09] the azure device tarball is not getting updated from the build, since 9-jun [17:09] but from the build log, it seems it was created [17:09] how to find out what is going on at the cdimage side of things? [17:11] there should be cdimage logs [17:11] http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu-core/wily/ [17:12] hmm, no mention of azure in there at all [17:12] could it be that they need to have manual intervention ? [17:12] maybe we're not building it [17:12] # now build the azure device tarball by adding walinuxagent [17:12] if [ -e binary/boot/filesystem.dir/var/lib/dpkg/info/walinuxagent.list ]; [17:12] the check [17:13] this is for vivid [17:13] yeah, we're not building it [17:14] right [17:14] # now build the azure device tarball by adding walinuxagent [17:14] if [ -e binary/boot/filesystem.dir/var/lib/dpkg/info/walinuxagent.list ]; [17:14] argh [17:14] if it exists cdimage downloads it as seen in http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu-core/wily/daily-preinstalled-20150609.log [17:14] daily-preinstalled-20150609.log [17:14] the last one that had it [17:14] yeah [17:14] daily-preinstalled-20150612.log already failed to generate it [17:15] * rsalveti looks for walinuxagent related changes [17:15] hmm http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu-core/wily/daily-preinstalled-20150610.log ... that downloaded it as well [17:15] ogra_: right, that's wily [17:15] for vivid we don't have 0610 [17:15] oh [17:15] http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu-core/vivid/?C=M;O=D [17:16] right [17:16] i missed the "vivid" in the url above [17:16] https://launchpadlibrarian.net/209278613/buildlog_ubuntu_vivid_amd64_ubuntu-core-system-image_BUILDING.txt.gz [17:16] but it seems it was actually created ^ [17:17] + tar -c -z -f /build/device.tar.gz system assets hardware.yaml [17:17] this from https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/vivid/ubuntu-core-system-image/ [17:17] checking the latest amd64 image [17:19] kgunn: can you try with seccomp policy as @unrestricted? [17:19] kgunn: (sorry, been in a meeting) [17:19] * jdstrand is still in the meeting [17:21] rsalveti: ogra_ we are probably not building it for wily and porbably don't want it either (to be replaced with gadget snaps) [17:22] sergiusens_: right, wily is fine to not build it [17:22] my concerned is that we're not building it for vivid [17:22] *concern [17:22] launchpad says we're building it, but the cdimage log is not showing that it copied it over [17:22] rsalveti: but, but, but we released a few weeks ago [17:22] sergiusens_: yeah, failed the next day [17:22] after the release [17:23] since the release day, we didn't get any other update [17:23] and nothing really changed on our side [17:24] rsalveti: system image's index says something different http://system-image.ubuntu.com/ubuntu-core/15.04/edge/azure_amd64/index.json [17:24] last entry, version_detail [17:24] sergiusens_: right, that is fine, since cdimage is publishing the tarball, but the older one [17:24] ah, the device tarball is stuck [17:24] that's the usual issue with the cdimage side [17:24] if something fails, it will just copy the older ones [17:24] http://cdimage.ubuntu.com/ubuntu-core/vivid/daily-preinstalled/pending/ [17:25] sergiusens_: check the date ^ [17:25] rsalveti: let's blame ogra! [17:25] rsalveti: yeah, that's why I said device tarball is stuck :-P [17:25] I'll brb [17:25] yeah, just blame me [17:28] ogra_: where can I find the previous live-build logs? [17:29] since we moved that to launchpad [17:33] slangasek: maybe you can help with this ^? [17:34] basically I'm trying to figure out why device-azure.tar.gz wasn't published as part of cdimage for the last few images [17:34] rsalveti: https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/wily/ubuntu-core for instance? [17:34] slangasek: I'm looking at https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/vivid/ubuntu-core-system-image/ [17:34] but I can only see a few [17:34] and not easily go back in time [17:35] ah [17:35] rsalveti: if you know which build you're looking for (from cdimage's POV), you can find the url to the exact build log in the log on nusakan [17:36] would be 20150609 [17:36] slangasek: do you know where to look? [17:37] rsalveti: /srv/cdimage.ubuntu.com/log/ubuntu-core/wily/daily-preinstalled-20150609.log [17:37] ubuntu-core-system-image-amd64 on Launchpad starting at 2015-06-09 04:56:02 [17:37] ubuntu-core-system-image-amd64: https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/wily/ubuntu-core-system-image/+build/29185 [17:38] slangasek: awesome, thanks! [17:39] yeah, build seems fine [17:39] * rsalveti looks for the cdimage cdo [17:39] code [17:41] slangasek, any idea why we stopped mirroring them to http://people.canonical.com/~ubuntu-archive/livefs-build-logs/ ? [17:41] it is quite handy to have all of them in one place [17:41] no idea, no [17:42] ogra_: slangasek: so there is no 'azure' at all in the cdimage code [17:42] and the only update, that happened at last 11, was to add the personal [17:43] and that probably caused a sync [17:43] who added the azure logic in there? [17:43] well, i had the impression the cloud stuff was some manual process [17:43] hum, so I got an image to boot in kvm [17:43] seb128, yay [17:43] lightdm fails to start though [17:43] oooh :( [17:43] ogra_: the manual part is after it gets in system-image [17:43] rsalveti, ah, k [17:43] "Error using VT_ACTIVATE 7 on /dev/console: Inappropriate ioctl for device" [17:43] does that ring a bell to anyone? [17:44] rsalveti: I don't think there were any changes required to the cdimage code for the azure device tarball. I thought the changes were only in livecd-rootfs [17:44] I can starts gallery-app with xinit though :p [17:44] so xorg is working [17:44] seb128, kgunn had fun with agetty and mir wrangling around a tty all day i think [17:44] slangasek: was thinking about cdimage because we need to copy the tarball over [17:44] after it gets published [17:44] if config.project in ("ubuntu-core", "ubuntu-desktop-next"): [17:44] device = "%s.device.tar.gz" % live_prefix [17:44] rsalveti: right; so I never made any changes to the cdimage code to accomodate this, and if I had it would have been in the bzr branch [17:44] if os.path.exists(device): [17:44] shutil.copy2( [17:44] device, "%s.device.tar.gz" % output_prefix) [17:45] this is the piece that copy it over [17:45] live_prefix = os.path.join(live_dir, arch) [17:45] rootfs = "%s.rootfs.tar.gz" % live_prefix [17:46] so unless I'm not reading it right, there is indeed nothing copying it around [17:46] well, that code is pretty generic [17:47] do we perhaps have an architecture called "amd64.azure" ? [17:48] (would be odd to have a dot in there, but who knows) [17:48] right [18:01] ogra_: slangasek: I fail to see how this ever worked =\ [18:01] rsalveti: indeed, I don't know either. I think mvo_ may have been the one doing the work on the azure device tarball at the time, maybe he remembers something? [18:01] https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/vivid/ubuntu-core-system-image/+build/29944 [18:01] the tarball is there [18:02] mvo_: if still around ^? [18:09] at least the problem is consistent with both vivid and wily, and also started at last 11 [18:12] rsalveti, asac, some clarification about the support status in the RPi thread "Snappy RPi2 stable image #3 now available" would be appreciated ... [18:12] (seems he is rather grumpy about our communication and marketing) [18:13] rsalveti: azure is built from livecd-rootfs [18:14] sergiusens_: right, and it's there [18:14] just not imported/copied in cdimage [18:14] ogra_: sure [18:14] sergiusens_, and how is it getting onto cdimage.u.c ? [18:14] ogra_: that's all black magic and I only wish someone explained it so we could simplify it ;-) [18:14] for some reason I only had your reply and not the original email, which is now there (and was unread) [18:14] * sergiusens_ winks [18:15] my gmail is kind of going crazy lately [18:15] ogra_: http://bazaar.launchpad.net/~ubuntu-core-dev/livecd-rootfs/trunk/view/head:/live-build/ubuntu-core/hooks/500-move-kernel-to-device-tar.binary [18:15] sergiusens_, ha, wishful thinking [18:15] rsalveti: I started using mutt again [18:15] sergiusens_: https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/vivid/ubuntu-core-system-image/+build/29944 [18:15] right, see the azure is there [18:15] sergiusens_: mutt + google imap? [18:15] I only go to gmail to search [18:15] rsalveti: yup, offlineimap [18:15] right, might be better indeed [18:15] evolution FTW :) [18:16] has still the fastest search tools [18:16] rsalveti: searching is good in gmail; organizing and cleaning up the queue and not missing content is better in a proper MUA [18:16] ogra_: makes you need a revolution in RAM though [18:16] my XPS copes fine with its 8G [18:17] and i dont have any device in use with less anymore [18:17] (for desktop that is) [18:17] rsalveti: so the build is there; what is missing? [18:17] sergiusens_, the code in cdimage that copies it [18:17] we dont know how it got from here to there [18:17] (the logs show it gets copied) [18:17] http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu-core/vivid/daily-preinstalled-20150609.log [18:18] 2015-06-09 14:18:50 URL:https://launchpadlibrarian.net/208661665/livecd.ubuntu-core.azure.device.tar.gz [142562062/142562062] -> "/srv/cdimage.ubuntu.com/scratch/ubuntu-core/vivid/daily-preinstalled/live/amd64.azure.device.tar.gz" [1] [18:18] yeah, was looking at the code, but can't see to find how it was copying ti before [18:18] sergiusens_: check lp:ubuntu-cdimage [18:18] grep for "Downloading live filesystem images" [18:18] that is where the magic happens [18:19] ogra@styx:~/Devel/branches/cdimage$ grep -r azure * [18:19] ogra@styx:~/Devel/branches/cdimage$ [18:19] thats the weird part [18:19] that copy the files from launchpad into /srv/cdimage.ubuntu.com/scratch/ubuntu-core/vivid/daily-preinstalled/live [18:19] which is the missing link, because it's not copying over the azure tarball [18:20] well, i dont get how it would even know about azure at all [18:20] rsalveti: let me read scrollback and look at the code. so the problem started on the 11th this month? [18:20] given it is neither mentioned in the code nor in the config [18:20] mvo_: yes [18:20] (or in any bzr changelog) [18:20] how confusing, my first guess is a livecd-rootfs upload [18:20] mvo_: it's not livecd-rootfs, since it's there: https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/vivid/ubuntu-core-system-image/+build/29944 [18:20] I'm sure the ubuntu-personal guys broke it [18:20] mvo_, not really, livecd-rootfs does what it should [18:20] the device tarball was built [18:20] meh [18:21] cdimage doesnt import it [18:21] there goes my theory and the blame I wanted to assign :/ [18:21] i got blamed already, find someone else :P [18:21] * ogra_ only takes the blame once a day [18:21] mvo_: so it might be connected with personal [18:21] mvo_: there was a personal change in cdimage at 11 [18:22] which would probably cause a sync in the bzr repo [18:22] so if the azure change was done outside trunk, then it would be lost [18:22] uh [18:22] but who would do that [18:22] https://wiki.ubuntu.com/ReleaseTeam/CDImageSetup ... [18:22] that's just a theory [18:23] (in case people dont use that way ... ) [18:23] unless I'm missing some magic in lp:ubuntu-cdimage [18:24] mvo_, nice blame try! [18:24] * mvo_ hugs seb128 [18:24] * seb128 hugs mvo_ back ;-) [18:24] and so subtle :) [18:27] yeah, the branch was pushed at 11 by Laney [18:27] reflecting the personal changes, but that has nothing to do with azure [18:27] rsalveti: we wouldn't normally do a 'bzr pull --overwrite'; if someone did that, that's quite bad [18:27] right, makes your theory more plausible [18:27] the usual protocol is 'bzr pull'; notice conflicts; yell at cowboys [18:28] right [18:29] and the branches are bound you wouldnt ever be able to commit [18:29] OH ! [18:29] but perhaps the config is in debian-cd, not cdimage [18:29] * ogra_ goes checking [18:30] hmm, no [18:30] ogra@styx:~/Devel/branches/debian-cd$ grep -r azure * [18:30] ogra@styx:~/Devel/branches/debian-cd$ [18:35] another funny thing I noticed because of this [18:35] the kernel config seems to be available via the rootfs [18:35] after creating the azure image I had vmlinuz-3.19.0-18-generic and config-3.19.0-21-generic [18:35] at /boot [18:36] yeah [18:36] rsalveti: yeah, we keep it in /boot because our grub needs it this way, we want to consolidate this [18:36] well, we dont have /proc/config.gz enabled [18:36] they are on the arm images too [18:36] needs cleanup ... [18:36] mvo_: right, but why is this part of the rootfs? [18:36] yeah, will open a bug in a few [18:37] rsalveti: oh, right, we don't want it there [18:43] mvo_: ogra_ rsalveti so is this what is needed? http://paste.ubuntu.com/11731958/ [18:43] I just blindly added that [18:43] sergiusens_: could well be, I'm trying to understand why it worked before :/ [18:43] not sure you want azure hardcoded there [18:43] sergiusens_: something similar to that is what I had in mind, but the question is indeed what mvo_ just said [18:44] for now I just manually copied over the new tarball and triggered a new image [18:44] see if it will show up at system-image, so I can unblock people (that were waiting for an updated initrd) [18:50] rsalveti, sergiusens_: it seems like your change is exactly what was there and is no longer, the logs even have "Publishing amd64 azure device tarball " (I guess you know that already). really strange [18:51] yeah [18:51] too bad this is not git [18:51] git reflog would explain it all [18:52] rsalveti, even if someone forcefully overwrote it ? [18:52] ogra_: yup [18:52] (including history) [18:52] interesting [18:52] rsalveti: mvo_ a bzr push --overwrite to lp or a bzr pull --overwrite on the client explains most of this to me [18:52] it's my solution when I do git reset --hard HASH [18:52] * ogra_ guesses some day he wont get around looking at git :P [18:52] ogra_: as soon as we move to it ;-) [18:52] and then omg omg omg I forgot to save a patch [18:52] (i must admit i'm surprised that i still do :) ) [18:52] git reflog gives me the complete history [18:53] sergiusens_: yeah [18:53] sergiusens_: yeah, I think thats it [18:53] and someone *cough* *cough* cowboyed it in [18:53] before without a proper branch [18:53] * mvo_ hides in shame [18:54] but to be fair, that azure enablement was really on a super tight deadline [18:54] mvo_, https://wiki.ubuntu.com/ReleaseTeam/CDImageSetup ... for next time :) [18:55] though still, while you cowboyed it in, someone overwrote it ignoring the error [18:55] bzr definitely complained [18:55] sergiusens_: this may need a additional download_live_items() in lib/cdimage/livefs.py [18:56] ogra_: indeed [18:56] * ogra_ blames Laney in absence :) [18:56] finally we can pass on the blame :) [18:58] yeah, the push --overwrite was even worse [19:01] mvo_: like http://paste.ubuntu.com/11732046/ ? [19:01] I don't really follow this code base :-P [19:02] i still think it should somehow be injected into source_prefix instead of hardcoding it [19:02] sergiusens_: yeah, I think something like this [19:03] ogra_: I would leave that to you :-P [19:03] well I pushed here: lp:~sergiusens/ubuntu-cdimage/azure [19:03] * rsalveti hands the cowboy hat to mvo_ [19:03] sergiusens_: \o/ [19:03] in case you want to merge [19:03] * sergiusens_ wants the cowboy hat [19:04] sergiusens_, noh, go ahead if we need it now ... i'm being "studio_'ed" and annoyed enough to not want to write on code ... [19:04] will use it Clint Eastwood style [19:05] (i can take annother look tomorrow to see if i find another way) [19:05] ogra_: heh, so I don't mind this azure device as it will go away as soon as we solve the update-grub issue [19:05] sergiusens_: heh, so you cowboy^Wmerge it for testing? [19:05] sergiusens_, ah, well, then leave it ... [19:05] mvo_: do I have permissions? I forget :-P [19:05] mvo_, sergiusens_, make sure to follow the wiki doc though (./run-tests etc) [19:06] sergiusens_: I think so, but I know that I have [19:06] sergiusens_, you do [19:06] you got them together with rsalveti back then [19:07] ogra_: heh, I was playing dumb :-P [19:07] yeah, cant cheat me :P [19:07] ogra_: the tests fail here even without my changes! [19:07] sergiusens_: lol [19:07] oh man [19:07] Laney, !!! [19:08] his change does not look like it would break tests [19:08] but who knows [19:08] * mvo_ does not [19:08] mvo_: coincidentally I have [19:08] mvo_: http://paste.ubuntu.com/11732086/ [19:08] but there are 4 failures regardless [19:09] kgunn_: did it work with @unrestricted? [19:09] sergiusens_: http://paste.ubuntu.com/11732091/ <- this is what I had to do back in the day to add support for device.tar.gz [19:10] sergiusens_: oh, I mean the changes that Laney did do not look like they would break stuff, but again, who knows [19:13] sergiusens_: it seems like item in live_item_path() might also want .azure in there :/ [19:16] mvo_: let me check; but I do have test parity now [19:16] jdstrand: i definitely get some app armor denials..but the way the script is written is kinda spins outta control...so i'm gonna fix that real quick and run again [19:19] mvo_: I have a good feeling about this one http://paste.ubuntu.com/11732146/ thanks for your base branch for device [19:20] sergiusens_: yay, lets give it a try! [19:20] mvo_: ok, let me do an actual checkout [19:23] sergiusens_: mvo_: when can we have a coven about changes to snaps for 16.04? [19:23] tomorrow? [19:23] yep [19:23] 16.04 ? thats so far out ! [19:24] ogra_: far out, man! [19:24] sergiusens_: mvo_: +1 [19:24] ogra_: I don't need to pull -d debian-cd nor production, right? [19:24] sergiusens_, if you didnt make any changes to it, no [19:24] only cdimage should be enough [19:24] ogra_: just in case; gets out of the server asap [19:25] * mvo_ is very curious [19:26] ogra_: mvo_ that's all I did http://paste.ubuntu.com/11732185/ [19:26] is there an argument for snappy ubuntu + consule + ... [19:26] vs coreos? [19:26] mvo_: about changes to snaps? [19:26] so I guess we can trigger a build now [19:26] consul, not consule [19:26] fikse: what's consul? [19:26] sergiusens_, perfect [19:26] consult maybe? [19:27] a tool for service discovery https://consul.io/ [19:27] Chipaca: about changes to snaps? [19:27] mvo_: * mvo_ is very curious [19:27] sergiusens_: now its time to run a build, no? [19:28] mvo_: yup [19:28] Chipaca: mostly about if sergiusens_ saved the day, I think he did! [19:28] is that nusakan? [19:28] Chipaca: but the snap stuff is interessting as well [19:28] sergiusens_: yes [19:28] * sergiusens_ logs back in [19:28] * sergiusens_ gets goosebumps [19:28] * ogra_ grins [19:28] mvo_: last time I was in there we were fixing that crazy system image server indexing problem :-P [19:31] fikse: depending on what you do, it might be perfect, or it might be too early for snappy :) [19:31] ogra_: mvo_ which one should we try wily or vivid? [19:31] * sergiusens_ goes for wily [19:32] yeah [19:32] +1 [19:33] Chipaca: i'm afraid it might be too early [19:34] sergiusens_: can you get your server running the snapcraft tests? [19:34] too early it is then [19:34] elopio: if it's in the .tarmac.sh, yes [19:34] geez, we have tests already ? [19:34] snapcraft doesnt even exist ! [19:34] you guys are so ahead [19:35] * ogra_ makes a note to check the test code to know what he will develop in a few months [19:35] tdd baby! [19:35] sergiusens_: yes, there's a .tarmac.sh. [19:35] elopio: just add it there [19:36] ogra_: I don't see the build here https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/wily/ubuntu-core-system-image [19:36] whats the line you ran ? [19:36] ogra_: SUBPROJECT=system-image EXTRA_PPAS=snappy-dev/image for-project ubuntu-core cron.daily-preinstalled [19:36] hmm, looks fine [19:37] ogra_: generally do crontab -l |grep something [19:37] oh [19:37] --live [19:37] fikse: give it a try if you want; there's a couple of non-framework services (e.g. the xkcd-webserver) in lp:~snappy-dev/snappy-hub/snappy-examples [19:37] ogra_: I thought you told me way back not to use that :-P [19:37] you should always use whats in the crontab [19:37] ogra_: ok; but I'll search for that conversation :-P [19:38] ubuntu [19:38] oops [19:38] ww [19:38] now change your password [19:38] kgunn_: we now know your password! [19:38] quick ! [19:38] (make it ubuntu1 ... nobody will guess that) [19:38] ogra_: mvo_ rsalveti ok, building here now https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/wily/ubuntu-core-system-image [19:38] ubuntu123 [19:38] yay [19:38] ubuntu123! [19:38] or maybe password :-) [19:39] nah, i use that already ... he cant take that [19:39] sergiusens_: ooooohhhh [19:39] ogra_: where is our powerpc build? :-) [19:39] sergiusens_, you just changed the cdimage code ... why didnt you add it ! [19:45] will snappy personal eventually merge back to the current branch of Ubuntu Desktop when it's deemed ready? [19:45] merge back ? [19:46] so for example when 16.10 comes out for example, and everyone decides snappy and unity8 is ready, everyone running Ubuntu desktop will upgrade to snappy and unity8 in 16.10 [19:46] you mean if there will be a deb based desktop install with unity8 ? [19:47] * ogra_ guesses thats a question for #ubuntu-desktop actually [19:47] nah it would be snappy [19:47] i suspect though that we wont auto-migrate users from deb based systems to snappy [19:47] so basically would the deb based version be dropped [19:47] ah okay [19:47] so if people want to stick with deb they will stay deb for good if they wish [19:47] no, there are too many people using the deb archive ... (flavours and such) ... [19:48] well for now at least [19:48] for a snappy install you will most likely do an install from scratch [19:48] cool thanks that answers my question :) [19:48] ogra_: I wasn't thinking straight, we need powerpc urgently for the device I don't have! [19:48] :-D [19:48] and also snappy is built from debs from the archive [19:49] sergiusens_, well, its a one line change in the config :) [19:49] (modulo image build failures etc indeed) [19:58] * rsalveti reads backlog (back from a meeting) [19:58] what's the process for adding/removing/modifying users? [20:00] DarwinF, there is no proper process yet ... /var/lib/extrausers/ has the user data though [20:01] rsalveti: btw https://launchpad.net/ubuntu/+source/goget-ubuntu-touch/0.24-0ubuntu2 [20:01] sergiusens_, your images are done btw [20:02] well, the livefs build [20:02] ogra_: thanks [20:02] sergiusens_: nice, so you fixed cdimage? [20:03] fixededed [20:03] rsalveti: I'm not sure yet :-P [20:03] rsalveti, but he forgot to add powerpc while doing that [20:03] lol, alright [20:03] that's super important [20:04] * rsalveti hands a few beers to sergiusens_ [20:04] hmm [20:04] rsalveti: http://cdimage.ubuntu.com/ubuntu-core/daily-preinstalled/pending still shows June 11 for azure [20:04] daily [20:04] http://cdimage.ubuntu.com/ubuntu-core/daily-preinstalled/20150617.3/ [20:04] not sure if I need to wait for all builds to finish here [20:05] ogra_: wily-preinstalled-core-amd64.azure.device.tar.gz 11-Jun-2015 05:13 140M [20:05] no, thats up to date i think [20:05] 11 and not 17 [20:05] yes [20:05] before it was 09 [20:05] wily was 11 [20:05] 09 was vivid [20:05] oh [20:05] and for vivid I manually replaced earlier today http://cdimage.ubuntu.com/ubuntu-core/vivid/daily-preinstalled/pending/ === j12t_ is now known as j12t [20:05] as a workaround [20:06] but, it seems the importer still didn't see the new image [20:06] ogra_: where is that "Publishing " message supposed to be logged? [20:06] which is super annoying [20:06] http://cdimage.ubuntu.com/ubuntu-core/vivid/daily-preinstalled/20150617.2/ [20:06] rsalveti: I suspect that, but I don't know :-) [20:07] jdstrand: ok...sorry, wanted to make sure and double check everything...so yeah, with @unrestricted i see aa denial for /sbin/killall5 [20:07] man, so many paths [20:07] which makes sense based on what tyler said earlier [20:07] e.g. pidof is actually part of killall5 [20:07] yeah, for wily it's still not there http://cdimage.ubuntu.com/ubuntu-core/daily-preinstalled/pending/ [20:07] http://paste.ubuntu.com/11732394/ [20:08] sergiusens_, /srv/cdimage.ubuntu.com/log/ubuntu-core/wily/daily-preinstalled-20150617.3.log on nusakan [20:08] http://people.canonical.com/~ubuntu-archive/cd-build-logs/ubuntu-core/wily/?C=M;O=D [20:08] yeah, more up-to-date log [20:08] no trace of azure there :( [20:08] no azure [20:09] kgunn_: ok, so use /sbin/killall5 ixr, [20:09] ogra_: how many hours is the importer taking nowadays? [20:09] the "publishing" actually comes from debian-cd i think [20:09] ack, was doing as you typed [20:09] rsalveti, not long for core [20:10] seems there is one importer running atm [20:10] jdstrand: so i should revert the /bin/pidof and bin/sleeps to ix as well ? (as i have them ux atm) [20:10] ogra_: I added Publishing logs in ubuntu-cdimage [20:10] yeah, and the word doesnt appear in a grep in debian-cd [20:10] only in cdimage [20:14] ogra_: something tells me this doesn't seem to be working if os.path.exists("%s.azure.device.tar.gz" % source_prefix) [20:15] yeah, i guess source_prefix is somehow a bit different [20:16] ogra_: different? livecd.ubuntu-core.azure.device.tar.gz vs livecd.ubuntu-core.device.tar.gz [20:17] i think it is more than just livecd.ubuntu-core ... moght be a full path [20:17] *might [20:18] kgunn_: yes please. we want to get rid of any ux's [20:18] kgunn_: is this something I could run in a vm? [20:18] ogra_: I grabbed those names from https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/wily/ubuntu-core-system-image/+build/29997 [20:19] well, if you look at http://paste.ubuntu.com/11732394/ ... source_prefix might have a stamp and all [20:19] jdstrand: yeah it is [20:19] https://launchpadlibrarian.net/209363495/livecd.ubuntu-core.rootfs.tar.gz ... [20:21] ogra_: oh, I found an issue in my diff [20:21] kgunn_: if you give me instructions, then I can maybe expedite this [20:21] jdstrand: sure....let me clean up and push and i'll share [20:22] that said, I do have something I'm working on atm and won't be able to get to it for a little while [20:22] but certainly full-force in the morning if not later today [20:22] jdstrand: i'm making some headway now...but i'll share later [20:23] kgunn_: ok, however you want to do it [20:23] i think the tricky bit was not realizing that @unrestricted would lead to more aa denials to add [20:24] ogra_: http://paste.ubuntu.com/11732451/ [20:24] sergiusens_, uuuh [20:24] yeah :) [20:24] ogra_: I could just make that amd64 as well :-P [20:25] haha [20:25] yeah [20:25] ogra_: is that an ack? [20:25] well it is only temporary anyway [20:25] sure, do it [20:26] ogra_: so just like http://paste.ubuntu.com/11732463/ [20:26] ? [20:26] yup, looks fine [20:27] ogra_: rsalveti ok, new build triggered [20:28] * ogra_ crosses fingers [20:28] * sergiusens_ takes a break to walk the dogs [20:33] sergiusens_: awesome [20:34] importer still not finished [20:34] wtf [20:35] sergiusens_: good luck, I need to go to bed, if it still not workss, please let me know by mail and I continue in the morning [20:47] rsalveti: build not done yet (I think all builds need to finish) [20:47] no, this was one importer from more than one hour ago [20:47] that was still running [20:48] sergiusens_: importer is another cronjob [20:48] everything is pull based [20:48] :-) [20:51] Jun 17 20:33:59 localhost kernel: [ 1905.624951] audit: type=1400 audit(1434573238.995:14269): apparmor="DENIED" operation="ptrace" profile="mir_system-compositor_snap1" pid=4484 comm="pidof" requested_mask="trace" denied_mask="trace" peer="unconfined" [20:51] jdstrand: so this is one i'm confuddled about ^ [20:52] denied mask "trace" ? [20:53] sergiusens_, rsalveti still 11th [20:53] http://cdimage.ubuntu.com/ubuntu-core/daily-preinstalled/20150617.4/ [20:54] ogra_: what I do? :) [20:55] Laney, how did you make your changes to cdimage ? [20:55] did you follow https://wiki.ubuntu.com/ReleaseTeam/CDImageSetup ? [20:55] more or less [20:55] seems there was some cowboyed code in there and it was overwritten [20:56] and usually cdimage screams loudly if thats the case to prevent you from merging [20:56] well maybe someone ran bzr revert or whatever [20:56] it'll only complain if there are conflicts, not sure I touched the same areas [20:57] yeah [20:57] reading, this looks related... [20:57] https://github.com/docker/docker/issues/7276 [20:57] Laney, thanks ... [20:57] sorry if it was me, didn't do it on purpose [20:58] is the code still alive? [20:58] no, seemingly not ... well, not the code we need [20:58] and nobody seems to have a backup [20:58] sergio is just trying his best and i dont feel awake enough to hack into it right now (i'll do it in my morning if there is no solution over night though) [21:02] kgunn_: what confuses you about it? mir_system-compositor_snap1 is trying to trace an unconfined peer process [21:03] jjohansen: i'm quite new to these concepts, and i'm going through the process of getting the security profile correct for [21:04] mir in order to be able to upload it to the store, so in my caveman brain i think "how can i make this go away" [21:05] kgunn_: well, generally speaking we don't allow a confined process to trace unconfined [21:05] kgunn_: do you know what it is trying to trace? [21:06] jjohansen: yeah, so this stems from some mir weirdness where there's a race between it and agetty, so there's a script that handles the race... [21:06] it's using pidof [21:06] that's where this is coming from [21:08] kgunn_: hrmm, so my initial reaction is don't do that :) [21:09] kgunn_: it can be worked around by giving the ptrace peer=(label=unconfined), rule to the profile but I'm not sure what is required to get that kind of rule into the store [21:09] jjohansen: :) i just found the line in docker [21:09] jdstrand: ^ any suggestions [21:09] rsalveti: yeah :/ [21:09] oops, ogra_ :-P [21:09] jjohansen: i'm sure jamie will spank me :) [21:10] sergiusens_, rsalveti ... erm ... [21:10] :) [21:10] sergiusens_, so your cdimage code is fine, it actually copied it over ... but i suspect livecd-rootfs isnt rebuilding it actually [21:10] ogra_: progress http://paste.ubuntu.com/11732678/ [21:10] 2015-06-17 20:50:58 URL:https://launchpadlibrarian.net/209366564/livecd.ubuntu-core.azure.device.tar.gz [147294112/147294112] -> "/srv/cdimage.ubuntu.com/scratch/ubuntu-core/wily/daily-preinstalled/live/amd64.azure.device.tar.gz" [1] [21:11] ogra_: heh, shared the same :-P [21:11] yeah :) [21:11] great minds and all that ;) [21:12] i see "+ tar -c -z -f /build/device-azure.tar.gz system assets hardware.yaml" in the build log [21:12] and no error [21:13] ogra_: well, it's getting the wrong link from librarian [21:13] hmm [21:13] ogra_: latest build is https://launchpadlibrarian.net/209363535/livecd.ubuntu-core.azure.device.tar.gz [21:13] ah, ok [21:14] slangasek: do you know why that can be? ^ [21:14] * ogra_ wishes back the days where cdimage was shell :/ [21:14] find_live_filesystem() was so much easier [21:15] ogra_: heh [21:15] it is all cody-summervilles fault ! [21:16] jjohansen, kgunn_: sorry, reading backscroll [21:16] just check the lp job, it should tell if the azure tarball is in there [21:17] rsalveti, yeah, that bit is fine [21:17] rsalveti: it's there, the librarian link is wrong [21:17] but we're not finding the right url [21:17] how that [21:17] rsalveti: well it's logged now ;-) [21:18] rsalveti: http://paste.ubuntu.com/11732678/ but the latest build's link is https://launchpadlibrarian.net/209363535/livecd.ubuntu-core.azure.device.tar.gz [21:18] but https://launchpadlibrarian.net/209366564/livecd.ubuntu-core.azure.device.tar.gz is also valid (just not the one you're looking for) [21:19] it is not publishing it though [21:19] according to the log [21:20] hmmm, but now i hit trace denied for all sorts of bespoke peers (webdm, docker, mir itself) [21:20] ogra_: sergiusens_: /srv/cdimage.ubuntu.com/scratch/ubuntu-core/wily/daily-preinstalled/live [21:20] ogra_: rsalveti I think I found one more location [21:20] it's there [21:20] yes, it is not publishing it [21:20] -rw-r--r-- 1 cdimage cdimage 147294112 Jun 17 20:34 amd64.azure.device.tar.gz [21:20] right [21:20] well, takes a while for it to be public [21:20] * ogra_ was just looking at the same file :) [21:21] rsalveti, cdimage logs when it publishes it [21:21] there is no azure in the log [21:21] /srv/cdimage.ubuntu.com/log/ubuntu-core/wily/daily-preinstalled-20150617.4.log [21:21] yeah [21:21] not at /srv/cdimage.ubuntu.com/www/full/ubuntu-core/daily-preinstalled/pending [21:22] man, this is so much more complicated than it should be [21:22] yep [21:22] i want shell back ! [21:22] involves 3, 4 different servers [21:22] would have taken me minutes [21:23] kgunn_: sure, ptrace is dangerous. you will need specify each of them [21:23] kgunn_: and the reverse relation as well, that is those apps have to declare the relationship as well [21:23] kgunn_: so, yes, use the rule jjohansen mentioned for now, but I'm pretty uncomfortable with the technique and the rule. I think we should figure out something better. the docker policy is not something to model your policy after, btw :) [21:24] ogra_: rsalveti http://paste.ubuntu.com/11732716/ [21:24] this prevents an attacker app being able to declare its allowed to ptrace without the peers saying yeah we know and trust him [21:24] kgunn_: the problem is that if you you are allowed to trace unconfined, unconfined allows anything to trace it [21:24] sergiusens_: looks ok [21:24] jdstrand: right, so no better suggestion for now [21:25] sergiusens_, yeah [21:25] jdstrand: really we need to fix that [21:25] jjohansen: fix the mir policy or that unconfined allows tracedby? [21:25] or both :) [21:26] both :) [21:26] so, I will probably make this a little better with a child profile for pidof [21:27] that way even if mir has an issue, it would need pidof to allow something unexpected [21:28] once kgunn_ gets through the initial policy, he'll give it to me to review and I can play with it [21:28] ogra_: rsalveti ok, this one is the one [21:28] jdstrand: hey, so what do I need to do for a snappy binary to be able to write to its user data dir (HOME)? [21:28] building ? [21:29] ogra_: yes [21:29] yay [21:29] * sergiusens_ starts to use http://whatthecommit.com/ again [21:29] lol === sergiusens_ is now known as sergiusens [21:29] jdstrand: I was hoping I wouldn't have to also make our client tool unconfined :) [21:29] stgraber: it should be allowed by the policy. You need to look in SNAP_APP_USER_DATA_PATH [21:29] stgraber, snappy binaries can not write to $HOME ... [21:29] right [21:29] error: mkdir /root/apps/lxd/0.11-0/.config: permission denied [21:30] SNAP_APP_USER_DATA_PATH is a subdir for the app in $HOME [21:30] yes [21:30] sergiusens: seems we're good regarding the --install=docker issue: https://bugs.launchpad.net/snappy/+bug/1465879/comments/5 [21:30] Ubuntu bug 1465879 in Snappy "docker framework does not install via ubuntu-device-flash" [High,Confirmed] [21:30] you don't get all of home, you get SNAP_APP_USER_DATA_PATH [21:30] Chipaca: http://whatthecommit.com/bf057fb0e2e7a4450250ebf7d6e1d084 :-P [21:30] sure sure, but clearly I don't even get write access to SNAP_APP_USER_DATA_PATH [21:30] lol [21:30] stgraber: is that an apparmor denial? [21:30] stgraber, because /root is readonly perhaps ? ;) [21:31] ogra_: isn't [21:31] and then there is that :) [21:31] [94170.804198] audit: type=1400 audit(1434576560.135:55): apparmor="DENIED" operation="mkdir" profile="lxd_lxc_0.11-0" name="/root/apps/lxd/0.11-0/.config/" pid=6085 comm="lxc" requested_mask="c" denied_mask="c" fsuid=0 ouid=0 [21:31] sergiusens: ? [21:31] Chipaca: I am just fooling around :-) [21:31] sergiusens: http://whatthecommit.com/0041a2c1bcc6d21895a46d0b92f64a88 then :) [21:32] \o/ [21:32] heh [21:32] stgraber: oh, that is because we are using @{HOMEDIRS}/*/ instead of @{HOME}. @{HOMEDIRS}/*/ does not include /root. [21:32] stgraber: I remember I asked other snappy devs about this [21:32] that probably ought to be fixed :) [21:32] and it was decided that /root/ was not needed. apparently that needs to be revisited [21:34] stgraber: before I fix it, I think there might need to be a discussion. can you file a bug agount snappy, the project? [21:34] stgraber: you can add an ubuntu-core-security task [21:34] stgraber: I have questions around /root, the FHS, rollbacks, etc [21:36] jdstrand: says that ubuntu-core-security doesn't use LP for bugs so can't add a task [21:36] jdstrand: anyway, bug 1466234 [21:36] bug 1466234 in Snappy "Apparmor denial for access to SNAP_APP_USER_DATA_PATH as root" [Undecided,New] https://launchpad.net/bugs/1466234 [21:36] stgraber: sorry, snappy the project and ubuntu-core-security under Ubuntu [21:37] I can add it [21:37] ah, done [21:40] stgraber: ok, I asked my questions. if the snappy devs are comfortable with it and respond in the bug, I'll adjust the policy [21:43] jdstrand: next question for you :) [21:43] (amd64)root@localhost:~# /usr/bin/ubuntu-core-launcher lxd lxd__0.11-0 /apps/lxd/0.11-0/bin/lxd.start [21:43] aa_change_onexec failed with -1 [21:43] . errmsg: No such file or directory [21:43] what does that mean? ^ [21:43] stgraber: it couldn't find the profile 'lxd__0.11-0' [21:43] 'lxd__0.11-0' is not formatted well btw' [21:44] so either the yaml has an issue or the launcher isn't calculating the appname part correctly [21:44] there should be something between the '__' [21:45] it should be the name of the service (from services) or the binary (from binaries) [21:45] oh, also, that thing is a framework, not sure if that's relevant [21:45] it isn't relevant to what should be happening. it might be a contributing factor to a bug [21:46] stgraber: can you paste /apps/lxd/current/meta/package.yaml? [21:46] oh, I think I know what's wrong in the yaml, testing [21:48] jdstrand: so a missing name attribute on the service was the source of the problem. Kinda surprised the tool isn't validating this though. [21:49] now to figure out the next problem [21:49] tedg: your opinion wrt SNAP_APP_USER_DATA_PATH might be relevant to #1466234 [21:49] bug #1466234 that is [21:49] Bug #1466234: Apparmor denial for access to SNAP_APP_USER_DATA_PATH as root [21:49] bug 1466234 in ubuntu-core-security (Ubuntu) "Apparmor denial for access to SNAP_APP_USER_DATA_PATH as root" [Undecided,Incomplete] https://launchpad.net/bugs/1466234 [21:51] stgraber: yes, the review tools are currently disabled (they would have caught) [21:51] jdstrand: so, http://paste.ubuntu.com/11732810/, any idea why I'm getting apparmor denials with that for the service? [21:52] Jun 17 21:50:22 localhost.localdomain audit[787]: apparmor="DENIED" operation="pivotroot" profile="lxd_lxd_0.11-0" name="/run/cgmanager/root/" pid=787 comm="cgmanager" srcname="/run/cgmanager/root/" [21:52] doesn't look unconfined to me :) [21:52] Chipaca: is there a bug to re-enable the review tools? [21:53] Wed Jun 17 21:53:12 UTC 2015 [21:53] Publishing amd64 ... [21:53] Publishing amd64 live manifest ... [21:53] Publishing amd64 device tarball ... [21:53] Publishing amd64 azure device tarball ... [21:53] \o/ [21:53] sergiusens, rsalveti ^^^ [21:53] \o/ [21:53] ogra_: it just just just finished :-P [21:53] i was tailing the log :=) [21:54] ogra_: http://cdimage.ubuntu.com/ubuntu-core/daily-preinstalled/20150617.5/ is up to date as well [21:54] stgraber: the unconfined template isn't trruly unconfined. http://bazaar.launchpad.net/~ubuntu-security/ubuntu-core-security/trunk/view/head:/data/apparmor/templates/ubuntu-core/15.04/unconfined [21:54] yippiie [21:54] stgraber: it is missing a 'pivot_root,' rule [21:54] jdstrand: ah, what do I do to get something that's completely unconfined? [21:55] * kgunn realizes no one likes being confined [21:55] because no pivot_root is going to be a bit of a problem for us :) [21:55] stgraber: I can add that [21:56] jdstrand: anyway I can easily do that locally so I can see what fails next? [21:56] jdstrand: nope [21:56] stgraber: in the mean time you can either just add the rule to /var/lib/apparmor/profiles/... or you can use 'security-policy' (https://wiki.ubuntu.com/SecurityTeam/Specifications/SnappyConfinement/DevelopingFrameworkPolicy) and use the unconfined template with pivot_root added [21:57] jdstrand: ok, I'll add the rule locally for now [21:57] kgunn: well, yes, but lxd is quite a different beast :) [21:57] :) [21:57] jdstrand: will that profile with this addition let me switch profile, specifically to real unconfined? [21:58] jdstrand: quick one i hope, so i worked through to the point now where i need to add "binaries" to my yaml for the actual mir-demo-server [21:58] and the question is about paths... [21:58] E=mc² [21:58] ;) [21:58] since mir-demo-server is actually in debs/usr/bin [21:59] (all relative to your SNAP_APP_PATH) [21:59] ogra_: ta [21:59] see i knew it was easy [22:01] jdstrand: ok, and with aa profile on this binary...since it's part of the framework, do i also need to do a seperate security pollicy on it ? [22:01] I need some go help in here. When I run go test ./... I get cmd/snappy/common.go:28: undefined: priv.WithMutex [22:01] but it is defined, and I don't get that error from my desktop. [22:02] since it needs some paths not needed by the main service [22:02] go is drunk. [22:07] elopio: tell me more [22:07] elopio: i have a good guess as to what's happening [22:08] elopio: in whatever you're running this, this not-your-desktop environ, has the GOPATH pointing at a snappy different from the one you're running the tests on [22:08] Chipaca: I don't know what else to tell you. I've pulled trunk and ran the tests. [22:08] elopio: because you're running the tests with ./... you're not picking up that snappy for the test runner [22:08] ogra_: sergiusens: lovely [22:08] elopio: but the imports in files under test use absolute paths [22:08] elopio: e.g. launchpad.net/snappy/potato [22:08] elopio: so those are coming from your GOPATH [22:10] kgunn: this is where framework-policy comes into play I think, or maybe not [22:11] well, i see i can add the client policies that the framework publishes [22:11] for use as a cap by the binaries [22:11] but, don't want to expose more through that than is needed.... [22:12] Chipaca: it works now ¬¬ [22:13] elopio: 🙌 [22:13] kgunn: right. so we want to expose the minimum in the framework-policy. are you saying that the mir-demo-client binary needs more than the minimum? [22:13] I removed my link from the GOPATH to my workspace. Then made it again... [22:13] Chipaca: how do you handle different branches in src/launchpad.net/snappy ? Do you use ln ? [22:14] elopio: you use a pipeline? [22:14] jdstrand: nope, sorry... so there's a binary mir-demo-server which is launched by (the service) mir-compositor [22:14] elopio: bzr gets confused by symlinks [22:14] kgunn: if it can use the minimum with the default template, then you can refer to your framework-policy [22:14] this is before clients actually show up... [22:14] kgunn: ok, in that case, you need to write "security-policy" for your binary [22:14] Chipaca: no, I have a workspace with all the branches as dirs. I put a link in src/launchpad.net to the one I want to test. [22:15] jdstrand: ok, so follow the same construct/form... [22:15] ta [22:15] kgunn: yes [22:15] elopio: ok [22:15] jdstrand: frighteningly i may understand this before i'm all done :) [22:15] elopio: that'll confuse either go or bzr, depending how you're doing it [22:16] elopio: you can make it work if you're careful [22:16] kgunn: I'll also (hopefully) helpfully remind you that https://wiki.ubuntu.com/SecurityTeam/Specifications/SnappyConfinement/DevelopingFrameworkPolicy has boilerplate [22:16] elopio: me, i gave up using symlinks and just move stuff around [22:16] elopio: when i'm not using pipelines that is [22:16] this is the first time it failed, so I guess I wasn't careful. [22:16] Chipaca: do you mean http://wiki.bazaar.canonical.com/BzrPipeline ? [22:16] elopio: yes [22:16] yessir [22:16] ok, I'll try that. [22:17] ok...nuff for now, maybe more later [22:17] elopio: and in bazar.conf, [22:17] n = swp :next [22:17] p = swp :prev [22:17] ps = pipes [22:18] Chipaca: those are alias, right? [22:18] elopio: in [aliases] i mean [22:18] yes [22:18] cool. Sounds less insane than links. [22:19] elopio: until it throws a stacktrace at you, sure :) [22:20] heh [22:20] I recall those [22:20] del-pipe has some issues [22:21] sergiusens: just switching with new files added but not committed seems to tickle it [22:21] anyway [22:21] Chipaca: oh, that works fine for me [22:21] elopio: everything is terrible. [22:22] elopio: we're moving back to C, and CVS [22:29] sergiusens: sorry, I have no brain state relevant to the azure files/builds/downloads. If you're stuck I'd suggest tagging the launchpad team [22:29] :) let me first hate pipeline before making the move. [22:31] slangasek: fixed 30' ago, but thanks :-) [22:31] sergiusens: hah, ok [23:05] rsalveti: do you plan to copy https://launchpad.net/ubuntu/+source/goget-ubuntu-touch/0.24-0ubuntu2 to tools-proposed? [23:06] sergiusens: already there [23:06] sergiusens: and utlemming just confirmed that it worked for him [23:06] so all good [23:06] rsalveti: oh neat :-) [23:06] one more thing to cross off the list :-) [23:06] rsalveti: are you moving it to tools per se? [23:06] or waiting on that one? [23:07] sergiusens: not now, want to migrate things at the same time we get to test our next stable image [23:07] so we can all be testing the same thing all together [23:08] rsalveti: sounds good [23:08] elopio: btw, the bzr bd command there doesn't play nice with my --builder option in bazaar.conf [23:09] sergiusens: we initially copied bzr-buildpackage from the original script, but it didn't work for me. [23:09] sergiusens: can you try that? [23:09] Building the package in /home/sergiusens/go/src/launchpad.net/build-area/ubuntu-snappy-1.1.2, using sbuild -d wily --arch=amd64 -c wily-amd64 -j9 -uc -us [23:09] Unknown option: u [23:10] I'm on trusty [23:10] oh, being called for dinner [23:19] * elopio goes to watch the game. [23:27] [ 5298.277754] audit: type=1400 audit(1434583615.201:38): apparmor="DENIED" operation="change_profile" profile="lxd_lxd_0.11-0" pid=4690 comm="lxd" target="lxc-container-default" [23:27] jdstrand: should be added to the unconfined profile too ^ [23:27] hmm [23:28] I'll add it to the list [23:28] * jdstrand heads out [23:29] * rsalveti also heads out, dinner and football === ManikTaneja is now known as manik_