=== chihchun_afk is now known as chihchun [06:33] ogra_: I blame fw_setenv for the breakage and I think I can prove it: http://paste.ubuntu.com/11923858/ [06:34] ogra_: thats operating on the pristine uboot.env I got from you and that was generated with mkenvimage, no snappy involved at all and notice the corruption at the end of the file [06:39] ogra_: I think I have a plan, lets see if it works [07:10] good morning [07:10] good morning [07:15] ogra_: I uploaded a fix to the ppa that should fix the corruption, its a bit ugly, I work on a nice version with proper tests etc now. its also very panic()y to ensure that any remaining issues are caught [07:15] ogra_: once the updated uboot-go is in, we need to rebuild snappy and make a new image and then fingers crossed :) [07:17] hey fgimenez and dholbach, good morning! [07:19] hi mvo dholbach and all :) [07:22] hey hey :) [08:02] mvo, thanks ! [08:03] mvo, i still dont get how it broke between 125 and 126 though ... only the config was different there [08:04] (and 125 was to 100% reliable) [08:16] reading through backlog you guys had some fun last night - anything i should test with 126? [08:16] longsleep, no, but the next build [08:17] ogra_: ok - did you make any changes to the saveenv logic? [08:18] not yet, i'm cleaning that up now, but since it worked for both of us before it should technically be fine, this is more cosmetic to rip out the inconsistent quoting [08:18] ogra_: i think i have alreayd put double quotes around everthing - are you doing that too? [08:18] yep [08:19] our current snappy_boot only has half of the vars and values quoted [08:19] (on the BBB that is) [08:20] well, i only put double quotes around stuff used for comparison, for the rest i never use double quotes [08:21] ok - i see what you mean i am inconsitent there too [08:21] * longsleep is fixing [08:21] heh [08:21] as i said ... simply cosmetic [08:22] by the way - what is the reason to have snappy_cmdline as seperate var ? [08:22] i think its unclean to just add that value at the end of mmcroot and will probably remove the var [08:22] i'm also putting a saveenv behind every setenv here [08:23] really [08:24] well, for the snappy_boot line ... not for everything :) [08:24] i need to make it multi line to make sense of it [08:24] yeah i figured that much :) [08:26] ogra_: mine currently looks like this https://pad.struktur.de/p/snappy_boot [08:26] ogra_: where do you put extra setenv? [08:26] saveenv [08:27] http://paste.ubuntu.com/11924149/ [08:27] i cant open your url ... insufficient privs [08:27] err [08:27] maybe its blocked from outside [08:27] let me check your diff [08:27] The requested URL /insufficient_privileges was not found on this server. [08:27] :D [08:28] thats like "the url "broken link" was not found" [08:28] hehe [08:29] i think the pad is not in outside DNS and you end up at some fallback server [08:29] * ogra_ notes he ends up at a fedora server .... thats all it says [08:29] so you saveenv whenever the snappy_ab is changed - not sure why this is required - i was under the impression that this is only for the current boot [08:30] well, it means you keep the condition if you powered off the board before the boot finished [08:31] mhm, what is the difference between boot failure and powered off - shouldnt it roll back in that case? [08:31] yes [08:33] ok i see your point - i was under the impression only snappy_mode was set to try on upgrade. But that would make no sense. [08:33] well ok hold on [08:34] it would make sense if the system upgrade would set snappy_ab to the upgraded partition value as well [08:34] it will roll back in both cases ... weather or not the var is set ... [08:34] then you would not have to save that change from u-boot ? [08:35] either all the vars are stored and the next boot will switch back to the former partition .... or none of the vars is stored so they are reset and it boots normal ... [08:36] ogra_: I can build you a custom snappy if you want to test/explore with the fixed code until the full image is ready [08:36] but in the latter case the bootloader will never know that an upgrade boot was attempted ... in the former it will ... the result is the same [08:36] the next boot will only switch to former partition if it failed to boot when the successfull boot does not reset snappy_mode and snapp_trial_boot [08:36] mvo, i can wait for the full image ... btw, did you make the snappy_trial_boot a boolean ? [08:36] longsleep, exactly [08:37] so why do you saveenv the snappy_ab change in u-boot then? [08:37] ogra_: its still a unset, I can change that now (would like to change it in both grub and uboot then in wily) do you think its important? i.e. will uboot be unhappy if its not set at all? [08:38] * longsleep does not get it - lets get some coffee [08:38] mvo, well, i'd like to exclude the possibility that it could get unhappy ... i have no proof, only gut feeling [08:39] longsleep, hmm, yeah, prehaps i have a brainfart here [08:40] ogra_: lets use this https://etherpad.mozilla.org/sglergmwrt - i hope you can open that :) [08:40] arg [08:40] it did format it [08:40] lol [08:40] in a weird way [08:40] that is just plain stupid [08:40] paste it in, gets formatted [08:41] what is "run setup" ? [08:42] thats setting up fdt based on variables [08:42] and how do you hand over the initrd, i dont see that in bootm [08:42] https://public.pad.fsfe.org/p/snappy_boot [08:43] copy paste fail - forgot the last line [08:43] ah ! [08:43] :) [08:43] so [08:43] i fail to see why the setenv snappy_ab parts need to be saved [08:44] looks good ... and i thihnk based on the above i'll leave out the saveenv [08:44] and i would like to get rid of snappy_cmdline [08:44] feels strage to just add it to mmvroot [08:45] well, it is good to have it separate for better readability ... [08:45] just added the setup - so you get the whole picture [08:46] so that you can see the actual cmdline options in one go without having to pick them out of an already way to long snappy_boot line [08:46] ogra_: so the only question is what does happen when the system gets updated. Someone sets snappy_mode to try - i get that much, the question is if snappy_ab is also changed or not [08:46] it is [08:46] snappy cvhanges it from userspace [08:47] it sets "ab" to the "other" partition and sets trial_boot to 1 [08:47] ogra_: good, then it would be bad to saveenv snappy_ab from u-boot [08:48] ogra_: i just changed snappy_cmdline to be added by mccargs to bootargs [08:48] i think that is cleaner [08:50] * ogra_ needs to check how mmcargs looks on the BBB and RPi [08:50] uh [08:51] i think it just sets the bootargs. It is called by snappy to pull in mmcroot as this contains the snappy_ab variable [08:51] never noticed that before ... BBB sets rootfstype there [08:51] yeah i saw that. I think that is redundant [08:51] no, thats dangerous [08:51] mhm ok :) [08:52] mmcrootfstype=ext4 rootwait [08:52] not so clever if we switch to some other fs :) [08:54] yeah [08:55] ogra_: so in the pad i now have my complete mmcargs with snappy_cmdline put there, instead of in the setenv in snappy_boot [08:55] feels cleaner [08:55] yes, i did the same here (anmd dropped mmcrootfstype) [08:58] ok great - so what is the problem with build 126 / why did you add the saveenv's in the first place? [09:01] ogra_: above you wrote the system sets ab to the other partition and trial_boot to 1. Is that really so? I think it should not touch trial_boot, but set snapp_mode to try instead? [09:02] Good morning all; happy Gorgeous Grandma Day! 😃 [09:03] longsleep, 126 corrupts the uboot.env file by adding a value without a var name to it [09:03] ogra_: I triggered a new armhf build for 15.04, fingers crossed [09:03] fw_setenv actually allows things like: "fw_setenv ' ' 1" [09:03] mvo, \o/ [09:04] oh [09:04] any news about using uboot-go instead? [09:08] longsleep, it is go ... [09:09] (meaning it is a static binary of 2MB size) [09:25] ogra_: thats great then [09:41] ogra_: btw - i wanted to ask you how to handle upgrades from snappy-system.txt to uboot.env. When an new image is released and gets updated to uboot.env [09:41] i'm not sure we do that [09:41] (we do use the old files if they exist, so it only affects new installs) [09:42] mhm [09:42] ok [09:42] so version 4 will contain the new and old behavior then? === howefield_afk is now known as howefield [09:48] ogra_: looks like the new armhf 15.04 is available [09:48] * mvo downloads [09:51] mvo: version 128 sounds good? [09:51] * ogra_ downloads too [10:01] mvo, ogra_ : all right i am booted into 128 - let me know if you want me to do any specific tests [10:02] Mhm - fw_printenv Read error on /boot/uboot/uboot.env: Success [10:09] davidcalle, I added an item to the trello card about generalising the markdown importer so we can import docs from lp:snapcraft too - I'll look into that now [10:09] mvo: well, fw_printenv cannot read or set anything in the .env file, while uboot-go works just fine [10:10] longsleep: hm, hm, let me investigate [10:11] mvo: i see the scary config in /etc/fw_env.config [10:11] mvo: i also just updated uboot-go to latest git and it still works [10:12] (ODROIDC)root@odroid:~# fw_setenv foo bar [10:12] Read error on /boot/uboot/uboot.env: Success [10:12] Error: environment not initialized [10:12] (ODROIDC)root@odroid:~# fw_printenv [10:12] Read error on /boot/uboot/uboot.env: Success [10:17] longsleep: so 113/rolling works with fw_setenv/fw_printenv, I create a 15.04 one now [10:20] mvo: did you fix the issue with the OEM snap in rolling? (bug #1477224) [10:20] Bug #1477224: OEM snap does not install when building image for rolling [10:20] bug 1477224 in Snappy "OEM snap does not install when building image for rolling" [Undecided,New] https://launchpad.net/bugs/1477224 [10:20] longsleep: no, I think its a race actually, after ~5 tries it worked for me [10:20] ok let me try that [10:22] mvo, hmm ... settin snappy_ab to b and snappy_mode to try doesnt get me booted from b [10:23] ogra_: does it show the correct values in the uboot prompt with printenv? [10:24] havent checked that yet [10:24] * ogra_ isnt sure if our logic is actually correct [10:26] did you see what i wrote above: ogra_: above you wrote the system sets ab to the other partition and trial_boot to 1. Is that really so? I think it should not touch trial_boot, but set snapp_mode to try instead? [10:26] mvo, http://paste.ubuntu.com/11924535/ [10:26] yeah, the trial_boot looks wrong [10:28] setting trial_boot to 1 and mode to try works [10:29] yes - but you should not set trial_boot, you should set mode to try and ab to the one which you want to try [10:29] whats how i see it at least [10:29] right, but thats not what the logic we have does [10:30] * ogra_ takes a look at the old code [10:30] no? i think it does just that [10:30] http://paste.ubuntu.com/11924535/ is the currentl logic [10:30] yes [10:30] but that is in uboot [10:30] it onl switches if mode is try and at the same time trial_boot is 1 [10:31] sigh [10:31] yes but it only has to switch if the boot failed [10:31] why cant cloud-init not write to /dev/null [10:31] so annoying [10:31] +1 [10:31] well, at least i dont see corruption now [10:31] from my point of view, uboot is only switching in case of roll back [10:32] the try is done from the system side after update [10:32] it shoudl also switch in case of upgrade :) [10:32] set try and ab to other [10:32] and it doesnt with the current logic [10:32] so the system does only set try but not ab to other [10:32] thats the error then [10:32] no, the system sets try and ab to the other [10:32] then all is good [10:33] well, no [10:33] it doesnt boot from b if i set it to b [10:33] well it should, because mode is try, it boots from whatever ab is set to because trial boot is 0 [10:34] http://paste.ubuntu.com/11924564/ [10:34] so either ab is a or if its b then trail boot is 1 [10:37] if ab is b and you want to boot it from b, then trial boot needs to be 0 [10:37] ogra_: hm, hm U-Boot# printenv snappy_ab [10:37] snappy_ab=b [10:38] ok, then trial boot is 1 [10:38] U-Boot# printenv snappy_ab [10:38] snappy_ab=b [10:38] and snappy_mode is try as well I assume? [10:38] i.e. the env is read correctly? [10:38] http://paste.ubuntu.com/11924581/ [10:38] ah, wait, didnt check mode [10:39] ab is definitely b after reboot [10:39] as you can see in the paste it boots a though [10:40] well, for me it boots just fine from b when i set it to b, saveenv and reset [10:40] just snappy_ab b ?without the try? [10:40] without hte snappy_mode try? [10:40] if ab is set to b and mode is try, then trial is 1 if it boots from a [10:41] yes without the try [10:41] let me try with try [10:41] ok that boots from a [10:42] so .. [10:42] script does not work [10:42] let me check [10:43] http://paste.ubuntu.com/11924593/ [10:43] mvo, ^^^ [10:43] so ab is set to b and mode to try [10:43] * ogra_ doesnt get that [10:43] ah i guess the problem is that snappy_trial_boot is not set and then always returns true [10:43] i remember test to behave that way [10:44] yeah, thats why i wanted a boolean [10:44] * ogra_ sets it ... once cloud-init lets me [10:44] if test ${snappy_trial_boot} = 1; then echo yes; fi [10:44] yes [10:44] echo ${snappy_trial_boot} [10:44] mvo, what sets it ? [10:44] thats…unexcpected [10:44] if snappy_trial_boot is not set then it always sets to a [10:45] u-boot, seriously? [10:45] if i set it to 0 [10:45] it works just fine [10:45] mvo, well, thats the hush shell implementation ... cant blame uboot itself :) [10:45] hm, quoting maybe? [10:45] if test "${snappy_trial_boot}" = "1"; then echo yes; fi [10:45] give no output [10:46] no, my script is quoted [10:46] i remember having this issue before [10:46] yeah, i have a local fix for the quoting, but its not that [10:46] http://paste.ubuntu.com/11924602/ [10:46] looks like quoting to me unless I miss something [10:46] hmm [10:47] mvo: no it does not work with the quoting either [10:47] http://paste.ubuntu.com/11924610/ [10:47] longsleep: how does yours look like? [10:47] longsleep: could you pastebin it please? [10:47] so what are the actual reasons for using cloud-init ? this is really annoying the hell out of me (additionally to the extra 30sec boot time it spams the console all the time) [10:47] i know we have distro ways of generating ssh keys [10:47] mvo: http://paste.ubuntu.com/11924615/ [10:48] and i guess it wouldnt be to hard to have user creation code adopted from d-i or some such [10:48] what else does cloud-init do for us ? [10:49] i have checked that a couple of weeks ago. It creates the default user too i think [10:49] and something with the hostname [10:49] right [10:49] see what i write [10:49] *wrote [10:50] we have tools for that in the distro that we use on the livecd for example [10:50] its a 10 line script or so [10:50] same goes for ssh ... its a few extra lines to have it create keys if they dont exist [10:50] longsleep: hm, thats confusing. does your uboot shell also behave like what I see in http://paste.ubuntu.com/11924610/ ? [10:50] and setting the hostname from /proc/cpuinfo isnt to hard either [10:50] mvo: i just checked that [10:51] mvo: mine behaves differently [10:51] mvo: if test "${snappy_trial_boot}" = "1"; then echo yes; fi [10:51] yes [10:51] i just dont get why we have to go through all the pain with cloud-init for these tzhree bits ... and make it really hard and uncomfortable for developers [10:51] (with unset trial boot) [10:51] longsleep, mvo most likely a version thing [10:52] ogra_: yeah - i think you should ask the one who added it - i would really love to see it go away [10:52] ogra_: let me check Git [10:52] seems everyone would [10:52] so yeah setting trial_boot to 0 gets me properly booted from b as expected [10:53] longsleep: woah, thats amazing, uboot is always good for a suprise [10:53] mvo: http://paste.ubuntu.com/11924645/ [10:53] mvo: indeed [10:53] mvo, well, vendors are good for surprises in that case ;) ... since they fork off from ancient uboot versions and never merge back usually [10:54] ah yes [10:54] http://git.denx.de/?p=u-boot.git;a=commit;h=2453de99df576fb907fe06cac58c628e3590833f [10:54] ogra_: yeah, looks like we need to make it bool and also change grub to be on the safe side, it looks on the bbb quoting will help, but if thats not universal its no good [10:55] yep [10:55] amazing [10:55] thanks longsleep [10:56] i did remember seeing that because i have another u-boot where i needed the -e for test and thus merged around in test command before [10:56] * longsleep is going to have some lunch [10:56] longsleep: do you also a yes for ' if "" = "1"; then echo yes; fi' [10:56] mvo: err sorry [10:56] mvo: what do you want me to run? [10:57] got it hold on [10:58] mvo: you missed a test in there right? else if gives unknown command [10:58] mvo: empty string works just fine [10:59] longsleep: yes, sorry. I wonder if "if printenv snappy_trial_boot; then echo yes; fi works [10:59] longsleep: interessting and crazy [10:59] WOAH ! [10:59] mvo, why is /var/lib/dpkg/info not wiped ? [11:00] ogra_: we only do that on wily [11:00] (BeagleBoneBlack)ubuntu@localhost:~$ du -hcs /var/lib/dpkg/info/ [11:00] 9.2M /var/lib/dpkg/info/ [11:00] ah, k [11:00] then i'm fine [11:00] mvo: that seems to work http://paste.ubuntu.com/11924669/ [11:01] mvo, that meanbs we have 9.2M extra space there in which we could ass uboot-go ;) [11:01] lol [11:01] *add [11:01] yeah, I thnk we should add it [11:01] http://people.canonical.com/~mvo/tmp/beagleblack_1.10_all.snap <- that has a snap with the quotes, building a new image+mp will be at least 2h that we can't test the rest :/ [11:02] pitti had suggested how we restart the network from inside the 'first boot' unit; does anybody remember how it was? [11:02] longsleep: I wonder if that could/should be used so that we can continue iterating, the image building is so slow :/ [11:02] mvo, does that have my MP merged ? [11:02] ogra_: I used your 1.9 snap [11:02] cool [11:02] as the base and just added the quotes [11:02] its writing now [11:02] mvo, hmm, no [11:02] mvo: so for your u-boot it works with quotes? [11:03] mvo: i could merge the fix into my u-boot [11:03] longsleep: it appears to be, I'm double checking [11:03] you probably also want snappy_trial_boot=0 in uboot.env.in [11:03] but i guess it will be trouble [11:03] ogra_: thats in there already afaict [11:03] as that change is from early 2014 [11:03] * mvo double-checks [11:03] oh [11:03] longsleep: yeah, I think the final version needs to set it to "0" [11:03] ah, yeah [11:04] heh ... i dont know my own code after two days anymore ... [11:04] longsleep: I'm just sick of waiting for another image and want to give it a good beating (for testing) [11:04] ogra_: I don't after 2h [11:04] * ogra_ guesses he should check for some vacation :) [11:04] mvo: but for testing now i am fine with whatever workaround [11:04] mvo, but you have kids distracting you ... thats fine :P [11:04] then I can prepare a branch after lunch, its trivial [11:04] ogra_: you have cats, about the same, no ;) ? [11:04] nah, they lseep during the day :) [11:04] *sleep [11:05] * longsleep now really goes for lunch [11:05] * mvo is also hungry [11:06] * ogra_ considers breakfast :) [11:10] Cogito ergo ipomoea batatal. [11:10] * Chipaca takes a break === kickinz1|afk is now known as kickinz1 [11:49] ogra_: http://people.canonical.com/~mvo/tmp/beagleblack_1.11_all.snap <- latest that works for me, i.e. it now boots b when I set snappy_ab b and snappy_mode try [11:50] next I will make it not unset the env [12:02] Chipaca: if you have a moment, it would be great if you could check https://code.launchpad.net/~mvo/snappy/snappy-no-unset-trial-boot and https://code.launchpad.net/~mvo/snappy/15.04-snappy-uboot-no-unset-trial-boot [12:02] ogra_: ^--- those branches set to "0" instead of unsetting [12:02] mvo: are those the result of what you and ogra debugged this morning? [12:02] Chipaca: yes, its the result of trying to be as compatible with u-boots quirks as possible [12:06] mvo: +1'ed; code looks sane, and i've watched you debug it :) [12:07] Chipaca: heh, thanks [12:07] Chipaca: crazy I say this u-boot [12:07] but it seems its under control [12:12] mvo: i did some more testing and found that all combinations seem to work as long as no variable is not defined. [12:12] mvo: See https://github.com/longsleep/snappy-odroidc/blob/uboot-env-support/oem/boot-assets/uboot.env.in for my latest env [12:16] longsleep: great, the above MP fixes the undefined vars [12:16] * mvo is still a bit disappointed by uboot, really [12:18] well u-boot is getting better and has improved a lot since 2014 [12:18] unfortunately most vendors have based their u-boot on 2013 something .. [12:18] yeah, sorry, I'm ranting a bit [12:19] at least it is open source - imagine you had to deal with closed source issues [12:19] longsleep: heh :) good point [12:19] i am pretty sure uefi implementations are even worse than u-boot is [12:23] mvo, could be a lot worse ... we could have to deal with redboot or aboot :) [12:24] merge looks good indeed [12:29] hmm [12:29] * ogra_ wonders why his phone screen doesnt turn on on notofocations [12:29] *notifications === fgimenez_ is now known as fgimenez [12:35] WOW ... [12:35] https://launchpad.net/~snappy-dev/+archive/ubuntu/image/+packages [12:35] why does it say "no signer" for the last snappy build [12:36] auto upload from the daily builds [12:36] thats ok [12:36] I triggered builds now and once they are in we can get the golden image [12:36] hopefully :P [12:36] does it correct that later ? [12:37] ogra_: the later? you mean the unset vs set = 0? [12:37] i mean the "no signer" i have never seen that before [12:37] I thnk its there all the time, for every daily build [12:37] * mvo could be wrong though [12:37] i wonder why i never noticed it before [12:38] i not only need new teeth and vacation apparently but also glasses :P [12:38] * ogra_ is happy he still has some hair at least :P [12:41] mvo: so you think the fw_* commands will work with the next build? [12:41] mvo: i did not manage to get the rolling release to accept my oem snap :/ [12:41] fw_* should work, yes [12:42] they already work here right now though [12:43] oh - so am an 128 and they do not work for me [12:43] oh ? [12:43] longsleep: crc error? or something else? [12:43] 4 vs 5 byte again? [12:44] no i got 5 byte env [12:44] parses fine with uboot-go [12:44] size also corret on disk and maches fw_env.config? [12:44] fw_env.config needs the hex size [12:44] if its nothing like this I run out of ideas :/ [12:44] the config has the two lines with the comment for 5 bytes [12:45] uboot-go is smarter and figures the size out by itself [12:45] mhm [12:46] http://paste.ubuntu.com/11925071/ [12:47] 5 bytes until the b [12:47] longsleep: what fize has the file? [12:48] mvo: 32768 bytes [12:48] fgimenez: I was wondering if we could leave your job code only for provisioning. [12:48] (thats the value in u-boot compile time) [12:48] fgimenez: like leave the --command flag out, and make the main return an ip. [12:48] longsleep, to small then [12:48] err [12:49] +uboot.env is created from uEnv.txt via: [12:49] +$ mkenvimage -r -s 131072 -o uboot.env uboot.env.in [12:49] elopio, sure, the cloud package takes care of that [12:49] from our beaglebone README in the snap branch [12:49] longsleep: hm, so we need to set 0x8000 in fw_env.config for you or you increase the env [12:50] ogra_: why too small? [12:50] probably best if you increase it as we have no way right now to change fw_env.config [12:50] how large should it be [12:50] longsleep, 128k [12:50] oh ok [12:50] just use the same mkenvimage call [12:51] * mvo will explore to put uboot-go into the image, thats all a bit insane IMNSO [12:51] i doo [12:51] see https://github.com/longsleep/snappy-odroidc/blob/uboot-env-support/oem.mk#L29 [12:51] mvo, +1000 [12:51] I mean, it seems bad to force everyone to use our env size just because of a silly config [12:51] longsleep, no, you use -s 32768 [12:51] yes [12:51] i was not aware that i have to use another size [12:51] fgimenez: so I'm thinking, on snappy/_integration-tests/main.go we can do a kvm-ok. If it succeeds, run adt-run ... -- snappy. If it doesn't, call your project and get an ip from canonistack, and then adt-run ... -- ssh ip. [12:51] +$ mkenvimage -r -s 131072 -o uboot.env uboot.env.in [12:51] but no problem [12:52]