=== CyberJacob is now known as CyberJacob|Away [03:31] roaksoax: can you bless this please? https://code.launchpad.net/~julian-edwards/maas/packaging/+merge/211668 === CyberJacob|Away is now known as CyberJacob [08:22] bigjools: is there something fundamentally wrong with https://code.launchpad.net/~rvb/maas/bootimage-ui/+merge/211761 or did you just forget to mark it 'Approved'? [08:23] Where did Oleg add his comments? I don't see them in the new-import-script-integration branch. [08:23] Oh, wait [08:23] jtv: https://code.launchpad.net/~strikov/maas/maas-new-metadata-format/+merge/210843 [08:23] trunk now, right? [08:24] Thanks. [08:24] Do we have parallel branches now? [08:24] I've been working off the new-*-integration one. [08:25] jtv: yeah, we need to integrate Oleg [08:25] 's work [08:26] Should we start by just making all of it live in trunk, even if not active yet? [08:27] Keep the old and the new entries in pserv.yaml, etc. [08:27] Or will it be too hard? [08:43] jtv: I think it's doable. [08:44] And probably the best option to avoid the big bang approach. [08:44] Yeah. It'll still be one functionally, but at least we won't be messing around with conflicting branches. [08:44] I went through a weird jet-lag phase earlier today, and I was just constantly wondering what was where. Scary! [08:45] Trunk is nice and safe. :) [08:53] rvba: I forgot, sorry. [08:53] no worries [08:54] rvba, jtv: I'd like to get oleg using the integration branch [08:54] however I emailed him earlier and said we'd just take over the work [08:54] Then we should figure out how to get his latest work into either maas or the integration branch, and the integration branch into trunk. [08:56] unless we changed any of the same code that is in trunk, we can just get rid of the one in trunk [08:56] did anything change in the integration branch? [08:57] Not yesterday or today. [08:57] I've been pulling it and not seeing changes. [08:57] Nothing since Tue 2014-03-18 10:53:34 +0100 [08:58] ok [09:06] rvba: I'm trying to run the import script from the integration branch, against source http://maas.ubuntu.com/images/ephemeral-v2/daily/ — but it just fails because the snapshot directory is never created. Do you get the same? [09:06] jtv: I didn't try using maas.ubuntu.com yet. [09:06] I am about to try [09:06] I've always used the fake data thus far. [09:06] jtv: I'll try now. [09:07] FWIW I got the same with the test repos, so I expect it's something I'm doing wrong. [09:26] jtv: bigjools: I've integrated Oleg's most recent fixes to the import script in our integration branch. [09:26] awesome [09:26] Great, thanks. I'll re-merge. [09:26] let's get him using it as well [09:27] meanwhile, my "make install-dependencies" is still going 20 minutes later on this lcy02 instance [09:27] Yeah, canonistack is a mess these days. [09:27] Machines on lcy02 and lcy01 are slow as hell. [09:28] popular I guess [09:28] (All the bootimages stuff is landed, I need to QA it but it should be quick) [09:31] maas seems to have about 6 million dependencies [09:34] jtv: when I merge trunk into the integration branch I get a conflict related to the 'label' work.. could you have a look? You can probably fix this quicker than me. [09:35] Sure [09:35] Ta [09:35] I don't know about the armhf template though. [09:37] We can take Oleg's version, 'highbank' is now explicitly supported by the 'generic' kernel. [09:37] "version": "14.04", [09:37] what/where do you configure the new source in the pserv.yaml? [09:37] "subarches": "generic,highbank,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t", [09:37] "release": "trusty", [09:37] "arch": "armhf" [09:38] bigjools: not sure what you mean…? [09:38] Comma-separated string? Not a list? [09:38] rvba: the new simplestreams source, where do I configure it in oleg's changes? [09:38] etc/maas/pserv.yaml [09:38] bigjools: http://paste.ubuntu.com/7124389/ [09:38] See the top of the diff in https://code.launchpad.net/~maas-maintainers/maas/new-import-script-integration/+merge/211470 [09:39] guys, I can see that much, but where in that config! [09:39] The path. [09:39] Sorry, I don't think I understand your question. [09:39] Going to be hard to resolve that conflict when tests don't pass in the existing branch... [09:39] jtv: yeah, I know :/ [09:40] If tests don't pass anyway, just go with the trunk version... [09:43] The armhf template conflict is just two identical changes conflicting. I thought they'd made bzr accept that without complaining. [09:45] formatting difference? [09:45] trailing spaces? [09:46] No. [09:46] I ran a diff. [09:46] :( [09:47] well I tried to run the new script and got this [09:47] IOError: [Errno 2] No such file or directory: u'/var/lib/maas/boot-resources//snapshot-20032014-094738//maas.meta' [09:49] Same for me. [09:49] That's the snapshot directory I mentioned earlier. [09:50] It looks as if simplestreams is supposed to create it implicitly while synchronising. [09:50] By the way, note the 20032014... yes that _is_ a date but it's little-endian. [09:53] Hi strikov! [09:54] rvba: Hey :) [10:19] strikov: that's the integration branch: lp:~maas-maintainers/maas/new-import-script-integration [10:28] bigjools: To confirm: when you had your empty drop down problem, what fixed it? Re-running import-(eph|pxe)? [10:30] gmb: in my case it was putting the tftp images back in the right place [10:30] bigjools: But wouldn't running the script again had the same effecT? [10:30] (I'm asking because the error should give some instruction as to how to get out of the problem state) [10:30] gmb: yes, but not in my case as I had a screwy package [10:30] Well, we don't care about that. :) [10:30] Okay, cool. [10:30] yes, the instruction should be the same as the bug warning banner about missing boot images [10:31] big* [10:31] Right. That's what I'm going to steal :) [10:32] gmb: parfait [10:34] Grr. [10:34] allenap: \o/ [10:34] ;) [10:34] wb allenap [10:34] Don’t talk to me. [10:34] ;) [10:35] so gmb, allenap, rvba, jtv: I shall make sure kanban cards are complete for the list on that doc [10:35] and then please feel free to start taking them [10:37] allenap: here's a good reason to start with a new config file, we need to write into it some config based on the current bootimages [10:37] does the yaml writer maintain comments? [10:37] bigjools: Nope, it doesn’t. [10:37] uh huh [10:37] Afaik. [10:37] :) [10:39] bigjools: However, you can just append to the yaml file and it’ll dtrt. [10:39] allenap: true! [10:40] When it comes to in-place updates to config files, there’s nothing really compelling out there. Configobj is the closest I’ve seen, but that’s kind of baroque. [10:40] Do we need to store this in a config file at all? [10:40] * allenap can’t remember exactly what we’re storing. [10:41] Parameters for the imports: arches, releases, source URLs. [10:41] Now, the source URLs we need to have somewhere. [10:41] configObj is what bzr used [10:41] But the rest I guess could be generated from a form. [10:41] allenap: http://paste.ubuntu.com/7124600/ [10:41] Hi lifeless [10:42] What I had in mind originally was: import link goes to a form page, where you can enter the releases and architectures you want, and _that_ gets pre-populated based on existing boot images. [10:42] Oh, that doesn't work with multiple simplestreams sources does it? [10:43] This is all the test failures we've got on the integration branch http://paste.ubuntu.com/7124602/ [10:43] I would suggest people start fixing in small branches... [10:43] wotcha lifeless [10:44] bigjools: can you land your packaging branch? I can't QA my changes right now because of the new dependency. [10:45] rvba: yes! [10:45] jtv: yes, the failures can be easily divided into "topics" [10:45] bigjools: ta [10:45] rvba: so you never had that error where the snapshot directory did not exist? [10:45] jtv: no [10:45] And you used that config you pasted to Julian earlier? [10:47] jtv: no, I was using the test data so that path was different. [10:47] What path? [10:47] Oh, the URL?\ [10:48] What did you use? The path to the repo1 (or repo2) directory? [10:48] Yeah, the URL. Named 'path' in the config. [10:48] file:// URL? [10:48] lifeless: Ah, interesting, that makes me want to look again. [10:48] Yes. [10:48] That didn't work for me either. Maybe it was the various problems in pserv.yaml... [10:49] Because I noticed that you passed a comma-separated list for subarches, not a list like in the integration branch. [10:50] Ugh. When I try it against the simplestreams-v2 daily URL, it breaks because it looks for an sjson index. [10:50] Which isn't there. [10:51] rvba: I can help fix tests [10:51] let me just get the int branch [10:51] strikov: do you know if, for manual testing, we can make the import script use index.json (as opposed to index.sjson) and skip any attempt at verifying signatures? [10:52] bigjools: cool, the card contains a link with all the test failures. [10:52] bigjools: I'm fixing the kernel opts-related failures now. [10:52] jtv: its better to put signatures in place, i think [10:52] jtv: let me tell how in a moment [10:52] *tell you how [10:52] ta [10:52] strikov: thanks. It's better with signatures of course, but right now, there aren't any and we do need try out some manual runs. [10:52] lifeless: While you’re there, would you be interested in something like https://code.launchpad.net/~allenap/maas/run-isolated-with-debugging/+merge/211803 in upstream subunit? [10:53] https://code.launchpad.net/~rvb/maas/integ-fix-tests/+merge/211903 [10:53] jtv: bzr branch lp:simplestreams simplestreams [10:53] cd simplestreams [10:53] make gnupg [10:53] ./tools/tenv js2signed /home/ubuntu/boot-resources-v2/ [10:54] jtv: that will sign your metadata with simplestream's default key which is in ./gpg/ folder [10:54] strikov: not what I needed! [10:54] jtv: then you can add this key to your keyring and all the checks will pass [10:54] But I'm trying to import a stream that doesn't have any signatures at all. [10:55] jtv: js2signed will add all the signatures automatically (create sjson from json) [10:55] Oh, it inserts that on my end somehow? [10:55] Instead of on the stream I'm trying to import from? [10:55] jtv: which meta do you use? self-generated? [10:56] No, Scott's online URL. [10:56] http://maas.ubuntu.com/images/ephemeral-v2/daily/streams/v1/ [10:56] jtv: Ah. Try mine: http://162.213.35.97/boot-resources-v2/ [10:56] jtv: it's basically the same but with signatures [10:57] jtv: you need to integrate simplestreams key though [10:57] Thanks. That will give me an exact mirror of Scott's otherwise? [10:57] rvba: can you talk me through that MP [10:57] jtv: let me see if scott made any changes yesterday [10:57] Thanks. [10:57] AIUI labels were added. [10:58] bigjools: this is to account for this http://paste.ubuntu.com/7124669/ [10:58] rvba: I guess that's pretty simplified! [10:58] Yeah, we don't need to patch things up anymore. [10:58] jtv: well, he did (and that's good), we indeed need to disable sign check then [10:59] rvba: but this means we only keep the most recent locally? [10:59] Yes. That's what the import script does. [10:59] strikov: yeah... for this kind of thing I really want to be close to the source, so we don't get everything jusssst right — for an obsolete version. [10:59] Also, the 'info' file is now completely gone. [11:00] rvba: get_ephemeral_name also gets a label, of course, in the branch I'm working on. [11:00] Right. [11:00] jtv: to disable sign check we need to (1) change index.sjson to index.json inside script and (2) implement stub policy handler for simplestreams [11:00] jtv: does it work for you? [11:01] strikov: sure, it's only for a temporary hack. Thanks! [11:01] jtv: give me a few moments [11:01] * jtv gives strikov a dozen moments [11:08] rvba: fwiw not sure I'd bother with MPs for the int branch fixes [11:08] bigjools: wfm [11:08] rvba: but for safety, did you set append_revisions_only? [11:08] bigjools: no [11:08] rvba: better do it :) [11:08] since we're all going to be pushing up [11:09] bigjools: how do I do that? [11:11] bigjools: found it; done. [11:11] rvba: you were quicker than me [11:11] sftp to the branch location basically [11:11] bzr config -d bzr+ssh://bazaar.launchpad.net/~maas-maintainers/maas/new-import-script-integration/ append_revisions_only=True [11:12] http://stackoverflow.com/questions/5413602/monotonically-increasing-bazaar-trunk-revision-numbers [11:15] allenap: hmm [11:15] allenap: so with v2 you can debug through the stream [11:15] lifeless: Ah ha, okay. We ought to upgrade at some point then. [11:16] allenap: but, that wouldn't stop the dup madness [11:16] allenap: so I think that this is a simple bug and we should fix it in subunit [11:16] separately to any discussion about v2 upgrades [11:17] lifeless: Okay, I’ll put a patch together. [11:17] let me look a little more [11:17] ok [11:17] so we replace fd 1 [11:17] which means sys.stdout will still go to that fd [11:18] and that fd does need to be the fd of the parent pipe, or you won't get the test data [11:18] so yeah - I think this is a case for upgrade the connection to v2 and route stdin and out to the child [11:18] via v2 packets [11:18] should be fairly straight forward [11:18] get you debugging [11:19] I think I need to read up on v2’s design. [11:23] jtv: https://pastebin.canonical.com/106840/ [11:23] jtv: it disables sign check and switches from md5 to sha256 [11:23] Thanks! I'll feel a lot better knowing that I can try things out against Scott's latest "real" data. [11:24] allenap, bigjools, gmb, rvba: see strikov's paste above — the ephemeral-vs stream is not signed yet, so Oleg's patch should help us work around that. Do Not Commit. :) [11:25] jtv: Right. Give me one more moment please -- it crashes with Scott's data due to some reason (not related to sign check) [11:25] Whoopsie [11:25] yay? :) [11:26] It's one step further than what I got. :) [11:26] jtv: we should probably steal the signature-related code from the old ephemeral import script. [11:27] jtv: heh, that's not my problem though -- so of the entries in Scott's meta doesn't contain checksums [11:27] And maybe extend it a bit to allow unchecked downloads. [11:27] *some of [11:27] rvba: For Freudian reasons, you probably know more about that code than I do... [11:28] rvba: I don't think we have much signing-related code in there anyway. [11:28] jtv: just an option to choose the keyring. [11:28] rvba: I am thinking that we can get rid of provisioningserver.import_images.tests.test_ephemerals_script entirely [11:29] bigjools: not entirely, there are tests for helpers we still use in there. [11:29] uec2roottar thingy, compose_filters [11:30] rvba: ok I'll do it [11:30] jtv: well, i'd say that Scott's meta is broken in many ways [11:30] ? [11:31] Yay [11:31] jtv: not all entries contain checksums (which is required for my script because its cache is checksum-indexed) [11:31] Ouch [11:32] jtv: that's definitely a requirement for meta to provide these checksums -- so Scott just didn't fix it yet [11:32] strikov: it's important to keeps a list of all these problems with the new metadata. Would you mind adding it to the document we looked at during the call? [11:32] rvba: well, i don't think that's a real problem because Scott didn't tell me that his meta is ready yet [11:32] rvba: so, he's working on it right now [11:34] strikov: I know, but we should keep track of what creates problems for the import script. So that we can make sure things are fixed when Scott releases a new version. [11:34] jtv, rvba: how about not pulling resources w/o checksums available? [11:34] jtv, rvba: that's a hack but it allows us to move forward [11:34] You mean just ignoring them? Sounds reasonable to me. [11:35] strikov: yeah, that's a good workaround. [11:35] rvba: understood, okay, will do [11:35] strikov: ta [11:35] Great, thanks. [11:35] spasiba. [11:38] rvba: just pushed up to r2140 [11:41] Real-world boot images display, for those interested: http://people.canonical.com/~rvb/boot_images_ui.png [11:41] Nice. [11:41] The sorting could be better :) [11:42] I was just writing that I wasn't going to ask about that, because... [11:42] priorities! [11:42] But it's nice to have some visibility at last. [11:42] Yes [11:48] Review needed: https://code.launchpad.net/~jtv/maas/labels-in-new-import-script/+merge/211910 [11:50] jtv: I'll take it [11:50] Thanks. [11:50] Disappointingly small, given the trouble it's caused. [11:50] Argh! I accidentally committed my pserv.yaml changes. [11:50] Let me fix that. [11:53] jtv: why? These changes are welcome. [11:53] Not with the commented-out lines... [11:54] I can add back the actual improvements. [11:55] who subscribed the team to the int branch? [11:55] don't do that [11:56] Are we subscribed automatically maybe? [11:56] dunno [11:56] Because it's owned by the team? [11:56] rvba, jtv, allenap, bigjools, gmb: Workaround to be able to pull Scott's meta (choose armhf in pserv.yaml to avoid pulling tons of images): https://pastebin.canonical.com/106841/ [11:57] \o/ [11:57] strikov: ta [11:57] This probably deserves a comment in the code though :) [11:57] This isn't for committing — it's for experimenting. [11:57] rvba: that a very dirty workaround :) [11:57] Ah ok. [11:58] Which I think is very empowering. [11:58] I thought we wanted to commit that… until simplestreams gets fixed. [11:58] No, it's just a workaround I requested so we could experiment. [12:00] Reviewer needed (tiny review) https://code.launchpad.net/~rvb/maas/bootimg-ui-sort/+merge/211912 [12:01] rvba: [12:01] jtv: you need to manually merge your branch to get it landed on the integration branch. [12:01] rvba: Approved, even :) [12:01] heh, thanks gmb [12:02] * gmb -> afk for a few hours; back later. [12:03] jtv, rvba: Scott is fixing this hash issue right now [12:04] strikov: cool [12:04] Even better. [12:05] rvba: I've been staring at code and re-reading over and over and finally I think I need to accept I am too tired to continue. I'll catch up tomorrow, don't forget your EOD email :) [12:06] I'll send one of my own [12:06] nn bigjools [12:06] nytol [12:06] nn bigjools [12:26] Oh, strikov, one thing I was wondering about: why the strange date format in the name of the snapshot directory? Why not YYYYMMDD? [12:26] Much better for sorting! [12:27] I noticed because the name looked like a date in 2003. :) [12:27] jtv: no idea.... that was the last thing i worried about. YYYYMMDD looks really well ;) [12:27] If it doesn't cause any problems, would you mind if I changed it? [12:28] jtv: sure [12:28] Thanks. [12:31] integra [12:31] ouch, sorry [12:31] allenap: ping [12:32] strikov: looks like I got a successful import... the files are still placeholders. [12:33] hi guys [12:33] jtv: really? [12:33] 1.4G -rw-r--r-- 9 root root 1.4G Mar 20 11:49 root-image [12:33] I'm working on the maas latest bbits, and with today's brz pull I have a few issues :-) === strikov is now known as strikov-lunch [12:36] jtv: I'll return back in 15 mins and do a deeper check [12:37] jtv: we shouldn't have placeholders there afaik and that looks like a bug [12:40] The first error in the logs is: [Thu Mar 20 14:22:20.513813 2014] [proxy:error] [pid 16550:tid 46944109119232] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:5243 (localhost) failed [12:40] it was working fine with yesterday lates bzr bits, so I wonder if anybody has an idea by any chance :-) [12:41] allenap: can you help alexpilotti please ^ [12:41] allenap: tx! [12:45] allenap rvba: a stack trace of a related error: http://paste.openstack.org/show/73903/ [12:49] allenap rvba: narrowed down to not being able to start: /home/cloudbase/maas/bin/python /home/cloudbase/maas/bin/maas-region-admin runserver 5243 --settings=maas.demo --noreload [12:49] getting "ImportError: cannot import name close_old_connections" === strikov-lunch is now known as strikov [13:02] rvba, jtv: Scott updated his meta. Now it's signed and contains all the required hashes [13:02] rvba, jtv: checking my script against it right now === cmagina-away is now known as cmagina [13:32] Hi alexpilotti. Looking now. [13:32] allenap: tx [13:34] alexpilotti: The first error implies that the Django app is not up. Try `make status` to check if it’s been started, then look at logs/webapp/current to see what might be the issue. [13:34] The second error the same. [13:35] alexpilotti: Ah, okay, I read a bit more of what you wrote :) [13:36] allenap: webapp is getting continuously respawned [13:36] e.g: [13:37] services/web: up (pid 7565) 125 seconds, normally down [13:37] services/webapp: up (pid 10030) 0 seconds, normally down [13:37] services/web: up (pid 7565) 140 seconds, normally down [13:37] services/webapp: up (pid 10251) 1 seconds, normally down [13:37] alexpilotti: Are you running this on Trusty? Can you check the version of Django you’re using? I wonder if close_old_connections only appears in 1.6. [13:38] yeap trusty, did an apt-get upgrade along with the bzr pull [13:38] alexpilotti: Run: apt-cache policy python-django [13:38] python-django 1.6.1-2 [13:39] python-django: [13:39] Installed: 1.6.1-2 [13:39] Candidate: 1.6.1-2 [13:39] Version table: [13:39] *** 1.6.1-2 0 [13:39] 500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages [13:39] 100 /var/lib/dpkg/status [13:39] alexpilotti: And: bin/py -c 'import django; print django.__file__' [13:39] In the branch. [13:39] "/usr/local/lib/python2.7/dist-packages/django/__init__.pyc" [13:40] Ah, you’ve got a version of Django in /usr/local that might be stale. [13:43] allenap: checking! [13:47] allenap: good ctach: python -c "import django; print django.get_version()" [13:47] 1.3.1 [13:53] removed, everything is good now, tx! [14:05] hi allenap, do you have some time to chat today? [14:07] tych0: Yes! I assume to talk about LXC? Do you want to get this done in time for Trusty? [14:07] allenap: that was the idea [14:07] i might just bag it, though [14:07] you guys don't seem to like the patch [14:07] tych0: Yeah, sorry. I do like the idea though. I guess it’s mainly for testing, right? [14:07] and for use in the cloud installer [14:08] we're already doing this but we have a race condition that this would solve [14:09] anyway, i'm open to suggestions [14:09] but i don't want to waste any more time if it's not going to go in [14:09] tych0: If it can wait a few weeks it would make me happy. I’d like to get it in, but properly. For example, we may want to add a hook to decommission a machine, to avoid that power-off-equals-destroy hack. [14:10] allenap: from my POV we could just power it off and leave it alive, and delete it next time it comes up [14:10] that's basically what maas already does [14:10] i don't mind either way, really [14:11] allenap: waiting a few weeks doesn't really do it for me. if it's not in trusty we can't use it in the cloud installer, so i have much less incentive to fix it. [14:12] tych0: We do have nascent plans to overhaul the transitions a node goes through. Supporting this would be a good use-case. If it can wait, let’s do it properly. [14:12] tych0: Okay, let’s talk at 1430 UTC. Is that okay? [14:13] allenap: that's in 15 minutes or so? [14:13] tych0: Yep. [14:13] allenap: sounds good [14:14] gmb, rvba, jtv: Either of you want to talk about LXC in MAAS too? [14:14] a/Either/Any/ [14:15] allenap: I think I'll pass… I'm too deep into the integration work. [14:32] tych0: Problems without hangout? [14:32] Or with even. [14:35] tych0: Try https://talky.io/maas [14:36] allenap: joining now === kevin is now known as Guest2426 === kevin is now known as Guest85094 === cmagina is now known as cmagina-away