/srv/irclogs.ubuntu.com/2014/04/10/#maas.txt

=== CyberJacob is now known as CyberJacob|Away
jtvbigjools: we can just make maas-test import from the daily images and ignore the releases repo, right?01:47
bigjoolsjtv: hmm good question01:47
bigjoolsI would say both, but default to release01:47
jtvMaybe we can have a chat about this?01:49
mwhudsonhm02:24
mwhudsonhow do you configure maas-dns?02:24
jtvmwhudson: maas generates the dns config using a template (in /etc/maas/templates IIRC)02:37
jtvSo you edit the template.02:37
mwhudsonhm02:37
mwhudsonwhat i want to do is tell it where to forward most requests to02:38
mwhudsonlike e.g. "python.org"02:38
jtvAh, that's already a thing.02:38
jtvFor some quantity of thing.  Let me look.02:38
mwhudsoni don't know much about configuring dns02:38
jtvSo what you want to set is the “upstream” DNS, for all requests for hostnames not managed by the MAAS, right?02:40
mwhudsonyes02:41
jtvThere's a UI setting for that.  Look for "upstream DNS."02:41
mwhudsonoh UI02:41
mwhudsoni wonder why the UI isn't responding02:42
jtvAt all..?02:42
mwhudsonjust hanging02:42
mwhudsonbet it turns out to be a firewall02:42
jtvThen why did it work before though?  Maybe a big ongoing upgrade?02:42
mwhudsonhm?02:43
mwhudsonit's never worked for me02:43
mwhudsonon this install02:43
mwhudsonwhich is a day old02:43
bigjoolsyou can set it via the API too, it's just a config item02:45
mwhudsonah ok i think the api not working was a stale ip address02:51
mwhudsonshouldn't maas createadmin prompt for a password?02:53
bigjoolshere we go again :)02:53
bigjoolscreatesuperuser does02:53
mwhudsonor maybe the docs should be fixed, if it's that sigh-worthy :)02:53
bigjoolspeople keep asking for one and then the other02:53
bigjoolsone is django's command, one is our wrapper02:54
mwhudsonbigjools: http://maas.ubuntu.com/docs/install.html#create-a-superuser-account seems to be just wrong (for trusty anyway i guess?)02:54
bigjoolsyeah the docs will be updated when 1.5 is released02:54
bigjoolsand we'll have docs for each supported release, at last02:55
bigjoolsseparate docs I mena02:55
bigjoolsmean02:55
mwhudsoni don't see a setting for dns in the ui :/02:55
mwhudsonoh that will be nice02:55
bigjoolsit's on the config page somewhere02:55
mwhudsonthis maas install seems less fresh than i thought it was :/02:56
bigjoolsheh02:57
bigjools2230 is latest trusty02:57
mwhudsonis it possible that this maas doesn't think it's managing dns?03:01
mwhudson1.4+bzr1693+dfsg-0ubuntu2.203:01
mwhudsonok that's ancient?03:01
bigjoolsmwhudson: ancient!03:01
mwhudsonffs03:01
bigjoolsthat's cloud-tools or saucy's version03:01
bigjoolsonly about 500 revision behind03:02
mwhudsonoh03:02
mwhudsonthis vm is saucy03:02
mwhudsonffs03:02
mwhudsonanti-progress: the dns server i want to forward to does not, in fact, appear to be listening to me03:19
bigjoolswinning03:22
bigjoolsjtv: is there anything else in your branch that needs attention?04:11
bigjoolsI will approve it04:11
jtvI hit a different problem in Q/A; part that worked before.  May just be my experimental change.04:14
jtvI always knew the uploading of the file would be the hard and risky part, but why did it work when Raphaël tried it?04:49
bigjoolsjtv: what goes wrong?05:02
jtvOh, I see it now.  The scp command.05:04
bigjoolsjtv: let me guess, quoting?05:10
jtvNo, just a missing username/host prefix.05:10
jtvI guess Raphaël fixed that up in his experiment.05:11
=== vladk|offline is now known as vladk
bigjoolsahah05:17
jtvYup, that seems to work.05:19
jtvI'll try another Q/A run.05:19
jtvHmm... images are imported just fine now (and according to spec), but Raphaël's new poll-until-images-are-imported code hits a timeout.05:43
jtvSo report_boot_images is not working in some way.05:44
jtvOr maybe it's just genuinely taking too long.05:45
jtvAh, wouldn't surprise me — he switched over from running the import script to triggering a run from the API.05:49
jtvThe script doesn't return until the import is done, and from there all we need to wait for is for report_boot_images to run.05:49
jtvBut when we call the API for importing images, it returns immediately — and so what we're waiting for is completion of the import itself.05:49
jtv(No longer report_boot_images, because the API-triggered task runs that right after the import).05:50
jtvThe timeout is 10 minutes, IIRC, which is fine for the report_boot_images cycle — but maybe not for the import.05:50
bigjoolsjtv: so, is it working or not? :)05:50
jtvNot yet, not yet!05:50
jtvCan't wait to have some hardware for this stuff...05:51
jtvNo, no, the change that triggers the API version of the imports doesn't seem to have landed yet...05:54
jtvAhhh, found the cause: chmod.06:14
jtvAnother easy fix.06:14
jtvMy branch just passed the enlist-a-node test.06:32
jtv\o/  Test successful.  Landing.06:36
=== CyberJacob|Away is now known as CyberJacob
=== vladk is now known as vladk|offline
jtvbigjools, is this what you had in mind?  https://code.launchpad.net/~jtv/maas-test/daily-option/+merge/21508807:09
bigjoolsjtv: boolean params always pique my interest07:10
bigjoolsthey are nearly always better replaced with some sort of enumerated value07:11
jtvVery true, but it's the UI choice that matters here.07:11
bigjools--image-type=07:11
jtvWhat happens under the bonnet can be changed easily.07:11
bigjools?07:11
bigjoolsand default to "release"07:11
jtv?07:11
bigjoolsthen it stays a little looser07:12
jtvI have similar misgivings about calling things types of things.  There are usually different aspects that you could call types.07:12
bigjoolsok07:12
jtvThis option is just a shortcut for an alternate config...  easier to improve once a real pattern forms.07:13
bigjoolsit was a rough suggestion, I'm open to options07:13
bigjoolsyou *could* just take a whole simplestreams path07:13
jtvYeah, but it's a lot more work for the user.  I figured that could be a next step in making it more flexible — if we need it.07:14
bigjoolsbut remember that this API needs to stick after bootresources is gone07:14
jtvI see that as a reason to keep the UI as simple as possible: keeps it easy to change the whole implementation, even if we may later have to build a more flexible, more complicated way of doing the same thing.07:15
jtvNot hard to say "you can't specify --daily-images and a custom path" later, if/when we add an option for the latter.07:15
bigjoolsthe mass-test UI would have to change if a new "type" was introduced07:15
bigjoolsit would not have to change in maas07:15
bigjoolssee my point?07:15
jtvWait... what's the connection to maas?07:16
jtvI mean, maas would never know about any of this anyway.07:16
bigjoolsmaas itself just cares about the url to the index07:17
bigjools"path" in the config07:17
jtvYes, that's what I mean.07:17
bigjoolsit knows nothing about dailies/release07:17
bigjoolsI think maas-test should be the same07:17
jtvAnd that's the way I like it.07:17
bigjoolsme too07:17
jtvWe can do that, but getting the right URL has been a bit of a pain in the past.07:17
jtvYou know: "I have a URL to an index listing here, but how many path components do I need to strip off to get the URL that simplestreams wants?"07:18
bigjoolsmy suggestion is to default the whole URL and allow knowledgeable users to override as they want07:19
bigjoolsdailies won't get used much after release, everyone will want to know if trusty works on their hardware07:19
jtvOkay.  It's an easy change — I just figured it was hard on the users.07:19
bigjoolsI would normally agree but I think the frequency of using this option will be limited to power users07:20
jtvWhat should I call the option then?  --image-stream?07:20
bigjools--image-stream-url07:20
bigjools?07:20
jtvOK07:20
bigjoolscool, thanks07:20
jtvrvba: if you want to use the API for starting the import job, that'll work now — but don't forget that the timeout for the BootImage records to appear will then have to cover the import time, not the report_boot_images interval.08:50
rvbajtv: not sure it's worth doing right now tbh.  It's a cleanup that can be done later.08:50
jtvOK, just a thing to keep in mind.08:51
rvbaYeah.08:51
jtvBecause 10 minutes is a long, long time for the report_boot_images schedule... but not a lot of time to import images!08:51
rvbaA very useful cleanup would be to avoid downloading the i386 images when it's not necessary.08:51
rvbaMaybe I'll fix that.08:51
jtvThat'd be great — look for "mipf" in the function name, in utils.py IIRC.08:52
rvbaallenap: bigjools: jtv: gmb: just a reminder (in case I forget): just before the release we want to change the label of the Trusty images from 'daily' to 'release'.08:52
jtvThat's what says "oh you asked for armhf?  I'll give you armhf/generic and throw in i386/generic as well."08:52
rvbaI know.08:52
jtvChange the label?  Of already imported images you mean!?08:53
rvbaNo, the label for Trusty in /etc/maas/bootresources.yaml.08:53
jtvSo Scott is going to throw the switch?08:53
rvbaWe probably need to get confirmation from him that the 'release' images for Trusty have been published.08:55
jtvSince AFAIR _all_ the images in the daily stream were labelled "daily," shouldn't we just use label "*"?  Saves us worrying about synchronicity.08:56
rvbaDoesn't make a great difference.  We would still have to change the 'path.'08:57
jtvIs he going to change that as well!?08:58
jtvISTM we can just put trusty under the "releases" source.08:59
jtvAnd just leave the label as *.08:59
jtvISTM we can do that before the upstream data changes, with no ill effects.08:59
bigjoolsrvba: we need to change that ASAP well before the FF.  If there's a problem with images then we just ask someone to put images in release.09:00
bigjoolsdid that make sense?09:00
rvbabigjools: I just checked and it seems the 'release' images for Trusty are already there.09:00
bigjools\o/09:00
bigjoolsJFDI then09:00
* gmb -> getting breakfast; bbiab09:01
rvbaYep.09:01
rvbaJFDoingI09:01
bigjoolso/09:01
allenaprvba, jtv: https://code.launchpad.net/~allenap/maas/dhcp-leases-parsing/+merge/215112 is my work-in-progress for speeding up lease parsing.09:03
allenapIt uses a regex to identify records, then it select only the latest ones, then it uses the full parser to break those apart.09:04
allenapIt saves a lot of work when the leases file is large.09:04
rvbabigjools: btw, the upcoming (still waiting for the RT ticket to go through —hint, hint) api doc page has a toc: compare http://people.canonical.com/~rvb/maas-docs/api.html with http://maas.ubuntu.com/docs/api.html09:04
jtvallenap: that's very little code indeed...  not sure what it does yet.09:05
bigjoolsrvba: can you go and pester them again for the RT please09:05
bigjoolsrvba: and the toc is a nice improvement, thanks09:07
rvbabigjools: we got a response on the RT, just no estimation of when it will be done.  I'll ask them about that.09:07
bigjoolsrvba: ok, thanks09:07
rvbaDone.09:08
=== vladk|offline is now known as vladk
jtvrvba: did you want to make the change to how/where we import trusty?10:12
jtvIf not, I can propose a quick branch.10:13
rvbajtv: I have a branch already.  I'm testing a package built from it right now.10:23
rvbajtv: https://code.launchpad.net/~rvb/maas/use-release-trusty-resources/+merge/215114 WIP until I get confirmation that it's working as expected.10:24
jtvWhy not just move the "trusty" selection under the other source?10:26
jtvI'm currently setting up for an import with that config btw.10:26
rvbajtv: yes, that would be even a bit simpler.  Fixing the branch now.10:28
jtvOh dear.  Import fails while trying to write the meta file: snapshot directory does not exist!10:31
jtvI have a horrible feeling we haven't been getting any new dailies for a while, and so failed to notice some problem or other...10:32
rvbauh-oh10:34
jtvJust filed it as bug 130575810:36
ubot5bug 1305758 in MAAS "Import fails while writing maas.meta: No such file or directory" [Critical,Triaged] https://launchpad.net/bugs/130575810:36
jtvOh, another possibility I guess would be that there is simply nothing to download, and that this gets handled particularly badly in some cases.10:46
rvbajtv: the import doesn't seem to work now.  I only got the precise image.10:46
jtvAny tracebacks?10:47
rvbaNo, no traceback.10:47
jtvSpooky.10:47
rvbaLike you said, maybe there is nothing to download.  Although the images seem to be there.10:48
jtvProbably identical to versions we already had...10:49
jtvAnd I don't see anything that would create that snapshot directory if there was nothing to download.10:49
jtvThe code would try to write that maas.meta file, but the directory wouldn't exist.10:49
jtvI seem to remember that this was always a weakness in the code, actually.10:49
rvbajtv: the images seem to be there but clearly, the index file doesn't contain an entry for Trusty/release.10:50
jtvMaybe because the same file was already in there as Trusty/daily?10:50
rvbajtv: I guess I'll wait for smoser to come online and ask him about that.10:50
jtvWow.  Latest images I have in cache are from March 31.10:51
jtvUh-oh.  Deleting everything from /var/lib/boot-resources doesn't fix the problem either.10:53
jtvAnd it fails suspiciously quickly — within a second.10:53
jtvMaybe because it's hitting the caching proxy.10:53
rvbapdb time :)10:54
jtvWhen I put the daily trusty import back in (in addition to the release trusty) it seems to get going again.10:56
jtvBut of course maybe it's just slower to fail because I made it download stuff.10:57
jtvI'm expecting to be called out soon.  May not have time to say so.10:57
rvbajtv: when importing (with the updated config) from a freshly installed package, I don't get a traceback.11:13
jtv!?11:13
rvbaOnly precise gets imported.  but no traceback.11:13
jtvThen I guess it may be getting confused because the same images are in different streams with different metadata...11:13
jtvAlthough then why did it keep failing for me when I deleted my cache & snapshots..?11:14
rvbaI doubt that's the problem; like I said, there is no reference to Trusty/release in the index.11:14
jtvOh, there still isn't?11:15
jtvAh yes, see that.11:15
rvbaNo.  Again: the images seem to be there but there is no reference to them in the index.11:15
jtvSo then the import procedure wouldn't find them...  I think.11:16
rvbaThat's the behavior I see.11:16
rvbaDoesn't explain the bug you're seeing though.11:16
jtvBut the behaviour you're getting sounds completely correct under the circumstances, so maybe it's just something weird in my setup.11:20
jtvAfter the ongoing import I'll try re-running it, and if that doesn't reproduce the problem, I'll try re-running but without the daily trusty source.11:20
rvbak11:20
rvbaBut like you said, this part of the code seems a bit fragile.  I'd be relieved if we could get to the bottom of this.11:21
jtvThat's exactly why I'm trying to reproduce the problem.11:23
jtvCan't promise that it'll work.  :(11:23
jtvAhhh, I've reproduced the traceback.11:46
jtvBy commenting out the "daily" source.11:46
jtvAh!  Maybe I understand why!11:47
jtvThe snapshot directory gets created, "en passant" as it were, when <something> is downloaded from the simplestreams repo.  Not sure what exactly, but no downloads means no snapshot directory.11:47
jtvBut then.11:47
jtv(Good thing I recently refactored this code — otherwise I wouldn't have understood it)11:48
jtvThen, the import code generates a yaml dump for the metadata it has downloaded from the repo, _in memory_, and compares it to what's in /var/lib/maas/boot-resources/current/maas.meta11:48
jtvIf the two do not match, then something has changed, and so the import code decides that the new snapshot directory is worth keeping.11:49
jtvIt writes a new maas.meta file in that snapshot directory.11:49
jtvWhich, oops, does not exist because all that happened is that things disappeared.11:49
jtvI think the sensible thing to do is to create the snapshot directory.  Assuming the rest of the code behaves sensibly.11:50
=== fader_ is now known as fader
jtvOh dear.  The story just got weirder.12:00
jtvThe metadata dict is empty.12:00
jtvThat means that no matching products were found.12:01
jtvBother.  All this time wasted on what is probably a syntax error.  I don't like yaml.12:35
rvbasmoser: around?13:17
rvbajtv: gmb: allenap: is one of you taking care of https://code.launchpad.net/~andreserl/maas/third_party_drivers/+merge/215065 ?13:20
jtvNot me, sorry!13:21
allenaprvba: Yeah, I was looking at it, but I’m not going to be able to review it until at least 1900 UTC.13:44
rvbaallenap: I've written the tests.13:44
allenaproaksoax: ^ is that going to be too late.13:44
allenaprvba: \o/13:44
rvbaallenap: I'm taking care of it unless gmb wants to review it :)13:44
allenaprvba: If gmb’s not around I think a selfie might be in order.13:44
rvbaallenap: that's the plan13:45
roaksoaxallenap: nope, we have until EOD today13:54
roaksoaxallenap: i'll be pushing the release team toapprove this too while jhobbs finishes up some changes13:55
roaksoaxallenap: all the hardcodede stuff (where the pPA is and stuff needs to go on a config file)13:55
rvbaroaksoax: btw, we'd like to land https://code.launchpad.net/~rvb/maas/use-release-trusty-resources/+merge/21511413:56
rvbaBut it seems the images for Trusty/release are not included in the index file yet.13:56
rvbaThe simplestreams index that is.13:56
roaksoaxrvba: so it would fail?13:56
roaksoaxrvba: maas-improt-pxe-files would fail?13:56
rvbaroaksoax: it would not fail for it would only import precise's images.13:57
rvbas/for/but/13:57
roaksoaxrvba: so there's something weird13:57
roaksoaxrvba: http://paste.ubuntu.com/7230893/13:58
roaksoaxrvba: and it that diff I see: arches: ["i386", "amd64"]13:58
rvbaroaksoax: what's weird?13:58
rvbaroaksoax: the config file gets rewritten… but it's semantically the same.13:59
roaksoaxrvba: ok13:59
roaksoaxrvba: so, i'm thikning that sicne there no release for trusty yet14:01
roaksoaxrvba: we could sru this, or tryu to get it post today?14:01
rvbaroaksoax: SRU works for me.  It's a bit of a chicken and egg problem.14:01
roaksoaxyeah14:01
rvbabigjools: see ;) ^14:01
roaksoaxrvba: https://code.launchpad.net/~andreserl/maas/third_party_drivers_1.5/+merge/21518514:03
rvbaroaksoax: approved (you can usually self-review simple backports)14:04
roaksoaxrvba: thanks :)14:07
roaksoaxrvba: it does nee dot be manually uploaed right?14:08
=== matsubara_ is now known as matsubara
rvbasmoser: when are the Trusty/release images going to be available in the simplestreams index on http://maas.ubuntu.com/images/ephemeral-v2/releases/ ?  i.e. should we land https://code.launchpad.net/~rvb/maas/use-release-trusty-resources/+merge/215114 as an SRU?14:49
roaksoaxrvba: don't lanmd anything to 1.5 please14:58
rvbaroaksoax: well, I'm not landing this unless the images are there, that's for sure.15:01
roaksoaxrvba: thganks15:01
faderHey folks, is there any documentation anywhere on authenticating against the REST API?15:12
faderI can use it properly from a web browser if I have logged into MAAS interactively already, but if I haven't logged in or try from a script I get something like "Unrecognised signature: GET list"15:12
faderSeems to be similar to: http://askubuntu.com/questions/402691/handling-ubuntu-maas-restful-api-login (and not covered that I see in the API docs)15:12
rvbafader: Hi, I just posted a reply to that question on askubuntu.com.  Although we know the documentation is a bit poor in this area, the example Python code should be you started.15:17
rvbafader: note that, if you're using Python, you can use the client library directly.15:17
faderrvba: Thanks, I'll take a look!  (Don't shoot me but I was planning to do it from PHP via its built-in URL grabbing as a prototype)15:19
faderI'm just trying to build a page that shows the status of multiple MAAS clusters as a proof-of-concept, nothing that will really be in production15:20
tych0hi smoser, roaksoax, it looks like maas defaults to syncing the daily ephemerals as well as the release versions15:24
tych0is that the way we are going to release it?15:24
rvbatych0: That was my question to smoser :).  We've got https://code.launchpad.net/~rvb/maas/use-release-trusty-resources/+merge/215114 ready to land.  But we're waiting on simplestreams having the right data.15:25
tych0oh, i see15:25
tych0rvba: yeah, cool15:25
smoserrvba, ?15:27
tych0rvba: so one thing i don't understand about that diff, can we not specify two releases in the same keyring path?15:27
smoseroh. i see.15:28
smoseri'd land it.15:28
rvbatych0: that's exactly what that branch does. The diff is a bit messed up.15:28
tych0or maybe a question better for smoser15:28
smoserwell you're not going to have something labeled release until ... after release.15:28
tych0rvba: ah, ok15:28
tych0rvba: i am not that good at reading diffs :-)15:28
smoserso that would put you in the realm of SRU15:28
rvbasmoser: okay, so I guess we will SRU this15:28
rvbatych0: this diff is particularly nasty15:29
tych0:-)15:29
smosersuck.15:29
smoseralright. so lets figure something out here.15:29
smoseri'll try to get something itnto daily today.15:29
rvbatych0: here is the file http://paste.ubuntu.com/7231245/15:29
smoserand we can test that, and then i think i'm willing to call somethinng release in this single case.15:30
smoserbut we could also possibly call it "rc" or something15:30
rvbasmoser: it's a bit of a chicken and egg problem.15:30
smoserand you allwo the label "rc"15:30
smoserlabel rc|release15:30
rvbasmoser: that would work15:31
tych0rvba: cool, that's what i was thinking15:31
smoseri dont think its terrible either. and actually, since you have (poorly) only 'trusty' in that list, then its safe in the future.15:31
smoseri really dont like that you've only extended the number of places with hard coded release names though.15:31
rvbaNot sure what you're talking about.15:32
smoser'trusty'15:32
smoseryou'll never see 'urban' (of 'urban unicorn')15:32
smoserwhen it would be better if you started pulling those things in (or at least allowed for that) when said unicorn was 'release' or 'rc'15:33
smosermaas should not have a database of ubuntu names.15:33
smoserthats what i'm complaining about.15:33
rvbaI see.  This is a long-standing bug.15:33
smoseryes. and this config file extended the number of places where naems are typed.15:33
smoseri will work today15:34
smoserand try to get daily updated.15:34
rvbaThis file replaces two other files with these names in it.  So technically, it's one step into the right direction.15:34
smoserrvba, thats good.15:34
smoser*and* its user editable config15:34
smoserrather than source code15:34
smoser:)15:34
smoseri'll get daily updated today. and we'll get that tested and then will promote a trusty to 'rc'15:35
smoserand then you can land that patch tomorrow15:35
rvbaSee, there is your silver lining.15:35
rvbasmoser: okay, please send an email to a maas ML when it's done.  Julian will probably be able to test and land the change to bootresources.yaml15:36
=== vladk is now known as vladk|offline
=== CyberJacob is now known as CyberJacob|Away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!