#maas 2012-10-08
<bigjools> jam: thanks for the review - not sure what difference your suggestion makes as everything seems to work anyway.  I think it's more a UI validation thing isn't it?
<jam> bigjools: I think if you get a form with multiple errors, the 'dict' method allows it to present both to users.
<jam> I could be 100% wrong, though.
<jam> I was just following along what other things did when I wrote a form recently.
<bigjools> heh, I honestly have no idea.  Given this was an API call I figured a human readable string would be for the best but....
<jam> bigjools: so the big win is to just not traceback, the rest is gravy
<jam> If it works for you, that's enough for me for now.
<bigjools> righto, cheers
<jam> I'm certainly not blocking the fix, just something I thought I saw.
<bigjools> well I am always open to improvements, otherwise what's the point of reviews
<bigjools> :)
<jam> rvba: merge conflict on your ssl-skip-check branch
<rvba> jam: right, I'm on it, thanks for pointing it out.
<rbasak> filter(arch=['armhf/highbank', 'armhf/armadaxp'])
<w7z> arch__in=
<allenap> filter(arch__in=['armhf/highbank', 'armhf/armadaxp'])
<rbasak> arch_filter('arm')
<rbasak> return filter(arch__in=['armhf/highbank', 'armhf/armadaxp'])
<mgz> bug 1062340
<mgz> ...nobot
<ubot5> Launchpad bug 1062340 in MAAS "juju provider interacts poorly with arch constraints" [High,In progress] https://launchpad.net/bugs/1062340
<mgz> thanks bot.
<jtv> Any reviewers available for a smallish code extraction?  https://code.launchpad.net/~jtv/maas/maascli-fake-config/+merge/128466
<jam> allenap: so after looking into it, it appears you *can* send application/json (or yaml, or a few things), but it doesn't come through transparetly.
<jam> Instead of coming in as request.POST or request.GET, you get it as request.data
<jam> (unless POST and GET also set data?)
<jam> allenap: and performance wise, 42s with encode_multipart_data, and 33s as JSON with my hacks
<allenap> jam: Cool. Not as conquer-all as hoped, but it's some improvement.
<mgz> request.data makes more sense
<jam> mgz: well, clearly I broke something as trying to revert to non JSON completed in 5s :) (with nothing getting updated, so clearly broken)
<mgz> if you hack it to just concat the raw output and return in as application/octet-stream, what's the timing on that?
<mgz> should give a baseline for the quoting, I suspect with C extentions json shouldn't be too bad though
<jam> mgz: well, I'd still need to parse it, since we're dealing in batches.
<mgz> (or is timing just that side difficult?)
<jam> mgz: I can run it under lsprof to get feelings for timing as well. For 'how much time is spent in this C extension' lsprof is quite good
<jam> because it doesn't hook the extension and slow it down.
<rvba> allenap: I was just running the test suite in your branch 'make-docutils-happy' and I noticed a test failure that is also there in trunk (!): http://paste.ubuntu.com/1267323/
<rvba> allenap: does it ring a bell?
<jtv> Thanks for the review rvba
<jtv> rvba: tests are passing for me... I hope I didn't accidentally land a bad maascli change.
<jtv> Actually, I just ran tests in trunk and they passed...
<rvba> jtv: really? That's weirdâ¦
<jtv> Ah!  The test may be affected by the existence of maascli profiles.
<jtv> Annoying global state.
<jtv> That branch I have up for review will make it easier to patch that part.
<rvba> jtv: you're right, if I move ~/.maascli.db the tests pass.
<jtv> :/
<jtv> rvba: if you move it back and run those same command lines that fail in the test, you may get a slightly more informative error.
<jtv> Probably not very, since maascli swallows tracebacks, but some.
<rvba> jtv: I'll file a bug.
<jtv> Thanks.
<mgz> if I do request.GET.get("something") what happens if non-ascii bytes are percent encoded in the query string value?
<mgz> arbitrary bytestring? decoded to unicode as utf-8? left quoted?
<allenap> jtv, rvba: Slighty related, but I may change the -d/--debug flag to be on the root argument parser, then we can consult it about displaying stack traces.
<jtv> allenap: I'm about to move the parser into its own module.
<allenap> jtv: Okay, I'll stay away from it for now.
<jtv> I was having the same thought, actually.
<allenap> Haha :)
<allenap> roaksoax: Hi there. How do you feel about some package renames, a la https://bugs.launchpad.net/maas/+bug/1053507
<ubot5> Ubuntu bug 1053507 in MAAS "maas installs python modules with out any namespacing" [Critical,Triaged]
<allenap> or is it too late for that?
<rbasak> mgz: I think I've got all the pieces, but what do I do about the test_POST_acquire_treats_unknown_arch_as_resource_conflict test? Now that I'm raising InvalidConstraint for an unknown architecture, the API returns an HTTP 400. Shall I just change the test accordingly, or is there some other behaviour that we want/
<rbasak> ?
<jam> rbasak: I think BAD_REQUEST is better than CONFLICT
<jam> it just did conflict because that is what it was doing.
<rbasak> thanks jam. I'll do that then
<mgz> right, that test just wants adapting, should have left a comment in it to that effect
<rbasak> smoser: do we have a precise ephemeral image released with a fix for bug 978127  yet, and if not, what's the schedule for this?
<ubot5> Launchpad bug 978127 in cloud-init (Ubuntu Precise) "incorrect time on node causes failed oauth" [High,Triaged] https://launchpad.net/bugs/978127
<rbasak> (please)
<roaksoax> allenap too late i think if u want to rename the packages you need to rename upstream so
<roaksoax> otherwise it makes no difference
<allenap> roaksoax: Okay, I'll stay away from that for now then.
<roaksoax> allenap im not saying is not possible thought but Daviey was gonna accept the upload to the archives today
<roaksoax> im on holliday today
<roaksoax> allenap you might wanna check with him
<rvba> Have a nice holiday roaksoax ;)
<allenap> roaksoax: Right, sorry, didn't mean to bother you then.
<roaksoax> rvba thanks ;)
<roaksoax> allenap lol no worries im gonna be around just trying to stay away from work
<roaksoax> ;)
<smoser> rbasak, you want to test the daily there?
<smoser> i can then promote that.
<rbasak> matsubara: ^^ could you try the daily ephemeral on highbank please?
<matsubara> rbasak, yes, will do today. I saw your email and the only workaround I didn't know how to do was to adjust the RTC for the arm board
<rbasak> matsubara: yeah it's really awkward, since MAAS is the preferred tool to get an Ubuntu install on highbank in the first place
<rbasak> matsubara: I think that if the daily does fix it then trying the daily will be the least work for all of us to verify it
<matsubara> rbasak, cool
<rbasak> matsubara: to verify that you have the problem, check the apache logs on the MAAS server. I think /var/log/apache2/access.log or something?
<rbasak> matsubara: if you see a load of 40(1? 3?) messages for the same URL within a minute or two of booting into commissioning, then you have the RTC issue
<matsubara> rbasak, ah ok. thanks for the pointer
<smoser> rbasak, the other thing we need to hvae qa'd is at least that they do not regress 12.04
<smoser> the 12.04 maas.
<smoser> so if you wanted to walk http://bazaar.launchpad.net/~smoser/maas/maas-pkg-test/view/head:/maas-ephemeral-test-precise.txt to make sure we get those done, that'd be great.
<rbasak> allenap: could you take a look at the test noise in http://paste.ubuntu.com/1267509/ please? The diff I'm working on is http://paste.ubuntu.com/1267475/. I can't see what I'm doing to introduce this, and the noise is particularly unhelpful.
<smoser> i'm not here today.
<allenap> rbasak: That's fixed in trunk.
<allenap> Ignore it for now.
<rbasak> allenap: how long ago? I pulled trunk an hour or so ago.
<allenap> rvba: r1217
<rbasak> allenap: ah
<rbasak> allenap: thanks!
<jtv> Any reviewers available?  I have some MPs waiting: https://code.launchpad.net/~jtv/maas/maascli-extract-parser/+merge/128501 and https://code.launchpad.net/~jtv/maas/maascli-profile-module/+merge/128510
<jtv> allenap, can I prevail on you perhaps?
<jtv> Or is "impose" the word I'm looking for?
<mgz> does this error trying to run tests after pulling changes from today mean anything to anyone? <http://pastebin.ubuntu.com/1267528/>
<jtv> Rings a bell.
<jtv> Try "make distclean" and then "rm -rf db"
<jtv> Then try again.
<mgz> make clean and rm -rf db didn't help
<mgz> I'll try distclean.
<jtv> Best do the distclean first; it kills the postgres instances running inside your branch.
<mgz> hm, and eggs, I might back them up first
<jtv> back them up?
<jtv> The db directory in your branch was designed to be utterly disposable.
<mgz> I haven't bothered setting up an egg cache here, as I'm only using one working tree
<jtv> Ah
<mgz> getting everything from pypi again would be mildly annoying
<jtv> Well you _can_ just kill postgres instances and rm -rf db.
<jtv> If that didn't fix this problem, I don't think "make distclean" would.
<rbasak> mgz: https://code.launchpad.net/~racb/maas/arch-constraint-wildcard/+merge/128513
<jtv> Ohhhh this wasn't the weird problem involving transactional tests, was it?
<rbasak> mgz: tests pass. I haven't tried a juju bootstrap yet though. I'll need to create a package and test from that, which will take a while
<allenap> jtv: Sure, I'll take a look.
<jtv> Thanks.
<jtv> I'll shove some food into my face.
<mgz> rbasak: ace, will have a look
<mgz> gah, something weird is going on
<mgz> I'll throw away the instance I think
<jam> mgz: any luck with the search string is juju constraints?
<jam> jelmer: any luck with the retry code?
<mgz> rbasak: looks good
<jam> After those things, the next important thing is to do pagination on the 'nodes' listings.
<mgz> jam: I'm done, am just trying to get the test suite to actually run...
<jam> mgz, allenap, jelmer: I'm trying to tune the 'batch_size' for tag processing. Bumping it from 100 to 1000 means downloading 25MB per batch (for the XML) vs 2.5MB for the batch.
<jam> And speeds up the 12,000 unit case from 37s to 35s.
<mgz> ....that doesn't seem much
<jam> Trying to bump it to 10,000 increases the batch size to 250MB, which crashes on my 2GB VM
<mgz> ehehe
<jam> (I'm guessing there are multiple levels so that at peak we have >1 copy of the full text)
<jam> json.loads() isn't streaming, etc.
<mgz> ...some how my db is hosed in a way rming and destroying the vm does not fix...
<mgz> hm, trunk is fine, rebase and hope time
<jtv> mgz: we have observed some problems that seem to be affected by interactions between tests, and maybe test setup/teardown.  Did you add or remove any test cases?
<jtv> Thanks for the reviews allenap!
<allenap> Welcome.
<mgz> jtv: yes, new test class, and nose stopped admitting to the existence of the whole module
<mgz> (generally means something inside the module is raising ImportError, but I can't spot it)
<jtv> mgz: then I'd look in that direction for suspects. Still a mystery why this happens.  Does your test case do anything weird?  Like rely on transactions behaving realistically, or models that exist only within the tests?
<mgz> bin/py -c "import maasserver.tests.test_views_nodes" should tell me that much...
<mgz> or not, because of django settings... >_<
<jtv> I'll bet the problem goes away when you run your tests in isolation.
<mgz> gah, no -1 option? noooose...
<jtv> Man, using 4 kinds of pepper in one meal is awesome.  Two kinds as spices, two as vegetables.
<mgz> okay, so I have an import error in a different bit of the tree
<jtv> Is it a circular import or something like that?
<mgz> typoed 'node' as 'name' from models when importing into views
<mgz> so, shouldn't be circular (models don't care about views)
<mgz> but is related to the test module that was misbehaving
<mgz> ...implies some cleanup on db level isn't happening if test fails at too early a stage?
<mgz> starts db setup, tries to setup tests, gets error, cleanup doesn't run, or similar?
<jtv> We are completely in the dark.  Otherwise we'd probably have fixed it by now!
<jtv> Note how incredibly early it happens.
<mgz> okay, I'm all proposed up.
<rbasak> mgz: may I have a +1 for https://code.launchpad.net/~racb/maas/arch-constraint-wildcard/+merge/128513 or do you not think it OK to land yet? I'm thinking that if I get it in the daily then we can easily test tomorrow - otherwise I'm tight for time for today's EOD anyway
<allenap> jtv: Did you forget a prerequisite branch, or is the diff wrong on https://code.launchpad.net/~jtv/maas/maascli-cli-subcommand/+merge/128538?
<mgz> rbasak: just done, with some minor comments
<mgz> getting it in the daily sounds good.
<rbasak> mgz: thanks, I'll get those changes done
<jtv> allenap: let me have a look
<jtv> allenap: I jumped the gun on that one and assumed the diff would update.  I'll see if I can trigger a re-diff.
<allenap> jtv: Don't worry about it too much; I have generated it for myself.
<mgz> rbasak: one option I didn't mention was you could avoid the aliased_value stuff by doing that in the same mapping too
<jtv> allenap: too late :)
<mgz> ie, `architecture_wildcards['arm'] = architecture_wildcards['armhf']`
<mgz> or equivalent.
<rbasak> mgz: that's very tidy - thanks!
<mgz> moving as much of that logic out of the constrain_architecture function and into precalculated data I like
<allenap> jtv: What did you do to make it update by the way?
<jtv> allenap: I replaced some instances of "log-in" and "log-out" (used as verbs) with "log in" and "log out" in the api.py docstrings.
<allenap> jtv: Haha :)
<jtv> OCD-R-Us
<allenap> rvba: Would you be able to give a second opinion on https://code.launchpad.net/~jtv/maas/maascli-cli-subcommand/+merge/128538?
 * allenap has to go, urgently, to collect kids.
<rvba> allenap: sure.
<rvba> allenap: could you have a look at the branch I just put up for review when you get back: https://code.launchpad.net/~rvb/maas/bug-1062607/+merge/128543
<rvba> jtv: I'm having a look at the comment Gavin made on your maascli-cli-subcommand branchâ¦
<rvba> jtv: but the 'cli' subcommand is long gone am I right?
<jtv> No, it's brand new.
<rvba> arg, I mean s/cli/api/
<rvba> jtv: So, I don't really get why you're introducing this new subcommand.
<jtv> The api subcommand is long gone, but the next step is to remove the profile name as a subcommand.
<jtv> Did I say "api"?  Mistake.
<jtv> Right now you'd say something like "maascli mylogin acquire_node <node>"
<rvba> Indeed.
<jtv> and that should become, for the case where mylogin is the default profile:
<jtv> "maascli acquire_node <node>"
<rvba> Right, I get thatâ¦ what does it have to do with 'api' or 'cli' then?
<rvba> I'm just trying to understand what Gavin said in his comment.
<jtv> If we _also_ have commands like "maascli login profile URL", then they live in the same namespace as the methods of the API.
<jtv> Not a good idea: "list" could easily be used in both the CLI commands and the API, for instance.
<jtv> So the CLI commands (login, logout, refresh, list) move into a separate subcommand.
<jtv> That leaves the root namespace free for API commands.
<rvba> Ah ok, I see.
<rvba> I wasn't aware of the intention to move the CLI commands under 'cli'.
<roaksoax> rvba: weird, you run into that db_ bug
<jtv> rvba: sorry -- I've been trying to mention the big picture in all my MPs but it's hard not to start skipping things with so much repetition.
<rvba> jtv: I understand.  But I've been asked to give a second opinion so I needed to put all the pieces of the puzzle together. :)
<jtv> I hope that doesn't mean trouble for my branch!
<jtv> It's a bit of a difficult transition, trying to do everything in small chunks.
<rvba> I imagine :)
<flacoste> roaksoax: has ipmi setting in enlistment landed?
<rvba> I think Gavin is right though.  I'll summarize my thoughts on the MP.
<roaksoax> flacoste: not as of friday... today I didn't check (i'm not really here, US holidays...)
<flacoste> roaksoax: aren't you the one working on that?
<roaksoax> flacoste: allenap was the one who was working on enabling the power settings for enlistment
<rvba> flacoste: roaksoax I /think/ this has been landed.
<rvba> Let me check.
* flacoste changed the topic of #maas to: Final Freeze is tomorrow| Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | MAAS jenkins: https://jenkins.qa.ubuntu.com/job/maas-trunk/
<rvba> flacoste: roaksoax: confirmed, https://code.launchpad.net/~allenap/maas/anon-power-setting/+merge/128127 has landed.
<roaksoax> flacoste: landed today so tomorrow I'll take care of the enlistment IPMI temporary credentials then
<flacoste> roaksoax: ok, and we'll 0-day SRU any hiccups from that?
<roaksoax> flacoste: yeah
<flacoste> landing this tomorrow seems very risky since we won't have time to react to any problem
<flacoste> nor have matsubara do a proper round of  QA around the new changes
<roaksoax> Daviey: until when can we make uploads?
<roaksoax> to archive
<roaksoax> flacoste: right, so landing this will only make use of the script that does the detection during the commissioning, but does need support from maas-enlist
<flacoste> roaksoax: tomorrow 21UTC according to the ReleaseSchedule wiki page iuc
<roaksoax> flacoste: right, so i'm wondering now, what's the latest upstream release that we are going to actually release to archives?
<roaksoax> flacoste: i made an upload over the weekend that Daviey will process
<roaksoax> flacoste: but I'm updating that (same upstream release though) with packaging fixes
<flacoste> roaksoax: there have a couple of bug fixes in progress that should be released
<flacoste> roaksoax: so we should plan on a new upstream tomorrow
<roaksoax> flacoste: ok, right, so since new upstreams are mostly bug fixes, then we should be good
<rvba> jtv: do you have time to review https://code.launchpad.net/~rvb/maas/bug-1062607/+merge/128543 ?
<jtv> I think I can, yes
<rvba> Thanks a lot.
<jtv> So it was Apache after all?
<Daviey> roaksoax: Final Freeze kicks in tomorrow
<Daviey> And this large upload certainly needs to be in by then
<Daviey> After that, we can sneak in some last minute fixes.
<roaksoax> Daviey: right, which means with the latests upstream release
<Daviey> right
<rvba> jtv: yes
<roaksoax> Daviey: ok so instead of uploading today, I guess I'll make an upload tomorrow
<roaksoax> Daviey: and the enlistment stuff... takes more than just add the support to MAAS
<roaksoax> Daviey: cause needs work on maas-enlist as well
<roaksoax> rvba: https://code.launchpad.net/~andreserl/maas/packaging_updates_bzr1099-2/+merge/128551
<roaksoax> please: )
<Daviey> roaksoax: erm, getting trunk onto tomorows ISO i would consider vital
<Daviey> so an upload today and another tomorrow
<roaksoax> Daviey: alright!
<roaksoax> Daviey: today's trunk on today's upload?
<rvba> Daviey: roaksoax don't you think it's pointless to make an upload before we fix: bug 1063857 ?
<ubot5> Launchpad bug 1063857 in maas (Ubuntu) "Cluster controller fails to start because MAAS_URL is not set." [Critical,Triaged] https://launchpad.net/bugs/1063857
<roaksoax> rvba: that's fixed already
<roaksoax> rvba: see above link  i sent you :)
<rvba> roaksoax: it that fixing it?  Ok then :)
<roaksoax> rbasak: i take that the branch you proposed for bug #1062340 takes care the i386 amd64 cases too?
<ubot5> Launchpad bug 1062340 in MAAS "juju provider interacts poorly with arch constraints" [High,In progress] https://launchpad.net/bugs/1062340
<Daviey> rvba: TBH, we keep saying that.. and the upload is embarrassingly large already
<Daviey> heck, it still uses ipmitool
<rbasak> roaksoax: I believe so. Could you clarify exactly what you mean? That "amd64" should work?
<roaksoax> rbasak: i mean, it completely replaces my branch and if juju sends amd64 it gets mapped correctly and won't fail to acquire node?
<rbasak> roaksoax: I believe so, yes
<roaksoax> cool
<rvba> jtv: I just posted a reply to your review on https://code.launchpad.net/~rvb/maas/bug-1062607/+merge/128543
<jtv> coming
<jtv> Replied.  I'll do a bit of rebooting to see if that'll bring back my window manager.  I've been working without one since before midnight.
<rvba> jtv: that's funny, my window manager disappeared a few hours ago :)
<jtv> Question is: who has them?
<roaksoax> rvba: around?
<roaksoax> rvba: can you test the package i just uploaded to experimental
<rvba> roaksoax: sure
<rvba> roaksoax: the problem is still there :/ http://paste.ubuntu.com/1267881/
<roaksoax> rvba: sudo dpkg-reconfigure maas-cluster-controller ?
<roaksoax> MAAS_URL=http://192.168.2.110/MAAS
<roaksoax> that;s me
<rvba> Yeah, now /etc/maas/maas_cluster.conf is populatedâ¦
<rvba> roaksoax: but now I don't see the cluster workerâ¦ even if I restart maas-cluster-celery.
<roaksoax> rvba: weird, I have  arunning cluster
<rvba> roaksoax: can you pastebin 'ps aux | grep celery' ?
<roaksoax> ubuntu@node2:~$ sudo service maas-cluster-celery status
<roaksoax> maas-cluster-celery start/running, process 8009
<roaksoax> https://pastebin.canonical.com/76065/
<roaksoax> err
<roaksoax> rvba: did you deploy on a new cloud instance?
<rvba> roaksoax: yes
<rvba> roaksoax: rarg, my fault, I made a mistake when I entered the url :/
<rvba> roaksoax: Now it's fine
<roaksoax> rvba: right, but initial autodetection is not working
<rvba> roaksoax: indeed.
<roaksoax> rvba: idk what bigjools might have changed
<roaksoax> if any
<roaksoax> rvba: i know what it is
<rvba> ?
<roaksoax> rvba: it doesn't ask for the dbconfig question if it couldn't be autodetected
<roaksoax> rvba: but then it replaces it from loclhost to nothing
<roaksoax> rvba: https://code.launchpad.net/~andreserl/maas/packaging_update_cluster/+merge/128564
<roaksoax> now it will be left as localhost/MAAS
<roaksoax> rvba: the problme is that maas-region-controller installs after maas-cluster-controller, causing that the latter cannot autodetect the URL
<rvba> roaksoax: can we control the order in which the packages are installed?
<roaksoax> rvba: not really no
<rvba> roaksoax: I've approved the branch.
<roaksoax> rvba: what could be done for such case is allow the region-controller set that value
<rvba> roaksoax: but that would mean having one package change the config of another one right?  Isn't it against the rules?
<roaksoax> rvba: it is indeed., tjhat
<roaksoax> that's where i was going next
<roaksoax> Daviey: uploading now
<jtv> smoser: having a look at bug 1050033... the old Precise version of maas-import-ephemerals has installed initrd files.  There is no .gz suffix, but I think they were already gzip-compressed.  Is that normal?  Can I assume that an initrd file will already be gzipped, so that I can just link or rename?
<ubot5> Launchpad bug 1050033 in MAAS "maas-import-pxe-files has copying errors after package upgrade" [Critical,Triaged] https://launchpad.net/bugs/1050033
<jtv> (I think gzip will do the right thing anyway if I try to compress an already-compressed file anyway)
<rvba> roaksoax: is that normal that debian/changelog contains "maas (0.1+bzr1223+dfsg-0ubuntu1~ppa1) quantal; urgency=low"â¦ I mean the 'ppa' part?
<roaksoax> rvba wheres that
<roaksoax> jtv: id you see the comment i made to your pscksging MPÂ¿
<Daviey> roaksoax: erm, this indicates 3 different uploads?
<Daviey> can you fold debian/changelog to be accurate for the ubuntu archive?
<roaksoax> Daviey it is just one upload
<Daviey> roaksoax: the changelog shows 0.1+bzr1223+dfsg-0ubuntu1, 0.1+bzr1110+dfsg-0ubuntu1, 0.1+bzr1063+dfsg-0ubuntu1,
<Daviey> two of them were never uploaded to quantal?
<roaksoax> Daviey: yeah I know what's wrong
<Daviey> PPA vs Archive changelog?
<roaksoax> Daviey: yeah, bigjools made uploades and treated as releases in the packaging branch
<roaksoax> uploads to PPA
<Daviey> ahh
<Daviey> blame bigjools then :)
<roaksoax> Daviey: my fault for not telling him earlier that changelog entreies were for releases in quantla not in PPA :)
<Daviey> roaksoax: don't try to cover for him.. :D
<roaksoax> oh well, i should have been clearer though
<Daviey> heh
<roaksoax> Daviey: do you want me to fix it though?
<roaksoax> and merge all of the changelogs?
<Daviey> roaksoax: Up to you.. Personally, i'd prefer it.. as it is currently a lie, but not enough to reject
<roaksoax> Daviey: alright fixing it then
<roaksoax> Daviey: i'm ready to upload
<roaksoax> Daviey: could you please reject the other one?
<Daviey> ok
<roaksoax> Daviey: ok, new version uploaded it's all yours :)
<bigjools> morning
<bigjools> roaksoax: around?
<roaksoax> bigjools: barely.. in class.. what's up?
<bigjools> roaksoax: just wanted to check in with you and see how things are going
<roaksoax> bigjools: maas is in quantal... tomorrow we'll be making a new upload :)
<bigjools> roaksoax: did anyone test upgrades from precise?
<roaksoax> bigjools: not from the new packages (there's no way to do it as it disables PPA's). That's something that has to be done now
<bigjools> there's always a way ;)
<bigjools> ok, I guess I'll be firing up some canonistack instances today
<roaksoax> bigjools: i tested from precise -> older quantal -> newer quantal so I'm pretty confidence it will wokr
<roaksoax> bigjools: and yes I was planning to do the same after I come back home
<bigjools> cool
<bigjools> I will look at some things I know we had to hack hard on to make work for upgrades
<roaksoax> bigjools: the cluster things will still be a problem
<bigjools> in what way
<bigjools> ?
<roaksoax> bigjools: the question is not being asked being medium priority
<roaksoax> rvba reported that today and I fixed a couple bugs there
<roaksoax> bigjools: http://bazaar.launchpad.net/~maas-maintainers/maas/packaging/revision/121 http://bazaar.launchpad.net/~maas-maintainers/maas/packaging/revision/120
<bigjools> ok
<roaksoax> bigjools: the problme is that in some cases it seems that it gets installed before maas-region, hence it fails
<roaksoax> bigjools: i think however, that in the installer i could override that questions
<roaksoax> that question
<bigjools> roaksoax: hmmm the config file should be on disk from the existing maas package though
<bigjools> hence no question should ever be asked
<roaksoax> bigjools: right, but it was resulting on the maas_cluster.conf having URL=
<bigjools> that can only mean that the existing config was broken then
<bigjools> oh this is the db_get cockup
<roaksoax> bigjools: yeah, so db_get was returning empty
<bigjools> so, the question should still not get asked
<bigjools> and that logic was fixed I see, so... where's the problem? :)
<roaksoax> bigjools: that the question should be asked on machines that do not have the region-controller
<bigjools> roaksoax: that is perfectly fine
<bigjools> the package is never auto installed
<roaksoax> alright
<bigjools> only on upgrades from maas
<bigjools> where the config is guaranteed to exist
<roaksoax> alright
<bigjools> is that ok? makes sense?
#maas 2012-10-09
<flacoste> bigjools: i'm in the hang out
<bigjools> flacoste: hi - I don't see an invitation
<bigjools> now I do
<flacoste> bigjools: it was the hang out linked to the event
<roaksoax> bigjools: right, but my point was that when you only install maas-cluster-controller you need to tell the MAAS URL too right?
<bigjools> hi jam
<jam> hi bigjools
<jam> (I was around earlier, but had to run some errands) anything you need?
<bigjools> jam: no, just saying hi :)
<bigjools> well cool, commissioning a new and I see hardware info appearing :)
<bigjools> new node*{
<jam> bigjools: yay
<bigjools> sadly the ipmi detection is not so good, it thinks the BMC has an ip address of  0.0.0.0
<jam> bigjools: that just means it is accessible from everywhere, right? :)
<bigjools> jam: that's *one* way of looking at it
<allenap> Morning.
<rvba> Morning allenap.
<jam> bigjools: I think I might know how we've had test suite failures in Jenkins.
<jam> It appears that if you submit a branch, and it fails the test suite
<jam> Submitting it again just accepts it (without running the tests)
<bigjools> !
<bigjools> I have done that in the past and it didn't land though
<jam> I could be wrong, but I just resubmitted to MAAS lander, and I think I can show it failing here.
<jam> bigjools: https://code.launchpad.net/~jameinel/maas/allow-json/+merge/128512
<jam> the first failure gave me the 404
<jam> so I just resubmit hoping to get an actual error I can work on.
<jam> But this time, magic it succeeded.
<bigjools> hummm
<jam> mgz: http://stackoverflow.com/questions/5907575/how-do-i-use-pagination-with-django-class-based-generic-listviews
<allenap> jam: Gah, can't import django in provisioningserver :-/
<jam> allenap: I have a fix re-submitted for it.
<jam> but see above
<allenap> I nearly added "because Django is an nearly unmitigated pos", but I bit my tongue.
<jam> Maas decided to land my branch again, when I retried because the first time gave me just a 404
<mgz> jam: I have fled upstairs, can mumble from here if we want though
<jam> mgz: well, I'll be chewing for a bit longer, then I'll poke you for mumble :)
<mgz> ^and I followed your bad example and told the lander to shut up and try again
<bigjools> allenap: lol
<jam> allenap: so if you check my latest 'lp:~jameinel/maas/allow-json' I should have fixed the import issues. However, Maas just rejected my change again, and I can't get the context because 404
<allenap> Gah.
<jam> I'm running a full make check to be sure
<jam> I can just land on trunk, and see if that resolves things?
<jam> (and why is it that console output used to work, but is now just always giving 404...)
<allenap> This is annoying. I guess land on trunk. rvba, do you know if/how we can turn off the gated landings in Tarmac, for a while at least?
<rvba> allenap: no idea, maybe Diogo documented that somewhere but I doubt it.
<allenap> We just walk round the back then.
<jam> allenap: it is possible that it was passing for me because I didn't have the latest trunk. I'll make sure to grab it and check one more time.
<allenap> jam: I'm running it too.
<bigjools> it'll be in the tarmac conf
<rvba> jam: the "Private Jenkins URL" works.
 * bigjools putting kids to bed, back in 20
<jam> rvba: I can get to the public jenkins, it just doesn't have the new job (172) that claims to be failing.
<jam> And I can't get to the private jenkins
<jam> allenap: I do see 2 failures in trunk this time, so maybe I can squash them.
<allenap> jam: You need the test lab VPN config to get to the private URL, see https://wiki.canonical.com/Launchpad/QA/MAASLenovoLab
<rvba> jam: like Gavin said :)
<jam> allenap, rvba: Technically, vpn's are illegal in Dubai, because they get around the content restrictions. but I'll probably give it a shot.
<jam> allenap: so the tests failing in trunk are the 'maascli' ones.
<jam> not mine.
<jam> so I'll land my patch to fix my own broken bits.
<jam> and then we can look to fix the maascli ones.
<mgz> don't get yourself hauled off to sit in content prison...
<jam> allenap: 'bin/test.maas maascli.tests.test_integration' has 2 failing tests for me.
<jam> mgz: is that 'conTENT' or 'CONtent' ?
<jam> :)
<rvba> jam: can you try moving away ~/.maascli.db and run the tests?
<rvba> jam: we have a bug about the tests using that fileâ¦
<jam> allenap: the maascli tests are failing because of 'resources=description['resources']' not existing.
<jam> rbasak: confirmed
<jam> Looks like it might be happening in Jenkins as  well?
<rvba> jam: yeah, that's bug 1063721.
<ubot5> Launchpad bug 1063721 in MAAS "The CLI tests depend on the content of ~/.maascli.db." [Low,Triaged] https://launchpad.net/bugs/1063721
<allenap> jam: Yeah, the ~/.maascli.db is probably causing that. Also, maascli tests should be run with bin/test.maascli.
<jam> allenap: they still fail, but I see them being run with both
<jam> as in running 'make check' they run under bin/test.maas from what I can tell, and they fail here.
<jam> allenap: if they shouldn't run under 'bin/test.maas' I think you need an --exclude=maascli in bin/test.maas
<jam> (which comes from buildout somehow)
<allenap> jam: I'll fix that later.
<allenap> jam: I get a clean run here, for make check with allow-json.
<jam> allenap: yeah, other than the maascli stuff, I have a clean run. So I landed it on trunk.
<jam> mgz: poke for great mumbling
<jam> allenap: maas lander is still failing follow up branches without any context
<mgz> third time lucky
<mgz> maas lander let the search change through
<jam> mgz: https://jenkins.qa.ubuntu.com/job/maas-merger-quantal/ only shows up to 169, not 170-175 which are the ones that are failing.
<jam> mgz: make harness
<jam> from maasserver.models import NodeGroup
<jam> ng = NodeGroup.objects.ensure_master()
<jam> from maasserver.testing.factory import factory
<jam> for i in range(?): factory.make_node(nodegroup
<jam> =ng)
<jam> allenap: I'm getting "twistd.pserv Unknown command: maas-pserv" any idea how to fix that?
<allenap> jam: What are you trying to do?
<jam> allenap: 'make run'
<jam> however, I think it is because I still had cruft leftover from failing to uninstall maas
<jam> So maas-txlongpoll was running in the background
<jam> rather than just the dev one.
<allenap> jam: Have you moved the directory or anything that might confuse buildout? Check bin/twistd.pserv to see if there's anything obviously wrong there.
<jam> allenap: still getting the same error after stopping the other.
<jam> allenap: Also, this is after doing 'make distclean' which should rebuild those things, right?
<allenap> jam: Yeah, it ought to. The maas-pserv plugin is picked up from twisted/plugins/maasps.py, so make sure that the root of the tree is mentioned in bin/twisted.pserv.
<allenap> s/twisted/twistd/
<jam> allenap: it is.
<allenap> Bugger.
<jam> root, etc, src, contrib/python-tx-tftp
<jam> allenap: so the actual failure is probably 'no module named tftp.backend'
<jam> the massps.py is suppressing the other import failure
<allenap> jam: That should be in contrib/python-tx-tftp.
<jam> which is in the twistd.pserv path
<allenap> jam: Can you run through twistd.pserv statement-by-statement and then try importing tftp.backend?
<jam> allenap: if I insert a PDB at the __name__ == '__main__' portion, I can manually import twisted.plugins.maasps and see that it has the ProvisioningServerMaker class in the namespace
<jam> allenap: however it still tells me 'unknown command maas-pserv'
<jam> availabe commands are ampq-longpoll, conch, dns, ftp inetd, mail, manhole, news, portforward, procmon, socks, telnet, tftp, txlongpoll, web, words, xmpp-router.
<jam> I *do* see 171 unhandled messages in the 'celery' queue... :(
<allenap> jam: I can't think of another reason why it wouldn't work, but that's obviously a failure of my imagination. I'll keep thinking. If you discover the reason, let me know?
<jtv> Anybody up for a review of some shell code?  It's not much: https://code.launchpad.net/~jtv/maas/bug-1050033/+merge/128685
<bigjools> jtv: sure
<jtv> Thanks.
<bigjools> jtv: just one point - the args are the wrong way around compared to the more traditional order :)
<jtv> Depends on the example you pick!  Unix tradition is: if one positional argument takes just one value, and another may be given multiple times, the one argument goes first.
<jtv> Insert long discussion here about whether ln's argument order is intuitive.  :)
<jtv> But yes, cp and mv do put the destination last.
<jtv> I'll change it.
<rbasak> Most of the time I use ln, I specify the destination only. It'd be confusing if ln were the other way round
<bigjools> depends on how you think about which is the destination
<jtv> Exactly.  That's what I hate about it.
<jtv> Now you don't just have to remember argument order, you have to remember which of several obvious ways to think about the whole thing!
<bigjools> I just remember original file :)
 * jtv wishes once again he'd pushed his openssh patch upstream half a decade ago, so he could stop it complaining about host keys for dynamic IP addresses changing all the time
<bigjools> the 99-temporary-fix-constraints.patch in packaging now needs removing
<bigjools> package is ftbfs
<allenap> mgz: Got time for a 1 minute review of https://code.launchpad.net/~allenap/maas/maas-cli-explain-after-login/+merge/128693?
<mgz> allenap: sure
<allenap> Ta.
<mgz> print() is weird
<allenap> mgz: Why?
<mgz> putting a bare print before and after is not how I'd do it, but I tend to the (equally weird) print s.join("\n\n")
<allenap> mgz: It's irrelevant here, because the target is Ubuntu only, but I avoid that so that line-endings are sorted out by Python, but in that I may be misguided.
<mgz> you're pretty much always printing to a file opened in text mode, so they get translated
<mgz> (redirecting stdout to something that has been opened as binary or hacking the stream properties is uncommon)
<smoser> jtv, you should not care about the extension of a initrd.  it never should have been changed.
<smoser> the 'gzip' executable is not at all involved in decompression of an initramfs.
<smoser> so, dont worry about it. rename it as you need.
<allenap> rvba, jtv: Anything in flight that needs to be landed today?
<rvba> allenap: no.
<jam> allenap: as near as I can tell, twisted/plugins/maasps.py is just not getting imported. Is there a way to tell what is going on? I guess just stepping from the startup?
<jam> mgz: poke
<flacoste> roaksoax: latest daily builds is uninstallable for smoser: http://paste.ubuntu.com/1269262/
<rvba> flacoste: for me too :/.  I just tested it on a branch new canonistack instance.
<rvba> brand*
<smoser> roaksoax, seems rooted in this: http://paste.ubuntu.com/1269276/
<rbasak> smoser: I have the bug reproduced and ready for a patch, based on my morning's daily (0.1+bzr1226+dfsg-0+1226+124~ppa0~quantal1) rather than current.
<jam> allenap: I think I found the problem... after uninstalling the package 'maas' it left a *stale* /usr/lib/python2.7/dist-packages/twisted/plugins/maasps.pyc file
<smoser> so, i would suggest that right now, archive install of freeipmi-tools is uninstallable
<Daviey> smoser: I did an upload earlier
<Daviey> Maybe it's still being published
<allenap> jam: Ach, that's bad. I guess that's a dpkg bug?
<Daviey> check which version is apt-cache policy
<jam> allenap: well, I had to manually mess things up because 'apt-get remove maas' was failing.
<jam> it was *starting* the services it was trying to remove.
<jam> and then, failing to remove them.
<jam> so I might have some things a bit broken here
<jam> however, after fixing *that*, I now have problems wiwth 'maasserver_config' does not exist.
<allenap> jam: Right. For now we assume it's an anomaly.
<allenap> Oh no :-/
<jam> that and 'response.getcode()' tuple object has no attribute 'getcode'
<roaksoax> smoser msybe relsted to being promoted to msinÂ¿
<roaksoax> smoser doko promoted it this morning
<jam> ah, I need to 'make syncdb' to get the db, right?
<roaksoax> smoser is the bug relsted to not being able to commission with console=ttyS0 still presentÂ¿
<roaksoax> ?
<jam> allenap: and it would appear I might have broken start_cluster_controller by changing the return value of MAASClient.post() checking
<smoser> roaksoax, probably.
<smoser> https://bugs.launchpad.net/maas/+bug/1061977
<ubot5> Ubuntu bug 1061977 in MAAS "Machine fails to commission when console=ttyS0 is present on kernel opts" [Critical,Triaged]
<smoser> i'm not *totally* opposed to dropping console=ttyS0 from the cmdline
<roaksoax> smoser ok.. so we still need to be able to modify that then in an easy manner
<smoser> as i have no other work around.
<mgz> jam: wiggle?
<jam> mgz: just checking how things are going, want to mumble?
<mgz> sure
<roaksoax> smoser i think we should be able to have an attribute on the node for that
<roaksoax> smoser shouldnt be that hard to hack
<bigjools> roaksoax: yes my machines failed to commission with console=ttyS0 today
<bigjools> also see the comment above about stale .pyc files left around?
<smoser> roaksoax, well, the one thing we can fix right now is to remove console=ttyS0.
<smoser> and i'm not completely opposed to that.
<smoser> i just think it sucks.
<roaksoax> smoser i think it is important to make it configurable
<bigjools> how does a bmc get at console output?
<roaksoax> adding a paramtet shold be a problem really
<smoser> it all depends.
<smoser> (how a bmc gets console output)
<smoser> some bmc will have serial console "SOL"
<roaksoax> bidjools you have to configure the bmc and then configure maas to tell what parameters to send
<smoser> which i would think is most common
<bigjools> also my BMCs are all failing to get a dhcp address, no idea why.  they ignore the dhcpoffer from the server
<jam> allenap: yay, it wasn't me who broke start_cluster_controller. Someone else changed the dispatcher from using urllib2.urlopen to httplib2.Http()
<smoser> but i really have not enough experience on server hardware to make broad generalizations on such a thing.
<jam> allenap: start_cluster_controller seems to be the only thing that is using 'response' directly after post
<roaksoax> bigjools those pyc files .. idk but its definiteky related to the package split as they werent present before the split but i cant figure out whats wrong
<bigjools> roaksoax: bug in postrm not pulling in debhelper?
<roaksoax> smoser in sabdfls cluster i had to odify to ttyS1 but he said he configured that on the bios
<smoser> so...
<roaksoax> bigjools i thought you merged that and it would have taken cre of it
<smoser> my personal feeling on this right now is to just drop all 'console=' params from the cmdline on intel
<smoser> unless bigjools or someone says "i can make that configurable via file in /etc/ "right now"'
<bigjools> there's an open bug about it
<jam> allenap: so your change to using httplib2.Http so that you could set the insecure flag
<roaksoax> i would go for addint the attribute for the node for custom kernel params
<jam> broke actually all users of dispatch_query
<smoser> so unless that happens, i think we live with "configurable" meaning 'change .py files'
<jam> because it used to return a urllib2 file-like object
<smoser> bigjools, yes, there is.
<jam> and it now returns a tuple
<jam> of response_headers, conte
<jam> content
<smoser> bug 1044503
<jam> we don't have tests that exercise the code
<ubot5> Launchpad bug 1044503 in MAAS "kernel command line is not easily customizable" [High,Triaged] https://launchpad.net/bugs/1044503
<jam> because all the tests mock out the connection.
<jam> because we never run an actuall HTTP server to query against.
<jam> all of the tags stuff would also show up broken, if we could actually get it running (but we are failing early during startup)
<bigjools> except in the test he added for that change... the irony ...
<jam> bigjools: :)
<bigjools> ok it's past midnight, I need to sleep
<bigjools> night all, good luck
<jam> bigjools: sleep well.
 * bigjools intends to :)
<rvba> jam: how could that change break things?  It's just a tiny encapsulation, and http_request still returns the result of http.request()â¦?
<jam> rvba: we used to return urllib2.open()
<jam> whicgh is a file-like object
<jam> with an attribute '.getcode()'
<jam> rvba: httplib2.Http.request() returns a very different object.
<jam> namely, a tuple
<rvba> I see.
<jam> of ({headers}, content)
<jam> rvba: it only shows up during 'make run' because that is the only time we actually connect to a real server.
<jam> rvba: so, do we try to update all clients to a new api? try to wrap it in something that looks like a urllib object?
<jam> revert the insecure=False change?
<rvba> I don't think we can revert that change just like that.
<allenap> jam: Revert I think.
<allenap> I only just made the change this morning.
<allenap> I'll do it.
<rvba> Also, sorry for being thick, but here is the change in question
<rvba> http://paste.ubuntu.com/1269307/
<jam> allenap: thanks
<rvba> We were using httplib2.Http.request() all along.
<jam> rvba: wrong code
<jam> this is the one allenap did
<rvba> Oh, you're talking about a different change I guess.
<allenap> rvba: apiclient, not maascli.
<rvba> Ok,
<rvba> k
<jam> allenap: yeah, I think we can revert it for now, and land it after tomorrow and think about what we want to do with the changes. Long term, you like httplib2 more?
<jam> (it at least lets you allow self-signed certs, I guess)
<allenap> jam: I was trying to "upstream" the changes in maascli to apiclient so that the former can properly use the latter. There's an impedance mismatch between apiclient and our Pistonesque API that's been mostly figured out in maascli.
<jam> allenap: yeah, I think it would be nice to update to something closer
<jam> and I don't think we need to have incremental reads from urllib2
<jam> (i don't know if we even have incremental there, but it pretends we do)
<allenap> jam: Care to rubberstamp? https://code.launchpad.net/~allenap/maas/revert-change-to-httplib2-in-apiclient/+merge/128726
<jam> allenap: done
<allenap> Thanks.
<jam> allenap: I'm glad I updated to trunk to test stuff today.
<allenap> jam: Me too :)
* flacoste changed the topic of #maas to: Final Freeze is today: waiting on upstream fixes for bug 978127| Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | MAAS jenkins: https://jenkins.qa.ubuntu.com/job/maas-trunk/
<flacoste> roaksoax: any way i can be a bug supervisor for the maas package? so that i can set priority
<roaksoax> flacoste: being member of the bugsquad or being a core dev :)
<flacoste> pff
<flacoste> smoser, roaksoax, Daviey: i'm just going to use a tag to highlight the bug fixed in upstream that aren't in the archive
<flacoste> too much hassle to create tasks when i cannot set the priority on it
<allenap> jam: I've filed bug 1064437 about it.
<ubot5> Launchpad bug 1064437 in MAAS "The expectations of MAASDispatcher.dispatch_query are not tested" [High,Triaged] https://launchpad.net/bugs/1064437
<jam> allenap: thanks
<smoser> flacoste, fair enough
<roaksoax> flacoste: what trunk rev are we looking to have in the archives today?
<jam> mgz: Vq2fSMwMvLXFuGJJyp:fZE7P5qGvgPSudZ8Hh:ZNqct3Pm8pFGh8mT5DVy8Py8JCtg4NaG
<flacoste> roaksoax: 1238 is the latest, but we are missing a few bug fixes still
<flacoste> roaksoax: so something like 1241 probably
<roaksoax> flacoste: ok I think we have until 21:00 UTC so we still have a few hours, but i'd require to do some testing first
<flacoste> roaksoax: 1239 -> revert of allenap revision breaking start_maas_cluster
<roaksoax> i'll check back in 1 hr
<flacoste> 1240 -> smoser fix for oauth in maas-enlist
<flacoste> 1241 -> IPMI auto-config in maas-enlist (optional really)
<flacoste> roaksoax, Daviey, smoser: https://bugs.launchpad.net/maas/+bugs?field.tag=missing-in-quantal
<smoser> flacoste, thanks
<roaksoax> flacoste: 1241 i'll try to make it happen, but this needs support from maas-enlist too
<roaksoax> flacoste: so there might not be time to finish it
<flacoste> roaksoax: right, fwiw, the API changes are available
<roaksoax> flacoste: yes I saw.. I'll be working on it todayt
<roaksoax> allenap: ping
<flacoste> jam, allenap: the reversal in 1239 does that mean that self-signed certificate cannot work with the maas-cli anymore?
<allenap> flacoste: No, it means it wont' work for users of apiclient. maascli doesn't use apiclient; before reverting this was an attempt to "upstream" the remote code in maascli to apiclient.
<flacoste> allenap: got it
<roaksoax> allenap: what am I doing wrong?
<roaksoax> curl --data "op=new&mac_addresses=e4:11:5b:13:81:9d&hostname=&architecture=amd64&subarchitecture=generic&after_commissioning_action=0&power_type=ipmi&power_parameters={"power_address":"test", "power_user":"test", "power_pass":"test"}" http://localhost/MAAS/api/1.0/nodes/
<roaksoax> Failed to parse JSON power_parameters
<rvba> roaksoax: power_parameters='{"power_address":"test"}'
<rvba> Note the quotes.
<rvba> json.dumps({"power_address":"test"})
<rvba> '{"power_address": "test"}'
<rvba> You have to pass a json string.
<roaksoax> rvba: got it :)
<allenap> roaksoax: Try: curl --data "op=new&mac_addresses=e4:11:5b:13:81:9d&hostname=&architecture=amd64&subarchitecture=generic&after_commissioning_action=0&power_type=ipmi" --data-urlencode="power_parameters={"power_address":"test", "power_user":"test", "power_pass":"test"}" http://localhost/MAAS/api/1.0/nodes/
<roaksoax> it was the quotes
<mgz> so, it seems django can't do internal redirects...
<rvba> mgz: you mean redirects to an address created with reverse('view-name')?
<mgz> rvba: I was just going to make /favicon.ico serve the same result as /static/img/favicon.ico ...but that turned out to be non-trivial
<mgz> I'd just use apache directives ideally, but don't want to upset django
<rvba> mgz: why do you want to do that?  Also, you cannot assume that the homepage of MAAS is '/'.
<mgz> because currently requesting /favicon.ico is a 404 and it wouldn't hurt to serve it instead
<rvba> mgz: src/maasserver/templates/maasserver/base.html:    <link rel="shortcut icon" href="{{ STATIC_URL }}img/favicon.ico"/>
<mgz> browsers tend to request that regardless of html giving a different path
<allenap> roaksoax: Ah right, yes, the quotes. From looking at --trace-ascii output, http://pastebin.ubuntu.com/1269411/ looks to be the correct way to do this. (You could s/--data-urlencode/--form/g I think too).
<rvba> mgz: really?
<allenap> roaksoax: --data is a recipe for failures and/or data corruption, because it assumes that the value is already URL encoded.
<mgz> yes, what if you're not fetching an html page? (not a common case here, but if was easy to fix I'd just fix it)
<roaksoax> allenap: cool, I;ll test that
<rvba> mgz: right.  Looks very minor to meâ¦ but in production, we have the prefix /MAAS in front of all the urls, so if the browser requests /favicon.ico, it won't get anything back and there is not much we can do about it.
<rbasak> rvba: could we redirect /favicon.ico to /MAAS/favicon.ico in packaging perhaps? Just an idea.
<rbasak> Not sure what implications that would have though
<rvba> rbasak: and what if you install another service in /AnotherServiceâ¦ it will get a wrong favicon.
<rbasak> Yeah exactly
<rvba> I don't think we want to hijack /favicon.ico.
<mgz> why are we trying to support serving multiple different things from one hostname exactly?
<rbasak> It'd be nice to have it on servers that are dedicated to MAAS
<rbasak> (by default)
<rbasak> But the only way I can think of doing it cleanly in packaging is perhaps not the best thing to do for today
<rbasak> I'm thinking of a separate maas-favicon recommends that conflicts with other services in some way
<mgz> the other magic top level name has a url mapping at the django level, but I guess just sits harmlessly at /MAAS/robots.txt in deployment
<mgz> anyway, put up a branch with the fix I was actually aiming at
<smoser> https://code.launchpad.net/~smoser/maas/drop-ttyS0-kernel-opt/+merge/128747
<matsubara> rvba, ping
<rvba> matsubara: pong
<matsubara> rvba, so, I just tried maas - 0.1+bzr1226+dfsg-0+1229+124~ppa0~quantal1 on a pristine quantal vm and maas-pserv is not started
<allenap> mgz: Mark said something about having to use an ico for full browser compatibility. Obviously you're still generating an ico, but perhaps he meant something else by it?
<rvba> matsubara: anything in the upstart log?
<rvba> mgz: I was about to ask the same question :)
<matsubara> rvba, just a bunch of lines like this: Removing stale pidfile /run/maas-pserv.pid
<rvba> matsubara: I'm trying to install the package from the daily ppa right nowâ¦
<matsubara> rvba, and apt/term.log shows this: maas-pserv start/running, process 10156
<matsubara> so I think it started but for some reason it failed
<mgz> allenap: I interpret what he said as assuming he'd generated an ico containing png files, but if fact he didn't
<rvba> matsubara: pserv not started here :/
<allenap> mgz: Right, cool.
<matsubara> rvba, aha!
<mgz> the larger sizes won't stop old (pre-vista internet explorer (do we even care..)) from picking out the 16x16 bitmap
<matsubara> rvba, so on a upgrade it keeps running fine but not on a new install
<smoser> someone want to ack that for me? i really hate this change. but i think its a requirement at this point in time.
<matsubara> rvba, do you know of any other logs that might be relevant?
<rvba> matsubara: yesterday it was fine :/
<allenap> matsubara: You'll need at least bzr 1239 to fix the pserv not starting issue, sorry.
<matsubara> does your upstart log show anything?
<rvba> allenap: what was the issue again?
<matsubara> allenap, ah so you knew about it all along
<allenap> rvba, matsubara: bug 1064437
<ubot5> Launchpad bug 1064437 in MAAS "The expectations of MAASDispatcher.dispatch_query are not tested" [High,Triaged] https://launchpad.net/bugs/1064437
<rvba> mgz: what's the benefit of having all but one of the image as a *raw* png?
<allenap> matsubara: Ha, I only just saw you having problems.
<matsubara> hmm I can't build new packages. LP is failing to build packages in the daily builds PPA due to a bzr error
<matsubara> allenap, no worries, just kidding :-)
<allenap> Phew :)
<matsubara> thanks for the pointer
<mgz> rvba: the 'raw' is a little confusing, but, in short, size
<matsubara> allenap, where do you see logs for this?
<matsubara> s/this/this error/?
<mgz> png is a compressed image format, bmp is not
<mgz> ico was originally a sort of container for bitmaps
<allenap> matsubara: I don't; jam told me it was broken.
<mgz> with recent format revisions it can also be a container for pngs
<mgz> so, storing the larger images as 'raw' png rather than exploding them to bitmaps saves space
<rvba> allenap: in this bug, you say is breaks start_cluster_controller, but in the package I'm testing, the cluster worker is started ok.
<rvba> mgz: I see, I got confused by the world 'raw' indeed :)
<allenap> rvba: What rev are you on? It was only a problem for revision 1234 to 1238 inclusive.
<matsubara> rvba, allenap: I got the log line in apt/term.log saying maas-pserv was started but after the package install finishes, the process is not running. Maybe it's failing and reporting a success?
<rvba> bzr1226
<rvba> The revision from the package in the daily ppa.
<allenap> Ah, so that bug isn't the problem.
<allenap> For matsubara either.
<rvba> Yep, there is a real problem then.
<matsubara> I'm using: 0.1+bzr1226+dfsg-0+1229+124~ppa0~quantal1
<matsubara> allenap, btw, this two revision numbers are confusing. when you say 1234..1238 do you mean trunk right?
<rvba> matsubara: if I start the service manually it seems to be started all right.
<matsubara> rvba, yep
<rvba> matsubara:trying one more time on another instance.
<rvba> matsubara: this time it worker :/
<matsubara> !
<rvba> worked*
<matsubara> how weird
<rvba> /var/log/upstart/maas-pserv.log is still full of "Removing stale pidfile /run/maas-pserv.pid"
<matsubara> yep, just like mine
<matsubara> I'm trying again as well
<matsubara> rvba, how are you starting a new pristine env?
<rvba> I just start a new canonistack instance, add the daily ppa, and then install maas.
<matsubara> ok
<matsubara> allenap, rvba have you seen the last package build failure: https://launchpadlibrarian.net/119221274/buildlog.txt.gz ?
<allenap> matsubara: Yeah, trunk.
<matsubara> allenap, is this because of your change?
 * allenap looks
<allenap> matsubara: Nope, I haven't been near the files in 99-temporary-fix-constraints.patch. I think that this has probably been fixed in trunk, so the patch can be removed. rbasak, can you help confirm?
<smoser> Daviey, roaksoax it seems to me that adding maas-ipmi-autodetect to commissioning broke kvm (and possibly anything that doesn't have such a device)
<smoser> is there a /dev/ entry that we could expect to be present to know whether or not we should go looking ? or some way to tell freeipmi to time out ?
<mgz> right, seems that patch just needs removing from the packaging branch
<smoser> it does seem to timeout eventually, but i'd say its in the 2 minute time frame.
<matsubara> does anyone know how I could install maas without having to answer the package installation questions? I tried to use a debconf seed file (maas.seed: https://pastebin.canonical.com/76135/) but it doesn't seem to work. Well, I get no questions asked but the default-maas-url value is not set to the one in the seed file.
<smoser> and then on failure, it still tries to call maas-signal with the power params.
<matsubara> smoser, roaksoax: maybe one of you know ^?
<smoser> http://bazaar.launchpad.net/~smoser/maas/maas-pkg-test/view/head:/maas-ephemeral-test-quantal.txt
<smoser> matsubara, ^
<smoser> see line 90. set DEBIAN_FRONTEND=noninteractive and for good measure set stdin from /dev/null
<matsubara> smoser, ah thanks
<Daviey> smoser: we *need* to silently fail, i think
<matsubara> smoser, you also change the string in the seed passed to debconf?
<rvba> I think that the pserv service depends on something that is installed by the region cluster.
<smoser> matsubara, i dont follow
<smoser> Daviey, it should not attempt to call maas-signal if the probe for ipmi fails.
<smoser> so we need to handle that bettter
<smoser> but we also need to not add 2 minutes to enlistment if ipmi is not present
<matsubara> smoser, lines 82 to 85
<matsubara> smoser, ah no, that's just setting the seed file, correct?
<smoser> matsubara, i'm setting the answer to that question
<smoser> Daviey, is ipmi a device ?
<smoser> that i can look for in /dev ?
<matsubara> right, thanks smoser
<matsubara> I'll give it a try
<matsubara> rvba, tried again and pserv is down
<smoser> awesome. strace on bmc-config shows that it reads /etc/freeipmi//freeipmi.conf 1 byte at a time.
<rvba> matsubara:  if I manually remove all the region stuff, then pserv refuses to be restarted.
<roaksoax> smoser: ok I see where it dies
<rvba> /usr/bin/twistd -n --uid=maas --gid=maas --pidfile=/run/maas-pserv.pid --logfile=/tmp/zz maas-pserv --config-file=/etc/maas/pserv.yaml
<rvba> usr/bin/twistd: Unknown command: maas-pserv
<smoser> roaksoax, do you know what devices it wants?
<roaksoax> smoser: smoser FATAL: Error inserting ipmi_si (/lib/modules/3.2.0-31-generic/kernel/drivers/char/ipmi/ipmi_si.ko): No such device
<smoser> roaksoax, but that doesnt mean anything in and of itself
<smoser> the driver could fail to load because it was already loaded
<smoser> or because it was built into a kernel
<matsubara> rvba, what does the log say? I restarted my environment and got the same problem.
<roaksoax> smoser: "No such device" --> this caould be 2 things 1. ipmi-locate might be erroneously detecting IPMI and continues, and bc-config gets stuck somewhere
<roaksoax> or 2. that causes ipmi-locate to get stuck
<smoser> from strance, it looks like it wants /dev/ipmi0 or /dev/bmc but then it goes looking in /dev/mem also before sleeping
<roaksoax> smoser: but after a while, it continues
<rvba> matsubara: the logs says nothing.
<smoser> roaksoax, yes, it times out after like 2 minutes-ish
<roaksoax> smoser: yeah.. such a PITA
<smoser> if i knew what it was looking for in /dev/mem, then i'd just do a udevadm settle after module load
<smoser> and hten if no /dev/ipmi* or /dev/bmc* then i would skip that.
<roaksoax> Daviey: so I need to upload maas-enlist source and add a maas-commission binary package
<smoser> but i'm afraid it might be looking in /dev/mem for some ipmi device that does not have a device driver but is a /dev/mem interface
<matsubara> rvba, I'm building a new package and we can test allenap fix deals with the issue
<roaksoax> smoser: http://paste.ubuntu.com/1269561/
<roaksoax> smoser: ok, so that will return an output in the way of {"power_address"="X", "power_user"="Y", "power_pass"="Z"}
<roaksoax> smoser: but I need to pass that to maas-enlist between single quotes, how can I do that?
<smoser> you actuall do not have to.
<smoser> $ output=$(echo '{"power_address"="X", "power_user"="Y", "power_pass"="Z"}')
<smoser> $ sh -c 'echo "1=^$1$"' -- "$output"
<smoser> 1=^{"power_address"="X", "power_user"="Y", "power_pass"="Z"}$
<smoser> ie, get the output, and quote the variable with doble quotes and you're fine.
<smoser> roaksoax, http://paste.ubuntu.com/1269572/
<smoser> thats in a kvm node.
<smoser> so there is 10 minutes of builtin sleep by that program by default. meaning enlistment of anything without ipmi is delayed 10 minutes.
<roaksoax> smoser: right...  but it order for it to get there, ipmi-locate must have succeeded
<smoser> unlikely.
<smoser> there is no ipmi.
<roaksoax> so maybe it is getting stuck in ipmi-locate
<roaksoax> unles it "detects" it
<smoser> https://bugs.launchpad.net/maas/+bug/1064527
<ubot5> Ubuntu bug 1064527 in MAAS "ipmi detection takes 10 minutes if no ipmi present" [Undecided,New]
<roaksoax> smoser: in my case it didn't take 10 mins
<roaksoax> :)
<rvba> matsubara: I think allenap and I found the problem,
<smoser> roaksoax, http://paste.ubuntu.com/1269579/
<smoser> really? did you try in a kvm instance?
<matsubara> rvba, what is it?
<rvba> Problems* I should say.
<roaksoax> smoser: yes I tried the kvm instance, it took like 1 min or so
<smoser> i dont know tha ti believe you, roaksoax
<smoser> 10m0.X seconds for the whole command, that I straced elsewhere and watched to 'nanosleep' repeatedly
<smoser> is a very suspicious time
<rvba> matsubara: 2 things: a) the cluster controller misses some of the required dependencies (txamqp at least) b) it tries to write its log to /var/log/maas which does not exist before the region is also installed.
<smoser> but none the less.
<rvba> It works sometimes because (I guess) the region gets installed right after the cluster.
<rvba> So the dependencies are there and /var/log/maas is created.
<rvba> I'm guessing the upstart script retries to start the service a few times.
<rvba> I the installation of the region is quick enough, it manages to start the pserv service.
<matsubara> I see
<roaksoax> rvba: if you pressume is because the /var/log/maas dir is not create, please file a bug , assign it to me, and explain which package should also create the log dir
<rvba> roaksoax: that's one of the 2 problems indeed.
<rvba> roaksoax: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1064539
<ubot5> Ubuntu bug 1064539 in maas (Ubuntu) "The pserv service logs to /var/log/maas/pserv.log and that directory (/var/log/maas/) is not created but the cluster controller package" [Critical,Triaged]
<rvba> s/but/by/
<allenap> roaksoax: I have to go, but can I bring https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1064542 to your attention? I can't mark it critical, but I think it is.
<ubot5> Ubuntu bug 1064542 in maas (Ubuntu) "python-maas-provisioningserver is missing several dependencies" [Undecided,New]
<roaksoax> allenap: ack
<roaksoax> thanks
* flacoste changed the topic of #maas to: Final Freeze is today: waiting on upstream fixes for bug 978127; packaging fixes: bugs 1064542 and 1064539| Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | MAAS jenkins: https://jenkins.qa.ubuntu.com/job/maas-trunk/
<roaksoax> allenap: so other packages no longer depend in those then
<allenap> roaksoax: They may do, so don't change other packages yet. I'll try to check their deps later.
<roaksoax> alright thanks
<smoser> roaksoax, more data for you. https://bugs.launchpad.net/maas/+bug/1064527
<ubot5> Ubuntu bug 1064527 in MAAS "detect_ipmi needs improvement. detects non-existant device in nested kvm" [Undecided,New]
<rvba> matsubara: I've documented the results of our findings on bug 1064020.
<ubot5> Launchpad bug 1064437 in MAAS "duplicate for #1064020 The expectations of MAASDispatcher.dispatch_query are not tested" [High,Triaged] https://launchpad.net/bugs/1064437
<matsubara> rvba, thanks
<rvba> matsubara: but basically, it all boils down to 2 packaging bugs.
<rvba> Bugs that roaksoax will be fixing soon ;)
<roaksoax> smoser: err passing the json format is a PITA
<roaksoax> smoser: i can't get to pass the full json string needed
<smoser> show me what you're doing ?
<smoser> it should'nt be bad.
<roaksoax> smoser: this is maas-ipmi-autodetect which returns in json format: http://paste.ubuntu.com/1269631/
<roaksoax> smoser: this is maas-enlist = http://bazaar.launchpad.net/~andreserl/maas/test/revision/32
<roaksoax> smoser: which would require to pass --power-params '{"test": "test", "test2", "test2"}'
<roaksoax> smoser: but every way I do it, maas-enlist doesn't get the whole json string it just get partial lilke {"test": "test",
<matsubara> rvba, so, this has nothing to do with bug 1064020?
<ubot5> Launchpad bug 1064437 in MAAS "duplicate for #1064020 The expectations of MAASDispatcher.dispatch_query are not tested" [High,Triaged] https://launchpad.net/bugs/1064437
<smoser> roaksoax, what is calling maas-enlist ?
<matsubara> rvba, because I dupe 1064020 against that one
<matsubara> err
<roaksoax> smoser: shell script
<smoser> what is taking output of maas-ipmi-autodetect and feeding it to maas-enlist
<smoser> where is that. that is your issue.
<roaksoax> smoser: it should go on maas enlist metadata
<matsubara> oh, wait
<matsubara> I got confused by the bot
<rvba> matsubara: no, nothing to do with that one.
<matsubara> ok, I'll undupe and close 1064020 as invalid since it's reported on the package
<matsubara> rvba, question for you, when I install maas-dns, shouldn't it ask for the config information to put in /etc/bind/named.conf.options? After every install in the QA lab I have to run this: https://pastebin.canonical.com/76145/
<smoser> roaksoax, what do you have there. thats where your sissue is.
<rvba> matsubara: not really sure why you had to do that.  What is the 'forwarders' stuff all about?
<matsubara> rvba, that's the only way I got dns to work in the lab. I got that change from Julian.
<rvba> matsubara: hum, I don't know about that.  Could you please file a bug?  Let's see what Julian has to say about that.
<matsubara> rvba, will do
<roaksoax> smoser: ubuntu@node2:~$ ./ipmi.sh
<roaksoax> ^{"power_address": "192.168.2.111", "power_pass": "6o9TuzEvY9fQ", "power_user": "maas-commissioning"}$
<roaksoax> Failed to parse JSON power_paramete
<roaksoax> smoser: http://paste.ubuntu.com/1269688/
<smoser> what does sudo ./maas-ipmi-autodetect --enlist output
<smoser> sudo ./maas-ipmi-autodetect --enlist
<roaksoax> smoser: prints the json string we need {"test": "test", "test":"test"}
<smoser> that'd be fine
<smoser> just quote "$var" then
<smoser> ./maas-enlist --serverurl localhost --power-type ipmi --power-params "${var}"
<roaksoax> smoser: ahhhh that;s was it "${var}"
<roaksoax> thanks
<roaksoax> lol :)
<roaksoax> never ocurred to me
<roaksoax> smoser: i'm including maas-signal in maas-commission binary package if that's ok with you
<smoser> roaksoax, well we need a fix there.
<smoser> still for this oauth bit
<smoser> i'm ok with that, but i'm working on that now.
<roaksoax> smoser: alright!
<roaksoax> smoser: ideally i think this should probably end up in maas-utils package or similar
<roaksoax> smoser: within the maas source
<roaksoax> Daviey: ping
<roaksoax> Daviey: is it too late for me to introduce this new binary package as part of maas-enlist source?
<matsubara> smoser, I set the debconf database with my seed file but when I installed the package it overrode the value. Is there anything I need to do so apt-get uses the value I supply in the seed file?
<matsubara> smoser, I installed the package with sudo apt-get install -y maas maas-dhcp maas-dns
<matsubara> and my seed file has: maas-region-controller  maas/default-maas-url   string  192.168.21.5
<smoser> probably just that maas-cluster-controller is the package that prompts you
<smoser> maas-cluster-controller	maas-cluster-controller/maas-url	string	http://localhost/MAAS
<smoser> matsubara, ^
<matsubara> smoser, I normally run sudo dpkg-reconfigure maas-region-controller to change that
<matsubara> so, maas-region-controller maas-region-controller/default-maas-url string 192.168.21.5
<smoser> well that seems wrong.
<matsubara> ?
<matsubara> it's what the package tells me to run after the package install :-)
<smoser> well, whats above works for me.
<smoser> what you're using doesn't work for you.
<smoser> i'm not able to spend time on why at the moment.
<smoser> but i'd suggest using what works for me/
<matsubara> ok
<matsubara> sounds good to me. trying that. thanks!
<smoser> rbasak, https://code.launchpad.net/~smoser/maas/trunk.maas-signal-clockskew/+merge/128794
<smoser> you can give that a sniff. i'm guessing it works.
<smoser> it does not ever set the hardware clock though.
<rbasak> smoser: looking
<smoser> it works for me. output at http://paste.ubuntu.com/1269818/
<smoser> rbasak, it works for me, testing when i apply this patch
<smoser> http://paste.ubuntu.com/1269821/
<smoser> which basically breaks the clock
<rbasak> smoser: looks good. https://pastebin.canonical.com/76152/ has enlistment and then commissioning
<rbasak> smoser: seeing multiple corrections but I presume that's juts multiple calls to maas-signal
<rbasak> smoser: RTC doesn't change during commissioning. Trying the install step now to see if that fixes it, which I think it does.
<smoser> rbasak, right. since it does not ever actually set the clock, each invocation of maas-signal has to re-discover bad clock.
<smoser> rbasak, do you know where install gets its clock from?
<rbasak> smoser: ntp I presume
<smoser> ntp.ubuntu.com
<smoser> ?
<rbasak> I assume so
<rbasak> I don't actually know
<rbasak> smoser: looks like yes - it's preseeded
<rbasak> smoser: /usr/share/maas/preseeds/preseed_master
<roaksoax> smoser: if in misc_bucket i define:
<roaksoax> - &maas_ipmi_detect |
<roaksoax> - &maas_enlist |
<roaksoax> etc etc
<roaksoax> each is run as a script?
<smoser> rbasak, ok. for now i think we just ship this.
<smoser> but for later, i think we hav a "ntp" setting in maas. and feed that to ephemeral environment.
<smoser> if it was set to something other than 'disabled' then set it based on that remote clock.
<smoser> roaksoax, where are you seeing that/
<roaksoax> smoser: enlist_userdata, and i'm asking if that's the case
<smoser> ah. contrib/preseeds_v2/enlist_userdata
<smoser> right.
<smoser> see
<smoser> runcmd:
<smoser>  - [ sh, -c, *maas_enlist ]
<smoser> that references the text in maas_enlist
<smoser> so just add another entry in that list for maas_ipmi_detect
<rbasak> smoser: tested and I've approved the MP
<rbasak> smoser: thanks!
<roaksoax> smoser: ok cool thanks
<roaksoax> smoser: and that's what i was doing just watned to make sure :)
<flacoste> roaksoax: upstream 1241 should be good to upload
<flacoste> roaksoax: unless you want your changes in as well which mean we'd ship 1242
<flacoste> roaksoax: but we have 25 minutes before the freeze happens
<roaksoax> flacoste: isn't it at 21 UTC?
<flacoste> indeed!
<flacoste> that's 60 more minutes!
<roaksoax> flacoste: indeed :)
<roaksoax> flacoste: i'm half way done
<flacoste> triple the time!
<roaksoax> so lets cross fingers :)
<smoser> https://code.launchpad.net/~smoser/maas/trunk.maas-signal-clockskew/+merge/128794
<smoser> is ready, not piced up yet
* flacoste changed the topic of #maas to: Final Freeze is today: upstream RC revision: 1241; packaging fixes: bugs 1064542 and 1064539| Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | MAAS jenkins: https://jenkins.qa.ubuntu.com/job/maas-trunk/
<flacoste> smoser: i received a merged message already
<smoser> really? Approved by: 	Scott Moser 1 minute ago
<flacoste> weird
<flacoste> smoser: actually, it wasn't the oauth fix, but the console fix
<flacoste> 1241
<flacoste> 1242 will contain the oauth fix
* flacoste changed the topic of #maas to: Final Freeze is today: upstream RC revision: 1242; packaging fixes: bugs 1064542 and 1064539| Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | MAAS jenkins: https://jenkins.qa.ubuntu.com/job/maas-trunk/
<rbasak> smoser: I understand that it might work, but it's still hard to read as it isn't obviously correct!
<smoser> it is obviously correct
<smoser> as the demonstrating script shows.
<smoser> it may not be obvious to you.
<smoser> but it is to python
<smoser> and in cases where the 2 differ, i agree with python.
<smoser> why would:
<smoser>  except Exception as X
<smoser> have any different scoping rules than
<smoser>  Y = X
<smoser> that would be confusing.
<flacoste> roaksoax: 1242 is in
<roaksoax> flacoste: ok, i'm almost there with the enlistment
<roaksoax> flacoste: if it doesn'y work withing the next 20 mins
<roaksoax> i'll give up
<roaksoax> and prepare an upload
<flacoste> roaksoax: perfect, am i right in that you already fixed the two packaging bugs mentioned in the topic?
<roaksoax> flacoste: yep i blieve they are fixed now
<Daviey> roaksoax: it is not too late to do that, no
<roaksoax> Daviey: the maas-nelist thing?
<Daviey> roaksoax: can we do it soon tho :)
<Daviey> roaksoax: not too late to introduce a new bin deb
<roaksoax> Daviey: ok cool
<Daviey> roaksoax: ETA?
<ppetraki> so I get this backtrace when I add a  node via mac address: http://paste.ubuntu.com/1269952/
<roaksoax> Daviey: i made a maas-enlist upload fixing a bug
<roaksoax> Daviey: so you could reject that and Ill upload the other thing now
<Daviey> ok
<roaksoax> Daviey: iuploaded
<Daviey> roaksoax: so here is the thing... i don't think i rejected it
<Daviey> 20:45 -queuebot:#ubuntu-release- Unapproved: accepted maas-enlist [source] (quantal-release) [0.4+bzr32-0ubuntu1]
<Daviey> someone else accepted it
<roaksoax> Daviey: alright, I;ll just prepare a different changelog
<roaksoax> Daviey: could you please reject maas-enlist-0.4+bzr35
<roaksoax> Daviey: i'll upload a new one
<Daviey> rejected
<flacoste> smoser: is bug 1006966 still relevant?%
<ubot5> Launchpad bug 1006966 in cloud-init (Ubuntu) "maas mirror values are overwritten by cloud-init" [High,Triaged] https://launchpad.net/bugs/1006966
<roaksoax> Daviey: ok uploaded a new one
<flacoste> ppetraki: everybody is heads down trying to hit feature freeze
<flacoste> ppetraki: and that's with the MAAS version still using cobbler
<Daviey> in a relaxed and orderly manner, due to our smooth planning and quality control
<flacoste>  ppetraki: and that's with the MAAS version still using cobbler
<smoser> flacoste, its fixed. i'll update with comment how.
<ppetraki> flacoste, I had duplicate cobbler entries, once I deleted them, it added correctly
<flacoste> Daviey: :-)
<ppetraki> flacoste, thanks
<flacoste> smoser: similary, is bug 1028501 fixed?
<ubot5> Launchpad bug 1028501 in cloud-init (Ubuntu Precise) "cloud-init selects wrong mirrors for arm" [High,Triaged] https://launchpad.net/bugs/1028501
<roaksoax> smoser: http://paste.ubuntu.com/1269987/
<smoser> flacoste, i put comments in them. you put the maas in the right state.
<roaksoax> flacoste: ok so I think the enlist stuff is finished... however, we still need publishing of the newer maas-enlist recently uploaded for it to work
<flacoste> roaksoax: is that a problem? would it prevent a new upload?
<roaksoax> flacoste: so, it could get merged, and later we file a bug saying "autodetection on enlistment is not enabled"
<flacoste> and upload as part of the 0-day updates next week?
<Daviey> 0-day if essential, right?
<Daviey> we are not currently planning a 0-day, right? :)
<flacoste> we are not
<flacoste> just would be nice to send a couple bug fixes more to users :-)
<flacoste> and autodetection only working during commissioning is not that interesting
<flacoste> roaksoax: can you merge it and make it part of the upload, or will it be rejected because maas-enlist hasn't been published yet
<roaksoax> flacoste: no it wont
<flacoste> roaksoax: if it can be accepted, but really will only work once maas-enlist is published, i think it'd be fine
<roaksoax> flacoste: ok, i'm running one more test
<smoser> roaksoax, so we're copying that whole commissining script basically?
<smoser> that sucks. :-(
<smoser> (i dont have a better way, but it sucks)
<roaksoax> smoser: yes, and no
<roaksoax> smoser: Daviey just accepted maas-enlist
<smoser> fyi, http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image
<roaksoax> smoser: with the new binary package
<roaksoax> smoser: but to be safe i'm copying it
<smoser> that takes care of your "remove this section"
<roaksoax> smoser: ok so we should be able to drop it as SRU
<roaksoax> smoser: is that ok with you?
<roaksoax> smoser: we temporarily merge all the copying of the script, and then we drop it, once the package hits the archive and I can test it works
<flacoste> smoser: how is bug 1044503  different than bug 1045390 ?
<ubot5> Launchpad bug 1044503 in MAAS "kernel command line is not easily customizable" [High,Triaged] https://launchpad.net/bugs/1044503
<ubot5> Launchpad bug 1045390 in MAAS "There's no way to add kernel parameters to a system read" [High,Triaged] https://launchpad.net/bugs/1045390
<smoser> flacoste, probably not. roaksoax ^ you have thoughts ?
<flacoste> roaksoax: that question can wait until the upload is done :-)
<smoser> roaksoax, i think your plan is fine.
<smoser> just make sure you get the latest version of maas-signal
<roaksoax> smoser: i did
<roaksoax> ok this is almost here
<flacoste> smoser: is bug 1053190 fixed?
<ubot5> Launchpad bug 1053190 in MAAS "multiple NICs could cause cloud-init nonet timeout" [High,Triaged] https://launchpad.net/bugs/1053190
<smoser> k. i'm going afk for a few hours.
<roaksoax> smoser: i get this
<roaksoax> }sh: 191: --power-params: not found
<roaksoax> smoser: https://pastebin.canonical.com/76160/
<Daviey> matsubara: Hey, what is your QA plan from now?
<matsubara> Daviey, currently I'm working on deploying with juju in the QA lab (which has 7 nodes arm and amd64)
<roaksoax> smoser: any idea?
<smoser> whitespace after \
<roaksoax> ok cool
<roaksoax> smoser: https://code.launchpad.net/~andreserl/maas/enlistment_ipmi_autodiscovery/+merge/128819
<roaksoax> flacoste: https://code.launchpad.net/~andreserl/maas/enlistment_ipmi_autodiscovery/+merge/128819
<roaksoax> please review
<flacoste> smoser: does bug 1034116 still need changes in MAAS?
<ubot5> Error: Launchpad bug 1034116 could not be found
<flacoste> roaksoax: line 17: it should be commented out :-)
<Daviey> matsubara: cool.  Do you have plans to try a cd image?
<roaksoax> flacoste: done
<matsubara> Daviey, nope
<roaksoax> flacoste: crap something failed in another test I just run
<flacoste> roaksoax: why the all-caps variable name at line 147?
<roaksoax> flacoste: "globals"
<flacoste> roaksoax: but they aren't globals?
<roaksoax> flacoste: I know :) I didn't have the time to update that
<roaksoax> flacoste: that's why the quotes :)
<flacoste> roaksoax: looks ok to me, but i'm not a shell expert
<roaksoax> smoser: could you please?
<roaksoax> smoser: https://code.launchpad.net/~andreserl/maas/enlistment_ipmi_autodiscovery/+merge/128819
 * roaksoax waiting for it to land
<roaksoax> to package it up
<roaksoax> flacoste: so maas-lander is taking forever to land the branch
<roaksoax> that's the only thing i'm waiting on to get the package read
<roaksoax> Daviey: just uploaded MAAS could you please take care of it?
<Daviey> yes
<roaksoax> Daviey: thank you!!
<flacoste> thanks roaksoax, Daviey and smoser!
 * roaksoax can now rest in peace :)
<roaksoax> smoser the bug you came across in nested kvm... is this canonistack?
<Daviey> roaksoax: it is
<Daviey> roaksoax: As IS use ksplice to patch the running kernel... i think smoser wasn't confident it was a regression on it's own
<Daviey> Maybe I miss understood
<roaksoax> so that means that it could have been related to the fact that that particular hardware does have ipmi hence the instances detect it?
<Daviey> roaksoax: No, i don't think that is passed through.
<Daviey> It's only the cpu
<Daviey> If that IS passed through, it's a massive security concern
<roaksoax> indeed
<roaksoax> i will look into it
<Daviey> roaksoax: you see, https://launchpadlibrarian.net/96102740/qemu_1.0%2Bnoroms-0ubuntu7.debdiff .. It only exposes the VMX feature, as an addition
<roaksoax> righ
<Daviey> and on that note.. nn
<roaksoax> Daviey: night!!
<smoser> roaksoax, no.
<smoser> the issue is not t hat there actually is ipmi in play
<smoser> its just the way ipmi-detect works
<smoser> it looks like it randomly walks through memory
<smoser> and i think is getting false positives.
<smoser> it opens /dev/mem and looks for stuff.
<roaksoax> smoser: right
<smoser> i'm surprised that it was so bad, and i could have just been being unlucky.
<smoser> as other times it did return nothing
<roaksoax> heh right
<smoser> i wondered if we could just limit it to the /dev/ipmi* and /dev/bmc*
<smoser> and disable its searches for /dev/mem
<smoser> great work on bug 1032004, roaksoax
<ubot5> Launchpad bug 1032004 in Indicator Remindor "Indicator-remindor don't start on Ubuntu 12.04" [Critical,Fix released] https://launchpad.net/bugs/1032004
<smoser> err... 1042004
<smoser> bug 1042004
<ubot5> Launchpad bug 1042004 in MAAS "Enlistment does not do BMC discovery" [Critical,Fix released] https://launchpad.net/bugs/1042004
<roaksoax> thanks... it was pretty straightforward though :)
<roaksoax> smoser: ok, I got it
<roaksoax> smoser: sudo bmc-config --disable-auto-probe --checkout
<smoser> ?
<smoser> but is that somethign thats going to mean we do not support "in band" cards?
<smoser> maybe we should make that a default and allow it to be configed overridden.
<smoser> anyway.
<smoser> again, good job. i'm on my way out. probably check back in.
<roaksoax> smoser: i'm assuming that if it works, it means we have local access to the IPMI card right?
<smoser> i dont know what it means really.
<roaksoax> smoser: that it doesn't probe for several IPMI interfaces
<smoser> if it means "don't crawl through /dev/mem", that sounds good at first.
<smoser> but if 75% of the existing cards would be found that way, that does *not* sound good.
<smoser> i really dont have enough information to make even a guess
<smoser> later.
<roaksoax> later
<bigjools> hi guys
<roaksoax> bigjools: good morning sir
<bigjools> roaksoax: hey there! how are we doing?
<roaksoax> bigjools: good, maas is in archives
<bigjools> good news!
<roaksoax> indeed
<bigjools> roaksoax: I see enistment is poking things in the BMC now ...
<bigjools> enlistment*
<roaksoax> bigjools: it is indeed :)
<bigjools> roaksoax: not sure it's a great idea to be changing parameters on enlistment :/
<roaksoax> sudo apt-get dist-upgrade
<roaksoax> err
<roaksoax> bigjools: you mean power parameters?
<bigjools> yeah
<bigjools> err no
<bigjools> I mean, setting user/pass on the bmcx
<roaksoax> bigjools: oh it is just a temporary one
<bigjools> or does it add a new one?
<roaksoax> bigjools: commissioning overrides it
<bigjools> does the old one still work or does it blow that away?
<bigjools> at enlistment I mean
<roaksoax> bigjools: blown away
<bigjools> :(
<roaksoax> bigjools: so ipmi basically manages user as User3, User4, etc etc
<bigjools> this means accidental pxe boots destroy BMC creds
<roaksoax> bigjools: well during enlistment we create maas-commissioning, which is then replaced by 'maas' during commissioning
<roaksoax> bigjools: but that's only for User3
<bigjools> roaksoax: did you see my bug about the 0.0.0.0 address?
<roaksoax> bigjools: nope, bug #?
<bigjools> https://bugs.launchpad.net/maas/+bug/1064224
<ubot5> Ubuntu bug 1064224 in MAAS "IPMI detection ends up with power_address of 0.0.0.0" [Critical,Incomplete]
 * bigjools brb
<roaksoax> bigjools: uhmm was it originally set to Static? if so, and the script changed it to DHCP then it might be due to it didn't wait enough time
<roaksoax> otherwise, it seems to not be getting an IP address of time
<bigjools> roaksoax: no it was always DHCP - it recently stopped getting an address
<bigjools> and now I can't access it at all of course
<bigjools> the dhcp server does a DHCPOFFER and the BMC just keeps doing DHCPDISCOVER
<roaksoax> bigjools: are these the MicroServer robbie had?
<bigjools> yeah
<bigjools> was working fine last week :/
<roaksoax>  bigjools did you delete the admin/password?
<roaksoax> the user admin with password password?
<bigjools> the commissioning code may have, I don't know
<roaksoax> bigjools: nope, it doesn't
<bigjools> well that's good :)
<roaksoax> the commissioning code uses User3 while the MicroServer were configured on User2 i think
<bigjools> roaksoax: how does this help me make it get an IP? :)
<roaksoax> bigjools: you might wanna try doing this: sudo bmc-config --commit --key-pair "Lan_Conf:IP_Address_Source=Static"
<roaksoax> and then
<roaksoax> sudo bmc-config --commit --key-pair "Lan_Conf:IP_Address_Source=Use_DHCP"
<roaksoax> that should reset the card and should try to DHCP again
<roaksoax> bigjools: if the card is already DHCP the cscript wont mess with it
<bigjools> okay, let me try
<bigjools> it is dhcp for sure, I can see it discovering
#maas 2012-10-10
<roaksoax> bigjools: i've seen similar issues but seemed to be DHCP issues
<roaksoax> bigjools: for example, once it DHCP's once, it seems to lock in the IP address until you set static and then DHCP again
<roaksoax> or if you reset the power (like disconnected the servers from power source)
<bigjools> I did that
<roaksoax> weird
<bigjools> very
<bigjools> roaksoax: ok bmc-config just returns "Unable to get Number of Users"
<bigjools> presumably I need some modules/options?
<roaksoax> bigjools: yes
 * bigjools looks in commissioning_user_data
<roaksoax> bigjools: during commissioning, enable the options in the commission-user-data in /etc/maas
<bigjools> well I am in deployment mode
<bigjools> can't ssh to a commissioning box :)
<roaksoax> bigjools: modprobe ipmi_msghandler modprobe ipmi_devintf
<roaksoax> modprobe ipmi_si type=kcs ports=0xca2
<bigjools> sorted, ta
<bigjools> roaksoax: didn't help :(
<roaksoax> bigjools: rmmod ipmi_si
<bigjools> roaksoax: as in, the bmc-config worked
<roaksoax> then modprobe ipmi_si type=kcs ports=0xca2
<bigjools> but the dhcp is still borked
<roaksoax> uhmm that's pretty werird though
<bigjools> DHCPDISCOVER followed by DHCPOFFER
<bigjools> over and over
<bigjools> it's simply not accepting the offered address
<roaksoax> bigjools: that's pretty weird though
<bigjools> very
<jtv> bigjools: when you install maas from a package, do you get a /home/maas?
<bigjools> jtv: no
<bigjools> deliberately so
<roaksoax> smoser: around?
<roaksoax> bigjools: do you happen to know if anny changes were made to the power handling? i'm getting an error when trying to execute the ipmi template, which wasn't there before
<roaksoax> bigjools: nevermind found the issue
<roaksoax> bigjools: https://code.launchpad.net/~andreserl/maas/maas_enlist_ipmi_username/+merge/128857 please :)
<bigjools> roaksoax: looking
<roaksoax> bigjools: thanks
<bigjools> de nada
<roaksoax> dpu:)
<roaksoax> :)
<bigjools> aren't you up late?
<roaksoax> bigjools: yeah :)
<mgz> fix for bug 1064734 <http://pastebin.ubuntu.com/1270729/>
<ubot5> Launchpad bug 1064734 in MAAS "ERROR Invalid 'tags' constraint 'set(['test-tag'])': No such tag using maas-tags to deploy with juju" [High,Triaged] https://launchpad.net/bugs/1064734
<jtv> mgz: I think you meant the juju bug.
<jtv> rvba: the WSGIApplicationGroup setting from the article I found works, but what I wanted to ask you is: can you think of any problems it might cause us?
<jtv> Meanwhile, I'm getting a JS test failure in trunk.  :(
<mgz> jtv: that is a juju bug
<mgz> jtv: we probably want to use the daemon configuration for mod_wsgi longer term
<jtv> mgz: argh, misread the number!
<mgz> blame matsubara! :)
<rvba> jtv: I can't think of anything really.
<rvba> mgz: you mean WSGIDaemonProcess?  If so, we already use that.
<jtv> rvba: thanks, that gives me a bit more confidence than me bunging it in and saying "hey look, it seems to work now!"  :)
<rvba> jtv: from what I can see, the combination WSGIDaemonProcess/WSGIApplicationGroup seems to be wildly used to deploy Django applications.
<jtv> Wildly?  That sounds exciting.
<mgz> :)
<rvba> Sorry, widely :)
<rvba> haha :)
<jtv> :(
<jtv> Does anybody want to review the fix?  https://code.launchpad.net/~jtv/maas/bug-1064737/+merge/128891
<mgz> hm, had misremembered, thought daemon/appgroup was either/or, but the internet does claim using both as in your branch is the right thing
<mgz> jtv: approved.
<jtv> Thanks mgz
<rbasak> Is longpoll broken in the packaging?
<rbasak> I see "GET /MAAS/longpoll/?uuid=amq.gen-wyS-xRaw0j0Xo1_9NW_leV&sequence=4 HTTP/1.1" 404 1885
<rbasak> Come to think of it I don't think I've seen it working in a while
<rvba> rbasak: I don't think this has been reported.  I'll test it.
<rbasak> thanks
<jtv> And I just saw a JS test failure in trunk... could it be related?
<rbasak> I also filed bug 1064922 btw
<ubot5> Launchpad bug 1064922 in maas (Ubuntu) "Enlistment fails" [Undecided,New] https://launchpad.net/bugs/1064922
<rvba> The failure rbasak is seeing looks like server-side stuff.
<rbasak> Enlistment is completely broken in quantal until the maas-enlist precise SRU lands, AFAICT
<rbasak> (since by default we use precise ephemerals which use precise maas-enlist)
<rvba> rbasak: I'm seeing the problem with txlongpoll too.
<rvba> rbasak: I'll file a bug and investigate.
<rvba> rbasak: just in case you'd like txlongpoll to work, all you need to do is: sudo a2enmod proxy_http && sudo service apache2 restart.  The fix is up for review.
<rbasak> thanks rvba!
<rvba> roaksoax: hi, could you please have a look at https://code.launchpad.net/~rvb/maas/packaging.fix-longpoll/+merge/128904 ?
<jtv> I'll be off.  Good night everyone.
<mgz> okay, folded down from a new 20 line test case, to a rewrite of a test class... to a single added assertion
<mgz> really want this to be a minimal patch
<evilnick_> anyone know what the point of the maas-cli files command is? I.e., when would you use it?
<mgz> is simplejson already a dependency?
<mgz> or will that require a packaging change?
<_rvba> Daviey: can you please have a look at bug 1065062?  I think we want that fixed in 12.10.
<ubot5> Launchpad bug 1065062 in maas (Ubuntu) "/var/lib/maas/celerybeat-cluster-schedule cannot be created by the cluster controller." [Critical,Triaged] https://launchpad.net/bugs/1065062
<_rvba> flacoste: can you please have a look at bug 1065055?  I think we want that fixed in 12.10.
<ubot5> Launchpad bug 1065055 in MAAS "celeryconfig_cluster.py imports utility method from maas (import_settings)" [Critical,Triaged] https://launchpad.net/bugs/1065055
<mgz> evilnick_: if you didn't get an answer, it's probably not used passed debugging
<mgz> maas needs a files api because juju expects to be able to poke some strings into file storage, and the cli just exposes all the apis
<mgz> evilnick: ^ in case your underscore breaks hilight
<evilnick_> mgz: yeah, thx. I think there may be a few other commands that fall into the same category, so it is a bit of a waste me writing docs for them :)
<mgz> some basic "internal thingy used for X" is fine, rather than nice user focuses stuff
<mgz> that way a curious kitten doing --help knows to move on
<flacoste> mgz: my understanding is that simplejson is part of python 2.7 now
<flacoste> mgz: in the json module
<flacoste> mgz: but maybe not the fast C implementation...
<flacoste> rvba: indeed we do :-)
<mgz> flacoste: mine too, but under the name 'json'... and the reason for that change is that bob ippolito continued hacking upstream and did various perf improvements... that didn't then get back into the json module
<mgz> so, just doing s/import json/import simplejson/ will break unless the python-simplejson is also intalled
<Daviey> rvba: Oh golly
<mgz> ...something is pulling it in currently, but I'm not sure how concrete that dep is
<rvba> Daviey: care to have a look at bug 1065080 ?
<ubot5> Launchpad bug 1065080 in maas (Ubuntu) "The host in BROKER_URL is hardcoded to 'localhost'." [Critical,Triaged] https://launchpad.net/bugs/1065080
<Daviey> anything for a giggle
<Daviey> done
<rvba> Daviey: medium really?
<rvba> Looks pretty critical to me.
<Daviey> rvba: https://wiki.ubuntu.com/Bugs/Importance
<Daviey> TBH, we rarely use critical
<rvba> Daviey: ah ok. "Renders essential features or functionality of the application or dependencies broken or ineffective"â¦ High then ;)
<Daviey> rvba: Right, but it's a config option?
<Daviey> it's REALLY a low :)
<rvba> Daviey: the hostname of the broker_url passed around is hardcoded to 'localhost'.  None of the cluster controllers apart from the one installed next to the region controller will be able to connect to the region controller.
<Daviey> rvba: I hate to go all technical on you.. but swirly alert
<Daviey> rvba: Do you want us to go to Blue alert ?
<rvba> Daviey: ok.  We have to fix that anyway :)
<Daviey> http://youtu.be/81W8tG3wH_4
<rvba> hehe :)
<Daviey> rvba: Oh, i thought it was coded into a config file
<Daviey> ?
<rvba> Daviey: it is in a config file on the region controller side, then send to the cluster controller over the API when the cluster controller 'registers'.
<rvba> s/send/sent/
<Daviey> rvba: so can a human fix this?
<rvba> Daviey: yes, sed -i s/localhost/real.ip.of.the.region/g /etc/maas/maas_local_celeryconfig.py will fix it (I've tested it).
<Daviey> well then.. it's a Medium!
<Daviey> :)
<rvba> All right :)
<Daviey> If it was in .py files.. then sure.. I could see the urgency
<roaksoax> rvba: the above bug, that's why you do  sudo dpkg-reconfigure maas-cluster-controller
<rvba> roaksoax: I don't think so.
<rvba> roaksoax: 'sudo dpkg-reconfigure maas-cluster-controller' lets you specific the url of the API.
<rvba> Then the cluster controller connects to the region (using that URL) and gets back the BROKER_URL.
<rvba> Then it tries to get celeryd to connect to RabbiMQ using that BROKER_URL.
<roaksoax> rvba: ah I see, so that should be in the region side
<rvba> roaksoax: it is on the region side indeed.
<rvba> In the config of the region.
<rvba> That broker url is also used by the region worker (in this case it's fine, because the region worker is on the same machine as the region controller)
<roaksoax> rvba: right, so a function needs to be added to update that when updating maas_local_settings to
<rvba> roaksoax: I'm a bit unsure what to do here.  Because the host of the region controller is already in many places.
<roaksoax> rvba: I;ll take care of it
<rvba> roaksoax: it's in DEFAULT_MAAS_URL in /etc/maas/maas_local_settings.py (region).
<rvba> roaksoax: we could also use the 'host' part of the url given when running 'sudo dpkg-reconfigure maas-cluster-controller'
<rvba> roaksoax: what do you think?
<roaksoax> rvba: well I think that whenever we update DEFAULT_MAAS_URL, the BROKER_URL should be updated too
<rvba> roaksoax: but what if the user later changes DEFAULT_MAAS_URL manuall?
<rvba> manually*
<roaksoax> rvba: that's their problem :). We don't care about that... I mean... the packaging should have taken care of setting that up on initial install
<roaksoax> rvba: if the user chances it manually, the packaging cannot do anything about it
<rvba> roaksoax: maybe we should extract the 'hostname' part out of DEFAULT_MAAS_URL.  That would solve that problem.
<rvba> I mean dynamically, in the upstream code.
<roaksoax> rvba: yes, that's what I was gonna say, if that sits in the region controller and will sit there all the time, why not infer the broker url from maas_url if at the end of the day, they are gonna be the same
<roaksoax> (as in the same IP/hostname)
<rvba> roaksoax: I think it's the best thing to do.
<rvba> roaksoax: I'll take care of it then :)
<Daviey> roaksoax: on update, when i get asked to keep orig config or maintainers version.. what should i do?
<roaksoax> Daviey: Yes to overwrite
<roaksoax> rvba: awesome thanks!!
<roaksoax> Daviey: are we going to be releasing a new upstream release by the end of the week?
<Daviey> roaksoax: good, i did that
<Daviey> roaksoax: Yes, a new upstream.. latest.. before i go to bed on Thursday
<roaksoax> Daviey: ack
<Daviey> That is the last point i can guarantee it to be on the cd
<Daviey> After that, it might be a 0-day SRU
<roaksoax> rvba: could you please update your packaging MP, I had to merge some other stuff first because that had hit archive
<roaksoax> Daviey: ok cool
<Daviey> roaksoax: I accepted that upload, as i assumed it would be overridden on your next trunk snapshot?
<rvba> roaksoax: sure
<roaksoax> Daviey: yes thank you! patches recently uploaded will be dropped with new upstream snapshot. I'm just keeping packaging branch in sync just in case
<roaksoax> rvba: thank you!!
<Daviey> ok
<rvba> roaksoax: changes pushed.
<roaksoax> rvba: thank you
<roaksoax> rvba: that proxy_http module comes with apache2 itself right?
<rvba> roaksoax: yes
<rvba> That module is shipped with apache:
<rvba> $ dpkg -S /etc/apache2/mods-available/proxy_http.load
<rvba> apache2.2-common: /etc/apache2/mods-available/proxy_http.load
<roaksoax> rvba: ok fcool,t hat's weird though when loading wsgi (i think) the proxy_http should have been loaded too
<rvba> roaksoax: apparently not.
<roaksoax> yeah that's very rare cases maybe
<roaksoax> i have never seen that issue :)
<roaksoax> i always saw it being loaded
<rvba> Yeah, I'm surprise too.  But rbasak found that problem and I was able to reproduce on a canonistack instance soâ¦
<rvba> surprised*
<roaksoax> rvba: are we wnating to eliminate maas-common some day?
<rvba> roaksoax: I haven't thought much about it TBH.  It's probably possible and not really complicated but definitely not something we want to do now.
<roaksoax> rvba: ok cool then, because I think i'll move the creation of /var/lib/maas /var/log/maas etc etc to it
<roaksoax> rvba: instead of doing it in cluster and region
<rvba> roaksoax: sounds good.  We're sure it will be installed before the cluster controller or the region controller right?
<roaksoax> rvba: the only thing that cluster installs in /var/lib/maas is the scheduler for celery right?
<roaksoax> rvba: maas-common is a dependency of both so it will
<rvba> roaksoax: cool.  Yes, that's the only thing that the cluster puts in there.
<roaksoax> rvba: so cluster-controller only has 1 log too?
<rvba> ls /var/log/maas
<rvba> celery.log  oops  pserv.log
<rvba> 2 actually
<roaksoax> rvba: oops are also needed?
<rvba> Well, it has been created by the oops package.
<rvba> I /think/.
<roaksoax> nope it is ceated by cluster
<roaksoax> rvba: https://code.launchpad.net/~andreserl/maas/packaging_update_lp1065062/+merge/128988
<rvba> It's referenced in /etc/maas/pserv.yaml (oops:directory: "/var/log/maas/oops")
<rvba> roaksoax: I thought you wanted to do that in maas-commonâ¦?
<roaksoax> rvba: yes, but I have to do it, test it,  etc etc, so I first prefer fixing that
<roaksoax> rvba: and then evaluate the other
<rvba> roaksoax: all right.
<roaksoax> rvba: could you approve though please?
<rvba> roaksoax: approved.
<roaksoax> rvba: thankyou
<roaksoax> :)
<Daviey> rbasak: still around?
<rbasak> Daviey: yes
<Daviey> smoser & rbasak: So how are we fixing the power problem?
<Daviey> enlistment + power setting
<rbasak> I thought that would be fixed with a precise sru? Or is the problem that this will take too long?
<Daviey> rbasak: take too long
<Daviey> smoser: How do you feel about including it in the ephemeral images?
<Daviey> (before it hits -updates)
<smoser> i'm not opposed to doing that.
<smoser> 2 comments
<smoser> a.) its not a big deal to me, since we're already ppa'ing 4 packages in the ephemeral images for precise
<Daviey> Next smoser Okay JFDI, and create a new ephemeral image pronto?
<mgz> what's the ubuntu equiv. of ftpmasters and how do I see what packages are queued to land in quantal...
<smoser> b.) i'd like to hold off on spinning a "release" image until after (or on) release.
<roaksoax> Daviey: enlist with quantal
<Daviey> mgz: Archive Admins, https://launchpad.net/ubuntu/quantal/+queue
<smoser> Daviey, so i'd do it to 'daily' build.
<smoser> which is easily configurable
<roaksoax> Daviey: you ca enlist with quantal in the meantime too
<mgz> Daviey: thanks, you beat google.
<Daviey> smoser / roaksoax: Okay. How do i fix this /right now/?
<Daviey> mgz: Wassup?
<roaksoax> Daviey: go to the Settings and select quantal as the commssiioning release
<roaksoax> Daviey: and make sure you have quantal, and that's it
<Daviey> mgz: I am an AA.. is there something you are expecting ?
<roaksoax> Daviey: quantal ephemerals by enabling it /etc/maas/import_pxe_files
<Daviey> roaksoax: how do i make sure?
<Daviey> ok
<mgz> Daviey: pinging SpamapS about it now.
<mgz> need my brown paper bag juju fix.
<SpamapS> mgz: sorry I got de-railed by wifi problems
<SpamapS> mgz: will hopefully finish shortly
<Daviey> How do i cleanly delete all nodes now?
<mgz> SpamapS: no problem!
<roaksoax> Daviey: is the problme commissioning on enlistment?
<Daviey> mgz: can you describe the issue ?
<Daviey> roaksoax: i have 16 nodes with no power paramas.. would make sense to delete and recycle  them.. no?
<roaksoax> Daviey: right, so you can delete them thru maas shell
<mgz> Daviey: the issue is I'm a moron. see bug 1064734
<roaksoax> or maybe the CLI if it is supported
<ubot5> Launchpad bug 1064734 in juju (Ubuntu Quantal) "ERROR Invalid 'tags' constraint 'set(['test-tag'])': No such tag using maas-tags to deploy with juju" [High,In progress] https://launchpad.net/bugs/1064734
<Daviey> roaksoax: we still don't have multi-slect in GUI, ffs
<Daviey> SpamapS: Can you get that patch in asap?
<roaksoax> Daviey: i have enlisted/commissioned with quantal without issues
<roaksoax> so you should be good using that
<smoser> i'll agree, enlistment should work fine from quantal.
<smoser> Daviey, do you want me to create a new daily of quantal?
<Daviey> okay, that is fine.. i want to delete all nodes
<Daviey> Please advice how :)
<Daviey> smoser: is there a need?
<roaksoax> i'm trying to check
<smoser> well, you want that maas-enlist in it.
<smoser> right?
<Daviey> SpamapS: Are you handling that upload?
<SpamapS> Daviey: yes its almost done, just want to finish the test build
<roaksoax> Daviey: these are not HP MicroServers either right
<roaksoax> ?
<Daviey> SpamapS: did you see Kapil's comment? <-- mgz ?
<Daviey> (it still strips operators
<roaksoax> Daviey: i guess from maas shell would be
<roaksoax> from maasserver.models import Node
<roaksoax> nodes = Node.objects.get()
<roaksoax> for node in nodes:
<roaksoax>    node.delete()
<Daviey> Yeah.. but earlier n the cycle doing that caused an explosion
<roaksoax> Daviey: i have done that verious times without issue TBH
<Daviey> ok
<Daviey> ITYM, node = Node.objects.all()
<mgz> Daviey: yup, and updated in response, will note that in the mp
<roaksoax> Daviey: yeah
<roaksoax> Daviey: i just deleted nodes without issue
<Daviey> roaksoax: seems to have worked
<Daviey> thanks
<roaksoax> Daviey: so now, are these HP MicroServers, no right?
<Daviey> roaksoax: no, these are proper rack servers
<roaksoax> Daviey: ok, cool. so the ponly thing would be to enable quantal ephemeral images
<roaksoax> maas-import-pxe-files to get them
<roaksoax> and that should be all
<roaksoax> adn of course changing the default release for commissioning (which will be used for enlistment too)
<roaksoax> and that's done on the WebUI settings
<Daviey> http://pb.daviey.com/in2m/ :-)
<Daviey> ahh, i imported ephermals
<roaksoax> yeah
<Daviey> roaksoax: Using maas-cli, how do i accept them all?
<roaksoax> Daviey: that i don't know i haven't played much with it
<roaksoax> matsubara: ^^
<Daviey> smoser: Did we remove serial console then?
<Daviey> (for intel)
<matsubara> maas-cli maas nodes accept-all
<smoser> Daviey, :-(. yes.
<matsubara> Daviey, ^
<Daviey> https://code.launchpad.net/~smoser/maas/drop-ttyS0-kernel-opt/+merge/128747 :(
<smoser> read the bug referenced.
<smoser> there wasn't a lot of option.
<smoser> if /dev/console gets assigned badly (which is possible if you specify console=/dev/ttyS0 and there is no hardware), then basically a lot of stuff ceases to work as writes to stdout start failing.
 * roaksoax lunch
<Daviey> well ok.. But the sitation now is i have 16 nodes in epidermal with NFI why they haven't enlisted
<smoser> you can "configure" console=ttyS0 on by editing the config file.
<smoser> which just happens to be python source code in /usr/share/pyshared
<smoser> :-(
<Daviey> and lost on upgrade, right?
<flacoste> smoser: i marked that bug as high
<flacoste> Daviey: have you figured out the way to delete systems?
<flacoste> Daviey: you can use the cli now to do that
<smoser> Daviey, yes.
<flacoste> Daviey: list-systems to get the id and then a for loop calling delete-node
<flacoste> i might get the name of the commands wrong though
<Daviey> flacoste: ok, and to accept?
<roaksoax> y
<roaksoax> err
<Daviey> flacoste: Are the doc's being updated for this?
<flacoste> Daviey: there is an accept-all command
<flacoste> Daviey: yes, evilnick has all the details
<SpamapS> Daviey: uploaded
<SpamapS> mgz: ^^
<roaksoax> Daviey: i have re-added the console stuff tothe code
<roaksoax> so you should be able to see it now
<Daviey> SpamapS: What was mgz's response to it still missing something, from Kapil?
<Daviey> roaksoax: You've added it to Mark's garage?
<roaksoax> Daviey: yes
<Daviey> roaksoax: I really think we need to get this fixed properly.. as an /etc/ config option.
<Daviey> flacoste: ^ What do you think?
<roaksoax> Daviey: i think that this should be kernel_opts node attribute
<roaksoax> Daviey: allowing to set it independently to some value
<Daviey> w.t.f.. http://pb.daviey.com/Nr2C/
<roaksoax> however, it should default
<roaksoax> I have an idea of how to do it
<flacoste> Daviey: yes, I,ve targetted that bug for 12.10
<Daviey> flacoste: bug number?
<Daviey> roaksoax: Did you spot that /etc/maas/import_pxe_files has oneiric as a comment?  shouldn't that be quantal as a comment?
<SpamapS> Daviey: 2nd patch, all included
<roaksoax> Daviey:yes i saw
<Daviey> SpamapS: cool
<flacoste> Daviey: bug 1044503
<ubot5> Launchpad bug 1044503 in MAAS "kernel command line is not easily customizable" [High,Triaged] https://launchpad.net/bugs/1044503
<Daviey> SpamapS / mgz: Accepted
<SpamapS> Daviey: ty
<roaksoax> matsubara: you can dailybuild now
<Daviey> flacoste: thanks
<matsubara> roaksoax, already did.
<matsubara> but thanks anyway :-)
<roaksoax> cool :)
 * roaksoax finally off to lunch
<Daviey> roaksoax: o/
<Daviey> hmm, still no serial console
<Daviey> smoser: do you know why we lost rsyslog?
<smoser> during enlistment?
<Daviey> yah
<smoser> because we used to get it via the installer
<Daviey> ah
<smoser> i suspec thtat we can probably get it back with some craftyness.
<Daviey> would be good, as currently i a SOB
<smoser> enable ttyS0
<smoser> ok. i'm copying maas-enlist and building a daily with it inside.
<flacoste> smoser, roaksoax: who is taking care of bug 1064922?
<ubot5> Launchpad bug 1064922 in MAAS "Enlistment fails with 12.04 ephemeral images" [Critical,In progress] https://launchpad.net/bugs/1064922
<smoser> flacoste, that is fixed
<smoser> i thought.
<smoser> were you referring to bug 1063946 ?
<ubot5> Launchpad bug 1063946 in maas-enlist (Ubuntu Precise) "maas-enlist does not take power-type and power-params" [Undecided,New] https://launchpad.net/bugs/1063946
<Daviey> smoser: ttyS1 is enabled.. i wonder if this is ttyS0
<flacoste> smoser: right, it's a dupe
<smoser> flacoste, not really a dupe
<smoser> one is fails
<smoser> one is "doesnt do ipmi enlistment"
<flacoste> indeed
<smoser> we are going to get ipmi enlistment done in 2 ways
<flacoste> do we support enlistment now?
<smoser>  a.) SRU the maas-enlist package to 12.04
<flacoste> or we reverted that?
<Daviey> smoser: I am massively concerned it is reporting an invalid mac address
<smoser>  b.) i'm going to build a daily image with quantal's maas-enlist inside.
<smoser> Daviey, do you want some help?
<Daviey> smoser: please
<Daviey> http://pb.daviey.com/ssaw/
<roaksoax> smoser: SpampS already accepted maas-enlist in precise-proposed
<smoser> hm.
<roaksoax> smoser: were you able to help Daviey with the mac error?
<smoser> getting there.
<roaksoax> smoser: i'd say just ssh into the enlistment image and see why it is doing that
<roaksoax> and that's it
<Daviey> done
<roaksoax> Daviey: fixed? What was it?
<Daviey> roaksoax: no..
<roaksoax> :(
<roaksoax> smoser: i figured out the issue
<roaksoax> join to explain
<smoser> https://code.launchpad.net/~smoser/maas/fix-login-block-reboot/+merge/129035
<smoser> anyone want that?
<Daviey> smoser: status?
<Daviey> roaksoax: ^?
<Daviey> smoser / roaksoax: How far off having it functioning are wek?
<Daviey> we*
<Daviey> It's blocking adam_g_ from doing a deploy.
<Daviey> I'd quite like it if I could reliably do a precise/essex deploy by my morning.
<roaksoax> Daviey: uploading fix to quantal
<roaksoax> Daviey: fix uploaded to quantal
<roaksoax> adam_g_: ping
<roaksoax> adam_g_: after the newwer masa-enlist lands, simply enlist/commission with quantal images and then you should be able to deploy precise
<adam_g_> roaksoax: cool
<smoser> Daviey, still awake?
#maas 2012-10-11
<bigjools> roaksoax: https://code.launchpad.net/~julian-edwards/maas/missing-import-bug-1065055/+merge/129080
<roaksoax> bigjools: can't approave it
<roaksoax> bigjools: it will conflict with maas-region-controller
<roaksoax> bigjools: i guess it will have to end up being shipping in maas-common and will have to add conflicts/replaces
<roaksoax> s/shipping/shipped
<roaksoax> ./win 8
<bigjools> roaksoax: grar
<bigjools> roaksoax: fixed, take a look if you are still there?
<Daviey> roaksoax, smoser: sorry had to go
<Daviey> allenap: maas-cli is certainly less than user friendly
<Daviey> http://pb.daviey.com/QdsM/
<jtv> Daviey: you want "maas-cli login" to act as "maas-cli login -h" ?
<Daviey> jtv: i'd say so... I worked it out tho
<Daviey> jtv: mind you, http://pb.daviey.com/jIU4/
<Daviey> still not that clear WTF i am supposed to do.
<jtv> Daviey: well what you have there shows the API commands you can issue directly.
<jtv> Try "maas-cli maas node-group -h", for instance.
<Daviey> jtv: right, but do you see how it feels like i'm using a dev tool like ipython?
<jtv> Absolutely.  It's tough to get going with.
<Daviey> jtv: so, deleting a node
<Daviey> jtv: so, deleting a node
<Daviey> I need to find the node first.. how?
<Daviey> davewalker@maas:~$ maas-cli maas node-group list-nodes uuid
<Daviey> Not Found
<jtv> What's the uuid for?
<Daviey> http://pb.daviey.com/cqtV/
<Daviey> ah
<Daviey> maas-cli maas node delete $system-id
<Daviey>  maas-cli maas nodes list | grep system_id
<jtv> Looks like we have some underdocumented methods in the API.  :(
<rvba> jam: I was wrong, the memory was reported ok on my microservers.
<mgz> jam: in reply to your xpath comments,
<mgz> you're right, it's much more painful until postgres 9.2 because of the must-return-nodeset limitation
<jam> mgz: care to mumble?
<mgz> be a little careful comparing things to nodesets though
<mgz> occasionally the result is suprising
<allenap> Daviey: I've been thinking about that. maas-cli is a thin wrapper around the API, but there's an impedance mismatch there.
<Daviey> allenap: there doesn't need to be, does there?
<allenap> Cousin Cletus (aka the shell) just can't deal with the richness of the API.
<Daviey> allenap: bash tab-complete?  NEVER mentioning: usage: /usr/lib/python2.7/dist-packages/maascli/__main__.py maas
<Daviey>        [-h] COMMAND ...
<Daviey> defaulting to providing -h if params are missing?
<allenap> Daviey: Perhaps for 13.04 we need to come up a more task-orientated command-line tool, and leave the API tinkering in a library.
<allenap> Daviey: Yeah, those are things we ought to have, but they're polish.
<allenap> I think there's a fundamental problem with trying to manipulate highly structured data from the shell.
<rvba> jtv: could you please review: https://code.launchpad.net/~rvb/maas/bug-1065406/+merge/1291
<Daviey> allenap: polish or DESIGN? :)
<allenap> Daviey: Polish. We want to do tab completion. Changing the error behaviour is something that has come out in practice.
<allenap> The need to change it I mean.
<allenap> Also, I mean: we have always wanted to do tab completion.
<allenap> It's not a new thing.
<Daviey> right
<Daviey> Is longpoll still busted?
<Daviey> not seeing the dash dynamically update?
<rvba> This should have been fixed by now.
<allenap> Daviey: Frankly I think shell is the wrong place to do anything other than start something else. What can you do when you ask for a listing of allocated nodes? bash can't grok that structure; we immediately need to go out to awk or something, which leads to scripts that are held together by dog lick.
<Daviey> allenap: which is exactly what i have done..
<allenap> Yeah :-(
<rvba> Daviey: fixed in r127 of the packaging branch.
<Daviey> but actually, as a application driven api.. it could be worse.
<Daviey> rvba: I wonder if it's because i am using SOCKS
<rvba> Should be ok.  Apache is acting a proxy to the longpoll server.
<rvba> Daviey: could you try to access the longpoll url?  (if you get a Django 404 there is a problem because Django should not be serving that url)
<ubot5> Django bug 404 in Metasystem "MySQL order_by=['?'] throws ProgrammingError" [Normal,Closed] http://code.djangoproject.com/ticket/404
<rvba> haha :)
<Daviey> rvba: linky for the lazy
<Daviey> ?
<Daviey> longpoll url
<rvba> Just one sec.
<rvba> Daviey: http://localhost:8080/MAAS/longpoll/?uuid=blah
<Daviey> ta
<rvba> You should get "Invalid request" (because the uuid is bogus) and not a reply from Django.
<Daviey> rvba: 404,  The requested URL /MAAS/longpoll/ was not found on this server.
<Daviey> http://192.168.9.1/MAAS/longpoll/?uuid .. right?
<rvba> Daviey: looks like the fix from r127 is not there.
<rvba> Yes.
<rvba> Daviey: sudo a2enmod proxy_http && sudo service apache2 restart
<Daviey>   Installed: 0.1+bzr1243+dfsg-0+1252+129~ppa0~quantal1
<Daviey> Now i get, "Async frontend for None"
<rvba> Daviey: can you try loading proxy_http?
<Daviey> rvba: your line worked, no?
<rvba> Yeah, but that is exactly what the fix in r127 should have done.
<Daviey> http://pb.daviey.com/pVKb/
<rvba> Daviey: right, so proxy_http was not enabled.
<rvba> Daviey: can you please double check wit: cat /var/lib/dpkg/info/maas-region-controller.postinst | grep proxy_html
<rvba> (yes, I know, the cat/grep combination is a bit stupid)
<Daviey> rvba: I think it's covered here, http://partmaps.org/era/unix/award.html
<rvba> heh
<Daviey> confirmed, not in there
<rvba> WTF
<Daviey> the good news is IPMI discovery is working REALLLLLLY well.
<rvba> \o/
<rvba> Daviey: I don't understand, "a2enmod proxy_http" is in debian/maas-region-controller.postinst revision 129.
<Daviey> No idea :/
<rvba> Maybe roaksoax will have an idea.  Smells fishy to me.  Packaging fishy.
<Daviey> stinks!
<jtv> rvba: not finding that review you linked to
<rvba> jtv: https://code.launchpad.net/~rvb/maas/bug-1065406/+merge/129131
<jtv> That works.
<jtv> STOP EATING MY SHOES!
<jtv> They make sneakers much too light nowadays.  Cats can drag them out of the shoe rack.
<rbasak> Just tried highbank on quantal and ipmi setting has completely disappeared. Nothing on enlistment and nothing on commissioning. Is this expected?
<rbasak> http://paste.ubuntu.com/1272933/
<rbasak> smoser, Daviey: ^^
<mgz> jtv: clearly you need to walk around the house in clogs only, cats would struggle to make an impression on those
<jtv> Good idea.
<jtv> Thing is, where do I put my sneakers in the mean time?  You don't wear shoes in the house here.
<jtv> I'm inside, the cats are outside.
<jtv> Eating my sneakers.
<jtv> They're new, too.  The old ones were at least a year beyond their best-by date, and as soon as I get myself a new pair, I have to visit a house that has cats prowling around.
<jtv> Looking for shoes to pounce on.
<mgz> maybe you could buy a very tall shoe tree. like a hat stand, but to hang shoes off
<mgz> then the cats would at least have to climb for their prize
<jtv> You haven't seen what these cats do.
<jtv> I've seen them walk up bug screens like Spider-man.
<mgz> ehehe
<jtv> Walk, I tell you.  Two meters up, then jump up on the curtain rails.
<jtv> rvba: the regular celeryconfig does not have the import_settings problem, just the cluster celeryconfig?
<rvba> jtv: right: debian/maas-region-controller.maas-region-celery.upstart contains env PYTHONPATH="/usr/share/maas/" so the region celery process has access to the utility method.
<jtv> Oh, that's used only from the upstart job
<jam> jelmer: how's memory stuff going?
<jelmer> jam: I was about to get lunch; I'm close to being able to push a branch up, was working on tests.
<jam> jelmer: enjoy your lunch, just checking in to see how it was going.
<jam> allenap: 'timeit' disagrees with you. indent=4 was only slightly slower, enforce_ascii was a huge hit.
<mgz> allenap: I think generally having pretty printing of json isn't too bad, as most api results are small and with cli get dumped straight onto the console
<jam> 10ms for dumps, 12ms for dumps with indent=4
<mgz> it would be easy enough to reformat client side, but just setting a few internal calls to bypass prettiness seems okay
<jam> ensure_ascii=False => 114ms.
<rvba> roaksoax: I've unlinked the upstream project from bug 1065080.  I've talked to Julian about it and after some discussion, we think it would be more clean to have it fixed in the packaging (see my comment on the bug).
<ubot5> Launchpad bug 1065080 in maas (Ubuntu Quantal) "The BROKER_URL sent to a cluster controller has its hostname set 'localhost'." [Medium,Triaged] https://launchpad.net/bugs/1065080
<jam> jtv: yeah, the cats in our yard jump from the ground about 2 meters up to get up on the wall. I think they push off the wall a bit on the way up, but it is pretty impressive.
<jam> mgz, allenap: I think the ensure_ascii difference is because with it set to True (default) you get back an 8-bit str that escapse all the non-ascii bytes. Vs getting back a Unicode string.
<jtv> Love 'em.
<jam> The only thing I can think of, is that simplejson internally ends up using a unicode string, and appends chars to it (or some other silly thing), vs using a proper buffer.
<jam> allenap: I think I found it. "encode_basestring_ascii" has a C level accellerator, but 'encode_basestring' does not.
<jam> So in this case, we have 1 24kb string, so it doesn't matter as much if the *encoder* object is C, it matters more if the function called to encode that string is in C.
<rvba> rbasak: maybe you can land lp:~racb/maas/comissioningâ¦  Looks very low-risk to me ;)
<rbasak> rvba: thanks. Had forgotten about that!
<Daviey> Hmm
<Daviey> If you destroy-enviroment and juju boostrap, is it expected that it loses it's DDNS?
<mgz> ...how do I see an oops so I know what I broke?
<mgz> the traceback doesn't seem to get logged...
<mgz> ..disregard that, looking in the wrong dir
<smoser> rbasak, you still busted on that ?
<rbasak> smoser: yes
<rbasak> smoser: precise ephemeral works. Quantal does not.
<Daviey> hmm.. i thought that was fixed last night?
<rbasak> I installed from quantal this morning
<smoser> this is commissioning?
<rbasak> yes
<rbasak> maas	0.1+bzr1243+dfsg-0ubuntu3
<smoser> rbasak, can i see?
<rbasak> smoser: sure. Same machine as last time.
<rbasak> smoser: which bits do you need?
<smoser> seems like bzr importer failed on maas-enlist
<smoser> suck
<rbasak> smoser: I'm using released quantal.
<rbasak> smoser: just realised
<rbasak> smoser: do I need daily quantal?
<rbasak> Or shouldn't it matter? I don't see why it would
<smoser> rbasak, at that point you're pulling from the archive.
<smoser> the ephemeral fixes have primarily landed in the earlier sstages of boot and cloud-init
<smoser> but you're getting past that.
<mgz> amusing but non-fatal: http://pastebin.ubuntu.com/1273139/
<mgz> wip, needs tests: lp:~gz/maas/paginate_nodes_page_1064672/
<jam> rvba: is there a good place to look up documentation about how to set up maas with a bunch of clusters?
<jam> I'm going to be doing some stress testing of the hardware constraints stuff, trying to set up a very large system (100,000 nodes).
<jam> And I figure I need a DB server, N_small appservers, N_large (~25) cluster workers.
<rvba> jam: I don't think there is documentation for that but that's precisely what I'm testing atm.
<rvba> I'm not doing stress testing, just setting up a cluster controller on a separate machine.
<jam> rvba: hints/docs/notes/access or whatever would be appreciated.
<rvba> jam: we need the fix that jtv is currently landing.
<jtv> Which one?
<jam> jtv: import_*
<jtv> Because one has already landed
<jam> I believe
<jtv> Ah
<jtv> That's waiting for the cron job.
<rvba> The fix for bug 1065055.
<ubot5> Launchpad bug 1065055 in MAAS "celeryconfig_cluster.py imports utility method from maas (import_settings)" [Critical,In progress] https://launchpad.net/bugs/1065055
<rbasak> smoser: I've filed bug 1065499
<ubot5> Launchpad bug 1065499 in MAAS "Power parameters fail to get set when using Quantal ephemerals" [Undecided,New] https://launchpad.net/bugs/1065499
<rvba> jam: I'll build a package in my ppa once this will be landed.
<rvba> Then you only need to do 2 things:
<rvba> Manually change the BROKER_URL setting on the region cluster celery config (s/localhost/ip.address/).
<rvba> When you install a cluster, dpgk-reconfigure maas-cluster-controller => give the URL of the region
<rvba> jam: that should be all.
<jam> rvba: so the 'region' is the master?
<rvba> jam: not sure what you mean by that.  There is a 'master' cluster controller installed with the region yes.
<jam> so region to me sounds like a subgroup of something larger.
<jam> but I don't think a region talks to anything else
<jam> so it seems to be the largest unit.
<rvba> The region controller is the MAAS server.
<rvba> All the cluster controllers talk to the region controller.
<jam> right.
<jam> but if you had 'multiple regions' they wouldn't talk to eachother. correct?
<rvba> No, not right now.
<jam> k
<jam> rvba: so I saw in the design that you're supposed to be able to have multiple 'appserver' processes (I'm guessing that is the 'maas' process). Is that something that has been tested?
<rvba> jam: no, this has not been tested.  The goal was to use juju to scale horizontally like that.
<jam> rvba: so theoretically you could use juju to configure maas appservers as well as cluster controllers?
<jtv> jam, rvba: the import branch just landed.  Are you going to test it?
<rvba> theoretically yes
<rvba> jtv: I'll see if I can build a package.
<jtv> Great.  Basically all we need to know is that installing a cluster without a region controller on the same machine doesn't break, right?
<rvba> yes
<rvba> Hunk #1 FAILED at 182.
<rvba> 1 out of 1 hunk FAILED -- rejects in file contrib/preseeds_v2/enlist_userdata
<rvba> Patch 04-fix-ipmi-enlistment.patch does not apply (enforce with -f)
<rvba> arg
<rvba> Looks like that change has made it upstrea.
<rvba> upstream*
<rvba> jtv: jam any idea how to fix this, I've removed the patch, removed the reference to it in debian/patches/series but now I'm getting: "No such file or directory: ..04-fix-ipmi-enlistment.patch". I've no idea where it's referenced.
<rvba> It's only mentioned in the changelog file.
<jam> rvba: in the patch queue?
<jam> (.pc)
<rvba> jam: where is that? find . -name "*.pc" => nothing
<jam> rvba: it would be in the root directory, named '.pc/'
<jtv> Leftovers in the build directory maybe?
<jam> quilt was the name I was looking for.
<jam> but rvba, jelmer is someone who knows deb packaging quite well.
<rvba> jam: the root directory of the packaging branch?
<jam> rvba: yes
<jelmer> rvba: are you using bzr builddeb?
<rvba> bzr bd -S -- -kemail
<jelmer> rvba: you might have to run "bzr rm" (no arguments)
<jelmer> otherwise bzr doesn't pick up that that file is removed, and bzr-builddeb will try to look for it while exporting the source tree
<rvba> Thanks jelmer, that did the trick indeed.
<rvba> Uploading to ppa.
<rvba> jam: the ppa package will be ready soon (ppa:rvb/maas) all the packages are in http://people.canonical.com/~rvb/maas/.
<mgz> ho hum hum.
<jelmer> rvba: note that you can avoid the -k bit by putting something like this in ~/.devscripts:
<jelmer> DEBSIGN_KEYID=1EEF5276
<rvba> jelmer: I didn't know that, thanks for the tip.
<rvba> jtv: your fix to the celeryconfig_cluster worked fine.
<jtv> Load off my mind, thanks
<rvba> So it txlongpoll is fine too.
<rvba> Daviey: ^
<Daviey> super
<rvba> jtv: the dhcp_key generation thingy fixed the dhcp restart problem.
<roaksoax> rvba: ack
<rvba> roaksoax: the fixes for bugs 1065055 and 1065406 as in as of r1261.  I had to remove debian/patches/04-fix-ipmi-enlistment.patch to build the package (looks like the fixes in that patch are now upstream).
<ubot5> Launchpad bug 1065055 in MAAS "celeryconfig_cluster.py imports utility method from maas (import_settings)" [Critical,Fix committed] https://launchpad.net/bugs/1065055
<ubot5> Launchpad bug 1065406 in MAAS "Nodegroup object is created with an empty dhcp_key." [Critical,Fix committed] https://launchpad.net/bugs/1065406
<roaksoax> rvba: yep
<roaksoax> rvba: i'm aware :)
<rvba> All right :)
<roaksoax> rvba: btw... maas-dns is only meant to be installed in the region?
<rvba> roaksoax: yes
<roaksoax> rvba: so that means that maas-dhcp too
<rvba> No, the dhcp can be installed on the cluster side too.
<roaksoax> rvba: ok cool
<roaksoax> thanks
<roaksoax> rvba: so maas-dns should not depend on maas-dhcp
<roaksoax> should it?
<rvba> roaksoax: no, I don't think it's required.
<rvba> roaksoax: but I'm not sure we've tested that very well, maybe you can keep it for now.
<rvba> roaksoax: the maas-dhcp package can be installed next to any cluster controller.  And there is one next to the region (it's a dependency).
<roaksoax> smoser: your branch https://code.launchpad.net/~smoser/maas/ipmi-locate-fallback/+merge/129202  also fixes bug #1064527
<ubot5> Launchpad bug 1064527 in MAAS "detect_ipmi needs improvement. detects non-existant device in nested kvm" [High,Triaged] https://launchpad.net/bugs/1064527
<roaksoax> right?
<roaksoax> rvba: ok
<smoser> no.
<smoser> it doesn't
<smoser> the issue in the nested kvm was it just took forever
<smoser> as it got a false positive and then bmc-config....
<roaksoax> rvba: cause the maas-dns package does maas set_up_dns on postinst, but IDK whether we want that
<smoser> and that was haning
<smoser> this increases that chance actually
<roaksoax> smoser: right but your branch is a fallback method for IPMI detection, so in nested KVM it won't detect ipmi, would it?
<rvba> roaksoax: yes, we want that, if you install maas-dns, you want maas to register its config in the running bind instance and that is what set_up_dns does.
<roaksoax> rvba: ok cool
<smoser> roaksoax, mine only increases the likelyhood of finding something
<smoser> (but is very unlikely to false-positive)
<roaksoax> smoser: ok
<roaksoax> smoser: btw.. maas-enlist is in archives now so it should work
<smoser> right.
<smoser> we need to get the sru back
<roaksoax> smoser: i already uploaded it
<roaksoax> rbasak: could you please verify SRU bug #1056816
<ubot5> Launchpad bug 1056816 in maas-enlist (Ubuntu Precise) "maas-enlist does not post subarch" [Undecided,Fix committed] https://launchpad.net/bugs/1056816
<smoser> roaksoax, we need this bug fix in maas-enlist though. with the mac addrs.
<smoser> we dont have that yet
<smoser> do we?
<roaksoax> smoser: for precise?
<rbasak> smoser: any tips on testing from precise-proposed? Shall I just add all of it in sources.list? Or is there any pinning functionality I should be using?
<roaksoax> smoser: it is uploaded to -proposed but needs to be accepted first
<roaksoax> smoser: so I think the previous upload we can mark verification-done and then the new one will address the third bug
<roaksoax> rbasak: yes just add -proposed and install and test
<smoser> roaksoax, just ask that the -proposed be rejected
<smoser> i think
<smoser> and upload a new one
<roaksoax> smoser: i asked that
<roaksoax> smoser: they said it won't
<roaksoax> smoser: ScottK told me to make a new upload with higher revision with everything of the previous, plus the new stuff
<roaksoax> smoser: so I did
<roaksoax> smoser: but SpamapS told me to make a new upload with a new revision that adds, or removes anything from the previous upload
<smoser> ok. thats fine.
<smoser> roaksoax, so you uploaded?
<roaksoax> smoser: ye
<smoser> i will then copy that version to maas-ephemeral images ppa
<roaksoax> smoser: yes
<smoser> and we'llget new precise dailies
<roaksoax> cool
<SpamapS> roaksoax: what scottk said, and what I said, is basically the same thing, isn't it?
<SpamapS> roaksoax: I see 2 uploads. Which one should be reviewed?
<roaksoax> SpamapS: The way I understood it is: 1. Make an upload which contains the previous fixes (in -proposed) + the new fix and increase the revision (ScottK). 2. (yours) Make a new upload with only the *new* patch because the other 2 are already in -proposed
<roaksoax> SpamapS: so I guess 2. is the correct approach, right? So please accept the latest one uploaded, and reject the older one :) Thank you
<SpamapS> roaksoax: use understanding #1. ScottK definitely put it more clearly than I did.
<roaksoax> SpamapS: yeah so the latest upload is the correct one
<SpamapS> roaksoax: lets back up. What you need to do is take the current -proposed package, modify it with a new revision, and upload that.
<roaksoax> SpamapS: so the latest upload I've made has both ubuntu1.2 + ubuntu1.3
<SpamapS> roaksoax: latest one does indeed look correct
<SpamapS> http://launchpadlibrarian.net/119496240/maas-enlist_0.4-0ubuntu1.2_0.4-0ubuntu1.3.diff.gz
<roaksoax> SpamapS: yes, that one :). Thanks for taking care of it :)
<rbasak> So do I still need to verify current precise-proposed or wait for the new one?
<roaksoax> rbasak: you can verify what's with current
<rbasak> roaksoax: ?
<roaksoax> rbasak: verify with what's in -proposed
<smoser> and build debdiff versus last accepted.
<roaksoax> rbasak: you don'yt need to wait
<rbasak> ok
<SpamapS> I'm accepting but..
<SpamapS> ++		args="${args} --data-urlencode mac_addresses=${i}"
<SpamapS> ++		#mac_address="$mac_address""&mac_addresses=""${i}";
<SpamapS> commenting out?!
<SpamapS> bad form ;)
<smoser> roaksoax, ^ i told you to get rid of that comment before uploading :)
<roaksoax> smoser: arg... I did... but after so many links I guess I got confused an applied the wrong patch :)
<smoser> it works though
<smoser> so i'm good with that.
<SpamapS> all good
<ppetraki> I've got a maas client that simply will not budge, precise repo, HP proliant, WOL/PXE enabled eth0, use existing dhcp. http://pastebin.ubuntu.com/1273410/
<ppetraki> I click start node and it does nothing
<roaksoax> ppetraki: i wouldn't be relying on precise's MAAS to work properly in what WoL is concerned
<ppetraki> roaksoax, :(, to the ppa then
<ppetraki> roaksoax, thanks
<roaksoax> ppetraki: welcome :)
<roaksoax> flacoste: so today is the last day we can upload new upstream release before a SRU.. until what rev do you want to get in?
<roaksoax> Daviey: until when can we make the upload?
<smoser> roaksoax, so maas-enlist ? expecting in a sru somewhere?
<roaksoax> smoser: we just need to wait for it to be available for verification
<smoser> i see.
<roaksoax> smoser: its already accepted
<smoser> https://launchpad.net/ubuntu/+source/maas-enlist/0.4-0ubuntu1.3
<smoser> right
<roaksoax> smoser: after verification, we wait 7 days
<flacoste> roaksoax: so I'd like smoser fix for ipmi-locate to be in
<flacoste> i just approved it
<flacoste> so that would be 1264
<flacoste> rvba, allenap: anything other fixes in-flight that could be landed shortly?
<flacoste> jelmer: how's your memory constraint fix going? can we upload that today?
<rvba> flacoste: all the fixes we (the red squad) wanted to check in have been landed as of r1261.
<rvba> flacoste: Not sure if jtv will have time to finish the improvements he wanted to make to the cli.
<rvba> Help message improvements.
<rbasak> roaksoax: verification failed. MAC address problem. I assume this was your other fix?
<roaksoax> rbasak: yes
<jelmer> flacoste: Yeah, I think I can propose it for review before EOD
<rbasak> roaksoax: it stops me verifying 0.4-0ubuntu1.2 since all highbanks have two NICs
<roaksoax> rbasak: you can just veirfy with 1 MAC address
<rvba> flacoste: but all our (known ;)) critical bugs have been fixed.  I tested a package I made (ppa:rvb/maas) 0.1+bzr1261+dfsg-0ubuntu4~ppa1 a few hours ago.
<roaksoax> rbasak: maas-enlist --serverurl XYZ -i eth0 etc etc
<roaksoax> rbasak: if you specify the interface it will only use 1 MAC
<rbasak> roaksoax: I'm verifying by actually using MAAS to enlist. And that doesn't work.
<flacoste> jelmer: well, it would needed to be _landed_ before EOD, so should we wait for it, or not?
<rbasak> roaksoax: landing this in -updates will cause a regression.
<rbasak> roaksoax: thus it should be verification-failed, no?
<roaksoax> rbasak: that's why .3 is for
<rbasak> OK, so I'll verify .3 then
<rbasak> When it lands in -proposed
<roaksoax> rbasak: cool
<jelmer> flacoste: that should be doable too; when is the upload happening?
<flacoste> roaksoax: what's the limit?
<roaksoax> flacoste: Daviey's EOD
<flacoste> roaksoax: he has one?
<roaksoax> apparently so :)
<roaksoax> rvba: i'll make available a new package for you to test with your fixes and some minor changes :)
<rvba> roaksoax: all right.  I'll test the connection between the region and the cluster.
<matsubara> roaksoax, hey, did you see the package build failure because of the ipmi-enlistment patch?
<ppetraki> roaksoax, so should I just pin quantal working from precise or pick one of the pockets from here? http://ppa.launchpad.net/maas-maintainers
<roaksoax> matsubara: yes: https://code.launchpad.net/~andreserl/maas/packaging_updates_bzr1263/+merge/129221
<matsubara> roaksoax, cool. thanks!
<roaksoax> ppetraki: the latest quantal is not yet available for precise, but you should be able to test ppa:maas-maintainers/testing
<roaksoax> with newer maas for precise
<roaksoax> without cobbler
<ppetraki> roaksoax, thanks
<jelmer> rbasak: hi?
<flacoste> roaksoax: so prepare an upload for 1264, if jelmer lands his fix before Daviey's EOD, let's include it, otherwise 1264 is a good fallback
<roaksoax> flacoste: ack
<roaksoax> rvba: http://paste.ubuntu.com/1273523/
<rbasak> jelmer: ?
<roaksoax> rvba: i'm not particularly thrilled with that fix... but it seems the only way to do it easily
<jelmer> rbasak: I was curious if you had some example lshw -xml output from an ARM machine
<rbasak> jelmer: yes, one moment
<rbasak> jelmer: https://bugs.launchpad.net/maas/+bug/1064638/comments/5
<ubot5> Ubuntu bug 1064638 in MAAS "Commissioning is failing to set node memory attribute" [Critical,Triaged]
<roaksoax> rvba: i'd much rather just set the url independently
<rvba> roaksoax: looks like a good enough fix to me.  What do you mean independently?
<roaksoax> rvba: see how configure_maas_default_url or configure_maas_squid_deb_proxy work
<roaksoax> rvba: I'd much rather do it that way
<jelmer> rbasak: thanks
<roaksoax> rvba: unless... let me see
<rvba> I don't really see the difference tbh.
<roaksoax> rvba: i don't like the fact to be changing the password of rabbitmq on dpkg-reconfigure maas
<roaksoax> rvba: cause it also means restart rabbimq
<rvba> roaksoax: I agree.  But this is not related to that ip address thing right?
<roaksoax> rvba: it is
<roaksoax> rvba: because if I pass the ipaddress to that function, everything we dpkg-reconfigure we will change rabbitmq password
<smoser> roaksoax, rbasak i started a precise ephemeral image build.
<smoser> for daily
<smoser> and upon that being done will do one for quantal
<rvba> roaksoax: ah right, now I see.
<roaksoax> rvba: ok just uploaded to experimental ppa, so it should build soon
<roaksoax> rvba: please test clean installs and uipgrades to see if it does work correctly
<roaksoax> rvba: in both single node, and multi nod eescneario
<roaksoax> i;ll do the same
<rvba> ok
<rvba> Still not built :/
<roaksoax> rvba: yeah they are slowtoday :/
<roaksoax> rvba: maybe someone is rebuilding a whole bunch of packages today
<rvba> I think the package I uploaded to my ppa a few hours ago took 40 minutes to get built.
<flacoste> roaksoax, rvba: the lp build farm sucks since the CD move
<flacoste> we are still missing several builders
<Daviey> roaksoax: keep it coming
<roaksoax> ah! that's the reasong then
<roaksoax> Daviey: will let you know as soon as I have a package ready to upload
<Daviey> roaksoax: cool
<rvba> roaksoax: package is currently building.
<roaksoax> \oi/
<roaksoax> \o/
<rvba> roaksoax: package published.
<rvba> roaksoax: when doing: sudo dpkg-reconfigure maas-cluster-controller, I accidentally added an extra space at the end of the url 'http://my.ip/MAAS ' and then I got: sed: -e expression #1, char 45: unterminated `s' command
<rvba> Probably not critical.
<roaksoax> rvba: right, so i guess we need to get rid of whitespaces
<rvba> yep
<roaksoax> rvba: is everything working as expected?
<rvba> roaksoax: yes, so far.
<roaksoax> rvba: in my local machine cluster didn't create the celery beat stuff
<roaksoax> but might be related to something else
<rvba> $ ls /var/lib/maas
<rvba> celerybeat-cluster-schedule  celerybeat-region-schedule  dhcp  dhcpd-interfaces  ephemeral  media  tftp
<rvba> Looks ok here.
<rvba> I see the tasks being processed.
<roaksoax> rvba: who creates celerybeat-schedule?
<rvba> In /var/log/maas/celery.log
<rvba> The cluster celery worker.
<roaksoax> doesn't it create celerybeat-cluster-schedule?
<rvba> Which is started by the sstart-cluster-controller upstart script.
<roaksoax> rvba: so I just purged a maas installation, and this was left running:
<roaksoax> maas       775  0.0  1.0 330912 21124 ?        S    12:47   0:00 /usr/bin/python /usr/bin/celeryd --logfile=/var/log/maas/celery.log --loglevel=INFO --beat --schedule=/var/lib/maas/celerybeat-schedule
<rvba> roaksoax: you're right, there is no celerybeat-schedule
<rvba> I don't have that.
<roaksoax> rvba: that might have been from a previous package then
<rvba> Yeah, I think so.
<rvba> /usr/bin/python /usr/bin/celeryd --logfile=/var/log/maas/celery.log --schedule=/var/lib/maas/celerybeat-cluster-schedule --loglevel=INFO --beat -Q b0933e01-ea42-4852-9a3e-52f7912b50d8
<rvba> That is the cluster worker.
<rvba> /usr/bin/python /usr/bin/celeryd --logfile=/var/log/maas/celery-region.log --loglevel=INFO --beat --schedule=/var/lib/maas/celerybeat-region-schedule -Q celery,master
<rvba> That is the region worker.
<roaksoax> yeah
<rvba> roaksoax: all the cluster/region connection stuff works ok.  I cannot really test upgrades.
<roaksoax> rvba: i just did an upgrade and things seemed to work fine
<roaksoax> rvba: but then purge maas and all its components
<roaksoax> rvba: and on reinstlal there was an error related to dbconfig-common
<roaksoax> rvba: cna you try that?
<rvba> Sure.
<rvba> Purging a cluster controller then reinstalling works.
<rvba> I'll try with the region controller now.
<rvba> roaksoax: I didn't get an error.  Did you answer yes to all the questions asked by the packaging when performing the 'purge'?
<roaksoax> rvba: yeah
<roaksoax> rvba: it is an error with dbconfig-common
<roaksoax> apparently
<rvba> I didn't see any.
<rvba> Does it break the package completely?
<roaksoax> rvba: it does
<rvba> My region cluster seems fully functional.
<rvba> After re-installation that is.
<roaksoax> rvba: cool,I guess it is just probably a race somewhere
<rvba> roaksoax: Races are rarely easy to track down.  Do you have an idea where it could come from?
 * rvba tries the purge/reinstall cycle again.
<roaksoax> rvba: yeah maas-region-controller.config (in packaging) which triggers something in dbconfig-common and that's where it fails
<rvba> Really a race, like you say, because it worked again here.
<roaksoax> rvba: indeed
<roaksoax> rvba: it worked for me too
<roaksoax> rvba: i think it might appear when upgrading from precise to quantal, then purge, and then install again
<rvba> Probably not related, but when I remove the package I see:
<rvba> userdel: user maas is currently logged in
<rvba> /usr/sbin/deluser: `/usr/sbin/userdel maas' returned error code 8. Exiting
<roaksoax> rvba: yeah I already fixed that
<roaksoax> rvba: update&upgrade
<roaksoax> and you should not longer see that
<rvba> Cool.
<rvba> Indeed, I don't see the error message now.
<adam_g_> at what point in the process of starting a node does its MAAS managed hostname become resolvable?
<roaksoax> adam_g_: in mark's lab you are not using maas DNS i believe, so you have to wait till it is installed
<adam_g_> roaksoax: what happens after it is installed that makes the hostname magically resolve?
<roaksoax> adam_g_: dunno TBH... i guess it triggers dchp to tell DNS? i dind't really could investigate
<roaksoax> rvba: yeah it is dbconfig-common for sure
 * roaksoax lunch
<ppetraki> ok, so I think I have an bug off the testing ppa, I can't delete a maas node, the button is literally grayed out. I tried using the maas console but hit a wall: http://pastebin.ubuntu.com/1273624/
<ppetraki> this node had a problem provisioning, it was installed and all but the credentials didn't seem to make it, I can't login. When I do juju destroy it's able to put the other node in my two node maas back to ready but it has no idea about this node
<globin> i get this error: http://paste2.org/p/2324832 if i try to connect to the web interface right after installing. machine is running quantal. any ideas?
<matsubara> globin, I just got a similar issue
<matsubara> globin, is this on a pristine environment?
<globin> yep just created new vm
<globin> matsubara: tried it twice even
<roaksoax> globin: what version of MAAS are you using?
<globin> 0.1+bzr1243+dfsg-0ubuntu3
<roaksoax> globin: weird.. can you upgrade to latest?
<roaksoax> globin: is rabbitmq running?
<globin> roaksoax: 1 mom please
<globin> roaksoax: upgrading fixed
<roaksoax> globin: cool!
<globin> roaksoax: tyvm, thought i'd done that
<roaksoax> :)
#maas 2012-10-12
<Daviey> jtv: are you alive yet?
<mgz> why are you still alive Daviey...
<Daviey> dammit.. it was HIM i wanted
<Daviey> git!
<Daviey> allenap: Mornin'
<Daviey> mgz: Mornin'
<Daviey> mgz: I could do with chatting to you :_
<mgz> Daviey: sur
<mgz> er... sure, not french for on.
<Daviey> mgz: Well.. tags.. they should be working right?
<Daviey> as in, commissioning should be discovering some characteristics ?
<mgz> Daviey: yup.
<Daviey> mgz: If i told you it was returning an empty set for arm and G-lab, what would you say?
<mgz> we don't create any default tags at the moment, so you need to set them up yourself
<Daviey> Oh!
<Daviey> how come?
<mgz> but filling in cpu_count and memory uses a similar mechanism, so if your nodes have those then tags will works
<mgz> mostly because we don't know what people actually want
<Daviey> mgz: ii'm just seeinf []
<Daviey> seeing*
<mgz> I like the idea of documentation explaining how to do things and having examples
<Daviey> mgz: I imagined we'd get defaults of mem-$(free -g) and cpu-(numbr of cores) at least
<mgz> well, we have that anyway with the mem and cpu values
<Daviey> how do i query it then?
<mgz> via what means? you use the juju constaint form, either when acquiring a node, or on the nodes page on the site
<Daviey> mgz: http://pb.daviey.com/E2TV/
<mgz> ie, http//maas.dev:5240/nodes/?query=cpu=4
<mgz> right, unless you add some tags, your nodes will not have any tags :)
<mgz> so, use the commandline client to create some
<Daviey> mgz: I'm sorry if i am being dumb.. but how does that work?
<mgz> you supply a tag name, a definition, and optionally a comment
<Daviey> how does a definition work?
<mgz> the name is what you use to look up against nodes, the definition is an xpath that runs against the lshw output
<Daviey> mgz: http://192.168.9.1/MAAS/nodes/?query=cpu=24 is null by default?
<mgz> a trivial test is using 'true' as the definition, that will then be set on all nodes
<Daviey> mgz: Ok, is this captured in doc's?
<mgz> you have nodes with 24 or more cpus?
<Daviey> yes
<Daviey> 15 of them :)
<mgz> if you select one of them, do they have the correct cpu count?
<Daviey> ah, interesting.. they do no
<Daviey> they show 2!
<mgz> there is no user-facing docs yet, we needed code in for freeze, we need docs for release
<Daviey> and 49152 MB mem
<Daviey> (should be 42GB!)
<mgz> okay, so the trick will be to find out if it's the lshw output that's wrong, or our query
<mgz> if you can extract that and post it somewhere, will tell you if it's a platform bug or a maas bug
<Daviey> ok
<mgz> so, if you look at that output, there may be some other facet you care about, which you can then add a tag for
<mgz> (it's not that useful with an entirely homogeneous cluster)
<Daviey> http://pb.daviey.com/fKJs/
<mgz> ...we probably want an easy way of getting that lshw stuff as I suspect this might be a common support question
<mgz> that lists two cpus.
<Daviey> hmm, only showing two cpu's.. but multiple cores i guess
<Daviey> this one has 24 cores, ubuntu@node-0025904c7eb4:~$ cat /proc/cpuinfo  | grep processor | wc -l
<mgz> with multiple cores and threads per cpu
<Daviey> 24
<mgz> what's the actual box? two hexacore chips with hyperthreading?
<Daviey> mgz: So.. is physical cpu, or core more important ?
<Daviey> mgz: NFI
<Daviey> ubuntu@node-0025904c7eb4:~$ sudo dmidecode | pastebinit
<Daviey> http://pb.daviey.com/SOhU/
<mgz> probably the core count.
<mgz> plus or minus hyperthreading
<mgz> okay, so example tag to match that
<mgz> I'm not sure the exact thing you use via the client, but it's along the lines of
<mgz> *... addtag* --name dual_hexacore --definition "count(//node[@class='processor']/configuration/setting[@id='cores' and @value=6]) = 2"
<Daviey> mgz: Do you have ideas how we could add some common ones OOTB?
<Daviey> such as that
<mgz> well, that one doesn't seem common, but adding anything you suggest as examples would be fine :)
<Daviey> Users can always delete them.. but this feels so complex, that nobody will bother using it without some already provided
<mgz> right, so one option is to fake instance types a little
<Daviey> mgz: Do you want to list some viable defaults ?
<mgz> so, have a few set tags that are "box with about cpu/mem/other of X'
<Daviey> sounds reasonable
<Daviey> what do others think?
<mgz> the downside is composite tags like that will be very complicated compared to single purpose ones that just look at one detail of a box
<Daviey> right
<mgz> so, it's good to provide them, but they're bad examples to work off
<mgz> Daviey: can you file a bug about your cpu_count and attach that lshw output?
<mgz> because that wants fixing regardless I suspect
<Daviey> mgz: cpu_count should be cores rather than physical cpu's?
<mgz> that makes more sense to me
<mgz> jam: it's meant to be your day off!
<mgz> bad jam.
<lifeless> Daviey: prob /proc reported cpus, so neither physical nore cores.
<Daviey> lifeless: I'm not qutie sure what i am seeing.  I assumed /proc/cpuinfo == lshw :)
<Daviey> but it does not.
<jam> mgz: I'm off, I'm browsing for cars, and figured I'd check email.
<jam> too much crap to do when moving...
<jam> mgz: what do you think about leaving out the 'bank:' in the xpath
<jam> better to have it or not have it?
<jam> It feels like we're getting to the place where that XPATH expression should be a configuration variable, so it can be tweaked in the field.
<jam> mgz: plus, if I can spend 5-10 min to keep you guys unblocked, as long as my wife doesn't notice, it is time well spent :)
<rbasak> Daviey: please could you see bug 1065763? This is dannf's issue from yesterday.
<ubot5> Launchpad bug 1065763 in MAAS "UI URL gives HTTP error 200 after CD install" [Undecided,New] https://launchpad.net/bugs/1065763
<rbasak> As a dpkg-reconfigure maas-region-controller fixes it, I'm concerned that there's a postinst ordering issue
<mgz> the startswith() is a bit ugly, maybe using @class instead would help, but from the two compeltely different ways of listing memory it's a bit tricky to make one sane expression
<mgz> I am getting the the point of thinking we need to give people a really easy way of getting at the lshw output if it's going to be this variable
<Daviey> rbasak: ugh
<mgz> otherwise trying to workout what's wrong with an expression will be taxing...
<mgz> the one issue with having the cpu and mem expressions stored as data rather than code is the migration needs them, and has limited access to stuff, but can probably get around that
<Daviey> rbasak: Have you identified what is exactly not happening ?
<rbasak> Daviey: no, I've not looked at all. I can't reproduce.
<rbasak> I've not attempted to reproduce by altering postinst ordering mind.
<jam> mgz: as for data vs code, we could always have a default, and a way to poke a new expression into the system. (store in the db, rather than in the config)
<jtv> allenap, jam: it's getting busy in the review department...  remember to claim reviews you're working on, to cut down on duplicate effort!
<mgz> okay, posted pagination bits.
<roaksoax> rbasak: it doesn't seem to me that the postinst ordering is the problem
<roaksoax> rbasak: it seems to me that credentials for maas_workers in rabbitmq are the problem
<roaksoax> rbasak: that's why it works on reconfigure
<roaksoax> Daviey: ^^
<rbasak> roaksoax: ok
<rbasak> roaksoax: I'm not familiar with the packaging
<roaksoax> dannf: do you happen to have an installed system that failed to install maas without reconfiguring?
<roaksoax> rbasak: thank your for looking at it though :)
<Daviey> roaksoax: Have a fix?
<roaksoax> Daviey: trying to reproduce and see if it is exactly what i think it is :)
<jtv> Oh, hi Daviey, hi roaksoax -- better help output for maas-cli is ready to land.  Unfortunately the cron job doesn't seem to be landing it yet.
<roaksoax> Daviey: http://pastebin.ubuntu.com/1274979/
<Daviey> jtv: sweet
<jtv> The formatting may look a bit crap; I'm working on that.
<roaksoax> Daviey: it seems rabbimq issue
<Daviey> crap
<Daviey> i saw a race with that earlier
<jtv> allenap: trying the regular HelpFormatter for maascli... actually looks better!  Any idea why we're using the RawDescriptionHelpFormatter?
<roaksoax> Daviey: yeah so the user doens't get created
<allenap> jtv: Because the default formatter mangles the help text that contains :param foo: bits. It also smushes everything into a single paragraph.
<roaksoax> Daviey: ok, so the postinst creates 2 rabbitmq users. The first one creates correctly. However, for the installer, before we create the user we restart rabbitmq (force restart).
<roaksoax> Daviey: now, that we have 2 rabbitmq users... the fix seems to be that we should restart rabbitmq *again* for the installer
<roaksoax> Daviey: this will restart rabbitmq 2 times on install
<jtv> allenap: ah yes...  I guess I'll just have to format the input text then.
<Daviey> roaksoax: why does it need restarting ?
<roaksoax> Daviey: in the installer... because we should not run dameons on installation? and apparently after the user creation it dies or somthing happens
<roaksoax> Daviey: that's the same issue we had last cycle
<roaksoax> Daviey: so we had to force rabbimq to be running during the installer
<Daviey> I wonder if dpkg-triggers could do what we need
<roaksoax> Daviey: it would be the same wouldn't it?
<roaksoax> Daviey: because it would also happen during the install when we need rabbitmq running
<Daviey> roaksoax: Hmm, i'm not sure
<Daviey> my gut says it's the solution, but i'd need to dig deeper
<roaksoax> Daviey: doesn't seem to be fixed by trying to restart rabbigmq again
<roaksoax> rabbitmq
<Daviey> poop
<dannf> roaksoax: i don't, but i can recreate one
<roaksoax> dannf: no worries, I know what the issue is
<roaksoax> Daviey: ok so it seems that there's no easy way to fix this
<Daviey> roaksoax: what is the hard fix?
<Daviey> roaksoax: Are you sure dpkg-trigger isn't the right fix?
<roaksoax> Daviey: so i tried trying to restart again, it doens't really create the user
<roaksoax> Daviey: i tried to force the start (as we do with postgresql), it doesn't work
<Daviey> 677840
<roaksoax> Daviey: i have a couple of other ideas but we'll see
<roaksoax> Daviey: and for the little I read... maybe not
<roaksoax> bug #677840
<ubot5> Launchpad bug 517746 in Ubuntu Tweak "duplicate for #677840 E:No se pudieron leer las listas de fuentes." [High,Fix released] https://launchpad.net/bugs/517746
<roaksoax> Daviey: is that a bug number?
<Daviey> err, no
<Daviey> it was an accidental OTP :)
<roaksoax> lol
<roaksoax> Daviey: well it seems tha triggers might be an option but idk whether that will work on a reboot
<Daviey> i thought they should work 'when satisfied'
<roaksoax> Daviey: right, so this would involve modifying rabbitmq
<roaksoax> Daviey: "This feature is heavily used to track changes of packaged files in a list of predefined directories, and to update other files based on this."
<roaksoax> uhmmm these talks about files
<roaksoax> but this cause is not handling filse, is having a server daemon running
<roaksoax> allenap: maas already supports https?
<roaksoax> smoser: around?
<smoser> here
<roaksoax> smoser: could you verify bug #1065259 please?
<ubot5> Launchpad bug 1065259 in maas-enlist (Ubuntu Precise) "Enlistment fails if multiple MAC addresses are sent" [Undecided,Fix committed] https://launchpad.net/bugs/1065259
<smoser> roaksoax, i can try
<roaksoax> smoser: thanks!
<roaksoax> Daviey: ok.. so ... tried many things by fixing the postinst and still no lock, os no idea of what might be wrong
<Daviey> urgh
<Daviey> This is potentially a big deal
<smoser> roaksoax, whats wroong?
<Daviey> maas install bug from cd
<roaksoax> smoser: idk whether you recall that during last cycle, we needed rabbitmq running in the installer in order for us to create the longpoll rabbitmq user, and etc
<Daviey> ie, release critical
<roaksoax> smoser: so now, we not only create 1 rabbitmq user, but 2, and the second one doesn't get created
<roaksoax> smoser: or how can I add -proposed automatically?
<roaksoax> to eh enlist user-data
<roaksoax> smoser: this is the installre bug #1065763
<ubot5> Launchpad bug 1065763 in maas (Ubuntu) "UI URL gives HTTP error 200 after CD install" [High,Confirmed] https://launchpad.net/bugs/1065763
<smoser> roaksoax, enlistment?
<roaksoax> smoser: yeah, so i'd like to add -proposed on cloud-init user data, is that possible?
<roaksoax> i mean, so that the userdata can grab the enlist image from there
<smoser> roaksoax, well, you dont really have to.
<smoser> you can, but in a daily, the maas-enlist is already there.
<smoser> the most recent daily
<smoser> i copied the SRU'd maas-enlist to maas-ephemeral ppa
<roaksoax> smoser: ok so then we can mark it as verified then?
<smoser> well, if you verify it you can :)
<roaksoax> smoser: that's why I was asking.. in order to verify we need to get it from proposed :)
<smoser> ok.
<smoser> a.) you dont need to do that. you could just use the daily of precise. it would have it.
<roaksoax> smoser: right but we need to veirfy what's in proposed not on the daily even though you copied it :)
<smoser> b.) you can (i suggest trying it) add a ppa by adding the appropriate cloud-config content to /usr/share/maas/preseeds/enlist_userdata
<smoser> apt_sources:
<smoser>  - source: deb $MIRROR $RELEASE-proposed main
<smoser> roaksoax, ^ add that and "should work"
<roaksoax> smoser: cool thanks
<roaksoax> allenap: does maas already support HTTPs?
<smoser> roaksoax, do you want me to try to do this?
<smoser> to verify?
<roaksoax> smoser: i already did
<smoser> oh. ok.
<roaksoax> thanks tho
<smoser> so my advice worked?
<smoser> did you use a daily ephemeral?
<roaksoax> smoser: no i haven't upgraded ephemerals and yes i used -proposed
<smoser> cool
<roaksoax> flacoste: howdy!1 unrelated question that you might help me with. Why do PPA's don't allow upoading a package, which is in the Ubuntu Archive, but has a different tarball?
<flacoste> roaksoax: you mean same version number than in the archive but actual different tarball?
<roaksoax> flacoste: yep
<roaksoax> flacoste: so I'm trying to upload a precise version of it, which has a different tarball
<flacoste> roaksoax: it's a good question, i know we don't allow two publication of the same version number with a different tarball
<flacoste> ah
<flacoste> but that would prevent the precise version from being upgraded when going to quantal, no?
<flacoste> so in this case, that check does make sense
* flacoste changed the topic of #maas to: Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | MAAS jenkins: https://jenkins.qa.ubuntu.com/job/maas-trunk/
<roaksoax> flacoste: right, but even if the quantla version has a higher ubuntu revision number, the upload fails
<roaksoax> flacoste: anyways, i guess this might prevent an SRU from happening due to this check
<roaksoax> because the upstream tarballs are gonna be different
<roaksoax> the precise version has to include python-tf-tftp and yui3
<flacoste> roaksoax: the only check that i am aware of is that we don't allow two publication of the same package version with two different tar ball
<flacoste> now, i wasn't aware that this was cross archive
<flacoste> i have no idea if this is intentional or not
<flacoste> best thing is to file a bug if you think it is
<roaksoax> flacoste: this is the message: "File maas_0.1+bzr1264+dfsg.orig.tar.gz already exists in Primary Archive for Ubuntu, but uploaded version has different contents."
<flacoste> and more knowledgeable folks will assess if it is or not
<roaksoax> flacoste: will do
<flacoste> roaksoax: right, so it seems that it's a cross archive check
<flacoste> hmm
<roaksoax> flacoste: indeed
<flacoste> my guess is that this is intentional
<flacoste> to prevent a ppa from hijacking a ubuntu package
<roaksoax> yeah it seems like it
<flacoste> or shadowing if you prefer
<roaksoax> right
<roaksoax> flacoste: alrught, thanks for the info
<lifeless> its a consistency thing
<lifeless> I *think* it applies to pockets as well
<lifeless> e.g. ubuntu-security/ubuntu-release
#maas 2013-10-07
<smoser> maas in ubuntu is still using old import_ephemerals ?
<smoser> :-(
<smoser> roaksoax, jtv thats correct, right?
<smoser> we *really* need to get that fixed, as we want to be delivering updated fast path installer images and the old import_ephemerals does not delete content, meaning guaranteed filesystem full over time.
<roaksoax> smoser: yup... they told me it was not ready
<smoser> imo that has to land.
<smoser> jtv, allenap ? is there a bug that explicitly addresses this? if not I'll open one.
<roaksoax> smoser: i dont disagree but if BBBthe tools is not ready we cant uae it
<allenap> smoser: jtv will need to answer that one.
<roaksoax> smoser: and there are merge proposals already so it is being taken care od
<jtv> smoser: I think it falls under the s-cloud-maas-maintainability blueprint.
<smoser> it also falls under grave-security-issue if you want me to push that route.
<smoser> how far are we from that being functional? i really had thought that this landed. sorry for not raising earlier.
<roaksoax> smoser: the tool has landed not the use of it
<jtv> What roaksoax said.
<roaksoax> jtv: whats the status on the user configurable settibgs?
<jtv> What settings specifically?
<roaksoax> i raised to rvba that imho disnt seem like a good idea to put everuthibg on pserv.yalm
<roaksoax> jtv: maas-import-ephenrals?
<jtv> No, we're ditching that.
<jtv> Yes, we're talking aboutg maas-import-ephemerals.
<roaksoax> jtv: so....?
<roaksoax> jtv how will a user select what releases wants to import and so on?
<jtv> There's three ways:
<jtv> 1. Command-line option.
<jtv> 2. yaml config â but we didn't like the way the script rewrote pserv.yaml, so we're moving that to a separate file.
<jtv> 3. The existing shell-style config still works.  The script converts it to yaml config.
<roaksoax> ok
<roaksoax> i guess 2/3 is what i meant
<jtv> I guess.
<roaksoax> jtv: so if 3 currently works... why add yaml?
<roaksoax> jtv: and... what is now preventing its usage?
<jtv> The fact that we have changes we want to make to the config.
<smoser> roaksoax, consuming shell from python is non ideal. i'd support moving away from that.
<jtv> Right now, if you run the script (even with --help!) it writes its new config.  After that it gets harder to move things around the way we intend to.
<jtv> What smoser said.
<roaksoax> smoser: roght i agree in the meantime it wouldnt hurt would it?
<jtv> We've already moved away from that, but we've got some changes we want to make to how we migrate away from it.
<roaksoax> jtv: ok
<roaksoax> jtv: ETA?
<smoser> and how is this planning to be run?
<smoser> previously the user could run 'maas-import-ephemerals' and when it returned they knew that the system was ready for use.  while that may not be an ideal interface, it does allow you to actually determine when maas was usable.
<smoser> after installation, there needs to be a way to basically block until usable.
<jtv> Hoping to be done with the config changes by Wednesday.  But hard to predict, given the state of documentation etc.
<jtv> smoser: it's still run in the same way â whether from the CLI or the UI.
<smoser> jtv, so in packaged version is this runnable now?
<smoser> is that what you're saying ?
<jtv> The packaged version still uses the old script.
<jtv> But you run it in the same way: CLI or web UI
<roaksoax> smoser: the new version is not being installed
<smoser> well, i opened bug 1236361.
<ubot5> bug 1236361 in maas (Ubuntu) "need new simple streams based maas-import-ephemerals" [Critical,Confirmed] https://launchpad.net/bugs/1236361
<smoser> i consider this critical to release of 13.10.
<jtv> smoser: by the way, I noticed that the old ephemerals script only seems to download the oldest versions of images.
<smoser> ?
<smoser> i dont think thats the case.
<jtv> It was downloading only beta versions of precise & quantal.
<smoser> as i'm running it right now against dailies and its dowlnoading saucy 20131007
<jtv> Could you check your precise & quantal versions?
<smoser> the *new* ehpemerals script was broken (due to server data being broken)
<smoser> in that way
<smoser> server data is now fixed.
<smoser> server data had 'beta' as the "serial" version which C locale sorts to > YYYYMMDD, so simplestremas code was correctly considering those to be the latest
<smoser> jtv, i certainly do not want to delay anything, but if you're consuming shell code as 'config' with python, https://gist.github.com/smoser/6466708 does that safely.
<jtv> smoser: if you see any specific problem that it would help with, by all means do.
<jtv> We do have some advantages that generic code doesn't have though: a statically known set of variables, and an existing relationship between the application and the config file.
<smoser> jtv, you dnot actually have a know set of variables.
<smoser> you exposed a shell program as a config mechanism.
<smoser> which then, users could quite reasonably use however they wanted.
<jtv> We have a known set of variables we are interested in.
<smoser> but you dont know how they're set.
<smoser> (note, you can pass in the list of variables you're interested in to that method, and it will correctly get their values)
<smoser> anyway. i'm fine if you dont use it. but it is as close to "correct" a way as I can come up with to solve the "shell was used as config" problem.
<jtv> That's nice.  By all means, once you have the tests done, propose a branch that uses this in MAAS!
<stokachu> will python-curtin be backported to raring, quantal, and precise?
<rbasak> stokachu: AIUI it's destined to go into the cloud-tools pocket of the cloud archive
<stokachu> rbasak: ok thanks
<smoser> stokachu, no.
<smoser> not necessary
<stokachu> so funny story if you run make install-dependencies in the maas code tree it removes all network-manager related packages
<stokachu> leaving you without internet
<smoser> good times.
<smoser> you didn't need that interweb stuff anyway
<stokachu> lol
<smoser> stokachu, you dont need to port python-curtin to raring/quantal or precise.
<smoser> because pytho-curtin is installed on the maas server
<smoser> it is packed up into a self extracting tarball, and shoved across to the installing node.
<smoser> and then invoked over there.
<stokachu> in the control file it depends on python-curtin
<smoser> thats right.
<smoser> maas does depend on that.
<smoser> you will not get maas with curtin support on Q or R.
<smoser> you get it on P through cloud-archive tools
<stokachu> so are all maas developers running saucy at this point?
<stokachu> or will buildout handle getting a maas development instance up locally
<smoser> oh. i see.
<smoser> i dont know the answer to that question.
<stokachu> i tried following the hacking.txt file but it left me without network connectivity
<smoser> stokachu, my response to any software package that says "hey, run this as root" is "run this in a lxc container or new cloud instance".
<smoser> which makes running saucy or precise easy, and alleviates the network manager hardship.
<smoser> i'm not saying thats a good thing, but its a general purpose solution to the problem.
<stokachu> ok
<smoser> roaksoax, https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1236433
<ubot5> Ubuntu bug 1236433 in maas (Ubuntu) "maas-cluster-celery did not survive upgrade" [Undecided,New]
<roaksoax> smoser: did you only upgrade maas or rabbitmq or something else too?
<roaksoax> smoser: you are also upgrading upstart: upstart:amd64 (1.10-0ubuntu3, 1.10-0ubuntu6)
<roaksoax> smoser: might be related?
<smoser> see bug.
<smoser> roaksoax, fwiw, it still seems unable to power on nodes for me :-(
<smoser> (even after starting the daemon)
<roaksoax> smoser: weird, I have been running a server myself without issues
<roaksoax> smoser: i'll check in a bit
<stokachu> so user provided preseed files are only available for enlist and commissioning? is there a way to override generic preseed during install?
<roaksoax> stokachu: you can modify the preseed as you wish
<stokachu> roaksoax: directly modifying the generic preseed or through a user supplied preseed?
<roaksoax> stokachu: directly modifying the preseed. you can create your own preseeds there and they will be added to the path, and you will have to add what you want to be added to the generic preseed from those user supplied ones
<roaksoax> stokachu: for example, preseed_master imports from generic, so you could techniclally create another preseed similar to generic, and in preseed_master you would need to "call" what's in your preseed
<stokachu> so generic inherits preseed_master so wouldnt i need to create a generic_custom?
<stokachu> or is this the problem with Tempita
<stokachu> the documentation states Tempita's inheritence is 'weird'
<strikov> hey guys. i just switched my toy maas server from 12.04 to 13.04 and met very strange issue. when i accept the node (either using dashboard or maas-cli) system doesn't create anything in pxelinux.cfg (in fact this folder is not created) and my nodes can't start commissioning. any ideas on how to fix that? thanks
<stokachu> so if i wanted to reconfigure the network on the node during installation do i edit generic or preseed_master
<stokachu> so creating amd64_generic_saucy will override generic, but i would still need to modify preseed_master to accept new network defintions, is that correct?
<stokachu> roaksoax: ah i see now what you meant by calling whats in my preseed
<roaksoax> stokachu: :)
<stokachu> roaksoax: so in the web ui when a preseed is created for a node it'll pick up amd64_generic_precise if the commissioning is set to that os image?
<stokachu> os image being precise
<roaksoax> stokachu: huh?
<stokachu> roaksoax: when commissioning a node will it pickup amd64_generic_precise if it exists?
<strikov> well, i figured out that maas from 13.04 uses python/twisted-based tftp server instead of original tftpd. it mean that it can handle requests w/o pxelinux.cfg folder. but my nodes are stopped trying to pull pxelinux.cfg/<mac> which means that this tftp server doesn't work well.
<roaksoax> stokachu: you added a preseed from the webui?
<roaksoax> stokachu: for commissioning/enlistment you can only add scripts that will do different taskls, not preseeds
<stokachu> roaksoax: where does the installer pick up amd64_generic_precise from if it exists?
<stokachu> preseed_master hardcodes the network, i need to override that
<roaksoax> stokachu: what's amd64_generic_precise?
<stokachu> roaksoax: that is my user-provided preseed
<stokachu> from the doc: As you can see this mechanism is also used to calculate the base preseeds for
<stokachu> all of installation, enlistment and commissioning.  It allows end users to
<stokachu> add, for example, a file named ``amd64_generic_saucy`` that would be used
<stokachu> instead of the ``generic`` template at installation time.
<roaksoax> stokachu: idk how will affect curint installation method which uses the style of enlistment/commissioning, but if you are using d-i
<roaksoax> commissioning/enlistment are completely different from installation preseeds
<stokachu> roaksoax: that i understand, i am more concerned with how to override the preseed during installation
<roaksoax> stokachu: ok, so have you added your preseed to /etc/maas/preseeds?
<stokachu> before the actuall install procedure
<roaksoax> stokachu: and defined a section
<roaksoax> and called it in preseed_master?
<stokachu> so thats my question do i create a amd64_generic_precise and inherit preseed_master
<roaksoax> stokachu: if you'd show me a diff of what you are trying to do maybe I could understand better :)
<stokachu> ok one sec
<stokachu> roaksoax: http://paste.ubuntu.com/6205699/
<stokachu> preseed_master call self.network_conf, but that only exists in amd64_generic_precise
<stokachu> is this the correct way to override the installation for a amd64 precise install via maas?
<roaksoax> stokachu: yeah that should work, but if network_conf is only for mad64
<roaksoax> then you would need to do stuff like: {{if node.architecture in {'i386/generic', 'amd64/generic'} }}
<stokachu> put that in the preseed_master right?
<roaksoax> stokachu: http://pastebin.ubuntu.com/6205703/
<stokachu> ah ok
<stokachu> roaksoax: thanks this clears up some stuff for me to work on
<stokachu> roaksoax: because i named it amd64_generic_precise this will only be read during a precise install correct?
<stokachu> everything else uses generic
<roaksoax> stokachu: i don't know if the naming convention works that way
<roaksoax> stokachu: better ask Ruetobas
<roaksoax> err
<roaksoax> rvba:
<roaksoax> sorry
<roaksoax> :)
<stokachu> roaksoax: im referencing the preseeds.rst documentation
<roaksoax> stokachu: I have not tested that, but if that what the documentation says then I would give it a try
<stokachu> there also doesn't seem to be a way to just create a install preseed for a series without defining the architecture
<stokachu> ok ill do some more testing on this
<smoser> strikov, i think likely the tftp server is not listening on the right interface
<strikov> smoser: i had pretty the same thought first, but my node can pull pxelinux.0 w/o any problems
<strikov> smose: i assume it pulls this file through tftp as well
<smoser> strikov, its od..
<smoser> i've seen this before.
<smoser> is your node virtual ?
<strikov> yes, the whole environment is vbox based
<smoser> https://bugs.launchpad.net/maas/+bug/1051626
<ubot5> Ubuntu bug 1051626 in MAAS "PXE boot not working in virtual environment" [High,Expired]
<strikov> smoser: interesting, but it looks like guys in the bug can't pull pxelinux.0 as well
<strikov> smoser: we may observe different sides of the same issue though
<smoser> strikov, no. se my coments in line 13.
<smoser> comment 13
<smoser> and even 7.
<strikov> ah, i see
<strikov> maas from 12.04 worked fine for me in the same environment
<strikov> i need to check if pxe binary changed a lot
<smoser> strikov, its not the pxe binary
<smoser> see my other comments there. . i basically tested everything hardy -> 12.04
<strikov> aha, well it looks like the issue began when maas switched to its own tftp server, right?
<strikov> because 12.04 was the last maas with original tftp
<strikov> did you try gavin's solution with restarting pserv?
<smoser> strikov, i have tried many things. and i'm not sure if i hit this or not any more.
<strikov> okay, i'll give it a try now. thanks for pointing to the bug.
<smoser> strikov, could you try adding 'next-server' entry in /etc/maas/dhcpd.conf
<smoser> and setting it correctly
<smoser> i'm just curious.
<strikov> okay, my virtual machine is reinstalling now, i'll check when it is ready
<strikov> smoser: restaring pserv doesn't help
<strikov> smoser: what did you mean by next-serv, do you want me to put my original server ip there or to setup external tftpd and put it there?
<smoser> add an entry in /etc/maas/dhcpd.conf for 'next-server YOUR.SERVER.IP.HERE'
<smoser> right underneith the 'filename' entry
<strikov> smoser: i see and actually i found tons of errors in /var/log/maas/pserv.log while doing pxe boot
<strikov> smoser: it can't find some images, trying to understand what's happening
<strikov> smoser: actually it can't find i386 images because i disabled pulling them
<smoser> strikov, yeah... so that sucks.
<smoser> i think this is basically  unavoidable.
<strikov> smoser, next-server doesn't help
<smoser> that you have to have the i386
<smoser> for enlistment at least
<smoser> as maas doesn't know the arch at tha tpoint.
<strikov> i did enlistment
<strikov> with the bootable cd
<strikov> at this point maas should know my arch i think
<strikov> smoser: eod for me. i'll keep trying tomorrow and let you know if i find the solution
<strikov> smoser: thanks for helping, ttyl
<smoser> strikov, i agree, it shoudl know
<smoser> roaksoax,
<smoser> <adam_g> smoser, [Mon Oct 07 18:50:06 2013] [crit] [client 127.0.0.1] configuration error:  couldn't perform authentication. AuthType not set!: /MAAS/static/images/amd64/generic/precise/xinstall/root.tar.gz   <- any hint?
<roaksoax> smoser: that's apache?
<adam_g> dropping 'Require all granted' from maas-cluster-http.conf helps
<adam_g> roaksoax, yea
<adam_g> smoser, curtin worked fine otherwise \o/
<roaksoax> adam_g: weird, before that required all granted was needed otherwise it wouldn't work
<roaksoax> adam_g: hold done, is this precise?
<roaksoax> adam_g: is this precise?
<adam_g> roaksoax, it is 1.4+bzr1656+dfsg-0ubuntu2~ctools0  on precise
<roaksoax> adam_g: that's the problem them, for saucy the http ocnfig is different
<roaksoax> which is why it fails in precise
<roaksoax> due to different apache
<adam_g> oh hum
<adam_g> that stinks
<smoser> gah.
<smoser> that does stink.
<roaksoax> smoser: i guess we could try fix that in postinst
<smoser> well, you probably shouldnt edit a conffile postinst
<smoser> though, right?
<roaksoax> smoser: yeah but that is a config file being shipped by maas btw
<smoser> right.
<smoser> bugger
<roaksoax> yup
<adam_g> some good pointers in https://wiki.debian.org/Apache/PackagingFor24 for supporting both apaches
<roaksoax> ok i think i know how to fix it
<smoser> oh nice.
<smoser> IfVersion
<roaksoax> yup
<smoser> roaksoax, i'll file a bug
<roaksoax> smoser: ok
<smoser> roaksoax, https://bugs.launchpad.net/cloud-archive/+bug/1236544
<ubot5> Ubuntu bug 1236544 in MAAS "canot access static images over http with apache 2.2 (precise)" [Critical,Confirmed]
<roaksoax> smoser: thank you
<sconklin> I'm unable to bootstrap the juju environment on my raring maas server. Best I can tell from searching, it's because all my nodes are "allocated to root", and not "ready". How can I return them to ready status
<smoser> sconklin, maas-cli $MAASNAME node release "$nodeid"
<roaksoax> smoser: makes sense? http://paste.ubuntu.com/6206324/
<roaksoax> adam_g: could you pelase test: http://paste.ubuntu.com/6206324/
<sconklin> smoser: what is $MAASNAME?
<smoser> sconklin, however you use maas-cli
<smoser> if you've not used maas-cli you need to set that up.
<sconklin> never used it
<smoser> http://paste.ubuntu.com/6206336/
<smoser> you can do the same stuff in the web ui, click 'release' for each of the nodes.
<sconklin> I don't see 'release' in the UI
<smoser> or 'stop node' i guess.
<sconklin> or stop node
<smoser> for the node?
<smoser> you have to go to the node' spage.
<roaksoax> smoser: proposed fix... just need someone to test it: https://code.launchpad.net/~andreserl/maas/lp1236544/+merge/189692
<adam_g> roaksoax, works but two things: should be closed with </IfVersion>, not </If> and you need to enable the 'version' module for that tag to work
<smoser> adam_g, you have a funny definiton of "works"
<stokachu> doing a clean upgrade from maas 1.2 to maas 1.4 in cloud-tools shows this error http://paste.ubuntu.com/6206361/
<stokachu> any way around that other than removing and installing maas 1.4?
<stokachu> hmm i did a clean install of maas and get that same error during the migration
<smoser> wow.
<smoser> stokachu, please file a bug.
<guntbert> did anybody see my report about http://maas.ubuntu.com/docs/install.html ?
<smoser> adam_g, did you test roaksoax config change at https://code.launchpad.net/~andreserl/maas/lp1236544/+merge/189692
<AskUbuntu> Can MAAS install non-ubuntu operating system | http://askubuntu.com/q/354996
<adam_g> smoser, i tested the equiv manually, but only works when the version module is enabled. there's a corresponding packaging change that needs to happen
<stokachu> smoser: figured it out, maas build depends on python-django and doesn't set a min version requirement as maas 1.4 needs at least django 1.4
<stokachu> ill file a bug with a patch
<smoser> roaksoax, ^
<roaksoax> adam_g: yeah typo :)
<stokachu> or i can just do a MP
<roaksoax> adam_g: is the module called 'version'? (a2enmode version)
<adam_g> roaksoax, yeah
<smoser> stokachu, we need a bug.
<roaksoax> adam_g: k thanks
<smoser> then you can do a MP
<roaksoax> smoser: i think we can drop the build-dep in python-django
<smoser> but we have to have a bug at this point in the ubuntu cycle
<smoser> roaksoax, id ont follow. stokachu is explicitly stating we need runtime depends (i think)
<stokachu> smoser: ok ill create it too
<stokachu> bug that is
<roaksoax> 15:54 < stokachu> smoser: figured it out, maas build depends on python-django and doesn't set a min version requirement as maas 1.4 needs at least django 1.4
<stokachu> GenericIPAddressField was introduced in django 1.4
<smoser> roaksoax, yeah, how would we drop the build-dep ? i'm confused.
<roaksoax> smoser: we don't really build anything (binaries and such)
<smoser> you could remove a 'Build-Depends' possibly, but who cares. the problem is that we are missing a 'Depends'
<smoser> well, roaksoax generally speaking i think the "best practice" is to build-depends on everything you Depends on and run 'make test' and use dh-python magic to determine your runtime depends wherever you can.
<roaksoax> smoser: we are missing versioning, not a depends
<smoser> right.
<smoser> so how would dropping a build-depends change anything.
<roaksoax> smoser: http://paste.ubuntu.com/6206456/
<roaksoax> makes sense?
<stokachu> roaksoax: yea that works
<smoser> roaksoax, is it 'maas-region-controller' that depends on it? or is it python-django-maas.
<smoser> i suspect the first, but just checking.
<roaksoax> smoser: the first one
<smoser> k. then that change sems to make sense to me.
<roaksoax> smoser: https://code.launchpad.net/~andreserl/maas/packaging_lp1236544/+merge/189694
<roaksoax> https://code.launchpad.net/~andreserl/maas/lp1236544/+merge/189692
<roaksoax> all yours
<smoser> roaksoax, you need the packaging change alos
<smoser> right?
<smoser> to enable the config mod
<roaksoax> smoser: i did that already, check both MP's
<smoser> oh i see.
<smoser> :)
<smoser> sorry
<roaksoax> :)
<roaksoax> no worries
<smoser> but you need sconklin's bug number for your debian/control change
<smoser> release team will want the bug number
<sconklin> smoser: ??
<smoser> oh. sorry.
<roaksoax> stokachu: what's the bug number
<smoser> stokachu's bug
<roaksoax> adam_g: what did you install to get the version module:
<roaksoax> roaksoax@pursue:/etc/openvpn$ sudo a2enmod version
<roaksoax> ERROR: Module version does not exist!
<stokachu> i need to create the bug
<stokachu> lemme do that right now
<stokachu> roaksoax: 1236572
<roaksoax> stokachu: thank you sir!
<stokachu> roaksoax: np :D
<guntbert> guys I can understand that you don't want to deal with apparent errors in the documentaion - in the meantime I filed a bug report - but it would have been nice if someone had said so :-)
<stokachu> guntbert: did you associate an MP with it?
<guntbert> stokachu: MP?
<stokachu> guntbert: a merge proposal with the documentation changes
<smoser> guntbert, i personally don't agree with bigjools' statement there.
<roaksoax> stokachu: yep
<adam_g> roaksoax, i didn't install anything to have that module available
<roaksoax> adam_g: weird... I don't seem to have it
<guntbert> no, I didn't have any proposal yet, atm I am just trying to follow some guide and stumbled over that discrepancy - anyway the bug I filed is #1236571
<stokachu> guntbert: cliking import boot images really just shells out to maas-import-pxe-files
<stokachu> afaics
<guntbert> stokachu: I see, so the instructions still apply - and it was only a misunderstanding
<stokachu> guntbert: i think you have a valid bug
<stokachu> guntbert: just needs some better wording on the docs is all
<roaksoax> adam_g: ok that module seems to be available in precise but not in suacy
<roaksoax> smoser: ^^
<guntbert> stokachu: well there are always the same problems with docs for a fast moving project :-)
<stokachu> guntbert: agreed -- i recently fixed a few just a few days ago
<smoser> roaksoax, really?
<roaksoax> y
<smoser> what is the module name ?
<roaksoax> smoser: version
<roaksoax> smoser: you give it a try to see im not crazy
<smoser> package ?
<roaksoax>  the package has to a2enmod version if 2.2 is installed. For 2.4, we will compile mod_version in statically.
<guntbert> have a nice time - bed time here :-)
<smoser> roaksoax, so you ahve a fix?
<smoser> who is "we" ?
<smoser> i'm confused
<roaksoax> smoser: i read that somewhere
<roaksoax> smoser: i don't have a fix yet
<roaksoax> smoser: we should probably be carrying a delta otherwise
<roaksoax> smoser: https://wiki.debian.org/Apache2Transition#Should_we_support_a_transitional_fallback_configuration_which_allows_web_apps_to_work_with_both_2.2_and_2.4_packages
<adam_g> roaksoax, you should only need to 'a2enmod version' from postinst if running apache2.2
<adam_g> https://wiki.debian.org/Apache/PackagingFor24 has a good example for detecing that from postinst
<roaksoax> adam_g: ok so this should be it: http://paste.ubuntu.com/6206625/
<roaksoax> smoser: http://paste.ubuntu.com/6206625/
<adam_g> roaksoax, looks reasonable
<roaksoax> smoser: https://code.launchpad.net/~andreserl/maas/packaging_lp1236544/+merge/189694
<roaksoax> adam_g: ty
<smoser> roaksoax, alternatively you just dont care if the a2enmod fails
<smoser> right?
<roaksoax> smoser: better? http://paste.ubuntu.com/6206668/
<smoser> well, thats not really waht i was saying.
<smoser> yours is fin ei thikn
<smoser> i was saying you can just only 'a2enmod version || true'
<smoser> on apache2 it runs fine.
<smoser> errr on 2.2 tha tworks
<smoser> on 2.4 it will fail, but you dont care
<roaksoax> smoser: right
<roaksoax> smoser: either way works i guess
<roaksoax> smoser: do you want me to change it?
<smoser> yours is fine. i think.
<smoser> if you've tested it.
<roaksoax> smoser: i haven't
<smoser> alright. i have to run for a couple hours. ill be in later.
<roaksoax> smoser: k enjoy!
<smoser> roaksoax, thanks for your work on this.
<roaksoax> smoser: np
<roaksoax> smoser: i think maas-signal or maas are broken :/
<bigjools> o/ roaksoax
<roaksoax> bigjools: howdy
<roaksoax> bigjools: back from holidays?
<bigjools> yes, 15m ago
<roaksoax> bigjools: hope you enjoyed it !
<bigjools> ETOOMUCHEMAIL
<bigjools> I did, thanks!
<bigjools> might not do so much driving next time
<roaksoax> bigjools: heh! the fun of driving
<roaksoax> bigjools: so now that you are here.. you can help me debug this
<roaksoax> bigjools: :P
<bigjools> roaksoax: normally it's fine but not so much with 4 kids
<bigjools> roaksoax: I can try but I am considering turning off IRC so I can catch up with email
<bigjools> so just to warn you :)
<roaksoax> bigjools: haha i think that might be a bad idea.. FinalFreeze on Thrusday :)
<bigjools> I know ...
<bigjools> I need to write release notes
<roaksoax> bigjools: ok so just quickly. I'm enabling moonshot to hopefully land it in saucy
<bigjools> cool
<roaksoax> bigjools: so the issue seems that maas doesn't seem to be updating the power params in maas
<roaksoax> bigjools: on commissioning
<roaksoax> bigjools: the power_type gets set, but not power_user,power_pass,power_address
<roaksoax> bigjools: where can I debug this?
<bigjools> roaksoax: turn on http debugging using the API call to change settings
<roaksoax> bigjools: (if i change power tyupe of ipmi the same result, so I know this is not only a moonshot thing)
<bigjools> and you can see the message that maas-signal sends
<bigjools> if you're not familiar with it I can tell you how
<roaksoax> bigjools: yeah i just need to know how to enable the logging fro apache2
<roaksoax> (http debugging)
<bigjools> oh it's changed in my absence....
<bigjools> roaksoax: it dumps at log level DEBUG
<bigjools> you might be able to change that in the web UI
<bigjools> but I don't know offhand how to set it in the API
<stokachu> what log file contains information about what happens after i press 'start node'
<stokachu> ive setup ssh keys and have virsh configured over ssh for the power type
<roaksoax> bigjools: uhmmm
<roaksoax> bigjools: can't we do this in apache2?
<bigjools> no
<bigjools> it's better to do it in Django
<bigjools> you get a nice formatted dump
<bigjools> just hack the log level at the end of src/maasserver/middleware.py
<bigjools> and restart apache
<bigjools> we're supposed to be able to set this on the fly but I don't know how now that someone changed this code
<roaksoax> bigjools: /etc/maas/maas_local_settings.py has the logging
<roaksoax> bigjools: so is this for handlers or loggers or what?
<roaksoax> adh dah
<roaksoax> handlers
<bigjools> all http comms are logged at DEBUG by default
<bigjools> so turn on DEBUG
<roaksoax> no luck doing it in the config
<roaksoax> bigjools: so this would output to maas.log right?
<bigjools> roaksoax: yes
<bigjools> did you restart apache after changing the config?
<roaksoax> bigjools: yup
<roaksoax> bigjools: no nothing
<roaksoax> i even restart the machine
<bigjools> roaksoax: just hack the log level in that source so it's at INFO
<bigjools> then restart apache
<bigjools> you'll see stuff in either apache's log or maas.log, can't remember which
<roaksoax> bigjools: logger.debug("%s\n%r\n%s", header, request, request.content)
<roaksoax> i changed that from logger.info to logger.debug
<bigjools>     log_level = logging.DEBUG
<roaksoax> bigjools: i can't find that in maasserver/middleware.py
<bigjools> you have an old source
<bigjools> trunk is newer
<roaksoax> bigjools: this is trunk from last week
<roaksoax> -_-'
<roaksoax> anyway,. gonna get a newer maas then
<bigjools> this change is in r1660
<roaksoax> bigjools: i have 1655 or so
<roaksoax> :)
<bigjools> the old code is changable with an API call then
<roaksoax> bigjools: what's the api call? :)
<bigjools> roaksoax: set "request_log_debug" in settings
<roaksoax> bigjools: that crashes the UI
<bigjools> sigh
<bigjools> probably why the code got changed :)
<roaksoax> probably
<roaksoax> ok i'll let this build and re-test
<roaksoax> bigjools: because if this is really a bug
<roaksoax> then is critical
<bigjools> sure
<roaksoax> bigjools: ok im in newer trunk
<bigjools> roaksoax: ok so look for "log_level = logging.DEBUG" near the bottom of middleware.py
<roaksoax> bigjools: changed from debug to info
<bigjools> and s/DEBUG/INFO/ then restart apache
<roaksoax> and don't really see any log
<bigjools> something must be logged somewhere
<bigjools> do some UI or API requests
<roaksoax> bigjools: i just see the normal stuff
<bigjools> :/
<bigjools> I know this stuff works, I've seen it
<bigjools> as I said, not sure which log though
<roaksoax> bigjools: i'm tailing apache logs, all of maas' logs
<roaksoax> etc
<roaksoax> bigjools: this is the only thing i see: ==> /var/log/apache2/access.log <==
<roaksoax> 10.18.1.40 - - [07/Oct/2013:18:58:51 -0500] "POST /MAAS/metadata//2012-03-01/ HTTP/1.1" 200 181 "-" "Python-urllib/2.7"
<bigjools> roaksoax: ah bugger,  this logging only works for maasserver, there's a different api for metadata
<roaksoax> plop
<roaksoax> bigjools: how can we see that then :)
#maas 2013-10-08
<bigjools> hack the api source that deals with maas-enlist
<bigjools> log the request there
<roaksoax> bigjools: this is not maas-enlist btw
<roaksoax> this is maas0singla
<roaksoax> maas-signal
<bigjools> yeah
<bigjools> that hits metadata service IIRC?
<roaksoax> yeah
<bigjools> roaksoax: any luck?
<kurt_> Any ideas why tags aren't working? http://pastebin.ubuntu.com/6207405/
<roaksoax> bigjools: nope
<roaksoax> bigjools: i gave up
<bigjools> roaksoax: urgh
<bigjools> roaksoax: I'll see if we can extend that logging to the metadata api
<bigjools> roaksoax: but surely a logger.info() hack will show you the data we need
<roaksoax> bigjools: i uploaded the paclage to experimwntal ppa
<roaksoax> that includes the
<roaksoax> moonshotbstudd
<roaksoax> stuff
<roaksoax> anyway i gotta go
<roaksoax> will be back later
<bigjools> roaksoax: ok
<bigjools> kurt_: are you sure the system id is right
<kurt_> I'll pastebin
<kurt_> bigjools: good call.  I'm working with two different test systems. Thanks
<stokachu> im still having issues getting the maas instance to power off/start a node via virsh
<stokachu> ive tested that i can virsh --connect qemu+ssh://ubuntu@10.x.x.x/system from the maas instance and can see the nodes
<stokachu> and starting/stopping from there works
<stokachu> ssh keys are deployed from maas instance to host as well
<stokachu> keys are added to my preferences
<smoser> stokachu, is your problem that maas can't power the nodes on?
<stokachu> smoser: yea
<smoser> are yo usure ?
<stokachu> smoser: i click start node from the UI and nothing happens
<stokachu> if i use virsh from maas instance i can start it there
<stokachu> ive looked through all the logs and don't actually see where it attempts to power the node on
<smoser> you can do that as the 'maas' user ?
<stokachu> smoser: yea running virsh --connect qemu+ssh://ubuntu@10.x.x.x/system from the maas instance works
<smoser> no
<smoser> as the maas user
<smoser> sudo -Hu maas virsh --connect ...
<stokachu> lemme try
<smoser> in saucy the virsh.template is /etc/maas/templates/power/virsh.template
<smoser> it might be elsewhere on raring.
<stokachu> it prompts for a password so i dont have ssh keys copied for that user
<smoser> right. so 2 options then.
<smoser> a.) get the maas user able to connect without prompt
<smoser> b.) change that template file in issue_virsh_command() to do sudo -Hu someuser
<smoser> and make someuser able to connect
<stokachu> ok, and where in the logs could i find errors like this?
<stokachu> like where it seems to be waiting on input
<stokachu> is that a celery thing?
<smoser> i dont know where you'd see logs liek this. probably would be in cluster-controller.
<smoser> it may be that its just still blocked int hat script
<smoser> and there *is* no long
<smoser> no log
<smoser> fwiw, http://bazaar.launchpad.net/~virtual-maasers/charms/precise/virtual-maas/trunk/view/head:/scripts/setup-maas
<smoser> see line 230 there. thats how we hacked it. 229 -> 243
<stokachu> ah ok
<smoser> but you shoudl prbably be able to get the maas user able to ssh itself like that
<stokachu> smoser: yea i need to set a shell for the maas user and all that
<smoser> right.
<stokachu> ok lemme do that now and see
<stokachu> smoser: http://askubuntu.com/questions/292061/how-to-configure-maas-to-be-able-to-boot-virtual-machines found this as well
<stokachu> maybe we should add that into the official docs for maas on libvirt
<smoser> that is a well written answer.
<stokachu> thats the way i just did it and it worked
<smoser> roaksoax, bigjools ... is there a way to create a non-admin user non-interactively ?
<smoser>  i guess i can just create admin and the drop staff or something.
<bigjools> smoser: don't think there is
<roaksoax> no idea
<roaksoax> bigjools: any luck?
<bigjools> roaksoax: well I was waiting for you to implement my suggestion
<roaksoax> bigjools: lol what's your suggestion again?
<bigjools> logger.info() in the relevant API call
<bigjools> dump the request
<bigjools> either that or you just tcpdump
<roaksoax> bigjools: well unfortunately i lost access to the environment
<roaksoax> so i guess nothing will get done today
<bigjools> damn
<bigjools> well we can reproduce elsewhere
<roaksoax> bigjools: yeah
<roaksoax> bigjools: one with ipmi
<bigjools> at least if it's not reproducible it's a little less urgent
<roaksoax> smoser: do we have anything available?
<roaksoax> bigjools: do we have somewhere to reproduce this?
<bigjools> roaksoax: qa lab?
<bigjools> garage maas?
<roaksoax> bigjools: garage maas is a no
<roaksoax> bigjools: i have an idea
<roaksoax> will try to reproduce locally faking ipmi discovery
<bigjools> roaksoax: we need an SRU for python-tx-tftp (0.1~bzr31-0ubuntu8)
<bigjools> for precise
<roaksoax> bigjools: cloud archive
<bigjools> is it in there already?
<roaksoax> should be yes
<bigjools> and is this the excuse for not doing SRUs now? :)
<roaksoax> bigjools: not, but the reason of the cloud-archive is we want to avoid the headaches of sru's right?
<bigjools> I thought it was to get around changes that were not SRUable
<bigjools> that said, I don't know if that tftp one is or not
<roaksoax> bigjools: probably
<roaksoax> well let's think about it post release
<bigjools> yeah, better time :)
<kurt_> bigjools: why can't maas-cli support multiple value pairs for tags? i.e. maas-cli maas tag nodes cntrlr-node, quantum-node OR cntrl-node quantum-node etc.
<roaksoax> bigjools: reproduced locally
<roaksoax> bigjools: so how can I see logs again?
<bigjools> roaksoax: good, yet bad :(
<bigjools> roaksoax: edit the maas-signal function to log the request
<roaksoax> bigjools:  payload = geturl(url, creds=creds, headers=headers, data=data) >
<roaksoax> ?
<bigjools> kurt_: because it wasn't implemented that way I guess
<kurt_> doesn't it make sense to do that?
<bigjools> roaksoax: hang on
<bigjools> kurt_: probably, but it is what it is
<kurt_> too much to ask for an enhancement?
<bigjools> feel free to file a bug
<kurt_> ok dokey
<bigjools> not sure if/when it would get looked at
<bigjools> but the source is open.... ;)
<kurt_> i'm almost certain there are bigger fish
<kurt_> :)
<bigjools> I can agree there
<kurt_> about your question about how to detect VMWare, I was going to respond in email.  But since you are hereâ¦how is kvm detected?
<roaksoax> bigjools: i think we could log where we receive the mparticular request
<roaksoax> on the metadata server side
<bigjools> roaksoax: I am making you a patch
<roaksoax> ok
<kurt_> bigjools: never mind.  I just added info to the bug.
<bigjools> roaksoax: untested, but: http://pastebin.com/zVdmRAPD
<roaksoax> bigjools: nothing logged :/
<bigjools> !
<bigjools> restarted apache?
<roaksoax> did that
<roaksoax> restarting machine
<bigjools> wtf
<bigjools> roaksoax: https://bugs.launchpad.net/maas/+bug/1232154
<ubot5> Ubuntu bug 1232154 in MAAS "Logs seem incomplete" [Critical,Triaged]
<bigjools> something is screwed
<bigjools> try using print() instead of logger
<roaksoax> yeah is saw a few MP's related to loggers
<roaksoax> bigjools: ok so tomorrow i'll confirm this bug is in trunk without the moonshot stuff
<roaksoax> bigjools: but this really shouldn't be affected by mnoonshot
<roaksoax> bigjools: lp:~andreserl/maas/moonshot_enablement
<bigjools> yeah
<bigjools> I agree
<bigjools> roaksoax: having said that, the daily tests in the QA lab would have been blowing up
<roaksoax> bigjools: yeah/ I'm wondering if it is because enlistment works, then commissioning for some reason keeps the same credentials
<roaksoax> i'll debug this tomorrow though
<bigjools> roaksoax: it's possible that is exactly what is happening
<bigjools> a bug was fixed where something was getting overwritten unnecessarily IIRC
<roaksoax> maybe that's related
<roaksoax> we'll see
<roaksoax> bigjools: so it seems the mparameters are not being passed
<roaksoax>     raise MAASAPIBadRequest("power_type %s, power_params %s" % (power_type, power_parameters))
<roaksoax> MAASAPIBadRequest: power_type ipmi, power_params None
<bigjools> roaksoax: where is that code?
<roaksoax> bigjools: api.py
<roaksoax> store_node_xyz
<roaksoax> maasserver/api.py
<roaksoax> store_node_power_parameters
<bigjools> so maas-signal is broken?
<roaksoax> bigjools: maybe! i'll debug this tomorrow against the latest trunk (without my enablement branch)
<bigjools> ok
<roaksoax> bigjools: ok
<roaksoax> i know what the issue is
 * roaksoax is blind
<roaksoax> roaksoax: params["power_parameters"] = json.dumps(power_parms)
<roaksoax> that's what was missing :(
<bigjools> roaksoax: missing where?
<roaksoax> bigjools: in my enablement branch :)
<bigjools> roaksoax: lol
<bigjools> sorry for laughing :)
<bigjools> roaksoax: TESTS!
<roaksoax> bigjools: yeah I haven't gotten to that part yet
<bigjools> can you see why we are anal about tests? :)
<roaksoax> bigjools: lol
<roaksoax> bigjools: anyway, sorry for the noise
 * roaksoax bed
<bigjools> roaksoax: sleep well!
<roaksoax> 2 more months till vacation
<roaksoax> yaaaaaaaay
<bigjools> :)
<mgz> how does the maas api versioning work?
<mgz> bug 1236734 mentiones that ip_addresses doesn't exist in the raring maas, can I not check that based on maas version somehow?
<ubot5> bug 1236734 in juju-core "juju 1.15.1 polls maas API continually" [Undecided,New] https://launchpad.net/bugs/1236734
<bbcmicrocomputer> be great if someone could take a look at bug 1236786
<ubot5> bug 1236786 in MAAS "ISO fails installing maas-dhcp" [Undecided,New] https://launchpad.net/bugs/1236786
<bbcmicrocomputer> it breaks the install CD currently
<rvba> roaksoax: hi, could you please have a look at bug 1236786?
<ubot5> bug 1236786 in MAAS "ISO fails installing maas-dhcp" [Undecided,New] https://launchpad.net/bugs/1236786
<roaksoax> rvba: there's really no information for me to figure out what's wrong on that bug, I'd need logs
<roaksoax> smoser: bug #1236786
<ubot5> bug 1236786 in MAAS "ISO fails installing maas-dhcp" [Undecided,New] https://launchpad.net/bugs/1236786
<roaksoax> bbcmicrocomputer: ^^
<rvba> roaksoax: thanks for looking into it.
<roaksoax> smoser: ok so the fix for the apache2 thing works in saucy
<roaksoax> i have not tested packaging
<roaksoax> in precise
<roaksoax> but saucy works
<roaksoax> rvba: around?
<rvba> roaksoax: yes
<roaksoax> rvba: so any updates on the maas-import-ephemerals stuff?
<rvba> roaksoax: well, yes; making progress but it's not yet done.  Julian wanted Jeroen to make a few changes to the migration stuffâ¦  sorry I can't be more precise than that.
<roaksoax> rvba: so there's no way we will have something for today ig uess then?
<rvba> roaksoax: no, not today.  Hopefully tomorrow.
<roaksoax> rvba: alright.. tomorrow is really our last day to upload stuff
<roaksoax> smoser: ^^
<bbcmicrocomputer> roaksoax: added syslog with debug mode to bug 1236786
<ubot5> bug 1236786 in MAAS "ISO fails installing maas-dhcp" [Undecided,New] https://launchpad.net/bugs/1236786
<roaksoax> bbcmicrocomputer: what's the workaround you had?
<bbcmicrocomputer> roaksoax: I was mounting securityfs on /target/sys/kernel/security, then installing, but in the preseed/late_command option
<bbcmicrocomputer> roaksoax: although you can't use that here
<roaksoax> bbcmicrocomputer: right cause judging from the logs... it doesn't really tell me what's failing exactly
<roaksoax> bbcmicrocomputer: can you try this though?
<roaksoax> bbcmicrocomputer: http://paste.ubuntu.com/6209648/
<roaksoax> bbcmicrocomputer: that should probably deal with it
<roaksoax> since invoke-rc.d is denied execution (which is expected) maybe that's causing the failure
<bbcmicrocomputer> yeah it's not obvious, 'Warning: unable to find a suitable fs in /proc/mounts, is it mounted?' comes from apparmor_parser inside maas-dhcp postinst I believe
<bbcmicrocomputer> which returns exit status of 1
<roaksoax> bbcmicrocomputer: righ
<bbcmicrocomputer> then we would just run preseed/late_command as 'in-target sh -c \"mount -t securityfs none /sys/kernel/security; apt-get -y install maas-dhcp"'
<bbcmicrocomputer> and that would work
<roaksoax> bbcmicrocomputer: right, so have you looked at any other package that can be installed during the installed that uses apparmor_parser?
<bbcmicrocomputer> roaksoax: nope
<roaksoax> bbcmicrocomputer: because i'm pretty sure there are other packages that are installed during the installer that install an apparmor profiel
<roaksoax> so i don't think that would be a cause of issues
<bbcmicrocomputer> ok
<roaksoax> bbcmicrocomputer: let me see what's happening in that postinst though
<bbcmicrocomputer> ok, thanks
<roaksoax> bbcmicrocomputer: taking for ever to download the latest ISO
<roaksoax> bbcmicrocomputer: is it possible for you to quickly edit the postinst script
<roaksoax> bbcmicrocomputer: before it gets run and add a set -x ?
<roaksoax> so it outputs what's actually doing?
<bbcmicrocomputer> roaksoax: sure, I figure something
<roaksoax> bbcmicrocomputer: thank you!
<roaksoax> bbcmicrocomputer: so I'm pretty sure this would avoid the issue on the cd install: http://paste.ubuntu.com/6209795/
<roaksoax> bbcmicrocomputer: but dunno what the effect would be
<bbcmicrocomputer> roaksoax: I can try, I'm just collecting the previous output :)
<bbcmicrocomputer> roaksoax: the install cycle takes 10 mins on my machine :(
<roaksoax> boomer
<roaksoax> bbcmicrocomputer: that seems to be the correct fix. bind9 for example does this: debian/bind9.postinst:        apparmor_parser -r "$APP_PROFILE" || true
<roaksoax> bbcmicrocomputer: so adding the|| true would allow this to work on CD isntall
<bbcmicrocomputer> roaksoax: yep
<bbcmicrocomputer> roaksoax: I'll try
<roaksoax> just confirmed the theory with jdstrand
<roaksoax> so I'll make the fix
<bbcmicrocomputer> roaksoax: I've added the output to the ticket
<bbcmicrocomputer> debug output, which gives up on that line, yes
<roaksoax> bbcmicrocomputer: ok, i just tested it
<roaksoax> bbcmicrocomputer: it works
<bbcmicrocomputer> roaksoax: cool
<bbcmicrocomputer> roaksoax: how long will it take for that fixed package to make it into the archive?
<roaksoax> bbcmicrocomputer: tomorrow
<roaksoax> bbcmicrocomputer: i'm not gonna upload anything until the final changes to trunk land
<bbcmicrocomputer> roaksoax: ok, cool, thanks
<stokachu> does maas utilize fast-path installation yet?
<stokachu> i see python-curtin
<stokachu> but there aren't any preseed template defined for it
<stokachu> i see curtin and curtin_userdata should i just extend userdata to contain post_Scripts?
<stokachu> has curtin/xinstall been tested with preseed_xinstall+generic?
<stokachu> so far its not picking up the preseeds
<roaksoax> stokachu: yes it has
<stokachu> roaksoax: i see where curtin_userdata has a spot for late_commands
<stokachu> but i dont see how preseed_xinstall is coming into play
<roaksoax> stokachu: it does, i haven't test it
<roaksoax> stokachu: preseed_xinstall is only a "preview"
<stokachu> so whats the format of curtin_userdata if i wanted to add multiple commands
<roaksoax> stokachu: you'd need to check with smoser :) I haven't got that far just yet
<stokachu> roaksoax: http://paste.ubuntu.com/6210689/
<stokachu> ok
<roaksoax> stokachu: but judging for the pb, i'd say something like: http://pastebin.ubuntu.com/6210707/
<stokachu> roaksoax: gotcha trying that now
<stokachu> hmm tried something like http://paste.ubuntu.com/6210716/ and it didn't work.. ill just wait for smoser to come back
<smoser> stokachu, here.
<stokachu> :)
<smoser> you rad too far
<smoser> preseed_xinstall is not invovled period.
<stokachu> that was from the docs on maas.ubuntu.com
<stokachu> http://maas.ubuntu.com/docs/development/preseeds.html
<smoser> k. docs suck.
<stokachu> smoser: so ive been messing with curtin_userdata
<stokachu> http://paste.ubuntu.com/6210716/ trying to get late_commands to work
<stokachu> fastpath installs and it's definately fast
<stokachu> 12 seconds
<smoser> right. so curtin_userdata is what you'd modify.
<smoser> what do you need to do
<stokachu> http://paste.ubuntu.com/6210716/ check that snippet
<stokachu> just want to download and run a shells cript in the late command
<stokachu> i was using http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/view/head:/doc/topics/overview.rst as a reference
<smoser> stokachu, right. thts not unreasonable.
<stokachu> smoser: is my config correct with the late_commands?
<smoser> stokachu, yeah, its not unreasonable.
<smoser> http://paste.ubuntu.com/6210737/
<smoser> might work also
<stokachu> smoser: nice let me try that
<smoser> you're just replacing the builtin "configure network" command
<smoser> which is
<smoser> 'network_commands': {'builtin': ['curtin', 'net-meta', 'auto']}
<smoser> stokachu, are you getting installs ?
<smoser> your 12 seconds doesn't seem reasonble.
<stokachu> smoser: so it installs and i can ssh to it
<stokachu> and i create a file to make sure it is destroyed on new node installs and its gone
<smoser> wow.
<smoser> well thats fast!
<stokachu> the logs indicate that its pulling the curtin images
<stokachu> dude its super fast... 25minutes using the old way, 12 seconds using curtin
<stokachu> keep in mind this is in a kvm though
<smoser> yeah.
<stokachu> so bare metal may be different
<smoser> and with unsafeio also
<stokachu> yea
<smoser> still i hadn't seen that fast.
<stokachu> :)
<stokachu> trying the install now to see if the network gets picked up
<smoser> the goal is for when maas knows about networking is for it to do basically what i told you to do there.
<stokachu> smoser: http://imgur.com/ytjNstd
<smoser> and it didn't land in saucy, but the goal was/is also to have a 'user-install-data' snippet.
<smoser> that can be provided to maas.
<stokachu> ah ok
<smoser> and it goes in as a config to curtin
<smoser> so you'd no thave to be able to modify the preseed, and could change your mind on every install.
<stokachu> thats awesome
<stokachu> the install didn't pick up the vlansetup script i have though :(
<smoser> you should configure console output on your vms.
<smoser> hm..
<smoser> serial console output log to a file
<smoser> is what i meant.
<stokachu> http://paste.ubuntu.com/6210772/ thats my script
<smoser> so you dont have ot deal with crappy vnc for watching
<stokachu> smoser: yea i am going to set that up today as well
<smoser> oh. well, fwiw, what i gave you above was not to be executed.
<smoser> just to be downloaded.
<stokachu> ah
<smoser> that would be what you have in /etc/network/interfaces
<smoser> :)
<stokachu> ahh
<stokachu> is $OUTPUT_INTERFACES defined or should i change that to /etc/network/interfaces?
<smoser> OUTPUT_INTERFACES should be defined by curtin when it executes the command.
<stokachu> ok
<smoser> to execute, something like:
<smoser>  mine: ['sh', '-c', 'wget "$1" -O - | sh', '--', 'http://192.168.122.206/vlansetup']
<stokachu> smoser: thanks ill try that
<smoser> but you'll have to change your script so that it writes to $OUTPUT_INTERFACES
<smoser> or
<smoser> yeah, you'll have ot do that reasonably. writing to /target/etc/network/interfaces wont work
<smoser> for 2 reasons
<smoser> a.) i dont think it will be mounted at target
<smoser> b.) the i think it'd get copied over. (or might fail later if there i sno OUTPUT_INTERFACES)
<stokachu> ok thats cool, ill mess around with it some more today and see what i can come up with
<stokachu> at least this gets me in the right spot for curtin
<stokachu> smoser: thanks :)
<smoser> no, clearly your devcycle is pretty fast here.
<smoser> now.
<smoser> but if you want, you can use curtin trunk there is a 'launch' that will boot kvm install to a disk and stop
<stokachu> smoser: niceee that would be handy
<smoser> but it doesn't reboot yet :)
<stokachu> thats ok it would still easier
<smoser> see doc/devel/README.txt
<smoser> and feel free to add a '--then-boot-it-dummy' flag
<roaksoax> smoser: when are we going to have saucy images?
<stokachu> cool ill check that out today too
<smoser> roaksoax, saucy images ephemeral ?
<smoser> they're there.
<roaksoax> smoser: released?
<roaksoax> smoser: or daily's?
<smoser> http://paste.ubuntu.com/6210816/
<roaksoax> smoser: cool thanks
<roaksoax> smoser: btw... so we need to decide what to do with the moonshot and maas-enlist and ipmitool
<stokachu> in my /etc/network/interfaces file after a node start i see ## This file is generated by cloud-initramfs-dyn-netconf
<stokachu> could that be overriding my customizations?
<lifeless> ooh dynamic network confic
<lifeless> smoser: ^ is that new ?
<smoser> lifeless, /etc/network/interfaces.d is new in saucy.
<smoser> that config there is for 'curtin' which is the "fast path installer" in saucy.
<lifeless> https://git.openstack.org/cgit/openstack/diskimage-builder/tree/elements/dhcp-all-interfaces
<lifeless> is a related thing we did
<smoser> yeah, curtin does that too.
<smoser> its kind of garbage though
<smoser> as dhcp eth0
<smoser> then wait some length of time
<smoser> which might be to short
<smoser> kinda sucks.
<lifeless> yeah
<smoser> http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/view/head:/curtin/net/__init__.py
<smoser> 'is_connected'  at least is sane.
<smoser> stokachu, progress ?
<stokachu> smoser: everytime i boot the node it just seems like /etc/network/interfaces is unchanged
<stokachu> i changed the script to output a interfaces file
<stokachu> and i added a config_hook
<smoser> hm..
<smoser> i'll play some
<stokachu> http://paste.ubuntu.com/6211001/
<stokachu> thats what i have so far
<stokachu> http://paste.ubuntu.com/6211003/ thats my interfaces file
<smoser> hm..
<stokachu> am i supposed to copy the /etc/network/interfaces file in the config_hook?
<stokachu> docs suggest that unless told so it just keeps the interfaces file in memory
<smoser> curtin shoudl copy OUTPUT_INTERFACES to the target /etc/network/interfaces
<stokachu> automatically right?
<smoser> right.
<stokachu> hmmm
<stokachu> smoser: could it be and then getting overwritten by cloud-initramfs-dyn-netconf?
<smoser> it shouldn't. that only wirtes /run/interfaces
<smoser> right?
<stokachu> it added the comment in /etc/network/interfaces
<smoser> hm..
<smoser> actually, no.
<smoser> that came from the ephemeral image boot
<smoser> and was the copied to OUTPUT_INTERFACES and then copied back in i think.
<stokachu> so that would've been earlier in the install right?
<smoser> let me see.
<smoser> ok.
<smoser> so wrt http://paste.ubuntu.com/6211001/
<smoser> i'm not sure why netowrk_commands wouldn' twork.
<stokachu> ok
<smoser> config_hook is wrong if thats what doc says
<stokachu> smoser: it wasn't really clear to me how to use it
<stokachu> config_hook: {{TARGET_MP}}/opt/curtin/config-hook
<stokachu> but then it talks about helpers
<stokachu> so wasn't sure the syntax
<smoser> yeah, so that probalby just rong
<smoser> you should be able to do:
<smoser> hook_commands:
<smoser>  myhook: [sh, -c, 'something']
<smoser> ['curtin in-target echo "8021q" >> /etc/modules']
<smoser> is probably wrong though
<stokachu> ok, i didnt see any documentation for hook_commands
<smoser> htat will try to execute a program called 'curtin in-target echo "8021q" >> /etc/modules'
<stokachu> there is final_commands
<smoser> stokachu, documentation lags a bit.
<stokachu> ok
<smoser> final_commands replaced by late_commands.
<stokachu> gotcha
<smoser> stokachu, i'm sorry that doc is behind.
<smoser> i have a system set up and i'll try to figure out a working ocnfig to do what you want.
<stokachu> smoser: thats ok, i plan on submitting MP's
<stokachu> smoser: i'd appreciate it, ill be hacking away at it in parallel
<stokachu> smoser: fwiw i tested the hook_commands and didn't see "8021q" added in /etc/modules
<smoser> well, see above.
<smoser> it wouldnt work like that.
<smoser> it should have failed
<smoser> 'curtin in-target echo "8021q" >> /etc/modules'
<smoser> not: ['curtin in-target echo "8021q" >> /etc/modules']
<stokachu> ok
<smoser> the first i think will get invoked under 'sh' as a convenience
<smoser> but if the value is an array
<smoser> then it will invoke it with execve
<smoser> which is going to try to run 'curtin in-target echo....' as a program in the PATH.
<smoser> ie, not with sh.
<stokachu> ah ok
<smoser> stokachu, i'm out for a bit. i'll be back in sometime later.
<stokachu> smoser: no worries
<stokachu> thanks again
#maas 2013-10-09
<smoser> stokachu, still there
<smoser> ?
<smoser> http://paste.ubuntu.com/6211669/
<smoser> so that "works for me". and I tested by following doc/devel/README.txt
<smoser> and my 'launch' command
<smoser> ./tools/launch $BOOTIMG --publish $ROOTTGZ --add config -- curtin -v install --config config/my-network.conf "PUBURL/${ROOTTGZ##*/}"
<roaksoax> bigjools: if you can review the moonshot stuff would be greatlt appreciated
<bigjools> roaksoax: I am overloaded with stuff at the moment, I will ask jtv to look
<jtv> What needs reviewing?
<roaksoax> thanks
<roaksoax> lp:~andreserl/maas/moonshot_enablement
<jtv> OK
<roaksoax> jtv btw is maas-import-ephemerals done?
<jtv> Just about.  We need to back up the old pserv.yaml before overwriting it, the "arches" setting needs to be in a place where it can be shared between the import scripts, and some settings can do with clearer names.  But that's all easy stuff.
<jtv> With those changes in place, we can land the switchover branch that is already pending.
<roaksoax> awesome
<roaksoax> jtv: thanks for the hardwork
<jtv> All in a good cause!
<jtv> Oh, I see RaphaÃ«l has the back-up-pserv.yaml branch up for review already.
<jtv> Excuse me while I do reviews...
<jtv> Argh.  Nope.  That looks buggy.  OK, so back to plan A.  :)
<jtv> Nope, nope, it works.  Go Raphers!
<jtv> roaksoax: that mechanism for including scripts in the userdata is really paying off, isn't it?
<roaksoax> jtv: yeah i love it...
<roaksoax> wanna put maas_enlist there when i rewrote it to pythin
<bigjools> can we bring maas-signal back there too?
<roaksoax> bigjools: xan you subscribe maas-maintainers to djorm-ext-pgarray .. kinda critical
<roaksoax> bigjools: and maas signal has always been in maas the reason why we ship itnwith maas-enlist is for end user usage on remote nodes
<bigjools> yes I was getting mixed up
<bigjools> roaksoax: what do you need the subscription for?
<roaksoax> bigjools: it is already.. to be able to promote the package to main
<roaksoax> it needs bug subscriber
<bigjools> roaksoax: I don't understand what you want
<roaksoax> bigjools: nevermind it is done already
<roaksoax> bigjools: but add maas-maintainers as a subscriber for launchpad.net/ubuntu/+source/curtin
<bigjools> roaksoax: bug subscriber you mean?
<bigjools> I will not add teams as subscribers to things as that's pretty obnoxious, people should subscribe themselves
 * jtv comes up from email, gasps for breath, goes under again
<roaksoax> bigjools: well maas-maintaoners maintain those packages
<roaksoax> not just me
<roaksoax> thats why it is subscribes
<bigjools> that's fine but team subscriptions suck
<roaksoax> thats a maas dependenxy that *we* maintain
<bigjools> I subscribed myself, feel free to ask the red guys
<roaksoax> as it is in ubuntu only for maas
<bigjools> now, which sprint track is maas in at SF ...
<jtv> roaksoax: bigjools also wrote some script to detect whether the host is physical or virtual...  He checked for qemu initially, but I think he found a more general way.
<bigjools> jtv: I didn't :)
<jtv> Oh blast.  Maybe it was allenap then.
<bigjools> there's a package called imvirt that we should use
<bigjools> jtv: well I wrote a commissioning script
<bigjools> but we need to generalise
<jtv> Found it.  roaksoax, have a look at src/metadataserver/models/commissioningscript.py and search for "notvirtual"
<jtv> I see now that it does mention qemu...
<jtv> Still, may be useful to compare, to see if we get consistent answers.
<stokachu> smoser: nice testing that now
<smoser> stokachu, oh. in-target wont work in your version
<smoser> :-(
<smoser> but this should
<smoser> http://paste.ubuntu.com/6212161/
<smoser> and i'm off to bed now.
<stokachu> smoser: thanks will test some more
<stokachu> interesting the web-ui doesn't update the nodes start/stop status if commissioning/starting nodes through maas-cli
<stokachu> and of course just after i type that it shows up
 * stokachu is tired apparently
<bigjools> jtv: we need to fix the Makefile so "make doc" does not try to rebuild the database.  Very annoying :/
<jtv> Urgent?
<rvba> jtv: did you merge lp:~rvb/maas/backup-config ?
<jtv> No...  I assumed you had.
<jtv> Because it's landed.
<rvba> Weird, I didn't
<rvba> jtv: is everything okay for me to land the branch that makes the switch to the new script?
<jtv> Not _quite_.  I need two simple reviews!
<rvba> Okay, I'll review them now.
<rvba> I really want us to do the switch so that I can do another round of testing.
<jtv> Thanks.  I'll be back by call time.
<jtv> Yes, quite.
<rvba> jtv: all reviewed, let's land!
<bigjools> rvba, jtv: apparently juju-core now supports maas-tags as constraints
<rvba> jtv: should I land the switch now?
<jtv> rvba: let me just check if my last branch landed â previous attempt failed because of the prerequisite.
<rvba> k
<jtv> Ah yes, that's landed.
<jtv> So go ahead!
 * rvba lands
<rvba> bigjools, jtv, allenap: All of the migration-related branches have landedâ¦ and the branch that switches to the new script has landed as wellâ¦ a package is being built in the daily ppaâ¦ test away!
<allenap> jtv: Thanks for looking at that PostgreSQL thing. I suspect we just take the hit first time around.
<jtv> allenap: I don't know much about the situation, but it seems reasonable...  Or maybe there's some heuristic that will get most of the rows you want to delete.
<rvba> bigjools, allenap, jtv: the package is ready.  fresh outta the oven.
<jtv> Does it work?
<allenap> rvba: What jtv said.
<rvba> Well, we all need to test it!
<jtv> And is this in the daily PPA?
<rvba> jtv: yes
<bigjools> allenap: have you got a quick example of how to use the LLDP constraints?
<rvba> bigjools: do you have time to have a look at: https://code.launchpad.net/~rvb/maas/fpi-button/+merge/190049
<bigjools> sure
<rvba> ?
<rvba> Ta
<bigjools> rvba: s/curtin/fast installer/ on the buttons
<rvba> bigjools: well, I thought you were okay with bothâ¦ I choose curtin because it is shorter.
<bigjools> rvba: it's too jargon-y for the UI
<allenap> bigjools: From Juju, no; Dan said he was going to talk to mramm about getting someone to sort that out on their side.
<bigjools> allenap: they said it was working I hear
<rvba> bigjools: then we need to re-work the wordings entirely, otherwise it won't fit
<allenap> bigjools: With regards to a tag, yes, I'll find something...
<bigjools> allenap: cheers
<bigjools> rvba: how many characters do we have?
<rvba> bigjools: let's say that what we have now fits.
<bigjools> ok but new people are going to say "wtf is curtin?"
<rvba> And we don't have much more space.
<bigjools> dude!
<bigjools> display = "Use the default installer"
<bigjools> is longer :)
<rvba> bigjools: the critical piece is display_bulk.
<bigjools> display_bulk = "Mark the selected nodes as using the default installer"
<bigjools> s/default/fast/ in the other button
<rvba> I'm testing that right now.
<bigjools> ok
<rvba> bigjools: s/default/fast/ in the other button?
<rvba> d-i is not fast :)
<bigjools> the UseCurtin one
<bigjools> you already have something *longer* for the d-i button
<rvba> "Use the fast path installer" does not fit on the node's page.
<bigjools> s/path// then
<rvba> "Use the default installer" fits
<bigjools> so two options:
<bigjools> "Use the fast installer"
<bigjools> and
<bigjools> "Use the default instaler"
<bigjools> ok?
<rvba> Yeah, that's the best option.
<bigjools> great
<rvba> Mark the selected nodes as using the fast installer/Mark the selected nodes as using the default installer
<rvba> That fits.  It's a bit tight but okay.
<bigjools> FWIW I got caught out the other week when I assumed should_use_fastpath_installer was a property :(
<allenap> bigjools: //lldp:chassis/lldp:id[@type="mac"]/text() = "20:4e:7f:94:2e:10"
<allenap> would select a machine that's connected to the router with that MAC.
 * bigjools blinks
<bigjools> remind me which API field...
<allenap> bigjools: Tag definition.
<bigjools> ok
<allenap> bigjools: routers are a special case for acquire(); it now accepts connected_to and not_connected_to arguments, which are MACs.
<allenap> Or multiple MACs.
<bigjools> allenap: nice
<allenap> bigjools: rvba did that bit.
<bigjools> allenap: this sounds like it needs its own doc
<bigjools> too much for release notes
<allenap> bigjools: Yep.
<bigjools> Ok when I land the release notes, someone can add a :ref: to the new doc
<rvba> bigjools: I pushed the change to my fpi-button branch.
<bigjools> rvba: should I be concerned that the ordering changed in test_views_nodes.py?
<bigjools> oh you changed the code being tested.   sorry
<rvba> bigjools: the ordering?  AFAIK I only changed the error messages.
<bigjools> yeah my bad
<bigjools> approvedÂ¬
<bigjools> approved!
<rvba> Ta
<rvba> Â¡Ta!
<bigjools> Â¡OlÃ©!
<rvba> allenap: I'll rebuild the package soon, if you want to land your lldp to be part of the build you can do so now (unless you want to tweak the UI some more before landing).
<bigjools> https://code.launchpad.net/~julian-edwards/maas/release-notes-13.10/+merge/190061
<bigjools> hello gmb
<gmb> Hi bigjools.
<gmb> bigjools: My account on Kanban appears to have been deactivatedâ¦ I'm assuming that wash;'t you and am contacting their help desk, but just to checkâ¦ ;)
<allenap> rvba: I probably won't be ready in time.
<bigjools> gmb: oh didn't anyone tell you yet?
 * bigjools joking
<gmb> Laugh? Nearly shat.
<bigjools> allenap: can you check my release notes branch please?
<allenap> bigjools: Sure.
<bigjools> gracias
<bigjools> or Â¡gracias!
<rvba> allenap: okay, no worries.  We can build a package quickly with https://code.launchpad.net/~julian-edwards/+recipe/maas-daily-saucy so just build a package when you land you branch and then we'll test it.
<rvba> your* branch
<bigjools> gmb: you're still marked as a board user for the maas board so that's all very odd
<gmb> bigjools: Hmmâ¦ sounds like some kind of password / authentication snafu. ISTR this happening earlier this year to awd and domas on the green squad board, actually.
<bigjools> Â¿yay?
<gmb> bigjools: If there were a phrase to accurately confer "We are about the only corporate users of that particular half-maintained web app," then I would use that. Â¿yay? will have to do.
<bigjools> I thought there were many users
<bigjools> anyway time to put kids to bed, back in a bit
<jtv> rvba: the new ephemerals script doesn't care.
<jtv> It'll just take releases that are in the stream, and optionally filter those.
<jtv> While maas-import-pxe-files won't care anymore because it  skips failing downloads.
<rvba> jtv: okay, makes sense.
<allenap> rvba: Do you think you'll have time to review https://code.launchpad.net/~allenap/maas/lldp-commissioning-results-in-ui/+merge/189663 again today?
<rvba> allenap: sureâ¦ is it much different from the branch Jeroen reviewed?
<allenap> rvba: Yes, it's almost all new.
<rvba> allenap: okay, I'll have a lookâ¦ could you provide a screenshot of the page?
<rvba> jtv: allenap: could any of you take care of fixing ./etc/maas/import_pxe_files please?
<jtv> rvba: by removing the RELEASES setting?
<rvba> jtv: that or adding Saucy.
<jtv> OK
<jtv> Testing is going to be hard though.  :(
<jtv> I have a large instance, but it's not large enough.
<rvba> You need to symblink to /mnt/
<rvba> symlink*
<rvba> Plenty of space there right?
<jtv> Oh, that worked for a "large"!?
<rvba> Yeah.
<jtv> Great.
<jtv> Oh dear
<jtv> I think our config conversion is broken...
<rvba> :/
<jtv> Need a different way to get the variables.
<rvba> allenap: I'm a bit confused by merge_details_cleanly/merge_details.  Does the usage of merge_details_cleanly() mean the change in backward incompatible?
<rvba> is*
<allenap> rvba: http://people.canonical.com/~gavin/ui/lldp-commissioning-results-in-ui/ for screenshots.
<rvba> Thanks.
<allenap> rvba: merge_details_cleanly() generates the XML without using the lshw details as the document root (so that they're in there once without a namespace, and once in the 'lshw' namespace).
<allenap> rvba: When tagging you get both, so that old expressions work.
<allenap> rvba: But we want to encourage people to use only the namespaced XML from here on.
<rvba> allenap: okay, thanks; makes sense.
<allenap> rvba: Do you think I should add highlighting support to the XML? It's very easy with highlight.js, and looks good.
<rvba> allenap: look like a nice addition, but probably not critical for today :)
<rvba> allenap: I can use a little help with the testing ;)
<allenap> rvba: Okeydoke :)
<rvba> allenap: jtv seems to be busy with the migration stuff, if you could fix (and, most importantly test) ./etc/maas/import_pxe_files, that would be one thing off the list.
<jtv> allenap, rvba â either of you willing to review a simple branch?  https://code.launchpad.net/~jtv/maas/use-unexported-config-vars/+merge/190089
<jtv> Yeah, this is distracting me from the /etc/maas/import_pxe_files change I'm afraid.
<jtv> I've got another thing to fix here.
<rvba> jtv: I'll take it.
<jtv> Merci beaucoup.
<allenap> jtv: I have one small suggestion for the env branch, hang on...
 * jtv hangs on
<allenap> jtv: Done, on the mp itself.
<jtv> OK
<jtv> Thanks
<allenap> rvba: Can I hit the build button again?
<rvba> allenap: go! :)
<jtv> allenap: I added the quoting, and will sleep better because of it.  But not the "&&," because AFAICT the script never ran with "-e" in the first place.
<allenap> jtv: Okay, cool.
 * allenap wonders why bash scripts don't default to -eu ...
<jtv> Because dried grapes are hilarious.
<allenap> jtv: Que?
<jtv> Wait... what was that other word for dried grapes?
<allenap> jtv: raisins... reasons...?
<jtv> And I don't mean hilarious...  I mean that other word that's sometimes used instead.
<allenap> jtv: depressing?
<jtv> Normally means something like insane, loudly irrational...
<jtv> Ah yes.  Hysterical.
<jtv> Raisins are hysterical.
 * allenap goes to have lunch because that's normal.
<jtv> In Unix shell tradition, bash is an interactive shell.  You don't want it to exit on the first mistake.
<jtv> Had it been developed purely as a batch shell, things might have been different...
<jtv> dash is more like that.
<jtv> In that it's not very accommodating to humans.
<allenap> bash accommodates all human error.
<jtv> Exactly.
<jtv> Because it's meant to support interactive use.
<jtv> Try running your terminals with "bash -e"...
<jtv> Actually, that might be a good way to train novices.
<jtv> Direct, merciless, effective.
<jtv> Drop and give me 50!
<rvba> roaksoax: we landed the branch that switches from the old script to the new script.  Testing is in progress but it's looking okay so farâ¦ could you test upgrading the package from older versions?  Something weird happened, I upgraded (using our daily ppa) from a package built one hour ago to a package built 10 minutes ago (the difference was only 2 revisions) and after the upgrade the 'maas-cluster-celery'
<rvba> service was down.
<jtv> allenap, rvba: another small branch needs reviewing...  If you'd be so kind: https://code.launchpad.net/~jtv/maas/load-immutable-config/+merge/190095
<jtv> This is the other fix for the config conversion.
<jtv> roaksoax: is maas-cluster-controller really supposed to depend on maas-dhcp?
<jtv> Isn't that a Recommends?
<rvba> jtv: I'll review your branch
<jtv> Thanks.
<jtv> Having a bit of trouble with the latest packaging branch though: can't install maas-cluster-controller without also installing the DHCP server!
<rvba> I think roaksoax told me they wanted to make maas-dns and maas-dhcp mandatory and not optional.
<rvba> roaksoax will confirm this.
<jtv> Isn't that going to wreak havoc on people just trying it out in small networks?
<rvba> No really, if you don't configure an interface to be "managed" the dhcp server won't be started.
<rvba> Not* really
<jtv> Phew.
<jtv> Thanks allenap & rvba for your reviews.  With this landed, the config conversion works.
<allenap> rvba: The package looks like it's used revno 1676, which is from yesterday. What am I missing?
<rvba> 1.4+bzr1676+dfsg-0+1689+209~ppa0~ubuntu13.10.1
<rvba> "1.4+bzr1676+dfsg-0" comes from the packaging.
<rvba> 1689 is the upstream revision used.
<rvba> 209 is the packaging revision.
<allenap> rvba: No one would believe me if I said packaging is arcane :)
<rvba> allenap: look at the recipe: it uses a funky pattern to create the package name.
<rvba> allenap: {debupstream}-0+{revno}+{revno:packaging}~ppa0
<allenap> rvba: And {debupstream} is where the weird bit comes from.
<rvba> allenap: yeah, that comes from the packaging's first line of debian/changelog.
<allenap> smoser: You're logged into the garage maas. Are you using it?
<jtv> allenap, rvba: I'm done as of r1689.  Unless there's anything urgent that either of you need help with.
<stokachu> smoser: filed bug 1237215
<ubot5> bug 1237215 in curtin "late_commands in maas provision node fails" [Undecided,New] https://launchpad.net/bugs/1237215
<smoser> allenap, i have a node in use, why?
<allenap> smoser: I want to test the latest daily package.
<smoser> allenap, garage maas really isn't the place to do that.
<allenap> jtv: Nope, I'm good :)
<allenap> smoser: Why not?
<smoser> because people expect for it to work.
<rvba> jtv: I'm good too :).
<smoser> cts people are expecting to be using that this week.
<allenap> smoser: I thought the deal was: you put your name in the calendar, then it's yours, and you leave it in the state you found it (or working, if it wasn't).
<allenap> smoser: My name's in the calendar :)
<smoser> allenap, i don't really think "go crazy and put it back" is a sustainable plan on a shared resource.
<allenap> smoser: What's it there for then?
<smoser> use of maas.
<smoser> not development of maas.
<smoser> how will you put it back to the state which you got it ?
<smoser> do you know the state that its in?
<smoser> do the packages correctly downgrade?
<smoser> how can you reasonably expect to "put it back the way you got it"
<smoser> honest question.
<matsubara> allenap, you can use the QA lab to do this sort of testing. it's supposed to be used like that, you use and throw away, next person  (or automated test) will re-build the whole thing anyway)
<allenap> smoser: I've not used the garage MAAS in anger before, so I would expect someone to have documented this by now. I know I've seen it broken more times than I've seen it working, but that's perhaps because I get asked to look with it is broken.
<allenap> s/with/when/
<smoser> you've seen it broken because of what i'm saying above.
<smoser> so my solution is not to let people break it.
<rvba> matsubara: I'm currently testing things in the lab, the idea was to parallelize the testing.
<matsubara> rvba, I see. ok then. I'll let you guys fight for the resource then :-)
<smoser> i'm perfectly fine if we can come up with a "put it back into known state" button.
<smoser> but we dont have that now
<smoser> and others will expect to be able to use that shared resource.
<smoser> i'd love a "redeploy garage maas" button.
<caribou_> Has the functionality in 1.2 to add archives for newly provisioned machines been dropped in Maas 1.4 ?
<smoser> rvba, https://bugs.launchpad.net/curtin/+bug/1237215
<ubot5> Ubuntu bug 1237215 in curtin "late_commands in maas provision node fails" [Undecided,New]
<stokachu> caribou_: boot images you mean?
<smoser> hm..
<caribou_> stokachu: nope, there is a button in the config section to add entries to the sources.list; it's no longer there  in 1.4
<stokachu> ah
<rvba> smoser: is the curtin installer supposed to work on ARM nodes?
<smoser> no.
<rvba> Okay. Thanks.
<smoser> it has no way of setting up bootloading on arm nodes (unimplemented).
<smoser> rbasak, could add that.
<allenap> rvba: MAAS supports one Juju environment per *user*, is that right, not per API key?
<rvba> allenap: that's how it's supposed to work indeed.  But there is that bug we talked about recently in juju-core that makes the maas provider take down *all* the instances it can get his hands on when bootstrapping an env.
<allenap> rvba: I'm thinking about the user preferences page, where it says "You'll need a separate MAAS key for each Juju environment".
<allenap> rvba: That's not just misleading, that's wrong :-/
<rvba> allenap: it's both indeed.
<allenap> rvba: I shall file a bug for it.
<smoser> allenap, i think that the real blocker is if you're admin its a global space
<smoser> regular users can be separated, but if an admin tries to use juju, he'll wipe out all nodes
<danwest> hi all, know today is going to be tough but I was curious if anyone has tested juju-core, maas tag constraints yet?
<allenap> smoser: I still thought that an admin would only destroy those nodes allocated to him/her. The problem is that Juju destroys nodes that have been allocated independently of Juju.
<allenap> danwest: Not yet.
<danwest> my understanding is that for hostname placement you need to create a tag with the hostname in it
<allenap> danwest: Speaking for me.
<danwest> which just seems wrong
<allenap> danwest: Eugh, I hope not.
<danwest> allenap: going to ping juju folks to check if they know
<smoser> allenap, well, i thought the bug was that if you juju bootstraped then juju would release all nodes that it could release.
<smoser> and as admin that is "all nodes"
<allenap> smoser: I'll check.
<danwest> allenap: confirmed with juju-core folks that new maas constraints would indeed require a hostname tag to use hostname as a constraint
<danwest> allenap: which means to make it 'just work' we would need to auto-gen a hostname tag but concerned that is unmaintainable
<allenap> smoser: You're right. That's fairly awful.
<allenap> danwest: It should be fixed in juju-core.
<danwest> allenap: agreed but I fear there are arguments against it, at a minimum we need to document this behavior/requirement in maas release notes
<allenap> rvba: In juju-core, maasEnviron.AllInstances (and therefore maasEnviron.instances) should not return all instances when handed an empty slice, it should return only allocated instances. Does that sound correct?
<allenap> danwest: Yeah, there's nothing we can do to fix that in time for release, other than documenting it.
<danwest> agreed, thx
<allenap> danwest: Do you know who's looking after the release notes for MAAS?
<danwest> allenap: thought it was bigjools, no?
<allenap> danwest: Of course, yes. I'll email him.
<danwest> roaksoax: ^^?
<smoser> the key release note there is "do not be admin"
<rvba> allenap: yes, but instances *allocated to this environment*.
<smoser> one thing that sucks about "do not be admin" is that there is no way that i'm aware of to create a user that is non-admin
<smoser> other than the GUI (which i dont consider useful)
<allenap> rvba: One step at a time :) Firstly we just want to only select instances allocated to the calling user, not just all instances.
<rvba> allenap: That's correct.
<allenap> rvba: Cool. I might see about fixing that then.
<roaksoax> danwest: bigjools is
<danwest> roaksoax: thx
<roaksoax> rvba: so ... did you guys land anything that would affect commissioning? i tested it yesterday with saucy on amd64 with no issues whatsoever
<rvba> roaksoax: we didn't change anything in this area.
<roaksoax> rvba: then maybe something with the image you used?
<rvba> roaksoax: I don't know, I tested using a fresh instance and AFAIK, we check the md5sum of the images we download.
<rvba> roaksoax: but I'm happy to test again.
<roaksoax> rvba: sure! ill also test but did yesterday with my moonshot enablement and worked just fine
<rvba> roaksoax: I just tested again, same result: "Failed tests,"
<roaksoax> k
<roaksoax> rvba: is there console output of the commissioning?
<rvba> roaksoax: no
<roaksoax> rvba: well i think it would be a good idea to obtain it
<rvba> roaksoax: no sure how to do that in the lab
<rvba> not*
<roaksoax> rvba: maybe the new import ephemerals has something to do with it
<rvba> roaksoax: possiblyâ¦ let's see if you see the same problemâ¦
<stokachu> smoser: http://paste.ubuntu.com/6213826/ everything works except for the network_commands, i was hoping to just append to the existing /etc/network/interfaces but im assuming it doesn't work like that?
<smoser> stokachu, well, theres not a lot of value in appending i dont think
<smoser> but if you want, just don't override builtin
<smoser> and make your entry run after builtin
<smoser> i think you will run before builtin as it is, and it would overwrite yours.
<stokachu> so would putting it in late_commands solve this?
<smoser> have to look. i wish i'd have made 'builtin' '00builtin'
<smoser> but i think if you just chainge '102_add_vlans' to 'c102_add_vlans' it'll be fine.
<smoser> oh, and remove your overwriting of builtin
<stokachu> ok ill try both ways and see what happens
<smoser> stokachu, you can definitely do it in late_commands.
<smoser> that paste bin doens't look right hough
<smoser> you dont have a clsing ] on 'maas:'
<stokachu> smoser: yea it chopped it off when copying from vi
<stokachu> i verified its there
<rvba> roaksoax: btw, we need to re-add "saucy" in etc/maas/import_pxe_files.
<roaksoax> yup
<stokachu> smoser: here is what is currently working http://paste.ubuntu.com/6213897/
<stokachu> it adds the vlan information into /etc/network/interfaces now
<smoser> i wish i had done a archive format for config.
<smoser> such that you could put your config changes separate from maas's stuff.
<smoser> so you didn't have to modify maas's late_command
<smoser> but could just add your own
<smoser> you can do that with multiple configs, but you only have one  config here.
<stokachu> yea so far just one
<smoser> what i should have done (and will do in the future) is support an archive format for config.
<smoser> ie, 'cloud-config-archive' format.
<smoser> so that woudl be:
<smoser> #curtin-config-archive
<smoser> - |
<smoser>   maas stuff here
<smoser> - |
<smoser>   stokachu's stuff here
<stokachu> that would be sweet
<smoser> jamespage, are you around ?
<AskUbuntu> MaaS dhcp cannot be detected at node installation | http://askubuntu.com/q/355847
<smoser> roaksoax, let me know if there is anything i can do to help you
<roaksoax> smoser: will do once I give this a test :)
<roaksoax> rvba: ok, so i'm building a package I;'ll use to test the moonshot stuff + the commissioning stuff
<roaksoax> rvba: other than that, how many more branches are we expecting to land?
<roaksoax> rvba: ok this is definitely something new and weird
<roaksoax> in my test cluster from yesterday i had machines commissioned
<roaksoax> all fine
<roaksoax> now I look at show "failed tests"
<roaksoax> smoser: "ci-info: no authorized ssh keys fingerprints found for user ubuntu.ec2:" ??
<smoser> probably not a big deal.
<smoser> thats ephenmeral boot?
<smoser> or install
<roaksoax> smoser: yeah, commissioning.. and commissioning fails
<smoser> well thats unrelated to commissionoing failure
<smoser> its just telling you that the user it configured did not have ssh keys
<smoser> ie, you cant get in here :)
<roaksoax> well then that's probably why
<roaksoax> it is failing
<roaksoax> so something must have changed...?
<rvba> roaksoax: the only thing we need to fix is /etc/maas/import_pxe_files.
<roaksoax> rvba: ok
<rvba> roaksoax: the cleanest way to do it is probably to remove the RELEASE variable entirely.  This way, we will download all the supported releases (the script now copes with missing images).
<roaksoax> rvba: if the scripts do that then it would be fine, but keep in mind it might not work on precise (since precise maybe doesn't know that saucy is a relese yet but rather development)
<roaksoax> smoser: ok
<roaksoax> smoser: so I see this issue again
<roaksoax> ci-info: no authorized ssh keys fingerprints found for user ubuntu.ec2:
<roaksoax> the machine failed to enlist
<smoser> that is unrelated
<rvba> roaksoax: you're talking about the backport right?
<smoser> i dont know why you're not enlisting. but it has nothing to do with the fact that the user has no keys.
<roaksoax> smoser: that's what I'll check in a bit
<roaksoax> rvba: yup
<rvba> roaksoax: good point, I'll just add "saucy"Â to the list then.
<rvba> roaksoax: on second thoughts, aren't we support to SRU distro-info everytime a series is released?
<rvba> roaksoax: I mean distro-info --supported on precise currently returns l/p/q/r/s
<roaksoax> rvba: we do
<roaksoax> but what if it is not therE?
<roaksoax> wouldn't be the first time
<roaksoax> :)
<roaksoax> so I think we should go through the safe path
<roaksoax> either way i don't mind
<rvba> What do you mean "it is not there"?
<roaksoax> just dont remove RELEASES
<roaksoax> just comment it out
<rvba> If the image is not there?
<roaksoax> rvba: yeah if the image is not there it will fail
<rvba> roaksoax: no, the script deals gracefully with that now
<roaksoax> ok
<rvba> I'll test it, just for safety, and propose a fix.
<roaksoax> rvba: ok, so is enlisting also failing for you?
<roaksoax> rvba: i haven't even upgraded to the latest maas
<roaksoax> and enlist fails
<rvba> roaksoax: I haven't tested enlisting using saucy
<roaksoax> rvba: could you also try and see if it fails
<rvba> roaksoax: ok
<rvba> roaksoax: I just got 4 nodes enlisted using Saucy's (commissioning) image.
<roaksoax> rvba: ok, try to commission them
<rvba> roaksoax: that is the part that will failâ¦
<roaksoax> still failing?
<rvba> â¦still commissioningâ¦
<roaksoax> i can't even enlist a node... this is completely weird
<rvba> roaksoax: 4 "failed tests"
<rvba> roaksoax: https://code.launchpad.net/~rvb/maas/re-enable-saucy/+merge/190159
<roaksoax> rvba: ok maas-import-ephemerals doesn't seem to be working either
<rvba> roaksoax: !
<roaksoax> rvba: i run maas-import-pxe-files
<roaksoax> and only downloads the pxe images
<roaksoax> and that's it
<rvba> roaksoax: I'm running it in a canonistack instance right now and I'm seeing it installing the ephemeral imagesâ¦
<rvba> roaksoax: I'm using the daily package from our ppa
<roaksoax> hdl@vm18-2:~/roaksoax/backdoor-image$ sudo maas-import-ephemerals
<roaksoax> hdl@vm18-2:~/roaksoax/backdoor-image$
<roaksoax> rvba: and I upgraded
<roaksoax> yours might be a clean install?
<rvba> roaksoax: yes, clean install
<roaksoax> ok so there's something there that needs to be fixed for an upgrade maybe
<roaksoax> rvba: ok, you will need to connect to the image using ssh
<roaksoax> rvba: but you will need to enable the user/pass for it
<roaksoax> kinda following https://lists.launchpad.net/maas-devel/msg00808.html
<roaksoax> \but since I don't know where the images are stored now
<roaksoax> you'll need to figure it out
<roaksoax> rvba: and is this only in saucy? other releases works fine?
<rvba> roaksoax: yes
<roaksoax> rvba: ok, so just try to connect to the image
<roaksoax> and block is power off
<roaksoax> and we can look from there
<roaksoax> because here i can't even
<roaksoax> obtain the ephemerals
<roaksoax> rvba: is there no output when i run maas-import-ephemerals?
<roaksoax> rvba: if I ctrl+c the import
<roaksoax> i think it wont try to run it again
<rvba> roaksoax: there is usually plenty of output
<roaksoax> rvba: not for me i guess
<roaksoax> i don't see anything
<roaksoax> rvba: and these are upgrades
<roaksoax> so there's something messed up real bad
<roaksoax> i thought we had agreed that ephemerals config file would be *separated* from pserv.yaml?
<rvba> roaksoax: I think Julian decided otherwise
<roaksoax> bigjools: ^^
<roaksoax> rvba: show me a pserv.yaml with various releases enabled
<roaksoax> i really don't like this
<roaksoax> -_-'
<rvba> roaksoax: I don't have one handy (because I always install from fresh) but it should contain something like: http://paste.ubuntu.com/6214348/
<allenap> roaksoax: I'm working on a bson encoding problem that's preventing tags from being evaluated. I have to feed my kids now, but I'll be back to work on it soon. When has this got to be done by?
<roaksoax> allenap: tonight :)
<allenap> roaksoax: Cool :)
<adam_g> roaksoax, http://people.canonical.com/~agandelman/failed_metadata.png any iea?
<adam_g> this is during comissioning
<roaksoax> adam_g: we are looking at commissioning issues right now...
<roaksoax> adam_g: did you upgrade or just magically started happening?
<roaksoax> smoser: ^^
<adam_g> roaksoax, upgraded on monday and added two new declared nodes, those are not comissioning
<smoser> what OS booted for commissioning?
<roaksoax> adam_g: ok so i had nodes commissioning/enlistment working yesterday and today they magically stopped working, so might be related. I was using saucy for enlistment/commissioning though
<adam_g> http://paste.ubuntu.com/6214708/ is from apache log
<adam_g> (err thats maas log)
<adam_g> smoser, trying to figure out what booted, by logs
<adam_g> roaksoax, hmm. the node appeared to be commissioned okay even with those errors. appears ready in MAAS
<roaksoax> adam_g: uhmm is ipmi discovered? are the cpu's/ram/etc being displayed?
<adam_g> roaksoax, yup
<roaksoax> adam_g: then i guess it works just fine then :/
<AskUbuntu> how to stop entire openstack so that i can run maas server on my localhost | http://askubuntu.com/q/355914
<smoser> adam_g, thats your clock is broken
<smoser> and cloud-init was adjusting its timestamps to fix oauth failed
<jamespage> smoser, I am now - sorry - no internet access all day
<smoser> :)
<AskUbuntu> how to run openstack and maas server on localhost together | http://askubuntu.com/q/355931
<roaksoax> rvba: let me know when around
<roaksoax> [    0.000000] Command line: nomodeset iscsi_target_name=iqn.2004-05.com.ubuntu:maas:maas-precise-12.04-amd64-ephemeral-20121008 iscsi_target_ip=10.18.1.9 iscsi_target_port=3260 iscsi_initiator=maas-enlist ip=:::
<roaksoax> :maas-enlist:BOOTIF ro root=/dev/disk/by-path/ip-10.18.1.9:3260-iscsi-iqn.2004-05.com.ubuntu:maas:maas-precise-12.04-amd64-ephemeral-20121008-lun-1 overlayroot=tmpfs cloud-config-url=http://10.18.1.9/MAAS/metadat
<roaksoax> a/latest/enlist-preseed/?op=get_enlist_preseed log_host=10.18.1.9 log_port=514 console=ttyS0,9600 initrd=amd64/generic/precise/commissioning/initrd.gz BOOT_IMAGE=amd64/generic/precise/commissioning/linux BOOTIF=0
<roaksoax> 1-38-ea-a7-0f-6e-ba
<roaksoax> err
<roaksoax> sorry
<roaksoax> rvba: ok so i think we are in big trouble...
<roaksoax> allenap: ^^
<roaksoax> rvba: so basically, I selected saucy to be the release for commissioning, first it PXE booted into precise.., and when I noticed that I rebooted the machine, and it then pxe booted into saucy.
<allenap> roaksoax: So... what are we talking about?
<roaksoax> rvba: once enlisted, i told it to commission... however, it PXE booted into precise again (when it should have been saucy, and obviously commissinnig failed) so I told it to PXE boot again and it now did on saucy
<roaksoax> rvba: and commissioned in saucy, but with failed tests
<roaksoax> allenap: simple enlistment/commissioning
<allenap> roaksoax: Is the problem that the commissioning series keeps changing?
<roaksoax> allenap: yes, but apart from that commissioning finishes "successfully" but shows Failed Tests on the maas webui
<roaksoax> which is what rvba reported in the morning
<roaksoax> allenap: so if I tell maas to enlist/commission with saucy, in the first attempt uses precise and obviously fails to do it
<roaksoax> allenap: so I reboot the node, and the second attempt does use saucy
<roaksoax> which works and enlists/commissions
<allenap> roaksoax: How do you tell it to commission with saucy? Via the global option?
<roaksoax> allenap: yeah, on the webui settings "Default distro series used for commissioning"
<roaksoax> allenap: which is what i always have used
<roaksoax> allenap: and never had problems with
<roaksoax> allenap: so this is something recent... from yesterday to today
<allenap> roaksoax: Very odd :-/
<allenap> roaksoax: I'll look soon; right now I'm this >< close to figuring out a weird bson issue.
<roaksoax> probably related to maas-import-ephemerals, which is also being problematic
<allenap> roaksoax: Is it going to be a problem to prevent python-bson-ext from being installed at the same time as maas?
<allenap> It's a C module that behaves weirdly in Apache+WSGI.
<roaksoax> allenap: maybe, we cannot control ordering of installation
<allenap> roaksoax: See https://bugs.launchpad.net/maas/+bug/1237615 for the problem. Right, I'll look at this commissioning issue now.
<ubot5> Ubuntu bug 1237615 in MAAS "python-bson-ext does not encode binary in Apache with mod_wsgi" [Critical,Triaged]
<roaksoax> allenap: ok, I;ll have a look in a bit
<allenap> roaksoax: Thanks.
<stokachu> oo we should do a responsive layout so people can manage their servers from their phone/tablet!
<allenap> roaksoax: I think you've set the wrong target branch on https://code.launchpad.net/~andreserl/maas/maas_saucy_fix_commissioning/+merge/190239... I'm not reviewing 107234 lines :)
<roaksoax> allenap: lol let me see
<roaksoax> allenap: https://code.launchpad.net/~andreserl/maas/maas_saucy_fix_commissioning/+merge/190244
<roaksoax> done
<roaksoax> that should fix the failed trsts stuff
<roaksoax> but not the pxe booting intonprecise when it shpuld be saucy
<allenap> roaksoax: Yeah, I don't understand that bit at all.
<roaksoax> allenap: ill look at it
<roaksoax> thanks though!
<allenap> roaksoax: Credit to you. I'm off now, I'm going to use my last energy to get upstairs to bed.
<roaksoax> lol goodnight
<rvba> roaksoax: I tested your fix for the commissioning issue and it worked!  Congratulations for fixing this!
<bigjools> morning
<roaksoax> bigjools: howdy!! regression day fixing today :)
<roaksoax> bigjools: do you know anything about the bson issue allenap ? i don't seem to understand fully
<roaksoax> since it seems we don't even depend on bson
<bigjools> roaksoax: I have no idea and I'm still catching up on email so don't know if I was told about it
<roaksoax> bigjools: do you know when python-bson was introduced as a dependency to maas and why the packaging was never updated to depend on it?
<bigjools> roaksoax: not entirely sure, I think it was in the last few weeks
<roaksoax> uhmmm
<bigjools> roaksoax: oh looks like allenap added it recently for the lldp work
<bigjools> sorry I am going to be a bit slow today, not feeling well
<roaksoax> bigjools: np.. he did leave this bug #1237615 before he left
<ubot5> bug 1237615 in MAAS "python-bson-ext does not encode binary in Apache with mod_wsgi" [Critical,Triaged] https://launchpad.net/bugs/1237615
<roaksoax> but I don't fully understand it
<bigjools> lookng
<stokachu> should we break up http://maas.ubuntu.com/docs/development/preseeds.html into sub-headings for Debian Installer and Curtin installer?
<stokachu> the documentation there now doesn't apply to curtin at all since preseed_xinstall doesnt exist
<stokachu> or maybe even do a new page dedicated to curtin
<stokachu> hmm, even have a heading in the TOC as like 'Node Enlistment, Commission, and Install'
<bigjools> stokachu: yeah it needs updating
<stokachu> bigjools: im just putting my thoughts out here as im going to take a stab at updating that part of the doc
<bigjools> thank you, that would be great
<stokachu> bigjools: if i make some doc changes within the TOC will that upset anyone or would it be open for discussion in an MP?
<bigjools> roaksoax: it looks like wsgi is possibly compiled differently to python
<bigjools> or some mismatch somewhere at least
<bigjools> stokachu: please agree on changes with us before starting
<bigjools> ie suggest something and we'll review before you commit to something
<stokachu> bigjools: do you want that in a bug or just here on irc
<bigjools> stokachu: bug would be ideal, tag it with "doc"
<stokachu> bigjools: sounds good ill do that
<bigjools> great, thanks very much
<stokachu> np
<bigjools> roaksoax: http://emptysqua.re/blog/python-c-extensions-and-mod-wsgi/
<bigjools> that might fix it
<roaksoax> bigjools: right, so we would need to test that
<roaksoax> too late now I think to take care of that
<bigjools> roaksoax: has the upload deadline passed?
<roaksoax> bigjools: kinda... but I'm uploading now
<bigjools> heh
<bigjools> sadly allenap did not give us a direct way to re-create the problem in the bug so I don't know what triggers it
<roaksoax> bigjools: exactly :)
<roaksoax> bigjools: but I just noticed that python-bson wasn't even a depends for maas
<roaksoax> package
<roaksoax> so I just added it
<bigjools> :(
<roaksoax> so we will see if an install of that causes any issues
<roaksoax> bigjools: my concern now is that if I add the depends on python-bson it might trigger the issue
<bigjools> roaksoax: probably
<bigjools> roaksoax: the bug talks about python-bson-ext though
#maas 2013-10-10
<roaksoax> bigjools:  ti is a recommends of python-bson
<roaksoax> which would get installed as part of bson's installation i think
<bigjools> ok
<jpds> smoser: Do you know why cloud-tools wants to remove ubuntu-minimal? https://pastebin.canonical.com/98848/
<smoser> jpds, don't know. but i'd like to knwo. same with dhcp-client doesn't make any senes.
<smoser> will investigate manana if you dont want to now
<jpds> I can now.
<jpds> If you want some apt-cache outputs ror something.
<danwest> smoser: current fpi image is 12.04.1, not 12.04.3?
<smoser> danwest, we can/should update that.
<smoser> we really need to get that on a 3 week like cadence.
<danwest> smoser: k, just noticed it
<smoser> .X is really garbage
<smoser> it does show its old
<smoser> but .X versus .Y doesn't mean anything.
<smoser> jpds, http://paste.ubuntu.com/6216432/
<smoser> thats the reason
<jpds> smoser: Nice.
<smoser> jtv, or bigjools, would you please take a cursory look at http://paste.ubuntu.com/6216361/
<jtv> Looking...
<smoser> it still has printfs in it. but some of tha tis important
<smoser> i think right now when we ryn 'sync' it will end up re-extrracting the tarball pointlessly
<smoser> and has no status on download at all.
<jtv> What am I looking at?
<smoser> patch to mass-import-ephemerals that adds
<smoser>  * some bad status on stderr
<smoser>  * not needlessly extracting a 250M tarball multiple times
<smoser>  * basically says "if stuff is in place, its good"
<smoser> and dont put files in place until they're good.
<jtv> Great.
<jtv> I think there's still some debug output in there...
<jtv> Of course tests will need updating.
<smoser> yeah i said that :)
<smoser> printf debugging still there
<jtv> Not what you said.
<jtv> But it falls into place when you word it that way.
<smoser> yeah.
<smoser> jpds, would you mind opening a bug ?
<smoser> with our discussion above ?
<smoser> i'll now upload iproute2 to staging
<smoser> or... to -next
<smoser> and i think that will actually solve our problem
<jpds> smoser: In a sec, doing a live MAAS demo.
<jtv> It's a lot of changes for one branch.  If feasible, I'd advise breaking it down into smaller branches.
<smoser> jtv, yeah.
<jpds> smoser: Bug where?
<smoser> you can open against cloud-archive project
<jpds> Will do.
<smoser> jtv, the other hting that we're doing now that we really dont need to do is actually caching the .tar.gz files that are downloaded.
<smoser> that increases the size requirements for /var/lib/maas
<smoser> we dont need them really. its a side affect of using the mirror that we're using.
<jtv> I was wondering about that.  Those files are huge.
<jtv> It's been complicating testing, even.
<jtv> Will the mirror writer still function well when we delete those files?
<jtv> I mean, it won't re-download images we already have, that sort of thing?
<jtv> The docstring for extract_image_tarball needs updating.  It's hard to figure out these changes from code without any explanation, but it's especially hard for the next person maintaining the code because they'll get documentation that is basically a lie!
<jpds> smoser: https://bugs.launchpad.net/cloud-archive/+bug/1237751
<ubot5> Ubuntu bug 1237751 in ubuntu-cloud-archive "cloud-tools removes ubuntu-minimal" [Undecided,New]
<roaksoax> smoser: are your fixes for import-ephemerals hitting trunk?
<jtv> smoser: another note about the diff, besides "document!": we normally require boolean tests to be explicit about what they test, e.g. "if name is not None:" or "if len(items) == 0:" instead of just "if name" or "if not items" respectively.
<roaksoax> jtv: is anything else landing nowish?
<jtv> roaksoax: not that I know.
<roaksoax> jtv: ok, just uploaded maas anyway
<jtv> There are two approved branches, from Gavin & RaphaÃ«l respectively.  I could see if they need immediate landing.
<jtv> Gavin's is irrelevant for production, and RaphaÃ«l may still want to consider options for his branch.
<jtv> smoser: don't forget to document dlstatus()... it also needs a clearer name, preferably a verby one!
<jtv> smoser: By the way, can you explain what "flat" is, in insert_item?  It gets returned from from util.products_exdata.
<jtv> Oh, and don't forget to run "make lint" â it catches some of the most obvious violations of coding standards.
<bigjools> jtv: any outstandng problems that you know of?
<jtv> bigjools: no, I was waiting to ask you the same thing.  Scott has a patch he's working on.
<bigjools> there's allenap's bson problem
<bigjools> that's all I know
<bigjools> jtv: if you're up for a technical challenge, see bug 1237615
<ubot5> bug 1237615 in MAAS "python-bson-ext does not encode binary in Apache with mod_wsgi" [Critical,Triaged] https://launchpad.net/bugs/1237615
<jtv> Slightly under the weather, probably post-deadline letdown, but I'll give it a look.
<bigjools> jtv: it's possible the garage maas is buggered - first st
<bigjools> jtv: it's possible the garage maas is buggered - first step is to recreate
<jtv> bigjools: afaict we're not actually implementing the workaround from that blog article â we have WSGIApplicationGroup %{GLOBAL} but not WSGIProcessGroup %{GLOBAL}.
<jtv> I still feel I'm missing something though...  If we're running only one application, what other interpreter in the same process is using the bson extension?
<bigjools> jtv: threads
<bigjools> it broke in other ways without the WSGIApplicationGroup too
<jtv> bigjools: it says threads=1, so that's part of the picture I'm missing.  But I'm not saying WSGIApplicationGroup %{GLOBAL} looks wrong, just that we don't have the workaround the blog article suggested.
<bigjools> jtv: hmmm
<bigjools> it also says processes=2
<bigjools> roaksoax: so is the bson issue resolved?
<bigjools> and rvba ^ ?
<lifeless> is bson biting?
<lifeless> is it from oops, or something else that you're using bson for? </idlecuriosity>
<bigjools> lifeless: python-bson and wsgi don't get along
<bigjools> https://launchpad.net/bugs/1237615
<ubot5> Ubuntu bug 1237615 in MAAS "python-bson-ext does not encode binary in Apache with mod_wsgi" [Critical,Triaged]
<lifeless> bigjools: wsgi in general, or mod_wsgi.. ah
<rvba> bigjools: I don't know, allenap was looking into itâ¦ Gavin ^?
<bigjools> rvba: the reason I ask is because of the last comment you made on the email thread
<bigjools> "Andres figured it outâ¦ I tested his fix in the lab and it worked! "
<rvba> bigjools: right, but the title of the email says "Commissioning with a Saucy image sets node status to "Failed tests"" and the bug referenced in there is bug 1237364 .
<ubot5> bug 1237364 in maas (Ubuntu) "Commissioning with a Saucy image sets node status to "Failed tests"" [Critical,Confirmed] https://launchpad.net/bugs/1237364
<bigjools> rvba: urgh right
<bigjools> see this? https://bugs.launchpad.net/bugs/1237615
<ubot5> Ubuntu bug 1237615 in MAAS "python-bson-ext does not encode binary in Apache with mod_wsgi" [Critical,Triaged]
<bigjools> argh
<bigjools> https://bugs.launchpad.net/bugs/1237615
<bigjools> no
<bigjools> paste fail
<bigjools> https://bugs.launchpad.net/bugs/1237159
<bigjools> that
<ubot5> Ubuntu bug 1237159 in MAAS "MAAS environment can not be bootstrapped or destroyed with Declared or Commissioning node" [Undecided,New]
<bigjools> rvba: ^ ?
<rvba> Lookingâ¦
<bigjools> looks like the "ownership" bug combined with the "cruft left in the filestore" bug
<rvba> "cruft left in the filestore"?
<rvba> I just think juju is trying to release all the nodes it can see (the juju-core bug) and some of the nodes' status don't allow that transition.
<bigjools> yeah
<jpds> bigjools: Are you stlil around?
<jpds> MAAS is doing something quite special.
<jpds> Actually, no, that's just very confusing.
<smoser> jpds, did you open me a bug ?
<jpds> smoser: Yes.
<smoser> oh?
<smoser> i didn't see it. and just opened one myself.
<jpds> smoser: https://bugs.launchpad.net/cloud-archive/+bug/1237751
<ubot5> Ubuntu bug 1237751 in ubuntu-cloud-archive "cloud-tools removes ubuntu-minimal" [Undecided,Confirmed]
<smoser> jpds, just copied iproute2 from -staging to -proposed.
<smoser> it has to re-build there, but then wlil land.
<jpds> smoser: Cool.
<roaksoax> allenap: howdy
<roaksoax> allenap: did you test the wsgi issue in a clean environment ?
<roaksoax> allenap: and unfortunately we cant do anything about pyhon-bson-ext being installed
<roaksoax> it will be
<allenap> roaksoax: I tried it in the garage maas. Moving _cbson.so out of the way (and restarting apache) fixed it. fixed it
<roaksoax> allenap: but what was the issue in the first placr.. the bug doesnt really say
<roaksoax> allenap: i would tests in a clean environment
<mgz> roaksoax: mod_wsgi uses a simplified threading model that not all python c extentions are compatible with
<mgz> symptom is generally segfaults
<roaksoax> mgz: right
<roaksoax> but my question is... do maas webui dies? does celery die?
<allenap> roaksoax: Nothing so obvious. It encodes binary data as UTF-8 strings, which causes an exception in the celery worker on the cluster, thus preventing tag expressions from being recalculated en masse.
<roaksoax> allenap: can you provide an exact way to reproduce and see how it fails to donso(we will need it eventuslly fornsru if we find a solution) but would like to test in a clean environment
<roaksoax> it would not be the first time i see issues doe tue a broken environment
<roaksoax> issues due to*
<roaksoax> thanks ;)
<allenap> roaksoax: I'll try to reproduce in a dev environment.
<roaksoax> allenap: thank you!
<stokachu> re: bug https://bugs.launchpad.net/maas/+bug/1237723, going to start work on this today, would like some feedback on comment #3
<ubot5> Ubuntu bug 1237723 in MAAS "preseed documentation is outdated and doesn't apply to latest curt installer bits" [High,Triaged]
<stokachu> so should the maas api be exposing things like network information (vlans) ?
<stokachu> i would think we'd need some sort of agent on the machines to respond to those types of requests
<stokachu> i did something like this before using qpid
<stokachu> amqp
<allenap> stokachu: Currently it exposes the routers to which a machine is connected, as discovered by LLDP. It also records the LLDP XML as produced from lldpctl in the lldpd package which can be used for tag expressions.
<stokachu> allenap: ok im not entirely familiar with lldp so ill need to research that
<allenap> stokachu: When I say "currently", I mean as of this week :)
<stokachu> ahh
<stokachu> allenap: im going to go digging into the code today to make sense of it
<allenap> stokachu: We are behind with documenting it, so please ask if you have questions. To the maas-devel mailing list is probably best, so that we have a record and others see it.
<stokachu> allenap: ok will do thanks!
<stokachu> allenap: awesome lldp gathers the vlan name
<stokachu> so i can work to expose that information b/c so far i think maas only exposes the mac address
<roaksoax> allenap: so have you guys tested DHCP with portfast + LLDP?
<roaksoax> allenap: in real world escenarios we've seen that that portfast prevents nodes from obtaining DHCP
<allenap> roaksoax: I haven't, and I don't think rvba or anyone else has.
<roaksoax> that's probably gonna conflict wouldn't it? since I read somewhere that the lldp needs portfast enabled
<stokachu> do you guys have switches to test against
<stokachu> i dont have any of that hardware and im not sure if openvswitch would emulate vlans in the same manner as cisco
 * roaksoax bbl
<smoser> matsubara,
<smoser> when the QA lab tests, does it test with released images or dailies ?
<matsubara> smoser, it uses the image from: CLOUDIMGURL=http://cloud-images.ubuntu.com/$RELEASE/$BUILDID as the testbed
<smoser> matsubara, i'm sorry. i meant the ephemeral images.
<smoser> maas.ubuntu.com/images
<matsubara> smoser, hmm we have a local mirror of maas.ubuntu.com/images here: http://10.98.0.13/mirrors/maas-ephemeral/, then the tests configure /etc/maas/import_ephemerals to use that mirror. (See http://bazaar.launchpad.net/~maas-maintainers/maas/qa-lab-tests/view/head:/utils.py lines 70  to 92). The test used to set up STREAM=daily but that's been disabled a long time ago as you can see by the XXX there
<smoser> matsubara, could i get a run done with daily ?
<matsubara> smoser, sure. let me trigger that for you. give me a few
<smoser> k. thanks.
<smoser> roaksoax, how does maas-import-ephemerals get run ?
<matsubara> smoser, with the saucy maas package, I assume?
<smoser> matsubara, sure
<roaksoax> maas-imoort-pxe-files runs it
<roaksoax> smoser ^^
<smoser> roaksoax, when does that get run
<matsubara> smoser, Currently rvba left a job running (#90) but I queued a test run with the STREAM=daily enabled. It's going to be build #91 on jenkins: http://10.189.74.2:8080/job/saucy-adt-maas-manual/
<matsubara> smoser, I'll keep an eye on it and let you know how it goes
<smoser> matsubara, thanks.
<matsubara> np
<roaksoax> smoser: i font think we have a cronjob anymore
<smoser> Hooray for us.
<matsubara> smoser, the test run you asked is running: http://10.189.74.2:8080/job/saucy-adt-maas-manual/91/ARCH=amd64,label=lenovo-RD230-01/console, looks like the STREAM=daily didn't cause any problems and the tests are running the juju tests now. You can inspect the test instance running MAAS here: http://10.98.0.90/MAAS/nodes/
<matsubara> I don't remember exactly the reason why STREAM=daily was commented out from import_ephemerals but I think there's a small window between when a release is made and the new devel images are avaialble in maas.ubuntu.com/images
<matsubara> need to ask rvba about that as it was his XXX :-)
<smoser> matsubara, there were bugs in the images.
<smoser> mainly i think before https://bugs.launchpad.net/ubuntu/+source/open-iscsi/+bug/1050487 was backported
<ubot5> Ubuntu bug 1050487 in open-iscsi (Ubuntu Precise) "resolvconf not updated on iscsi root" [Medium,Fix released]
<matsubara> smoser, tests passed. should I MP the change to re-enable the STREAM=daily on import_ephemerals? I kicked off test runs for precise, quantal and raring too just to be sure.
<smoser> matsubara, i think i'd prefer that, yeah.
<smoser> it probably only acutally tests precise enlistment though, right?
<matsubara> smoser, yes
<smoser> ewll, for this purpose thats pretty good.
<smoser> matsubara, would it be easy to tell it to use a different release ?
<smoser> i dont recall how you tell it to use a different release for commissioning than precise
<matsubara> smoser, we have a NODE_SERIES variable but that's used to tell juju which release to use for deployment. If the maas cli allows us to change the main maas setting for commissioning, then it should be easy to do.
<smoser> roaksoax, do you know ?
<matsubara> smoser, apparently it's possible, yes: http://maas.ubuntu.com/docs/configure.html#choosing-a-series-to-install
<smoser> thats just the defaul treleease.
<smoser> i dont know if it that is used for commissioning
<smoser> hm..
<smoser> matsubara, well, its             release=Config.objects.get_config('commissioning_distro_series'))
<matsubara> well, looks like that's setable using the maas-cli: $ maas-cli maas maas get-config name="commissioning_distro_series"
<matsubara> "precise"
<smoser> nice.
<matsubara> cool. I added a new COMMISSIONING_SERIES var to the test (Default to precise) which allows us to run the tests with MAAS commissioning with different releases. I'll MP and ask for a review from the maas guys
<roaksoax> matsubara: commissioning has it owns drfault variable so does deployment
<roaksoax> juju passes fhe release for deployment
<smoser> yeah. we saw. and go tthat right.
<roaksoax> if juju does pass it then it uses default or the one set
<smoser> juju
<roaksoax> thats why we need default-series in the environment for juju
<smoser> there are other things than juju ;)
<roaksoax> in environments.yaml
<smoser> we were talking about commissioning though
<smoser> which has nothing to do with juju
<smoser> (i know, roaksoax, saying something has nothing to do with juju is shameful to you)
<roaksoax> commissioning you only need to set the variable
<roaksoax> smoser: haha it is not was just trying to give background info
<allenap> roaksoax: I've got a simple fix for the bson bug.
<allenap> roaksoax: I've confirmed it works on the Garage MAAS, and I have a reproducible example (see https://bugs.launchpad.net/ubuntu/+source/pymongo/+bug/1237615).
<ubot5> Ubuntu bug 1237615 in MAAS "python-bson-ext does not encode binary in Apache with mod_wsgi" [Critical,Triaged]
<allenap> roaksoax: I'll put up a branch for review in a few seconds.
<allenap> roaksoax: https://code.launchpad.net/~allenap/maas/fix-wsgi-application-group/+merge/190458
<roaksoax> allenap: cool thanks
<allenap> roaksoax: If you review it, I land it, can we rebuild and upload in time?
<roaksoax> allenap: so what's the regression potential of doing this "application-group=%{GLOBAL}"
<roaksoax> allenap: i mean, could that affect how other things such as logs? or anything else that requires everything under wsgi to run under the maas user/grpup
<allenap> roaksoax: I honestly don't know. The change ensures that the wsgi.py script is imported in the first Python interpreter to be created instead of a later one. My guess is that this is what we wanted all along anyway.
<allenap> roaksoax: Right now tag recalculation isn't working at all :-/
<roaksoax> allenap: right, so we are between the sword and the wall...
<allenap> roaksoax: Indeed!
<roaksoax> allenap: let's get that tested in many ways possible and if there's no apparent issue, let's merge it
<allenap> rvba: Are you able to help test a new package in the lab?
<roaksoax> allenap: we will need to test both upgrades and fresh installs
<allenap> roaksoax: I don't have the means to test those. matsubara, do you?
 * matsubara reads backlog
<matsubara> I can test on the qa lab
<matsubara> would that help?
<roaksoax> we need to check that everything starts, that all the logs are written
<roaksoax> we need to make sure no errors are present in apache logs
<roaksoax> nor maas logs
<roaksoax> power commands
<matsubara> allenap, roaksoax: what's the package that needs fresh install testing? And what upgrade path do you want me to test? raring to saucy and or precise to saucy (not even sure if that's possible)?
<roaksoax> everything that involes wsgi
<roaksoax> matsubara: saucy archives, to saucy fix
<allenap> roaksoax: Can you +1 https://code.launchpad.net/~allenap/maas/fix-wsgi-application-group/+merge/190458 and I'll start the ball rolling.
<matsubara> the lab is currently tied up running some tests for smoser, once that's done (should be done in a couple of hours) then I can do the testing
<matsubara> which will give you guys time to land and get the recipe going to build a new package, right?
<allenap> matsubara: Should be. Thanks Diogo, that's great.
<allenap> matsubara, roaksoax: I've requested a built at https://code.launchpad.net/~julian-edwards/+recipe/maas-daily-saucy
<matsubara> allenap, there's a message there saying it couldn't build because sp was superseded. is this related to build you requested?
<allenap> matsubara: I think that's an old message.
<matsubara> allenap, ok
<allenap> roaksoax: That built fine, but https://code.launchpad.net/~maas-maintainers/+archive/dailybuilds says a newer version is in the distro. How can I fix this?
<roaksoax> allenap: oh hold on
<roaksoax> allenap: ok changes to packaging should land in a few minutes
<roaksoax> allenap: https://code.launchpad.net/~andreserl/maas/packaging_bzr1693/+merge/190488
<allenap> roaksoax: When they're in I'll kick off a new build.
<roaksoax> allenap: merged, go for it
<roaksoax> allenap: i need to leave now, will be back tonight. Just email me an update and I'll do my share of testing when I get back
<allenap> roaksoax: That's brilliant. It's bedtime for me, so I'll speak to you tomorrow.
<roaksoax> allenap: alright! thanks for thr hard work
<allenap> roaksoax: You too :)
<matsubara> allenap, could you cc me too? would be helpful to have some pointers on what I need to test for
<matsubara> allenap, roaksoax: from the conversation above, I'm thinking of doing: start a new saucy testbed, install the saucy package from the archive, upgrade it to the PPA version and then check logs, web ui. Anything else I should check for?
<allenap> matsubara: The thing that has to work is tag expression recalculation. When you define a new tag, or update and existing tag, containing an XPath expression, jobs get sent to the clusters asking them to do the work. The jobs on the clusters were crashing with ValueError (https://bugs.launchpad.net/maas/+bug/1237463).
<ubot5> Ubuntu bug 1237463 in MAAS "ValueError using lxml when evaluating tags in worker" [Critical,In progress]
<matsubara> allenap, cool. thanks for the bug.
<allenap> matsubara: What you need is a node that has completed commissioning (and so has lshw and/or lldp XML recorded against it) and then add a new tag with an expression (the expression "true()" - without quotes - is enough to test this).
<matsubara> allenap, perfect. thank you
<allenap> matsubara: 1.4+bzr1693+dfsg-0+1695+210~ppa0~ubuntu13.10.1 is now available in ppa:maas-maintainers/dailybuilds.
<matsubara> allenap, the lab should be available soon and will get the testing started
<allenap> matsubara: Tip top, thanks again. I'm going to get some sleep now. Cheerio until tomorrow.
<matsubara> allenap, you're welcome. good night!
<bigjools> morning
<bigjools> roaksoax: why add power/moonshot.template which is an almost carbon copy of ipmi?
<bigjools> it has an old bug in it (bug 1171418)
<ubot5> bug 1171418 in maas (Ubuntu Quantal) "MAAS fails to power up machines when trying to install nodes" [High,New] https://launchpad.net/bugs/1171418
<roaksoax> bigjools: ipmitool?
<roaksoax> moonshot uses ipmitool
<roaksoax> bigjools: besides we need a template named moonshot eithwrway
<bigjools> really?
<roaksoax> to use thst power method
<roaksoax> yep really
<bigjools> why not make it generic "ipmitool"
<bigjools> either way, it has a bug
<bigjools> and I am sure we can refactor it with ipmipower
 * bigjools files bug
<roaksoax> bigjools: it does not worl exactly as the one using ipmipower
<bigjools> roaksoax: what are the main differences?
<roaksoax> the behavior is different
<roaksoax> bigjools: t
<roaksoax> wwhat moonshot returns on ipmi commands
<roaksoax> on ipmi power we use cycle
<roaksoax> ipmitool doesnt
<bigjools> :/
<roaksoax> yeah it sucks
<roaksoax> but btw when the power typenis set to "moonshot" the code looks for a template cslled moonshot.template
<roaksoax> anyway im off
<bigjools> roaksoax: ok fair enough... it's a shame.  Thanks for getting loads of stuff fixed, tremendous job!
#maas 2013-10-11
<bigjools> roaksoax: in case you are not really asleep yet, there's one more fix that needs to get in to maas, the WSGIImportScript change to fix the bson problem.  It's possibly only a small packaging change for the Apache config.
<bigjools> might have to SRU it though
<bigjools> either way, it's urgent of course
<jpds> bigjools: Any idea how I can figure out why MAAS is no longer serving TFTP?
<jpds> pserv.log shows: TFTP Listener started at 172.30.171.5:69
<jpds> But it's not.
<jpds> Oh, wait, might have fied it.
<bigjools> can you co..
<jpds> So, I chwoned pserv.log back to maas:mas.
<bigjools> ...
<jpds> But it still can't do its thing: chinstrap.canonical.com/~jpds/maas.png
<jpds> Interesting how it doesn't say that it's starting up...
<bigjools> restart it
<bigjools> it didnt shut down
<jpds> If i stop it, I get another  Received SIGTERM, shutting down. line.
<jpds> start doesn't being anything up.
<bigjools> when you stop it, do you see the process actually go away?
<jpds> Yep, it's no longer listening on port 69.
<bigjools> that means nothing, is the process still there?
<jpds> No, no pserv.
<bigjools> ok
<jpds> $ sudo service maas-pserv start
<bigjools> what user does it run as?
<jpds> I have a process, port 69, user maas.
<jpds> Nothing in logs.
<bigjools> is the log file +w?
<jpds> Yep.
<bigjools> apparmor ?
<bigjools> having said that your last error was hours ago
<bigjools> so once it's up, can you use a cmd line tftp client?
<bigjools> looks like that part worked actually, from the bios screen
<bigjools> unless the dots mean it's waiting and not downloading
<jpds> Interestinly, if I download from the cmd line, I actually get logs in pserv.log.
<jpds> Sorry, suffering the _worst_ Internet connection I've seen here.
<davecheney> jpds: do you have negative bandwidth ?
<jpds> davecheney: I wouldn't be surprised, took me 5 minutes to write that last line it kept dropping out.
<jpds> tftp> get pxelinux.0
<jpds> Received 26837 bytes in 0.0 seconds
<jpds> Works fine, VM on the same phyiscally host as the MAAS VM, doesn't work... :-/
<bigjools> sounds like your VM network setup is wrong then
<jpds> bigjools: They're both sitting on br0.
<bigjools> well something is wrong
<jpds> bigjools: Can you see anything breaking if I "sudo -u postgres pg_dump maasdb" and import that into a new MAAS install?
<bigjools> yes
<bigjools> but if all the nodes are Ready then it's ok
<bigjools> I  think
<jpds> They're ready, I don't want to re-enlist everything though.
<jtv> Does anybody know why bug 1232174 was reset from Fix Committed to Triaged?
<ubot5> bug 1232174 in MAAS "maas-import-pxe-files fails because Saucy's images information is not available." [Critical,Triaged] https://launchpad.net/bugs/1232174
<bigjools> nfi
<bigjools> does it work?
<bigjools> btw our sampledata seems screwed
<jtv> I thought it did, but...  Wanted to try just now, but couldn't get a large canonistack instance.
<bigjools> you get "#### INVALID STRING ####" in the node listing for each node's name
<jtv> I think that's hard-coded IDs in the sample data fixture.
<bigjools> clicking on them gets a crash
<jtv> Yes.
<jtv> Say the sample data creates a nodegroup, and the database assigns it ID 3.
<jtv> Unfortunately the fixture assumes it gets ID 1.
<jtv> I don't know how exactly that would work, given referential integrity, but ISTR that was how it happened.
<roaksoax> bigjools: yep that's on my radar
<bigjools> roaksoax: cool, SRU?
<roaksoax> bigjools: i think we might be able to get in critical fixes
<bigjools> even more better
<roaksoax> bigjools: what other critical fixes we have?
<bigjools> roaksoax: that's the only one I know of
<bigjools> I'm doing some doc changes now but that won't affect the release
<roaksoax> bigjools: this aaffects maas-import-ephemerals, and I';m filking a new one
<roaksoax> https://bugs.launchpad.net/simplestreams/+bug/1237990
<ubot5> Ubuntu bug 1237990 in simplestreams "FileStore checksumming will always fail on resumed download" [Undecided,Fix committed]
<roaksoax> bigjools: i think we should unify maas-import-pxe-files and maas-import-ephemerals config to be just one
<bigjools> roaksoax: +1
<bigjools> it will happen when we convert mipf
<roaksoax> bigjools: https://bugs.launchpad.net/maas/+bug/1238376
<ubot5> Ubuntu bug 1238376 in MAAS "maas-import-ephemerals no longer inherits config from maas-import-pxe-files" [Undecided,New]
<roaksoax> bigjools: is the bug clear or am I too tired?
<bigjools> reading
<bigjools> roaksoax: hmmm,  jtv any comments ^
 * jtv reads
<jtv> If you still have the shell-style config for maas-import-ephemerals, it includes the maas-import-pxe-files by default.
<jtv> So AFAICS the bug only applies to a fresh install.
<jtv> And there, the behaviour seems reasonable per se â as long as we can make the configuration clear to the user.
<roaksoax> jtv: might also affect in upgrades I think I had issues with upgrades as well
<roaksoax> that's how I think I discovered it first
<jtv> Let me just check the old config file...
<roaksoax> i can't really remember though
<roaksoax> jtv: but either way, on a fresh install, if before we used to modify 1 file, now we need to modify 2
<roaksoax> which can become painful
<jtv> The old /etc/maas/import_ephemerals config started with:
<jtv> [ ! -f /etc/maas/import_pxe_files ] || . /etc/maas/import_pxe_files
<roaksoax> jtv: right but that upgrade should remove import_pxe_files
<roaksoax> err
<roaksoax> import_ephemerals
<jtv> It moves it, but after reading the variables it sets.
<roaksoax> technically speaking, and due to packaging standards, it shold be removfed on upgrade IIRC
<jtv> So the resulting config _should_ include any settings from /etc/maas/import_pxe_files
<roaksoax> right
<roaksoax> jtv: so I've seen upgrades where it generated a new pserv.yaml
<roaksoax> while I've seen others that didn't
<roaksoax> i think it requires more corner case escenarios
<roaksoax> but the thing is that should not happen in fresh installs either
<roaksoax> because we are supposed to only run maas-import-pxe-files which also runs maas-import-ephemerals
<jtv> What exactly should not happen in fresh installs?
<jtv> If you've seen a fresh install rewrite pserv.yaml, that's a bug.
<roaksoax> jtv: not  fresh, an upgrade
<roaksoax> jtv: in fresh install.. because we run maas-import-pxe-files, which runs maas-import-pepehemrals, then it should inhering
<roaksoax> whatever config for release is in /etc/maas/import_pxe_files
<jtv> It doesn't.  I think we should update the documentation for this.
<roaksoax> jtv: right, it doesn't iunherit, so that *is* a bug
<roaksoax> jtv: that's a "regression"
<roaksoax> from how it used to work
<jtv> Or "change in behaviour" as we software engineers like to justify ourselves.  :)
<roaksoax> a regression because maas-import-pxe-files *does* run maas-import-ephemerals
<roaksoax> lol
<roaksoax> jtv: yeah unfortantely users say otherwise and I deal with that :)
<roaksoax> but anytways
<roaksoax> simply because maas-import-pxe-files runs maas-import-ephemerals, then it is a regression and should be fixed
<roaksoax> jtv: that's why i suggested bigjools to somehow unify the config between maas-import-pxe-files and maas-import-ephemerals
<roaksoax> but either way that's an standing bug
<roaksoax> jtv: there's users been hitting that issue already (internal and in important projects that caused blockers)
<jtv> That's fast.
<jtv> We have been unifying the config, but this is a transitional issue.
<roaksoax> so I'm happy it wasn't just me
<roaksoax> jtv: yeah, transitional should mean backwards compatible
<roaksoax> and fresh installs are not
<roaksoax> and might affect upgrades
<roaksoax> i;m not 100% sure it doesn't cause I found the issues on an upgrade
<jtv> It shouldn't be too hard to preserve the old behaviour â it's just a bit ugly to keep running one shell config but moving the other into the unified config.
<roaksoax> yeah
<roaksoax> anyway
<roaksoax> i'm off to bed
<roaksoax> have a good night
<jtv> It's important to know when you hit those issues.
<jtv> Good night... dream of MAAS.  :)
<jtv> Â¡Buenas!
<bigjools> nn roaksoax
<bigjools> https://code.launchpad.net/~julian-edwards/maas/more-docs/+merge/190525  but no rush I am going to eat
<jpds> Any way I can force a rewrite of dhcpd.conf ?
<jpds> I tried editing eth0 on the web UI, but it's not writing dhcpd.conf for some reasons.
<jpds> reason*
<jpds> Did it by hand, and got MAAS to work.
<bigjools> jpds: edit the cluster IIRC
<jpds> bigjools: Yeah, I tried that, it doesn't write the file.
<jpds> Tried removing the interface too.
<jpds> And readding it.
<bigjools> why did you need to rewrite it?
<jpds> bigjools: I didn't have one, this is a fresh MAAS install.
<bigjools> sounds like your cluster's celeryd is not workig
<jpds> maas-cluster-celery start/running, process 599 - hmm.
<bigjools> or perhaps dhcp not set to managed?
<jpds> Ah, /var/log/maas/celery.log as HTTPErrors.
<bigjools> or the cluster is not accepted
<jpds> INFO/PoolWorker-4] The DHCP leases file does not exist.  This is only a problem if this cluster controller is managing its DHCP server.  If that's the case then you need to install the 'maas-dhcp' package on this cluster controller.
<bigjools> if editing the cluster doesn't write it one of those conditions is not met
<jpds> It is installed, so I guess the file was just missing to begin with.
<bigjools> that error is orthogonal to this
<jpds> Aha, i seem to have a pending cluster.
<bigjools> jtv: we don't have any loud announcement of pending clusters do we?
<jtv> No, they're just listed on the settings page.
<bigjools> jtv: "  If multiple tags attached to a node have the kernel_opts defined, the first one ordered by name is used."
<bigjools> ok?
<jtv> bigjools: yes, thanks
<bigjools> cheers
<kampy> hi all
<kampy> i have installed maas
<kampy> but facing issues with dhcp
<kampy> when i am trying to get ip for  machine through dhcp
<kampy> i am geting the ip from the range i have configured in /etc/dhcp/dhcpd.conf
<kampy> but when i am trying the same conf through maas
<kampy> it is not able to fetch the ip
<kampy> please help me in configuring dhcp with maas correctly
<kampy> http://maas.ubuntu.com/docs/install.html#configure-dhcp followed the steps in this link
<jpds> bigjools: I think DOWNLOAD in masa-import-pxe-files should have a -c by default.
<rvba> Hi smoser, I'm getting this http://paste.ubuntu.com/6221791/ when running the import scriptâ¦ I assume this is a transient issueâ¦ ?
<jamespage> rvba, smoser's not around today
<jamespage> rvba, I'd hoped that commits to published simplestream data might be atomic - does it resolve if you re-run straight away?
<rvba> jamespage: I ran the script twiceâ¦ but maybe this is due to the caching proxy we use in the labâ¦ I'm investigatingâ¦
<jamespage> rvba, that could certainly created some issues depending on config
<nesusvet> Hello everyone!
<nesusvet> I tryied to deploy the node, but after deployment it returns: 2013-10-11 07:15:54 DEBUG juju.state open.go:88 connection failed, will retry: dial tcp 127.0.0.1:37017: connection refused
<nesusvet> I guess there a problem in different version on the mongodb
<nesusvet> and I guess I should add the stable-juju repository to this file: /etc/apt/sources.list before installation, but I don't know how
<jtv> nesusvet: sounds like something the Juju folks would be better able to diagnose.  It doesn't seem to be anything in MAAS per se.
<nesusvet> thanks
<jtv> rvba: just hit that checksum error too
<jtv> On a "precise" download this time.
<jtv> But also 20131010, strangely enough.
<jtv> roaksoax: are you here?  If so, could you review a packaging change for me?  It's https://code.launchpad.net/~jtv/maas/pkg-bug-1238376/+merge/190658
<roaksoax> jtv: branch needs fixing
<roaksoax> huge diff :)
<jtv> Uh-oh
<jtv> I bet I proposed for the wrong branch!
<jtv> roaksoax: hoping this one'll work out better... https://code.launchpad.net/~jtv/maas/pkg-bug-1238376/+merge/190694
<jtv> Definitely a smaller diff...
<jtv> roaksoax: I also landed another critical fix in r1702, so maybe I should just bump the revision number to that...
<jtv> Thanks for the review.
<roaksoax> yep
<roaksoax> jtv: if you guys can make a list of critical bug fixes would be great
<jtv> roaksoax: there's no "us guys" at the moment I'm afraid...  I think the rest of the squad's gone home, and I'm about to do the same.
<jtv> I know there's bug 1238376 and 1238681.
<ubot5> bug 1238681 in MAAS "Legacy import config treats arches/releases as lists of characters" [Critical,Fix committed] https://launchpad.net/bugs/1238681
<ubot5> bug 1238376 in MAAS "maas-import-ephemerals no longer inherits config from maas-import-pxe-files" [Critical,Fix committed] https://launchpad.net/bugs/1238376
<roaksoax> jtv: next week is fine
<roaksoax> over email better ;)
<jtv> roaksoax: this listing should give you a pretty good idea, on an ongoing basis: https://bugs.launchpad.net/maas/+bugs?field.searchtext=&orderby=-date_last_updated&search=Search&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=FIXRELEASED&field.importance%3Alist=CRITICAL&field.importance%3Alist=HIGH&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=&field.tags_combinator=ANY&fie
<jtv> ld.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on
<jtv> Hum.  Pretty long URL.
<jtv> Call it http://bit.ly/1bLZzBX for short.
<jtv> roaksoax: that's critical bugs, fix committed & fix released, sorted from most recently changed to less recently changed.
<matsubara> roaksoax, have you seen this before: http://pastebin.ubuntu.com/6222957/?
<matsubara> roaksoax, triggered during a test run for the saucy daily package
<roaksoax> jtv: awesome! thanks yoiu
<roaksoax> matsubara: looking
<roaksoax> matsubara: yeah, it is pretty self explanatory
<roaksoax> matsubara: do you have the images for the release?
<matsubara> roaksoax, well, it did run maas-import-pxe-files
<roaksoax> matsubara: make sure the directory and images exist
<matsubara> so I guess maas-import-pxe-files is failing quietly
<roaksoax> matsubara: probably is, or maas-import-ephemerals
<matsubara> roaksoax, there's a /var/lib/maas/ephemeral/precise/ephemeral/i386 dir but there's not a amd64 dir
<roaksoax> matsubara: so you only imported i386 images
<roaksoax> you need both, amd64 and i386
<matsubara> roaksoax, the test ran maas-import-pxe-files, that should be enough to get both right?
<matsubara> I'm investigating why it didn't, just wondered if you had seen it before
<roaksoax> matsubara: by default it should get both yes
<roaksoax> matsubara: but if you modified the config to only get i386 then it would fail that way
<roaksoax> smoser: can i mark this bug as fix released? 1218530
<roaksoax> bug #1218530
<ubot5> bug 1218530 in maas (Ubuntu) "FFE 13.10: curtin new development" [Undecided,Triaged] https://launchpad.net/bugs/1218530
<adam_g> roaksoax, ping
<roaksoax> adam_g: pong
<adam_g> roaksoax, do you have any curtin'd insatlled 12.04 machines up atm?
<roaksoax> adam_g: nope
<adam_g> roaksoax, ah, k
<roaksoax> adam_g: btw i have a fix for the upmi thing but havent tested yet
<adam_g> roaksoax, oh right i saw you pinged me
<adam_g> roaksoax, unfortuantely i wont be able to test today
<roaksoax> adam_g: no worries ;)
<roaksoax>  
<matsubara> roaksoax, I'm still getting the checksum failure running maas-import-pxe-files while using a proxy (after I cleared the cache). rvba mentioned he tried on a canonistack instance and the script completed successfully. Could be a bug of the script interacting with a proxy? Or perhaps I'm not cleaning the cache properly (although I do see lots of TCP_MISS in the squid logs, so it indicates the cache has been cleared fine)
<roaksoax> uhmmm
<roaksoax> matsubara: remove squid cache with rm -rf
<roaksoax> and restart squid and retry?
<matsubara> that's what I tried
<matsubara> I stopped squid, removed the cache_dir and restarted
<matsubara> but got the error again
<matsubara> but I'm seeing in the squid log: 11/Oct/2013:15:10:16 -0400.124      0 10.98.0.90 TCP_MEM_HIT/200 2221 GET http://maas.ubuntu.com/images/ephemeral/releases/streams/v1/index.sjson - NONE/- text/plain
<matsubara> and 11/Oct/2013:15:10:16 -0400.387    163 10.98.0.90 TCP_REFRESH_UNMODIFIED/200 13497 GET http://maas.ubuntu.com/images/ephemeral/releases/streams/v1/com.ubuntu.maas:download.sjson - DIRECT/91.189.90.19 text/plain
<matsubara> so, maybe it's using some other cache I'm not aware of. squid's memory cache? sudo service squid3 stop should clear the cache in memory, no?
<roaksoax> matsubara: check in /var/lib/cache/squid....
<roaksoax> or somewhere around there for the actual squid cache
<matsubara> -bash: cd: /var/lib/cache: No such file or directory
<roaksoax> matsubara: restart squi-deb-proxy
<matsubara> yeah, the config is pointing to /var/spool/squid3 and that's the one I removed
<matsubara> squid deb proxy too? this is just regular squid, not the deb-proxy one
<matsubara> I mean, the maas-import-pxe-files uses the squid cache not the s-d-p one, AFAICT
<matsubara> s-d-p is used for the archive stuff
<matsubara> (I'll try restarting s-d-p in anyway)
<roaksoax> matsubara: we dont ship anyconfig for squid3
<roaksoax> only for squid-deb-proxy
<matsubara> roaksoax, that's in the QA lab, it's not part of maas, we use a cache there to speed up maas-import-pxe-files
<roaksoax> ok so if the logs say theres a hit on the cache then probably the cache was not cleaned
<roaksoax> i dunno where rlse would it be
<roaksoax> but thats weird
<roaksoax> tzzzzt
<adam_g> roaksoax, do you know much about curtin in terms of its preseed data/cloud-config?
<adam_g> i know smoser is out
<roaksoax> adam_g: no unfortunately
<roaksoax> what do you need to do?
<adam_g> roaksoax, do you know if the /etc/maas/preseeds/curtin_userdata is userdata that is executed during provisioning?
<adam_g> trying to work around https://bugs.launchpad.net/bugs/1238915 by adding a curtin hook to just remove that file after install / before first boot
<ubot5> Ubuntu bug 1238915 in curtin (Ubuntu) "Keystone fails to start after installation: invoke-rc.d: policy-rc.d denied execution of start." [Undecided,New]
<adam_g> adding a final_command seems like it shoudl work
<roaksoax> adam_g: yeah it is
<roaksoax> so thats where you eould put lay
<roaksoax> te_commands
<roaksoax> err
<roaksoax> late_commands
<adam_g> roaksoax, http://paste.ubuntu.com/6223940/
<adam_g> roaksoax, late_commands is a DI thing right?  that doc makes it look like curtin's final_command is the same thing
<roaksoax> adam_g: correct
<adam_g> roaksoax, ill add that to curtin_userdata and see if it works
<roaksoax> it should
<roaksoax> i think stokachu did it for vlan config
<Tim> Quick Question: Is there a way to set a password for commissioned nodes?
<Tim> I need to access the node locally because networking doesn't come up properly after reboot
<Nik_> Not sure if I understand the question, but ubuntu user gets the ssh key placed in their authorized keys
<Tim> No?
<Tim> I understand, but I can't ssh
<Tim> I have to log in locally
<Tim> Nik_: hence the password
<Nik_> Oh i see...
<Nik_> If you are local, you can go into maintenance mode
<Nik_> and set any password
<Nik_> You'll get root access
<Tim> maintenance mode?
<Nik_> Umm.. that kernel boot option
<Tim> ah
<Tim> in grub?
<matsubara> Tim, this might help: https://lists.launchpad.net/maas-devel/msg00808.html
<Nik_> yeah something like that
<Nik_> It's called single-user mode or maintennace mode
<Nik_> You get a shell prompt and no drives mounted
<Nik_> maybe this can help: http://www.cyberciti.biz/faq/linux-reset-forgotten-root-password/
<Tim> matsubara: Interesting, thanks for the link
<Tim> Nik_: I'll try that as well
<Nik_> You got juju working or just maas?
<matsubara> np
<Tim> Nik_: I'm tryinig to get juju working, but since networking does come up on my node after reboot juju deploy times out
<Tim> *does not
<Tim> Its a problem with out 10gb switch
<Tim> if anyone cares
<Nik_> I see. Well hopefully I understand how maas works and how things happen with PXE enough for this to fork for you :)
<Nik_> And I didn't check mat's link. That may help too
<Tim> Nik_: Lol. I wouldn't worry about, I've got the pxe covered. It sounds like I might just be able to manually set the password using your suggestion
<Tim> and if not, modify the preseed file via matsubara's suggestion
<Tim> It took long enough to get MAAS and juju working
<Tim> it would really suck if this was what killed it
<AskUbuntu> MAAS Set password for node | http://askubuntu.com/q/356936
<Tim> Yes, that was me
<Tim> I asked that
<Tim> Nik_: Damn, skips the GRUB screen during boot
<Nik_> You can try hold shift I think
<Nik_> though that may be  a no-go on ubuntu...
<Nik_> http://askubuntu.com/questions/71867/grub-menu-doesnt-appear-when-pressing-shift
<Nik_> :(
<Tim_> Yeah, tried that
<Tim_> either it wasn't passed via KVM, or it doesn't work
<Tim_> Nik_: Thanks! that worked!
<Nik_> Tim_: Glad to hear! Anyways, I'm off for today. Hopefully you get your all solved :)
#maas 2013-10-12
<AskUbuntu> Some questions about juju and maas | http://askubuntu.com/q/357309
<AskUbuntu> Can MaaS Cluster Controller be on a VM | http://askubuntu.com/q/357322
#maas 2013-10-13
<bigjools> roaksoax: on the offchance you are around, we have critical revisions in maas trunk that need to get into saucy.  SRU is probably ok if it's done immediately on release.
<roaksoax> bigjools: i. not really ;) automated response
<roaksoax> lol
<bigjools> :)
<roaksoax> bigjools: send me an email with the buglist
<bigjools> will do
<roaksoax> thanks
<bigjools> on its way
<bigjools> there's a packaging change as well
<roaksoax> bigjools: cool thanks
<bigjools> roaksoax: and thank you
<roaksoax> bigjools: ill have to cherry pick btw
<bigjools> roaksoax: ok
<bigjools> let me check revs
<roaksoax> ill take care of that so no worries :)
<bigjools> roaksoax: ok there are more bugs
<bigjools> will email
#maas 2014-10-06
<lovea> Upgraded my MAAS controller to 1.7 and have lost my custom images (cloudbase windows server 2012).
<lovea> Lost in the UI that is
<lovea> In MAAS 1.5 if the images were in the /var/lib/maas/boot-resources/current structure they appeared in the UI.
<lovea> Anyone any ideas?
<roaksoax> lovea: MAAS 1.7 does image import differently. MAAS 1.7 now has the Image Store that sotres images differently. The API has methods to upload images to the image repository
<lovea> roaksoax: Ah, ok. So I should remove my custom images from /var/lib/maas/boot-resources/current and import them using the API?
<roaksoax> lovea: yes, i would keep your images for backup just in case
<lovea> Never used the API before. I'm guessing I use a browser as the REST client?
<lovea> roaksoax: Never used the API before. I'm guessing I use a browser as the REST client?
<roaksoax> lovea: there are commands for maas' CLI
<roaksoax> lovea: something along thse lines IIRC: maas local boot-resources create name=windows title="My Custom Windows Image" architecture=amd64/generic content@=  filetype=ddtgz
<lovea> roaksoax: Many thanks, I'll try this
<roaksoax> lovea: content@=/path/to/image
<lovea> roaksoax: and local is a maas profile?
<lovea> or a keyword
<roaksoax> lovea: maas profgile
<lovea> roaksoax: cheers
#maas 2014-10-08
<newell> bigjools, for implementation of the moving and regenerating the DHCP leases file, that needs to go on the provisioningserver since that is per cluster controller.  However, as far as actually putting stuff into the leases file, this will need to be done on the region since that is where the db and the hostmaps are.  I guess there are a couple ways this could be done.
<newell> 1.  We could update the leases file when the formModel is saved for the interface.
<newell> 2. Have rpc calls that call the region to then call update_host_maps?
<newell> I am thinking (1) is better no?
<bigjools> newell: sorry I'm doing too many things at the moment and I need some time to look at your situation
<newell> bigjools, no worries, when you get some time is fine.  Appreciate it since I am kinda isolated for implementation reviews at the moment.
<bigjools> yeah I understand
<jtv> gmb: review done.
<gmb> jtv: Thanks
#maas 2014-10-10
<bigjools> jtv: just landing a packaging fix now
<jtv> Thanks
<bigjools> jtv: it's in
<jtv> bigjools: sent you my draft reply.
<bigjools> jtv: looking
<jtv> bigjools: could I bother you for a review?  https://code.launchpad.net/~jtv/maas/bug-1379568/+merge/237893
<bigjools> jtv: doing one already but I'll take a look
<jtv> Thanks.
<bigjools> gmb: instead of the red error banner, do you think a separate oops page would be better?
<bigjools> it confused me until I realised what was going on
<gmb> bigjools: If the red error banner is confusing, then yes, we should do something different. I think we have a chunk of work to do on error reporting, actually: at the moment errors that happen in bulk actions are barely reported at all (just "couldn't be executed on N nodes:").
<jtv> Any reviewers available?
<jtv> rvba: I'll also need landing approval, but I'm going to run some local Q/A first.
<allenap> jtv: I want to get my branches on the way to landingdom, then Iâll get stuck into your reviews.
<jtv> Thanks!
<jtv> allenap: thanks for the swarm of reviews.
<allenap> jtv: Youâre welcome. The code was good so it was easy.
<jtv> Thanks.  Really making a point of facilitating reviews.
<gmb> allenap, jtv, (rvba): Review needed when you have chance gents: https://code.launchpad.net/~gmb/maas/bug-1379154/+merge/237936
<jtv> I'll take it.
<gmb> Thanks
 * gmb -> moar tea
<gmb> jtv: Gah, I missed the ", you"! But yay, made you look :)
<gmb> Thanks for the review :)
<zezom> I'm currently running proxmox at home on a single server + NAS but I'm quite impressed with what I've seen MASS + JUJU do. Unfortunately I am unable to tell if I can run MAAS, JUJU and my virtuals all on a single server. I would like the ability to add more servers in the future but for now one server is all I have. I've read through the MAAS install doco on ubuntu.com but it doesn't say if the MASS controller install can run any virtual's on i
<zezom> t's self or if it's only used to bootstrap other nodes to run virtual's.
<zezom> I also have a Ubuntu desktop but I shut it down when not in use.
<rbasak> zezom: no need for MAAS at all if you only have one server. You can just use Juju on it directly, and Juju can create VMs or containers as needed.
<rbasak> zezom: you can use the "manual" Juju provider for this. Or the "local" provider if you never want to go beyond one server (it's a bit easier).
<zezom> rbasak, that's great. and in the future I can install maas when required?
<rbasak> For just a small handful of servers, the "manual" provider should be fine.
<rbasak> Beyond that, to switch to MAAS, you'll need to redeploy to a new MAAS environment.
<zezom> should I install the cloud version of the Ubuntu server image then? Or just the standard server image?
<rbasak> For the manual or local providers, just the standard server image will be fine.
<rbasak> If local, bootstrap Juju from the server itself. If manual, I think you can bootstrap it from your desktop.
<rbasak> Hmm.
<zezom> when you say redeploy will that mean I'll have to recreate the setup? or I can just move the setup to the new hardware after presenting it to the existing juju?
<rbasak> If you want to manage the deployment from your desktop, the manual provider might be easier in that case.
<rbasak> Anyway, #juju would probably be more relevant for this conversation. People there know more than I do about this.
<rbasak> You'll have to recreate.
<rbasak> Juju has functionality that makes this easier though, with backup/restore stuff (I'm not familiar with it though)
<zezom> thank you for the info. it's appreciated.
<gmb> rvba, https://code.launchpad.net/~gmb/maas/bug-1379154/+merge/237936 needs approval for landing when you have a moment.
<jtv> And a bunch of mine as well...
<rvba> gmb: just reviewed your branch.
<gmb> rvba: thx
<jtv> rvba: could you review landing-approval-review mine as well?
<rvba> jtv: doing it now
<jtv> I've got 4 up, but they're not big.
<jtv> Thanks.
<gmb> jtv, rvba: https://code.launchpad.net/~gmb/maas/fix-register_event_and_event_type-bug-1379401 needs reviewing if you have the time and bandwidth.
<gmb> allenap: ^^
<allenap> gmb: I have spare girth in my band.
<jtv> Simple documentation review: https://code.launchpad.net/~jtv/maas/update-ipv6-limitations/+merge/237982
 * gmb -> out; errands. Back later
<th3rt> I'm trying to use MaaS to setup some super micro nodes with the IPMI on a separate maas network.  I can't seem to get MaaS to associate the IPMI interfaces with the correct host (I think because they are on a different network in MaaS).  Can anyone tell me what I'm doing wrong?
<jtv> th3rt: having them on a separate network should be fine.  No idea what could be going wrong there.
<th3rt> If I delete all the nodes from MaaS,  how can I get MaaS to re-discover the nodes?
<th3rt> ... or is that the wrong thing to do.  I didn't set this MaaS server up originally (and have never used MaaS before) so I'm not sure how the discovery process works once a node has been deleted.
<jtv> th3rt: as long as they're set to netboot, just turn them on.
<jtv> Assuming you're letting MAAS manage DHCP, and letting the nodes netboot,
<jtv> turning a node on will netboot it off the maas server.
<jtv> It will boot into an ephemeral image that registers the node with the server.
<th3rt> jtv,  thanks much.  That helps.  I am letting MaaS manage DHCP.
<jtv> np
<DominiKiD> can anyone help me with my maas configuration?
<DominiKiD> i beent ryign to do this for weeks and i have no clue of what im doung..
<DominiKiD> doing
<DominiKiD> i been  trying to make my own home server and add a node
<DominiKiD> i installed mass on one of the computers
<DominiKiD> and eveyrthign seems to be working..
<DominiKiD> now i dont have a blue how to configure the dhcp
<DominiKiD> the clusters
<DominiKiD> help enyone?
#maas 2014-10-11
<th3rt> Silly question here,  but under username->preferences->  is the ssh-key section for logging into maas or is it for maas to add to newly bootstrapped hosts?
<th3rt> I'm trying to figure out what the default creds for a maas bootstrapped host are
<alexz> hello
<alexz> I cannot commision a node, when I try check-commissioning I get empty array: []
<alexz> any ideas?
<alexz> I try to use maas (with latest ubuntu 14.04)
<alexz> I don't understand how to get a node from state=1 to anything else
#maas 2014-10-12
<rabejens> hello, does anyone know of the problem that MAAS provisioned nodes do not get partitioned?
<rabejens> I tried to install MAAS on three different setups now (virtual box, behind proxy; virtual box, with direct net access, and physical machines, behind proxy) and I get the same results every time. The node boots and enlists via PXE, provisioning seems to succeed (I get no errors), but the hard disk is neither partitioned nor formatted nor any system is transferred to it.
<rabejens> If there was an old system on it before, it is not even overwritten, i.e., after rebooting, the old system is booted again.
<rabejens> well, does not seem so, apparently...
#maas 2015-10-05
<mup> Bug #1502839 opened: CentOS node deployment fails <centos> <maas> <MAAS:New> <https://launchpad.net/bugs/1502839>
<voidspace> daily-qa-ok doesn't have vivid builds?
<voidspace> so if I need to test 1.9 I need to use dailybuilds?
<voidspace> rvba: ping
<voidspace> anyone able to help me un-screw a maas installation?
<narindergupta> voidspace, whats the issue? I can give a try if i knows the answer?
<voidspace> narindergupta: I use the dailybuilds ppa (because daily-qa-ok has no vivid build)
<voidspace> narindergupta: occassionally migrations are added and then backed out
<voidspace> which results in a screwed installation
<narindergupta> voidspace, so you want to clean up?
<voidspace> narindergupta: http://pastebin.ubuntu.com/12690416/
<narindergupta> voidspace, i am not that expert but you can clean by using apt-get purge
<voidspace> narindergupta: heh, no - the django db needs cleaning up
<voidspace> but thanks
<voidspace> gotta go
<narindergupta> voidspace, ok in that case i do not have that expertize
<voidspace> rvba is probably (possibly?) in Seattle this week, so I may be out of luck
<voidspace> he's helped me with this in the past
<binoy> how to get details using curl
<binoy> from maas-api
<stokachu> if you visit http://localhost/MAAS#/nodes it fails to connect to the websocket
<stokachu> is that considered a bug and should it redirect to http://localhost/MAAS/#/nodes instead?
<stokachu> roaksoax: ^
<roaksoax> stokachu: http://localhost/MAAS#/nodes -> this never existd IMHO
<mup> Bug #1424849 changed: "Permission Denied" while trying to configure the Windows Administrator Password after deployment completes with MAAS <hyperscale> <qa-missing> <MAAS:Invalid> <https://launchpad.net/bugs/1424849>
<mup> Bug #1502566 changed: Rack ID for nodes <suggestions> <MAAS:Opinion> <https://launchpad.net/bugs/1502566>
#maas 2015-10-06
<binoy> is it possible to do maas api using curl
<binoy> ?
<binoy> get and post
<binoy> https://answers.launchpad.net/ubuntu/+source/maas/+question/272100
<mup> Bug #1503300 opened: maas does not add nodes to bind <MAAS:Incomplete> <https://launchpad.net/bugs/1503300>
<mup> Bug #1503300 changed: maas does not add nodes to bind <MAAS:Incomplete> <https://launchpad.net/bugs/1503300>
<mup> Bug #1503300 opened: maas does not add nodes to bind <MAAS:Incomplete> <https://launchpad.net/bugs/1503300>
<mup> Bug #1497392 changed: maas text-mode "node-group-interface" bug fails with MAAS 1.8.2+bzr4041-0ubuntu1~trusty1 <MAAS:Invalid> <https://launchpad.net/bugs/1497392>
<manglav_> hey all. Is it possible to use MAAS with distributed nodes, connecting them via openVPN so they seem like they're on the same LAN?
<mup> Bug # opened: 1503467, 1503468, 1503472, 1503473, 1503474, 1503475
#maas 2015-10-07
<mup> Bug #1503479 opened: [1.9.0 alpha3] can't add physical interface <ui> <ux> <MAAS:New for carlaberkers> <https://launchpad.net/bugs/1503479>
<mup> Bug #1503483 opened: [1.9.0 alpha3] can't select all interfaces or disks <ui> <MAAS:New for blake-rouse> <https://launchpad.net/bugs/1503483>
<mup> Bug #1503484 opened: cannot deploy on disks larger than 2TiB <storage> <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1503484>
<mup> Bug #1503488 opened: [1.9.0 alpha3] insufficient padding between IP fields on node networking <ui> <MAAS:New for ricgard> <https://launchpad.net/bugs/1503488>
<mup> Bug #1503484 changed: cannot deploy on disks larger than 2TiB <storage> <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1503484>
<mup> Bug #1503484 opened: cannot deploy on disks larger than 2TiB <storage> <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1503484>
<mup> Bug #1503484 changed: cannot deploy on disks larger than 2TiB <storage> <MAAS:In Progress by blake-rouse> <https://launchpad.net/bugs/1503484>
<mup> Bug # opened: 1503529, 1503530, 1503531, 1503533, 1503534, 1503536
<mup> Bug # changed: 1503529, 1503530, 1503531, 1503533, 1503534, 1503536
<mup> Bug # opened: 1503529, 1503530, 1503531, 1503533, 1503534, 1503536, 1503538
<mup> Bug #1503538 changed: [1.9.0 alpha 3] on alias, fabric and VLAN appear deactivated <ui> <MAAS:Triaged by ricgard> <https://launchpad.net/bugs/1503538>
<mup> Bug #1503538 opened: [1.9.0 alpha 3] on alias, fabric and VLAN appear deactivated <ui> <MAAS:Triaged by ricgard> <https://launchpad.net/bugs/1503538>
<mup> Bug #1503538 changed: [1.9.0 alpha 3] on alias, fabric and VLAN appear deactivated <ui> <MAAS:Triaged by ricgard> <https://launchpad.net/bugs/1503538>
<mup> Bug #1503538 opened: [1.9.0 alpha 3] on alias, fabric and VLAN appear deactivated <ui> <MAAS:Triaged by ricgard> <https://launchpad.net/bugs/1503538>
<sac_> Help required regarding MAAS setup.
<rbasak> sac_: try asking your question. People will read this channel over the next few hours and so might have a reply for you if you have already asked it.
<sac_> I want to add the same server hosting MAAS as the node in the MAAS. The server has two interfaces : one public network and one private. Which MAC address I should add in the MAAS ?
<rbasak> I don't see how that would work. The server would obliterate itself, surely? Why do you want to do this?
<sac_> @rbasak, the MAAS and node has to be compulsory separate ?
<rbasak> sac_: I don't see why you'd want to use MAAS in production in any other way. If just experimenting you can use VMs.
<rbasak> sac_: MAAS by definition wipes out a machine with a fresh install. Why would you want to do that on the machine that MAAS is using and expect it to keep working?
<sac_> Got it. Thanks for transforming my mis-understanding into understanding.
#maas 2015-10-08
<mup> Bug #1503925 opened: [1.9.0 alpha3] sabdfl request: keep selected nodes selected after action <ui> <MAAS:New> <https://launchpad.net/bugs/1503925>
<mup> Bug #1504061 opened: MAAS should have ability to specify default kernel for commissioning <MAAS:New> <https://launchpad.net/bugs/1504061>
<mup> Bug #1504061 changed: MAAS should have ability to specify default kernel for commissioning <MAAS:New> <https://launchpad.net/bugs/1504061>
<mup> Bug #1504061 opened: MAAS should have ability to specify default kernel for commissioning <MAAS:New> <https://launchpad.net/bugs/1504061>
<mup> Bug #1504066 opened: hwe-v for Trusty isn't available for ppc64el <MAAS:New> <https://launchpad.net/bugs/1504066>
<mup> Bug #1504066 changed: hwe-v for Trusty isn't available for ppc64el <MAAS:New> <https://launchpad.net/bugs/1504066>
<mup> Bug #1504066 opened: hwe-v for Trusty isn't available for ppc64el <MAAS:New> <https://launchpad.net/bugs/1504066>
<mup> Bug #1504173 opened: Cannot add a device, got a "django.db.utils.IntegrityError: duplicate key value violates unique constraint "maasserver_interface_pkey" 	DETAIL:  Key (id)=(2) already exists." message in the regiond log <MAAS:New> <https://launchpad.net/bugs/1504173>
<mup> Bug #1504173 changed: Cannot add a device, got a "django.db.utils.IntegrityError: duplicate key value violates unique constraint "maasserver_interface_pkey" 	DETAIL:  Key (id)=(2) already exists." message in the regiond log <MAAS:New> <https://launchpad.net/bugs/1504173>
<mup> Bug #1504173 opened: Cannot add a device, got a "django.db.utils.IntegrityError: duplicate key value violates unique constraint "maasserver_interface_pkey" 	DETAIL:  Key (id)=(2) already exists." message in the regiond log <MAAS:New> <https://launchpad.net/bugs/1504173>
<mup> Bug #1449402 changed: 1.8b4: 504 GATEWAY TIMEOUT (Unexpected exception: TimeoutError) <oil> <MAAS:Invalid> <https://launchpad.net/bugs/1449402>
<mup> Bug #1449402 opened: 1.8b4: 504 GATEWAY TIMEOUT (Unexpected exception: TimeoutError) <oil> <MAAS:Invalid> <https://launchpad.net/bugs/1449402>
<mup> Bug #1449402 changed: 1.8b4: 504 GATEWAY TIMEOUT (Unexpected exception: TimeoutError) <oil> <MAAS:Invalid> <https://launchpad.net/bugs/1449402>
<mup> Bug #1504268 opened: [1.9.0 alpha 3] sabdfl request: rename fabric (default) <ui> <MAAS:New> <https://launchpad.net/bugs/1504268>
<mup> Bug #1504340 opened: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 changed: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 opened: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 opened: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 changed: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 changed: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 opened: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 opened: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 changed: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 opened: Upgrade from 1.5 to 1.9 fails in network migration <maas (Ubuntu):New> <https://launchpad.net/bugs/1504340>
<mup> Bug #1504340 changed: Upgrade from 1.5 to 1.9 fails in network migration <MAAS:New> <https://launchpad.net/bugs/1504340>
#maas 2015-10-09
<mup> Bug #1504061 changed: MAAS should have ability to specify default kernel for commissioning <MAAS:New> <https://launchpad.net/bugs/1504061>
<mup> Bug #1504066 changed: hwe-v for Trusty isn't available for ppc64el <maas-images:Confirmed> <https://launchpad.net/bugs/1504066>
<mup> Bug #1501605 changed: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1501605 opened: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1501605 changed: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1501605 opened: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1501605 changed: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1501605 opened: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1501605 changed: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1501605 opened: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1501605 changed: Upgrade to 1.9 turned a device into a node <MAAS:New> <https://launchpad.net/bugs/1501605>
<mup> Bug #1496469 opened: MAAS still uses 3.13 kernel which does not work on power <blocks-hwcert-server> <MAAS:Confirmed> <https://launchpad.net/bugs/1496469>
<mup> Bug #1496469 changed: MAAS still uses 3.13 kernel which does not work on power <blocks-hwcert-server> <MAAS:Confirmed> <https://launchpad.net/bugs/1496469>
<mup> Bug #1496469 opened: MAAS still uses 3.13 kernel which does not work on power <blocks-hwcert-server> <MAAS:Confirmed> <https://launchpad.net/bugs/1496469>
<JZTech101> ok, so I changed the IP range from /8 to /16 on my DHCP server and changed the approriate settings in MaaS accordingly
<JZTech101> yet the nodes aren't powering on properly :|
<JZTech101> or rather, they aren't commissioning as they should
<JZTech101> it seems the gateway IP remains 0.0.0.0
<JZTech101> help?
#maas 2015-10-10
<mup> Bug #1504861 opened: Device action button in incorrect position when selecting a device <MAAS:New> <https://launchpad.net/bugs/1504861>
<mup> Bug #1504861 changed: Device action button in incorrect position when selecting a device <MAAS:New> <https://launchpad.net/bugs/1504861>
#maas 2015-10-11
<mup> Bug #1504861 opened: Device action button in incorrect position when selecting a device <MAAS:New> <https://launchpad.net/bugs/1504861>
<mup> Bug #1504863 opened: Unhandled error <MAAS:New> <https://launchpad.net/bugs/1504863>
<mup> Bug #1504865 opened: Include and install maas-backup utility <MAAS:New> <https://launchpad.net/bugs/1504865>
<mup> Bug #1481813 changed: Failed trusty deployment on Lenovo X3650 M5 in UEFI mode <oil> <curtin:Incomplete> <MAAS:Expired> <https://launchpad.net/bugs/1481813>
<mup> Bug #1504956 opened: When deploying CentOS or any other OS, MAAS should not partition the disk <MAAS:New> <https://launchpad.net/bugs/1504956>
<mup> Bug #1504956 changed: When deploying CentOS or any other OS, MAAS should not partition the disk <MAAS:New> <https://launchpad.net/bugs/1504956>
<mup> Bug #1504956 opened: When deploying CentOS or any other OS, MAAS should not partition the disk <MAAS:New> <https://launchpad.net/bugs/1504956>
<mup> Bug #1496469 changed: MAAS still uses 3.13 kernel which does not work on power <blocks-hwcert-server> <MAAS:Confirmed> <https://launchpad.net/bugs/1496469>
<mup> Bug #1496469 opened: MAAS still uses 3.13 kernel which does not work on power <blocks-hwcert-server> <MAAS:Confirmed> <https://launchpad.net/bugs/1496469>
<mup> Bug #1496469 changed: MAAS still uses 3.13 kernel which does not work on power <blocks-hwcert-server> <MAAS:Confirmed> <https://launchpad.net/bugs/1496469>
<mup> Bug #1504971 opened: Internal Server Error when trying to configure a cluster interface <MAAS:New> <https://launchpad.net/bugs/1504971>
<mup> Bug #1504971 changed: Internal Server Error when trying to configure a cluster interface <MAAS:New> <https://launchpad.net/bugs/1504971>
<mup> Bug #1504971 opened: Internal Server Error when trying to configure a cluster interface <MAAS:New> <https://launchpad.net/bugs/1504971>
<mup> Bug #1504971 changed: Internal Server Error when trying to configure a cluster interface <MAAS:New> <https://launchpad.net/bugs/1504971>
<mup> Bug #1504971 opened: Internal Server Error when trying to configure a cluster interface <MAAS:New> <https://launchpad.net/bugs/1504971>
<mup> Bug # opened: 1505029, 1505030, 1505031, 1505032
#maas 2016-10-10
<mup> Bug #1611481 changed: IPMI autoconfiguration and startup not working <MAAS:Expired> <https://launchpad.net/bugs/1611481>
<mup> Bug #1611980 changed: Upgrade from MAAS 2.0 RC3 to RC4 has pre-removal script error messages <MAAS:Expired> <https://launchpad.net/bugs/1611980>
<Prabakaran> Hello Team, Getting timeout issue while commissioning the node in MAAS 1.9, Logs are pasted in here  http://pastebin.ubuntu.com/23301741/ could somebody help me on this?
<mup> Bug #1631908 opened: VLAN over balance bondig doesn't work <MAAS:New> <https://launchpad.net/bugs/1631908>
<mup> Bug #1631934 opened: MTU values should not be on the bridged interface <MAAS:New> <https://launchpad.net/bugs/1631934>
<mup> Bug #1631946 opened: daily images have broken cloud-init <MAAS:Incomplete> <https://launchpad.net/bugs/1631946>
<mup> Bug # changed: 1274168, 1279460, 1326485, 1363074, 1364097, 1367843, 1368307, 1370371, 1372794, 1374242, 1376075, 1378865, 1379639, 1381390, 1622971
<mup> Bug #1631971 opened: TestDHCPNetworkLayout.test__dhcp_configurations_rendered is dependent on /etc/hosts <ipv6> <tests> <MAAS:Triaged> <https://launchpad.net/bugs/1631971>
<mup> Bug # changed: 1272016, 1343317, 1374478, 1393997, 1394277, 1402086, 1437251, 1438780, 1453749, 1465735, 1467694, 1476254, 1476746, 1481281, 1481283, 1483694, 1498854, 1503483, 1503530, 1507670, 1510070, 1513695, 1515975, 1519177, 1526859, 1556431, 1590768
<mup> Bug #1628114 changed: [FUJ] SSH input field not indicated for invalid username & the error is incomprehensible <MAAS:Fix Released> <https://launchpad.net/bugs/1628114>
<mup> Bug #1629080 changed: [2.1 ipv6] deploying machine needs connectivity to maas (address family) <maas-ipv6> <MAAS:New> <https://launchpad.net/bugs/1629080>
<mup> Bug #1621090 changed: rack controller broken after a period of time when deployed on a seperate machine from the region <MAAS:Invalid> <https://launchpad.net/bugs/1621090>
<Prabakaran> Hello Team, Getting timeout issue while commissioning the node in MAAS 1.9, Logs are pasted in here  http://pastebin.ubuntu.com/23301741/ could somebody help me on this?
<mup> Bug #1514436 changed: Support redirects for maas.ubuntu.com/images <MAAS:Fix Released by andreserl> <https://launchpad.net/bugs/1514436>
<baldpope> sup all - am I correct in understanding that maas cannot deploy to a vm running on the free version of esxi?
<baldpope> when attempting, I get a generic error - Unable to find a rack controller with access to chassis
<baldpope> i get that when running that from cli or webui when attempting to add chassis
<baldpope> and if I attempt to add a machine, i get this error in the webui -No rack controllers can access the BMC of node:
<roaksoax> baldpope: maas *can* deploy to a VM running on ESXI
<roaksoax> baldpope: the issue, based on what you mention, is that the rack controller cannot access the ESXI host IP
<baldpope> definitely not a firewall issue this time - no blocked traffic between the two
<roaksoax> baldpope: for example, you may not have routing between MAAS and the ESXi host you are trying to control
<roaksoax> baldpope: can you ping your ESXi host from the machine MAAS is running ?
<baldpope> i can run openssl s_client -connect against the vm server and do showcerts
<baldpope> yea, ping works as well
<roaksoax> baldpope: is the rack controller connect to MAAS ?
<roaksoax> baldpope: as it, it is showing a green check that everything is working ?
<baldpope> roaksoax, not sure I know where to look for that
<roaksoax> baldpope: in the UI, under the 'Nodes' tab, click on 'Controllers'
<baldpope> yea, green
<roaksoax> baldpope: strange then
<roaksoax> baldpope: so do this: tail -f /var/log/maas/rack.log and then add a chassis, grab the log it shows, and please file a bug :)
<roaksoax> baldpope: also, provide output pingin the IP where ESXi host is running and such
<baldpope> roaksoax, if I do attempt to add chassis, supplying the hostname, user/pass and prefix filter, I receive the following in the webui
<baldpope> Unable to find a rack controller with access to chassis <myhost>
<baldpope> but I see no output in rackd.log
<roaksoax> baldpope: what if you supply an IP instead of a hostname ?
<baldpope> tried both, same
<baldpope> I can kinda proceed if I set the power to manual and then manage power via the UI through vmware
<baldpope> was able to complete the commission step
<MrDanDan> hi guys
<MrDanDan> at the end of the commision phase, the nodes should power off, right?
<baldpope> yea
<roaksoax> baldpope: if you could file a bug with the logs above would be great
<MrDanDan> well, mine enter prompt
<baldpope> roaksoax, ok, will give you what I got
<MrDanDan> and on web UI hangs at commisioning
<roaksoax> MrDanDan: have your machines changed from "Commissioning" to "Ready"  ?
<MrDanDan> no
<roaksoax> MrDanDan: so then they are probably not accessing the commissioning environment
<MrDanDan> they stay commisioned
<MrDanDan> well, the network is a host-only on vmware
<MrDanDan> if that is of any issue
<roaksoax> MrDanDan: either a fail in PXE booting or loading the image, to tring to commission and something blocking it
<MrDanDan> think it was the fact the vms had ide disks attached
<MrDanDan> trying with scsi
<MrDanDan> issue was caused by some networking issues with HDCP
<MrDanDan> issue was caused by some networking issues with DHCP
<baldpope> roaksoax, https://github.com/conjure-up/conjure-up/issues/484
<stokachu> baldpope: yea the charm is attempting to locate a network device that isn't accessible
<baldpope> not even sure where to begin with that?
<baldpope> stokachu, what if deployment just takes a really long time - i ask because after waiting (for about 10+ mins) on 4 of the 5 blades, I see started lxd - main daemon
<baldpope> but the conjure-up session was already dead
<baldpope> is it possible to continue or?
<stokachu> it'll continue without conjure-up running
<stokachu> you'll just need to use juju for interfacing with it
<baldpope> juju status shows everything either in a waiting or pending state - nothing appears to have completed
<stokachu> neutron-gateway is in a failed state because of the network config
<stokachu> so that needs to be resolved first
<baldpope> yea, i just read your comment on github (assume that's you) - how is the network config broken, at the moment, all pretty flat with a single vlan
<baldpope> or is this more a basic issue like expecting eth[0-x] where my ethernet devices are named something else?
<smgoller> hey all, I've got maas set up and running. The maas DNS component seems to register all the machines in a .maas domain by default. This works fine. Machines that are provisioned correctly get the maas dns server as their default resolver. however, there is no 'search maas' line in the resolv.conf. Is there a maas setting that manages this?
<smgoller> so for example, I have a host called r01-s02. ping 'r01-s02' fails because it can't resolve the hostname because of no 'search maas' line in resolv.conf
<smgoller> Does anyone have any pointers to best practices when it comes to IPMI?
<smgoller> What I've been doing up to this point is putting IPMI traffic on a different vlan that is connected to the rack controller. I provide DHCP on that network via the rack controller. However, I'm running into a problem where there seems to be duplicate lease entries or something and I keep getting errors about keys already existing in the database for a given ip address.
#maas 2016-10-11
<mup> Bug #1631775 opened: canonical-kubernetes will fail to come up on bare metal <juju:Triaged> <MAAS:New> <https://launchpad.net/bugs/1631775>
<mup> Bug #1631775 changed: canonical-kubernetes will fail to come up on bare metal <juju:Triaged> <MAAS:New> <https://launchpad.net/bugs/1631775>
<mup> Bug #1631775 opened: canonical-kubernetes will fail to come up on bare metal <juju:Triaged> <MAAS:New> <https://launchpad.net/bugs/1631775>
<mup> Bug #1628114 opened: [FUJ] SSH input field not indicated for invalid username & the error is incomprehensible <MAAS:Triaged> <https://launchpad.net/bugs/1628114>
<mup> Bug #1628070 changed: [FUJ] In the Ubuntu section, maas.io should be the default selection <MAAS:Invalid> <https://launchpad.net/bugs/1628070>
<mup> Bug #1628070 opened: [FUJ] In the Ubuntu section, maas.io should be the default selection <MAAS:Invalid> <https://launchpad.net/bugs/1628070>
<mup> Bug #1628070 changed: [FUJ] In the Ubuntu section, maas.io should be the default selection <MAAS:Invalid> <https://launchpad.net/bugs/1628070>
<mup> Bug #1632395 opened: [2.1, Yakkety, UI]  UI error when adding a chassis <MAAS:Triaged> <https://launchpad.net/bugs/1632395>
<mup> Bug #1632395 changed: [2.1, Yakkety, UI]  UI error when adding a chassis <MAAS:Triaged> <https://launchpad.net/bugs/1632395>
<mup> Bug #1632395 opened: [2.1, Yakkety, UI]  UI error when adding a chassis <MAAS:Triaged> <https://launchpad.net/bugs/1632395>
<mup> Bug #1629909 changed: [FFE] New upstream release MAAS 2.1 <maas (Ubuntu):Fix Released> <https://launchpad.net/bugs/1629909>
<mup> Bug #1570934 changed: Reconfigure DHCP does not work <MAAS:Triaged> <https://launchpad.net/bugs/1570934>
<mup> Bug #1570934 opened: Reconfigure DHCP does not work <MAAS:Triaged by newell-jensen> <https://launchpad.net/bugs/1570934>
<baldpope> stokachu, just read your latest comment on the bug I sub'd
<baldpope> at least - i'm assuming that's you?
<stokachu> baldpope: yea
<stokachu> which bug was it?
<baldpope> 1632092
<baldpope> messed up net config for neutron
<stokachu> baldpope: ah yea, i just added additional options you can configure including the bridge-mappings
<stokachu> so you can give it a list of interfaces to try
<stokachu> useful if all machines have different network interfaces defined
<baldpope> i think I was talking with either you or roaksoax - if I were trying to setup up lag/lacp/bonded interfaces, would that be done in config, pre deploy or when? or should I not bond the nics?
<baldpope> and do you prep that in maas or modify one of the charms ?
<baldpope> i'm on the jujucharms setup - what is the difference between openstack base and telemetry?
<baldpope> err, jujucharms site demo.jujucharms.com
<vmorris> openstack-base doesn't include any ceilometer services
<mup> Bug #1632480 opened: Latest Yakkety grub timesout when attempting to UEFI boot over IPv6 <maas-ipv6> <MAAS:New> <grub2 (Ubuntu):New> <https://launchpad.net/bugs/1632480>
<mup> Bug #1632480 changed: Latest Yakkety grub timesout when attempting to UEFI boot over IPv6 <maas-ipv6> <MAAS:New> <grub2 (Ubuntu):New> <https://launchpad.net/bugs/1632480>
<mup> Bug #1632480 opened: Latest Yakkety grub timesout when attempting to UEFI boot over IPv6 <maas-ipv6> <MAAS:New> <grub2 (Ubuntu):New> <https://launchpad.net/bugs/1632480>
<mup> Bug #1632498 opened: [2.1] `maas-rack observe-dhcp` command only supports IPv4 <MAAS:New> <https://launchpad.net/bugs/1632498>
<mup> Bug #1632504 opened: [2.1] External DHCP detection logging should be more specific <MAAS:New> <https://launchpad.net/bugs/1632504>
#maas 2016-10-12
<cmart> has anyone tried deploying the Ubuntu 16.04 LTS image since it last updated today? I'm having errors during cloud_init, fails commissioning. on a set of hosts that I just deployed successfully last week.
<cmart> I get a Python traceback and "failed to start initial cloud-init job" on the console, unsure how to troubleshoot.
<cmart> is there a way to roll back an image to a version from a few days ago?
<cmart> It looks like you're storing some old versions here: http://images.maas.io/ephemeral-v2/daily/xenial/amd64/ Is there a way to tell MAAS to grab one of those?
<mup> Bug #1632638 opened: [2.1 UI] Include table sorting for device discovery <ui> <ux> <MAAS:Triaged> <https://launchpad.net/bugs/1632638>
<mup> Bug #1632642 opened: [2.1] Device discovery click on row functionality <MAAS:New> <https://launchpad.net/bugs/1632642>
<mup> Bug #1632642 changed: [2.1 UI] Device discovery click on row functionality <ui> <MAAS:Opinion> <https://launchpad.net/bugs/1632642>
<mup> Bug #1632642 opened: [2.1 UI] Device discovery click on row functionality <ui> <MAAS:Opinion> <https://launchpad.net/bugs/1632642>
<mup> Bug #1632642 changed: [2.1 UI] Device discovery click on row functionality <ui> <MAAS:Opinion> <https://launchpad.net/bugs/1632642>
<baldpope> stokachu, ping
<stokachu> baldpope: o/
<baldpope> had a question in response to bug 1632092
<baldpope> andrew's question (I believe to you) was asking where is the config done, but I had a follow-up question from yesterday's conversation about the net config
<baldpope> i was checking out the juju demo pages and (what appears to be) a charm creator - pretty slick btw
<stokachu> baldpope: ah the juju gui
<baldpope> up to this point, i had assumed that the combination of conjure/juju understood the infrastructure it was being deployed to automagically
<baldpope> stokachu, yea - that's some cool shit
<baldpope> so anyway - then it dawned on me - with all the different nics, out there, it's possible that I should be prepping the installation scripts with some specifics about my environment
<baldpope> so to finally get to the question - instead of just running the conjure-up openstack - is there some prep work I should have completed to tell about the target environment?
<stokachu> so with juju you can set constraints to deploy certain applications to specific systems in maas
<stokachu> which you would set in the bundle itself and use conjure-up that way
<baldpope> but the application constraints are more about (asking) which applications will be installed, the components, but not necessarily (again asking) about the nics, bonding, vlans, etc?
<stokachu> baldpope: correct
<stokachu> in a normal scenario we ask people to make sure all machines have at least 2nics and 2hdds for a successful openstack deployment
<stokachu> via maas
<baldpope> ok, so a 'stock' openstack charm/spell installation is probably fine but then where do I set bonding and the desired vlans, etc
<baldpope> right, and each of my nodes are 6 nics and 8 hdd
<stokachu> baldpope: so the charm would expose some options for you to setup bonding
<baldpope> and ignoring the conversation we had last week about nfs/iscsi root, I'd like each node to share the disks to the stack for storage, and each node to be a compute node (if htat's possible)
<baldpope> stokachu, and the charm can be run either before or after base installation?
<stokachu> both, the config changes should apply without failure at any point of the deployment
<baldpope> hm
<stokachu> baldpope: maas also gives you some flexibility on defines your networks
<baldpope> ok.. i'm not whining (it's going to sound like it...) ... then why does the conjure-up openstack seem to fail deployment every time I've attempted to deploy, if the base charm/spell should work without any customization to the nics
<stokachu> baldpope: for openstack novalxd we have control over that environment so it can deploy successfully without any configuration
<baldpope> ok
<stokachu> baldpope: with maas you still need to perform some configuration via the charm options within conjure-up
<baldpope> ok, then I think that's the part I've been missing
<baldpope> during the conjure-up, it deploys to 1 host first, from conversation with either you or roaksoax , I recall being advised that most people run that in a VM?
<baldpope> does that host also need dual nics and dual drive?
<baldpope> mine currently does not
<stokachu> baldpope: you dont need dual nics/hdds for the controller node
<baldpope> and not knowing how to size it, i simply doled out 4gb ram and 32gb disk
<stokachu> yea that should be plenty
<baldpope> k
<stokachu> conjure-up has --bootstrap-to that you can set to make sure that vm host is used for the controller node
<stokachu> only applies to a maas deployment
<baldpope> cool, I noticed that when conjure-up is running, it must sort node names, so I have mine correctly numbers blade0 through blade5
<baldpope> with 0 being the vm
<baldpope> but I like the cli option better - no guess work
<stokachu> yea i wouldnt rely on that sorting im not sure if that's intentional or not
<baldpope> heh
<stokachu> that would probably be a juju question
<stokachu> i thought juju just randomly picked an avaialble machine for the controller node
<baldpope> fair enough - i'll use the cli option going forward
<baldpope> i've done it a couple of times now and it always seemed to grab the lowest numbered one - assuming you set them up intentionally through maas instead of doing the auto discover with random host names
<stokachu> that would be interesting to find out
<stokachu> i should ask juju guys about it
<baldpope> i could be wrong - maybe (as you said) could be unintentional side effect
<stokachu> im asking now ill let you know what they say
<baldpope> cool
<baldpope> feature request for maas - can we get the mac address added to the display in maas under the node interface listings - it would help identify which nics are which
<baldpope> it says name/mac - but only name is listed
<baldpope> ah fuck,..
 * baldpope sigh
<baldpope> nevermind
<stokachu> :D
<brendand> baldpope, but it could be more obvious, honestly
<baldpope> thanks, appreciate it
<baldpope> actually, my issue was i had the browser window relatively small, and MAC was partially covered
<baldpope> when I expanded the window I realized (finally) the link
<baldpope> stokachu, possible reason why it's failing, eno1 is listed as the first nic, but it is not the nic that is plugged in
<stokachu> did you set bridge-mappings to a list of interfaces to try?
<baldpope> eno1, along with ens3f1, ens3f2, and ens3f3 are the add-on card
<baldpope> stokachu, not yea, back and forth with work work
<stokachu> baldpope: gotcha, try to put them all in the bridge-mappings list
<stokachu> see if it'll pick up the plugged in one
<baldpope> stokachu, would I want to bond them together in maas first?
<stokachu> sure you can do that
<baldpope> k
<stokachu> you'd want to do it for all machines
<stokachu> because you won't know which one gets picked for neutron-gateway
<baldpope> yea, I had planned to configure each in maas the same way
<stokachu> cool yea that should be it then
<batkins61> I can't find maas-image-builder in either maas-maintainers/stable or maas/stable, as described in the docs.  How do I get an RHEL custom image to install?
<brendand> batkins61, yeah that changed
<brendand> ltrager, ^
<ltrager> batkins61: you need RHEL not CentOS?
<roaksoax> batkins61: maas-image-builder is no longer available provided it only gave support for CentOS and we now provide centos automatically
<brendand> roaksoax, docs claim it supports RHEL as well
<batkins61> roaksoax the web page says RHEL7 too.
<batkins61> roaksoax 1.7+
<batkins61> https://maas.ubuntu.com/docs/os-support.html
<batkins61> Red Hat logo is also on the main page "Zero-touch deployment of Windows, Ubuntu, CentOS, RHEL and SUSE".    Is there docs on how to access that in 1.9 or 2.0?
<blake_r> batkins61: maas.io in the pricing section
<roaksoax> batkins61: that documentation is OLD and it is in the process of being removed
<blake_r> batkins61: with Ubuntu Advantange you get access to All operating systems
<roaksoax> batkins61: if you want support for RHEL7, at the moment, you'd need to contact Canonical and get Ubuntu Advantage for MAAS as blake_r just mentioned
<batkins61> roaksoax blake_r Thanks, that clarifies things
<baldpope> stokachu, ok, just finished wiring up the bonded nics - you mentioned bridge-mappings earlier - where am I supposed to configure that? in conjure, juju, ?
<stokachu> baldpope: yea you can do it in conjure-up
<stokachu> when you see the list of applications press the [configure] button next to neutron-gateway
<stokachu> there you can enter your mappings
<baldpope> also created the bonded nic in maas
<baldpope> ok, will do that now
 * baldpope sigh
<baldpope> stokachu, have to pick it up in a bit - dev issue going to take my time for a little bit
<stokachu> ok
<mup> Bug #1631934 changed: [2.1] Option to not have MTU values on the bridged interface <MAAS:Invalid> <https://launchpad.net/bugs/1631934>
<mup> Bug #1632759 opened: Support for images needs to be clarified <MAAS:New> <https://launchpad.net/bugs/1632759>
<mup> Bug #1632763 opened: [2.1] Device discovery IP Assignment label incorrect <MAAS:Incomplete> <https://launchpad.net/bugs/1632763>
<Prabakaran> Hello Team, Getting timeout issue while commissioning the node in MAAS 1.9, Logs are pasted in here  http://pastebin.ubuntu.com/23301741/ could somebody help me on this?
<Prabakaran> Is there any document or video ref to create a Vrish network which uses DHCP /DNS configuration in maas?
<roaksoax> Prabakaran: no there's not. you just need a virsh network that doesn't have DHCP managed by the virsh network
<brendand> Prabakaran, either use virt-manager -> Edit -> Connection details, or look up virsh net-create
<brendand> Prabakaran, you can do like:
<brendand> virsh net-dumpxml default > maas.xml
<brendand> remove the <dhcp> section
<brendand> virsh net-create maas maas.xml
<brendand> or rather
<brendand> virsh net-create maas.xml (and edit <name> in maas.xml as well)
<Prabakaran> brendand:  in the maas console i have edited etho interface management to DHCP + DNS and configured IP, Dynamic ips and Static ip.. Should i reset it also?
<mup> Bug #1632763 changed: [2.1] Device discovery IP Assignment label incorrect <MAAS:Won't Fix> <https://launchpad.net/bugs/1632763>
<mup> Bug #1632804 opened: ROUNDTTT in configure_networking is effectively ignored <MAAS:New> <initramfs-tools (Ubuntu):New> <https://launchpad.net/bugs/1632804>
<mup> Bug #1632804 changed: ROUNDTTT in configure_networking is effectively ignored <MAAS:New> <initramfs-tools (Ubuntu):New> <https://launchpad.net/bugs/1632804>
<mup> Bug #1632804 opened: ROUNDTTT in configure_networking is effectively ignored <MAAS:New> <initramfs-tools (Ubuntu):New> <https://launchpad.net/bugs/1632804>
<mup> Bug #1632808 opened: configure_networking exits without any ipv6 routes <maas-ipv6> <MAAS:New> <initramfs-tools (Ubuntu):New> <https://launchpad.net/bugs/1632808>
<mup> Bug #1632815 opened: Node failed to be released, because of the following error: 'NoneType' object has no attribute 'addErrback' <MAAS:New> <https://launchpad.net/bugs/1632815>
<mup> Bug #1632853 opened: [2.1] Observed neighbours should be avoided when assigning IP addresses <MAAS:Triaged by mpontillo> <https://launchpad.net/bugs/1632853>
<baldpope> stokachu, neutron-gateway/bridge-mappings
<baldpope> set to bond0.$vlan ?
<mup> Bug #1632853 changed: [2.1] Observed neighbours should be avoided when assigning IP addresses <MAAS:Triaged by mpontillo> <https://launchpad.net/bugs/1632853>
<mup> Bug #1632853 opened: [2.1] Observed neighbours should be avoided when assigning IP addresses <MAAS:Triaged by mpontillo> <https://launchpad.net/bugs/1632853>
<mup> Bug #1632862 opened: [2.1. Yakkety] "Map subnet" action doesn't work <MAAS:Triaged> <https://launchpad.net/bugs/1632862>
<mup> Bug #1632862 changed: [2.1. Yakkety] "Map subnet" action doesn't work <MAAS:Triaged> <https://launchpad.net/bugs/1632862>
<mup> Bug #1632862 opened: [2.1. Yakkety] "Map subnet" action doesn't work <MAAS:Triaged> <https://launchpad.net/bugs/1632862>
#maas 2016-10-13
<qgriffith> Hello all I am having a weird issue I can't trace down. Running MAAS Version 2.0.0+bzr5189-0ubuntu1 (16.04.1) using conjure-up to build out an Openstack on MAAS. Some of the lxc containers that are build during the deploy are not able to get on the network. They get an IP but I can't ping or juju ssh into them. When I access these containers via lxc exec they have an IP assigned but can't ping anything. It is not happening on all the
<qgriffith> They seem to happen randomly on different pieces of hardware. So one of the machines will have three lxc containers and all three can't ping.
<mup> Bug #1633064 opened: [0-day SRU] MAAS 2.1.0 RC1 <maas (Ubuntu):New> <https://launchpad.net/bugs/1633064>
<circ-user-Jnn09> NICK ramzgt
<circ-user-Jnn09> woops sorry, irc newb here
<ramzgt> is Blake on?
<ramzgt> hello cmart
<cmart> howdy ramzgt
<cmart> have we met?
<ramzgt> no not yet, I was introduced to MAAS last night at a meetup event and wanted to get involved in testing
<cmart> cool. I was introduced to MAAS a few weeks ago. I'm using it to provision servers that run OpenStack. overall experience has been positive despite a few hiccups. definitely better than installing by hand!
<mup> Bug # changed: 1536354, 1569365, 1598470, 1603466, 1605476, 1608555, 1628114, 1629061, 1629475, 1629604, 1630633, 1630667, 1631022, 1631024, 1631079, 1631152, 1631358, 1631420, 1632395, 1632815, 1632862, 1633067
<ramzgt> same, I am looking forward to transitioning off of AWS and onto my own hardware. I believe Canonical is doing everyone a huge service
<mup> Bug # changed: 1065456, 1382258, 1483206, 1483947, 1515188, 1532359, 1536134, 1546208, 1549303, 1555178, 1575337, 1581219, 1581224, 1582070, 1587542, 1587544, 1587549, 1587864, 1590946, 1593219, 1597968, 1599223, 1600259, 1600264, 1600267, 1600285, 1602177, 1602487, 1603147, 1604111, 1604987,
<mup> 1605124, 1606264, 1606499, 1607560, 1610276, 1610277, 1610278, 1610294, 1610397, 1610414, 1618752, 1618763, 1618785, 1619337, 1619693, 1621285, 1621446, 1625707, 1625709, 1626942
<cmart>  ramzgt are you running your own cloud? OpenStack or something similar?
<mup> Bug #1633181 opened: Rack and Region disconnections <docteam> <MAAS:New> <https://launchpad.net/bugs/1633181>
<ramzgt> hey joe
<circ-user-WBQeB> \nick jeauxfranklin
<mup> Bug #1631132 changed: [2.1 yakkety] jquery script failure on MAAS UI Nodes page <cdo-qa> <MAAS:Invalid> <https://launchpad.net/bugs/1631132>
<mup> Bug #1631132 opened: [2.1 yakkety] jquery script failure on MAAS UI Nodes page <cdo-qa> <MAAS:Invalid> <https://launchpad.net/bugs/1631132>
<mup> Bug #1631132 changed: [2.1 yakkety] jquery script failure on MAAS UI Nodes page <cdo-qa> <MAAS:Invalid> <https://launchpad.net/bugs/1631132>
<mup> Bug #1633209 opened: Daily Images appear to have broken for xgene-uboot <MAAS:New> <https://launchpad.net/bugs/1633209>
<mup> Bug #1633209 changed: Daily Images appear to have broken for xgene-uboot <MAAS:New> <https://launchpad.net/bugs/1633209>
<mup> Bug #1633209 opened: Daily Images appear to have broken for xgene-uboot <MAAS:New> <https://launchpad.net/bugs/1633209>
<mup> Bug #1633247 opened: Used fallback datasource. PXE boot fail <MAAS:New> <https://launchpad.net/bugs/1633247>
#maas 2016-10-14
<smgoller> Hey all, I'm running into a situation with a fresh maas install where it's trying to download cloud-init information from the subnet's router instead of the rack controller. Any ideas on how to debug this?
<smgoller> basically cloud-init runs through all these things then gives up.
<smgoller> so looking at /etc/maas/preseeds/enlist, I see a reference to a variable metadata_enlist_url. Anyone know how that is calculated?
<pmatulis> smgoller, what version of maas? are you talking about importing boot images?
<smgoller> maas 2.0. I'm talking about trying to add a node to maas automatically via PXE.
<smgoller> the machine pulls down the boot image, starts up, then cloud-init runs on the enlisting machine to get information, but it is never able to get anything from the rack controller so enlisting doesn't happen.
<smgoller> it basically ends up netbooting an ubuntu image that just sits there and i can't get into it.
<pmatulis> odd
<smgoller> ayup
<pmatulis> did you check the rackd and regiond logs?
<smgoller> nothing particularly interesting but i'll check again.
<mup> Bug #1633271 opened: [1.9,2.0,2.1] IP addresses without a subnet can cause IP allocation to permanently fail <MAAS:Triaged> <https://launchpad.net/bugs/1633271>
<smgoller> regiond logs has a bunch of stuff something trying to connect to itself and failing
<smgoller> "provisioningserver.rpc.clusterservice.ClusterClientService"
<smgoller> ok, it looks like the problem stems from the fact that i'm running the region and rack controllers on the same machine.
<smgoller> so i'm going to split the region and rack controllers back into separate machines.
<mup> Bug #1633064 changed: [0-day SRU] MAAS 2.1.0 <verification-done> <maas (Ubuntu):Fix Released> <maas (Ubuntu Yakkety):Fix Released> <maas (Ubuntu Z-series):Fix Committed> <https://launchpad.net/bugs/1633064>
<pmatulis> smgoller, that would be surprising. the recommended setup is to have them together. i've certainly never had the problem you're describing
<smgoller> eh, it's a home setup, nothing production, so i'm taking it as an opportunity to learn how maas works under the hood
<smgoller> so I see when it pxe boots that it's getting a metadata url but it seems to be ignoring it. anyway, I'll mess with it more tomorrow.
<mup> Bug #1633378 opened: [Device Discovery] Rename the section Header in the Settings page <MAAS:Triaged> <https://launchpad.net/bugs/1633378>
<mup> Bug #1633397 opened: [Device Discovery] In the subnet details page change the label of the  "Active discovery" to "Active mapping" <ui> <MAAS:New> <https://launchpad.net/bugs/1633397>
<mup> Bug #1633399 opened: [Device Discovery] Rename the action in the action button in Subnet details to Map <ui> <MAAS:New> <https://launchpad.net/bugs/1633399>
<mup> Bug #1633401 opened: [Device Discovery] In device discovery in Settings, remove the header of the first dropdown field (Host discovery and network observation) <ui> <MAAS:New> <https://launchpad.net/bugs/1633401>
<mup> Bug #1633452 opened: [Device discovery] In settings, rename the option Disabled (suppress active scanning) in the Active discovery interval field to Never (disabled). <ui> <MAAS:New> <https://launchpad.net/bugs/1633452>
<mup> Bug #1633457 opened: [Device Discovery] In settings rename the header "Active discovery interval" to "Active subnet mapping interval" <ui> <MAAS:New> <https://launchpad.net/bugs/1633457>
<mup> Bug #1633462 opened: [Device Discovery] In settings in the device discovery section reduce the text of the explanation of the fields. <ui> <MAAS:New> <https://launchpad.net/bugs/1633462>
<mup> Bug #1633462 changed: [Device Discovery] In settings in the device discovery section reduce the text of the explanation of the fields. <ui> <MAAS:New> <https://launchpad.net/bugs/1633462>
<mup> Bug #1633462 opened: [Device Discovery] In settings in the device discovery section reduce the text of the explanation of the fields. <ui> <MAAS:New> <https://launchpad.net/bugs/1633462>
<mup> Bug # opened: 1633467, 1633468, 1633469, 1633470
<mup> Bug #1633555 opened: [2.1] link-subnet API should allow arbitrary subnets if interface is disconnected <MAAS:Triaged> <https://launchpad.net/bugs/1633555>
<mup> Bug #1633598 opened: [2.1.1] Unable to set local boot-source via the API <MAAS:Confirmed for ltrager> <https://launchpad.net/bugs/1633598>
<mup> Bug #1633600 opened: [2.1.1] Docs do not mention the need to mirror bootloaders <MAAS:Confirmed for ltrager> <https://launchpad.net/bugs/1633600>
<mup> Bug #1633598 changed: [2.1.1] Unable to set local boot-source via the API <MAAS:Confirmed for ltrager> <https://launchpad.net/bugs/1633598>
<mup> Bug #1633600 changed: [2.1.1] Docs do not mention the need to mirror bootloaders <MAAS:Confirmed for ltrager> <https://launchpad.net/bugs/1633600>
<mup> Bug #1633598 opened: [2.1.1] Unable to set local boot-source via the API <MAAS:Confirmed for ltrager> <https://launchpad.net/bugs/1633598>
<mup> Bug #1633600 opened: [2.1.1] Docs do not mention the need to mirror bootloaders <MAAS:Confirmed for ltrager> <https://launchpad.net/bugs/1633600>
<mup> Bug #1595279 changed: [2.0] ipaddresses API doesn't show hostname when reading <oil> <MAAS:Won't Fix> <https://launchpad.net/bugs/1595279>
<TheCompWiz> quick question... on 16.04lts... what do I need to get MAAS to manage DHCP? ... I've followed every bit of direction I can find on the wiki... but I'm still stumped.
<TheCompWiz> do I need to install the dhcpd?  or is that part of the default install? or what am I missing?
<TheCompWiz> scratch that... I see isc-dhcp-server is already installed...
<TheCompWiz> so... why didn't it start?
<mup> Bug #1633641 opened: A "termination protection"-like feature would be cool <MAAS:New> <https://launchpad.net/bugs/1633641>
<mup> Bug #1633641 changed: A "termination protection"-like feature would be cool <MAAS:New> <https://launchpad.net/bugs/1633641>
<spaok> hey guys, how can I create a GPT partition via the maas command line? I don't see where I can set partition_table_type
<mup> Bug #1633641 opened: A "termination protection"-like feature would be cool <MAAS:New> <https://launchpad.net/bugs/1633641>
#maas 2016-10-15
<mup> Bug #1633663 opened: Unlink interfaces creates a new link <MAAS:New> <https://launchpad.net/bugs/1633663>
<ai114> Hi! I've just a (maybe stupid) question. Do I have to run the MAAS all the time or is it possible to deploy the OS on all the nodes and then shutdown the MAAS? What happens if I start the MASS again - will I see the nodes?
<spaok> ai114: MAAS tends to be be DHCP and DNS for the nodes, so you would need to account for that lost
<spaok> the new MAAS/Juju 2.0 HA uses DNS in MAAS, so that would also be lost
<ai114> spaok: THANK you!
<spaok> you can run MAAS ina container if you are trying to save resources
<ai114> that's a good idea!
<spaok> my dev env is MAAS in a container and a management VM, Juju VM, and two comp VMs in KVM, then the nova compute uses LXD
<spaok> works pretty good
<ai114> Thanks again - your config should work for me too.
<mup> Bug #1633696 opened: regiond log leaks a lot of twisted <MAAS:New> <https://launchpad.net/bugs/1633696>
<mup> Bug #1633717 opened: DHCP probe reports 'failure' when its really just completed <MAAS:New> <https://launchpad.net/bugs/1633717>
<mup> Bug #1633726 opened: rackd tftp logging leaks a lot of python-isms <MAAS:New> <https://launchpad.net/bugs/1633726>
<mup> Bug #1633727 opened: connectionpool leaks python-isms in log <MAAS:New> <https://launchpad.net/bugs/1633727>
<mup> Bug #1633763 opened: MAAS wants to do a dhcp probe on a slave interface in a bond <MAAS:New> <https://launchpad.net/bugs/1633763>
<mup> Bug #1633764 opened: MAAS dhcp probes on an internal bridge <MAAS:New> <https://launchpad.net/bugs/1633764>
<mup> Bug #1633763 changed: MAAS wants to do a dhcp probe on a slave interface in a bond <MAAS:New> <https://launchpad.net/bugs/1633763>
<mup> Bug #1633764 changed: MAAS dhcp probes on an internal bridge <MAAS:Invalid> <https://launchpad.net/bugs/1633764>
<moparlakci> ola
#maas 2016-10-16
<jiffe> so if I want to still use my router's dhcp server for the desktops on my network I will have to segment my network and put maas machines on a different network from the desktops?
<mup> Bug #1633822 opened: Device discovery ignores reverse DNS <MAAS:New> <https://launchpad.net/bugs/1633822>
<mup> Bug #1633822 changed: Device discovery ignores reverse DNS <MAAS:New> <https://launchpad.net/bugs/1633822>
<mup> Bug #1633822 opened: Device discovery ignores reverse DNS <MAAS:New> <https://launchpad.net/bugs/1633822>
<smgoller> hey all, any ideas on troubleshooting remote virsh management? I've followed the instructions on the maas website for setting up ssh keys for a user and such, and virsh -c <my host> list --all works fine. however maas still says there's a problem logging in. any ideas?
<pmatulis> smgoller, in what way does maas report a problem logging in?
<smgoller> pmatulis: Failed to login to virsh console.
<smgoller> that is via the gui
<smgoller> no other interesting information seems to be in the logs that I can find
<pmatulis> smgoller, so you became the 'maas' user and this works: virsh -c qemu+ssh://<user>@<kvm host>/system list --all ?
<pmatulis> smgoller, but the web UI has a problem?
<smgoller> yup
<smgoller> pmatulis, that is exactly correct
<smgoller> let me try something else.
<pmatulis> smgoller, so the power check for a node must also not be working. i would 'tail -f' /var/log/auth.log on the kvm host and try a power check from the UI. see that pops up there
<smgoller> ok
<pmatulis> s/that/what
<mup> Bug #1633247 changed: Used fallback datasource. PXE boot fail <MAAS:Invalid by charlienz> <https://launchpad.net/bugs/1633247>
<smgoller> hm. it doesn't actually try to connect. I'm going to delete the node and re-add it.
<pmatulis> smgoller, maybe you didn't configure the BMC correctly
<smgoller> weird. So I copied the URL from the one not working, deleted the machine, then used the 'add chassis' with the virsh power driver and information, found it just fine
<smgoller> <shrug>
<smgoller> chicken and egg problem maybe? Originally I had booted the VM to let maas discover it, then set the power type afterwards
<mup> Bug #1633247 opened: Used fallback datasource. PXE boot fail <MAAS:Invalid by charlienz> <https://launchpad.net/bugs/1633247>
<mup> Bug #1633247 changed: Used fallback datasource. PXE boot fail <MAAS:Invalid by charlienz> <https://launchpad.net/bugs/1633247>
#maas 2017-10-09
<plumo> Good morning everyone!!
<plumo> Question! Is that normal that my MAAS Server do not correctly answers DNS Requests when I'm doing something like: dig @10.0.49.254 my-little-poney.maas.mycompany.org ? even if I've set my MAAS to be Authoritative DNS Server on this subdomain and my servers deployed with maas only using MAAS DNS Resolver?
<roaksoax> plumo: what version of maas are you using ?
<plumo> roaksoax: Hey ! I'm using 2.2.2
<roaksoax> plumo: try upgrading to 2.2.3 which is in ppa:maas/proposed
<roaksoax> plumo: it contains a couple of fixes that should improve DNS
<plumo> ok, I'll do it right now.
<plumo> roaksoax: PERFECT!! it fix everything regarding our DNS issue \o/ Amazing job!
<plumo> here is another question
<plumo> Is juju able to use MAAS tags as constraints to deploy stacks?
<roaksoax> plumo: yes
<plumo> Nice! Got it.
<plumo> roaksoax: Just a quick improve that could be nice to have. When a proxy is set into the setting tab, it should be set as http_proxy and https_proxy vars for the default user and the default sudoers file should get the "Defaults env_keep = "http_proxy https_proxy ftp_proxy no_proxy"" values set.
<chamar> Hello.  Is there a way to disable (or not enable) swap on deployed server using MAAS?
<plumo> chamar: You can delete the swap partition from the storage tab.
<chamar> plumo, Hum.. Can't see it.  it's on a KVM (as a Pod), can it be different?!
<plumo> Sorry, I'm not working with PODS and VMs on MAAS, only using it for BM Purposes.
<chamar> no worries.  I'll dig  a bit more.  Thanks.
<ybaumy> roaksoax: hey ho
<ybaumy> roaksoax: is the bug fixed with the dns resolv
<mup> Bug #1722406 opened: [2.3] API allows "deploying" a machine that's already deployed <MAAS:Triaged> <https://launchpad.net/bugs/1722406>
#maas 2017-10-10
<mup> Bug #1635107 changed: maas.power Error changing power state (on) while commissioning the node <MAAS:Expired> <https://launchpad.net/bugs/1635107>
<mup> Bug #1635107 opened: maas.power Error changing power state (on) while commissioning the node <MAAS:Confirmed> <https://launchpad.net/bugs/1635107>
<mwe1> hi there, does anybody knows website with a good tutotial how to trigger ansible via maas?
<v92> mwe1: I'm planning to use ansible-pull method and do the ansible installation and configuration during deployment phase
<mup> Bug #1722526 opened: [2.3] Cannot power manage a region/rack controller (which was a node deployed by MAAS) <MAAS:Triaged> <https://launchpad.net/bugs/1722526>
<mup> Bug #1722531 opened: [2.3] No way to know what version of MAAS/curtin deployed  amachine <MAAS:New> <https://launchpad.net/bugs/1722531>
<mup> Bug #1722532 opened: [2.3] No way to know what version of MAAS/curtin deployed  amachine <MAAS:Triaged by andreserl> <https://launchpad.net/bugs/1722532>
<rbasak> mwe1: I don't know if this is relevant to you, but a common way to get newly started Ubuntu images to do things is to use cloud-init - including on MAAS, lxd, EC2, etc.
<rbasak> It can add apt repositories, install packages, and then run commands on first boot, for example, it's present on all Ubuntu cloud images (and most other distributions' cloud images), and you can use MAAS to pass instructions to it.
<roaksoax> /wi/win 4
<mup> Bug #1722538 opened: Server header should provide MAAS version <papercut> <MAAS:Triaged> <https://launchpad.net/bugs/1722538>
<mup> Bug #1722538 changed: Server header should provide MAAS version <papercut> <MAAS:Triaged> <https://launchpad.net/bugs/1722538>
<mup> Bug #1722538 opened: Server header should provide MAAS version <papercut> <MAAS:Triaged> <https://launchpad.net/bugs/1722538>
<mup> Bug #1722578 opened: Controller <> is running an older version of MAAS (less than 2.3.0) <cdo-qa> <foundations-engine> <MAAS:New> <https://launchpad.net/bugs/1722578>
<mup> Bug #1722589 opened: syslog full of "topology hint" logs <cdo-qa> <foundations-engine> <MAAS:In Progress by mpontillo> <https://launchpad.net/bugs/1722589>
<jamesbenson> does anyone work with maas and juju?
<mup> Bug #1722607 opened:  [2.3, HWTv2] Logs still sent with node object <MAAS:Triaged> <https://launchpad.net/bugs/1722607>
<mup> Bug #1722609 opened:  [2.3, HWTv2] Add ability to view previous results on the node-results page <MAAS:Triaged> <https://launchpad.net/bugs/1722609>
<mup> Bug #1722620 opened: [2.3] Failed to update regiond's process and endpoints; records may be out of date <MAAS:Triaged> <https://launchpad.net/bugs/1722620>
<mup> Bug #1722620 changed: [2.3] Failed to update regiond's process and endpoints; records may be out of date <MAAS:Triaged> <https://launchpad.net/bugs/1722620>
<mup> Bug #1722620 opened: [2.3] Failed to update regiond's process and endpoints; records may be out of date <MAAS:Triaged> <https://launchpad.net/bugs/1722620>
<mup> Bug #1722646 opened: interfaces do not have ips <cdo-qa> <cdo-qa-blocker> <foundations-engine> <MAAS:Incomplete> <https://launchpad.net/bugs/1722646>
<mup> Bug #1719643 changed: [2.3, UI, HWTv2] Storage Card needs scrolling for multiple different disks <MAAS:Won't Fix by newell-jensen> <https://launchpad.net/bugs/1719643>
<mup> Bug #1721743 opened: [2.3b2] Rack and region controller versions still not updated <MAAS:Triaged by mpontillo> <https://launchpad.net/bugs/1721743>
<mup> Bug #1722665 opened: [2.3, HWTv2] MAAS stores a limited amount of test results <MAAS:Triaged> <https://launchpad.net/bugs/1722665>
<mup> Bug #1722671 opened: [2.3, pod] Unable to delete a machine or a pod if the pod no longer exists <MAAS:New> <https://launchpad.net/bugs/1722671>
#maas 2017-10-11
<mup> Bug #1722693 opened: Websocket is constantly resending known data <MAAS:Triaged> <https://launchpad.net/bugs/1722693>
<vinay_> hi
<vinay_> anybody there ??
<mup> Bug #1722790 opened: Node commissioning fails with Hash Sum Mismatch error for repo <MAAS:New> <https://launchpad.net/bugs/1722790>
<mup> Bug #1722790 changed: Node commissioning fails with Hash Sum Mismatch error for repo <MAAS:New> <https://launchpad.net/bugs/1722790>
<mup> Bug #1722790 opened: Node commissioning fails with Hash Sum Mismatch error for repo <MAAS:New> <https://launchpad.net/bugs/1722790>
<LikeLuc> hey, my maas don't commissioning storage
<LikeLuc> do u have any idea?
<LikeLuc> i am using maas 2.2.2 i want to add a node with 30 GB stoage but mas only finde ram and cpu
<mup> Bug #1722803 opened: Unable to decompose KVM pod <MAAS:New> <https://launchpad.net/bugs/1722803>
<LikeLuc> so i have to wait for the next patch right?
<mup> Bug #1722790 changed: Node commissioning fails with Hash Sum Mismatch error for repo <MAAS:Invalid> <https://launchpad.net/bugs/1722790>
<mup> Bug #1722803 changed: Unable to decompose KVM pod <MAAS:New> <https://launchpad.net/bugs/1722803>
<LikeLuc> hey, my maas don't commissioning storage, i am using maas 2.2.2 i want to add a node with 30 GB stoage but mas only find ram and cpu
<mup> Bug #1722790 opened: Node commissioning fails with Hash Sum Mismatch error for repo <MAAS:Invalid> <https://launchpad.net/bugs/1722790>
<mup> Bug #1722803 opened: Unable to decompose KVM pod <MAAS:New> <https://launchpad.net/bugs/1722803>
<mup> Bug #1722790 changed: Node commissioning fails with Hash Sum Mismatch error for repo <MAAS:Invalid> <https://launchpad.net/bugs/1722790>
<mup> Bug #1722803 changed: Unable to decompose KVM pod <MAAS:New> <https://launchpad.net/bugs/1722803>
<mup> Bug #1722848 opened: [2.3, HWTv2] Memtester test is not robust <MAAS:Confirmed> <https://launchpad.net/bugs/1722848>
<mup> Bug #1722851 opened: [2.3 beta 1, UX improvement] Surface on the UI the ability to add a comment for an action and present the comments on the UI <MAAS:New for m-vrachnis> <https://launchpad.net/bugs/1722851>
<jamesbenson> roaksoax: you around?
<mup> Bug #1722863 opened: [2.2.x./2.3.x, Accessibility] Apply accessibility guidelines in the features that have been developed in the latest releases by the MAAS team and are not accessible <MAAS:New for m-vrachnis> <https://launchpad.net/bugs/1722863>
<mup> Bug #1722863 changed: [2.2.x./2.3.x, Accessibility] Apply accessibility guidelines in the features that have been developed in the latest releases by the MAAS team and are not accessible <MAAS:New for m-vrachnis> <https://launchpad.net/bugs/1722863>
<mup> Bug #1722863 opened: [2.2.x./2.3.x, Accessibility] Apply accessibility guidelines in the features that have been developed in the latest releases by the MAAS team and are not accessible <MAAS:New for m-vrachnis> <https://launchpad.net/bugs/1722863>
<mup> Bug #1722902 opened: [2.3, HWTv2] MAAS storage test reports 7 tests passed, 1 timed out, but only 7 tests have been  <MAAS:New> <https://launchpad.net/bugs/1722902>
<mup> Bug #1722903 opened: [2.3, HWTv2] MAAS storage test reports 7 tests passed, 1 timed out, but only 7 tests have been  <MAAS:New> <https://launchpad.net/bugs/1722903>
<mup> Bug #1722910 opened: [2.3b1] In Devices->Interfaces when the focus is int he MAC address the Enter key opens the menu of the previous row <ui> <MAAS:Triaged> <https://launchpad.net/bugs/1722910>
<mup> Bug #1722913 opened: [2.3 beta 1, Keyboard nav] In machine details->Interfaces, when I am editing a bond/interface and I am focused in the bond/interface name and I hit Enter, the contextual menu (of the same bond/interface) opens <ui> <MAAS:New> <https://launchpad.net/bugs/1722913>
<mup> Bug # changed: 1710092, 1711760, 1719015, 1719353, 1719361, 1721105, 1721108, 1721111, 1721113, 1721273, 1721276, 1721524, 1721525, 1721548, 1721587, 1722589
<mup> Bug # changed: 1681872, 1681876, 1681887, 1682379, 1683810, 1683816
#maas 2017-10-12
<mup> Bug #1722962 opened: Connection broken: IncompleteRead on rackd <MAAS:New> <https://launchpad.net/bugs/1722962>
<TheGOro> Hi all
<TheGOro> Can I have a Dell related question?
<TheGOro> When I am trying to comission manually defined nodes, MaaS duplicates them and defines the duplicates with different IPMI credentials
<TheGOro> It seems that it creates maas IPMI user for some reasons
<roaksoax> TheGOro: maas autodetects IPMI and auto-configures it for usage, yes
<spaniard> apologies if this is a know issue: bootstrapping Juju controller Attempting to connect <IP>:22
<spaniard> what did i miss?
<catbus> spaniard: did the node with <IP> come up right? Can you ping the node via that <IP> from maas server?
<spaniard> yes the node comes up, displays the correct info <Ubuntu... number of cores and shows powered on > and is pingable
<catbus> spaniard: does it show deployed successfully on maas? if so, how long have juju bootstrap been stuck at Attempting to connect <IP>:22/
<catbus> ?
<spaniard> catbus: maybe an hour stuck on "Attempting to ssh"
<catbus> spaniard: can you ssh to the machine from MAAS?
<catbus> spaniard: I'd kill current juju bootstrap, make sure the machine is released on MAAS, then juju bootstrap again with --debug flag to get more info.
<spaniard> catbus: I can not ssh, did i miss adding a key somewhere?
<catbus> spaniard: if you can't ssh ubuntu@<IP> from MAAS, you should check if an ssh key is imported in MAAS, click on the admin account on the top right, then it will bring you to a page where you can import ssh keys.
<spaniard> ah user ubuntu ..yes i can ssh to the node
<catbus> spaniard: I'd kill current juju bootstrap, make sure the bootstrap machine is released on MAAS, then juju bootstrap again with --debug flag to get more info.
<spaniard> catbus: thank you! i'll start again with the --debug, approximately how long should it take to pass this step Attempting to connect <IP>:22/
<catbus> spaniard: However long it takes the machine to finish OS install and reachable via that IP. When you see it shows deployed successfully on maas, juju should be able to contact the node.
<mup> Bug #1723232 opened: [2.3, UI] Move event log dates to the left <MAAS:Triaged> <https://launchpad.net/bugs/1723232>
<mup> Bug #1707999 changed: pod VM fails to PXE boot after receiving multiple DHCP offers, for different IPs, from the dhcp server <cdo-qa> <cdo-qa-blocker> <cdo-release-blocker>
<mup> <foundations-engine> <internal> <MAAS:Invalid by blake-rouse> <MAAS 2.2:Invalid> <ipxe (Ubuntu):New for andreserl> <https://launchpad.net/bugs/1707999>
#maas 2017-10-13
<boolman> how do I add custom cloud-config in the default installation?
<boolman> or do I have to run curtin?
<boolman> curtin commands*
<mup> Bug #1723425 opened: [2.3, HWTv2] Hardware tests do not provide start, current running and estimated run time <MAAS:Triaged> <https://launchpad.net/bugs/1723425>
#maas 2019-10-07
<mup> Bug #1847011 opened: Return osystems and distro_series requiring keys from the general.osinfo websocket API <MAAS:New for blake-rouse> <https://launchpad.net/bugs/1847011>
<mup> Bug #1846897 changed: Feature request: Support signed SSH public keys/SSH certificates <MAAS:Invalid> <https://launchpad.net/bugs/1846897>
<mup> Bug #1846897 opened: Feature request: Support signed SSH public keys/SSH certificates <MAAS:Invalid> <https://launchpad.net/bugs/1846897>
<mup> Bug #1846897 changed: Feature request: Support signed SSH public keys/SSH certificates <MAAS:Invalid> <https://launchpad.net/bugs/1846897>
<mup> Bug #1847046 opened: MAAS doesn't detect the right PXE interface <MAAS:Triaged by bjornt> <MAAS 2.6:Triaged by bjornt> <https://launchpad.net/bugs/1847046>
<mup> Bug #1847046 changed: MAAS doesn't detect the right PXE interface <MAAS:Triaged by bjornt> <MAAS 2.6:Triaged by bjornt> <https://launchpad.net/bugs/1847046>
<mup> Bug #1847046 opened: MAAS doesn't detect the right PXE interface <MAAS:Triaged by bjornt> <MAAS 2.6:Triaged by bjornt> <https://launchpad.net/bugs/1847046>
<mup> Bug #1847058 opened: MAAS deployment failed with vgchange command <MAAS:New> <curtin (Ubuntu):New> <https://launchpad.net/bugs/1847058>
<mup> Bug #1847058 changed: MAAS deployment failed with vgchange command <MAAS:New> <curtin (Ubuntu):New> <https://launchpad.net/bugs/1847058>
<mup> Bug #1847058 opened: MAAS deployment failed with vgchange command <MAAS:New> <curtin (Ubuntu):New> <https://launchpad.net/bugs/1847058>
<mup> Bug #1847058 changed: MAAS deployment failed with vgchange command <MAAS:New> <curtin (Ubuntu):New> <https://launchpad.net/bugs/1847058>
<mup> Bug #1847058 opened: MAAS deployment failed with vgchange command <MAAS:New> <curtin (Ubuntu):New> <https://launchpad.net/bugs/1847058>
<mup> Bug #1847086 opened: Authentication failure for users with username >32 char length <MAAS:New> <https://launchpad.net/bugs/1847086>
<mup> Bug #1847058 changed: MAAS deployment failed with vgchange command <MAAS:Invalid> <curtin (Ubuntu):New> <https://launchpad.net/bugs/1847058>
<mup> Bug #1794080 changed: [Feature, UI] allow passing cloud-init user-data on deployment and power-on via UI <cpe-onsite> <feature> <track> <ui> <ux> <MAAS:Invalid> <https://launchpad.net/bugs/1794080>
<mup> Bug #1632642 changed: Device discovery click on row functionality <ui> <MAAS:Invalid> <https://launchpad.net/bugs/1632642>
<mup> Bug #1632642 opened: Device discovery click on row functionality <ui> <MAAS:Invalid> <https://launchpad.net/bugs/1632642>
<mup> Bug #1632642 changed: Device discovery click on row functionality <ui> <MAAS:Invalid> <https://launchpad.net/bugs/1632642>
#maas 2019-10-08
<mup> Bug #1847195 opened: web interface locks up in connecting state <maas (Ubuntu):New> <https://launchpad.net/bugs/1847195>
<mup> Bug #1847244 opened: Authentication failure when using username with special characters and Candid+RBAC services <field-high> <MAAS:New> <https://launchpad.net/bugs/1847244>
<mup> Bug #1847244 changed: Authentication failure when using username with special characters and Candid+RBAC services <field-high> <MAAS:New> <https://launchpad.net/bugs/1847244>
<mup> Bug #1847244 opened: Authentication failure when using username with special characters and Candid+RBAC services <field-high> <MAAS:New> <https://launchpad.net/bugs/1847244>
<mup> Bug #1833618 changed: MAAS can't deploy Ubuntu if ID_SERIAL of any block device is broken (USB pendrive in this case). <curtin:Invalid> <MAAS:Invalid> <sg3-utils (Ubuntu):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Bionic):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Disco):In Progress by
<mup> rafaeldtinoco> <sg3-utils (Ubuntu Eoan):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Ff-series):Confirmed for rafaeldtinoco> <https://launchpad.net/bugs/1833618>
<mup> Bug #1833618 opened: MAAS can't deploy Ubuntu if ID_SERIAL of any block device is broken (USB pendrive in this case). <curtin:Invalid> <MAAS:Invalid> <sg3-utils (Ubuntu):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Bionic):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Disco):In Progress by
<mup> rafaeldtinoco> <sg3-utils (Ubuntu Eoan):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Ff-series):Confirmed for rafaeldtinoco> <https://launchpad.net/bugs/1833618>
<mup> Bug #1833618 changed: MAAS can't deploy Ubuntu if ID_SERIAL of any block device is broken (USB pendrive in this case). <curtin:Invalid> <MAAS:Invalid> <sg3-utils (Ubuntu):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Bionic):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Disco):In Progress by
<mup> rafaeldtinoco> <sg3-utils (Ubuntu Eoan):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Ff-series):Confirmed for rafaeldtinoco> <https://launchpad.net/bugs/1833618>
<mup> Bug #1833618 opened: MAAS can't deploy Ubuntu if ID_SERIAL of any block device is broken (USB pendrive in this case). <curtin:Invalid> <MAAS:Invalid> <sg3-utils (Ubuntu):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Bionic):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Disco):In Progress by
<mup> rafaeldtinoco> <sg3-utils (Ubuntu Eoan):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Ff-series):Confirmed for rafaeldtinoco> <https://launchpad.net/bugs/1833618>
<mup> Bug #1833618 changed: MAAS can't deploy Ubuntu if ID_SERIAL of any block device is broken (USB pendrive in this case). <curtin:Invalid> <MAAS:Invalid> <sg3-utils (Ubuntu):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Bionic):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Disco):In Progress by
<mup> rafaeldtinoco> <sg3-utils (Ubuntu Eoan):In Progress by rafaeldtinoco> <sg3-utils (Ubuntu Ff-series):Confirmed for rafaeldtinoco> <https://launchpad.net/bugs/1833618>
<mup> Bug #1782363 changed: [2.5, UI] Upgrade from 2.4 pod refresh message needs improvement <ui> <ux> <MAAS:Invalid by newell-jensen> <https://launchpad.net/bugs/1782363>
<mup> Bug #1782363 opened: [2.5, UI] Upgrade from 2.4 pod refresh message needs improvement <ui> <ux> <MAAS:Invalid by newell-jensen> <https://launchpad.net/bugs/1782363>
<mup> Bug #1782363 changed: [2.5, UI] Upgrade from 2.4 pod refresh message needs improvement <ui> <ux> <MAAS:Invalid by newell-jensen> <https://launchpad.net/bugs/1782363>
<mup> Bug #1782363 opened: [2.5, UI] Upgrade from 2.4 pod refresh message needs improvement <ui> <ux> <MAAS:Invalid by newell-jensen> <https://launchpad.net/bugs/1782363>
<mup> Bug #1782363 changed: [2.5, UI] Upgrade from 2.4 pod refresh message needs improvement <ui> <ux> <MAAS:Invalid by newell-jensen> <https://launchpad.net/bugs/1782363>
<mup> Bug #1847337 opened: MaaS 2.6.1 - Failed to retrieve curtin config: 'NoneType' object has no attribute 'url' <MAAS:New> <https://launchpad.net/bugs/1847337>
#maas 2019-10-09
<tosaraja> how do i debug commissioning? I have a host that does PXE boot, but the commissioning fails constantly. I can log in to the host and see that the network connection works and all..it's all fine. just doesn't commission
<tosaraja> as i removed the host from maas, the pxe boot also gave it a new host name. so that part also works
<mup> Bug #1847464 opened: Missing disable/delete options for default repositories. <MAAS:New> <https://launchpad.net/bugs/1847464>
