/srv/irclogs.ubuntu.com/2012/10/02/#maas.txt

=== matsubara-afk is now known as matsubara
roaksoaxbigjools: waking up early lately :)00:44
bigjoolsroaksoax: it's actually late for me - bad night of twins00:46
* bigjools otp00:46
roaksoaxaw :(00:46
bigjoolsroaksoax: where are you guys with the ipmi config?00:48
roaksoaxbigjools autodetect? needs integration will be done by EOW00:50
roaksoaxbigjools is trunk broken? i cannot make run00:50
* bigjools has not tried make run for a while00:50
roaksoaxlol00:52
bigjoolspackaging baby, packaging00:53
* roaksoax bbl00:53
roaksoaxbigjools: around?02:36
bigjoolsin body, yes02:36
roaksoaxbigjools: you prefer to leave the discussion about the packaging for tomorrow ?02:36
bigjoolsroaksoax: yeah if you don;t mind - I have a million other packaging bugs to fix02:37
bigjoolsI might be more compos mentis tomorrow02:37
roaksoaxbigjools: cooil, anything ai can help with?02:38
bigjoolsroaksoax: always! :_02:38
bigjools:)02:38
bigjoolsroaksoax: bug 1059556, bug 105945902:38
ubot5Launchpad bug 1059556 in MAAS "/etc/init/maas-celery.conf not removed on upgrade" [Critical,Triaged] https://launchpad.net/bugs/105955602:38
ubot5Launchpad bug 1059459 in MAAS "Existing DHCP server not stopped" [Critical,Triaged] https://launchpad.net/bugs/105945902:38
bigjoolsbug 105941602:39
ubot5Launchpad bug 1059416 in MAAS "When upgrading from the maas package, the cluster controller package doesn't detect MAAS_URL but it could" [Critical,Triaged] https://launchpad.net/bugs/105941602:39
bigjoolsbug 105956902:39
ubot5Launchpad bug 1059569 in MAAS "Impossible to start cluster controller as maas user" [Critical,Triaged] https://launchpad.net/bugs/105956902:39
bigjoolstake your pick02:39
bigjoolsI need to fix all these today02:39
bigjoolsoh and bug 105945302:39
ubot5Launchpad bug 1059453 in maas (Ubuntu) "The celery cluster worker is not properly stopped" [Critical,Triaged] https://launchpad.net/bugs/105945302:39
roaksoaxbigjools: I think this would solve the first one: http://paste.ubuntu.com/1255220/02:41
bigjoolsyeah we talked about that, thanks, I'll stick it in02:41
roaksoaxbigjools: i'll test it02:42
roaksoaxbigjools: i'll grab the latest from PPA and start pushing my fixes there, and if they get fixed I will propose abranch02:42
bigjoolscool02:43
roaksoaxi wanted to fix these stuff this morning but got stuck with something else02:44
bigjoolsbrb02:45
roaksoaxbigjools: bug #1059569 is weird, becuase installing the sudoers file should have solved the issue. I think i actually found myself against that error to before02:48
ubot5Launchpad bug 1059569 in MAAS "Impossible to start cluster controller as maas user" [Critical,Triaged] https://launchpad.net/bugs/105956902:48
roaksoaxgonna check the logs02:48
bigjoolsroaksoax: it's not a sudoers problem02:49
roaksoaxoh i see, what does the maas-provision command require as root?02:49
bigjoolsroaksoax: we either need to not enforce that maas-provision runs as root, or make it start the celeryd as a different user.  the latter is problematic as we'd have to put more packaging smarts through02:50
bigjoolsI don't know why it is required to be root02:50
roaksoaxbigjools: i'm pretty sure the old maas-celery didn't have that problem02:51
roaksoaxand was running as non-root02:52
roaksoaxso should be the same thing02:52
bigjoolsroaksoax: no - the process is a wrapper around celery now02:52
roaksoaxbigjools: right, so that shouldn'y have changed anything02:52
roaksoaxbigjools: so maybe you guys are storing something on a location which is non maas owned?02:52
roaksoaxbigjools: are logfiles being pre-created with correct permissions?02:52
roaksoaxmaybe is that02:53
bigjoolsroaksoax: you don't understand, let me explain again02:53
bigjoolsmaas-provision now starts the cluster celeryd.  maas-provision needs to run as root, but celery needs to run as maas02:53
bigjoolsthe upstart conf starts the maas-provision wrapper, not celeryd02:54
bigjoolsif conf is set to run it as maas user, maas-provision bails out as it checks to see if it's root02:54
bigjoolsI don;t know why that check is there02:54
roaksoaxbigjools: right, I see the point now02:55
roaksoaxbigjools: so that's design falw then02:55
roaksoaxbigjools: maas-provisiond should be starting the daemon02:55
roaksoaxor something02:55
roaksoaxbigjools: the problme is that we are using a wrapper both a config/admin tool to do certain stuff (such as install pxe images), and you are also using to start a daemon02:56
roaksoaxand that should not be done that way02:56
roaksoaxIMHO02:56
bigjoolsI agree02:56
roaksoaxa dameon is different from a command line tool02:56
bigjoolsit's not a daemon02:57
roaksoaxbigjools: hence, maas-provision wrapper makes sure we run it as root to do the necessary operations02:57
roaksoaxbigjools: maybe a good idea would be to have another wrapper that starts the daemon02:57
roaksoaxwithout the check02:57
roaksoaxbut either way, celery should be started differently IMHO02:58
bigjoolsI might just remove the check. it's pointless02:58
roaksoaxbigjools: it is not02:58
roaksoaxbigjools: it displays nasty errors when run as user02:58
bigjoolsif it needs root, it'll fail anyway02:58
roaksoaxbigjools: it needs root to be able to write directories02:58
roaksoaxbigjools: if you do maas-provision -install-pxe-image as normal user it will display a nasty error that should not be displayed02:59
roaksoaxthen, the root check should be done within provisioningserver code02:59
bigjoolsthe check should be done elsewhere then02:59
roaksoaxthe wrapper simply checks that the user has appropriate permissions to execute those operations02:59
bigjoolsin fact the code should catch the nasty errors03:00
bigjoolsand display a nice one03:00
bigjoolswrapping the whole script in a root check is wrong IMO03:00
roaksoaxk, etiher way i don't agree with having a command line tool starting a daemon03:00
roaksoaxbigjools: IMO, it is not, because the wrapper runs code that is (or was) meant to be run as root only03:01
roaksoaxso it should make the check03:01
roaksoaxbut yes, the check should go in the python code now03:01
roaksoaxthat a command line tool starts a daemon03:02
roaksoaxthe daemon should be independent from a command line tool03:02
bigjools"meant to be run as root only" - I disagree :)03:02
roaksoaxbigjools: the wrapper belongs to /usr/sbin03:03
roaksoaxwhy? because it messes up with the system03:03
roaksoaxit does not only mess with the environment of a user03:03
roaksoaxfor that reason, is mean to be run as a privileged user03:03
bigjoolsmaas-provision is a command utility. we never intended it to be turned into a root only tool03:05
roaksoaxbigjools: right, but look, maas-provision is a command line utility that interacts with MAAS operations that require privileged user03:05
bigjoolsyes03:05
roaksoaxhence, it is a tool that belongs in usr/sbin03:05
roaksoaxwhich is meant to be run as root03:05
bigjoolsbut not exclusively as rot03:06
bigjoolsroot03:06
roaksoaxbigjools: ok, let e rephrase, privileged user (this means users under sudo).03:07
roaksoaxbigjools: the check done, is for root user, and sudo users03:07
roaksoaxbigjools: for example, apachectl can only be run as priviliged user03:08
roaksoaxmaas-provision is exactly the same thing03:08
roaksoaxthe check it has, is to check it is a privileged user03:09
bigjoolsbasically we have friction between upstream's design and requirements, and Ubuntu's policies03:09
roaksoaxbigjools: I dont agree :). These are not Ubuntu policies. These are Operating System policies03:10
roaksoaxdaemons and interaction with daemons are meant to be done by privileged user03:10
bigjoolsthe fact that you want it root-only and in sbin is not ubuntu policy?03:11
roaksoaxbigjools: maybe adding parameter for user/group to run the celery dameon are needed for maas-provision03:11
roaksoaxbigjools: thta's OS principles, not only ubuntu's03:11
roaksoaxfedora, redhat, debian, etc etc etc03:11
bigjoolsthat was one of my potential solutions yeah03:11
bigjoolswell look, we'll do a separate command03:12
roaksoaxok, longtemr thoguh, it should support the arguments for user/group I think03:15
roaksoaxbecause it is better to tell the daemon under what user to run03:15
roaksoaxthat way03:15
bigjoolsthat's upstart's job isn't it? :)03:16
roaksoaxbigjools: not completely no03:16
roaksoaxbigjools: twistd allows passing the user/group that you would like the daemon to run as03:17
roaksoaxcelery doesn't03:17
roaksoaxbut since celery is now run by maas-provision, then maas-provision should probbaly allow passing user/group to run the daemon as03:17
bigjoolsyeah03:19
bigjoolsperhaps that's a better solution here03:19
roaksoaxindeed03:19
bigjoolseasier, I mean.  jtv ^ ?03:19
roaksoaxbigjools: and as you mentioned, the other operations should check they are run as root, and display an error accordingly03:19
roaksoaxbigjools: so that we ditch that check in the wrapper03:19
jtvI don't quite understand: if the command to start a cluster will run celery as another user than it itself runs under, that makes it a privileged command.  What's the argument for making it a command separate from maas-provision?03:22
roaksoaxjtv: right, it makes it a privilleged command, that the upstart job will run as root, and will tell the daemon "run as the user"03:23
bigjoolsjtv: maas-provision is root-only according to roaksoax, so either it runs celery as a different user or another command run by maas has to start it03:23
roaksoaxjtv: so for exmaple, the upstart job for txlongpoll is invoked as root, but we are telling it that the daemon (twistd) should run as the 'maas' user group03:23
roaksoaxbigjools: not just according to me. They way I see it, it is a design principle. If a utility messes up with the system, it is a privileged utility03:24
bigjoolsroaksoax: ok you and others :)03:25
roaksoaxbigjools: hehe isn't it fun to be in between different worlds ? :)03:25
bigjoolsjtv, if it's easy to become a different user in Python, taking a -uid and -gid cmd line option seems ok to me03:25
jtvBut for maas-provision, or for a separate command?03:27
bigjoolsjtv: for maas-provision's start_cluster_controller03:28
roaksoaxjtv: the "separate command" was simply a wrapper to start_cluster_controller03:28
jtvOK, so keep start_cluster_controller but make it startable as another user.  And obviously that means it'll have to fork() as well.03:29
bigjoolsjtv: indeed03:29
bigjoolsjtv: but fork() is good because it means I can start tracking it properly in upstart03:29
bigjoolssince at the moment it fails to do so03:29
jtvWell I say "obviously" -- I don't actually know there's no other way if you're root.  :)03:30
jtvYeah, I just hate to build on two variables.03:30
jtvIt'd be bloody annoying to find that the solution for changing the user makes the upstart problem impossible to solve!03:30
bigjoolsjtv: fork(), change user, exec().  Sorted.03:31
bigjoolsroaksoax: are you landing that Conflicts: change or shall I?03:32
roaksoaxbigjools: i'm about to test it03:32
bigjoolsroaksoax: excellent, thanks03:32
bigjoolsroaksoax: what is the best way of stopping/disabling the existing dhcp server when installing maas-dhcp?03:39
jtvAdded note: have to do it synchronously, I think, or it may keep ours off its port.03:40
bigjoolsyeah03:40
bigjoolseasy03:40
bigjoolsroaksoax: I presume invoke-rc followed by update-rc03:41
roaksoaxbigjools: yes, if it is using sysvinit script03:41
bigjoolsit is03:42
bigjoolsiirc03:42
roaksoaxbigjools: this should probbaly done on preinst03:42
bigjoolsroaksoax: oh actually it's an upstart conf03:44
bigjoolshow do you disable those?03:44
roaksoaxbigjools: probably by an override, let me check03:46
roaksoaxbigjools: http://upstart.ubuntu.com/cookbook/#disabling-a-job-from-automatically-starting03:47
bigjoolsah thanks03:47
bigjoolsroaksoax: should that be a package-installed file?03:48
roaksoaxbigjools: not necessary as long as it gets removed on postrm03:49
bigjoolsok03:49
bigjoolsso preinst and postrim03:49
* bigjools hacs03:49
* bigjools eats first in fact03:50
jtvroaksoax: do we still need a pid file, with upstart watching our celery process?04:19
roaksoaxjtv: i think we do04:20
roaksoaxjtv: for twistd04:21
roaksoaxbut not for celery04:21
* roaksoax checks04:21
jtvOh, I meant specifically about celery.04:21
roaksoaxno we dont04:21
roaksoaxold celery was exec /usr/bin/celeryd --logfile=/var/log/maas/celery.log --loglevel=INFO --beat --schedule=/var/lib/maas/celerybeat-schedule04:21
roaksoax so no pid04:21
jtvYeah, I'm asking whether that's right though.  Because we had trouble tracking it with upstart.04:22
roaksoaxjtv: you can specify it if you like04:23
jtvI don't like, I'm just not sure whether it's needed!04:23
roaksoaxjtv: i don't think it is mandatory04:24
roaksoaxwe have been without a pidfile04:24
roaksoaxso i don't think we need it04:24
jtvBut it hasn't been working properly, which is why I'm asking.04:24
roaksoaxjtv: idk TBH. :(04:25
roaksoaxbigjools: uhm conflicts/replaces didn't seem to work04:25
jtvBecause it's going to get more complicated to maintain a pidfile with the setuid.04:25
bigjoolsroaksoax: damn :(04:25
roaksoaxbigjools: maybe just conflicts! i'll give it a try04:26
bigjoolsjtv: it's not working because the wrapper execs in a way that doesn't use fork but somehow changes its pid04:26
jtvSo we know for sure it's not just lack of pidfile?  Good.04:26
roaksoaxyes I04:27
roaksoaxi'd agree with bigjools04:28
roaksoaxs/i'd/i04:28
bigjoolsjtv: once you use fork we can tell upstart about that04:29
bigjoolsand it'll DTRT04:29
jtvYes, that's the hypothesis.  I just don't know enough about what's going on to be confident, which is why I ask around.04:30
jtvbigjools: well, it turns out one of my hypotheses yesterday was right.  We do fork something else before we fork off celery.04:59
jtvIt's ifconfig.04:59
=== matsubara is now known as matsubara-afk
bigjoolsjtv: gnargh!05:06
jtvYup.05:06
jtvSo this might be a good time to consider netifaces.05:06
bigjoolsyes05:07
bigjoolsJFDI05:07
bigjoolsand it's in main, unbelievable05:07
jtvI was wondering why changing fork-exec to fork-setuid-exec would fix anything :)05:07
jtvIn good news, it was the tests that brought this to light.05:08
jtvWhat needs doing to add the netifaces dependency?05:08
bigjools2 places: required-dependencies and packaging dependencies05:11
bigjoolsI'm doing loads of packaging changes, I'll do it there for you05:11
bigjoolsjust in cluster controller right?05:12
jamjam05:19
jammorning all05:19
bigjoolshey jam05:24
jtvHi jam05:31
jtvbigjools: yes, just cluster controller.05:31
jtvIt's lunchtime.  I'm going to try to get some equipment.  Almost bought a nice little lightweight machine, but found that unity probably won't support its AMD graphics chip.  :(05:32
jtvOh crap here comes the rain.  Perfect timing as always.05:32
bigjoolsheh05:34
jambigjools: question for you about using celery05:52
bigjoolsyup05:53
jamfor the Tag table, we know that when we create a tag, it can take 10s for 10,000 nodes to get checked.05:53
jamSo we want to push that out into an async job05:53
jam(cron or rabbit comes to mind)05:53
jambigjools: how do we integrate that with the system?05:53
bigjoolsjam: pretty easy, I'll take you through the steps:05:53
bigjools1. Edit src/provisioningserver/tasks.py and define the async job05:54
jamk05:54
jambigjools: just as a function with @task on it?05:54
bigjoolsit's a function decorated with @task05:54
jamk05:54
bigjools2. keep the function small - just call out to something in provisioning server so that the tasks file stays small05:55
jam(and jinx, I guess :)05:55
jamsure, I expect the logic to stay on the Tag object.05:55
bigjools3. from the appserver code, call it with funcname.apply_async(queue=nodegroup.uuid)05:55
bigjoolsand it'll run on that nodegroup's worker05:55
jambigjools: so we have to loop over N nodegroups?05:56
bigjoolsjam: if your nodes are across multiple groups, yes05:56
jam(and the nodegroups talk directly to the DB, or we need another bit to send a message back)05:56
bigjoolsto talk back you need an API method defined that the nodegroup can call05:56
jamk05:57
bigjoolss/nodegroup/celeryd/05:57
bigjoolsthe celeryd caches credentials for the API05:57
bigjoolssee report_boot_images() for an example05:57
jambigjools: so what data do the individual nodegroups have access to? (they don't have a local DB do they?)05:58
bigjoolsthey don't05:58
bigjoolsthey can only see what is passed in the task call05:58
jamdo they have access to the main? or we need 2 apis, one to scrap the data out of the db, and another to put it back in?05:58
bigjoolswhat are you doing, exactly?05:58
jambigjools: we need to take the hardware details, filter it by the xpath statement, and determine a list of Nodes that match particular Tags05:59
bigjoolsah ok then let's do this in another way05:59
jamthe long-term idea is that the hardware details would sit on the cluster controler05:59
bigjoolswe have a worker local to the region for this stuff05:59
jamhowever, they don't have anywhere to put it *today805:59
jambigjools: sounds like the worker we'd like to talk to for the initial implementation, then.06:00
jambigjools: do you set that by the 'queue=...' stuff?06:00
bigjoolsjam: yes06:00
bigjoolsso if you look in etc/celeryconfig_common.py06:00
bigjoolsyou'll see WORKER_QUEUE_DNS = 'celery' and WORKER_QUEUE_BOOT_IMAGES = 'celery'06:01
bigjoolsAdd another one, WORKER_QUEUE_<something>06:01
bigjoolsset to "celery", which is the name of the region's worker06:01
bigjoolsthen when you define the task you can decorate it like this:06:01
bigjools@task(queue=celery_config.WORKER_QUEUE_<thing>)06:02
bigjoolswhich ensures it always gets sent to the same worker06:02
jamso I'm guessing we should have a similar config as WORKER_QUEUE_TAG_??? and then also point it at 'celery'06:02
jamright06:02
bigjoolsand you change the apply_async() to a delay()06:02
bigjoolsso function.delay()06:02
bigjoolsand pass any args in the delay params06:02
jambigjools: ah, ok06:03
bigjoolsit sounds like you'll need some way of querying the API for the data you need06:03
bigjoolsand to store it back06:03
jambigjools: so we still need api calls, but it is just run locally.06:03
jamNP06:03
bigjoolsright06:03
bigjoolsmake sure the api calls run *quick* then the job can take a bit longer06:04
jambigjools: is there plans in say 13.04 to add some sort of local db/storage on the cluster controllers?06:04
jamor is the goal for them to be fully stateless?06:04
bigjoolspossibly.  Mark wants to do that.06:04
bigjoolswe might thrash that out a bit more in COP06:04
jambigjools: so how fast is quick? <1s, <100ms, <10ms?06:05
bigjools<1s is probably ok06:05
jam(shoot for 100ms assuming that sometimes load will cause it to be 1s?)06:05
bigjoolswhat I mean is - optimise the DB queries :)06:05
bigjoolsotherwise you're negating the effects of offloading jobs from the appserver06:06
jambigjools: so we have 2 options, we can compute the XPATH content in the DB (postgres has native support for it), or we can compute it in the local process using LXML. It is slightly faster to do it in the DB, because we don't have to read out the XML content, but obviously it adds more load in the DB06:07
jameither way we can probably do it in batches06:07
jamso we update, say 1000 nodes at a time.06:07
bigjoolsjam: yeah batching is important here for 1000s of nodes06:07
bigjoolsnot sure if we have a batching mechanism in Piston06:07
bigjoolsjam: if the xpath runs quick enough on the DB, there's no problem with that06:08
jambigjools: 'quick enough' is 6s for 10,000 nodes.06:08
jamso batching at 1,000 nodes would be 600ms, as a ballpark sort of thing.06:08
bigjoolsmmmm might be pushing it06:08
bigjoolsyeah06:08
jambut you still spend 6 total seconds doing the work06:08
bigjoolsyou'll probably have to manually batch06:08
jamor in the 100k node space, you are talking 60s total CPU time.06:09
bigjoolsthis is fine to dump on the celery worker06:09
bigjoolssince most of the time it is not doing much ATM06:09
bigjoolsthe DB is potentially more precious resources-wise06:09
jambigjools: and the long term goal is to dump down to the individual regions06:09
bigjoolsright06:10
jamI'm wondering it should be architectured as: 1) grab the list of nodes that need touching right away, 2) farm out to each nodegroup for their respective nodes06:10
jam3) pass just the node ids06:10
jam4) the provisioning_servers then batch requests for XML content, and parse it.06:10
jam5) and poke the results back into the DB as they go.06:10
jamThe main trick with all this, is knowing when everyone is "done"06:11
jambut I think that gets us a good CPU story06:11
jambecause the processing is properly farmed out across the 'scaling' portion of the system.06:11
jamWe have a small bandwidth issue tod06:11
bigjoolsyou can do it like that if you like - I'm just saying that the region celeryd is fairly idle06:11
jamtoday, because the data is in the central DB06:11
jambut it is where we want it to be done in 13.04 or whatever.06:12
bigjoolsbut scaling out is a good plan for the future06:12
jambigjools: from what I can tell, lshw -xml is about 24kB * 10,000 nodes is 240MB being downloaded in these requests.06:13
jamIs that too much load?06:13
jamon the DB06:13
bigjoolssounds ok06:13
jam(note that they probably compress fantastically well, if that is possible in the API)06:13
jamthough adding that may be premature optimization at this point.06:13
jtvThe rain's letting up.  I'm going to make my shopping run.06:30
jtvbigjools: any chance of a review?  https://code.launchpad.net/~jtv/maas/netifaces/+merge/12742006:33
jtvWhile I'm out?06:33
bigjoolsone sec06:33
bigjoolsyes, OTP06:33
jtvWow, that's fast!06:33
jtv:P06:33
Fajkowskyhello guys, I have problem with my maas server I described it here - http://www.tinyurl.pl?QSsXP6oX - its "no instance data found in start local"08:17
bigjoolsFajkowsky: your link is invalid08:29
Fajkowskyhttp://askubuntu.com/questions/195115/nodes-cant-connect-to-server-after-bootstrap08:30
FajkowskyI try install again maas and add nodes, maybe it works this time.09:19
bigjoolsFajkowsky: I am just adding an answer09:20
Fajkowskyok09:21
allenaprvba: Sorry, I just switched your branch back from Approved.10:03
rvbaallenap: a) I don't agree with what you say in your comment.  b) we've got many other place where it's done this way.  If we've going to change that behavior, then we better do it everywhere.10:06
rvbaplaces*10:06
allenaprvba: Well, start here then. That we've done it wrong elsewhere doesn't make it right.10:08
rvbaallenap: indeed.  But I'm really not sure it's right :)10:08
rvbaallenap: ok you might be right about that idempotent stuff.  But, if you don't mind me saying that, your method is wrong here.  You should let me land that branch, file a bug about the problem, an *then* someone will pick up that branch and change all the delete methods.10:14
rvbaallenap: because that bug is marked critical and changing the behavior of all the 'delete' methods is not.10:15
rvbaIt's 'high' at best.10:15
allenaprvba: Okay, fair enough; I just wanted to save you an extra branch and proposal for what seems like a simple change.10:21
rvbaallenap: I just wanted to land a quick fix to unblock Diogo.  And also, who's tell you that I'm going to be the one doing that extra branch ;).  No, seriously, I've got to focus on my UI stuff right now.10:22
rvbas/tell/telling/10:23
allenaprvba: I wasn't suggesting you fix everything. I was just commenting on this one proposal. I changed it back to Needs Review to stop Tarmac from landing it, so that we could talk about amending it, saving the effort of filing a bug, proposing a merge, making a card, etc. I didn't realise it was a bigger problem. I'm sorry that I caused such distress! :)10:25
rvbaallenap: no distress, really, but if your goal was to save time for the both of us, then that's a fail :). We definitely will have to file a bug for the other delete() methods so fixing up this one and have it half-done is not the way to go I think.  "Ni fait ni Ă  faire", here is another nice french expression :).10:29
jtvAh, is the "one more small change" worm rearing its ugly head?10:32
jtvMeanwhile, I wonder why one of our postinst scripts uses '[a-z]\{0,\}' instead of '[a-z]*'10:33
jtvActually, it's pretty sick to "sed" for an entry in a multi-line dict.  What if some other dict contains the same key?  I think I'd rather define a variable, and have the dict refer to that.  The dict will not need any patching, there'll be no leading whitespace, etc.10:36
jtvallenap: maybe you can help me out with this question.  What is the relationship between the patch we have in packaging that sets the db password to 'maas', and the postinst code that sets a proper password?  Why do we have both?10:49
allenapjtv: Eugh, I don't know. Intriguing.10:52
jtvOr wtf dbc_dbpass comes from... it seems to be coming from thin air.10:52
jtvBit annoying when you want to verify that it really consists only of ASCII letters and digits.10:52
jtv(I don't see why the regex needs to check for exact contents of the string: '[^']*' is both easier and afaict, more appropriate)10:53
jtvReview needed!  Spot the stupid mistake that I keep repeating... https://code.launchpad.net/~jtv/maas/pkg-bug-1060095/+merge/12745110:55
allenapjtv: I'll do it; I haven't a clue about dbc_wibble so I'll keep my head down reviewing.10:56
jtvThat's good too.  :)  Thanks.10:56
rvbajtv: Andres will probably know the answers to these packaging questions :).10:57
jtvYeah.  Wonder if he's here yet... I need to leave soon.10:57
jtvroaksoax, are you here?10:57
allenapjtv: It's definitely maas_local_settings not maas_local_settings.py, right?10:57
jtvallenap: what is?  I think the module is maas_local_settings and the file is maas_local_settings.py.10:58
jtvThe latter is what we mostly refer to.10:58
allenapjtv: It's chmod'ing /etc/maas/maas_local_settings <-- no .py10:58
jtvOoo!10:58
jtvFixed.11:00
jtvGood thing you spotted that.11:02
allenapjtv, rvba, anyone: I have to stop work early today to collect Robin from school, at 1340 UTC, but I'll be back around this evening, after 1900 UTC.11:10
jtvI'll be away for the rest of the night as well.11:10
jtvI'll email roaksoax with my questions.11:10
mgzallenap: as worded, I don't think bug 1060114 is true, or in need of fixing.11:21
ubot5Launchpad bug 1060114 in MAAS "DELETE operations are not idempotent" [Medium,Triaged] https://launchpad.net/bugs/106011411:21
allenapmgz: Fancy rewording it? ;)11:24
allenapmgz: Ah, I've just seen your comment on the proposal. Interesting point.11:25
allenapmgz: I like your explanation. Can you add it to the bug and mark it Invalid?11:26
mgzallenap: sure11:27
allenapmgz: You don't happen to be in town today?11:28
allenapI'm trying to find an excuse to go out for lunch.11:28
mgzallenap: alas :)11:28
mgzyou're making lunch on thursday though, right?11:32
mgzwell, as it, getting there, kat will be making it :D11:32
mgz*as in11:32
mgzbah, I can't type for toffee11:33
allenapmgz: Yeah, I'll be there, probably at about 1200, because Chantal and I are collecting a puppy before then.11:35
mgzbring the puppy to lunch! :D11:36
mgz...how house trained is it going to be?11:38
allenapmgz: Not very, and it has to stay at home until it's fully inoculated so I'm told, so a few weeks.11:40
mgz;_;11:40
allenapmgz: We can't even take it out for walks at first. It gets to shit in the back garden, that's it :)11:40
jtvOkay folks, I'm off for tonight.11:41
allenapCheerio jtv.11:41
jtvnn11:41
jtvWhen Raphers comes up to breathe, tell him I wish him good luck and God speed with his UI branches.  :)11:41
jamjelmer: how's it going? Want to skype some more?11:48
jammgz: how are things looking for you?11:48
jamallenap: I have some questions about how the celery stuff works. are you knowledgable or should I chat with bigjools tomorrow?11:49
allenapjam: rvba is the man on that front, but I might be able to help.11:49
jelmerwb jam11:49
jelmerjam: making some progress, trying to get some tests going11:50
jamthe big question is that the workers need someone to call 'record_secrets' before they can do any work11:53
jambut the examples we have seem to just drop the request on the floor if they don't haev the secrets yet11:53
jambut how do we make sure that the work is always done11:53
jamdo we need to put the work in the DB as 'todo'11:53
jamand then have it marked 'done' by the callback?11:53
jamrvba, allenap^^11:53
mgzjam: landed some tag sample stuff11:56
=== matsubara-afk is now known as matsubara
jamand if the work is still pending who retries it?11:58
rbasaksmoser: ping, for when you get in12:03
rvbajam: (was out having lunch) Celery can handle that for you I think, you just need to get the task retried instead of dropping it on the floor if the secrets are not there.12:04
jamrvba: how does one signal that?12:05
rvbajam: see rndc_command in src/provisioningserver/tasks.py.12:05
jam(and not have it go into a death spiral trying to retry the job)12:05
rvbahttp://docs.celeryproject.org/en/latest/userguide/tasks.html#retrying12:05
jamrvba: ah the function itself calls func.retry12:07
rvbaYeah.12:07
mgzokay, now I feel a lot less clever about not using real xpaths...12:15
jammgz: we have code that asserts they are valid12:16
jamlike when allocating a new node12:16
jammgz: quick poll for you12:17
mgzI'll fake that up here12:17
jamwe know that after updating a tag12:17
jamthere will be some time where the node <=> tag mapping is inconsistent.12:17
jamIs it better to drop all mapping, and then add them slowly?12:18
jamor is it better to slowly update the nodes to make it consistent?12:18
mgzah, interesting12:18
jamA user fixes a tag definition, and then goes to start nodes based on that tag.12:18
jamis it better to not match anything, and get tried again later12:18
jamor better to match something that it used to match, on the premise that a small update is unlikely to actually change the node set dramatically.12:18
jamI'm tending towards the former, on the premise that 'juju deploy' will keep trying to fulfill your request12:19
jamand then you won't accidentally deploy on machines that no longer match the new tag vaule.12:19
jamvalue.12:19
mgzI feel a common "update the same name" case might be to slightly tweak what gets selected12:19
mgzso, having an interval where the stuff that used to be selected and will still be selected is not, is probably the most suprising12:20
jammgz: right 'has_nvidia' and you realize that sometimes it is NVIDIA and sometimes nvidia12:20
jamso you want to make it case insensitive12:20
jammgz: the flip side is 'big >= 2GB' and you update it to 'big >= 4GB' and you deploy, and it picks a 2GB node.12:21
mgzthat seems less harmful, having an interval where the old rules apply12:21
jammgz: my argument for 'not selected for a while' is that it will eventually be selected and retrying the query will get you the right value.12:21
mgzthis is true, if we stick with only using tags as positives12:22
jammgz: so I think jelmer's preference is to do delta updates12:22
mgzthe other option...12:22
jelmerjam: if it's retrying it might be that the tag is still only half up-to-date when the set of nodes is non-empty12:22
mgzis to set an update time12:22
mgzand if asked to acquire something with an old tag set, defer until the tags are fresh12:22
jamjelmer: it may be only have up-to-date, but everything that is tagged *definitely* matches the new value.12:22
jamso a 'big' node will always have 4GB after changing the definition.12:23
mgzacquire is expected to be slow12:23
jammgz: we do have an updated field already (it is part of the model)12:23
mgzadding 6s (currently) or per-cluster update delay, would not be too bad12:23
jamand we can do other work to detect if all nodegroups have given their responses.12:23
mgzif a cluster doesn't do any acquire till after it's done tag updates, you'd still get some responses fast12:24
jam(essentially some sort of db entry that indicates whether nodegroup X has responded for tag Y)12:24
mgz(slow/big clusters would tend not to be used straight after tag updates, but that's not terrible)12:24
jammgz: in the end, this sounds like something to bring up at the standup, and we can move forward with what we have until then.12:25
jamI have the small feeling that a coin flip may end up involved somewhere.12:25
mgzyup, all options are reasonable, and changing strategy after we have running code is fine.12:25
jamallenap: also, for the 'nodegroup' changes, are all nodes going to have a nodegroup?12:26
jammgz: it does change the api, 'add_nodes' vs 'update_nodes', etc.12:26
allenapjam: They should do already, but let me check.12:26
rvbajam: it is already the case.12:26
jamrvba: I see that it says 'this should be not null, but we can't do that yet'12:26
smoserrbasak, here now.12:27
allenaprvba: Node.nodegroup is null=True, blank=False  -- what does that mean on a foreign key?12:27
allenaprvba: Ah, I've read the comment now ;)12:27
rvbaallenap: :)12:28
smoserhm... did we get a bug for my issue with daily ppa not installable?12:28
mgzbah, pants: TypeError: 'Tag' instance expected, got <Tag: big>12:30
mgzsouth wants to make my life miserable12:30
mgzcan I refactor that bit out...12:30
jammgz: I think you can do add(Tag.id) but I might be wrong.12:31
mgzjam: I'm trying to share code with the migration... when they're really using different model classes12:32
rbasaksmoser: good morning!12:32
mgzpassing it in might be th lease stupid option...12:32
rbasaksmoser: the precise daily ephemeral image seems to work. I didn't add BOOTIF_DEFAULT and it works.12:33
mgz*the least12:33
rbasaksmoser: I had a question though. Looking at making maas-import-ephemerals work for ARM without hackery. It should be a pretty minor change, but I can't remember what we said about adding multiple subarchs in ephemeral images12:33
rvbamgz: I don't think you should try to share code with the migration because the migration runs with the models being in a special state (when not all of the migrations have been run yet) so if you change that code later, it might break the migration.12:34
rbasaksmoser: is the plan to have one image for all of armhf? In which case, how does that work with install_tftp_image?12:34
smoserrbasak, it only works for you because you get a dhcp response on both interfaces.12:34
smosers/both/all/12:34
mgzrvba: this is all basically a trigger12:35
mgzwhen hardware_details field changes, update these other fields12:35
smoserrbasak, for multipel sub-arches' the plan would just to change the format of the tar file so that there were multiple kernels pulled out.12:35
rbasaksmoser: OK. I want to break it before I fix it in order to test the fix12:35
smoserrbasak, fi you want more than "highbank" then ephemeral images and import scripts probably need work.12:36
mgzI don't know how to do the migration correctly if setting that field does not also do the (db contents specific) updates to the rest of the stuff12:36
rbasaksmoser: OK, but does that mean that the maas-provision install-pxe-image interface will need to be changed? Currently it expects an entirely different image directory per subarch, which would be a waste of space I think12:36
mgzand it's complex, writing it in two places, one of which is not tested, does not appeal to me.12:36
rvbamgz: ok, I don't know what particular problem you're facing but it's just that we've been bitten by that once :)12:37
mgzI could just copy the current code though12:37
smoserone could just complain that "subarch" should never have been invented :)12:37
rbasak:-)12:37
rbasakOne day we'll have device tree and it'll all go away12:37
smoseri'd have to look at it, rbasak, but yeah, our goal would be to have one ephemeral iamge12:37
smoserand multiple tftp'd kernels12:37
rbasaksmoser: ok. So I think it's not practical to get multiple subarch support into maas-import-ephemerals right now, so I think I'll add some kind of ugly hack for highbank and leave it at that for 12.10. Is this OK with you?12:39
smoserwhat is it that you're concerned about?12:41
rbasaksmoser: right now it doesn't import highbank at all, since "generic" is hardcoded. I need to have this fixed by 12.10, that's all.12:41
smoserand it doesn't fail?12:42
rbasaksmoser: I've been patching it by hand up to now12:42
rbasaksmoser: and now I'm at the point where I'd like to get it working in trunk and in the package12:42
rbasaksmoser: and have it import all three arches by default12:42
sanderjHow come MaaS have a bug not including udev in the initrd script? So it dosn't work to boot up remote vms?12:53
sanderjscripts/init-bottom/udev should look like this: http://paste.ubuntu.com/679222/12:54
rbasaksanderj: what exactly is the problem?13:03
sanderjrbasak, when booting up a vm from spx with maas.. I get the error: cannot find "bnx2-mips-09-6.2.1a.fw"13:04
rbasakwhat's spx?13:05
sanderjwhen I unpacked the initrd and added two lines to the abow script, then it workes.13:05
sanderjpxeboot13:05
sanderjI mean13:05
sanderjSorry13:06
rbasakwhich lines did you add?13:06
rbasakany idea what bnx2-mips is?13:06
sanderjnetwork card driver13:06
sanderj. /scripts/functions13:07
sanderjwait_for_udev13:07
sanderjThose two lines I added13:07
sanderjinto scripts/init-bottom/udev13:07
rbasakwhich initrd did you modify?13:07
sanderjThe initrd for the kernel maas uses to boot up remote machines.13:08
rbasakThere are a few13:08
rbasakWhat was the path?13:08
rvbaallenap: I'm reviewing your branch: https://code.launchpad.net/~allenap/maas/query-strings-and-request-bodies/+merge/12747913:10
allenaprvba: Thanks.13:10
rvbaallenap: I think it will fix many bugs in one go :)13:10
allenaprvba: Yeah, I hope so. What *was* I thinking before?13:10
=== flacoste changed the topic of #maas to: 1 week until Final Freeze | Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | MAAS jenkins: https://jenkins.qa.ubuntu.com/job/maas-trunk/
sanderjrbasak, /var/lib/maas/ephemeral/precise/ephemeral/amd64/2012042413:12
sanderjrbasak, /var/lib/maas/ephemeral/precise/ephemeral/amd64/20120424/initrd13:13
rbasaksanderj: so I think that's not a maas specific issue, although perhaps maas is the only thing to exhibit it. I think the file you modified is coming from initramfs-tools or some package like that13:13
rbasaksanderj: can you check for existing bugs and if you can't find one, then please file a bug report with as much detail as you can?13:14
sanderjrbasak, where do I find maas bugs?13:15
sanderjrbasak, I think the bug is corrected with ubuntu, but not in maas.13:15
rbasaksanderj: https://bugs.launchpad.net/maas/13:15
sanderjNo bugs reported when searching for udev there.13:16
jammgz, jelmer: https://code.launchpad.net/~jameinel/maas/get-nodes-for-group/+merge/12748413:16
mgzjam: looking13:27
sanderjrbasak, ok, now it's reported. Let's hope it helps.13:28
rvbamatsubara: the fix for 1060079 should land any minute now.13:41
matsubararvba, great13:41
roaksoaxrvba: howdy!! Is make run broken?14:05
roaksoaxerr upstream trunk borken?14:05
rvbaroaksoax: jenkins seems happy, let me check14:05
rvbaroaksoax: everything seems fine, what error are you seeing?14:07
roaksoaxrvba: an import error. give me a sec an i'll show you14:10
rvbaroaksoax: apt-get install python-netiface maybe?14:11
roaksoaxrvba: django.db.utils.DatabaseError: relation "maasserver_config" does not exist14:11
roaksoaxLINE 1: ..._config"."name", "maasserver_config"."value" FROM "maasserve...14:11
roaksoaxrvba: also this:  it is not being cleaned: setlock: fatal: unable to lock /run/lock/maas.dev.cluster-worker: temporary failure14:13
rvbaroaksoax: the first error makes me thing that the database is simply not there because "maasserver_config" is an old table that was created months ago.14:15
rvbaroaksoax: the second one: sometimes celery gets stuck, just killall the celery processes and remove that file (/run/lock/maas.dev.cluster-worker).14:15
roaksoaxrvba: ImportError: No module named netifaces --> even though it is installed14:15
roaksoaxthere's really something weird going on14:16
roaksoaxrvba: what do you think might be causing that in my ssytem?14:23
rvbaroaksoax: difficult to say remotely, can you make sure first that all the processes have been killed?14:24
roaksoaxrvba: yeah they were, I just had rebooted my machine. IU'm trying again now14:25
mgzyeay, working migration14:43
mgznow that wasn't at all painful or owt14:43
mgz...which still falls over if run from scratch...14:45
mgzhm, the maasserver gets migrated before anything at all is done with metadataserver? that's fun.14:47
mgzwell, should be easy to fix (ho ho ho)14:49
roaksoaxrvba: http://pastebin.ubuntu.com/1256069/14:50
roaksoaxrvba: that's weird postgresql is running14:50
rvbaroaksoax: you can try to remove db/.s.PGSQL.543214:52
roaksoaxrvba: the file doenst exist14:54
rvbaroaksoax: not even db/.s.PGSQL.5432.lock ?14:54
roaksoaxnope14:54
rvbaroaksoax: can you wipe out the db or is there things in there you'd like to keep?14:56
roaksoaxrvba: so i rm -rf db/ and make run again and same issues14:57
rvbaroaksoax: does 'make sampledata' work?14:58
roaksoaxit did14:58
roaksoaxi'm re-making the whole environment14:58
rbasaksmoser: I can't get the precise daily armhf image to fail. I tried disabling dhcp on eth1 and it still works14:59
rbasaksmoser: but whichever way, please could you promote it to a release? Then I can have maas-import-ephemerals import armhf by default without breaking anything14:59
smoserrbasak, can you send me a console log ?15:00
rbasaksmoser: ...of it working?15:00
smoserbecause i dont like that i dont think it should work15:00
rbasakOK15:00
rbasaksmoser: err15:01
rbasaksmoser: BOOTIF seems to have arrived15:01
rbasakWell this is embarrasing15:01
rbasaksmoser: I'll check after this test but it seems that IPAPPEND support might h ave appeared in the lastest highbank U-BOot update15:01
smoserwell that'd be neat.15:02
Davieylol15:03
roaksoaxrvba: make sampledata works15:07
smoserroaksoax, ping15:19
rbasaksmoser: yeah IPAPPEND now works!15:20
smoserwell that is nice indeed.15:20
rbasaksmoser: one note though. With DHCP disabled on eth1, everything worked all the way through except after the installation cloud-init hung15:20
rbasaksmoser: and at that stage it's a local boot so no BOOTIF expected15:20
rbasaksmoser: thoughts?15:20
smoserit looks like little intel-jr is growing up.15:20
rbasak:-)_15:21
smoserprecise15:21
smoser?15:21
smoserquantal should work15:21
rbasakYes15:21
rbasakI'll check quantal15:21
smoseryou need a couple fixes SRU'd to precise15:22
smoser(they happen to be in that maas-ephemeral ppa , so you could try just adding that ppa and seeing if that makes it magic)15:22
rbasaksmoser: which ppa please?15:23
rbasaksmoser: ephermal-fixes?15:24
smoserhttps://launchpad.net/~maas-maintainers/+archive/maas-ephemeral-images15:24
rbasakthanks!15:24
roaksoaxsmoser: pong15:26
rbasaksmoser: is sources.list expected to be wrong in precise still?15:26
smoserrbasak, yeah15:27
smoserroaksoax, so ... we are to fix ipmi today15:27
rbasaksmoser: the PPA seems to have fixed it15:27
smoseryou did an install that quickly?15:28
rbasakI just updated15:28
roaksoaxsmoser: ok15:28
rbasaksmoser: unless first boot is expected to be different for some reason?15:29
roaksoaxrbasak: quick question... if we upgrade from precise to quantal, how is te change in arch from i386 to i386/generic is handled?15:31
rbasakroaksoax: the db migration just slaps /generic on the end of all existing nodes' architectures15:31
roaksoaxrbasak: cool!15:31
roaksoaxallenap: aroung?15:41
roaksoaxaround*15:41
rvbaroaksoax: allenap will be back around 1900 utc.15:41
roaksoaxrvba: alright. So you might help then :). So for the power related stuff, IPMI specifically, we need to ship a especial config file that will be used every time an IPMI command is executed15:42
roaksoaxrvba: were do you think the file should live, and how should it be referenced15:42
roaksoaxi was thinking it should live with the templates15:44
roaksoaxrvba: oh wai,t you were the one who did the power stuf right?15:44
rvbayeah15:44
roaksoaxrvba: hehe alright so it its you then :)15:44
rvbaroaksoax: "live with the templates"… which templates? :)15:45
roaksoaxrvba: http://paste.ubuntu.com/1256181/15:46
roaksoaxrvba: live with the templates, as in this file should be placed in the power template directory, (were ipmi.template is)15:46
rbasakroaksoax: mind that workaround stays in the right place in that patch15:47
rvbaroaksoax: src/provisioningserver/power/config/ seems like a good place to me15:47
rbasakroaksoax: also ipmi-chassis-config might need it too. Probably worth testing with </dev/null15:47
roaksoaxrbasak: the workaround is not being affected15:47
rbasakroaksoax: your patch puts it on the wrong line15:48
roaksoaxrbasak: ack!15:48
roaksoaxrbasak: k15:48
roaksoaxrbasak: i see now15:48
roaksoax:)15:49
rbasak:)15:50
roaksoaxrvba: do you know anything about jtv's changes on running maas-cluster-celery under user/pass?16:14
roaksoaxuser/group?16:14
rvbaroaksoax: yeah, I think he has landed that branch.16:15
rvbaroaksoax: is there a problem with that change?16:17
rvbaroaksoax: btw, did you manage to get rid of that weird problem you had?16:17
roaksoaxrvba: yteah i did manage to get rid of the problme i had16:17
roaksoaxrvba: and i think there is, i saw maas-cluster-celery bein unable to start16:18
roaksoaxbut now it starts :/16:18
rvbaBe aware of the fact that it does not start celery instantly, if first need to get the credentials from the region controller.16:18
roaksoaxrvba: yeah so it seems to do that16:19
roaksoaxbut still16:19
roaksoaxlet me check again16:19
rvbaroaksoax: arg, it seems the packaging has not been cleaned up: debian/maas-cluster-controller.maas-cluster-celery.upstart still contains setuid maas/setgid maas16:21
rvbaroaksoax: so apparently, he made the upstream change, but not the related packaging change.16:21
rvbaroaksoax: and /usr/sbin/maas-provision will refuse to do anything if not run as root.16:22
rvbaroaksoax: so now I wonder, how can you see it running… ?16:22
roaksoaxrvba: i didn't the problme was that it failed to start on an upgrade16:22
roaksoaxhence leaving the package unconfoigured16:23
roaksoaxrvba: i'll upload a fix16:23
rvbaOk, thanks.16:23
rbasaksmoser: just finished testing quantal daily ephemeral for quantal install. It works all the way through without problems.16:42
rbasaksmoser: can we get the precise armhf daily converted to a release soon now please? Is there anything blocking this? Then I can land a change for maas-import-ephemerals to import armhf by default.16:43
smoseri can do that now.16:44
rbasakthanks!16:44
rbasakAlthough you changed 'cloud-init boot finished' to 'Cloud-init v. 0.7 finished' so my expect script didn't match for success :-P16:44
mgzrbasak: yeah, smoser likes doing small cloud-init changes to break your scripts :)16:45
mgzI do now work with lucid->quantal though16:46
smoserrecent changes can be more blamed on harlowja16:46
smosersorry about that.16:47
mgzsmoser: the main annoyance is needing to support all versions, the new improved is very nice but having to work with lucid still makes it painful...16:48
mgzI'd like to use the file injection stuff now josh implemented it, but having two versions loses the simplifications...16:50
roaksoaxrvba: do you think it is possible to ship maas_local_settings.py in /usr/share/maas and have that source somthing in /etc/maas/local_settings.py or similar?16:55
roaksoaxrvba: or have a proper conffile?16:56
mercsniper_is the cloud init package still out of date for 12.04?16:57
melmothmercsniper_, i do not know, but last time i used maas (2 weeks ago) i did not experience problem with cloud-init16:58
mercsniper_k16:58
rvbaroaksoax: that's possible but that would simply add one additional level of complexity.16:58
rvbaroaksoax: what would it give us?16:59
roaksoaxrvba: ok so the problme is that we can no longer modify /etc/maas/maas_local_settings.py in packaging16:59
roaksoaxrvba: so it should only be modified by the user16:59
roaksoaxnot the package16:59
roaksoaxso if the user makes changes, on upgrade it gets prompted16:59
roaksoaxrvba: it would be like adding .d support16:59
roaksoaxrvba: cause if it is not done upstream, i'm gonna have to patch maas up and do it myuself17:00
roaksoaxDaviey: what was the package with the .d support for cobbler?17:00
Davieyroaksoax: it was a custom thing i did17:04
Davieynever ht the archive, still in my PPA17:04
rvbaroaksoax: that's definitely doable, we've got a tiny utility method to do that so the change should be simple.  Could you please file a bug with the details.  I'm probably not gonna be able to do it right now but Gavin might be able to do it later today.17:04
mercsniper_Is there a way to remove a node if the status is commissioning?17:05
roaksoaxrvba: cool, that way, we can simply edit /usr/share/maas/maas_local_settings.py or whatever in packaging17:06
roaksoaxrvba: and if the user wants to override something he can do it17:06
mercsniper_melmoth: are you using 1204 or 12.10 for your maas17:22
melmoth12.0417:22
mercsniper_still getting commissioning...17:22
melmothmercsniper_, did the machine restarted ? did you try to reboot it ?17:24
melmothi did not understand exactly how the power management thing worked, so some times i just rebooted nodes.17:25
mercsniper_I tried rebooting, I get cloud-init-nonet killed(300)17:25
melmothsome times they rebooted on their own (i m still puzzled as to why :-) )17:25
mercsniper_while it boots i get a landscape-client is not configured17:26
melmothi think i have seen that but did not looked like a real problem.17:26
melmothbut i did not see a cloud-ini-nonet killed17:26
mercsniper_init: clount-init-nonet main process (269) killed by TERM signal17:27
melmothhttps://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/101522317:28
ubot5Ubuntu bug 1015223 in cloud-init (Ubuntu) "cloud-init-nonet main process killed by TERM signal" [Low,Triaged]17:28
melmothdont panic i think it says :)17:28
melmothhmm, but there s a link to https://bugs.launchpad.net/ubuntu/+source/maas/+bug/992075 that is worryiing17:30
ubot5Ubuntu bug 992075 in maas (Ubuntu) "Commissioning status persists with cloud-init 0.6.3-0ubuntu1" [Undecided,Confirmed]17:30
melmothmercsniper_, is the date and time the same on the maas server and the machine you try to comission ?17:32
mercsniper_hm.... i would imagine17:32
mercsniper_is there a standard login for nodes?17:32
melmothnot untill they got the public juju ssh key injected17:33
melmothif you want a password login you need to make your own image (i dont know how, i saw once a doc telling how to)17:33
mercsniper_ah17:33
melmothi m asking about the clock thing because it hit me several time with juju stuff and because of comment 2 in https://bugs.launchpad.net/ubuntu/+source/maas/+bug/99207517:34
ubot5Ubuntu bug 992075 in maas (Ubuntu) "Commissioning status persists with cloud-init 0.6.3-0ubuntu1" [Undecided,Confirmed]17:34
melmothmercsniper_, see comment 12 https://answers.launchpad.net/maas/+question/19679117:36
melmoth(and what about booting on a live cd and running a ntpade so the local clock is roughly on the correct date and time on next reboot ? )17:37
mercsniper_true17:40
mercsniper_trying to bootstrap juju, i get an unexpected http 500 code....mean anything to anyone?17:52
mercsniper_this directory didnt exist /var/lib/maas/media/storage17:59
mercsniper_how long does commissioning take?18:35
melmothcouple of minutes18:35
mercsniper_then it must not be commissioning properly18:35
melmoththe only real long stuff is when you deploy a service, then it s  areal install18:35
mercsniper_gonna try virtual box instead of vmworkstation18:43
=== mercsniper__ is now known as mercsniper
roaksoaxsmoser: please :) https://code.launchpad.net/~andreserl/maas/packaging_updateS_bzr1134/+merge/12757018:54
smoserroaksoax, you copied bug https://bugs.launchpad.net/maas/+bug/1039513 to import-squashfs19:03
ubot5Error: ubuntu bug 1039513 not found19:03
roaksoaxsmoser: ack19:05
roaksoaxyeah we need to do verifications19:05
rbasakallenap: thanks for the review!19:07
allenaprbasak: Welcome :)19:08
smoserhttps://code.launchpad.net/~smoser/maas/trunk-remove-hostname-kludge/+merge/12757119:08
smosersomeone can tak that too19:08
rbasakallenap: there's https://code.launchpad.net/~racb/maas/arch-detect/+merge/127458 too, and then I'm done :-P19:09
allenapOkay, I'll try to look at those both.19:09
rbasakthank you!19:09
rbasakAfter that the daily PPA should in theory work for ARM19:10
allenapsmoser: Is there any way to make set -e work for broken command substitution?19:25
smoserwhat does that mean?19:25
allenapwrt. bug 106041119:25
ubot5Launchpad bug 1060411 in MAAS "maas-import-pxe-files does not catch failure of compose_installer_download_files" [Undecided,New] https://launchpad.net/bugs/106041119:25
smoseroh.19:25
smoserthats not hte issue.19:25
smosercommented in bug.19:26
smoserthe issue is just 'local' as the declaration succeeds, and that is what is checked.19:26
smoserso you just do those on 2 separate lines.19:26
allenapsmoser: If I do:19:27
allenapset -e; echo $(does_not_exist); echo $?19:28
allenapI get 0.19:28
allenapAh! But if I do variable=$(something) it will break.19:28
allenapOkay, got it.19:28
roaksoaxallenap: https://code.launchpad.net/~andreserl/maas/use_squashfs_filesystem_2/+merge/127577 --> I adressed 1, the rest is addings tests19:30
smoserthis is one reason i prefer:19:30
roaksoaxallenap: if you could take care of that, I'd deeply appreciate it19:30
smoser myfunc && myvar=$_RET || return 119:30
smoserto19:30
smoser myvar=$(myfunc) || return 119:30
smoserin addition to the fact that the second incurs a fork19:30
smoseryou're welcome to make fun of my hatred of forks. but you'll see why i hate them next time you dist-upgrade.19:31
melmothmercsniper, you install things in virtual machines ?19:31
roaksoaxsmoser: uhmm enlistment doesn't seem to be working :19:32
roaksoax:s19:32
allenapsmoser: Are you sure it incurs a fork, when calling a shell builtin? Try: echo $$ $(echo $$)19:32
smoseri'm positive.19:32
mercsniperMel: I am doing this work as a learning experience on my work laptop19:33
melmothhow much ram ?19:33
roaksoaxsmoser: did you change the console to which it is displaying the output?19:33
mercsnipervmworkstation lets you switch between machines19:34
smoserallenap, compare:19:34
smoser$ time sh -c 'for i in "$@"; do echo $(echo $i); done' -- $(seq 1 1000)  >/dev/null19:34
smoserto19:34
melmothi m using kvm to play with things and learn maas here19:34
smosertime sh -c 'for i in "$@"; do echo $i; done' -- $(seq 1 1000)19:34
melmothmercsniper, http://bazaar.launchpad.net/~pierre-amadio/+junk/c6100-jumpstart-maas/view/head:/README.txt19:36
roaksoaxsmoser: nevermind :)19:36
melmothir you want to give kvm a try instead of virtuabox..Should just work "out of the box"19:36
allenapsmoser: Yeah, I can see it now; another way to demonstrate it is looking at the process list when running: echo $(read)19:36
mercsniper_unfortnately, im on a windows host19:40
melmothhey, good reason to install something new on your laptop :)19:40
mercsniper_laptop needs to stay windows per company policy19:41
mercsniper_thats why its all virtual19:41
melmothi do it in kvm because it s easier to get one machine with lots and lots of ram, than 10 little ones with switch and wire and stuff19:42
roaksoaxsmoser: updated19:44
* roaksoax hates chromium crashing19:45
smoserreguarding wrap-and-sort, i was only really complaining about the python-netifaces19:46
roaksoaxallenap: what extra tests did you have in mind for the suqashfs19:46
smoserand it turns out i was wrong there anyway19:46
smoser:)19:46
smoser(i thought that would sort after the ${misc:Depends}19:46
smoseri'll ack this because wrap-and-sort is generally a good thing.19:46
smoserah. but you approved already.19:47
smoser:)19:47
roaksoaxsmoser: yeah :)19:48
roaksoaxthanks19:48
smoseroh... roak!19:50
smoserset -e ?19:50
smoserer...19:50
smoserset -x ?19:50
smoserreally?19:50
roaksoaxsmoser: in maas-import-squashfs?19:50
roaksoaxsmoser: yeah I forgot19:51
roaksoaxsmoser: a branch is ready for review that allenap needs to review19:51
roaksoaxthat completes that19:51
roaksoaxand fixes it19:51
roaksoaxcompletes the support and fixes that19:51
smoserroaksoax, isntall of maas-dhcp from experimental results in no /etc/maas/dhcpd.conf19:53
roaksoaxsmoser: tbh i hjave not been following up on what they've been doing with maas-dhcp19:53
roaksoaxsmoser: but i'll audit19:53
smoserok. well i'm looking for a way to have a maas functional19:55
smoseryou suggested experimental19:55
smoserthat didn't work19:55
roaksoaxsmoser: yeah so that means that's broken somehow19:58
roaksoaxsmoser: i don't think the dhcp server is still functional19:58
roaksoaxclear20:04
smosermatsubara, did you install maas-dhcp recently?20:08
matsubarasmoser, yes20:09
matsubarawell20:09
matsubarafound a bug with the package this morning20:09
matsubarahttps://bugs.launchpad.net/ubuntu/+source/maas/+bug/106023720:09
ubot5Ubuntu bug 1060237 in maas (Ubuntu) "apt-get install maas maas-dhcp maas-dns fails" [Undecided,New]20:09
smosermatsubara, roak has a fix for that https://code.launchpad.net/~andreserl/maas/packaging_updateS_bzr1134/+merge/12757020:10
smosermatsubara, do you have notes available on how you install and configure?20:11
matsubarasmoser, I follow the checkbox tests and have a local note like this: https://pastebin.canonical.com/75751/20:13
smoserwhere are the checkbox tests?20:14
smoserallenap, https://code.launchpad.net/~smoser/maas/trunk-remove-hostname-kludge/+merge/12757120:19
roaksoaxsmoser: you want me to integrate it inside maas-signal right?20:22
smoserwell, we want maas signal to call it.20:23
smosererr..20:23
smoserthe scripts there to call yours, and then post back the results20:23
roaksoaxsmoser: ok20:23
roaksoaxsmoser: i'm doing this too: http://paste.ubuntu.com/1256796/20:23
smoserwell config probably shouldnt be executable20:26
smoserbut other than that i think it looks reasonable20:27
roaksoaxcool20:30
smosermatsubara, how do i enable dhcp?20:35
matsubara$ maas-cli api maas node-group-interfaces new master ip=192.168.21.1 interface=eth0 management=2 subnet_mask=255.255.255.0 broadcast_ip=192.168.21.255 router_ip=192.168.21.1 ip_range_low=192.168.21.10 ip_range_high=192.168.21.5020:36
matsubarasmoser, ^20:36
smoserso where do you have those notes?20:36
smoserie, is this part of the "checkbox install" that you mentioned?20:36
matsubarasmoser, checkbox tests are linked in this doc: https://docs.google.com/a/canonical.com/document/d/1GNrJCL8EyfSw7ypCCYjH0BuIgIEDP2E6Y9Xbb7Gx8rs/edit20:37
matsubaraand my notes are in the pastebin20:37
matsubaraand I rely a lot in the shell history as well :-)20:37
smoserthis *really* needs to not be a private google doc20:37
smosermatsubara, and how do you set up a maas user and such ?20:38
smoserit seems like you probably have done a lot of things that i want to do20:39
smoserand i'm just trying to avoid us both doing them.20:39
matsubarasudo maas createadmin --username=admin --password=test --email=example@canonical.com20:39
roaksoaxsmoser: any thoughts "1349210267.806     72 192.168.123.101 TCP_DENIED/403 3728 GET http://192.168.123.2/MAAS/static/images/amd64/generic/quantal/filesystem/filesystem.squashfs - NONE/- text/html"20:39
roaksoax?20:39
smoseri would say you are being denied access to that20:39
smoser:)20:39
smosercheck /var/log/apache/*.log20:40
smoser(including error)20:40
roaksoaxsmoser: lol yeah I mean, squid-deb-proxy doesn't allow the installer to download the squashfs image20:40
smoseri suspect your being expected to oauth20:40
roaksoaxsmoser: any thoughts on how can we fix it?20:40
roaksoaxsmoser: i was thinking on telling squid-deb-proxy to allow access to the maas server in question20:40
roaksoaxby hacking on the packaging20:41
roaksoaxbut maybe you know of a better way20:41
smoserwell why is that going through the proxy20:41
smosersquid-deb-proxy is explicitly a *deb* proxy20:41
roaksoaxsmoser: because we are telling the installer to use the proxy20:41
smoserhm..20:41
smoserwell that would seem like a bug one way or another20:42
smosereither in that we're gelling it there is a generic proxy20:42
smoseror in that it is assuming what we said it should use for an archive proxy it can use for other things20:42
smoserbut yeah, to fix that i guess you will probably have to have it allow proxying of /MAAS/static/images/*20:43
roaksoaxmaybe squid itself  is blocking it20:43
smoserroaksoax, i thought htat is what you were saying20:47
smoseri'm confused now.20:47
smoserwere you thinking maas was saying that?20:47
roaksoaxsmoser: i meant squid3 itself (not the instance squid-deb-proxy spawns)20:47
roaksoaxsmoser: but it is squid-deb-proxy20:48
roaksoaxif I add the IP address of the MAAS server facing that network, it allows it20:48
smoserroaksoax, i'll be back in later tonight. (probably 3+ hours from now)21:05
allenaproaksoax: I'm looking at use_squashfs_filesystem_2 now.21:07
roaksoaxsmoser: alright, i'll be later here too21:08
roaksoaxallenap: awesome thank you!21:08
roaksoaxallenap: are you gonna be at UDS btw?21:25
allenaproaksoax: Yeah, you?21:25
roaksoaxallenap: yeah!! is the rest of the team gonna be there?21:26
roaksoaxs/team/squad21:26
allenaproaksoax: Yeah, I think we're all going. smoser, you at UDS?21:27
roaksoaxallenap: yeah all of our team is gonna be there21:27
allenapCool :)21:28
roaksoaxallenap: alright, so I hope you guys don't run away from Peruvian Pisco :P21:28
allenaproaksoax: Oh god, I was given Pisco by Nicolas at the Barcelona UDS. I haven't been able to drink spirits since then.21:33
allenapI'll give it a go though :)21:34
roaksoaxlol21:34
allenaproaksoax: I've changed is_squashfs_image_present in lp:~allenap/maas/use_squashfs_filesystem_2, and updated the tests. It doesn't test the expansion of the templates, but I need to go and sleep now ;)21:35
roaksoaxallenap:  is there any example?21:36
roaksoaxon how to do it?21:37
allenaproaksoax: There are general tests for expansion, but not for the specific templates in contrib/...21:38
allenaproaksoax: Don't worry about it. It's something we ought to address after 12.10. The confusion about inheritance means that we should revisit this stuff anyway.21:39
roaksoaxallenap: alright, cool21:39
roaksoaxthanks for helping out!21:39
allenaproaksoax: Pull my branch (it directly follows on from yours), push it up, and I'll +1 that mp.21:39
allenaproaksoax: So, sorry I didn't get to this yesterday.21:44
roaksoaxallenap: no worries :)21:44
roaksoaxallenap: thank you for helping tho21:44
roaksoaxallenap: pushed the changes to the MP!! thanks a lot again! and have a good night!21:47

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!