=== Aaton_off is now known as Aaton [01:23] Can anyone tell me what the mechanism is in maas that copies ssh keys from ~/.ssh/id_rsa.pub to the boot image? [03:20] Can anyone tell me what the mechanism is in maas that copies ssh keys from ~/.ssh/id_rsa.pub to the boot image? [03:21] kurt_: yes, 2 ways [03:21] 1. juju does it for you if you use juju [03:21] 2. you need to register them in maas against your maas user [03:21] if you start a node manually use #2 [03:22] is this mostly done by maas, or juju? [03:22] so, create your key, configure it in the settings, right? [03:22] kurt_: is what done? [03:23] if you are using juju you don't need to worry about it [03:23] I guess I wondering what mechanism/script does a chroot and copies the key in your home directory to the image that gets booted? [03:24] it gets passed via preseed data [03:24] because, even though my clocks are all relatively up to date, I am having trouble with one or two servers booting [03:24] (these are VMs again, remember) [03:25] watch the console log when the VM boots [03:25] it should give some info [03:25] maas.log or the VM's console? (ie. dmesg or syslog/messages)? [03:25] VM console === Aaton is now known as Aaton_off [03:27] kk - how does server/VM force a reseed of the pre seeded software? is there some way to force this? [03:27] presided data rathere [03:27] preseeded data rather [03:28] kurt_: http://pastebin.ubuntu.com/1040179/ [03:29] thnx Julian [03:29] np [03:29] can you tell me when the next full release is scheduled vs git releases? [03:30] you mean bzr? we don't use git [03:30] apt-get update maas :) [03:30] there's a point release scheduled with 12.04.01 [03:31] and it'll run without cobbler [03:31] kk - are you guys looking at forcing delete even when node is in state "commissioning" or will that always be left to the shell? [03:32] that's fixed in trunk already [03:32] cool, nice to have [03:32] will shell features be more completely documented? :) [03:32] unlikely [03:32] I've been learning what I know via newsgroups [03:33] or postings [03:33] they are for developer and debug usage [03:33] we'll document supported stuff for sure [03:33] but people are free to document unsupported stuff themselves :) [03:34] If I can figure it out, I'm happy to ;) [03:35] Like the section on the non-dhcp set up of maas :) [03:35] that's not supported yet [03:35] that will be really nice to have [03:35] presuming you mean external dhcp [03:35] I do [03:36] it's not really something that we're concentrating on because it makes things much harder to use [03:36] understood - get things working first [03:36] get it solid [03:36] well, the dhcp actions are integral to how maas operates [03:36] yes! [03:37] we're moving all this out of cobbler and into maas at the moment [03:37] so when maas does it itself it should be more obvious how to set it up manually [03:37] because of the pxe stuff, this very much reminds me of clonezilla too [03:38] a question about the documented system requirements - are those pretty strict? [03:38] ie. 16 GB for the master node? [03:39] I'm not sure, but it largely depends on the size of your cloud [03:39] I am debugging locally on a 4GB box [03:39] for testing and demo purposes, I'm just trying to put together a minimal functional installation [03:39] which has a master and a few VMs running on it. Admittedly it gets sluggish :) [03:39] I don't care about performance at this point [03:40] but what about the requirement of 9 nodes for openstack? [03:40] is that strict, or just a "suggested" number? [03:40] I don't know much about that [03:40] my end goal is to get a functional openstack installation working [03:40] ok [03:40] we'd love to help [03:40] ie. on top of maas [03:41] and juju [03:41] when are you normally around on IRC? [03:41] I'm GMT -8 (PST) [03:41] I'm on all day mostly [03:42] ok - there are people who know more intricaces of the server stuff than me who are in -5 [03:42] and I am happy to contribute to the documentation and such [03:42] cool [03:42] this is great stuff you guys are doing btw [03:42] I'm in GMT+10 so barely overlap with anyone [03:42] glad you like it [03:43] it's still a little raw but there's some cool stuff coming [03:45] so you are a core developer, how many others? Are you employees of canonical? [03:48] yeah, there's me and 3 others working on the core web app [03:48] and a few others doing packaging and server stuff [03:48] are you dedicated, ie. full time on this? [03:48] yup [03:49] nice. [03:49] I take it you are in austrailia or asia? [03:49] yeah Brisbane [03:49] excellent [03:50] I almost went to work for a contract with Telstra a few years back [03:51] my wife put the brakes on that :( [03:52] did I see above you guys are trying to get rid of cobbler out of the equation? [03:52] we are [03:52] I think the more you can streamline this, the better [03:52] cobbler is pretty horrible; the security team dislikes it and we don't think it will scale [03:53] are you beginning to think of how things can scale in much larger deployments? [03:53] not just beginning, it was a core goal [03:54] what processes are the best candidates for scaling? [03:54] not sure I understand [03:54] scaling = spreading the love [03:55] what do you mean by processes? [03:56] for example maas-dhcp, which configures... [03:56] trying to think of the dhcpd process used [03:56] there are two :) [03:56] dnsmasq or isc-dhcpd [03:57] dnsmasq is what I was thinking of... [03:57] we're not supporting dnsmasq when cobbler has gone [03:57] what will take that function over/ [03:57] ? [03:57] the other one :) [03:57] it's more proven in large environments [03:57] will it be capable of perhaps supporting VLANs? [03:58] multiple large scale VLANs? [03:58] vlans are orthogonal really [03:58] network admins are free to set any topology [03:58] we're just providing a worker on a per-subnet basis [03:58] what kind of api's are you guys thinkng of? [03:59] the existing API won't change much [04:01] ok, I've bothered you enough tonight :) [04:01] lol [04:01] heh np [04:01] thanks for fielding my questions... [04:01] my pleasure [04:01] I'll look forward to your releases [04:01] bfn [04:01] we look forward to you trying them out [04:02] I'll be happy to give input [04:06] what is your suggestion when clock is in sync, and VM gets stuck in "init: cloud-init-nonet main process (307) killed by TERM signal"? [04:06] this has been killing me with only one of my VMs [04:07] I've tried recreating the mac address, destroying and recreating the VM numerous times [04:07] I have to manually delete the name via the maas sheel [04:07] shell rather [04:08] but always gets stuck in commissioning with that above problem [04:11] that's rather odd [04:11] is there no other clue as to why the SIGTERM happens? [04:12] since I can't get access to the VM until its booted, I'm limited in what I can see [04:12] I usually hard power the VM down [04:13] is there some mapping associated with the naming? [04:13] ie. my hosts name is maas02 [04:13] smoser or roaksoax, are you guys around? Any idea with the above? [04:14] so is some key in the db maybe not being deleted and some artifact in the db remains that's being recycled? [04:14] or some cobbler files that the maas shell doesn't delete that need to be manually deleted? [04:14] what does cobbler think about the node's power details? [04:15] can you tell me how to look? [04:15] sorry…not cobbler saavy [04:15] well it's turning it on so it's probably ok [04:15] its not [04:15] I have to manually boot [04:15] aha [04:16] vmware doesn't really support WOL [04:16] well you can manually set the virsh parameters in cobbler [04:16] it's not really supported but it'll work [04:16] virsh = kvm, not vmware, right? [04:17] vmware should work too [04:17] not tried it personally [04:17] is there some rtfm I can do around this? [04:18] browse to localhost/cobbler_web [04:18] Not Found [04:18] The requested URL /cobbler_web was not found on this server. [04:18] must it be on the local node? [04:19] no it's on the maas server [04:20] ummm different port [04:20] * bigjools tries to remember [04:20] 8080? [04:20] nope :) [04:20] you have 64943 more tries ;) [04:20] heh [04:21] hmmmm it's possible that the web part is not installed [04:21] you'd have to dink about in the command line [04:21] there's an exe called "cobbler" [04:22] it sounds like the cobbler DB is out of whack with maas anyway [04:22] that sounds logical [04:22] is there a sync process? [04:22] the only way I know of fixing it is to delete the nodes on both sides [04:23] how do I fully delete the node on the cobbler side? I think I know how to on the maas side [04:23] you can sync manually but it's error prone [04:23] via the cobbler web or command liner [04:24] do I need to install the cobbler web application? [04:24] and why do you guy want to get rid of cobbler? Its so easy and painless? [04:24] LOL! [04:24] heh [04:25] what are your timelines for getting rid of cobbler? [04:26] ASAP [04:26] couple of weeks [04:27] so, if I begin to install cobbler-common, etc - is that going to overwrite some of the maas related configuration? [04:27] yes [04:27] ugh [04:27] the packages conflict [04:27] :) [04:28] looks like I'm not going to be able to install cobbler-web [04:28] fraid not [04:29] the way I get around it is to install cobbler on a separate VM [04:29] and point maas at it [04:30] otherwise, I need to figure out the command line stuff, right? [04:31] do any of your comrades have a better understanding of this part? [04:32] I do see the problematic node is listed 3x: [04:32] ituser@maas01:~$ sudo cobbler system list [04:32] default [04:32] node-2e11a386-b5c1-11e1-8715-000c29de71df [04:32] node-dc7aa15e-b5ce-11e1-8715-000c29de71df [04:32] node-ff6a0be4-b5d5-11e1-8b34-000c29de71df [04:33] ah nice [04:33] delete everything and re-add it [04:34] I think I've tried that…but I'll try again [04:34] make sure it's deleted in cobbler [04:34] so, just do cobbler system remove --name=xxx for all, then relist the systems as shown above? [04:35] or is there some other sync too? [04:35] or other place in cobbler it needs to be deleted? [04:35] remove in maas and cobbler [04:35] make sure you do a cobbler sync [04:35] as in the command "cobbler sync" [04:35] ah, ok [04:46] what is the difference between "cobbler sync" and "maas-import-isos" ? [04:46] the former tells cobbler to sync its internal database [04:47] the latter is the script that pulls the images from maas.ubuntu.com [04:48] they look very similar in their functions [04:48] i.e.,after images are downloaded [04:49] maybe maas-import-isos does a cobbler sync as a part of its functions? [04:50] it does [04:51] k, I've managed to screw something up now. web process is stuck. [04:52] I removed all of those extra cobbler systems, but it somehow had an association with the newer mac adds associated with maas02 [04:52] so when I went to try to remove that via the web interface, things went south [04:55] ah, like windows. rebooting fixed everything :) [04:58] File "/usr/lib/python2.7/dist-packages/maasserver/provisioning.py", line 254, in __call__ [04:58] raise friendly_fault [04:58] ExternalComponentException: Unable to reach provisioning server. [04:58] looks like there is artifact in maas db [04:58] will try to reprovision to see if that fixes... [05:52] ok, discovered issue… note to self - ensure you have a complete os and system writable disk before proceeding with netboot [05:56] heh [05:56] configuration couldn't write to vm's disk [06:02] so process when creating maas VMs is to build your VM first with complete install, then switch VM to PXE boot. [06:02] glad it works [06:03] me too. so far so good [06:03] I just got powering on with virsh working without cobbler too, so it's all going well today [06:03] nice [06:03] is that some that works in the current build? [06:03] no [06:03] ok, something to look forward too :) [06:04] ah - ok, I see - cobbler is a lot of the reason you need to do that... [06:04] it has the PXE functionality you need [07:20] bigjools: here [07:20] o/ [07:20] ok [07:20] Doesn't something in there block until at least the files have been gathered? [07:20] no idea [07:21] I'll find out shortly [07:21] I hope it's not the code that (AFAICT) decides to deal with umount failure by doing the umount again. [07:22] grepping for "import" doesn't help .... [07:23] Oh, I'm sure the right answer will be in the output. [07:23] Somewhere. [07:23] * bigjools stabs a pin in [07:24] qq: can the hostname be changed in the maas interface without repercussion after the hostname has already been created like "node-000c25abcc"? [07:25] kurt_: the code for maas itself doesn't care, but something else that maas relies on might. [07:25] Not very helpful, I know. [07:25] :) [07:26] ok, best to leave it alone [07:26] Specifically, if it's okay with juju and with cobbler, it should be okay. [07:27] MAAS in its current incarnation (which still uses cobbler) propagates such changes to cobbler immediately. But we're on the cusp of radical change. [07:27] that's a wee bit of a frustrating part of that interface is having to maintain my own mac adds to human friendly name mapping, esp where VMs are concerned [07:29] You can leave the hostname blank. [07:29] I'd assign it only in cases where the hostname has a special meaning to what you're doing. [07:30] and yes, je was talking about the whole rework of the cobbler stuff [07:30] that's great work [07:30] Where “easy work” would be preferred, of course. :) [07:31] go for the low hanging fruit of course... [07:32] bigjools: looks to me like cobbler's import_tree rsyncs the whole given directory into its own storage. [07:32] jtv: yeah just looking at it [07:33] Not too surprising, really, given how tiny the two files we want are compared to the whole image we download [07:33] nice of it to say "importing from a network location" regardless [07:35] bare excepts.... yegads [07:36] More detail in modules/manage_import_debian_ubuntu.py — with just a hint of suspicion that we've got several layers of scripts and code duplicating similar support functionality. [07:36] yup [07:38] This is going to drive me to drink. [07:38] at least you'll be sober in the morning [07:38] the code will still be shit [07:39] ok, last question of the night…then off to bed... [07:39] trying to delete a node from maas, seeing "Unable to reach provisioning server." [07:39] restarted pserv [07:39] no joy [07:39] any ideas? [07:39] what's in the pserv log? [07:40] here's little excerpt from maas.log, will get pserv [07:40] * jtv braves the 30—40kg dog in a desperate dash to the bathroom [07:40] File "/usr/lib/python2.7/dist-packages/maasserver/provisioning.py", line 254, in __call__ [07:40] raise friendly_fault [07:40] ExternalComponentException: Unable to reach provisioning server. [07:41] not really much.. [07:41] 2012-06-14 00:38:35-0700 [QueryProtocol,client] 127.0.0.1 - - [14/Jun/2012:07:38:35 +0000] "POST /api HTTP/1.1" 200 1265 "-" "xmlrpclib.py/1.0.1 (by www.pythonware.com)" [07:41] 2012-06-14 00:38:35-0700 [QueryProtocol,client] 127.0.0.1 - - [14/Jun/2012:07:38:35 +0000] "POST /api HTTP/1.1" 200 1265 "-" "xmlrpclib.py/1.0.1 (by www.pythonware.com)" [07:41] 2012-06-14 00:38:35-0700 [QueryProtocol,client] Stopping factory [07:41] 2012-06-14 00:38:35-0700 [QueryProtocol,client] Stopping factory [07:42] ok, completely knackered [07:42] see you guys tomorrow [07:42] kurt_: that's the maas log [07:42] not much to go on [07:43] as distinct from the pserv log [07:43] last bit is pserv.log [07:43] guess I need to turn up debug [07:43] Sounds as if the maasserver really can't reach the pserver then [07:43] Try a ps -ef | grep pserv? [07:44] See if it's really running or not [07:44] ituser@maas01:~$ ps -ef | grep pserv [07:44] maas 15396 1 0 00:35 ? 00:00:00 /usr/bin/python /usr/bin/twistd -n --uid=maas --gid=maas --pidfile=/run/maas-pserv.pid --logfile=/dev/null maas-pserv --config-file=/etc/maas/pserv.yaml [07:44] Love that --logfile option [07:45] interesting... [07:45] default - is that set in pserv.yaml? [07:45] No idea tbh [07:45] jtv: so import looking for repos, kickstarts and bootloaders [07:45] ok, good night [07:45] nn kurt_ [07:45] nn [07:45] can't focus any more [07:46] then don't overdo it :) [07:46] thnx guys [07:48] jtv: I think we need more help from server guys [07:48] and get their take on what needs doing [07:48] Can't disagree with you. [07:49] yay, late night for me [07:50] So sorry about that. :( I've been trying to throw extra hours at this, but haven't had the energy lately. [07:51] It does look as if the core of the Cobbler import code looks for initrds and kernels. [07:52] But it's all much more OS-agnostic, of course. [07:53] yeah [07:53] but there's then these layers of bash [07:55] Suddenly the name makes sense to me. [07:55] bigjools: I've updated the Questions section of the google doc. [07:56] ok [07:59] bigjools: we'll also want to deal with updates that happen concurrently to net-boots. [07:59] In other words, we can't just replace an initrd/kernel pair while nodes are using it. [08:00] depends if tftp keeps a file handle open? :) [08:00] So we may need to number versions. The server guys suggested something like this, but I cut it short since it wasn't part of the immediate problem. [08:00] Bear in mind that it's stateless, UDP. [08:01] There's no such thing as an ongoing session or even an ongoing download. A request for a chunk of file comes in and gets serviced, and total amnesia follows. [08:01] At least conceptually — optimization may be a different issue. [08:04] I've no idea how it works internally [08:04] anyway [08:05] dinner, back later [08:05] bon appétit [08:13] hi there mrevell [08:13] Hello jtv === matsubara is now known as matsubara-lunch === mrevell_ is now known as mrevell === matsubara-lunch is now known as matsubara [13:32] hey maas gurus, is there a command line way of adding/deleting nodes from a maas? [13:33] the command 'maas' on the command line gives me a number of options but not many seem specific to maas [13:35] cheez0r: http://paste.ubuntu.com/1040776/ [13:36] right, that does the delete [13:37] To create a node manually: Node(hostname='sss', status=3) note that you will need to know the internals a bit to provide the right parameters to the node creation method. [13:37] node = Node(...) [13:37] node.save() [13:39] can I do node = Node.objects.get(hostname='myhostname') [13:39] node.show() [13:39] ? [13:39] node.print()? [13:39] trying to figure out how to get an existing node out to use as template [13:41] node.__dict__ [13:41] danke [13:42] I don't know your constraints but note that you would probably be better off using the API. [13:43] just trying to come up with a way to create a file that has mac,arch,hostname and can reproduce what I'm doing in the web gui [13:43] kind of slow to add eleven nodes via the GUI when I'm adding/removing them a lot [13:44] plus for auto-commissioning of a maas cluster that'd be useful [13:46] if I was more of a programmer Id just write a command-line front end to the API, but I'm not much of one :p [13:47] That's something that's on our TODO list. [13:47] I'm sure, now that I think about it. [13:57] roaksoax: The fix to support fetching YUI's file from the package has landed. Please ping me if you update YUI's packaging as Francis suggested yesterday. I'll need to change one line in contrib/maas_local_settings_sample.py if you change the place where the files are located. [13:58] roaksoax: you'll see a new config option in contrib/maas_local_settings_sample.py: YUI_LOCATION. [13:58] rvba: awesome! thanks. I'm emailing the java script team in Debian to talk about that [13:58] roaksoax: cool. [13:59] rvba: I cannot follow Francis suggestion libjs-yui35 because that'd mean introducing a new source and I'm not sure whether we wanna do that or not yet [13:59] ^^ [13:59] smoser: ^^ [14:08] rvba: was the directory suggested 'yui//build/*.js'? [14:09] roaksoax: it's not really suggested, but that's what the JS side requests by default. [14:10] rvba: the JS side in *maas*? or generally? [14:11] roaksoax: sorry for being vague, the JS side in YUi. YUI3 has this mechanism where you load a small initial file and then this tiny bit of code requests the needed additional modules all in one request. [14:13] roaksoax: does that make sense? [14:15] rvba: kindof, is there any documentation that you could point me at? [14:16] roaksoax: http://yuilibrary.com/yui/docs/yui/loader.html [14:17] roaksoax: you'll see that the default 'root' option is '{version}/build'. Because the idea is to be able to have multiple versions of YUI3. [14:28] rvba: right, but even so, we can only have 1 version of YUI 3.5.X [14:28] ah nevermind :) [14:30] rvba: does this make sense to you? http://pastebin.ubuntu.com/1040864/ [14:31] Looking. [14:34] roaksoax: Look good. One remark: I'd drop the part about having 'build' in the path. I think this is the default because of the way the code is structured in the yui tarball (src/ docs/ build/ etc.). But when it comes to having that packaged, it really makes sense to have only the files in build/ and drop everything else. The main question is the about having the version in the path and the package name. [14:36] rvba: right, ah I thgouht you wanted everything to be installed under ./build/ [14:37] No, YUI 'likes' it by default. but I think it's just a convenience. In a package I think it makes sense to drop that. [14:37] rvba: ok [14:39] rvba: ok cool. email sent! === med_ is now known as medberry === medberry is now known as med__ === med__ is now known as med_ [14:53] paultag: could you pelase repeat this for rvba to give his opinion [14:53] rvba: ^^ [14:54] I was wondering how you suggested dealing with node.js policy, since it's a slightly different subset of javascript policy it's self - adding a version path will cause some pretty gnarly behavior (as much as I would kill to have scripted languages major-versioned) [14:55] so I was wondering if you had an idea for how to keep javascript and node.js normalized, or if this was just to solve the problem at hand [14:55] also, using update-alternatives for a "current" version of the lib would be nice to add, IMHO anyway [14:56] (also wrt package naming, since node.js folk have some jacked up version schemes) [14:56] so, TBH i'm not a JS expert, but was just given the task to upgrade the mpackages, and obviously by doing so, some concerns arised [14:56] major version doens't always mean API stability :( [14:56] roaksoax: of course, I totally ACK that there is a problem [14:56] not that I do much javascript work, I was just here, saw @ubuntu and figured I'd see what you thought [14:56] I'm just a casual member of that team === med_ is now known as med_out === med_out is now known as med_away === med_away is now known as med_ [14:57] javascript policy was written by dapal on the wiki, and it's very short -- getting it changed will not be so much of an issue, there are two active folks on that team ATM [14:57] paultag: so I do not have an idea of keeping js and node.js normalized but would definitely make sense doing so, as well as using update-alternatives [14:58] OK, I was just wondering :) [14:58] node.js isn't in testing, so it's not so much an issue [14:58] (whereas plain js libs are) [15:00] right. [15:00]