=== wallyworld_ is now known as wallyworld === CyberJacob|Away is now known as CyberJacob === vladk|offline is now known as vladk [07:20] bigjools: Do you really need to resort to raw SQL in get_hostname_ip_mapping? [07:21] hoist. petard. etc [07:21] rvba: yes. [07:21] same as the other same-named func does [07:23] Ah, right, I think I see why now… [07:24] me too [07:24] debugging it made it clear, it's not the same dict [07:24] bigjools: I /think/ Django grew support for "Distinct ON". [07:25] well, still not sure why it's lazy, but uh [07:27] oh maybe it's a memory thing [07:30] rvba: "I'd rather properly test the behavior" [07:30] bigjools: in this branch you've added StaticIPAddressManager.get_hostname_ip_mapping but it's not used anywhere yet… do you have a follow-up branch that makes use of it? [07:30] testing behaviour: that's what I am avoiding [07:30] it's slow [07:31] yes I am using it locally but struggling to get tests to pass [07:31] Yeah, I meant end-to-end behavior. For the reason I stated. But I'm okay with mocking, it's just not as reliable (and that's why you missed the fact that the old get_hostname_ip_mapping wasn't doing the right thing). [07:32] these tests are a nightmare [07:32] they are running the same code twice because of signals [07:33] The ones in test_dns? [07:33] what wasn't it doing? [07:33] yes test_dns [07:33] make_nodegroup etc etc causes signals [07:34] Your tests aren't exercising the code up to the point where get_hostname_ip_mapping is called. === CyberJacob is now known as CyberJacob|Away [07:34] so the same dns code is run [07:34] rvba: yes I know, that is the intention [07:34] Ah, okay, cool then. [07:34] it's tested elsewhere [07:34] well, ideally in these :) [07:35] rvba: every time create_managed_nodegroup() is called in the test it fires a dns job through a signal [07:35] bigjools: yeah, but until we call 'self.patch(settings, 'DNS_CONNECT', True)' that doesn't trigger a real task. [07:36] bigjools: you'll see that 'self.patch(settings, 'DNS_CONNECT', True)' is called *after* all the setup is done. [07:36] true, but, still feels like one of the main reasons tests are slow [07:36] It's possible. [07:36] they have gone massively slow on me lately, something has changed [07:37] I didn't notice anything this dramatic on my side. [07:37] and we have not put enough importance on the fact that fast tests are a necessity [07:37] I agree with that. [07:39] hurray a passing test [07:39] * bigjools fixes others [07:41] rvba: very glad you made create_nodegroup_with_lease() [07:41] easy to change to create static ip instead and thusly fix many tests in one fell swoop [07:42] Encapsulation FTW :) [07:43] wooo passing tests [07:43] I'll push a new rev in a sec [07:45] rvba: annoyingly there's a bi-dependency between node and dns now, so inline imports are needed :( [07:45] rvba: dns.py can't import from maasserver.models.staticipaddress === roadmr is now known as roadmr_afk [07:45] I see. [07:46] rvba: anyway new rev pushed, can you have a look please? I am going to eat, back in a little while [07:47] bigjools: looking at the code, I think you're right and the only reason why lazydict was introduced was to make use of __slots__ and thus save memory space in case the mapping is huge. [07:47] bigjools: sure, looking now. [07:47] yeah [07:47] bbiab === mwhudson is now known as mwhudson- === mwhudson is now known as mwhudson-bip [08:27] rvba: "deviation from the original design"... not as I understood it! We spoke yesterday and said that dynamic IPs do not need dns CNAME entries === mwhudson-bip is now known as mwhudson === roadmr_afk is now known as roadmr === vladk is now known as vladk|offline === mwhudson is now known as mwhudson-bip [09:48] gmb: I remember seeing a bunch of "6" suffixes on keywords in an IPv6 sample config. [09:48] jtv: Yeah, it really is pretty much that simple :) [09:49] Ran 2647 tests in 1889.564s [09:49] (at least on my test network, on a small scale) [09:49] urgh [09:49] Why, why, why does factory.make_node() perform two update queries on the node it just created, and then another query to load back the object!? [09:50] bigjools: The v6 stuff couldn’t have increased runtimes could it? I.e. if there’s lots of “if IP in network” where network is a /64; that kind of thing? [09:50] doubt it [09:50] Mind you we’re not doing it in many places yet. [09:50] exactly [09:50] profile. we need a profile [09:57] rvba: the branch is all done now, if you want to look again. All I did was fix a couple of failing tests and remove the dns update for new leases. [09:58] * jtv longs for a utilities package [10:01] gmb: did you land your make-a-random-ipv6-address factory method? [10:02] jtv: Yes. it’s factory.get_random_ipv6_address(). [10:02] bigjools: LGTM [10:05] gmb: wha!? Why not just make_ipv6_address()? [10:05] Maybe I should have said... [10:05] bigjools: are you witnessing a general slowdown or is it confined to a subset of the tests? [10:05] rvba: general [10:06] rvba: thanks, BTW [10:17] The branch I described this morning is up for review: https://code.launchpad.net/~rvb/maas/utopic-syslinux/+merge/224182 === roadmr is now known as roadmr_afk === vladk|offline is now known as vladk [11:04] rvba: blason added a file to the list of boot loader files, but it's present in syslinux 4 and 6 alike. [11:04] poweroff.com, I think [11:04] Yeah. I think we can live with a bit of duplication here. === vladk is now known as vladk|offline === vladk|offline is now known as vladk === vladk is now known as vladk|offline === vladk|offline is now known as vladk [11:32] Unable to add nodes to ubuntu MAAS server on static vlan | http://askubuntu.com/q/487549 === roadmr_afk is now known as roadmr === roadmr is now known as roadmr_afk [13:00] bigjools: gmb: turns out we *can* use omshell to query the leases. I just tested it and it works. [13:00] rvba: Sweet. === vladk is now known as vladk|offline === jtv_ is now known as jtv === roadmr_afk is now known as roadmr [13:10] jtv: "maasutils" package would be very nice [13:10] jtv: "ensure_dir" is not in maasserver, ugh! [13:12] blake_r: yeah, so far we've mostly had to dump generic helpers in provisioningserver, because provisioningserver can't import from maasserver (but vice versa works). [13:13] jtv: oh yeah forgot, was thinking it was the other way around [13:14] There shouldn't be a need to remember... "maasutils" === vladk|offline is now known as vladk [13:20] blake_r: btw, FYI using a symlink /syslinux/ at the root of the TFTP dir worked. [13:21] rvba: i was looking at that [13:21] rvba: so you created a symlink from current/syslinux -> /usr/lib/syslinux ? [13:21] blake_r: exactly [13:22] blake_r: but syslinux 4 and 6 seem to have different requirements for what *really* needs to be stored at the top level. [13:22] rvba: i thought pxelinux expected it all to be in tftproot [13:22] rvba: does it check both? [13:22] blake_r: syslinux 6 definitely does [13:23] rvba: cool [13:23] rvba: that makes it easy [13:23] Yeah === jfarschman is now known as MilesDenver [14:14] newell_: Hi there. I see you haven't had time to work on the MP I marked as "Needs fixing". I realize what I asked is probably a bit more work than you had anticipated because it implies dealing with forms. But in the long run, it's a worthy investment (for the various reasons I stated in my review); I'm happy to give you a hand with that if you want. [14:35] rvba: lander's tests broke on your branch. :( [14:35] The syslinux one. [14:36] jtv: I saw that. The failure looks completely unrelated. [14:36] Yeah. And I remember that was a function I disliked for some reason. [14:36] I wonder if it's a function I messed with. [14:37] jtv: it's a spurious failure. It passes locally from time to time. [14:38] You mean it _fails_ locally from time to time, I hope? It's been passing for me. [14:39] Not sure, I haven't run it a great number of times. [14:40] But yeah, it seems success is more frequent than failure. [14:40] I bet there's an undetected clash in networks or ranges. [14:40] I'm going to investigate but I'd like to get my syslinux branch landed first. === jfarschman is now known as MilesDenver [14:44] rvba, I haven't worked on that because I was told that there were some other items that were higher priority, so I have been working on those. [14:44] Yes, your help would be appreciated [14:45] Once I have finished the higher priority items that I am currently working on, I will ping you so you we can go over it. [14:45] Thanks for lending a helping hand. [14:48] newell_: okay, cool. [15:03] where in the source is the startup script for bootstrapping a node? [15:07] based on packet captures on both hosts, they're making a bunch of DNS queries that are failing, but based on the hosts they're querying, I'm guessing some of them are supposed to fail? [15:39] Who's up for a code review? https://code.launchpad.net/~jtv/maas/arp-on-failed-entries/+merge/224322 [15:42] blake_r: ^^ [15:42] roaksoax: sure [15:43] blake_r: thanks! [15:52] jtv: done! [15:52] Thanks blake_r! === vladk is now known as vladk|offline [15:52] jtv: np [15:53] You say that, but actually it hasn't even been proven that np isn't the same thing as p. [15:54] Yeah that makes since! But I will keep saying "np"! [15:57] blake_r: rarg, poweroff.com is poweroff.c32 in Utopic's syslinux. [15:57] roaksoax: it should have both [15:58] rvba: ^ [15:58] rvba: poweroff.com at /usr/lib/syslinux [15:58] rvba: from syslinux-common package [15:58] That's not the case. I just installed syslinux-common and syslinux-dev. [15:58] rvba: ugh [15:59] rvba: http://packages.ubuntu.com/search?searchon=contents&keywords=poweroff.com&mode=exactfilename&suite=utopic&arch=any [16:00] blake_r: http://paste.ubuntu.com/7695727/ [16:01] rvba: that is so wierd, guess the packages site is incorrect [16:01] rvba: guess need a symlink from poweroff.com -> poweroff.c32 [16:01] blake_r: indeed. Maybe roaksoax will know how that could happen. [16:01] blake_r: ewww [16:01] rvba: temp[late needs to work fro both trusty and utopic [16:02] blake_r: I know. [16:02] And precise? jtv interjects with a stupid look [16:02] But I don't want to do a symlink if we can avoid it. [16:03] blake_r: I wonder if, with the other symlink I just added it won't just work out of the box if we remove poweroff.com from the list of things we copy over. === roadmr is now known as roadmr_afk [16:04] blake_r: I guess it's worth a shot. I'll test that tomorrow. [16:08] blake_r: are, I didn't realize the template was referencing the file directly… [16:08] s/are/arg/ [16:10] I have question about charms deployment and juju .. If i deploy the same charm , on different machine, (Let's say mysql and rabbitMQ) , how will the deployment work their relation, will the mysql instances be standalone , or are they going to replicate the same data trough all instances ? [16:10] blake_r: isn't there a way to say: "poweroff.com or poweroff.c32"? [16:11] loki27_: you may have more luck in #juju with questions specific to a Juju charm. [16:12] haa great [16:12] (Not saying nobody will know, but I think it's the best chance) [16:13] gmb: looks like you put code in make_interface_settings to leave static_ip_range_{low,high} set to None if the random range has no room for them. [16:13] Guess what happens when you enter None in the dict that goes into a POST? [16:13] jtv: Not exactly; it was to allow for the old (dynamic only) behaviour to be maintained… I’m guessing that that doesn’t work properly :/ [16:14] Right. It blows up, but only on rare runs. [16:14] * jtv doesn't really know what gmb just said, but finds enough common ground to bluff his way through [16:15] These are the things that need documenting in code! [16:15] I thought I had; my apologies :/ [16:15] jtv: Can you paste me a failure? [16:15] Urgh. I think I just lost that shell. Let me reproduce it... [16:16] rvba: I don't think so [16:16] rvba: I will look and see [16:17] Okay, ta. [17:08] rvba: from what I can find, doesn't seem to be anything about handling a missing file or alternatives === CyberJacob|Away is now known as CyberJacob [18:06] strange. When i remote into my VMAAS nodes and issue df -h i dont see a root parition listed [18:06] is this expected behavior? === CyberJacob is now known as CyberJacob|Away [18:31] lazypower: you don't see a root partition? how is your system operating? [18:33] d_`: my sentiments exactly [18:34] let me rebootstrap maas, spin up a machine with juju and pastebin the output for df -h [18:58] http://paste.ubuntu.com/7696549/ [18:58] d_`: sorry for the delay, but thats what i see out of df -h on my maas nodes [18:59] they all have 8gb disks allocated. This is a new one on me where df doesn't report all mounted filesystems. [19:05] lazypower: whats the content of /etc/fstab [19:18] blake_r: LABEL=cloudimg-rootfs / ext4 defaults 0 0 [19:18] that's the only line in there - so is this a byproduct of using some kind of overlayfs? [19:28] lazypower: that is correct [19:28] lazypower: i mean the corrent fstab [19:29] ok [19:29] lazypower: what does blkid, show [19:29] this appears to have changed recently - within the last 4 months. I recall a while back df-h showed the contents of / mounted and its disk space [19:29] blkid shows nothing, returns 0 [19:29] do i need to pass it an arg? [19:29] no [19:30] try "ls /dev | grep sd" [19:30] nothin [19:31] try [19:31] try "ls /dev | grep vd" [19:31] that yields 2 disks. vda device and vda1 [19:31] so your in a physical machine? [19:31] well, disk / partition [19:31] i mean virtual machine [19:31] i'm in a virtual machine. correct [19:31] i'm orchestrating VMAAS. my HOST (metal) is the region/cluster controller [19:32] i'm using kvm w/ about 13 nodes pre-allocated as my maas cluster. [19:32] so vda1 is your "/" [19:32] fdisk /dev/vda [19:32] print [19:32] should show partition, correct? [19:32] correct [19:32] what version of maas? [19:33] apt-cache policy maas [19:33] Installed: 1.5.1+bzr2269-0ubuntu0.1 [19:34] can you view the curtin log, as the node is installing ubuntu? [19:34] i'm not sure whre that lives - i normally open up the VM from VMM and watch the logs there. [19:34] okay, do this [19:35] on the webui of maas, go to the settings page and set the global kernel parameters to this value [19:35] console=ttyS0 [19:35] then "start" a node [19:35] then on your host do "virsh console {name-of-vm}" [19:35] that will show the booting log and curtin output [19:36] right on, neat [19:36] you should do "virsh console", quickly [19:36] yeah i'm not seeing anything. should i see the TFTP output? [19:37] er, the pxe boot sequence [19:41] blake_r: ah, found the problem. I've got the curtin log now. [19:41] lazypower: perfect [19:47] blake_r: am i looking for something specific in here or do i need to pastebin this for you? [19:48] lazypower: oh thought you saw the problem [19:48] lazypower: yeah you can just pastebin the whole thing [19:48] [ 2.565478] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) [19:48] thats teh only thing i see pertinent to what i'm looking for... [19:49] ok let me re run this with output redirected to a logfile [19:54] blake_r: http://paste.ubuntu.com/7696786/ [19:55] wait, i see this here. init bottom says something about tmpfs as the overlayfs [19:55] wat [19:56] thats just for install [19:57] your system reboots [19:57] ah ok [19:57] does this new node have the same problem? [20:01] blake_r: yeah :( [20:02] lazypower: did you change the preseed in anyway [20:03] blake_r: nosir [20:03] lazypower: that is so wierd [20:03] lazypower: i would recommend re importing the boot images [20:03] lazypower: rm -rf /var/lib/maas/boot-resources/current/* [20:03] lazypower: then "maas-import-pxe-files" [20:04] lazypower: just to see if that is it [20:04] already on it. i wiped the entire directory though [20:04] /var/lib/maas/boot-resources [20:04] that should be okay [20:47] blake_r: yeah still seeing the same behavior after reimporting the pxe images [20:48] lazypower: you are deploying precise, can you try to deploy trusty [20:48] strange... my default series is set to trusty [20:49] ah wait, i'm wrong. [20:53] blake_r: mind = blown [20:53] works as expected with trusty [20:53] lazypower: strange precise should work [20:53] lazypower: alteast it is working now [20:54] i mean, it appears to work. the thing that got me was being unable to report disk usage [20:54] i'm deploying a mongodb cluster, and ran out of disk space thanks to preallloc - which is what prompted this discovery session [20:56] might ask in #ubuntu-server they might know an answer for precise [21:00] thanks for the troubleshooting tutorial blake_r. this was informative [21:00] lazypower: np [21:01] i thought i had a good grasp on maas until this :P [21:01] now i realize there's so much more === CyberJacob|Away is now known as CyberJacob === mwhudson-bip is now known as mwhudson === CyberJacob is now known as CyberJacob|Away === jfarschman is now known as MilesDenver === jfarschman is now known as MilesDenver