[04:53] <jtv> Ohhhh WHAT?  Test failure while landing?  I tested locally!
[04:53] <jtv> Rabbit.
[04:54] <jtv> maasserver.tests.test_rabbit.TestRabbitBase.test_base_channel_creates_exchange
[04:54] <jtv> Timeout.
[04:54] <jtv> bigjools: is that related to what you just debugged with Scott?
[04:54] <bigjools> jtv: no
[04:54] <bigjools> random
[04:54] <bigjools> not seen that for a long time
[04:55] <jtv>                     {cannot_listen,{127,0,0,1},59471,eaddrinuse}}},
[04:55] <jtv> That seems to be the nub of it.
[04:55] <bigjools> yeah
[04:55] <bigjools> nub
[04:55] <jtv> I guess it was just an unlucky choice of port... retrying.
[04:58] <jtv> Oh dear. And an error in simplestreams.
[05:20] <jtv> bigjools: should I backport my fix for the inscrutable-import-error bug to 1.5?  Or is there no point now?
[05:20] <bigjools> jtv: no point IMO
[05:21] <jtv> OK
[05:21] <bigjools> downgrade the bug from panic status
[05:23] <jtv> Done.
[05:50] <jtv> That error in the simplestreams went away, but now I'm getting one about an unexpected sha256 checksum.
[05:51] <bigjools> jtv: I expect the mirror might still be running
[05:52] <bigjools> which prompts another question ...
[06:12] <jtv> Same exception again.  That makes it hard to do full Q/A.
[06:15] <bigjools> jtv: worked here
[06:15] <jtv> hmmm
[06:15] <bigjools> if we have a label of "rc" is anything going to bork?
[06:16] <bigjools> IOW did we rely on  "release" anywhere?
[06:29] <rvba> bigjools: I think it will work just fine.
[06:29] <rvba> We were using 'daily' before.
[06:29]  * bigjools likes the optimism
[06:31] <rvba> bigjools: the node failed to boot in the lab.
[06:31] <rvba> :/
[06:31] <bigjools> shit
[06:31] <rvba> 'No such image'
[06:31]  * rvba investigates
[06:31] <rvba> …
[06:31] <bigjools> rvba: is it hard-coded anywhere?
[06:31] <bigjools> in the tests
[06:32] <rvba> bigjools: no, the test just want an image for Trusty.  Again, it was working with the 'daily' images.
[06:32] <rvba> bigjools: no Trusty image was imported.
[06:32] <rvba> I just see the precise images.
[06:32] <bigjools> rvba: are the tests using the default config?
[06:32]  * rvba checks
[06:32] <bigjools> I imported trusty "rc" ok on a new install
[06:34] <rvba> yes (http://paste.ubuntu.com/7233732/ that's the config from the lab)
[06:35] <rvba> /var/lib/maas/boot-resources/ contains only the precise images.
[06:37] <rvba> bigjools: I don't see any reference to 'rc' images in http://maas.ubuntu.com/images/ephemeral-v2/releases/streams/v1/com.ubuntu.maas:v2:download.json
[06:37] <rvba> What am I missing?
[06:37] <bigjools> WFM, so, WTH
[06:38]  * rvba tries on canonistack
[06:38] <bigjools> could be a mirroring problem
[06:38] <jtv> I do see "rc" in the sjson.
[06:39] <jtv> Oh, that's in the regular json too.
[06:39] <bigjools> jtv: I don't see it!
[06:39] <bigjools>  "updated": "Fri, 11 Apr 2014 00:45:23 +0000",
[06:39] <rvba> bigjools: arg, I was looking at a ached version of the json
[06:39] <rvba> Now I see it too
[06:39] <rvba> cached*
[06:40] <rvba> bigjools: maybe that's what happened in the lab too
[06:40] <jtv> Science: “I'll believe it when I see it.”  Religion: “I'll see it when I believe it.”
[06:40] <jtv> Sounds as if maybe the caching headers on the index are set to cache too aggressively.
[06:40] <rvba>  "updated": "Fri, 11 Apr 2014 00:45:23 +0000",
[06:40] <rvba>  "products": {
[06:40] <rvba>   "com.ubuntu.maas:v2:boot:14.04:armhf:generic-lpae": {
[06:40] <rvba>    "label": "rc",
[06:40] <jtv> Maybe the http headers were designed for etags, then had the etags removed or not implemented?
[06:40] <bigjools> oh see it now
[06:41] <bigjools> rvba: the lab will work once it can import the right image then :)
[06:41] <rvba> bigjools: trying to clear the cache now…
[06:41] <rvba> The cached data in the proxy that is.
[06:45] <jtv> I got a checksum error, but disabling imports of the "rc" label seems to get me past it.
[06:51] <rvba> bigjools: my run on a canonistack machine imported the 'rc' images all right. (Same as what you saw.)
[06:52] <bigjools> jtv, rvba: I suspect proxies are screwing things over
[06:52] <rvba> Yep
[06:52] <rvba> Investigating that now
[06:55] <rvba> Arg, I don't have permission to clear the cache on the proxy machine.
[06:57] <bigjools> rvba: I do know that it worked in the Garage MAAS come to think of it
[06:58] <bigjools> not this exact package but the rc data worked IIRC
[06:58] <rvba> Okay
[06:58] <rvba> Would be good to get an end-to-end test to pass though.
[06:59] <rvba> But I'm not too worried.
[07:04] <bigjools> k
[07:05] <jtv> I would suspect the configuration or the original httpd more than the proxies.
[07:06] <jtv> For example, if the directory with the index is set up for aggressive caching because they look like they only have static files...
[07:07] <rvba> What's weird is that when I wget the index in the lab (using the same proxy the test script uses), I get the right index with the references to the 'rc' images.
[07:11] <jtv> Well, the headers from the httpd say what caching is _allowed_.  After that, every proxy or client gets to decide to what extent it wants to use that.
[07:11]  * jtv hates the quagmire of over-aggressive proxies
[07:11] <jtv> There is no more objective reality in that situation.
[07:16] <rvba> Running the int. tests manually…
[07:22] <rvba> If maas-import-pxe-files was a bit more verbose, that would help…
[07:44] <rvba> bigjools: importing the images in the lab without going through the proxy.  It's going to take time, but we will have our answer.
[07:44] <bigjools> rvba: ok
[07:45] <bigjools> rvba: need to get the proxy fixed then if it's aggressively caching like that
[07:45] <rvba> Yeah
[07:45] <rvba> I'll need Diogo to help me with that cause I don't have permissions to manipulate the proxy.
[09:08] <rvba> Import done.
[09:08] <rvba> 'rc' images are there.
[09:09] <rvba> Running the rest of the test now…
[09:28] <rvba> allenap: time for a tiny review? https://code.launchpad.net/~rvb/maas/show-version-doc/+merge/215377
[09:29] <gmb> rvba, allenap: How can I find out why maas-dhcp-server won’t start for me? `sudo service maas-dhcp-server-start` just give me “maas-dhcp-server stop/waiting”.
[09:29] <rvba> gmb: upstart log/syslog
[09:30] <gmb> Gah!
[09:30] <gmb> rvba: Thanks. Forgot about upstart log. syslog is silent on the issue.
[09:30] <rvba> gmb: the question you just asked should go into the FAQ.
[09:30] <gmb> Hmm: “/var/lib/maas/dhcpd-interfaces does not exist.  Aborting”
[09:30] <gmb> rvba: Agreed.
[09:30] <gmb> Will take care of it.
[09:31] <rvba> Great, thanks.
[09:33] <allenap> rvba: How about a bzipped normal review? Ha. Now I’m scraping the bottom of the barrel in a different way :)
[09:41] <rvba> allenap: thanks for the review
[13:19] <rvba> allenap: gmb: could one of you have a look at bug 1306303 please?  I'm still trying to figure out this proxy problem.
[13:34] <allenap> rvba: I might not be able to do it until much later, but sure.
[14:58] <dimitern> allenap, ping
[15:45] <rvba> gmb: found another gaping hole in the documentation (since you seem to be in a writing frenzy): the commissioning phase doesn't seem to be described anywhere.  Nothing tells you that you have access to the commissioning scripts' results for instance.
[15:49] <d_`> Hi MAAS people, I really like the idea of MAAS, but I already have chef instead of juju. what is the relative difficulty of using not juju to configure servers when deploying with MAAS?
[15:50] <rvba> d_`: juju is just a client of MAAS.  MAAS is pretty agnostic in this regard.  It just provisions servers and hands them over to clients.
[15:51] <rvba> d_`: in other words, you shouldn't have trouble using Chef with MAAS.
[15:51] <d_`> rvba: thanks for the quick reply, that's really encouraging. I think I'm going to set up a lab today
[15:53] <d_`> rvba: since you have a canonical masked host, do you know what's up with the 14.04 RC? why it's not out?
[15:56] <rvba> d_`: no idea, sorry.
[15:57] <rvba> d_`: you might want to ask around in #ubuntu-release.
[15:58] <d_`> rvba: haha, no problem. thanks for the quick answers