[00:00] <wallyworld_> the error isn't tools related
[00:00] <thumper> wallyworld_: sorry, was lazy helping as I'm eating :)
[00:00] <wallyworld_> np:-)
[00:01] <sinzui> noodles775, beuno then upload the images and streams to a location in that region
[00:02] <noodles775> OK, I've got a bit of learning to do there. sinzui if there's any docs outline the process, let me know. Thanks.
[00:02] <sinzui> noodles775, beuno: this article covers some of the details, but it would be nice to find a location in that region that provides the streams for free: https://juju.ubuntu.com/docs/howto-privatecloud.html
[00:02] <noodles775> Also, how can I check which juju version was the first to include a goamz with cn-north-1? I see it's in current, but it's not in the juju installed on a deployment machine (which I'm trying to get updated, but is currently on 1.18.4)
[00:03] <noodles775> Great, thanks sinzui
[00:03] <sinzui> noodles775, juju 1.20.0 had cn knowledge
[00:04] <sinzui> noodles775, you can still the tools/agents parts of that doc to get running. --upload-tools will get you an env to play with.
[00:09] <sinzui> s/still/skip/
[00:16] <noodles775> sinzui: thanks. So, just checking my understanding, the issue is because cn-north-1 isn't included here right? http://cloud-images.ubuntu.com/daily/server/trusty/current/
[00:16] <sinzui> noodles775, try this in environment.yaml: image-metadata-url: https://cloud-images.ubuntu.com/releases
[00:17] <sinzui> noodles775, the url we see in your paste is for daily, and there aren't and cn in that stream. the released stream lists on cn
[00:17] <noodles775> sinzui: right, I was specifically using daily in the environments.yaml, let me switch that and try first, then I'll try with the image-metadata-url.
[00:17] <sinzui> noodles775, I am not sure about the url. I think juju will see the streams/ dir below that path and work
[00:21] <noodles775> sinzui: here's the output when not using the daily stream: https://pastebin.canonical.com/124270/ - I assume that means I don't need to try with the image-metadata-url (as it seems to be using /releases)?
[00:23] <sinzui> noodles775, right, that is the default url, but I see we do have cn in the file http://cloud-images.ubuntu.com/releases/streams/v1/index.json
[00:25] <sinzui> noodles775, ah, the url is slightly different https://ec2.cn-north-1.amazonaws.com.cn in the paste != to https://ec2.cn-north-1.amazonaws.com.cn in the json
[00:26] <sinzui> Not sure why juju isn't looking at https://ec2.cn-north-1.amazonaws-cn.com.cn as specified in the json
[00:26] <noodles775> sinzui: right, good spot. (s/amazonaws/amazonaws-cn).
[00:32] <sinzui> noodles775, I wonder if that region is behind a caching proxy. Your env is reading stale data.
[00:32] <sinzui> noodles775, if you are in that region, can you curl http://cloud-images.ubuntu.com/releases/streams/v1/index.json to verify the index states it was made  26 Jan 2015
[00:35] <noodles775> sinzui: I'm not, but I migth be able to ask someone who is.
[00:37] <noodles775> sinzui: do you know that an old version did in fact have the incorrect url that we're seeing? (I've pinged nessita, but might not hear straight back)
[00:39] <sinzui> noodles775, I don't. I don't know much here. I am applying my experience of missing agents to missing images. Stale caching is often the cause of urls that don't match the recently published data
[00:45] <noodles775> sinzui: seems correct from cn also: https://pastebin.canonical.com/124272/ . Note, I'm bootstrapping on a canonistack instance in the DC, and I can confirm that it also sees the correct url. So I'm not sure where the amazonaws.com.cn is coming from?
[00:46] <noodles775> s/on a/from a/
[00:47] <sinzui> noodles775, that is the file I was looking at
[00:50] <noodles775> sinzui: actually, - looks like it's goamz https://pastebin.canonical.com/124273/
[00:50] <noodles775> which would mean a fix would require a new juju version :/ (if that's right)
[00:50] <sinzui> :(
[00:50] <sinzui> noodles775, yep
[00:51] <noodles775> sinzui: do you want me to create a bug (guessing you can probably create something more specific, but happy to create it if you're ready to EOD or whatever)
[00:52] <sinzui> noodles775, you can report the bug. Include when this needs to be fixed. that will help determine if we change 1.21.1 or 1.22.0
[00:53] <noodles775> sinzui: We're trying to deploy to cn-north-1 now, so I'm guessing beuno will say asap. Creating the bug now.
[00:53] <sinzui> noodles775, thank you. I will bring the bug to the various parties
[00:54] <hloeung> can we backport the fix to the juju version in Trusty as well?
[00:55] <noodles775> sinzui: actually, we should first check whether the url we think is the incorrect one actually is (ie. could it be that the goamz one is correct, while the stream is not - I've not checked)
[00:56] <sinzui> noodles775, good point. I think #cloudware and utlemming are the experts. I think they will say their streams are correct and the code from last year is wrong
[00:57] <noodles775> sinzui: OK, I'll just note that on the git bug.
[00:57] <noodles775> sinzui: did you see hloeung's question above? (as he's providing the juju version which we're using)
[01:03] <noodles775> sinzui: actually, hloeung just pointed out that amazonaws.com.cn resolves, while amazonaws-cn.com.cn does not - so I'm hopeful it is the stream :)
[01:04] <hloeung> yeah, it looks like it's just the stream - "endpoint": "https://ec2.cn-north-1.amazonaws-cn.com.cn"
[01:04] <hloeung> $ host ec2.cn-north-1.amazonaws-cn.com.cn
[01:04] <hloeung> Host ec2.cn-north-1.amazonaws-cn.com.cn not found: 3(NXDOMAIN)
[01:05] <hloeung> "com.ubuntu.cloud:released:aws-cn": { "updated": "Wed, 10 Dec 2014 20:45:42 +0000",
[01:05] <hloeung> taken from http://cloud-images.ubuntu.com/releases/streams/v1/index.json
[01:07] <sinzui> noodles775, okay, his is good news. I think we can contrive our own streams using what already exists
[01:08] <noodles775> sinzui: OK, I'd just created the bug, but feel free to mark it invalid (or git equiv) https://github.com/go-amz/amz/issues/21
[01:08] <hloeung> sinzui: I checked all the other endpoints, it seems us-gov-west-1 needs fixing as well
[01:08] <hloeung>      "endpoint": "https://ec2.us-gov-west-1.amazonaws-govcloud.com"
[01:09] <noodles775> sinzui: How often are the streams updated?
[01:09] <sinzui> noodles775, still a bug since we are talking about work arounds. We can copy just a few files from cloud-images, the correct the urls.
[01:09] <hloeung> sinzui: should be ec2.us-gov-west-1.amazonaws.com
[01:09] <sinzui> noodles775, I cannot speak for image streams. I think they are updated every time a package in the image changes or a new region is added
[01:10] <sinzui> hloeung, I'll take your word for that. I can claim to be an expert on juju agent streams, but not cloud images
[01:11] <noodles775> sinzui: How is it a bug in the goamz code though? I thought the proper fix would be for the stream to be fixed (by the people you mentioned earlier)? (ie. not trying to get you to do something, just wondering what the correct fix is here)
[01:11] <hloeung> sinzui: heh, I'm just going with what's in goamz (https://github.com/goamz/goamz/blob/master/aws/regions.go) and that seems to resolve
[01:12] <sinzui> noodles775, as I said. I don't know anything about image streams, but I know the people who test in china are the authors of the streams. I don't know of anyone using goawz to test cn
[01:16] <noodles775> sinzui: Great, are you able to ping/contact those people about the bug then? (I'm assuming they're using the same juju with goamz compiled in).
[01:17] <sinzui> noodles775, I am sure they are NOT
[01:17] <sinzui> noodles775, cloudware makes os images. I test juju. I don't work with them and vice versa.
[01:18] <sinzui> So I am not surprised juju doesn't just work
[01:18] <noodles775> sinzui: Sorry, I misunderstood - I thought they were testing juju in china.
[10:10] <Tug> Hey, I just read that Google Cloud Platform was an ubuntu partner http://partners.ubuntu.com/partner-programmes/public-cloud
[10:11] <Tug> Do you think we're going to have a google cloud provider for juju in the future ?
[10:48] <dimitern> Tug, yes, it's under development and should be ready soon
[10:50] <Tug> dimitern, cool Thank you
[15:05] <hazmat> Tug, there's a merge proposal for extant re gce
[17:43] <mwak> hello
[17:59] <marcoceppi> hey mwak, how's it going?
[18:00] <mwak> good and you marcoceppi
[18:01] <marcoceppi> doing great
[18:05] <mwak> :)
[18:25] <mattrae> hi, i'm using juju 1.20.14. it appears that when doing debug-hooks and using 'exit 1' to pause the queue does not work. it moves on to the next queued hook and clears the hook error. this is making debug-hooks mostly useless
[18:26] <mattrae> and we may have to redeploy because i don't know if i can retrigger the same hook that was incorrectly resolved
[18:26] <mattrae> is there any version of juju that fixes the 'exit 1' debug-hooks issue, or any way to retrigger the hook that was originally in error state?
[19:04] <drbidwell> How do I find out why a "juju deploy" stays in a pending state?  I can ssh to the host as ubuntu@host and I can do "juju ssh 1" .  The auth.log doesn't show any attempts to log in and I don't find anything in any of the /var/log/juju/ logs.
[19:04] <lazyPower|Travel> drbidwell: what provider?
[19:05] <drbidwell> lazyPower|Travel: maas on ubuntu 14.04.1 using maas 1.7.1 rc4 and juju 1.20.14
[19:06] <lazyPower|Travel> drbidwell: can the unit communicate with the bootstrap node? thats typcally why a unit will stay in pending and not come online in my experience with maas.
[19:06] <lazyPower|Travel> drbidwell: best guess is to ssh into that pending unit and attempt to ping the ip of the bootstrap node an ensure there is connectivity
[19:07] <drbidwell> lazyPower|Travel: I can ping from the boostrap node to the target node
[19:07] <Guest75565> hi
[19:08] <drbidwell>  lazyPower|Travel: I can ping from the target to the bootstrap node also
[19:09] <Guest75565> has anyone come across this error "getInstanceNetworkInterfaces failed: invalid hardware information" while deploying a charm
[19:09] <lazyPower|Travel> drbidwell: hmm seems like its in order then
[19:09] <Guest75565> i am running juju and maas as a vm
[19:10]  * lazyPower|Travel nods
[19:10] <lazyPower|Travel> thats how i got started with maas too - however at the moment i dont have any other ideas off the top of my head and I'm about to board a flight
[19:11] <lazyPower|Travel> drbidwell: i'll be back around in just over an hour, i'll follow up if you're still here
[21:58] <noodles775> axw, sinzui, utlemming : I've just retried bootstrapping for cn-china-1, but it fails with a different error. I've updated the issue with a (private) paste, but don't see an option to reopen the bug: https://github.com/go-amz/amz/issues/21
[21:58] <noodles775> hrm, maybe it should be a separate issue anyway, as the endpoints now match.
[21:59] <utlemming> noodles775: yeah, that is a seperate issue...and may be related to the Great Firewall of China. Unless you are playing in China...yeah, good luck with that.
[21:59] <utlemming> (and I am only have kidding)
[22:02] <sinzui> noodles775, I think you need to be in china to use the region. it is for Chinese businesses
[22:03] <sinzui> noodles775, Juju QA requires  Chinese host, preferably one we can setup a jenkins slave to test stream health.
[22:05] <noodles775> sinzui: I don't think that's the case for aws itself (but will check by seeing if I can launch instances there sans juju), but it certainly looks like we'll need a deployment host there from which to manage deployments :/ I'll try.
[22:06] <sinzui> noodles775, that is good phrasing of what Juju QA wants.
[22:12] <noodles775> utlemming: Have you confirmed that `juju bootstrap` on cn-north-1 works there with the updated streams? (I don't want to go down that path unless we know it is indeed working)