[19:44] <smoser> powersj: did you test console output ?
[19:45] <smoser> in your ec2
[19:45] <powersj> I added the call to grab it, however as you will note about my comment it isn't always ready to get pulled
[19:45] <smoser> seems like it should return a base64 encoded string
[19:45] <smoser> http://boto3.readthedocs.io/en/latest/reference/services/ec2.html#EC2.Instance.console_output
[19:45] <powersj> the latest commit got it from: https://git.launchpad.net/~powersj/cloud-init/commit/?id=677c60738d026a9bc1272c3b6e2366ce4bdc98f8
[19:45] <smoser> ie, if you *did* get something i think it will be base64
[19:46] <powersj> also the base64 encoding doesn't apply since "If using a command line tool, the tool decodes the output for you."
[19:46] <powersj> when using the aws-cli == command line tool
[19:47] <smoser> right
[19:47] <smoser> you're *not* using it
[19:47] <smoser> so you need to decode
[19:47] <powersj> or using the python library seems to cover that as well
[19:47] <smoser> that seems wrong
[19:47] <powersj> same for user-data... we don't encode it
[19:47] <smoser> https://eucalyptus.atlassian.net/browse/TOOLS-407
[19:48] <smoser> err.. the redux
[19:48] <smoser>  https://eucalyptus.atlassian.net/browse/TOOLS-675?attachmentViewMode=gallery
[19:49] <smoser> basically, though, i'd expect that the content that went to the console of the device is not necessarily anything that be represented losslessly as a string
[19:49] <smoser> it is bytes
[19:49] <smoser> so if ec2 did an encode to utf8, that'd be lossy
[19:51] <powersj> smoser: your comments stem from my use of .encode() on the output?
[19:52] <smoser> have you tested this?
[19:52] <powersj> yes ran it through all the tests, only one fails and we need to fix that test
[19:52] <powersj> let me try again in case I just never got any output at all
[19:52] <smoser> because i think what you will be '.encode()'ing is b'aGVsbG8gd29ybGQK'
[19:53] <smoser> so sure, it will work, buit the console log will be base64 encoded string
[19:53] <smoser> er,...bytes.
[19:54] <smoser> i have to run, i'll take a further look later.
[19:59] <powersj> smoser: it produces a string
[19:59] <powersj> https://paste.ubuntu.com/26321229/
[20:00] <powersj> here is also an example of no output: https://paste.ubuntu.com/26321219/
[20:03] <powersj> here is part of the raw string: https://paste.ubuntu.com/26321243/
[20:07] <blackboxsw> powersj minor comments on my last pass of your ec2 branch posted
[20:07] <blackboxsw> approved on my end once those nits are reviewed and handled or rejected by you
[20:09] <powersj> blackboxsw: thank you, I'm +1 on all of those
[20:12] <dojordan_> hey smoser, I'm working on using url_helper instead of raw requests, but I need infinite retry support so I was planning on using max_wait==None. I wanted to make sure that is okay, as well as wanted to point out that right now, using the default parameter of None will trigger a somewhat unexpected exception
[20:13] <dojordan_> def timeup(max_wait, start_time): return ((max_wait <= 0 or max_wait is None) or (time.time() - start_time > max_wait))
[20:13] <dojordan_> the problem is the short circuit check for a None parameter should happen before the none to int comparison. (this is in url_helper)
[20:15] <powersj> smoser: in addition to the above, take a look at: https://paste.ubuntu.com/26321285/
[20:39] <powersj> blackboxsw: changes pushed
[20:39] <powersj> spelling fix, docstring format, try/except change, and more details in debug messages
[20:42] <blackboxsw> powersj: looks good, just saw another highlighted in your latest changest.   'deleting secuirty' in tests/cloud_tests/platforms/ec2/platform.py
[20:42] <blackboxsw> all good otherwise.
[20:44] <powersj> blackboxsw: thx are you using a spell checker of some sort?
[20:45] <blackboxsw> I have two of them, and no prescription glasses ;)
[20:45] <powersj> lol
[20:45] <powersj> mine don't work very well
[20:45] <blackboxsw> nope just reading through things generally. that's why I missed the typos first time around (I should use a spell checker) ispell or something
[21:01] <powersj> blackboxsw: https://paste.ubuntu.com/26321529/ may have some test escapes again
[21:01] <powersj> that was on my local branch, let me see if the nightly run shows the same
[21:02] <powersj> actually ignore me... I am looking at the decimal wrong doh
[21:02] <powersj> coffee time
[21:02] <blackboxsw> powersj: hrm, yeah that is taking a bit long..... I thought when we saw bad escapes we'd see > 1 second (but that was probably with retries etc)
[21:02] <powersj> yeah I saw that and went 5 seconds?!
[21:02] <powersj> but it isn't 5 seconds :P
[21:04] <blackboxsw> powersj: yeah I went on a date to zquila over break and had a decimal place issue. we almost ordered 2 margs that were $75 each. thank goodness the waitress asked us to double check
[21:04] <powersj> hahaha
[21:05] <blackboxsw> I can't believe enough people would actually order a $75 marg to warrant putting something like that on the menu
[21:06] <blackboxsw> #wrongchannel
[21:06] <blackboxsw> :)
[21:09] <blkadder> LOL
[21:58] <smoser> powersj: then the bug is in boto3 :-(
[21:58] <powersj> bug?
[21:58] <smoser> i'll file it upstream.
[21:59] <powersj> you believe it should return bytes over a string?
[21:59] <smoser> it is taking a base64 encoded blob and decoding it into a string
[21:59] <smoser> and losing data along the way
[22:01] <powersj> https://github.com/boto/botocore/blob/develop/botocore/handlers.py#L142
[22:01] <smoser> yeah, thats the bug :)
[22:08] <smoser> https://github.com/boto/botocore/issues/1351
[22:08] <smoser> thanks for digging, powersj
[22:09] <powersj> smoser: heh thanks for calling it out :)
[22:09] <powersj> I am always amazed by your ability to pull out pugs from years ago heh :P
[22:09] <smoser> firefox history
[22:10] <smoser> i think euca must not use boto3
[22:10] <smoser> otherwise it would never have seen the bug that i reported (twice)
[22:10] <smoser> i  know it didn't origintally use boto3, but i figured it might have moved by now. but the project is kind of dead now.
[22:13] <smoser> dojordan_: yes, that seems sane. 0 might also make sense.
[22:14] <smoser> powersj: fwiw i'm not just making this stuff up. the data that systemd writes to /dev/console is almost all the time *not* utf-8 during a boot cycle on ubuntu at least.
[22:14] <smoser> i only know this because of tools that assumed it was not working :)
[22:15] <powersj> ok good to know, my quick look at the data made me go looks like normal output and move on
[22:27] <smoser> powersj: i hit 'submit' on review
[22:27] <smoser> i think i might be missing something though... i just went through my last round of comments, and it didn't look like you addressed many of them.
[22:29] <powersj> smoser: hmm thought I did
[22:29] <powersj> I'll look
[22:40] <powersj> smoser: looks like I misunderstood your comment about snapshots, otherwise hit all your previous requests
[22:40] <powersj> I'll get these fixes done
[23:40] <powersj> smoser: pushed updates, with a question on the review