/srv/irclogs.ubuntu.com/2016/05/19/#juju-dev.txt

anastasiamac_davecheney: m a bit concerned with the work that you are doing in returning *all* errors that hav previously been logged00:10
anastasiamac_davecheney: what is the bug that set u on this warpath?00:10
anastasiamac_davecheney: for example, m not convinced that callback failure that meant to set the status should really fail the whole operation of container creation00:12
anastasiamac_davecheney: m sure that there are other instances where encounting error should not affect the whole operation. m specifically thinking of simplestreams but thre are others...00:13
davecheney                if err != nil {00:21
davecheney                        logger.Warningf("FIXME: ignoring unexpected CIDR format %q: %v", netConfig.CIDR, err)00:21
davecheneyanastasiamac_: here is my philosophy00:21
davecheneyif an error occurs one of two things happen00:22
davecheneya. the error means you cannot continue processing because you are missing the requireed data00:22
davecheneyb. the error does not stop processing00:22
davecheneyif a, then we don't want to log the error asynchrnously into a log file, we want to return it to the caller synchronusly00:22
davecheneyif b, then it isn't an error because we can continue00:23
davecheneyanastasiamac_: can you point out a specific part of the change you are talking about00:23
anastasiamac_davecheney: for example,  container/lxc/lxc.go, line 257... this now means that if u could not successfully callback to set the status, you'd fail container creation00:28
anastasiamac_davecheney: m looking at what u've committed yesterday... and m a bit puzzled... some of the areas did not seem like they warrented to throw an error... and I recall considerable amount of logic in some areas where it was ok to get an error and continue (ur b)00:31
anastasiamac_so we'd log error at another level (like warning or info) and keep going..because it's not significant enought to do anything about but might be of interest during debug00:31
davecheneyanastasiamac_: this line ? https://github.com/juju/juju/commit/19f0dbb069916c83007b3f7293e079cb5e450fa4#diff-edf2b984c531037601cf518c0a0b039eL25600:33
davecheneyif we can keep going then it isn't an error00:33
anastasiamac_davecheney: yes, this line. that's the point.. it's not a show-stopper. it's might be useful for when you are diagnosing a field failure - what has happenned leading to my failure - but it's not signifiant enough to stop the operation...00:36
anastasiamac_we have millions of places like this00:36
davecheneyyes, and they all need to be fixed00:38
davecheneyhere is my rule of thumb00:38
davecheneyno warnings, ever00:38
anastasiamac_a call to some function may fail but we either have other means of achieving what we want or we don't really care enough00:38
davecheneyit's either something the user needs to know about because they need to fix it, or we don't say anything00:38
davecheneyso, if we don't care if it succeeds or fails,00:38
davecheney1. why do we call it00:38
davecheney2. why do we log the result00:39
anastasiamac_logs are not jsut for user.. they are also for diagnosing field failures... warning means "this fails, we may need to worry about it if something else goes worng or at a later stage or <insert condition here>"00:40
davecheneyno, i'm sorry, my policy on warnings is absolute00:41
davecheneyno warnings, ever00:41
davecheneyit's either an error or an info message00:41
anastasiamac_i do think that this should not be tackled at a global level and cetainly not without further discussion00:41
davecheneyi don't think the ideas of not having warnings is contraversial00:41
davecheneyi'll send a PR to fix lxc/lxc.go00:41
anastasiamac_sure... m not talking about log levels really \o/ my concern is that you are returning errors where they should not be propagated00:42
anastasiamac_plz consider all other places where u have started throwing erros too \o/ u'd know best what was touched :D00:42
davecheneywhat are you asking ?00:43
davecheneythe code was reviewed, all the tests pass, juju.fail is clear00:43
anastasiamac_i there are any other places similar to https://github.com/juju/juju/commit/19f0dbb069916c83007b3f7293e079cb5e450fa4#diff-edf2b984c531037601cf518c0a0b039eL25600:43
davecheneythis sounds like fear that something that we don't know about might break because we don't know about it because its scarey adn we don't trust out code review or our tests00:44
anastasiamac_plz revert them whenevr possible.. (i dunno if there are - i trust u to know) \o/00:44
anastasiamac_as for log levels, there is a team meeting tonight at our 8pm (well, bne at least)00:44
anastasiamac_let's discuss it then \o/00:44
anastasiamac_davecheney: i agree that there mayb some places where we are swallowing errors..but i'd like these to be fixed on a per-need basis, prefereably with a bug backing the need00:45
davecheneyanastasiamac_: https://github.com/juju/juju/pull/543200:46
anastasiamac_davecheney: FWIW, all the places where we appear to swallow an error need to have a clear explanation as to why...00:58
davecheneyi agree00:59
anastasiamac_davecheney: i have seen instances where these are explained in the review comments... but i believe explanantion needs to be in the codebase00:59
anastasiamac_davecheney: \o/00:59
davecheneybut stil, don't log errors at warning level00:59
anastasiamac_davecheney: plz come to team meeting - let's talk :D00:59
davecheneyanastasiamac_: all the decision makers will be asleep01:00
davecheneythey are all in the US timezone01:00
anastasiamac_davecheney: i don't think it requires executive decision :D just ocnsensus and europe will b online01:01
davecheneywhy do we need to have a call01:01
anastasiamac_bleh.. consensus :D01:01
davecheneycan't we just talk about it now01:01
davecheneyor send an email ?01:01
anastasiamac_email is just as good but f-2f may be more efficient01:02
anastasiamac_personal preference, for majority of cases - errors should b logged at error-level01:03
anastasiamac_i just can;t think of a places where it would not need to.. and don;t want to omit someplaces that other ppl mayb aware of :D01:03
davecheneyhave a look at the branches i've been landing01:05
davecheneymost of this is s/Warningf/Errorf01:05
davecheneywith the exceptoins where we do something like01:06
davecheneyWarningf("ignore error, carrying on anyway")01:06
davecheneyin whcih case that is debugging information because there is nothing a user can do with this knowledge01:06
davecheneythat's just for us developers01:06
anastasiamac_well, I think that as long as there is a good explanation in the comment and in the log message as to why an error is ignored, whether it it's logged t Warning or Error level is less important... Explanation is what really matters :D01:14
natefinchFWIW... I hate errors in logs01:14
natefinchunless it's a showstopper01:15
anastasiamac_natefinch: warning might be better as we aer not treating it as an error ;D01:15
anastasiamac_agreed \o/ all errors proper should be logged at error-level01:15
natefinchnope, I'm with Dave on that one.  Warnings are dumb.  Debug, Info, Error, and error should almost never be used01:15
natefinchthe problem is that, to anyone who is not the specific developer who wrote that warning/error log... it's impossible to know if an ERROR is really an ERROR, or just "hey, something went wrong, that's ok"01:17
anastasiamac_yeah :D i bbelieve that all traffic lights should loose yellow - u other stop or u go :)01:17
davecheneyanastasiamac_: there is no need to be facetious01:17
davecheneyhere is my rule of thub01:17
natefinchyellow doesn't mean "maybe it's ok, maybe it's not"01:17
natefinchbut that's what warning is01:17
davecheneytell the user if there is something they need to do01:17
anastasiamac_warning-level is like yeelow :D01:17
davecheneysorry, no, that wasn't right01:17
anastasiamac_m not being f** :D01:18
davecheneytell the user if there is something they can do01:18
davecheneyif there is nothing the user can do, then don't tell them01:18
davecheneyeither carry on, or the program quits01:18
davecheneyanastasiamac_: for reference, http://dave.cheney.net/2015/11/05/lets-talk-about-logging01:19
anastasiamac_davecheney: m not convinced that in ideal world user should need to look in the logs (only when something went horribly wrong, the logs should be trotted)01:19
anastasiamac_user should receive an intelegent feedback..01:19
natefinchthe logs are for developers01:19
natefinchor extreme power users01:19
anastasiamac_+100 \o/01:20
davecheneyanastasiamac_: logs are for develoeprs01:20
davecheneythe return value of a function is for the user01:20
anastasiamac_davecheney: i hope u mean return value of an operation... our functions should not matter to user eaither01:21
davecheneyfunction/method/opertation, what'ever works for you01:21
anastasiamac_davecheney: granularity is different01:22
davecheneyi disagree01:22
anastasiamac_if as auser I want to add a machine, i don't care if u r calling 100 functions underneath (and some of the functions may not make sense to me)01:23
davecheneyi agree01:23
davecheneythat is what I meant01:23
davecheneyan operation either worked or it didn't01:24
davecheneyif it didn't it returns an error01:24
anastasiamac_\o/ m glad we agree01:24
davecheneyit should not fail, but rturn no error while poopping out some warning messages on the side01:24
davecheneythis is what I am working on fixing at the moment01:25
natefinchlogging at ERROR level is confusing to *developers* though.  Does it mean juju is in a bad state?  What *does* it mean?  If we can ignore it, it must not really be that much of an error.  Warning is just as bad.01:27
natefinchWARNING juju.worker.uniter upgrade126.go:26 no uniter state file found for unit unit-resource-0, skipping uniter upgrade step01:27
natefinchis my juju broken? is it a bug?01:27
natefinchI presume that is actually a perfectly fine state for juju to be in... but I don't know01:28
anastasiamac_i think it's ok to say "warning: blah blah did not happen, will try bleh".. and as a user m ok to see these go by :D01:28
anastasiamac_it's kind of like saying "my 1st preference is this but it did not happe, i'll try my 2nd beest"01:29
davecheneynatefinch: exactly01:29
anastasiamac_(without the typos)01:29
davecheneyanastasiamac_: so in that case that's just debugging information01:29
anastasiamac_not necessarily dbugging (althou it can be).. it's more of an operation progress :D01:31
a123I'm seeing something very odd with juju2.0 while installing charms. A day ago, my charms were installing fine, then today one is failing in the cloud-init section when trying to run jujud-machine-xx. After connecting to the machine and running initctl reload-configuration, I can now start the job. Otherwise, the error is 'Unknown job' Any ideas?01:33
davecheneyanastasiamac_: here is my rule of thumb01:35
davecheneywhen reading a log file, if you every reach for grep -v to filer out "operating in progress" informational messges01:35
a123As far as I can see, the logs are identical between the charms which worked(ie. the cloud-init-output.log) and the one that doesn't. It appears upstart doesn't recognize the script.01:35
davecheneythey shouldn't be logged in the first place01:35
davecheneyevery single line reported to the user must be something actionable01:35
davecheney"i'm still working on this" is not actionable01:35
natefinchman, so much of our logging is utter crap.  I mind it less when it's at debug level, because, well, it can be useful for debugging.  But info should be something at least useful.  important changes to state, etc.01:37
redirI think warnings are reasonable for broadcasting information such as API deprecation or change when such an API or other such announcements. However, retries (my first pref didn't happen trying second pref) are a functioning codepath and shouldn't be logged to the end user IMO.01:57
davecheneyredir: no, sorry, no warnings ever01:57
mupBug #1583274 changed: Openstack base bundle 42 fails to start rabbitmq when deployed with Juju/MAAS 2.0 <cdo-qa> <cdo-qa-blocker> <deploy> <openstack> <juju-core:New> <https://launchpad.net/bugs/1583274>01:57
davecheneyit's either an error or an info01:57
redirwhen such an api is called01:57
davecheneyok, so that's an informational message "attention: this api call is deprecated and will be removed shortly"01:58
redirwell I guess if you could then just have it be an error when the API changes or is removed.01:58
davecheneyyou'll get the error for free when the api method is removed :)01:59
redirI usually suppress info only warn and worse when I care about logs in production02:00
davecheneyredir: precisely02:00
davecheneywhich is why I loose my shit when errors are logged at warning level02:00
davecheneyfor context: http://dave.cheney.net/2015/11/05/lets-talk-about-logging02:00
redirI agree 100% errors need to be errors02:00
redirbeen there read that02:01
redir:)02:01
redirAPI warnings are probably moot in the context of juju since end users aren't using juju as library and dont' need to know about internal deprecations. So, my bad example.02:02
=== redir is now known as redir_afk
davecheneyping, https://github.com/juju/juju/pull/543303:04
davecheneythis is a simple update to depenedncies.tsv03:04
natefinchdavecheney: ship it03:07
natefinchhonestly, I think that if we're just updating a dependency we wrote, we shouldn't need a review03:07
davecheneynatefinch: thanks nate03:08
davecheney                if err != nil {03:11
davecheney                        logger.Warningf("FIXME: ignoring unexpected CIDR format %q: %v", netConfig.CIDR, err)03:11
davecheney                        continue03:11
davecheney                }03:11
davecheney... i don't even know where to begin03:11
natefinchuh, yeah03:11
davecheneyOH MY GOD03:14
davecheneywe have an apiserver/common package03:14
davecheneyand we have an apiserver/common/networkingcommon package03:14
davecheneyHOW DID THIS HAPPEN !?03:14
natefinchlol03:14
davecheneycommon, but not that common03:14
davecheneysorta in the in crew, but not really03:14
mupBug #1583409 opened: help on status-history is confusing <juju-core:New> <https://launchpad.net/bugs/1583409>03:15
mupBug #1583412 opened: status-history doesn't complain about badly formatted names <juju-core:New> <https://launchpad.net/bugs/1583412>03:15
natefinchis status-history supposed to work?03:16
davecheneyyou might have to be flexbile in your definition on work03:18
natefinchwell, mine seems to always just return "ERROR no status history available"03:18
mupBug #1583409 changed: help on status-history is confusing <juju-core:New> <https://launchpad.net/bugs/1583409>03:18
mupBug #1583412 changed: status-history doesn't complain about badly formatted names <juju-core:New> <https://launchpad.net/bugs/1583412>03:18
natefinchhmm, just updated my client and now I get ERROR json: cannot unmarshal object into Go value of type names.Tag03:27
mupBug #1583409 opened: help on status-history is confusing <juju-core:New> <https://launchpad.net/bugs/1583409>03:27
mupBug #1583412 opened: status-history doesn't complain about badly formatted names <juju-core:New> <https://launchpad.net/bugs/1583412>03:27
davecheneyanastasiamac_: natefinch http://reviews.vapour.ws/r/4870/03:30
davecheneynatefinch: urk, how are tags leaking out of the apiserver03:30
mupBug #1583409 changed: help on status-history is confusing <juju-core:New> <https://launchpad.net/bugs/1583409>03:31
mupBug #1583412 changed: status-history doesn't complain about badly formatted names <juju-core:New> <https://launchpad.net/bugs/1583412>03:31
natefinchdavecheney: lemme rebuild the server and make sure I haven't screwed things up somehow03:31
davecheney                logger.Debugf("coercion failed attributes: %#v, checker: %#v, %v", attrs, checker, err)03:32
davecheney                return nil, err03:32
davecheneylog an error, at debug, then return a different error03:32
mupBug #1583409 opened: help on status-history is confusing <juju-core:New> <https://launchpad.net/bugs/1583409>03:34
mupBug #1583412 opened: status-history doesn't complain about badly formatted names <juju-core:New> <https://launchpad.net/bugs/1583412>03:34
natefinchdavecheney: oops: https://github.com/juju/juju/blob/master/apiserver/params/status.go#L13803:41
davecheneythat'll never work03:46
davecheneygood think it's got a unit test03:46
natefinchoh, there's even more broken there04:07
natefinchalso json related... the request object uses a *time.Time... then checks for nil after deserialization04:07
natefinchexcept that it serializes as a zero time and so deserializes as a zero (but non-nil) time04:08
mupBug #1583422 opened: status-history is broken <juju-core:New> <https://launchpad.net/bugs/1583422>04:22
mupBug #1583443 opened: juju 2.0 doesn't support bundles with 'to: ["service=0"]' placement syntax <juju-core:New> <https://launchpad.net/bugs/1583443>06:04
mupBug #1583443 changed: juju 2.0 doesn't support bundles with 'to: ["service=0"]' placement syntax <juju-core:New> <https://launchpad.net/bugs/1583443>06:13
mupBug #1583443 opened: juju 2.0 doesn't support bundles with 'to: ["service=0"]' placement syntax <juju-core:Invalid> <https://launchpad.net/bugs/1583443>06:22
mupBug #1583443 changed: juju 2.0 doesn't support bundles with 'to: ["service=0"]' placement syntax <juju-core:Invalid> <https://launchpad.net/bugs/1583443>06:31
babbageclunkdimitern: ping?08:54
dimiternbabbageclunk: pong08:55
babbageclunkdimitern: So, I think I've found the immediate thing causing these tests to fail (although I don't know what it means or how it happens yet) :).08:57
dimiternbabbageclunk: oh, go on?08:57
babbageclunkIn a successful test I get volumeattachments with volumeids 0, 0/1, 0/208:57
babbageclunkbut in a failing one they are 0/0, 0/1, 208:58
dimiternhmm08:59
babbageclunkWhat's the difference between the 0/1 ones and the 0 ones? What does the / indicate?09:00
dimiternbabbageclunk: weird.. what's expeced format of those IDs?09:00
dimiternbabbageclunk: ISTM "0" is the machine ID, whereas "0/1" is the second volume ID on that machine? but might be wrong..09:01
babbageclunkdimitern: well, they look more cryptic because 0 is the machine id.09:01
babbageclunkha ha yup09:01
babbageclunkoops standup09:01
voidspaceomw09:02
dimiternme 209:02
babbageclunkoh no - nothing until 1109:02
dimiterntechnically we don't have scheduled standup today09:02
dimiternwe can still do it with yesterday's slot if you like?09:03
babbageclunkyeah, sure - would be helpful for me!09:03
dimiternbabbageclunk: that comment in names/volume.go explains it: Volumes may be bound to a machine, meaning that the volume cannot exist without that machine. We encode this in the tag to allow09:26
dimiternso "0" and "0/1" are both valid volume tags09:27
dimiternbabbageclunk: also looking at TestWatchMachineVolumes ISTM the watcher is only started after adding the service, initial unit, and the following addUnit()09:32
dimiternbabbageclunk: are you stopping all watchers before resetting the DB09:33
dimiternbabbageclunk: it looks like if the watcher keeps going while the DB reset happens, it will still report the initial events *before* it was supposed to start at line 351 in state/volumes_test.go09:34
babbageclunkdimitern: No, that's probably it - thanks!09:39
voidspacedimitern: heh, so I have three places marked with XXX and no failing tests09:58
voidspacedimitern: moar tests09:58
voidspacedimitern: to be fair - I don't think those tests were needed before, so it's not really a lack of coverage09:58
dimiternvoidspace: moar tests == good :)09:59
voidspacedimitern: well, so long as they're not flaky...09:59
dimiternvoidspace: oh yeah10:00
voidspacedimitern: mtg?10:02
dimiternomw10:02
anastasiamac_davecheney: call?10:04
babbageclunkfwereade: re: logging context, I really liked the distinction between spans and baggage in http://opentracing.io/spec/10:58
fwereadebabbageclunk, so baggage is effectively high-level task context, and spans correspond to the various different components that might happen to be doing stuff in response?11:24
fwereadebabbageclunk, if so, yeah -- I'm currently thinking spans mainly, but baggage would be a fantastic addition11:24
fwereadekatco, worth considering in the context of audit? ^^11:25
babbageclunkfwereade: yeah, that's right. At the stock exchange there was a component that stamped every message flowing through the system with an id that let you follow the flow for that message through all of the logs across lots of different subsystems.11:27
babbageclunkfwereade: Was absolutely essential for tracking stuff down.11:27
fwereadebabbageclunk, I have some concerns about how far we can meaningfully take it in juju, because at a certain point everything is happening because of everything else11:34
fwereadebabbageclunk, but the useful 80% would be relatively cheap and easy, I think11:34
fwereadebabbageclunk, (that point is once you get down to relation settings changes, basically)11:35
fwereadebabbageclunk, (and we don't have any useful insight into what relation changes *actually* cause things to happen -- you could probably do something probabilistic with a suitably overarching view of the system, but that's a bit scope-creepy ;p)11:38
babbageclunkfwereade: :)11:38
babbageclunkfwereade: Yeah, that system was a lot less dynamic than juju - adding a machine or process while the system running was a rare, scary, fiddly task, where here it's kind of the point.11:41
mupBug #1583564 opened: failed to download URL for lxc images <landscape> <juju-core:New> <https://launchpad.net/bugs/1583564>11:53
mupBug #1583564 changed: failed to download URL for lxc images <landscape> <juju-core:New> <https://launchpad.net/bugs/1583564>11:56
mupBug #1583564 opened: failed to download URL for lxc images <landscape> <juju-core:New> <https://launchpad.net/bugs/1583564>12:02
dimiternvoidspace, babbageclunk: please take a look at http://reviews.vapour.ws/r/4871/, which fixes bug 157484413:39
mupBug #1574844: juju2 gives ipv6 address for one lxd, rabbit doesn't appreciate it. <conjure> <juju-release-support> <landscape> <lxd-provider> <juju-core:In Progress by dimitern> <rabbitmq-server (Juju Charms Collection):New> <https://launchpad.net/bugs/1574844>13:39
dimitern(I still need to add a few extra unit tests, but otherwise it won't change much)13:39
* dimitern steps out for ~30m13:40
mupBug #1583564 changed: failed to download URL for lxc images <landscape> <juju-core:New> <https://launchpad.net/bugs/1583564>13:48
babbageclunkdimitern: Why have duplicate attributes and methods for IPv6 and IPv4 addresses? Why not pass in a type (or use the address.Type() where approro13:54
babbageclunkOopp13:54
babbageclunkGah. ...where appropriate?13:54
dimiternbabbageclunk: because of the way address selection works, you can't rely on addr.Type == network.IPv6Address when trying to set the preferred IPv6 address (e.g. when only IPv4 addresses are available)13:59
dimiternbabbageclunk: I'd appreciate suggestions on the review :)13:59
dimiternbabbageclunk: like what's unclear and possibly the need for extra comments, etc.14:00
dimiternnow I'm really afk for a while..14:00
babbageclunkdimitern: Was doing that, but wanted to make sure I wasn't going off half-cocked.14:00
katcobabbageclunk: fwereade: thanks, i'll give it a read over14:01
voidspacedimitern: why not just have the preferred address algorithms prefer IPv4?14:02
voidspacedimitern: I don't see a need for all these extra methods14:03
dimiternvoidspace: because the whole point of the preferred addresses is that they don't change once set14:59
dimiternvoidspace: and if providerAddresses = ["public:fc00:123::/64"] the very first time SetProviderAddresses is called, both the preferred public and private IPv4 and IPv6 will be the same and they cannot change if later SetProviderAddresses is called with e.g. ["public:8.8.8.8", "public:fc00:123::/64", "local-cloud:10.0.1.2", "local-cloud:fe80::/64"]15:01
dimiternthat's essentially what the bug is about15:02
dimiternvoidspace: that's why preferred IPv4 and preferred IPv6 need to be stored separately15:03
voidspacedimitern: hmmm...15:04
katconatefinch: standup time15:04
voidspacedimitern: you still don't need two *setters* as preferred ipv4 and ipv6 can be determined from the type of the input to SetPreferredAddresses15:05
dimiternvoidspace: you mean for migrations?15:05
voidspacedimitern: I mean you don't need SetPreferredIPv4Addresses and SetPreferredIPv6Addresses15:06
dimiternvoidspace: that's for migtations only15:06
voidspacedimitern: migration of what/15:06
voidspacewhat?15:06
dimiternvoidspace: SetPreferredIPv4Addresses is a method of description.Machine interface in core/descriptiob15:07
dimiterndescription15:07
voidspacedimitern: I don't think it needs to be15:07
natefinchyou certainly don't need the word "Addresses" in there, at the very least.  This isn't java.15:07
voidspacedimitern: you can SetPreferredAddresses twice, once with ipv4 addresses and once with ipv615:07
voidspacedimitern: and if there's a type switch internally it will "do the right thing" without needing two methods15:08
katcoericsnow: natefinch: redir_afk: my hangouts has frozen15:08
* katco waits for it to unstick15:08
dimiternvoidspace: which SetPreferredIPv4Addresses method are you talking about/15:09
dimitern?15:09
voidspacedimitern: machine15:11
voidspacedimitern: and of course the interface in core descriptions15:11
voidspacedimitern: have SetPreferredAddresses sort the addresses it is given by type and set both ipv4 and ipv6 preferred addresses15:11
voidspacedimitern: and if it is called twice, once with ipv4 only and once with ipv6 only, it will still "do the right thing"15:12
dimiternvoidspace: I don't mind changing the description.Machine methods - they are less relevant anyway and will be likely called with all 4 preferred set15:12
dimiternvoidspace: there's no SetPreferredIPv4Addresses on state.Machine (or anything like that)15:12
voidspacedimitern: ah, it's core/description/machine15:13
voidspacedimitern: I care less about extra methods there I guess15:13
dimiternvoidspace: yeah :)15:13
dimiternvoidspace: my point exactly :) also I was following the pattern for existing migrations interface methods15:13
voidspacedimitern: ok15:13
dimiternsorry, brb again15:13
lazyPoweris anyone available to help troubleshoot a 1.25 controller stuck in an erroneous "upgrade", the user apparently didn't request the upgrade, but rebooted the host controller and its stuck doing an upgrade.15:27
lazyPowerthey're over in #juju if anyone has a moment to lend a hand. i'm out of my depth on this one, i'm directing them to aggregate logs and file a proper bug15:27
natefinchlazyPower: I noticed he said this was a case study and would get published.... sigh.  bad day for all the managers to be at a sprint15:42
natefinchrick_h_: you around?15:42
lazyPoweragreed :/15:43
katcoi also need a volunteer to tal @ bug 1537585. this is blocking the landscape team from releasing.15:50
mupBug #1537585: machine agent failed to register IP addresses, borks agent <2.0-count> <blocker> <landscape> <network> <juju-core:Triaged by dooferlad> <juju-core 1.25:Triaged by dooferlad> <https://launchpad.net/bugs/1537585>15:50
katconatefinch: you're the only one without feature work atm. are you looking at this other customer-facing issue in #juju?15:51
katcoperrito666: what are you up to?15:51
natefinchkatco: was just trying to figure out if I should help the guy in #juju... I've gotten flack for doing that in the past, so was hesitant this time.15:52
katconatefinch: i was thinking the same thing. we have paid support for that reason15:55
katconatefinch: i would say pick up this bug if you can. it will unblock landscape and make all of juju better15:56
natefinchkatco: before or after I fix status-history?15:56
katconatefinch: how long do you think it will be for that fix?15:57
natefinchkatco: just an hour or so to tweak the tests to pass15:57
katconatefinch: i'd say wrap that up first then15:57
natefinchkatco: yeah, my thought as well. Cool.15:57
natefinchkatco: then I'll hop on the landscape one15:57
katconatefinch: you are a bug fixing machine15:58
perrito666katco: I was in bed until about now, I am trying to wrap up the restore work that ian inherited me16:08
katcoperrito666: ok, ta. hope you're feeling better :(16:08
perrito666katco: nope, but bed is driving me crazy so I get up and work every now and then16:08
katcoperrito666: sorry to hear that16:09
perrito666katco: you know how these things are, you just have to wait and eat chicken soup16:09
katcoperrito666: yeah. i'm not a good sick person.16:09
mupBug #1583683 opened: juju thinks it is upgrading after a reboot <juju-core:New> <https://launchpad.net/bugs/1583683>16:24
mupBug #1583683 changed: juju thinks it is upgrading after a reboot <juju-core:New> <https://launchpad.net/bugs/1583683>16:30
mupBug #1583683 opened: juju thinks it is upgrading after a reboot <juju-core:New> <https://launchpad.net/bugs/1583683>16:43
=== redir_afk is now known as redir
lazyPowerkat/nate - is there anyone in particular i should ping with the bug dweaver submit for their issue? (the #juju casestudy user)16:57
lazyPowerkatco natefinch ^  - sorry pinging you directly as you made mention. I want to route this appropriately16:57
katcolazyPower: i pinged cheryl, alexis, and rick_h_16:58
lazyPowerok, bugs right up there ^16:58
katcolazyPower: ta16:58
lazyPowerthanks for the handoff16:59
katcolazyPower: just want to be crystal clear: we haven't committed to investigating this yet16:59
lazyPowerok, if it gets dropped it gets dropped. Theres very little i can do about it, and i know we're on a tight schedule/deadline.17:00
lazyPowerwe've done what we can17:00
katcolazyPower: curious if they can hang on until next week? half our resources are sprinting atm17:01
lazyPoweri made that clear that we were short handed due to a planning sprint.  Their response: [11:59:18] dweaver: lazyPower, yes, willing to be patient, thanks.  I'll ping you with a bug ID once it is submitted with all the info.  Will be most appreciative thanks.17:02
katcolazyPower: awesome, ta chuck17:02
lazyPowernp np, good hustle17:03
katcoalexisb: are we still doing our 1:1? figured you were busy with sprint17:50
alexisbkatco, we are not17:50
alexisbhow are things?17:50
katcoalexisb: things going fine17:50
perrito666did someone forget a couple of machines running in our ec2 since apirl 14?18:24
* perrito666 is about to pull the trigger18:24
alexisbnatefinch, ping18:26
natefinchalexisb: pong18:28
alexisbhey natefinch what are you up to?18:29
natefinchalexisb: fixing status-history, which I figured out last night is completely broken18:30
alexisbnatefinch, how long will you be working those bugs?18:30
natefinchalexisb: well, wrapping up status-history soon.  There was a bug reported by landscape guys that I was going to work on next, but what's up?18:31
alexisbnatefinch, please continue18:32
alexisbnatefinch, katco thanks18:32
redirericsnow: got a minute?18:45
ericsnowredir: sure18:45
redirmoonstone?18:45
ericsnowredir: yep18:46
katcoredir: spec updated19:54
natefinchperrito666: you around?19:57
natefinchperrito666: http://reviews.vapour.ws/r/4872/19:57
perrito666natefinch: looking19:59
perrito666natefinch: ship it20:01
perrito666natefinch: and incidentally I think I opened a "we should not use *time.Time" bug some time ago which you just partially addressed20:01
natefinchperrito666: I actually undid my addressing of it... it just propagated out too far. I was going down a rat hole and just wanted this thing fixed.20:02
natefinchperrito666: it wouldn't be that hard to fix, it just touches a lot of files20:02
perrito666oh, I thought you where doing that, then I noticed you didnt :)20:02
perrito666i might do it in the trip to the sprint20:02
mupBug #1583771 opened: RunnerSuite.TestOneWorkerStartWhenStopping timed out <ci> <intermittent-failure> <race-condition> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1583771>20:13
mupBug #1583772 opened: MachineSuite.TestDyingModelCleanedUp state.Life dead not alive <ci> <ppc64el> <regression> <s390x> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1583772>20:13
natefinchI wish we had juju add-container... I can never remember the invocation to add a container to an existing machine20:33
natefinchalso wish juju add-machine lxc:0 -n 2 worked :/20:34
natefinchcannot use -n when specifying a placement directive.... why the hell not?20:34
mupBug #1583787 opened: get-* CLI commands should be called show-* <juju-core:New> <https://launchpad.net/bugs/1583787>20:52
mupBug #1583789 opened: juju uses wrong gpg for validating images on streams.canonical.com <centos> <ci> <simplestreams> <juju-core:Triaged> <https://launchpad.net/bugs/1583789>20:52
redirkatco: tx20:53
=== redir is now known as redir_lunch
=== redir_lunch is now known as redir
redirfwereade: yt?21:45
katcoanastasiamac_: i'm in the hangout when you're ready22:08
* redir stuck22:13

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!