[01:25] thumper: ping [01:29] thumper: emailed you my question [01:33] ok [01:34] waigani: you need to cast it to what you know it is [01:34] oh, so simple [01:43] mramm: can i get editing perms for that doco? [02:18] wallyworld: what is the card "mass provider picks maas name using placement" ? [02:18] wallyworld: I already implemented maas-name placement [02:19] is there something missing? [02:19] axw: i got that from the next steps on a previous mp [02:19] maybe it's been done [02:19] all those 4 cards are from the same source [02:20] probably talking about supporting it in "juju deploy --to" and "juju add-unit --to" [02:20] yeah [02:20] I spoke with William at Vegas about placement, and he reckoned we should just stick with machine-id in those commands for now [02:21] ok [02:21] feel free to update the cards as necessary [02:21] supporting placement in those is a little trickier [02:21] ok [02:21] i just wanted to capture out of the mp the next steps [02:23] wallyworld: going o delete the maas-name one, since that's done. anything remaining is provider-independent [02:23] sounds good [02:24] I'm going to prod my AZ changes along at the same time as this address stuff [02:26] ok [03:12] * thumper has hit friday afternoon itis [03:57] * thumper makes a sad face [03:57] ugh... [04:04] on that note, if anyone feels like having a look at this to see why $schema values are causing nil exceptions in gojsonschema, input would be welcome [04:04] https://codereview.appspot.com/94540044 [04:04] I'll be looking at it tomorrow [04:04] going to bed in a minute here === vladk|offline is now known as vladk [05:22] Anyone wanna chat about logging / mongo and the state package? [05:22] https://codereview.appspot.com/94540044 fwereade mgz [05:23] I'm working on adding a 'results' collection for Actions, and if I'm reading it right the 'log' collection on watchers is in a separate mongo db from the state db? [05:31] okay.. I see now there are 3 db's: juju, presence, and admin [05:48] and I now see that the watcher logs are on the juju db === Ursinha-afk is now known as Ursinha === vladk is now known as vladk|offline [06:51] axw: i have to go to soccer so won't get to look at your branch before i got, i'll look when i get back later if it's still unreviewed [06:52] wallyworld: thanks [06:52] no particular rush [06:52] I'm going to work on manual AZ support now [06:56] ok [06:56] you might be EOD before I get back so have a good long weekend :-) [07:03] thanks wallyworld . enjoy soccer === Ursinha is now known as Ursinha-afk [07:11] jcw4, if you're still around, I can talk about it: I suspect if you're needing to look at the log collection you're doing it wrong somehow [07:11] axw, just responding to your review [07:12] fwereade: thanks [07:13] bodie_, I might not get to that in time for this evening, hopefully mgz will though === Ursinha-afk is now known as Ursinha [07:36] axw, so, I wrote you an essay: https://codereview.appspot.com/102860043/ [07:37] mgz, are you around? in dimitern's absence I'd like what clarity you can provider on container addresses -- see my comment in that review ^ [07:38] fwereade: thanks. when I said "relation unit in state", I really meant the scope doc (but that doesn't invalidate anything) [07:39] axw, I don't think the unit wants to watch its own scope/settings docs though [07:39] axw, it wants to watch something else, and the settings only get updated according to the unit's having responded to those changes, detected elsewhere [07:40] axw, anyway, I'm going to make some coffee and have a ciggie; ping me when you're ready to talk and I'll be there soon [07:41] fwereade: thanks, I'll just digest this first [07:44] fwereade: cool, it all makes sense. I kinda came to the same conclusion, after assuming that we could allow the RU to enter scope first. you mentioned the addresses-watching worker the other day, but it didn't click at the time === lazyPower is now known as lazypower-travel [07:55] axw, also I suspect we already have the seed of that worker underway, because the networker is going to be responsible for setting up the machine's networks in the first place; having it update the machine with its addresses seems entirely natural to me [07:55] * axw nods [07:58] axw, btw, is there anything new for review in that or can I wip it? [07:58] fwereade: nope, it can go to WIP [07:58] axw, (btw, if you want to propose that driveby cleanup on its own I would look favourably on such an endeavour :)) [07:59] certainly [08:00] fwereade: not sure what else there is to discuss, but I am free now if you want to chat still [08:01] axw, the main thing that's on my mind is whether we can and/or should combine the relation model with the exposure model, and if so what it'd take to do so [08:01] axw, but that's not directly relevant to what you're doing [08:01] fwereade: frankly I don't think I know enough to have any useful input on that right now [08:02] maybe once I've gotten through the initial bits I will [08:03] axw, yeah, nobody does really tbh, I should probably find a cuddly toy and explain my thoughts to them and see if I still think they're a good idea afterwards [08:03] ;) [08:13] Morning all === vladk|offline is now known as vladk [08:20] morning void [08:20] voidspace rather [08:20] * axw looks into the void [08:24] fwereade: I'll take a look [08:28] axw: you know what they say - the void looks back [08:31] heh :) [08:51] axw, https://codereview.appspot.com/99660047/ LGTM but might benefit from a gentle massage to move some of the functionality to slightly more natural-feeling places [08:51] axw, let me know if the review doesn't give enough direction [08:52] wallyworld, hey, are you around? I wanted to talk about md5 vs sha256 [08:53] fwereade: thanks, sounds fine [08:54] fwereade: my predilection for methods is to avoid polluting the namespace, but I understand the desire to keep methods near their types too [09:09] axw, one thing that floats around my mind is the idea that the functionality that depends entirely on the exposed interfaces of exposed types should probably be kept separate from the stuff that needs access to the internals [09:09] fwereade: yeah, that makes sense [09:09] axw, it's tricky especially in go, though, because if you want to use them in-package (which you very often do) you can't move them out (lest import cycles) and therefore you can't actually enforce those boundaries [09:10] axw, but free funcs in their own files are probably the closest we have [09:11] axw, (or ofc you can explode the package into a *load* of smaller packages but that's often a dauntingly big job even for a small package, and with something like state it's devastatingly hard [09:11] ) [09:12] yeah, I'll just go with free funcs ;) [10:01] `Added charm "cs:precise/ntpmaster-3" to the environment. [10:01] ERROR cannot assign unit "ntpmaster/0" to machine: cannot assign unit "ntpmaster/0" to new machine or container: cannot assign unit "ntpmaster/0" to new machine: use "juju add-machine ssh:[user@]" to provision machines` [10:01] That's a pretty bad error message === niemeyer_ is now known as niemeyer [10:09] niemeyer, it is, isn't it :/ do we have a bug fior it? [10:10] niemeyer, we definitely ought to catch that case earlier, regardless -- axw, are you still around? [10:10] fwereade: That seems rooted on how these new error helper libraries work.. I bet this is not the only one [10:11] niemeyer, it's rooted in the old way of doing things, actually, it's a balance it's been difficult to maintain across the board [10:12] niemeyer, I have pretty high hopes that they'll help us present clearer error messages to users but record the context usefully as well [10:12] niemeyer, nobody ever said that too much annotation is not a problem when it leaks up to the user [10:13] dimitern, would you take a look at https://codereview.appspot.com/102860043/ and contribute your understanding of how we'll be handling the container-address/machine-address issues? [10:13] fwereade: The problem is not just leaking to the user.. this is adding the same annotation repeatedly, which isn't helpful to developers either [10:14] niemeyer, quite -- it's a failure to annotate helpfully, which is orthogonal to the mechanism we use for annotation [10:14] fwereade: It might be orthogonal, or it might not [10:15] fwereade: If you stipulate that every level should annotate, this encourages developers to add boilerplate to fill up the form [10:15] niemeyer, and fwiw it's not *quite* the same annotation each time -- it does give you a trail of breadcrumbs that helps me as a developer see how we came to be in that situation [10:16] niemeyer, this is not a defence of the UX, or of the particular sequence of messages [10:21] niemeyer, on a separate note, I was thinking I would grab a lift from cath when she comes into town this pm, so I could come and see you -- do you have any time this pm? [10:22] niemeyer, and this evening I was thinking of going into floriana (easy frequent bus ride from sliema) for http://www.whatson.com.mt/en/home/events/9093/ghanafest-2014.htm?fb_action_ids=10152073938286851&fb_action_types=og.likes&fb_ref=.U4gRG8XT_7Q.like (if you or anyone else might be interested) [10:22] niemeyer, I have no idea if it's any good [10:23] fwereade: Oh, that looks nice [10:23] fwereade: Yeah, pm is mostly free, besides a few events [10:27] niemeyer, ok, I'll probably get to the meridien a bit after 3 [10:29] fwereade: Cool [10:35] fwereade: I was wondering if you would pair with me on a review at some point today. I find it beneficial to do a review, thinking out loud, and then watch someone else do the same review. [10:36] fwereade, looking [10:37] wwitzel3, it would be a pleasure, we should do it in the next 2 hours or so, but not right now I think, I need a bit of a lunch break [10:38] fwereade: at your convenience, just ping me [10:38] wwitzel3: morning [10:39] wwitzel3, it would be great if you would take a pre-look at https://codereview.appspot.com/92610043/ https://codereview.appspot.com/100810045/ https://codereview.appspot.com/94540044/ in the meantime -- and if you can come up with comments please go ahead and post them [10:39] voidspace: morning :) and any break on that precise regression? [10:39] wwitzel3: nope, although I have one indication it may just be a timing issue - I wonder if sleeping and retrying would help :-) [10:39] wwitzel3, regardless it will be good if you have a bit of context loaded for them before we begin [10:39] wwitzel3: http://efreedom.net/Question/1-9992535/Azure-MongoDB-Replica-Sets-Initialization-Error-Localoplogrs-Empty [10:40] There was no problem at all. This exception is normal behavior. Just need to wait couple of seconds and replica set will be initialized. [10:40] - See more at: http://efreedom.net/Question/1-9992535/Azure-MongoDB-Replica-Sets-Initialization-Error-Localoplogrs-Empty#sthash.sbXHTaFd.dpuf [10:40] Hah, original reference for that: http://stackoverflow.com/questions/9992535/azure-mongodb-replica-sets-initialization-error-local-oplog-rs-is-not-empty [10:40] fwereade: indeed [10:41] or maybe even that the replica set is already initialized [10:41] we see many logging lines of the same failure - maybe we're trying to initiate multiple times [10:45] TheMue, vladk, fwereade, standup? [10:48] I have to nip out for an early lunch [10:48] biab === Ursinha is now known as Ursinha-afk === vladk is now known as vladk|offline [11:19] fwereade: i am now [11:20] i used md5 cause it was native in gridfs, but can change it === Ursinha-afk is now known as Ursinha [11:33] fwereade: just ping me if you want to chat [11:41] hi ppl === vladk|offline is now known as vladk [11:42] morning [11:47] g'morning [11:52] wallyworld, bugger, my schedule is becoming problematic [11:52] wallyworld, did that review make any sense to you? [11:52] wallyworld, I fear it was a bit light on direction [11:53] wallyworld, mainly I really want us to get the existing storage api in place against gridfs before we try to build in the content-addressability stuff [11:54] wallyworld, or at least deduplication, sorry, my brain is firing on relatively few cylinders today [11:56] fwereade: the gridfs storage is already impelemnted [11:56] this is the next step [11:57] i also considered pulling out a separate txn type base class but wanted to wait till see how it evolved [11:57] there's a few more brnaches to go [11:59] the next step will be a ManagedResource service which uses ResourceCatalog and ResourceStorage to provide an exported, authenticated storage manager which handles dups etc [12:00] we probably need to talk face-face. if you get a break in your schedule, ping me or we can do it later, whatever suits [12:56] voidspace, I am rebooting juju-ci-vapour.ws; my last desperate effort to address the lxc deploy issue [12:58] * sinzui waits for jenkins to return [12:59] sinzui: still not able to reproduce that somewhere else? [13:00] perrito666, I have tried else where, but I played over revisions of juju and the test passes. [13:01] * sinzui rebuilds after upgrade and restart [13:33] so... import aliases: import (coretesting "launchpad.net/juju-core/testing"; jujutesting "launchpad.net/juju-core/juju/testing"; ???? "github.com/juju/testing") [13:34] oh yeah, i missed: stdtesting "testing" [13:34] and apiservertesting as well :P [13:34] any suggestions for github.com/juju/testing. i'm currently using gitjujutesting. [13:35] is the github package specific to juju or just living in that org namespace? [13:35] wwitzel3: it's under juju control, but it's not specific to -core [13:36] wwitzel3: i believe that juju-core/testing will continue to exist [13:38] ha, envtesting, toolstesting [13:38] rogpeppe: I haven't explored, but I've always wondered the difference between juju-core/testing and juju-core/juju/testing [13:38] wwitzel3: the former can be used by lower level packages - it doesn't import state [13:38] rogpeppe: ahh, ok [13:39] ugh yuck [13:40] rogpeppe: no matter what you do, you won't like it [13:40] rogpeppe: does that help? lol [13:41] wwitzel3: lol [13:41] I guess my initial reaction was to name the gh/juju/testing package something else [13:43] rogpeppe: also I really hate that we alias the go stdlib testing package as stdtesting .. if anything should just be testing, it should be the stdlib package, but I digress. [13:43] rogpeppe: I guess gitjujutesting .. it will only make your eyes bleed occasionally [13:44] if we just stopped writing tests this wouldn't be a problem ;D [13:47] rogpeppe: you could name it something like .. yatp [13:47] moartesting [13:52] fwereade: wrt. the log collection, I was just trying to understand where actionResults would go. It seems clear now that it would be a collection in the juju db in mongo [13:53] hey, perhaps some of you might find this useful, patches are welcome, it is a pretty quick hack https://github.com/perrito666/golazytools/blob/master/gogrep [13:53] and probably held in the state struct like actions too [14:02] wwitzel3, voidspace: standup? [14:02] natefinch: yep, thanks [14:08] jcw4, yeah, I think it's just another collection, like actions [14:08] morning all [14:08] bodie_, heyhey [14:09] fwereade: yeah. I started looking for logs 'cause it was the closest analog I could think of, and noticed that the watchers had their own mgo.Collection on them, and that's what led me astray [14:09] hi bodie [14:09] jcw4, yeah, the logs are really at a level below state [14:11] fwereade: I think I'm clear on where the collection should be now, but I'm not sure how much detail to put in the results... timestamp, unit?, output, error?, running time? etc. etc. [14:12] jcw4: good questions [14:14] all of htat sounds good, I'm not sure on the form of the unit link though [14:14] mgz: I suppose we can start light just action name, unit id?, and output, and then add more detail later as needed? [14:14] mgz: yeah, globalKey? u.doc.Name? etc. [14:14] mgz: encoded in _id like the actions collection [14:19] okay, I'm going to see about getting the next steps plugged in, I guess an Actions() method on the Charm interface [14:20] bodie_: sounds good to me [14:21] anyone have a few spare cycles to give me a quick LGTM on https://codereview.appspot.com/94540044 ? [14:21] I think we've just about beaten out every possible flaw there, heh [14:21] unless fwereade has input about whether to move the yaml back to global scope [14:21] or to a file? [14:21] bodie_: I wouldn't go *that* far :) [14:22] lol [14:22] "that far" being "beaten out every possible flaw"? [14:22] lol [14:22] :D [14:23] I'm more than willing to go back over it with an industrial-strength iron, but I'd personally really like to be out of that file [14:23] wwitzel3, did you get a chance to look at that one? ^^ [14:24] fwereade: that is the one I was just pulling up now, I just did them in the order you linked them and that one happened to be last :) [14:24] bodie_: lgtm, land it. go to the launchpad page, set the commit message, and mark approved [14:24] aye-aye :) [14:24] mgz, you might need to land it for him? [14:24] fwereade: I may need to toggle the approved [14:24] he can set the commit message [14:25] wwitzel3, it's worth casting an eye over it anyway (but this is not a blocker on landing it) [14:25] mgz, +1 [14:25] ...I guess making him do it isn't that useful as we're changing all this, but hey [14:25] fwereade: sure, doing that now [14:26] dimitern: machine-1-lxc-0 can't authorize machine-1/lxc/0, is it a bug? [14:27] mgz: it got pushed back at least once... maybe again... [14:27] fwereade: the first one I reviewed, add user info to juju switch, was fairly heavily reviewed, so I didn't have any comments on it. But I did read through the progression of the code and all the comments. [14:27] wwitzel3, I'm afraid I'm disappearing for half an hour to regather my sanity, because today has somehow been stupidly relentless, and you have been very nice about my inattention so I propose to trespass upon it further :/ [14:27] jcw4: I'll get yelled at if it does [14:28] mgz: lol [14:28] vladk, machine-1/lxc/0 is not a thing, is it? [14:28] vladk, it's a mix of tag and id syntax [14:28] vladk, machine-1-lxc-0 is the tag, 1/lxc/0 is the id [14:28] fwereade: haha, no worries, I'm here for hours yet :) enjoy your siesta [14:29] fwereade: thanks [14:29] vladk, (or possibly my brain has turned to cheese, double-check in the juju-core/names package) [14:29] head cheese [14:30] uhh, cheese [14:30] * perrito666 is near lunchtime [14:32] I see there is no much consistency on the tests about jc.IsTrue vs gc.Equals, true) sounds to me like jc.IsTrue is the best way to express the assertion, is there any reason not to use it? [14:33] perrito666: nope [14:33] perrito666: the latter is what we had to use before jc.IsTrue [14:33] ah, I see [14:34] perrito666: (personally, i think gc.Equals expresses the logic perfectly, but i think some people begrudged the extra characters) [14:34] frankban, anyone else: review appreciated: https://github.com/juju/testing/pull/7 [14:34] btw mgz, I think the nil exception was coming from a logical error in my tests [14:34] I caught that thanks to your suggestion to test the one edge bit [14:35] so, thanks :) [14:35] :D [14:35] how detailed / verbose do we like our commit messages to be? [14:35] frankban: it's trivial BTW [14:36] bodie_: i generally prefix the commit message with the package name(s) where reasonable, and a single line that describes the change to some degree [14:36] bodie_: the important one is the MP message, which turns into the eventual trunk commit msg [14:37] I see, thanks [14:37] rogpeppe: done [14:37] frankban: thanks [14:38] rogpeppe: why do you have to skip C (cgo)? for it to work. [14:38] wwitzel3: because build.Default.Import fails for cgo's import "C" [14:38] wwitzel3: and the net package uses cgo [14:39] wwitzel3: "C" is a fake package - it doesn't actually exist anywhere [14:39] rogpeppe: reading about that now, it is essentially a marker for C preamble. [14:40] wwitzel3: yeah [14:40] wwitzel3: and C names are inside its name space [14:40] rogpeppe: yep, makes sense [14:59] rogpeppe, fwereade, mgz do you think I need to amend the code with a comment for the spec the Actions YAML is expected to take, or is that more of a docs/ addition? [14:59] I'm just combing the last kinks out of the MP message [14:59] bodie_: i think that a doc comment is always good [15:00] bodie_: for anyone trying to parse charms from go and using the package, that's all they'll see [15:00] bodie_: although you could add a link to the spec instead [15:00] hmm [15:00] okay [15:00] give me a few minutes to land something appropriate, I'm just putting my kid down for a nap [15:00] then hopefully we can finally get this thing in [15:04] bodie_: sorry, can we land your branch yet? [15:07] review appreciated of this: https://codereview.appspot.com/99670045/ [15:07] pretty trivial stuff - mostly just code movement [15:12] natefinch: wwitzel3: perrito666: hey guys, sorry I missed standup [15:12] natefinch: wwitzel3: perrito666: got stuck in traffic and have been out a stupidly long time [15:13] natefinch: I assume you didn't make any progress on the precise CI problem yesterday? [15:13] mgz, I'd like to basically copy the MR message into a comment for the Actions type, just give me a minute here [15:13] sorry for the delay [15:13] natefinch: from "further research" this morning I have a theory - going looking in the code to see if it could be correct [15:15] mgz, do I need to re-propose? [15:16] (I know I'd want this change as a user of the type) [15:16] voidspace: yeah, no, sorry, no progress [15:16] bodie_: nope [15:17] mkay [15:19] okay, it says I need review since I pushed a change, even though it's a simple one... [15:20] just a richer comment on the type [15:20] so I guess that means I need to lbox propose [15:21] I promise the next iteration won't have such a low signal to noise ratio :/ [15:22] bodie_, you just need to refresh the launchpad page and approve on the frsh page, I think [15:22] alrighty [15:26] fwereade, I'm not seeing the approved button any longer [15:26] can I just get a quick approval here... https://codereview.appspot.com/94540044 [15:37] someone/anyone? All I added was a couple of comments to types that needed to be more informative [15:37] https://codereview.appspot.com/94540044/ [15:38] fwereade, mgz, natefinch, rogpeppe, just want to land this thing already, sorry to pester but apparently the comments I added to the code mean it needs a fresh review [15:38] just so it's on your radar [15:38] bodie_: landing [15:39] thank ye sir [15:46] was that for merge 219926? [15:46] why does their url for v4 have /latest/ rather than /v4/ or something >_< [15:47] bodie_: I poked the commit mesage a little and flagged for the bot [15:47] oy... I thought I linked the v4 draft. bleh [15:47] yeah, that fact was driving me nuts [15:47] oh, poo, I need to go pull in the deps [15:48] I think I made up for it by linking the v4 _schema_ rather than the draft since I couldn't find a copy of the v4 draft [15:48] technically, JSON-Schema lets you include a $schema key to define the schema to use -- however, I noticed the "latest", i.e. v4 draft, is from last august [15:48] I don't even know if they landed a final version, or if it fell out of date, or what's up with all that [15:50] heh `fail`... I should totally make that tyop an alais for `tail -f` [15:50] bodie_: bot is chewing on it [15:50] mgz lol [16:04] bodie_: landed [16:05] woot [16:05] everyone: you'll want to pull trunk and get the new gojsonschema related dependencies [16:05] pilgrim's progress === Ursinha is now known as Ursinha-afk === vladk is now known as vladk|offline === vladk|offline is now known as vladk === vladk is now known as vladk|offline [17:07] fwereade, anyone else, where do you think the Actions params validator should go? since Actions is a map belonging to Charm, which needs to be loaded by gojsonschema in order to validate a Params map, it seems really sensible to me to implement that as a member of the Charm interface [17:08] something like func (c *Charm) Validate(map[string]interface{}) error { ... } [17:08] however, I could also see that being the responsibility of a method in State [17:10] bodie_: I don't think Charms or State should know about Action parameter validation (directly), I would expect the Validate to be on an ActionDefinition type (not sure what we'll call that type) [17:10] bodie_: in the example you give... How does Charm know *which* action you're validating? [17:12] sorry, one moment, helping the girls up [17:13] * jcw4 wil brb [17:18] maybe we could use an Action interface to tie it all together? [17:18] like Charm [17:19] that probably isn't what we need since Charm has a different meaning for Actions then State will have [17:19] than* [17:22] natefinch: I'm approaching EOD, and it still looks like we're no nearer a solution [17:22] natefinch: although interestingly it looks like the specific failure has changed - we're now seeing "Closed explicitly" as the error message [17:23] natefinch: we need to time bound how long we're going to continue looking at this I think [17:23] natefinch, voidspace : do you have a minute to review https://codereview.appspot.com/100880046 [17:24] sinzui: looking [17:24] that was tricky :-) [17:29] natefinch: although this reference implies that the call to replSetInitiate can error, but be successful [17:29] http://stackoverflow.com/questions/9992535/azure-mongodb-replica-sets-initialization-error-local-oplog-rs-is-not-empty [17:29] There was no problem at all. This exception is normal behavior. Just need to wait couple of seconds and replica set will be initialized. [17:29] Inside Initiate we wait 1 second (well, ten sleeps of 100ms) [17:31] we could wait a bit longer (up to a few seconds) whilst polling for the replica set status [17:31] it could be a timing issue (slow machine) which is why it only happens on the build machine [17:32] voidspace: back [17:33] voidspace: hmm slow machine sounds possible [17:33] natefinch: hey, hi - only a few lines of scrollback [17:33] natefinch: the question is whether or not that specific error is genuinely recoverable [17:34] natefinch: we have replicaset.CurrrentStatus() [17:35] natefinch: I could add polling for that - if we get an alive status (or equivalent) then continue [17:35] upping the timeout seems harmless. bootstrap is already relatively slow, making it a few seconds slower in the edge case where there's a non-recoverable error seems like not a big deal [17:35] natefinch: MaybeInititiateMongoServer only returns an error [17:36] natefinch: ok I'll create an mp adding that and we can see if it fixes the problem [17:36] I hate this blindly swinging at invisible errors stuff. [17:36] me too :-/ [17:37] natefinch: this log with debug (the CI console output logs are INFO only) shows us attempting to call replSetInitiate multiple times [17:37] http://pastebin.ubuntu.com/7544771/ [17:37] gustavo might have a better idea of what's going on, but he's playing squash or something in Malta, I believe [17:37] and if it's "in progress" it makes sense that subsequent calls fail with this error [17:37] effectively meaning "I'm already initiating a replSet for you"... [17:37] you are only supposed to call it once [17:38] ahh right, good point [17:38] the odd thing is that the loop for multiple calls is explicitly checking for unreachable servers [17:38] voidspace: well the first call declares it fails, those can be retries [17:39] perrito666: we actually have two loops [17:39] one inside initiate.go and one inside replicaset.go [17:39] the initiate.go one logs "replica set initiation failed, will retry" and we're *not* seeing that [17:39] so it must be the replicaset.go one [17:40] hmmm... ah wait [17:40] the other loop is: if err != nil && err.Error() == rsMembersUnreachableError [17:41] looks like a debug log has been removed [17:41] perrito666: http://pastebin.ubuntu.com/7544771/ [17:42] voidspace: migration step? [17:42] that log is showing repeated retries inside Initiate - but not repeated calls to the initial logging line [17:42] perrito666: what do you mean? [17:43] voidspace: sounds like something intended to introduce peergrouper in a harmless way [17:43] 31 // MaybeInitiateMongoServer checks for an existing mongo configuration. [17:43] 32 // If no existing configuration is found one is created using Initiate. [17:44] also there is a comment by rogpeppe [17:44] 43 // TODO(rog) remove this code when we no longer need to upgrade [17:44] 44 // from pre-HA-capable environments. [17:44] perrito666: right, but the logging shows we go into Initiate and then retry multiple times inside there [17:45] perrito666: and this reference implies that this error can be non fatal (i.e. initiation is still happening) [17:45] http://stackoverflow.com/questions/9992535/azure-mongodb-replica-sets-initialization-error-local-oplog-rs-is-not-empty [17:47] ok, I need to EOD [17:47] g'night all [17:47] voidspace: well in restore, after running init of rs I do run a loop trying to connect as the client until it actually works [17:47] voidspace: bye [17:47] night voidspace, thanks for the help. Sorry it's been such a hassle [17:48] natefinch: heh, such is life [17:48] voidspace: the reason to remove that code is that in post HA-version environments, MaybeInitiateMongoServer should only be called once, at bootstrap time. at that time, there are no users, so there's no need to connect as a specific user [17:49] rogpeppe: right, but it's currently harmless and definitely *not* the source of the current problem [17:50] voidspace: ok, that's fine [17:50] which is a problem of initiating failing at bootstrap time [17:50] on one machine [17:50] and that machine only so far... [17:50] voidspace: ok, that's what i was about to ask [17:50] so, latest theory is that initiating is just slow on that machine [17:51] so I'll poll replicaset status and if we see a healthy replica set within a few seconds we'll ignore that error [17:51] as I've found one reference saying that this can be the case [17:51] if that fixes the problem then great [17:52] but for now... [17:52] good night :-) [17:53] rogpeppe: yeah, it's just the precise CI machine that can't run local provider with --replset for whatever reason. [17:53] rogpeppe: works on other peoples' precise VMs [17:53] natefinch: have we tried doing it manually on the CI machine? [17:55] I think Michael had said he tried it manually, but I'm not 100% sure [17:55] certainly worth testing out [17:56] sinzui: is there a way I can get on the precise CI machine to noodle with mongo? === alexisb is now known as alexisb_lunch [18:04] natefinch: I think Michael mentioned not having access to the machine to try it manually [18:07] perrito666: ahh yeah, right, well, we should get on there. [18:40] okay, so I've gone and forgotten to switch branches before writing code, and committed to master [18:40] is there a quick and dirty way to just yank that commit into a new branch somehow without taking the fact that it came from master? [18:41] I'm not entirely clear on how much bzr cares about branch history vs file contents [18:42] like a cherry-pick [18:43] I guess that's a merge -c [18:44] sorry, no idea [18:44] bodie_: If that doesn't work, I would personally just generate a patch of the changes and then remove your local repo and start again [18:45] bodie_: otherwise, you'll be fighting the changes in master when you pull from lp:juju-core, and when you try to create new branches [18:45] right [18:45] I think I can merge -r 1 and then cherry-pick the incorrect changes from master, which should bring them in free of branch history info [18:45]