[01:21] <jelmer> hmm, I'm seeing weird behaviour talking to the qastaging server
[01:22] <jelmer> retrieving a wadl resource I sometimes (I don't see a pattern yet) only get 3486 bytes back
[01:22] <wgrant> How big are the headers?
[01:22] <wgrant> And what is the content?
[01:23] <jelmer> the content is JSON (but cut off halfway through, so the JSON parser barfs)
[01:23] <wgrant> We occasionally see that when production is overloaded.
[01:23] <wgrant> The response, including headers, gets truncated to a power of two.
[01:23] <wgrant> I do not remember which power of two.
[01:24] <jelmer> it seems to consistently be 3486 here, and it appears to happen randomly
[01:24] <jelmer> I wouldn't be surprised if the size of the headers was (4096-3486)
[01:24] <wgrant> That sounds about right.
[01:24] <wgrant> Which resource?
[01:24] <wgrant> Authenticated or not?
[01:25] <jelmer> yes, should be authenticated
[01:25] <jelmer> would anonymous access make a difference in any way?
[01:26] <wgrant> Probably not on qastaging.
[01:30] <jelmer> do you know if there is a bug of some sort?
[01:30] <jelmer> ehm, bug /report/
[01:32] <wgrant> jelmer: Bug #636713
[01:32] <_mup_> Bug #636713: during sustained overload situation replies are truncated in production <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/636713 >
[01:33] <jelmer> wgrant: thanks!
[01:33] <jelmer> unfortunately my script can only run against qastaging at the moment, so I'll try again later
[01:35] <wgrant> Why can it only run against qas?
[01:35] <jelmer> it uses some new stuff that Ian introduced recently
[01:35] <jelmer> http://samba.org/~jelmer/recipe-status/
[01:36] <wgrant> It's not even on staging?
[01:36] <jelmer> has some of the output
[01:36] <wgrant> ah, the new recipe build collections?
[01:36] <jelmer> yep
[01:36] <jelmer> it wasn't on staging a couple of days ago, let me check again..
[01:36] <wgrant> Ah, staging's update failed overnight.
[01:36] <wgrant> Will look tomorrow.
[01:36] <wgrant> Too many incidents already this weekend.
[01:37] <jelmer> thanks, it does actually appear to be on staging now
[01:39] <wgrant> Odd.
[01:39] <wgrant> I was expecting staging to be down.
[01:40] <wgrant> Hmm.
[01:41] <wgrant> jelmer: You may be interested to know that they are on prod now.
[01:42] <wgrant> But they only show up in 1.0, not devel...
[01:43] <wgrant> Ah, no, they're in devel too, but you have to kill your cache.
[01:43] <jelmer> oh, cool
[01:43] <jelmer> they don't show up in launchpad.net/+apidoc/devel
[01:45] <wgrant> They do if you avoid your cache.
[01:45] <wgrant> eg. Ctrl+Shift+R.
[01:46] <jelmer> oh, wow. I didn't realize Launchpad did heavy caching on that too. I thought it was just the wadl.
[01:46] <wgrant> Neither did I.
[01:46] <wgrant> But I thought it was worth a try.
[01:48] <lifeless> jelmer: isn't https://bugs.launchpad.net/bugs/725811 a dupe?
[01:48] <_mup_> Bug #725811: expose ISourcePackageRecipeBuild.binary_builds on the web UI <api> <recipes> <Launchpad itself:Triaged> < https://launchpad.net/bugs/725811 >
[01:49] <jelmer> lifeless: a dupe of which other bug?
[01:49] <lifeless> ians work
[01:49] <wgrant> No.
[01:49] <jelmer> lifeless: No, Ian exposed the source package builds
[01:49] <lifeless> can you not traverse to the binaries ?
[01:49] <jelmer> this is about the binary builds created from those source builds
[01:50] <lifeless> yes, I get that
[01:51] <jelmer> lifeless: I can't find any way to traverse to the binaries, the source_package_recipe_build doesn't have anything that links it to the source package it created
[01:52] <wgrant> We should probably guess at that.
[01:52] <jelmer> wgrant: the source_package_recipe_build has a binary_builds attribute that's not exposed
[01:53] <jelmer> I currently parse the build log for the source package name and the version, and then call archive.getPublishedSources(source_name=source_name,version=version)[0].getBuilds()
[01:53] <wgrant> jelmer: Ah, so it already does guess.
[01:53] <wgrant> We should expose that and the guessed SPPH.
[01:53] <jelmer> wgrant: yep - the web page already shows that information
[01:54] <wgrant> Yay, fpc built correctly on i386 and amd64 this time.
[01:54] <jelmer> at least it works against production now :) e.g. http://samba.org/~jelmer/recipe-status/testing-cabal.html
[01:54] <jelmer> wgrant: fpc?
[01:54] <wgrant> jelmer: The last bit of fallout from https://wiki.ubuntu.com/IncidentReports/2011-02-25-Permissions-build-failures
[02:00] <lifeless> woo - statement count gropuing useful again
[02:00] <wgrant> Oh?
[02:00] <lifeless> you've killed enough of the checkwatches stuff
[02:00] <lifeless> 1992  OOPS-1883CCW1185  https://bugzilla.novell.com/show_bug.cgi?id=604251
[02:00] <wgrant> Almost.
[02:00] <lifeless> is still in the list
[02:00] <wgrant> Still got lots of ProtocolErrors.
[02:00] <wgrant> Hmm.
[02:00] <wgrant> That's odd.
[02:00] <wgrant> Looking.
[02:00] <wgrant> It should be mostly bugtracker-wide errors now.
[03:51] <wgrant> Woah.
[03:51] <wgrant> Looks like LP now uses the Light variant of the font...
[04:41] <lifeless> hmm
[04:41] <lifeless> I -really- have to fix this design issue preventing test optimisation with baseline fixtures.
[04:42] <lifeless> making a new keyspace and schema with cassandra is a little slow to do per-test.
[09:00] <LPCIBot> Project db-devel build #401: STILL FAILING in 5 hr 48 min: https://hudson.wedontsleep.org/job/db-devel/401/
[14:51] <maxb> The front-page of staging OOPSes - yay :-(
[17:57] <lifeless> maxb: what is hte oops ids
[18:04] <maxb> OOPS-1884S112844 for example, but it's entirely repeatable
[18:05] <maxb> geser mentioned an "S" OOPS id in #launchpad recently too
[18:07] <lifeless> maxb: the reason I asked for anexisting one is that the oops only sync every $hours on staging
[18:07] <lifeless> looking at gesers
[18:07] <lifeless> https://api.staging.launchpad.net/1.0
[18:14] <SpamapS> lifeless: o/
[18:14] <lifeless> oh hai
[18:14] <SpamapS> lifeless: so I wanted to setup a bug tracker for the packages...
[18:15] <lifeless> SpamapS: upload them once
[18:15] <SpamapS> and at one time I did have the project cassandra-packages registered
[18:15] <SpamapS> but now its like that project disappeared
[18:15] <SpamapS> I can't create a new cassandra-packages ... but I can add branches of it.
[18:15] <lifeless> 2010-06-16 Curtis Hovey: Project disabled and Clint Byrum (Clint Byrum) notified
[18:16] <lifeless> I can reenable it
[18:16] <SpamapS> Doh maybe I missed that notification
[18:16] <lifeless> but why not just upload to natty
[18:16] <SpamapS> its not suitable
[18:16] <lifeless> that will activate the package bug tracker
[18:16] <SpamapS> thrift is embedded..
[18:16] <SpamapS> avro..
[18:16] <lifeless> ok
[18:16] <SpamapS> :(
[18:16] <SpamapS> we're a lot closer to distro packages... but there are still things embedded. :-P
[18:17]  * SpamapS notes that the .debian.tar.gz is bigger than the .orig.tar.gz
[18:18] <lifeless> reenabled
[18:18] <lifeless> I suggest a more clear description and title
[18:18] <lifeless> 'e.g. 'staging for cassandra packages' and 'cassandra embeds too much stuff to be blah blah blah
[18:19] <lifeless> you'll want to configure the bug tracker
[18:19] <lifeless> SpamapS: ^
[18:21] <lifeless> SpamapS: it was a bit disapointing that cassandra can't do schema changes in *separate keyspaces* concurrently :>
[18:21] <lifeless> SpamapS: I have some /awfully/ crufty code you'll want to see :>
[18:23] <SpamapS> lifeless: everybody using cassandra has some awfully crufty code to deal with their still-evoling admin capabilities
[18:23] <lifeless> OTOH I have most of a cassandra based oops system
[18:23] <lifeless> but their inability to do range requests is plain annoying
[18:23] <lifeless> I've mailed our contact at riptano again :)
[18:24] <SpamapS> wait ranges for reads are supported
[18:24] <SpamapS> as long as you use OPP
[18:24] <SpamapS> which, given the OOPS focus, would make sense, and be only slightly confusing.
[18:24] <lifeless> not on secondary indices because they are bound to the node of the primary row
[18:25] <SpamapS> right. not much you can do there.
[18:25] <lifeless> sure there is; they can fix the broken assumption :>
[18:26] <lifeless> for now I've use a trick
[18:26] <lifeless> you have a column with value ''
[18:26] <lifeless> and you index it
[18:26] <lifeless> then you also index the thing you want ranged
[18:26] <lifeless> and you do a search for '' and $range
[18:27] <lifeless> it reads every row and does the range stuff as a loop
[18:27] <lifeless> this will suffice for a while
[18:27] <SpamapS> Right.. is that worth it?
[18:27] <lifeless> given I only need that for gc
[18:28] <lifeless> for aggregates and reports I'm keeping running totales
[18:29] <SpamapS> there's some rudimentary count suppot in 0.7 .. and 0.8 is supposed to have powerful counting
[18:29] <lifeless> one of the weirdnesses
[18:30] <lifeless> in twissandra they use a timestamp to build the timeline
[18:30] <lifeless> but timestamps - even @ microseconds - are not unique
[18:30] <lifeless> it might be /hard/ to collide, but get enough activity and frontends and it will happen
[18:31] <SpamapS> yeah there's some work on making it a vector clock instead of a straight microsecond timestamp
[18:31] <lifeless> see, I could ues that
[18:31] <lifeless> the other thing I could use is a ttl
[18:31] <lifeless> but open bug references need to impede gc
[18:31] <lifeless> I'm thinking of having two cfs
[18:31] <lifeless> identical
[18:32] <lifeless> but one referenced by bugs and no ttl
[18:32] <lifeless> that would leave aggregated cruft behind though, because AIUI I wouldn't get a callback on ttl expiry
[18:33] <SpamapS> Couldn't you just remove the ttl when a reference is reated?
[18:33] <lifeless> wasn't 100% sure that doing a new write with no ttl would do that
[18:33] <lifeless> and was coding at us unfriendly times
[18:33] <lifeless> SpamapS: so I reactivated https://launchpad.net/cassandra-packages
[18:34] <lifeless> SpamapS: but you need to click on the bug tracker and configure it
[18:34] <lifeless> it thinks you don't use LP at the moment
[18:34] <SpamapS> lifeless: thanks. :) I looked in my nbox and I must have inadvertently deleted the notification
[18:38] <lifeless> are you familiar with the bug settings bit I'm refering to ?
[18:38] <SpamapS> very
[18:38] <lifeless> kk
[18:38] <SpamapS> just having dinner now.. somebody reminded me that there is no food to be had in capetown this late on sunday ;)
[18:39] <lifeless> I'm just polling till I can file these bugs for you :)
[18:39] <SpamapS> EAGAIN
[18:43] <SpamapS> lifeless: ok, bug reporter configured
[18:50] <lifeless> cool
[18:51] <lifeless> want all the things I mailed you as bugs?
[18:54] <lifeless> SpamapS: ^
[18:54] <SpamapS> yes please!
[18:55] <SpamapS> I think some of them will be solved by doing actual lucid/maverick backports rather than copying the bin packages
[19:00] <lifeless> done
[19:00] <lifeless> SpamapS: ^
[19:00] <SpamapS> awesome... I sent my reply as well
[19:00] <SpamapS> to your email I mean
[19:00] <lifeless> cool
[19:01] <SpamapS> These Afrikaaners have filled me up with delicious food.. hopefully I'll pass out soon. :)
[19:03] <lifeless> nice
[19:06] <SpamapS> its good to see that you're making some headway w/ the cassandra packages.
[19:06] <SpamapS> I was beginning to wonder if anybody would ever actually use them. :)
[19:08] <lifeless> heh
[19:08] <lifeless> so the only critical one is the classdefnotfound
[19:08] <lifeless> not being able to compact is, uhm, operationally significant
[19:12] <SpamapS> indeed
[19:12] <SpamapS> $ ./get-ppa-stats cassandra-ubuntu stable
[19:12] <SpamapS> cassandra	0.7.0-0ubuntu1	42
[19:12] <SpamapS> 42 downloads.. not too shabby ;)
[19:25] <maxb> Where does get-ppa-stats live?
[19:28] <SpamapS> in my home dir ;)
[19:29] <SpamapS> actually
[19:29] <SpamapS> lp:~clint-fewbar/+junk/lptools
[19:29] <SpamapS> maxb: ^^
[19:30] <SpamapS> lifeless: one point of pain right now is that they up and changed the build system entirely from 0.7.0 -> 0.7.1
[19:37] <maxb> thanks
[19:38] <lifeless> SpamapS: of course they did
[19:39] <lifeless> SpamapS: get-ppa-stats should be in lptools ;)
[19:43] <SpamapS> ok.. passing out eminent
[19:50] <lifeless> night
[20:16] <lifeless> hi flacoste
[20:17] <flacoste> hi lifeless
[20:19] <lifeless> flacoste: I spiked a oops-in-cassandra in the weekend; its on my local disk only at the moment
[20:19] <lifeless> flacoste: ok with you if I make a project in lp, agpl3 the code and push publicly?
[20:19] <lifeless> flacoste: also, how was your flight?
[20:20] <flacoste> lifeless: fine by me
[20:20] <flacoste> lifeless: my flight was awesome, even though it started by a cancelling flight
[20:20] <lifeless> ><
[20:20] <flacoste> cancelled
[20:20] <lifeless> so the replacement they upgraded you or something ?
[20:21] <flacoste> lifeless: kind of, instead of flying continental to London, i got an air canada flight
[20:21] <lifeless> oh nice
[20:21] <flacoste> and i managed to use one of my upgrade ticket
[20:21] <lifeless> \o/
[20:21] <flacoste> so i first business for the first time
[20:21] <flacoste> slept better than at home, 5 hours straight without interruptions
[20:21] <lifeless> no kids :)
[20:21] <flacoste> exactly
[20:21] <flacoste> :-)
[20:22] <flacoste> and on the london cape town flight, it wasn't fully booked, so i got an empty seat besides me
[20:22] <flacoste> which made sleeping also easier
[20:22] <flacoste> so one of the best flight experience overall!
[20:22] <flacoste> feel fresh and ready for this week sprint
[20:24] <lifeless> excellent
[20:24] <lifeless> we have fresh water again, came on on friday
[20:24] <lifeless> :)
[20:25] <flacoste> that's always useful
[20:25] <lifeless> very
[20:33] <lifeless> all software sucks ;)
[21:03] <wallyworld> thumper: mumble?
[21:07] <lifeless> wallyworld: two bugs you might like to know about - model defects exposed by your nice patch to show merge proposals in the revision lists for branches
[21:07] <lifeless> https://bugs.launchpad.net/launchpad/+bug/726190
[21:07] <_mup_> Bug #726190: bugs shown against merge proposals in revision list lists all bugs ever linked to branch and differ to those the MP itself show <code-review> <Launchpad itself:Triaged> < https://launchpad.net/bugs/726190 >
[21:07] <lifeless> https://bugs.launchpad.net/launchpad/+bug/726195
[21:07] <_mup_> Bug #726195: merge proposals show closed bugs <code-review> <Launchpad itself:Triaged> < https://launchpad.net/bugs/726195 >
[21:13] <lifeless> rhythmbox just rickrolled me
[21:27] <lifeless> mornign wgrant
[21:27] <lifeless> hmm, no sinzui
[21:29] <lifeless> anyone know if there is a bug about things like ubiquity showing up in +needs-packaging at all ?
[21:30] <wgrant> lifeless: What's special about ubiquity?
[21:30] <wgrant> It still has a project, doesn't it?
[21:31] <lifeless> no
[21:31] <lifeless> you may be htinking unity
[21:31] <lifeless> dpkg, ubiquity, apport - those things the distro folk often explicitly do not want 'upstream' projects
[21:31] <wgrant> Erm.
[21:31] <wgrant> Have you seen devel lately?
[21:32] <wgrant> It needs a new apt-ftparchive that is not everywhere yet.
[21:32] <wgrant> I suspect we want to roll that back.
[21:32] <lifeless> you just changed topic, right ?
[21:32] <wgrant> I did.
[21:33] <wgrant> So, there is a bug about how we don't really support Ubuntu-native packages well at all.
[21:33] <lifeless> for context
[21:33] <lifeless> I'm looking at https://bugs.launchpad.net/launchpad/+bug/722794
[21:33] <_mup_> Bug #722794: DistroSeries:+needs-packaging timeouts <dba> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/722794 >
[21:33] <lifeless> which does crazy stuff to get bugs and strings
[21:33] <lifeless> and bug heat?!
[22:10] <wallyworld> lifeless: thanks for the heads up on those bugs - that post implementation clean up stuff is on my radar for this maintenance phase
[22:11] <lifeless> wallyworld: the timeout I mentioned last week preexisted
[22:11] <lifeless> wallyworld: I have a branch playing to amliorate it
[22:11] <wallyworld> lifeless: yeah, i had a quick look and had come to the same conclusion
[22:11] <lifeless> wallyworld: https://code.launchpad.net/~lifeless/launchpad/bug-710685/+merge/51258
[22:12] <wallyworld> was going to get in today and try and fix it but you already are on it
[22:16] <huwshimi> Uh what? 0 tests run in 4:12:24.883708, 0 failures, 0 errors. It took four hours to do exactly what then?
[22:16] <StevenK> Haha. That's the question.
[22:21] <wgrant> huwshimi: Bug #726243
[22:21] <_mup_> Bug #726243: ec2 no longer keeps track of failed tests <Launchpad itself:Triaged by wgrant> < https://launchpad.net/bugs/726243 >
[22:24] <huwshimi> wgrant: Oh, that's not very helpful.
[22:25] <wgrant> It does at least detect whether it has failed or not, though.
[22:27] <huwshimi> wgrant: How do I know what failed then? The log is not obviously showing anything that went wrong, but there's a lot of the log that I haven't looked at :)
[22:27] <wgrant> huwshimi: You'll need to grep it.
[22:27] <wgrant> Or run subunit/testr over it
[22:27] <huwshimi> wgrant: For what?
[22:30] <wgrant> I presume there's some subunit tool, but I tend to search for "error:" and "failure:", or just pipe it into 'testr load'.
[22:32] <StevenK> Grab the log from the e-mail and: zcat <file> | subunit-filter --no-passthrough | subunit-ls
[22:33] <huwshimi> I'm guess it is this: http://paste.ubuntu.com/573231/
[22:33] <StevenK> wgrant: subunit *tool*? Why have one confusing UI when you can have ten!
[22:34] <wgrant> huwshimi: I fixed that on Friday.
[22:34] <wgrant> huwshimi: If it's the only failure, lp-land it.
[22:34] <wgrant> Except that we're in testfix.
[22:34] <wgrant> Because buildbot doesn't have the new apt.
[22:34] <wgrant> And I guess we are ISless today.
[22:34] <huwshimi> wgrant: Ah right, it appears to be. Do I need to merge devel first?
[22:35] <wgrant> No.
[22:35] <lifeless> wgrant: the new apt is in our ppa
[22:35] <lifeless> wgrant: IS will be here when spm shows up
[22:35] <StevenK> Given he flew back from London on Saturday? You tell funny jokes.
[22:35] <wgrant> lifeless: Right, but given that he may well still be in the air...
[22:36] <lifeless> he should get in at 9am canberra time or thereabouts
[22:36] <lifeless> and he will want to stay up to beat jetlag
[22:36] <lifeless> [whether he will succeed... different question]
[22:37] <wgrant> Heh
[22:39] <wgrant> lifeless: What does subunit.TestProtocolServer do? You give it a result object and a stream, and it plays the stream's events through the result?
[22:40] <wgrant> Ah, no, that stream is something else...
[22:42] <lifeless> wgrant: pydoc subunit
[22:43] <lifeless> '    Subunit has support for non-blocking usage too, for use with asyncore or
[22:43] <lifeless>     Twisted. See the ``TestProtocolServer`` parser class for more details.
[22:43] <lifeless> '
[22:43] <lifeless> wgrant: what do you want to achieve?
[22:43] <wgrant> lifeless: Just working out how ec2test.remote does its stuff.
[22:44] <wgrant> So I can unbreak it.
[22:44] <lifeless> ok
[22:44] <lifeless> well  - class TestProtocolServer( - find that section in pydoc
[22:44] <lifeless> I just reread it and it seems useful to me :)
[22:46] <lifeless> BranchSet:CollectionResource:#branches never completes in < 4 seconds :(
[22:53] <LPCIBot> Project devel build #481: STILL FAILING in 5 hr 56 min: https://hudson.wedontsleep.org/job/devel/481/
[22:53] <wgrant> StevenK: Jenkins needs the new apt.
[22:54] <wgrant> StevenK: And you need to tell me what the checkwatches.txt failure is, since it 404s in the web UI...
[22:57] <StevenK> Yes, it doesn't like doctests.
[22:58] <wgrant> :(
[22:59] <StevenK> http://pastebin.ubuntu.com/573238/
[22:59] <wgrant> Thanks.
[22:59] <StevenK> Just checking the current build slave out
[23:00] <StevenK> ubuntu@ip-172-56-124-250:~$ COLUMNS=150 dpkg -l apt | tail -n 1
[23:00] <StevenK> ii  apt                             0.7.25.3ubuntu9.3+ppa1          Advanced front-end for dpkg
[23:02] <StevenK> wgrant: Is that the new apt? ^
[23:02] <wgrant> I presume so.
[23:02] <wgrant> But looking.
[23:03] <wgrant> Yes
[23:03] <wgrant> :(
[23:04] <StevenK> Hm?
[23:04] <wgrant> Well, if it has the new apt and is still failing...
[23:04] <StevenK> Oh, sorry.
[23:04] <StevenK> That output is from the current build slave
[23:04] <StevenK> As in, the test run that started ~ 30 minutes ago
[23:05] <wgrant> Ah. I guess we'll see.
[23:05] <wgrant> Thanks.
[23:15] <lifeless> whats the ? to set webservice batch size ?
[23:18] <wgrant> ?ws.size
[23:18] <lifeless> thanks
[23:19] <lifeless> hmm, I think I need more sample branches to fix properly.
[23:19] <lifeless> shrug
[23:20] <michaelh1> Morning.  Is there a URL to get the latest download release from a series?  I'm writing a Makefile and would like to point wget at one URL that always resolves to the latest tarball.
[23:20] <michaelh1> (such as the latest from the 4.5 series at https://launchpad.net/gcc-linaro/+download)
[23:22] <lifeless> I don't know
[23:22] <lifeless> you could use the uwatch stuff that packages (like bzr) use to track upstream
[23:23] <lifeless> wgrant: do you happen to know where the top level collections are glued into the web service?
[23:23] <wgrant> lifeless: LaunchpadRootNavigation?
[23:24] <michaelh1> lifeless: OK, ta.  I'll file a wishlist feature request
[23:24] <StevenK> michaelh1: I'd suggest uwatch as the easiest way currently
[23:25] <lifeless> indefinitely
[23:25] <wgrant> lifeless: export_as_webservice_collection() makes it appear in launchpadlib, though.
[23:25] <lifeless> realistically, such a link is going to be a long way away
[23:25] <lifeless> we're sorting out many rather heavy lifting issues atm
[23:25] <lifeless> wgrant: yes, I see.
[23:25] <lifeless> wgrant: and then
[23:25] <lifeless> *another* decorat
[23:25] <lifeless>  @collection_default_content
[23:25] <lifeless> to select what is shown.
[23:26] <lifeless> This is just so byzantine.
[23:26] <wgrant> Right.
[23:30] <StevenK> Ran 56 tests with 0 failures and 48 errors in 17.653 seconds.
[23:30] <StevenK> "Bugger"
[23:31] <wgrant> At least it was quick.
[23:34] <StevenK> Total: 77 tests, 2 failures, 3 errors in 2 minutes 13.821 seconds.
[23:34] <StevenK> Progress!
[23:47] <huwshimi> Is anyone else getting make errors (ImportError: No module named widgets) when they try and make on a new branch?
[23:48] <lifeless> run utilities/update-sourcecode
[23:49] <lifeless> wgrant: it would be nice for folk if when changing shipit you make a small song and dance on the dev list
[23:59] <huwshimi> lifeless: Thanks all working now.