[12:00] i'm assuming they are working and i'm not, in fact, breaking pages :-) [12:03] that's the problem with tests -- you can only rely on them to the extent that they themselves have enough coverage :) === limi loves tests too [12:06] we always write our tests before our code in our company now (really) [12:06] it rocks [12:06] I do too; can't do that with functional doctests though. :) [12:06] no, of course not :) === sabdfl [~mark@host217-37-231-28.in-addr.btopenworld.com] has left #launchpad [] === sabdfl [~mark@host217-37-231-28.in-addr.btopenworld.com] has joined #launchpad === sabdfl [~mark@host217-37-231-28.in-addr.btopenworld.com] has left #launchpad [] === sabdfl [~mark@host217-37-231-28.in-addr.btopenworld.com] has joined #launchpad === BradB is now known as BradB|away [12:53] night guys === sabdfl [~mark@host217-37-231-28.in-addr.btopenworld.com] has left #launchpad [] [01:00] Kinnison: ping [01:10] yes? [01:12] cprov: pong [01:17] so how do I find a project by its name ? [01:17] when I try to execute the launchpad's tests I get this error: [01:17] Traceback (most recent call last): [01:17] File "test.py", line 27, in ? [01:17] from zope.app.tests.functional import FunctionalTestSetup [01:17] File "/home/carlos/Work/dists/launchpad2/lib/zope/app/tests/functional.py", line 32, in ? [01:17] from ZODB.DB import DB [01:17] File "/home/carlos/Work/dists/launchpad2/lib/ZODB/__init__.py", line 21, in ? [01:17] from persistent import TimeStamp [01:17] File "/home/carlos/Work/dists/launchpad2/lib/persistent/__init__.py", line 19, in ? [01:17] from cPersistence import Persistent, GHOST, UPTODATE, CHANGED, STICKY [01:17] ImportError: No module named cPersistence [01:18] which python module should I install? (It was working before merging with rocketfuel) [01:21] carlos: Have you built Zope's C modules? [01:22] lifeless: With the SQLObject? [01:22] well I had a convenience mapper, where I could just go 'mapper.findByName('foo')' [01:22] but thats gone away... and I just found that not all the code using it was converted. [01:22] Well; on the assumption that cprov isn't gonna wake up [01:23] lifeless: That's still there, hiding in soyuz/importd.py, according to ctags... :) [01:24] You should be able to say "from canonical.database import Project; foo = Project.byName('foo')", but someone needs to set alternateID=True in the declaration of the name column. [01:24] spiv: seems like that was the problem, thank you. I did not saw that zope was updated since my last rebuild... [01:24] Project.byName('foo') would be good. [01:25] Product.byNameAndProject('bar', foo) [01:25] would be the next request. [01:25] lifeless: I think I'm going to have to run importd on my laptop to track down this taxi bug... my guesses at causes haven't found any problem.s [01:25] spiv: ok. [01:25] its easy to setup, or at least, I've always found it easy. [01:26] step 1 - check out a buildbot config. [01:26] lifeless: There's a wiki page of something describing how? [01:26] 1. got that already :) [01:26] spiv: yes [01:26] ImportProcess [01:27] Ok, I'll try that. I recall getting those instructions to work on my desktop a few months ago... [01:27] now, there are a couple of minor nits with that page. [01:27] we've changed to reading from the database - but you can ignore that [01:28] taxi in the slave is not affected by that change. === spiv nods [01:31] basically, let me know of any irregularities, and I'll fix them with you right now and update the wiki page [01:32] Exception exceptions.TypeError: in > ignored [01:32] has started happening for me.. just a heads up [01:32] Yeah, I've seen that too. [01:32] It's a real pita to track down. [01:33] Because it happens during Python's shutdown, so things liek teh sys module aren't available any more... [01:33] Makes debugging tricky :) [01:33] ewww [01:33] import pdb; import sys; :) [01:33] lifeless: You'd think so, wouldn't you? ;) === debonzi [~debonzi@200.158.100.251] has left #launchpad ["Leaving"] [01:46] lifeless: pqm is not sending success/failure email to me anylonger . can you verify why ? the last time on Soyuz sprint I sent hin an unsigned message, after that no messages are comming from my PQM request, but they still working fine [01:49] lifeless: email me if you have news about it, tks [01:57] spiv: well, there goes the mapper. [01:57] :[ [02:01] spiv: got it going ? [02:02] lifeless: Just getting to that now... [02:03] ok. [02:03] I think my lappy is gonna meltdown [02:04] Not enough fans? ;) [02:04] 15 minutes on 100% cpu, and its boiling hot here right now [02:04] was 41' in north ryde two days back [02:04] Ouch. [02:07] just about burn my hand in the airstream from the fans [02:12] Hmm. The slave won't start for me. [02:12] Fails in arch._twisted in _spawn on assert not threadable.isInIOThread() [02:13] ewww. [02:14] that sounds like either a) twisted has changed breaking davids test suite (try make check in pyarch), or b) its not making the thread in buildbot_slavecommand.py in importd/ [02:16] make check in pyarch has one failure for me: ERROR: Changeset.get_index supports escaped file names. [02:16] Otherwise, it passes. [02:17] ok, that sounds like you have an old tla. [02:17] shouldn't matter for this. [02:17] 'shouldn't. [02:17] so, when does it fail - twistd -f buildbot.tap, or when you 'force job' ? [02:17] The former. [02:18] the slave doesn't do any arch stuff on startup. [02:18] (I have an integration tla from a few days ago, I think...) [02:18] unless you'd already done a force job, which is queued. [02:18] got a backtrace on the failure? [02:18] Oh, my bad, I think this is in botmaster. [02:18] oh ok. [02:19] I know what that will be. the master *is* running the io thread. [02:19] do this. [02:19] lifeless: do we have any schedule for arch.gnome.org? [02:19] in master.cfg, before the 'scan arch directories' code, [02:19] import arch._tla [02:20] arch._tla.spawn=arch._tla.PyArchSpawningStrategy('tla') [02:20] that should fix it. [02:20] carlos: no. [02:25] ok [02:26] Oops, need a cvsmail dir... [02:28] Ok, it's running... [02:29] Hmm, the slaves aren't connecting... [02:30] Ok, the port number on the wiki instructions is wrong... [02:31] check master.cfg [02:32] I'm noting all these down to document later. [02:33] What should I be looking for? builders is being passed to processDB... [02:34] Oh, hmm. [02:34] It has paths like '/home/robertc/source/canonical/buildbot/launchpad/botslave/' :) [02:34] oh right. === spiv tries the obvious thing... [02:34] that is new [02:35] in your botslave dir [02:35] mkdir gpg [02:35] mkdir mirrorarchives [02:35] set the other path to point at mirrorarchives [02:36] and in the gpg dir create a keypair [02:37] ddaa: btw, .netrc isn't needed, curl already does the auth with the current config. [02:37] you have to put it in the url - https://foo@bar:host/ [02:37] and it will just work. [02:37] Do you know the gpg command off-hand? [02:38] gpg --no-default-keyring --keyring ./arch@canonical.com.pub --secret-keyring ./arch@canonical.com.secret --gen-key [02:38] it must have arch@canonical.com as the uid [02:41] [Broker,client] setBuilderList [] [02:41] ddaa: and no, I wrote the buildbot conversion logic. bob2 did taxi. [02:41] spiv: hmm. no builders found. [02:41] Oh, and the master says: [Broker,0,127.0.0.1] slave trogslave has leftover directories (gpg,mirrorarchives): you can delete them now [02:41] try visiting /reload on buildbot [02:42] Ooh. [02:42] I got an error in my new stuff. [02:42] hmm ? [02:43] you know I need more detail to help :) [02:43] I'm checking it's not my fault ;) [02:44] I'm saving you for the important questions ;) [02:44] It's from the initZopeless call in jobstuff... [02:45] Hmm, now an SQLObject error. [02:45] Oh, no, my bad :) [02:46] whats the big O for list.append ? [02:48] O(1) [02:49] Python lists are basically like a vector ;) [02:51] constant time ? [02:51] serious ? [02:51] Amortized, of course. [02:52] cause buildbot slows /waaaaay/ down when the log gets long [02:52] and I mean /way/ [02:52] Intriguing. [02:53] From listobject.c in Python: [02:53] /* This over-allocates proportional to the list size, making room [02:53] * for additional growth. The over-allocation is mild, but is [02:53] * enough to give linear-time amortized behavior over a long [02:53] * sequence of appends() in the presence of a poorly-performing [02:53] * system realloc(). [02:53] * The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... [02:53] */ [02:53] (in list_resize) [02:54] ah, linear != O(1) [02:54] that O(n) [02:55] O(1) is constant time [02:55] i.e. searches in a trie are constant time if the maximum length allowed in the trie is set. [02:57] STL vectors do amortised constant time insertion at the end, linear time insertion at beginning or middle [02:58] lifeless: If you are trying optimizations, it might work if you initialized the list to some big chunk using range(0,100000) and keep track or the next insert index [02:58] That comment is actually a bit ambiguous. [02:58] stub: I'll profile first. [02:59] if its the list, I'll probably just throw a splay container in instead of a list. [02:59] I'm not sure if it's referreing to the sequence of appends being linear time, or the individual appends... [03:00] dictionaries or sets might be useful aternatives too [03:00] stub: it needs to be ordered, so a dictionary with the index as the first key is a solid option. [03:01] spiv: I'm could have sword it was O(1) with some additional overhead when it needed to realloc [03:01] we are talking about /big/ lists here [03:02] its highly likely that the < double copy behaviour starts to dominate. [03:02] stub: Ditto. [03:03] spiv: whats the current error ? [03:04] stub: I just found a post by Tim Peters asserting that it's amortised O(1), so that's good enough for me ;) [03:04] lifeless: Still no builders. [03:04] spiv: what was that error you encountered ? (does /reload work) [03:05] Even after making reload work by fiddling with the initZopeless stuff. [03:05] ok. what in your master.cfg - processDir or processDB ? [03:06] processDB. [03:06] ok, run your luanchpad, goto the do not use prject, unassigned jobs. [03:07] pick a52dec from the +sources lust [03:07] clik on 'enable' up the top, then down the bottom choose a project - say 'ubuntu'. [03:07] this will fail, but its what you need to do :) [03:07] I have a patch in draft here. [03:07] or... [03:07] comment out the processDB and infoIMporter calls in master.cfg [03:08] uncommment the processDir call. === spiv does that [03:10] Ok, lots of jobs on the web page now... [03:10] Although the slave still says [Broker,client] setBuilderList [] [03:10] now thats bizarre [03:10] bounce the slave. [03:13] No change. [03:13] no way. [03:13] refresh /importd/status [03:13] It still reports them all as "offline" inthe web page.. [03:14] offline means the slave hasn't attached. [03:14] I already have :) [03:14] oh. [03:14] I know. [03:14] in jobstuff.py, there is a slave name in the builder [03:14] change that (lifelesswks) to (spivsslavename) [03:14] Ah. [03:15] That looks likely :) [03:15] a52decv is a nice small one === limi [~limi@212.80-202-72.nextgentel.com] has left #launchpad [] [03:16] Oops, updated the wrong part of jobstuff.py... [03:17] Ah, "idle" :) [03:17] great [03:17] Hmm, and error... [03:17] Error during call to `vu_chdir' for /srv/importdtest/archives/ (No such file or directory) [03:18] (from the slave...( [03:18] Ah, that's easy.. [03:18] master.cfg again. [03:18] unchanged path in master.cfg === spiv nods === spiv hmms [03:21] Hmm, it expects "archives" in my slavedir, not "mirrorarhives" [03:22] oh, there are two paths, it depends on how you set them [03:22] exceptions.AttributeError, 'WorkingTree' object has no attribute 'logger' [03:22] ??? [03:22] have you got the latest cscvs and buildbot patches from rocketfuel ? [03:23] Hmm, maybe not cscvs... [03:23] heh. [03:23] that will be it [03:23] Ok, I was missing a couple from cscvs :) [03:23] ok, the two paths for me are: [03:24] '/home/robertc/source/canonical/buildbot/launchpad/botslave/', '/home/robertc/source/canonical/buildbot/launchpad/botslave/mirrorarchives/', [03:24] the first path is where it makes the archives, and gets the gpg key [03:24] the second path is where it makes the public mirrors of the archives. [03:24] I left the paths as is, but had to mkdir "archives" in the first to make the error go away. [03:24] the second cannot be the first + 'archives' [03:24] yes, thats fine [03:25] Ok, it's actually doing stuff. [03:25] "creating changesets", etc :) [03:26] great. [03:26] I presume I just need to wait, now? [03:26] to see the slowdown I mention, in buildbot/importd/buildbot_slavecommand.py, change the debug level - its at WARNING now I think [03:26] if you make it INFO, then run a big job... weeooo. [03:27] in the buildbot console [03:27] web page [03:27] thing [03:27] do a refresh [03:27] on the waterfall display of that job [03:27] I'm at http://localhost:8000/importd/status/a52dec-HEAD-import.job/events/5/log [03:27] And it's still streaming... [03:27] hit back [03:27] then refresh [03:28] when you get an exception there, you are ready for the next step my chjild. [03:28] Yeah, but I shouldn't get an exception until the log finishes streaming... :) [03:28] right. [03:28] if you want to watch the stream, go right ahead :) [03:32] Well, I'm watching the throbber ;) [03:32] mmmm, throbber [03:39] ok, I'm going for a shower. [03:39] then lunch, then RMS. [03:39] that a52dec should have finished by now tho. [03:42] Nope. [03:42] No action for quite a while... [03:42] Last output was: A libao/audio_out_null.c [1.1] [03:43] check the slaves log [03:43] Nothing. [03:43] if the last thing is roughly 'problem encountered', you've hit a bug [03:43] serious? nothing in the botslave/twistd.log ? [03:43] No new lines in that log for almost 20 minutes. [03:43] 2004/10/15 03:24 CEST [Broker,client] startCommand:job [id 1 760629] [03:43] 2004/10/15 03:24 CEST [Broker,client] method 'runJob' with args [] , kwargs {'dir': '/home/andrew/warthogs/code/dists-buildbot/launchpad/trogslave/buildbot-jobs/a52dec-HEAD-import.job'} [1 760629] [03:44] what is the last entry in there then ? [03:44] now, you aren't using tail -f are you ? [03:44] I'm mainly using less, with the magical G key... [03:45] try a 'tail' on its own [03:45] But that's definitely the lsat thing in that file :) [03:45] twistd often breaks log observers I've foudn :() [03:45] ok, well thats plain weird and sucky. [03:45] Confirmed... [03:45] kill the slave [03:45] spiv: how could I execute a SELECT DISTINCT with a object.select(...) ? [03:46] in cscvs/modules/CVS/__init__.py, line 372 [03:46] change "-d%s" to "-d%s<" [03:46] carlos: you currently can't... [03:47] then start the slave and try again. [03:47] carlos: The current workaround is to wrap a Set around it. See SQLObjectGuide's notes on DISTINCT. [03:47] ok [03:47] spiv: thanks [03:47] also change the log level in buildbot/importd/buildbot_slavecommand.py from INFO to WARNING [03:52] spiv: hows that looking ? [03:53] lifeless: It's doing stuff :) [03:53] N changeset MAIN.58, etc [03:55] ok [03:57] Ooh, an exception... [03:57] File "/home/andrew/warthogs/code/dists-buildbot/launchpad/lib/cscvs/cmds/totla.py", line 508, in totla [03:57] raise ValidationFailed("cannot cross_check yet") [03:57] cscvs.cmds.totla.ValidationFailed: cannot cross_check yet [03:58] Interesting, but not mine as far as I can tell :) [04:01] oh right [04:01] add this: [04:02] launchpad/lib/canonical/sourcerer/util to the pythonpath for the slave [04:03] Ok. [04:04] _sqlite.OperationalError: database is locked [04:05] the change won't take effect until you kill off all the slave threads [04:05] ... [04:05] I killed and restarted the slave... I have to stop and restart the botmaster too? [04:05] no [04:06] ps x | grep twistd [04:06] there are old slave threads still running [04:06] I'm shutting down now, ring me if there is other issues. [04:07] Ok. [04:07] I'll have to sleep soon anyway. [04:08] Ok, back to creating changesets... [04:08] Hmm.. [04:09] File "/home/andrew/warthogs/code/dists-buildbot/launchpad/lib/cscvs/cmds/totla.py", line 396, in enforceTargetNewOrCleanAndUpdated [04:09] assert (not target_tree.has_changes()) [04:09] exceptions.AssertionError: [04:09] lifeless: I guess you've alrady really gone? :) [04:24] stub: ping === Signon time : Wed Oct 13 08:43:40 2004 === Signoff time : Fri Oct 15 04:33:35 2004 === Total uptime : 1d 19h 49m 55s === warthylog [~warthylog@port49.ds1-van.adsl.cybercity.dk] has joined #launchpad === Topic for #launchpad: lunchpad: home of the sandwich artists | fogo na bomba | "qorking along happily" === Topic (#launchpad): set by daf at Tue Oct 12 23:03:04 2004 [10:14] lifeless: it's a pita to see you and spiv struggling with buildbot, while it would have been a simple matter of running make with my botmaster tree. [10:15] lifeless: I still do not understand why you object to having an official testing botmaster tree, that would save everyone time and frustration. [10:16] lifeless: also, please do not say "no" in the middle of some unrelated discussion to answer something I said 12 hours ago. I know that bob2 wrote taxi.py, I have no idea what you were answering to. === sabdfl [~mark@host217-37-231-28.in-addr.btopenworld.com] has joined #launchpad === Kinnison [~dsilvers@haddenham.pepperfish.net] has joined #launchpad [11:13] sabdfl: Classes for database tables should be CamelCase or just Initialcapital ? [11:14] CamelCase === Kinnison nods and corrects daniel's overzealous decamelling :-) [11:16] I guess he got the wrong end of the stick wrt Sourcepackage [11:16] and didn't notice us agree to SourcePackage === stub [~stub@dsl-246.248.240.220.dsl.comindico.com.au] has joined #launchpad [12:11] >>> PackagePublishing.get(9).distroarchrelease.distrorelease.name [12:11] u'warty' [12:11] I'm finally getting the hang of this SQLObject stuff [12:15] Anyone here an sqlobject guru who can tell me why: PackagePublishing.selectBy(distroarchrelease=1) gives me: KeyError: 'distroarchrelease' given that I have: ForeignKey(name='distroarchrelease', foreignKey='DistroArchRelease', dbName='distroarchrelease'), [12:15] ? [12:23] Because in that syntax, distroarchrelease should be a DistroArchRelease object, not an int. You want selectBy(distroarchreleaseID=1) I think [12:23] PackagePublishing.selectBy(distroarchrelease=DistroArchRelease.get(1)) [12:23] gives me the same error [12:24] and you might want to do selectBy(PackagePublishing.q.distroarchrelease == 1) [12:26] >>> PackagePublishing.selectBy(PackagePublishing.q.distroarchrelease=1) [12:26] SyntaxError: keyword can't be an expression [12:26] ==, not = [12:26] that gives a different traceback but still a KeyError [12:29] Check the definition of PackagePublishing - make sure it has that column defined and the case is correct [12:30] I don't know the technical difference between select and selectBy - I've only used the former [12:31] ForeignKey(name='distroarchrelease', foreignKey='DistroArchRelease', dbName='distroarchrelease'), [12:31] that's what's in PackagePublishing._columns [12:33] aha [12:33] I am supposed to selectBy(distroarchreleaseID=DistroArchReleaseInstance) [12:33] eh? hmm [12:34] oh, maybe not === Kinnison gets different errors for that [12:34] That might work because DistroArchReleaseInstance is castable to an int [12:35] oh no; that gives me: [12:35] ValueError: Unknown SQL builtin type: for [12:36] right; so PackagePublishing.selectBy(distroarchreleaseID=1) [12:36] works [12:37] >>> PackagePublishing.selectBy(distroarchreleaseID=1)[0] .section.name [12:37] u'universe/games' [12:38] so we're getting somewhere [12:38] Kinnison: isn't universe the component? [12:39] >>> PackagePublishing.selectBy(distroarchreleaseID=1)[0] .component.name [12:39] u'universe' [12:39] I imagine that's a gina-needs-teaching issue [12:39] although it's probably come straight out of katie; so I guess it should stay === Kinnison hugs sabdfl for harness.py [12:40] it's *so* useful === SteveA|way is now known as SteveA [12:45] Hey SteveA, aren't we meant to have had a meeting by now? [12:45] Oh no, my ability to count is broken === Kinnison blames late nights and python [12:50] spiv: back === BradB|away is now known as BradB [12:55] Hey brad [12:56] hi [12:59] Launchpad meeting in 30 seconds! [01:00] SteveA: you said 12:00 UTC [01:00] oh, right.. [01:00] darn timezones [01:00] dsilvers@petitemort:~$ TZ=:UTC date [01:00] Fri Oct 15 11:00:21 UTC 2004 [01:00] it's 11:00 UTC isn't it? [01:00] hien!? [01:00] i thought 12 UTC was now [01:00] nah [01:01] `date -u` [01:01] wow, i guess the time just changed [01:01] SteveA: where do we have meetings? [01:01] here [01:01] right [01:01] right here === BradB goes back to sleep for an hour === BradB is now known as BradB|away [01:02] sabdfl: I'm going to put SourcePackageReleaseFile and BinaryPackageFile into publishing.py unless you'd rather they went into library.py or somewhere else entirely [01:06] Kinnison: no, please either packages.py or library.py === Kinnison thinks packages.py then [01:06] what about files.py? === limi [~limi@212.80-202-72.nextgentel.com] has joined #launchpad [01:07] I suppose I could do a files.py with them === cprov [~cprov@200.158.100.251] has joined #launchpad [01:13] hey cprov === carlos [~carlos@69.Red-80-33-181.pooles.rima-tde.net] has joined #launchpad [01:15] morning === warthylog [~warthylog@port49.ds1-van.adsl.cybercity.dk] has joined #launchpad === Topic for #launchpad: lunchpad: home of the sandwich artists | fogo na bomba | "qorking along happily" === Topic (#launchpad): set by daf at Tue Oct 12 23:03:04 2004 [01:42] (stub/#launchpad) Yup [01:42] (stub/#launchpad) Trivial rename since nobody is importing it anymore [01:42] it is all using names? [01:43] Yup. Might as well actually use some of the infrastructure Z3 gives us ;) [01:43] ok. so, my next question is, are vocabularies sufficiently different from database/domain objects to be put in one place on their own? [01:45] Nope, although a number of them inherit common behaviour from a base class [01:45] we have some different kinds of thing that we're considering as sufficiently different to be in their own distinct places in the source tree: interfaces of database objects, database object implementations, page templates, zcml glue [01:46] what we don't yet have a good place for is the adapters and other code that does not fall into those categories [01:47] this code is currently in canonical/rosetta, canonical/malone etc. [01:48] SteveA: i originally tried a vocabulary package with an __init__.py that imports the separate files [01:48] but importing one of the subsidiary files was always resulting in an execution of __ini__.py [01:48] which gave me big circular import headaches [01:48] yes, it will do. [01:49] that's why I structured it this way === thom [~thom@amnesiac.heapspace.net] has joined #launchpad [01:49] happy for you to do it differently [01:49] I'll look into it. [01:49] hi [01:49] SteveA: adapters that MAY have functional use outside of a single app should be with their classes or in a dedicated place [01:50] stuff like RosettaApplication belongs in rosetta [01:50] what do you mean by "with their classes" ? [01:51] SteveA: basic problem with the site is that it uses absolute urls everywhere, so having multiple vhosts pointing at the same plone "site" doesn't work right. [01:51] is that fixable with a virtualhostmonster? [01:51] thom: can you give me some context please? [01:51] ubuntulinux.org ? [01:51]

Edit "..." Product [10:18] (sabdfl/#launchpad) Details