| lifeless | spm: hi | 00:02 |
|---|---|---|
| lifeless | spm: I'm sure you have urgents, but I have some today | 00:02 |
| lifeless | spm: firstly, gmb needs to know some stuff about the release. | 00:02 |
| lifeless | gmb: ^ | 00:02 |
| spm | lifeless: heya | 00:02 |
| * gmb is paying attention, may mumble further questions | 00:02 | |
| gmb | Have run out of Monday. | 00:03 |
| lifeless | spm: specifically; how long did the staging db migration take (for downtime estimation); secondly, are the lucid upgrades tom is doing sequential or concurrent with the schema migration | 00:03 |
| lifeless | also we seem to have a dead-but-accepting launchpad edge server - see #launchpad backlog with nightrose | 00:03 |
| lifeless | and can I have a pony. please. | 00:03 |
| spm | sorry. no ponies. we do have shiny ponies tho? does that count? | 00:04 |
| lifeless | shiny will do | 00:04 |
| spm | lifeless: <stub> spm: 25 mins for the db patches to apply anyway | 00:05 |
| lifeless | gmb: ^ | 00:05 |
| gmb | spm: Ta. | 00:05 |
| spm | tho. um. migration? I perhaps misunderstood the question? | 00:05 |
| spm | did you mean the 8.3 -> 8.4 migration? or just the regular do the DB juju? cause the ans provided is for the latter; not former. | 00:06 |
| spm | staging DB has not been migrated. | 00:06 |
| lifeless | regular DB juju | 00:06 |
| spm | coolio | 00:06 |
| lifeless | oh and speaking of staging | 00:07 |
| lifeless | I'd like to get a profile pretty please | 00:08 |
| * thumper is finding :(( | 00:13 | |
| * gmb exits stage right in search of sleep. | 00:15 | |
| gmb | 'Night folks. | 00:15 |
| lifeless | shiny shiny - http://wiki.postgresql.org/wiki/Postgres-XC | 00:15 |
| elmo | "Failover/High Availability: No" | 00:23 |
| lifeless | yeah | 00:24 |
| lifeless | but its the first multi master thing in the pg world that I've heard of | 00:24 |
| lifeless | so things are starting to move | 00:24 |
| thumper | ah no | 00:24 |
| thumper | I was all about to go \o/ | 00:24 |
| thumper | but no failover is somewhat sucky | 00:24 |
| lifeless | oh god no, we can't /move/ to this today | 00:25 |
| lifeless | its simply 'if you need more writes than one machine can give you. I think it would be possibly to run slony on top if it if we wanted (arrghh :P) | 00:25 |
| thumper | jelmer: ping? | 00:38 |
| jelmer | thumper, pong | 00:38 |
| thumper | jelmer: why am I not too surprised that you are up? | 00:38 |
| thumper | jelmer: http://staging.launchpadlibrarian.net/54835281/chicken-chicken-git-mirror.log | 00:38 |
| thumper | jelmer: looking at some staging import logs | 00:38 |
| thumper | jelmer: Exception AttributeError: "'NoneType' object has no attribute 'close'" in <function terminate at 0x414d140> ignored | 00:38 |
| thumper | jelmer: any ideas? | 00:39 |
| thumper | jelmer: seems on bzr-git and bzr-svn imports | 00:39 |
| jelmer | thumper: as far as I can tell they are to do with paramiko | 00:39 |
| thumper | jelmer: ah | 00:39 |
| jelmer | thumper: I've seen them from e.g. update-sourcecode as well | 00:39 |
| jelmer | thumper: but I'm not clear as to what the problem is exactly | 00:39 |
| thumper | ok | 00:39 |
| thumper | jelmer: I've got a question about bug 599397 | 00:40 |
| _mup_ | Bug #599397: code imports break when tdb is installed <code-import> <qa-needstesting> <Bazaar Hg Plugin:Triaged> <Launchpad Bazaar Integration:Fix Committed by jelmer> <https://launchpad.net/bugs/599397> | 00:41 |
| thumper | jelmer: should we be running the import machines with tdb? | 00:41 |
| jelmer | thumper: Yes-ish | 00:42 |
| thumper | ? | 00:42 |
| thumper | meaning? | 00:42 |
| jelmer | thumper: There are some advantages to using tdb - it allows concurrent write access for example, so multiple clones from the same svn repository at the same. In my experience it's also faster than SQLite. | 00:42 |
| thumper | I feel a but coming... | 00:42 |
| jelmer | there is.. :-) | 00:43 |
| jelmer | maxb has reported he's seen worse performance from tdb than from sqlite | 00:43 |
| jelmer | there is also the fact that all existing caches are already in sqlite, moving to tdb would mean regenerating them or keeping the existing ones in sqlite and the new ones in tdb | 00:44 |
| jelmer | concurrent access isn't really an issue except when people register svn imports for different branches in the same svn repository at the same time. after the initial import the incremental imports are quite quick so they hardly ever overlap, and even if they do they will be retried. | 00:46 |
| thumper | jelmer: suggestion on an import to try to verify bug 617078? | 00:46 |
| _mup_ | Bug #617078: RedirectRequest errors trying to access .git/objects <qa-needstesting> <Bazaar Git Plugin:Fix Committed by jelmer> <Launchpad Bazaar Integration:Fix Committed by jelmer> <https://launchpad.net/bugs/617078> | 00:46 |
| mwhudson | any github http import i think? | 00:46 |
| jelmer | thumper: xwax, lp:~dholbach/xwax/trunk | 00:47 |
| thumper | ok | 00:47 |
| jelmer | mwhudson: I think most non-github failing git imports over http are failing because of that particular bug | 00:47 |
| mwhudson | ah ok | 00:48 |
| jelmer | mwhudson: github still won't work with the new rollout, they don't support "dumb" access to git repositories over http | 00:48 |
| jelmer | mwhudson: s/github/github over http/ | 00:48 |
| lifeless | spm: can we do a profile on stagin gplease? | 00:54 |
| spm | lifeless: sure; sorry was just stopping the librarian from running outa space and doing a major update on how docco on how to achieve that happy state :-) | 00:55 |
| lifeless | yeah | 00:55 |
| lifeless | I'd like to do the multi-disk-support thing properly. | 00:55 |
| lifeless | if you have design constraints/ideas add em to the bug. | 00:55 |
| spm | https://wiki.canonical.com/InformationInfrastructure/OSA/LPHowTo/IncrementTopLevelLibrarianDir for the bored | 00:55 |
| lifeless | I wonder if we could just move to an S3 impl | 00:56 |
| spm | written for the "AAAAA! Librarian! No Space! FIX NOW" I don't wanna have to work this out from first principles in a hurry losa | 00:56 |
| spm | obvious jokes aside; it is something we've rambled about internally. I'm unsure if backups and/or IO would be sufficient tho. | 00:57 |
| lifeless | spm: challenges challenges | 00:57 |
| lifeless | anyhow, lets get our current system bug free | 00:58 |
| lifeless | then we can transform what it does into a requirements spec for evaluating replacements. | 00:58 |
| lifeless | spm: so profiling ? yes/no ? | 00:59 |
| spm | getting there.... :-) | 00:59 |
| thumper | lifeless: no | 01:00 |
| thumper | lifeless: spm is doing something for me first :) | 01:00 |
| thumper | lifeless: I'll take 3 minutes | 01:00 |
| lifeless | ok | 01:00 |
| lifeless | there, thats 6 :P | 01:06 |
| thumper | lifeless: I'm all done | 01:07 |
| lifeless | spm: oh hhhaaaaaiiiii | 01:09 |
| spm | heh, you have not been forgotten; just another crit state I had to deal with... | 01:09 |
| lifeless | I know | 01:10 |
| spm | only 10 reds atm ; 2 since dealt with. progress... :-/ | 01:10 |
| lifeless | >< | 01:10 |
| lifeless | any I can help with ? | 01:10 |
| spm | kill the c o u c h d b developers? | 01:11 |
| lifeless | I have a small moral objection there | 01:11 |
| spm | oh. did that come out loud. oops. | 01:11 |
| mwhudson | :) | 01:13 |
| spm | lifeless: go for it! | 01:20 |
| lifeless | grabbing the new docs | 01:22 |
| lifeless | the new one only permits it rather than doing it on every request (unless you set another key as well) | 01:22 |
| rockstar | wgrant, I disagree. They provide a good clean system, but they also provide a system that can be safely compromised. | 01:23 |
| lifeless | 30782 0 16.7424 1.1619 storm.store:669(_load_object) | 01:25 |
| lifeless | that might be related to the assignments perf problem | 01:25 |
| thumper | ✁☹ | 01:26 |
| thumper | that's how I'm feeling right now | 01:26 |
| lifeless | where are enums again? | 01:30 |
| lifeless | spm: can you kick the profiling rsync? | 01:31 |
| thumper | lifeless: which enums? | 01:32 |
| thumper | lazr.enum | 01:32 |
| thumper | or for a particular lp part? | 01:33 |
| spm | yarp | 01:33 |
| lifeless | thumper: particular part | 01:33 |
| lifeless | registry | 01:33 |
| thumper | try lp.registry.enum | 01:34 |
| thumper | I know they were moving them | 01:34 |
| thumper | they may not have moved all yet though | 01:34 |
| lifeless | thumper: and th eold place? | 01:34 |
| thumper | lifeless: interface imports | 01:34 |
| spm | lifeless: done | 01:34 |
| lifeless | spm: thanks I'm done | 01:36 |
| lifeless | turn off allow profiling please | 01:36 |
| lifeless | so, storm's load-objects is 80% of Distribution:+assignments | 01:37 |
| * lifeless again considers removing storms object cache. | 01:37 | |
| wgrant | rockstar: The VMs protect from compromise. Not the chroots. | 01:37 |
| thumper | damn | 01:38 |
| thumper | something is wrong with my groovey new code | 01:38 |
| * thumper lunches and mulls it over | 01:38 | |
| spm | lifeless: reverted | 01:39 |
| * spm goes to correct a ZOMG-ULTRA_VIOLENT Critical alert: -ENOCOFFEE | 01:41 | |
| lifeless | spm: zomg the librarian thing scares me shitless | 01:50 |
| spm | lifeless: ahh. welcome to the club. | 01:51 |
| lifeless | how often do you do this? | 01:51 |
| nigelb | can I interest one of you folks to do a session during https://wiki.ubuntu.com/UbuntuAppDeveloperWeek | 03:02 |
| lifeless | what sort of thing did you have in mind? | 03:03 |
| nigelb | its about appp development, so something about how lp fits in, recipies, dailies, etc | 03:03 |
| nigelb | how LP can be awesome to an app developer | 03:04 |
| lifeless | recipes are a significant differentiator over e.g. sf | 03:05 |
| lifeless | jamesh: hi | 03:13 |
| jamesh | lifeless: hi | 03:13 |
| lifeless | https://bugs.edge.launchpad.net/blueprint/+bug/618367 | 03:13 |
| _mup_ | Bug #618367: Distribution:+assignments is taking a long time maybe 20% of the time <timeout> <Launchpad Blueprints:Triaged> <https://launchpad.net/bugs/618367> | 03:13 |
| lifeless | jamesh: storm seems to spend about 1/2ms per object in load_objects | 03:14 |
| lifeless | the last three comments are relevant | 03:14 |
| nigelb | lifeless: sf? | 03:15 |
| lifeless | source forge and others | 03:15 |
| nigelb | yes, so that's why asked if somone could do a session about how LP could help :) | 03:16 |
| lifeless | I htink its a good idea | 03:16 |
| lifeless | ask on the lp-dev mailing list though | 03:16 |
| lifeless | you'll capture more peoples attention | 03:16 |
| nigelb | ok, I will :) | 03:17 |
| nigelb | err, which team is that list attached to? | 03:17 |
| nigelb | nm, got it | 03:18 |
| lifeless | jamesh: any thoughts on that bug ? | 03:25 |
| lifeless | jamesh: I'm going to reduce the columns and not return full objects for the eager loading | 03:25 |
| jamesh | lifeless: I guess one obvious question is how many distinct objects are you getting from that query? | 03:27 |
| lifeless | 3000 distinct -rows- | 03:27 |
| lifeless | many repeated person/account/email triples | 03:27 |
| jamesh | While it is 3000 rows x 10 tables, I assume it isn't 30K different objects, right? | 03:27 |
| lifeless | thats right | 03:27 |
| lifeless | if storm could skip deserialising when the primary key for a thing was already retrieved in the same query that would be useful | 03:28 |
| jamesh | from the third last comment, is it set_values or _set_values you saw in the profile? | 03:34 |
| lifeless | _set_values, attached the kcachegrind to the bug for you | 03:36 |
| lifeless | to do seperate person + specs queries without code contortions at higher layers we need an even smarter resultset | 03:37 |
| jamesh | so, from how _load_objec() is structured, the _set_values() calls occur when we're updating the values in an existing Python wrapper | 03:37 |
| lifeless | I think investing in a sqlalchemy like mapping layer is probably well worth it | 03:37 |
| jamesh | the get_object_info() calls are when _load_object() creates a new wrapper | 03:38 |
| lifeless | jamesh: so having a way to skip that if its from the same query would be uesful ? | 03:38 |
| jamesh | the _load_object() calls with neither are the cases where the row was all NULLs (I assume you're doing a left join here) | 03:38 |
| lifeless | yes | 03:39 |
| lifeless | the one I was pulling my hair out over the other week | 03:39 |
| jamesh | I don't think I can give you a quick answer to the problem: we don't really keep track of whether an ObjectInfo has been fully filled out, so we'd need to track that to decide whether to attempt to fill it in | 03:45 |
| jamesh | maintaining such a flag would need to track when the user assigns lazy values to properties too | 03:45 |
| lifeless | hmmm | 03:46 |
| lifeless | so the challenge is that we have apis which return resultsets, which shouldn't cost a lot to e.g. call and do count() | 03:53 |
| lifeless | but /have/ to eager load this related data | 03:53 |
| lifeless | its quite possible the db would be faster doing two queries as well (though in theory not; only if its misphrased) | 03:54 |
| lifeless | I'd like to get rid of the use of count() in tales | 03:54 |
| jamesh | creating a ResultSet and calling its count() method won't hit _load_object() | 03:55 |
| jamesh | you should only hit that when calling an API that actually returns objects | 03:55 |
| lifeless | let me rephrase | 03:55 |
| lifeless | if we restructure this into two queries | 03:55 |
| lifeless | there will still only be one attribute (specifications) that the browser/view/template iterate | 03:55 |
| jamesh | first grab all specs, then grab all referenced (person, account, email) triples? | 03:56 |
| lifeless | so it needs to eager load the people, for the rows being returned, before iteration, and only when iteration is hjappening | 03:56 |
| lifeless | jamesh: right,thats what one would want to happen. | 03:57 |
| lifeless | I doubt that we'll save much time doing that on the DB end: its 500ms to get the specs alone with no people | 03:59 |
| lifeless | probably due to freakishly wide rows or something | 03:59 |
| thumper | mwhudson: I need to talk through this problem if you have a minute | 04:54 |
| mwhudson | thumper: ok, but a minute would be about right | 04:55 |
| thumper | mwhudson: you don't have long? | 04:56 |
| mwhudson | thumper: i just seem to have ever mounting distractions | 04:56 |
| thumper | :( | 04:56 |
| lifeless | garh | 05:24 |
| lifeless | what is | 05:24 |
| lifeless | +class Array(NamedFunc): | 05:24 |
| lifeless | + """Implements the postgres "array" function in Storm.""" | 05:24 |
| lifeless | + name = 'array' | 05:24 |
| lifeless | + | 05:24 |
| lifeless | doing in product.py?! | 05:24 |
| thumper | lifeless: because that is where it is being used? | 05:25 |
| lifeless | also lib/lp/services/database/prejoin.py should be deleted. | 05:27 |
| lifeless | thumper: it should in storm itself or lp.services.database.storm | 05:27 |
| lifeless | so other folk can find it | 05:27 |
| thumper | lifeless: agreed | 05:27 |
| * thumper just found a bug... | 05:36 | |
| lifeless | spm: hi | 05:50 |
| spm | yo | 05:50 |
| lifeless | spm: I have a small favour to ask | 05:50 |
| spm | LIES! | 05:50 |
| lifeless | I'd like you to pull a patch onto the staging appserver so I can see if it actually helps | 05:51 |
| spm | sure. you have a patch file? | 05:51 |
| lifeless | its one commit - rev 11516 in lp:~lifeless/launchpad/bug-618367 | 05:51 |
| lifeless | I'll make a patch, one sec | 05:51 |
| lifeless | or you can just do a merge -c 11516 lp:~lifeless/launchpad/bug-618367 | 05:52 |
| lifeless | all it changes is specifications (hah what could go wrong) and it passes the tests I wrote for this the other day | 05:52 |
| spm | "hah what could go wrong".... oh where do I start.... ;-) | 05:53 |
| lifeless | http://paste.ubuntu.com/489589/ | 05:54 |
| lifeless | but the merge may be easier to do; you tell me :) | 05:54 |
| spm | generally a patch is tbh | 05:55 |
| lifeless | the pastebin should do then ? | 05:55 |
| spm | yup. ta | 05:56 |
| spm | lp is a little easier these days being open; but any locked branches - eg configs - are a pita. so patches are generally better. | 05:56 |
| spm | lifeless: patched clean; restarting... | 05:57 |
| lifeless | ok, its not enough | 05:59 |
| lifeless | can you please enable profiling again | 05:59 |
| spm | sure | 05:59 |
| lifeless | and I'll get a profile and then be out of your hair for a while | 05:59 |
| * spm restrains mightily against an outpouring of cynical laughter | 06:00 | |
| spm | lifeless: should be abck again | 06:02 |
| lifeless | kick the profile rsync ? | 06:03 |
| spm | oh yeah. sorry - should have anticipated that one. | 06:05 |
| lifeless | spm: ok, I have it | 06:09 |
| lifeless | profiling off, patch off, gtg. | 06:09 |
| lifeless | thanks | 06:09 |
| lifeless | spm: actually, hang a sec, i fyou haven't | 06:10 |
| spm | I haven't. | 06:10 |
| lifeless | trying to see if its DB IO | 06:11 |
| lifeless | ok, there is clearly 10 seconds of storm parsing time in there | 06:14 |
| lifeless | whats the status of 'mentoring' | 06:34 |
| lifeless | should we be deleting stuff for it ? | 06:34 |
| mwhudson | i think so | 06:36 |
| thumper | k | 06:39 |
| thumper | I've got the main fubar fixed | 06:39 |
| spm | lifeless: can I reset that btw? | 06:41 |
| lifeless | spm: sorry, yes please | 06:46 |
| spm | np | 06:46 |
| stub | lifeless: So I can't maintain the data structures for the page performance report in RAM now that devpad has less RAM | 06:55 |
| stub | lifeless: So I need to spool parsed requests to disk | 06:55 |
| stub | lifeless: sqlite3 or spooling to an xdr file (xdrlib) seem suitable | 06:57 |
| stub | lifeless: xdrlib will be fastest to write, but generating 500 rows in the report will involve scanning it 500 times. sqlite3 is much slower to write as it needs to update indexes, but the whole file does not need to be scanned to generate a row as data can be pulled out with an index | 06:59 |
| lifeless | stub: we'll be getting more ram agian | 07:20 |
| stub | lifeless: So don't bother? | 07:20 |
| lifeless | it seems to be generating OK at the moment | 07:20 |
| stub | Whatever I do, it will make things slower. | 07:20 |
| lifeless | so yeah, I wouldn't bother. | 07:20 |
| stub | k. I'll dump the kanban card and see if IS keep wining. | 07:21 |
| stub | whining even. | 07:21 |
| lifeless | rt 41200 | 07:21 |
| lifeless | charlie mentions a different ticket to swap th ebits back | 07:22 |
| lifeless | the monthlies may need a disk spool regardless | 07:24 |
| lifeless | but dailies should be fine I think - no? | 07:24 |
| spm | stub: we can whine no matter what you do. | 07:27 |
| lifeless | rotfl | 07:27 |
| lifeless | timing is everything | 07:27 |
| spm | reckon. /me sulks | 07:27 |
| nigelb | hrm, wgrant isn't around today? | 07:31 |
| wgrant | nigelb: Am I not? | 07:31 |
| wgrant | I'm reasonably busy. | 07:31 |
| wgrant | But I'm around. | 07:31 |
| wgrant | Although I'm about to be unbusy. | 07:31 |
| nigelb | wgrant: I was asking around eariler if somone wanted to talk about LP awesomeness on app developer week... interested? | 07:32 |
| * nigelb hugs spm | 07:32 | |
| nigelb | someday you'll get the timing right ;) | 07:32 |
| spm | :-) | 07:33 |
| wgrant | nigelb: Not really, sorry. | 07:33 |
| lifeless | wgrant: you bought a gun and found your project team? | 07:33 |
| wgrant | lifeless: Sadly not. | 07:33 |
| nigelb | lifeless: wait, what? | 07:33 |
| mwhudson | wgrant: you just did all the work yourself? | 07:34 |
| nigelb | wgrant: np, I'll look for more victims..err candidates | 07:34 |
| wgrant | mwhudson: It's looking that way, although we're only half-way through. | 07:34 |
| lifeless | nigelb: ? | 07:45 |
| nigelb | lifeless: talk of guns and project teams.. nm, I figured it out | 07:46 |
| lifeless | wgrant: did you get a chance to look at the +history thing? | 07:51 |
| lifeless | ok, I'm off for a while, I'll be back later to nab jkakar / therve etc and talk about big picture refactorings of storm | 07:51 |
| lifeless | jamesh: ping | 08:24 |
| jamesh | lifeless: pong | 08:26 |
| adeuring | good morning | 08:42 |
| lifeless | jamesh: wanting to discuss storm perf with you more | 08:54 |
| lifeless | jamesh: looks like 66% of the time was in 10% of the objects | 08:54 |
| mrevell | Hello | 09:12 |
| wgrant | bigjools: That fix hasn't landed yet, has it? | 09:31 |
| bigjools | wgrant: I'll find out shortly :) | 09:31 |
| lifeless | statik: if you want to catch up nowish I could do a relatively brief skype call | 09:33 |
| statik | lifeless: i'm toggling between an in-person meeting on OpenERP and a security bug, i'll probably have to wait until next week. sorry man | 09:35 |
| lifeless | thats ok | 09:35 |
| lifeless | just bein gopportunistic | 09:35 |
| lifeless | I have to wait around a bit to stalk jkakar about storm performance anyhow | 09:35 |
| bigjools | lifeless: still around? | 11:31 |
| allenap | Is there a staging codehosting environment? | 11:34 |
| mwhudson | allenap: yes | 11:35 |
| mwhudson | allenap: but it's a bit odd because it has hardly any other underlying branches | 11:35 |
| allenap | mwhudson: It's not refreshed at the same time as the database? | 11:35 |
| mwhudson | allenap: all 600 gigs of it? | 11:39 |
| allenap | mwhudson: Yeah :) Okay, guess not. If I'm interested in manipulating some of those branches, I suppose I can push them first, and things will work? | 11:43 |
| mwhudson | yeah | 11:44 |
| mwhudson | the bzrtools branchess are synced | 11:44 |
| mwhudson | iirc | 11:44 |
| allenap | mwhudson: Excellent, thank you. | 11:54 |
| deryck | Morning, all. | 12:00 |
| lifeless | bigjools: not really; whats up? | 12:13 |
| bigjools | lifeless: it can wait. I want to talk about how zopless scripts interact with the webapp with regard to table locking and transactions | 12:13 |
| lifeless | 'badly' :P | 12:13 |
| bigjools | I figured | 12:14 |
| bigjools | let me re-phrase | 12:14 |
| bigjools | I want to make it work :) | 12:14 |
| lifeless | I'm growing more and more of the opinion that we need the same protection the webapp has, in scripts | 12:14 |
| bigjools | which protection? | 12:14 |
| lifeless | there are a few ways to do that | 12:14 |
| lifeless | timeouts | 12:14 |
| lifeless | enforced | 12:14 |
| bigjools | hmmm | 12:14 |
| bigjools | in my case I want to add a build cancellation feature, but the webapp would need to change build states potentially at the same time as the buildd-manager | 12:15 |
| lifeless | how so, and how is it different to two people changing a bug status at the same time ? | 12:16 |
| bigjools | because the b-m is a script | 12:17 |
| bigjools | and that is where my uncertainty lies | 12:17 |
| bigjools | there's a race condition where the build is NEEDSBUILD and the webapp wants to cancel it and the b-m wants to dispatch it | 12:18 |
| bigjools | I need to make sure that they behave. In the old days you'd have mutexes or something. | 12:18 |
| lifeless | ok | 12:19 |
| bigjools | I can avoid it if I do something like make the webapp use another status that's a "request to cancel" and then make the b-m handle it | 12:20 |
| lifeless | so you can model this with a table | 12:20 |
| bigjools | but that's not ideal from a user PoV | 12:20 |
| lifeless | you have NEEDSBUILD-><something> | 12:22 |
| lifeless | and you have two mutators | 12:22 |
| lifeless | webapp chooses cancelled | 12:22 |
| lifeless | bm chooses dispatched | 12:22 |
| lifeless | if the webapp ins the bm commit will fail | 12:24 |
| lifeless | if the bm wins, the webapp request will be retried | 12:25 |
| lifeless | (and won't see it as NEEDSBUILD the next time around) | 12:25 |
| lifeless | if the bm commit fails, it shouldn't dispatch | 12:25 |
| bigjools | what makes the commit fail? | 12:25 |
| lifeless | conflict | 12:25 |
| bigjools | what detects that? | 12:25 |
| lifeless | pg | 12:26 |
| lifeless | open up two pgsql launchpad_dev sessions | 12:26 |
| bigjools | does it need a special txn mode? | 12:26 |
| lifeless | in both do 'begin;' | 12:26 |
| wgrant | The webapp uses READ COMMITTED. | 12:26 |
| wgrant | So watch out. | 12:26 |
| lifeless | then do an update in one like the bm will do, and in the other like the webapp - just the status field they both write to is enough | 12:26 |
| wgrant | Really really watch out. | 12:26 |
| bigjools | exactly my point wgrant :) | 12:26 |
| bigjools | the b-m also uses the @write_transaction decorator which retries IIRC? | 12:27 |
| lifeless | write locks are still txn wide in read committed | 12:27 |
| wgrant | bigjools: It does retry, yes. | 12:28 |
| StevenK | Does python have a sh -n equivlant? | 12:28 |
| * StevenK is trying to remember | 12:28 | |
| lifeless | I think it will work; you will definitely ant to verify that it does - even if only by a manual test script | 12:29 |
| bigjools | I'm going to run the b-m and the webapp in 2 debug sessions and see what happens | 12:29 |
| bigjools | I don't trust it simply because I think this sort of thing has gone wrong before | 12:30 |
| lifeless | plenty of ways it can | 12:30 |
| * lifeless would be happiest if b-m didn't talk to the db | 12:30 | |
| lifeless | it would make this trivial | 12:30 |
| bigjools | you mean talk through a webapp using the api don't you | 12:30 |
| * bigjools shudders | 12:30 | |
| lifeless | or the internal xml server | 12:31 |
| * bigjools shudders again | 12:31 | |
| bigjools | xmlsucks.org | 12:31 |
| StevenK | Oh dear, this argument again? | 12:31 |
| bigjools | :) | 12:31 |
| bigjools | I'd be very happy to be persuaded that it would all work fine | 12:31 |
| lifeless | oh xml is terrible, no argument. | 12:31 |
| bigjools | but I'm unconvinced since it's pushing the problem to a different place | 12:32 |
| lifeless | the LP API already has semantics for in flight collisions | 12:32 |
| lifeless | it gives a patch error | 12:32 |
| lifeless | FWIW | 12:32 |
| bigjools | what about the webapp? | 12:33 |
| bigjools | well, the UI I mean | 12:33 |
| StevenK | lifeless: I think we need to make the API *much* quicker before we can argue about b-m using it with any credence. | 12:33 |
| lifeless | when it retries -AIUI, I haven't checked this code myself- it will start over from scratch. | 12:33 |
| lifeless | StevenK: yes, would you like a list of soyuz API calls that are shocking? Available on demand. | 12:34 |
| bigjools | StevenK: I think that's already understood :) | 12:34 |
| StevenK | lifeless: Not assigning blame | 12:34 |
| lifeless | StevenK: the API is not slow because its the API - in general; its slow because LP is slow and the API is a thin skin over the LP model objects. | 12:34 |
| lifeless | StevenK: its -trivial- to make a set of dedicated extremely fast API calls for services like the codehosting machines have | 12:34 |
| bigjools | part of the problem with the API is that it serializes all of the object - even bits I don't care about | 12:35 |
| lifeless | anyhow, yes thats a definite constraint | 12:35 |
| bigjools | and often the developers don't remember this when adding new "properties" | 12:35 |
| lifeless | yes | 12:35 |
| * StevenK cries since making the wadl is dying with SyntaxError | 12:36 | |
| lifeless | I need to collate some of the lessons in this area for general discussion | 12:36 |
| lifeless | but back to the question | 12:36 |
| lifeless | the webapp starts over | 12:36 |
| bigjools | lifeless: anyway, I shall try some debug sessions to see how the webapp interacts with a b-m under conflict conditions | 12:36 |
| lifeless | it will read the *new* state, and as long as the form handler is sensible (that is that a 'cancel' returns a 'cannot cancel, its <something>' when the thing wasn't in NEEDSBUILD, then the web ui will be fine. | 12:37 |
| StevenK | And the SyntaxError is *so* helpful :-( | 12:37 |
| bigjools | lifeless: the problem is that the transaction boundaries in the b-m are shit | 12:37 |
| bigjools | I need to review it and figure out WTF it has commit() inside @write_transaction | 12:38 |
| lifeless | bigjools: I can believe that; thats part of the stuff making me speculate about APIs | 12:38 |
| lifeless | bigjools: the merge proposal worker is currently causing timeouts in the webapp | 12:38 |
| lifeless | similar story | 12:38 |
| lifeless | long transaction | 12:38 |
| bigjools | lifeless: I would be on board with the API idea iff it was *thin* and super super fast | 12:38 |
| lifeless | webapp wants to update same thing, timeout. | 12:38 |
| bigjools | blocked on a lock? | 12:39 |
| lifeless | bigjools: While I can be very critical of some parts of it - and I have, on the list - it is pretty darn thin and quite lean; for the use case of b-m I think it would be fine. | 12:39 |
| bigjools | lifeless: the restful aspect of it makes me cry | 12:40 |
| lifeless | certainly you could try putting selected bits of b-m into the API and see whether they are good or bad, low risk, easy rollback etc. | 12:40 |
| lifeless | bigjools: rest has some serious limitations that I dislike too. | 12:40 |
| lifeless | and we're not even really doing rest all that faithfully. | 12:40 |
| bigjools | honestly, I just can't see me ever making the b-m use the API as it is | 12:41 |
| lifeless | I'd appreciate it if you made sure there are (reasonably) focused bugs for what you need changed. | 12:41 |
| bigjools | lifeless: anyway, thanks, I'll let you know how I get on with the testing | 12:41 |
| bigjools | I don't know what I would need changed :( | 12:42 |
| lifeless | I definitely want the API suitable for implementing backend services on. | 12:42 |
| lifeless | bigjools: well, 'I tried to put <x> in the API and it was 10 seconds slower' would be a great bug :) | 12:42 |
| bigjools | heh | 12:42 |
| lifeless | seriously, make sure that one update you do is in the API | 12:42 |
| bigjools | before I take the time to do that I want to see clearly what the pros are | 12:42 |
| lifeless | secured appropriately | 12:42 |
| lifeless | the pros are: | 12:43 |
| bigjools | the b-m would have to be an authenticated user | 12:43 |
| bigjools | unless we give it a private webapp | 12:43 |
| lifeless | - clear transaction boundaries. Crystal clear, and enforced via the primary stack we use. | 12:43 |
| lifeless | - easier testing : the twisted stuff which doesn't play - that - well with storm. postrgresql & zope - will become more pure twisted, and less godzilla hybrid from the deeps | 12:44 |
| lifeless | - we can, if we go the full monty, get rid of zope from the b-m *entirely* | 12:44 |
| lifeless | bigjools: yes, it would be a celebrity, like the software centre agent, I suspect. | 12:45 |
| lifeless | or something approximately like that | 12:45 |
| bigjools | well I am sprinting with jml in a couple of week's time and we're overhauling the remaining synchronous stuff in the b-m, I'll make sure we consider your concerns when we redesign bits of it | 12:45 |
| lifeless | it is at this point just an idea; I think its worth exploring, but its an idea. | 12:46 |
| lifeless | Its *one* implementation strategy for 'please ensure that ALL things connecting to the DB have *enforced* transaction timeouts. Thanks.' | 12:46 |
| bigjools | the latency on the requests would have to be much much quicker (consider that versus a native storm query) | 12:46 |
| lifeless | there are other implementation strategies, and we probably want to try them all in different places and see what works best. | 12:46 |
| lifeless | bigjools: apples and oranges; you'd structure it a little differently | 12:47 |
| bigjools | well, like I said, I'm sceptical but very happy to be proven wrong | 12:47 |
| bigjools | wgrant: do you remember when the package copier was fixed to prevent file checksum conflicts? | 12:54 |
| * bigjools is clearing up the 6 PPAs with unpublishable packages | 12:54 | |
| lifeless | bigjools: so I don't think its a matter of proving you wrong - there are plenty of services in the dc already running against the API | 12:58 |
| bigjools | lifeless: I'm sure there are, but the b-m is an extremely sensitive component | 12:59 |
| lifeless | bigjools: I'm totally sure there would be things to fix. I just can't predict them more accurately than saying - give it a shot, in a safe incremental way, and when we hit a problem, file a bug and stop there until its fixed. | 12:59 |
| lifeless | anyhow, irssi tells me it just went midnight; have a great day! | 12:59 |
| lifeless | gnight | 12:59 |
| bigjools | lifeless: gnight! | 13:00 |
| wgrant | bigjools: April some time. | 13:09 |
| bigjools | wgrant: figures, the most recent corrupt publication is 2010-04-03 | 13:09 |
| wgrant | Excellent news. | 13:09 |
| bigjools | I'll just delete them all | 13:10 |
| wgrant | Delete or DELETE? | 13:11 |
| bigjools | latter | 13:11 |
| wgrant | Sounds reasonable. | 13:11 |
| bigjools | whee storm is rolling in, I expect to go offline | 13:11 |
| bigjools | the third world substation here trips as soon as I get thunder | 13:12 |
| wgrant | Heh. | 13:12 |
| wgrant | Lovely. | 13:12 |
| === almaisan-away is now known as al-maisan | ||
| === mrevell is now known as mrevell-lunch | ||
| === mrevell-lunch is now known as mrevell | ||
| === thekorn_ is now known as thekorn | ||
| thefish | sorry for silly question, but is it possible to run launchpad locally, similar to something like redmine? Ive installed as per the /Getting etc instructions, but the instance I have contains loads of other oss projects - would like to use this for internal-only dev if possible/allowed | 15:32 |
| deryck | hi thefish. That's probably more appropriate for #launchpad, but the short answer is that the code base is designed for running locally for development, not deployment. | 16:02 |
| thefish | deryck: thank you for the answer, thats exactly what we want to do, just a replacement for redmine - sorry about wrong channel :) is there any docs about how to use it for local-only stuff (we dont want to see bugs for anything other than our own projects, and we dont want our own projects pushed to landscape.net) | 16:04 |
| thefish | ^ shall i ask in #landscape instead? | 16:04 |
| deryck | thefish, it's #launchpad not #landscape. Too similar I know :-) And sure, let's move it there. | 16:05 |
| thefish | :) | 16:05 |
| === mpt_ is now known as mpt | ||
| cr3 | if I'm extending launchpad outside the core of the project, which naming would be preferable/more consistent: launchpad_foobar or launchpadfoobar? | 16:39 |
| cr3 | it seems that the former is more readable but that other projects prefer the latter, like launchpadintegration for example | 16:40 |
| benji | cr3: if you mean a Python package name, many people have a strong dislike for underscores in package names | 16:41 |
| cr3 | benji: I recall something in the PEP8 that underscores are preferable for readability, but I guess many people prefer typability over readability :) | 16:42 |
| benji | the pertinant bit of PEP-8 is "Python packages should also have short, all-lowercase names, although the use of underscores is discouraged." | 16:44 |
| benji | (although I certainly don't treat PEP-8 as gospel) | 16:45 |
| cr3 | benji: ditto, thanks for the advice :) | 16:45 |
| === benji is now known as benji-lunch | ||
| === beuno is now known as beuno-lunch | ||
| === benji-lunch is now known as benji | ||
| === deryck is now known as deryck[lunch] | ||
| === al-maisan is now known as almaisan-away | ||
| === deryck[lunch] is now known as deryck | ||
| === beuno-lunch is now known as beuno | ||
| m4n1sh | I am preparing a presentation for giving a talk at PyCon India on Launchpad. I just wanted to make sure that Soyez has been made open source or it is still proprietary ? | 19:20 |
| beuno | m4n1sh, it was open sourced with everything else | 19:26 |
| beuno | nothing got held back | 19:26 |
| m4n1sh | benji: I can't find the branches under it;s launchpad's project | 19:27 |
| m4n1sh | https://code.launchpad.net/soyuz <-- can't find any source code | 19:27 |
| benji | m4n1sh: me neither! | 19:27 |
| * benji wonders what we're talking about. | 19:28 | |
| m4n1sh | benji: if it is open sourced, then where is the code? | 19:28 |
| m4n1sh | even I heard it was getting open sourced. Didn't follow after the news came out | 19:28 |
| beuno | m4n1sh, it's all part of the same source tree | 19:29 |
| m4n1sh | under launchpad.net/launchpad ? | 19:29 |
| beuno | yes | 19:29 |
| m4n1sh | beuno: thanks. | 19:29 |
| beuno | m4n1sh, the full bzr history got released | 19:30 |
| m4n1sh | col | 19:30 |
| === almaisan-away is now known as al-maisan | ||
| m4n1sh | *cool | 19:30 |
| cr3 | when using openid to authenticate against login.launchpad.net, is there a way to get the email address of the user? | 20:24 |
| cr3 | I tried sreg.SRegRequest(required=["email"])) but this results in an empty sreg.SRegResponse.fromSuccessResponse(response) | 20:25 |
| lifeless | moin | 20:30 |
| james_w` | cr3: you have to get you app added to a trusted list | 20:30 |
| cr3 | james_w`: I'm trying to find a simpler way and I think I got it thanks to the openid identifier | 20:32 |
| lifeless | jkakar: hi | 20:37 |
| EdwinGrubbs | abentley: ping | 20:37 |
| abentley | EdwinGrubbs, pong | 20:38 |
| lifeless | abentley: btw got your mail the other day; am ruminating on it | 20:38 |
| abentley | lifeless, cool. | 20:38 |
| EdwinGrubbs | abentley: I was looking at the various BranchJob classes to understand how to create my own for adding members to teams. Why is there a branch foreign key on the BranchJob table, but there is also a branch id in the metadata sometimes? Why not just put all the data in the metadata? | 20:40 |
| abentley | EdwinGrubbs, I'm not sure. Which BranchJob has a branch id in its metadata? | 20:41 |
| EdwinGrubbs | abentley: ReclaimBranchSpaceJob | 20:41 |
| jkakar | lifeless: Hiyas | 20:41 |
| lifeless | jkakar: hi | 20:41 |
| lifeless | jkakar: want to talk storm performance ? | 20:41 |
| abentley | EdwinGrubbs, probably because that one has to refer to the branch id of a branch that has been deleted. | 20:42 |
| jkakar | lifeless: Uhm, sure. Want to hop over to #storm? | 20:43 |
| lifeless | sure | 20:43 |
| === lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is CLOSED | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | ||
| EdwinGrubbs | abentley: I'm wondering why we go to the trouble of creating new tables for new foreign keys when it is ok to put some foreign keys in the metadata. | 20:43 |
| cr3 | james_w`: easy question for you: if the same user authenticates using openid against login.launchpad.net and oauth against launchpad.net, what would be the simplest way to obtain a common handle about that user? | 20:44 |
| abentley | EdwinGrubbs, at that point, the branch id is not a foreign key. There is no id corresponding with that branch in the Branch table. The purpose of the id in that case is to refer to the location of files on disk. | 20:45 |
| cr3 | james_w`: s/launchpad.net/api.launchpad.net/ | 20:45 |
| james_w` | cr3: I don't really understand the question. openid gives you an identity, oauth gives you an auth token. If your app cares about identity it would normally tie the latter to the former. | 20:46 |
| EdwinGrubbs | abentley: ok, so for an AddMemberJob table, I should have a foreign key for super_team and for new_member. | 20:46 |
| james_w` | cr3: or are you talking about from the LP point of view? | 20:46 |
| jelmer | what's up with staging? | 20:46 |
| abentley | EdwinGrubbs, that sounds like a good idea, because you might want to find that job by querying on super_team and/or new_member. | 20:48 |
| cr3 | james_w`: how could I tie the latter to the former, other than by making api calls with the handle obtained from oauth like handle.me.preferred_email_address for example | 20:48 |
| EdwinGrubbs | thanks | 20:48 |
| abentley | EdwinGrubbs, no problem. | 20:48 |
| james_w` | cr3: depends on your app. I assume this is a webapp? | 20:48 |
| cr3 | james_w`: yes | 20:48 |
| james_w` | cr3: and you want act on behalf of the user to do some things on LP? | 20:49 |
| cr3 | james_w`: but the webapp does not have a dedicated user for doing things on launchpad, it proxies request from the user interacting with the webapp using launchpadlib | 20:49 |
| james_w` | cr3: don't use a single user! | 20:49 |
| cr3 | james_w`: on behalf of the user, but as the user | 20:49 |
| cr3 | james_w`: right, no single user | 20:50 |
| cr3 | james_w`: however, users will sometimes access the webapp using oauth and something openid. I'd like both to refer to the same person object somehow | 20:50 |
| cr3 | s/something/sometimes/ | 20:50 |
| james_w` | cr3: right, so to "impersonate" me, you would have me to the "oauth dance", which would give you an oauth token. You then make API requests signed by that token, which make the changes on Launchpad. | 20:50 |
| james_w` | cr3: users /never/ access your webapp using /Launchpad's/ oauth | 20:51 |
| cr3 | james_w`: I've got that dance working already, it's pretty cool and I can use launchpadlib as is | 20:51 |
| james_w` | cr3: you use oauth on their behalf to talk to Launchpad | 20:51 |
| cr3 | james_w`: exactly, the user does the oauth dance | 20:51 |
| james_w` | cr3: great. Then how do you stop me from using your oauth token to do bad things that will be attributed to you? | 20:51 |
| james_w` | or, how do I stop you? ;-) | 20:52 |
| cr3 | james_w`: you answer read-only access during the oauth dance or something | 20:52 |
| cr3 | james_w`: the concern you raise for the webapp is exactly the same as for any other application | 20:53 |
| james_w` | cr3: I mean from the webapp's point of view | 20:53 |
| james_w` | consider a really simple webapp with a text box, and a button that says "add comment to bug #1" | 20:53 |
| cr3 | james_w`: I'm not sure I follow, the "you" and "me" in your question confused me | 20:53 |
| _mup_ | Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> < | 20:53 |
| james_w` | and the webapp wants to add that comment as the user who is accessing it | 20:54 |
| james_w` | it will do that using their oauth token. | 20:54 |
| cr3 | james_w`: I wouldn't do anything like that, changes to launchpad can only occur as a result of using the api, not the web interface | 20:54 |
| james_w` | cr3: which api? | 20:54 |
| cr3 | james_w`: the webapp's api | 20:55 |
| james_w` | cr3: ok, same thing applies though | 20:55 |
| cr3 | james_w`: conceptually, since I have the oauth information, I could very well make changes in launchpad as anyone | 20:55 |
| james_w` | cr3: a webapp method, that adds a comment to that bug. How does the webapp know which user's Launchpad oauth token to use when making the Launchpad oauth request? | 20:55 |
| james_w` | it may have thousands of oauth tokens | 20:56 |
| cr3 | james_w`: isn't that what /dev/srandom is for? | 20:56 |
| james_w` | you are just going to pick a random user? | 20:56 |
| cr3 | james_w`: seriously though, I expect the session to tell me but, you're right, nothing prevents me from using an arbitrary user | 20:57 |
| james_w` | cr3: right. I'm not trying to expose the fact that your webapp could be malicious, just trying to expose the real question underlying yours. | 20:57 |
| james_w` | cr3: the session needs a user identity for which it can find the oauth token to use | 20:58 |
| james_w` | therefore your webapp needs a user database, which contains at least a link to an oauth token to use | 20:58 |
| james_w` | if there is no oauth token then you refuse the request and make the user do the oauth dance | 20:58 |
| james_w` | but, you don't want to manage identity in the webapp, you want to delegate that to Launchpad | 20:59 |
| cr3 | james_w`: I'm trying hard to avoid having a user database. if I could get a common handle somehow, I was hoping to just be able to instantiate volatile users with the handle passed to the constructor | 20:59 |
| james_w` | that's where openid comes in | 20:59 |
| james_w` | you create a user when someone logs in using an openid that you have not seen before, and you tie the openid identity to the user, and hence the Launchpad oauth token | 21:00 |
| cr3 | james_w`: from the user, I could then get oauth_access_tokens, just like in launchpad, or identity_url for openid. all from the same volatile user object | 21:00 |
| james_w` | then, assuming you are using oauth for the webapp's API, you also implement the server-side part of oauth to hand out your apps oauth tokens | 21:01 |
| cr3 | james_w`: so users need to use the web interface in order to create an identity in a persistent store, right? | 21:01 |
| james_w` | cr3: you can call it something else than User, but that's what it is, it's an identity table, and there is no way to avoid it | 21:01 |
| james_w` | cr3: indeed | 21:01 |
| cr3 | ok, so I need an identity table, whatever the name is. then, when the same user uses the api and authenticates using oauth, how do I relate that to the existing identity? | 21:02 |
| james_w` | you need something to tie identity (delegated to Launchpad via openid), to the oauth token to use for authorization when acting on behalf of that user with Launchpad, and to the valid oauth tokens that they can use to talk to your webapp's API. | 21:02 |
| james_w` | cr3: by implementing the server-side part of oauth, it should be fairly obvious how to do that part when you implement that | 21:03 |
| james_w` | when you get an oauth signed request you can look up a user from that and retrieve the Launchpad oauth token to use | 21:04 |
| cr3 | james_w`: I started implementing the server-side part of oauth and I reverted that to proxy requests straight to launchpad | 21:04 |
| james_w` | cr3: so oauth requests that your webapp receives would be forwarded straight to Launchpad, without re-signing or something? | 21:05 |
| cr3 | james_w`: one of the major roadblocks to having my own server-side part is that I'd really benefit from using launchpadlib as-is | 21:05 |
| cr3 | james_w`: that's where I'm at now, yes | 21:05 |
| james_w` | cr3: ok, so two comments, firstly, I don't think that's possible under oauth, because the headers are signed, and the headers aren't valid to send to Launchpad, and two, why does your application exist at all if they can just talk directly to Launchpad? | 21:06 |
| cr3 | james_w`: firstly, my webapp seems to proxy the oauth dance to launchpad successfully, I just need to make sure the Referer points to the right place and voila | 21:07 |
| cr3 | james_w`: secondly, it exists because it extends the launchpad api with additional functionality. for example, using the same handle against my api, you could use launchpad like lp.projects['bzr'] and you can get results like lp.results[-1] | 21:08 |
| james_w` | ok | 21:08 |
| cr3 | james_w`: sane, almost sane or completely insane? | 21:09 |
| james_w` | if you can get it to work, then go for it, but I think you will find that you get requests that you can't proxy without re-signing | 21:09 |
| cr3 | james_w`: I'm not sure my understanding of oauth is good enough to validate the re-signing part and I suspect I'll find out at my own cost :( | 21:10 |
| james_w` | cr3: you say you proxied the dance, did you proxy any signed requests? | 21:10 |
| james_w` | because I would expect that it would just say 401 Unauthorized to any proxied requests | 21:11 |
| cr3 | james_w`: haven't tried yet, just the +request-token, +authorize-token and +access-token | 21:11 |
| james_w` | +access-token I would expect to reject it | 21:11 |
| cr3 | james_w`: I'll give that a try next at least to validate I'm going in the right direction | 21:11 |
| cr3 | james_w`: the whole dance process worked, that I can confirm | 21:11 |
| james_w` | plus, without your own oauth access tokens you have no way to validate requests like lp.addResult(...) | 21:11 |
| cr3 | james_w`: I was hoping that if launchpad returned an access-token, then requests are valid | 21:12 |
| sinzui | thumper, ping | 21:12 |
| cr3 | james_w`: the only thing is that I don't necessarily have the access granularity of launchpad: read-only, read-write, etc. | 21:12 |
| james_w` | cr3: but you can't do that on every request | 21:12 |
| cr3 | james_w`: when the information is successfully returned from +access-token, that's when I cache the oauth consumer and token in my own database and use that for subsequent requests instead of doing the dance every request | 21:13 |
| cr3 | james_w`: my understanding of oauth is very limited, so I'm not claiming this is necessarily correct, so your critique is very welcome :) | 21:13 |
| james_w` | cr3: but how do you associate my access token that you just stored with me when I make my next request? | 21:14 |
| cr3 | james_w`: isn't that sent in the HTTP_AUTHORIZATION header? | 21:15 |
| cr3 | james_w`: however, note that I can only assess that the user has previously successfully gotten an access token from launchpad, I'm at the point of associating that with a user which is when the openid <-> oauth debacle all started | 21:16 |
| james_w` | cr3: yes, but how do you have any idea that is is correctly signed? | 21:17 |
| cr3 | james_w`: crap, I need nonce information for that, right? | 21:17 |
| james_w` | yes | 21:18 |
| cr3 | james_w`: and that's never sent over the wire, just server side, right? | 21:18 |
| james_w` | yes | 21:18 |
| cr3 | james_w`: actually, I observed this was sent over the wire when doing the dance: janrain_nonce=2010-09-07T15%3A15%3A54ZoRpj7Y | 21:18 |
| cr3 | james_w`: however, that's for the request token rather than the access token, right? | 21:18 |
| james_w` | cr3: in order to verify it you have to MITM the communication between LP and the user, which oauth is explicitly designed to prevent | 21:19 |
| james_w` | cr3: basically, if you find a way to do it, it will be a security bug in the oauth protocol, and promptly closed | 21:19 |
| thumper | sinzui: hi | 21:19 |
| sinzui | thumper, We need to talk about the death of gmaps. Do you have time today for a skype call | 21:20 |
| cr3 | james_w`: dude, you totally broke my fun. now I have to redesign the whole oauth thing! but thanks, better to know early than late :) | 21:20 |
| thumper | sinzui: sure, just let me get the kids off then we can chat | 21:20 |
| james_w` | cr3: I suggest you add a User table (or whatever you want to call it), and implement oauth client and server | 21:20 |
| sinzui | thansk | 21:20 |
| sinzui | thanks | 21:20 |
| james_w` | cr3: it's exactly how this stuff is supposed to be used, so you won't be fighting against it | 21:21 |
| cr3 | james_w`: I had most of that implemented already, so it's just a question of combining that with the launchpad dance | 21:21 |
| mwhudson | janrain_nonce is to do with openid, not oauth, fwiw | 21:21 |
| cr3 | mwhudson: thanks | 21:22 |
| cr3 | james_w`: so, if my webapp handles the oauth server side, can the webapp turnaround and get a handle against launchpad or should it not do that? | 21:22 |
| james_w` | cr3: it can indeed | 21:23 |
| cr3 | james_w`: I'm not sure what the user experience would be. lets say someone uses launchpadlib against my webapp's url with their email address, what would happen next in terms of the user experience? | 21:24 |
| james_w` | cr3: they can't use their email address with launchpadlib | 21:25 |
| james_w` | cr3: let me doodle in a pastebin | 21:25 |
| cr3 | james_w`: when I used launchpadlib, I've used something like: Launchpad.get_token_and_login(consumer_name...) where consumer_name is my email address | 21:26 |
| james_w` | consumer_name is arbitrary | 21:27 |
| thumper | sinzui: skype up and we can chat now | 21:27 |
| thumper | sinzui: I have to take the little car for a warrent this morning | 21:28 |
| cr3 | james_w`: ah! that clarifies another question I had, thanks! | 21:28 |
| * cr3 takes a little break to let all of this sink in | 21:28 | |
| cr3 | james_w`: I totally understand how to link the openid to the oauth token now, that's why +request-token redirects to an openid page like https://launchpad.net/+authorize-token/+login?oauth_token=5V7gnDVP2MqTjw2TKPg2 which can then link that token to the identity, right? | 21:38 |
| james_w` | cr3: if you are not logged in, then it will ask you to login, yes. It requires that so that it knows who is granting access. | 21:39 |
| james_w` | http://paste.ubuntu.com/490014/ | 21:39 |
| cr3 | james_w`: do you think I can conceptually achieve this user experience using launchpadlib as is, assuming I adhere to the urls expected by launchpad? | 21:42 |
| james_w` | cr3: yes, if you use lazr.restful then you should be good to go | 21:43 |
| cr3 | james_w`: yep, I'll give that a try and let you know how it goes :) | 21:43 |
| james_w` | great | 21:44 |
| cr3 | james_w`: by the way, do you happen to know of any code that does what I'm trying to do? I think that the software-center-agent uses a single user instead of going through the hoops we've discussed | 21:55 |
| james_w` | cr3: I believe it does | 21:55 |
| james_w` | cr3: the font testing tool did something like this IIRC | 21:55 |
| cr3 | james_w`: this code in sca seems to indicate they have a hard coded access token for all launchpad requests: oauth_token = OAuthToken(settings.LP_ACCESS_TOKEN, settings.LP_ACCESS_SECRET) | 22:01 |
| cr3 | james_w`: and I just got confirmation from achuni that actions are performed as a single user, hopefully my code could be reused by that project then :) | 22:01 |
| lifeless | is staging meant to be down ? | 22:04 |
| lifeless | hmmm, I guess it might be codeupdating | 22:05 |
| === abentley_ is now known as abentley | ||
| lifeless | devpad, wherearthough ? | 22:15 |
| lifeless | I hat emuscle memory | 22:15 |
| EdwinGrubbs | abentley: does BranchMergeProposalJobDerived delegate instead of subclass from BranchMergeProposalJob just because storm doesn't like multiple classes linked to the same table? | 22:18 |
| abentley | EdwinGrubbs, no, it does it because AFAIK, Storm doesn't support inheritance. (Other stormified classes inherit from BranchMergeProposalJobDerived) | 22:31 |
| EdwinGrubbs | abentley: according to the storm docs, you can inherit, but you have to override the __storm_table__ attribute in the subclasses. | 22:37 |
| abentley | EdwinGrubbs, wouldn't that make it impossible to use the data provided by that table? | 22:38 |
| EdwinGrubbs | abentley: right, it doesn't help in this situation. Storm usually expects subclasses to have a table with all the columns. Actually, now I see in storm's infoheritance.txt, it does have a pattern for subclasses that have an extra table with just the extra columns it needs. However, it doesn't seem to provide any benefits over using delegates, and it has a confusing registration process. | 22:45 |
| wallyworld_ | morning | 22:55 |
| lifeless | is it possible to specify a host in the database config ? | 23:02 |
| thumper | wallyworld_: morning | 23:13 |
| wallyworld_ | thumper: standup meeting happening today? | 23:14 |
| thumper | wallyworld_: the other two had to head | 23:14 |
| thumper | wallyworld_: but I'm around | 23:14 |
| thumper | wallyworld_: but I need more coffee | 23:14 |
| wallyworld_ | thumper: np. i've got a couple of bugs left in my queue | 23:14 |
| thumper | wallyworld_: let me make a coffee then we can have a call | 23:16 |
| wallyworld_ | thumper: ok, i'll have a flat white :-) | 23:16 |
| thumper | wallyworld_: here you go -> ☕ | 23:17 |
| wallyworld_ | :-D | 23:17 |
| thumper | wallyworld_: what is the name of the IDE you use? I've a friend asking about python IDEs | 23:30 |
| wallyworld_ | thumper: PyCharm (from the guys in do Intellij for java). It's still in beta though (beta2). | 23:30 |
| wallyworld_ | s/in/who | 23:31 |
| === al-maisan is now known as almaisan-away | ||
| bdmurray | If I want to use a css class but don't like one part of it how can I change it on just my page? | 23:52 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!