| === EdwinGrubbs is now known as Edwin-zzzz | ||
| === lifeless_ is now known as lifeless | ||
| === almaisan-away is now known as al-maisan | ||
| poolie | stub, lifeless, is there any facility in launchpad at the moment for configuration that can be changed by sysadmins? | 08:11 |
|---|---|---|
| poolie | in particular if i want a list of trusted dkim domains | 08:11 |
| poolie | or a switch to turn dkim on/off | 08:11 |
| lifeless | yes | 08:11 |
| lifeless | the config files | 08:11 |
| poolie | ok | 08:11 |
| lifeless | they have settings for dev, staging, edge, production | 08:11 |
| lifeless | hm | 08:39 |
| lifeless | oops | 08:39 |
| james_w | anyone around who knows the job system and can provide some advice? | 08:53 |
| james_w | I would like to know the preferred way to implement a new job | 08:54 |
| james_w | Whether a fairly generic job with json, implying serialising some things to strings to lookup as keys when the job is run, or a more-specific job type that can have References to the other models, saving that serialisation step, would be preferred. | 08:54 |
| poolie | hi james_w | 08:55 |
| poolie | unfortunately i don't know about that | 08:55 |
| james_w | hi poolie | 08:56 |
| mrevell | Morning | 09:21 |
| james_w | BjornT: hi, do you have an opinion on whether a new job type should use as many References as possible, or serialise to JSON and look up the objects again based on keys in the JSON? | 09:25 |
| lifeless | james_w: hi | 09:28 |
| lifeless | james_w: I'm not sure there is a general answer to that question | 09:28 |
| james_w | hi lifeless | 09:28 |
| lifeless | james_w: I'd consider things like: | 09:28 |
| lifeless | - are there security implications on any referenced tables | 09:29 |
| james_w | I'm happy to give specifics | 09:29 |
| lifeless | - are there performance implications ... | 09:29 |
| lifeless | - what would be simplest for your code | 09:29 |
| lifeless | - are there race conditions - do you need to freeze parameters so they don't change under you | 09:29 |
| lifeless | james_w: sure, though I'm signing off - 4am rise tomorrow - shortly | 09:30 |
| lifeless | james_w: oh and obviously the inverse of the previous one, do you have parameters that *have* to be fresh | 09:30 |
| james_w | g'night | 09:32 |
| thumper | james_w: hi | 10:01 |
| thumper | james_w: what are you looking to do? | 10:01 |
| james_w | hi thumper | 10:01 |
| james_w | I want to create a job type for the creation of copy archives | 10:02 |
| james_w | these have a source archive, target archive, source distroseries, source packagesets, source component, etc. | 10:02 |
| thumper | james_w: what we did for BranchJob was to just have the branch that it was linked to, and everything else in a json blob | 10:02 |
| thumper | the general idea is: | 10:03 |
| james_w | an ArchiveJob to parallel BugJub and BranchJob would be useful elsewhere, but there is far more linked data here than in any of the other jobs | 10:03 |
| thumper | if you need to query against the value, it needs to be a column | 10:03 |
| thumper | otherwise a blob is fine | 10:03 |
| thumper | james_w: the question relates to what you end up querying against | 10:03 |
| thumper | and what the job really links to | 10:03 |
| thumper | and how it needs to be selected... | 10:04 |
| thumper | what invalidates the job... | 10:04 |
| thumper | how it will be deleted... | 10:04 |
| james_w | just target archive needs to be queried against I think | 10:04 |
| thumper | what other jobs will there be for archives? | 10:05 |
| james_w | don't know yet | 10:05 |
| james_w | there are a bunch of things that could be possible to have there, bulk copying packages, even publication | 10:06 |
| thumper | but it is likely that there will be more? | 10:06 |
| thumper | I'd suggest make a mirror of the branch job table for now | 10:06 |
| thumper | although I hate myself for saying it... | 10:06 |
| thumper | the duplication rubs me the wrong way | 10:07 |
| thumper | but is the best we have for now | 10:07 |
| thumper | but instead of a branch reference, have the archive | 10:07 |
| james_w | there's a hell of a lot of boilerplate to create a new job type | 10:07 |
| thumper | I wouldn't say there is that much... | 10:07 |
| thumper | a lot has been moved into base classes | 10:07 |
| thumper | but yes, | 10:07 |
| thumper | there will be duplication and boilerplate | 10:07 |
| james_w | https://dev.launchpad.net/Foundations/JobSystem | 10:08 |
| * thumper isn't working :) just helping | 10:08 | |
| thumper | I'm in Perth having a holiday | 10:08 |
| james_w | well, thanks :-) | 10:08 |
| thumper | but as you can see, I have my laptop :) | 10:09 |
| thumper | can't go on holiday without it, well, not for more than a few days | 10:10 |
| thumper | I need my fix | 10:10 |
| deryck | Morning, all. | 11:01 |
| maxb | In relation to the just-gone discussion in #launchpad, maybe we should have uses_soyuz and uses_lp_distromirrors attributes on distributions? | 11:21 |
| wgrant | maxb: We already have IDistribution.full_functionality | 11:39 |
| wgrant | Currently a celebrity-based hack, but it works. | 11:39 |
| jml | maxb, or even IDistribution.build_system :) | 11:41 |
| bigjools | ICanHazBuilds | 11:42 |
| wgrant | But even just hiding series that don't have any PPA-capable DASes would be a good start. | 11:42 |
| bigjools | wgrant: BTW did you notice that we got the first successful recipe build done in prod yesterday? | 11:44 |
| wgrant | bigjools: I saw some bugs fly past that suggested it. | 11:45 |
| wgrant | Is this after chmoding the source dir on cesium? | 11:45 |
| bigjools | no, fixing the init script for the b-m | 11:45 |
| wgrant | Ah. | 11:45 |
| bigjools | which had an interesting "cd" in it | 11:45 |
| bigjools | so, part of this success is down to you, thanks a lot man :) | 11:45 |
| wgrant | I'm glad it's finally running. | 11:46 |
| wgrant | A few months later than I expected, but..... | 11:46 |
| wgrant | mrevell: The PPA terms of use state that anything OK for Ubuntu is OK. | 11:57 |
| wgrant | And just about anything redistributable is OK for Ubuntu. | 11:58 |
| mrevell | wgrant, Thanks for that. I'm not saying you're wrong but that's not how we've enforced it before. For example, we nearly closed a PPA a year or two back that had the flash installer in it but the owner reworked it. | 12:00 |
| wgrant | Right, it seems like it's wrong, but it seems to be what the terms of use say. | 12:00 |
| wgrant | Hmm. | 12:01 |
| wgrant | bigjools: The bzr-builder PPA doesn't also have a modern bzr in it? | 12:02 |
| bigjools | is that a question or a statement? | 12:02 |
| wgrant | I just tried a Hardy build, and it seems to be using real Hardy bzr. | 12:02 |
| wgrant | I didn't even know bzr-builder would work with something so old. | 12:02 |
| bigjools | the production system doesn't use a PPA for the bzr-builder dep | 12:03 |
| wgrant | Oh, bzr-builder is in -updates? | 12:03 |
| bigjools | no, there's an internal archive | 12:03 |
| wgrant | Ah, right. | 12:03 |
| mrevell | wgrant, Thanks for raising that. As ever, you're the guy who keeps me on the right track :) I'll review the terms and how we enforce them. | 12:03 |
| wgrant | s/bzr-builder PPA/internal archive/ | 12:03 |
| bigjools | heh - no idea | 12:04 |
| bigjools | I'll let abentley chase that one up | 12:04 |
| james_w | wgrant: bzr-builder doesn't do anything particularly interesting with bzrlib, just equivalents of branch and merge, which have been around for ever. | 12:05 |
| wgrant | Ah. | 12:06 |
| wgrant | james_w: http://launchpadlibrarian.net/50794525/buildlog.txt.gz <- do I need to backport python-debian too? | 12:16 |
| james_w | wgrant: quite possibly | 12:16 |
| james_w | we could perhaps fix it in bzr-builder | 12:17 |
| wgrant | Is the PPA publisher working? I copied a package about 12 minutes ago -- it's still Pending. | 12:31 |
| wgrant | Ah, there. | 12:32 |
| wgrant | Looks like it missed two runs. | 12:32 |
| === matsubara-afk is now known as matsubara | ||
| === mrevell is now known as mrevell-lunch | ||
| === salgado-afk is now known as salgado | ||
| === mrevell-lunch is now known as mrevell | ||
| james_w | how do I get the key for a enum item? | 14:04 |
| james_w | I want to store something that I can use to get that enum value back | 14:04 |
| james_w | .value? | 14:05 |
| === Edwin-zzzz is now known as EdwinGrubbs | ||
| jml | james_w, have you figured it out yet? | 15:52 |
| james_w | no | 15:52 |
| james_w | yes | 15:52 |
| jml | james_w, cool. :) | 15:53 |
| jml | sorry for not trying to answer earlier. | 15:53 |
| james_w | PackagePublishingPocket.items[PackagePublishingPocket.RELEASE.value] I think | 15:53 |
| james_w | just waiting for my tests to confirm | 15:54 |
| james_w | yes | 15:54 |
| === salgado is now known as salgado-lunch | ||
| james_w | which layer should I use for a test class that will just be using the db (and possibly librarian), but doesn't care about the web ui and stuff? | 16:23 |
| james_w | is LaunchpadZopelessLayer just for script tests? | 16:23 |
| === matsubara is now known as matsubara-lunch | ||
| james_w | bigjools: do you have a problem with PackageCloner creating build records for the packages it copies too? Currently it just copies sources and the builds are created elsewhere. I would like to consolidate for ease of testing. | 17:07 |
| bigjools | james_w: remind me where they're created at the moment? | 17:08 |
| james_w | scripts/populate_archive.py | 17:08 |
| bigjools | urgh | 17:08 |
| bigjools | yeah change it | 17:08 |
| === beuno is now known as beuno-lunch | ||
| === salgado-lunch is now known as salgado | ||
| === beuno-lunch is now known as beuno | ||
| === matsubara-lunch is now known as matsubara | ||
| === al-maisan is now known as almaisan-away | ||
| jml | I'm getting an error trying to build gettextpo | 18:22 |
| jml | it's the first time I've tried to build lp in a few months, so I might have missed some obvious / crucial step | 18:22 |
| jml | http://paste.ubuntu.com/454033/ | 18:22 |
| jml | help? | 18:26 |
| dobey | anyone know why the merge proposal diffs aren't updating in launchpad, and it's not sending the mails for proposal status changes? | 18:30 |
| jml | dobey, no, I don't, but I saw abentley say something vaguely operational about it on another channel | 18:32 |
| abentley | dobey, no, at this point it's just under investigation. | 18:33 |
| james_w | jml: that's odd, if pydoc was failing it would normally say "Error" before "Leaving directory" I think | 18:33 |
| dobey | abentley: ok, thanks | 18:34 |
| james_w | that's probably the last step either way, so that doesn't tell you whether it failed | 18:34 |
| james_w | if you are using parallel make then the actual error can be further back | 18:35 |
| jml | james_w, how would I know if I'm using parallel make? | 18:35 |
| james_w | I don't know, apart from it doing more than one thing at once | 18:36 |
| james_w | if there's no error before it starts printing pygettextpo stuff then you probably aren't | 18:36 |
| james_w | I don't think it's default for LP, so unless you ran "make -j"... | 18:36 |
| jml | hm. | 18:37 |
| jml | oh, I missed a tb | 18:37 |
| jml | http://paste.ubuntu.com/454037/ | 18:37 |
| jml | oh huh | 18:44 |
| jml | I think I have a local copy of Python | 18:44 |
| jml | If I want to add some tests for bugtask, should I add them to test_bugtask.py, test_bugtask_0.py or test_bugtask_1.py?+ | 18:51 |
| james_w | heh | 18:54 |
| james_w | probably make test_bugtask_jml.py? | 18:55 |
| jml | james_w, maybe I should use the database id for my row in Launchpad to avoid confusion | 18:55 |
| jml | deryck, If I want to add some tests for bugtask, should I add them to test_bugtask.py, test_bugtask_0.py or test_bugtask_1.py? | 18:56 |
| deryck | jml, I don't know. Let me look more after the call. | 19:01 |
| jml | deryck, thanks. | 19:01 |
| james_w | gah, IPackagesetSet has a get method that works different to all the other get methods I have come across | 19:11 |
| jml | yeah. we need some consistency. | 19:11 |
| james_w | are there any scripts in LP that would be run by an admin and just create a job for later processing? Would we want such a script to create the job but synchronously process it? Or just do the work and leave the job for webservice requests? | 19:18 |
| jml | james_w, I'm not sure what you mean. | 19:24 |
| james_w | populate_archive.py currently creates a copy archive and sits there while everything is copied | 19:24 |
| james_w | I'm working on making copy archive creation a job such that we can do it over the API and the like if we wish | 19:25 |
| james_w | I'm wondering if I should: | 19:25 |
| james_w | a) have the populate_archive.py script create the job and exit, leaving it up to a cronscript to do the work and the person running the script to poll the web ui to find when it is done | 19:25 |
| james_w | b) have the populate_archive.py create the job, but pre-claimed, and then kick off the running of the job synchronously such that it can report the results to the admin running the script | 19:26 |
| james_w | c) have populate_archive.py ignore the job system and just run the code in pretty much the way it is now, and just have the job system as an alternative entry point | 19:27 |
| james_w | d) other | 19:27 |
| james_w | e) none of the above | 19:27 |
| james_w | 3) Uzbekistan in the 1920s | 19:27 |
| jml | james_w, sorry for the delay, I'm on a call that got interesting :) | 19:33 |
| james_w | np | 19:34 |
| james_w | it's an idle question as I hack the code to actually make any of this possible | 19:34 |
| jml | james_w, I don't know enough about what populate_archive.py does, I think. a) seems the most consistent | 19:35 |
| jml | and we should have a widget on the ui for tracking in-progress jobs | 19:35 |
| james_w | so populate_archive.py creates something like launchpad.net/~jml/+copy-archive/some-new-archive | 19:36 |
| james_w | then it spends a few minutes copying all the sources from Ubuntu in to it and creating build records so that you can testbuild a snapshot of Ubuntu | 19:36 |
| james_w | this is currently something that is usually done by an admin for doko | 19:37 |
| james_w | so I guess it's a question for the losas? they are the only ones that will use the populate_archive.py interface. | 19:37 |
| jml | james_w, yeah, maybe. | 19:38 |
| jml | james_w, now that you explain it though, I'm more inclined toward a) | 19:38 |
| james_w | Perhaps it will be changed to do an API call instead, in which case we can't really do synchronous, as we don't even have a way to poll for completion of this job | 19:38 |
| jml | james_w, although it's something of a regression in the UI of the script, we should just fix the webservice api so that the script can behave the way it did. | 19:39 |
| jml | james_w, after all, this isn't the only case where someone wants to run scripts that kick off long-running jobs. | 19:39 |
| james_w | yeah | 19:39 |
| jml | deryck, thanks for looking at the test_bugtask thing btw | 19:40 |
| james_w | jml: do you have any notes from your earlier call with Julian? | 19:40 |
| deryck | jml, yes, just back from coffee. Looking now.... | 19:41 |
| abentley | dobey, the script got stuck and we've restarted it. Please let us know if you're still experiencing problems. | 19:43 |
| dobey | abentley: ok thanks | 19:43 |
| * dobey waits for a flood of e-mail | 19:43 | |
| abentley | james_w, we do already have scripts that create jobs and let a cronjob handle running them. e.g. the branch scanner. Daily builds is similar, though it hands off to the build farm, not a cron script. | 19:51 |
| deryck | jml, so I think just use test_bugtask. | 19:52 |
| deryck | jml, the others seem like someone was thinking something that is not now clear :-) | 19:52 |
| jml | deryck, that had occurred to me :) | 19:52 |
| jml | deryck, thanks. | 19:52 |
| james_w | abentley: they are scripts run by admins? | 19:52 |
| abentley | james_w, they are normally run from cron scripts themselves. | 19:53 |
| abentley | Or as cron jobs, I should say. | 19:53 |
| james_w | abentley: right, this will only ever be run by hand on request | 19:53 |
| * deryck will make a note to confer and refactor test_bugtask* as appropriate | 19:53 | |
| jml | 5 seconds to start the functional layer | 20:06 |
| jml | gary_poster, do you know by what mechanism the API documentation lists the possible choices for "hardware_bus" on https://edge.launchpad.net/+apidoc/devel.html#bug_target for searchTasks? | 21:12 |
| gary_poster | jml: no, but my guess would be by introspecting a vocabulary specified in an interface. I could probably find out quickly. Should I? | 21:14 |
| jml | gary_poster, yes please. I'm trying to figure out how it knows to introspect as well. | 21:14 |
| gary_poster | k | 21:15 |
| gary_poster | jml: it almost certainly introspects the Choice field on line 95 of lp/bugs/interfaces/bugtarget.py . This in turn points to a vocabulary, which is apparently, per the comment, set to HWBus in ./canonical/launchpad/interfaces/_schema_circular_imports.py:165 | 21:25 |
| === ubuntujenkins_ is now known as ubuntujenkins | ||
| james_w | lazr.restful.marshallers.SimpleVocabularyLookupFieldMarshaller knows how to iterate items on an enum to get the valid choices | 21:27 |
| jml | does that get used to generate the wadl | 21:28 |
| gary_poster | yes | 21:28 |
| jml | ok. thank you gary_poster & james_w | 21:31 |
| gary_poster | np | 21:31 |
| jml | actually, it looks like it's used to make the json, not the wadl | 21:42 |
| james_w | jml: http://paste.ubuntu.com/454134/ | 21:49 |
| jml | james_w, thanks. | 21:49 |
| james_w | jml: in return any hints you could give me of how I would get a SourcePackagePublishingHistory given it's id would be greatly appreciated | 21:58 |
| james_w | there's no ISourcePackagePublishingHistorySet I can see | 21:58 |
| james_w | hmm, there's an IPublishingSet | 21:58 |
| jml | james_w, you can cheat & use storm. getUtility(ISomethingOrOther).find(SourcePackagePublishingHistory, SPPH.id=your_id). | 21:59 |
| james_w | jml: thanks, I'll resort to that if this doesn't work | 22:00 |
| === matsubara is now known as matsubara-afk | ||
| james_w | getUtility(IPublishingSet).getByIdAndArchive() apparently. Though for some reason that returns you a ResultSet and you have to call .one() on it yourself. | 22:06 |
| jml | :( | 22:06 |
| jml | james_w, there's only one callsite for that, and no unit tests. | 22:08 |
| jml | (or doctests) | 22:08 |
| james_w | jml: are you suggesting I fix it up? | 22:09 |
| === salgado is now known as salgado-afk | ||
| jml | james_w, I am. But I can completely understand if you'd rather not. | 22:09 |
| james_w | I'm already in the middle of a monster branch | 22:10 |
| james_w | so adding one more thing to it isn't going to hurt that much | 22:10 |
| james_w | hmm, my ec2 instances seem to be dying without mailing me | 22:11 |
| jml | james_w, good news and bad, even while I momentarily gaze away | 22:11 |
| jml | gary_poster, james_w, fwiw, I was working on https://bugs.edge.launchpad.net/lazr.restful/+bug/590561. Now I've stopped. | 22:12 |
| _mup_ | Bug #590561: searchTasks API documentation doesn't document acceptable status arguments <lazr.restful:New> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/590561> | 22:12 |
| gary_poster | on call | 22:12 |
| jml | np. it's not urgent. | 22:12 |
| james_w | more fun: http://paste.ubuntu.com/454140/ | 22:12 |
| james_w | oh, that one's my mistake | 22:12 |
| jml | james_w, nyah-ah-ah | 22:13 |
| jml | james_w, if they are dying without mailing, I'd recommend running them attached. | 22:13 |
| jml | james_w, there's no other way of guaranteeing you'll get output. | 22:14 |
| james_w | jml: just kicking that off now | 22:15 |
| jml | james_w, cool. | 22:16 |
| jml | james_w, I'm off now to find something to eat. | 22:17 |
| james_w | jml: enjoy | 22:18 |
| jml | james_w, and thence to not work. | 22:18 |
| jml | james_w, enjoy your evening | 22:18 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!