[07:57] When I run "cronscripts/process-job-source.py IInitializeDistroSeriesJobSource", it output deug message "Running (ID 2044) in status Waiting". How to debug it ? [08:00] maozhou: Debug what? [08:02] wgrant: How to make it initialize the series ? [08:02] maozhou: You've said that it said it initialised the series, but nothing further. [08:02] There's no problem. [08:05] wgrant: But , it's this status for hours [08:05] Ah, that is an important piece of information. [08:05] Is it doing anything? [08:05] if your machine is slow that may not be unreasonable. [08:07] No, my machine is not slow.And there is no building job now. [08:08] The build queue is empty [08:08] Series initialisation does not create builds. [08:08] Is the script still running? [08:09] The script maintain this status [08:10] I don't understand what you mean. [08:15] http://i1.tietuku.com/74d40e72869ead18.png [08:15] So it is still running, and has been running for several hours? [08:15] Is it issuing database queries? [08:16] Yes, it's has ben running for several hours. [08:18] I search the ID which belong to the package ,the package has been builded successful. [08:19] The package? [08:19] IDS does not touch builds. [08:20] The ID of 2044 , is it the package ID ? [08:21] No, that's a job ID. [08:21] IDS doesn't touch a single package. [08:21] It touches an entire series. [08:22] You should find out from postgres whether it is still actively issuing queries. [08:23] which is postgres? [08:23] PostgreSQL, the database that Launchpad uses. [08:25] Which table can I find the job ID 2044? [08:26] Either Job or DistributionJob, I forget. [08:28] Need I modify the database manual ? [08:28] No [08:28] You need to work out whether the job is actually hung. [08:28] strace it, look in pg_stat_activity to see if it's running DB queries, etc. [08:30] ok, thank you [09:08] wgrant: In table pg_stat_activity , I find the job is actually hung. How to deal with it? [09:08] http://i1.tietuku.com/7bcb98edbf3a5831.png [09:08] The transaction start time isn't very interesting, but the query start time is. [09:12] I don't understand. [09:14] maozhou: IDS can use long transactions, that's fine. [09:14] The question is whether a single query has hung, and if so what that is. [09:14] Those are other columns in pg_stat_activity that you haven't disclosed. [09:20] I only select "datid,datname,pid,usename,Xact_start" from the table "pg_stat_activity" [09:20] There should also be the query start time and query text. [09:20] Those are interesting. [09:21] just select *, it's not that wide [09:21] better than messing about [09:22] and use a text pastebin rather than pointlessly turning text into an image :-) [09:24] http://i3.tietuku.com/e31c6b656b8c47ce.png [09:25] The output message is like this. [09:28] the query is "idle in transaction | SELECT BinaryPackageBuild.arch_indep, BinaryPackageBuild.archive, BinaryPackageBuild.build_farm_job, BinaryPackageBuild.builder, BinaryPackageBuild.date_created, BinaryPackageBuild.date_finished, BinaryPackageBuild.date_first_dispatched, BinaryPackageBuild.date_started, BinaryPackageBuild.dependencies, BinaryPackageBuild.distribution, BinaryPackageBuild.distro_arch_series, BinaryPackageBuild.distro_series, Binar [09:28] yPackageBuild.failure_count, BinaryPackageBuild.id, BinaryPackageBuild.is_distro_archive, BinaryPackageBuild.log, BinaryPackageBuild.pocket, BinaryPackageBuild.processor, BinaryPackageBuild.source_package_name, BinaryPackageBuild.source_package_release, BinaryPackageBuild.status, BinaryPackageBuild.upload_log, BinaryPackageBuild.virtualized FROM BinaryPackageBuild WHERE BinaryPackageBuild.archive = 18 AND BinaryPackageBuild.source_package_release [09:28] = 6079 AND BinaryPackageBuild.distro_arch_series = 71 ORDER BY BinaryPackageBuild.id" [09:28] That would appear to be making progress. [09:29] The query start time is "2015-08-20 15:39:18.258064+08" [09:29] query_start two minutes before you gave the paste URL here, if I have your timezone correct [09:29] No, it's not [09:29] That's the xact_start [09:30] query_start is 2015-08-20 17:22:12.816189+08 (which if you'd used a text pastebin I would have been able to copy and paste rather than having to retype it) [09:33] Oh, yes , query start time is "2015-08-20 17:22:12.816189+08" [09:44] Why It's idle in transaction? [09:45] Because it is thinking, most likely. [09:45] How big is the series you're copying from? [09:46] Or waiting for a lock [09:46] No, idle means no query is in progress. [09:46] (and waiting is false) [09:47] oh, true [09:47] (I blame openssh having eaten my brain) [09:47] Heh [09:47] What are you doing to it? [09:47] Or is it conch? [09:52] This series is derived from Utopic [09:53] Your utopic. [09:53] How big is your utopic? [09:53] Will it spend long time for that? [09:54] 114G [09:55] wgrant: 6.9p1 packaging, followed by fixing conch, followed by 7.0p1 packaging [09:55] Aha [09:55] or hopefully fixing conch, anyway [09:55] which will then require upgrading twisted, hence my arguing with pip recently [09:56] btw, do you already have a yak-shaving machine factory or do I need to build one? [09:56] All the yak-shaving machine capacity for the next year is alread spoken for, I'm afraid. [09:57] damn. [09:59] Oh, it's finished ,thank you. [15:02] wgrant: tempted to make an LP-specific fork with https://github.com/msabramo/setuptools-git/pull/10 [15:02] (OK, won't help once we move to git, but would already have saved me the time I spent writing it) [15:04] A pip install of LP's dependencies is down to 36 seconds with that plus a wheel cache, which is still a lot slower than buildout's ~5 seconds but maybe now not prohibitively so