[00:03] robru: So, in my branch, the latest released version corresponds to revision 342 [00:03] The revisions on trunk and devel are not the same. [00:03] michi: right, but this is going by devel because your MP targets devel. [00:03] Because we use devel as a staging branch, bundling several changes, and then bulk-merge to trunk. [00:03] michi: unless you want to make a different MP that targets trunk? [00:03] Right, so I tried to tag my branch with 342 [00:03] bzr tag 1.0.0+15.10.20150821.3-0ubuntu1 -r 342 [00:03] bzr: ERROR: Tag 1.0.0+15.10.20150821.3-0ubuntu1 already exists. [00:04] michi: can you show me the output of 'bzr tags' please? [00:04] Sec [00:05] Here are the last few: http://pastebin.ubuntu.com/12188812/ [00:05] So, I guess I need to use the final one? [00:05] No, because that one’s there already. [00:06] michi: were those already there? I don't understand why there aren't any on your devel branch [00:06] and why the train would grab every commit ever. [00:06] Let me check the tags on the devel branch. The ones I pasted are from the single-tree MR. [00:07] These are on the devel branch: http://pastebin.ubuntu.com/12188826/ [00:07] michi: I don't understand how you got up to 621 revisions when the tag for r163 is dated a few days ago [00:09] michi: anyway whatever [00:09] I don’t get that either. [00:09] But I don’t know how to fix the dch problem now. [00:09] michi: I'm seeing http://paste.ubuntu.com/12188840/ in the log, maybe your build depends aren't quite right yet? [00:10] slangasek: do you have any reccomendations for doing series detection? lsb is giving us some hassle ^ [00:10] robru: Oh, I get it :-( [00:10] I added the build dep to control.in. [00:10] But, of course, it uses control initially. [00:10] oh haha [00:10] yeah [00:10] Not lsb_release’s fault [00:10] I’ll do another push [00:13] robru: done [00:13] michi: ok one more go: https://ci-train.staging.ubuntu.com/job/ubuntu-landing-008-1-build/4/console [00:14] Something is definitely fishy with those revision numbers. [00:14] I’m seeing 625 [00:14] In the devel tree, the latest revision is 624, as you’d expect. [00:15] But bzr tags on the devel tree shows 163 as the last one [00:19] michi: it must be that you do that much extra devel on the devel branch and then only merge occaisionally into trunk or something [00:19] Yes, that’s something we definitely do a lot. [00:20] michi: still seeing "No lsb modules..." [00:20] Shit [00:20] I definitely added lsb-release to the build deps in control. [00:21] Hmmm… From the log: http://pastebin.ubuntu.com/12188826/ [00:21] Sorry, wrong link [00:21] michi: although actually hmmm. it might be working, that error might be a red herring [00:21] Yes [00:21] make[1]: Entering directory '/var/lib/jenkins/silos/ubuntu/landing-008/unity-scopes-api_+vivid' [00:21] gen-debian-files: running lsb_release -a [00:21] Distributor ID: Ubuntu [00:21] Description: Ubuntu 15.04 [00:21] Release: 15.04 [00:21] Codename: vivid [00:21] dh_auto_clean [00:22] michi: https://launchpad.net/~ci-train-staging-area/+archive/ubuntu/landing-008/+sourcepub/5325235/+listing-archive-extra scroll to the bottom here, are those the correct binary package names? [00:23] Is this vivid or wily? [00:23] michi: https://launchpad.net/~ci-train-staging-area/+archive/ubuntu/landing-008/+build/7833477 oh you're depwaiting in vivid. [00:23] For vivid, this looks right [00:23] Sorry, for *wily* this looks right. [00:23] michi: right, that one is wily [00:23] michi: the vivid one didn't built [00:23] :( [00:24] How come we need to wait here? [00:24] Ah, OK, one of our build deps is currently being built. [00:24] michi: no, not "waiting". "depwaiting". it means dependencies aren't available: https://launchpad.net/~ci-train-staging-area/+archive/ubuntu/landing-008/+build/7833477 [00:25] right. OK, that should be resolved in a few minutes, I would think? [00:25] michi: no? why would you think that? [00:25] Isn’t it building right now? [00:25] michi: I'm not aware of anybody building libnet-cpp-dev anywhere [00:25] https://launchpad.net/builders/lgw01-51 [00:26] Started 4 minutes ago [00:26] That page is linked to from https://launchpad.net/~ci-train-staging-area/+archive/ubuntu/landing-008/+build/7833477 [00:26] michi: i don't see libnet-cpp-dev on that page? [00:26] This is very strange. [00:26] Check the second link I just pasted. [00:27] It says that we are waiting for libnet-cpp-dev [00:27] then follow the lgw01-51 link [00:27] michi: the last link you pasted says the build is depwait, which is for the most part a failure mode. the builder you linked to is building emacs-snapshot of all things. [00:27] Yes. Now why is that linked to from the page that says scopes-api is waiting on netcpp? [00:28] michi: because the version of netcpp that you want is not available in vivid. [00:29] michi: https://launchpad.net/ubuntu/+source/net-cpp vivid has 1.1 [00:29] michi: your deps say ">= 1.2" [00:29] Ah... [00:29] michi: depwait means "this has failed and here's why" [00:29] nobody is building netcpp [00:29] More conflicts between control and control.in, sigh... [00:30] So, for vivid, the build deps are less stringent. [00:30] michi: well, then you can update your control hackery to make it say >= 1.1 in vivid and >= 1.2 in wily [00:31] robru: yes [00:31] Except that we need to get the build-deps from control *before* it is generated, no? [00:32] hmmm [00:32] Or would it be good enough to generate the build-deps from the clean target [00:32] If so, that’s not a problem. [00:32] michi: no slangasek told me that the build deps are expected to be installed before you run the clean target. [00:32] Sigh... [00:33] Looks like I’m stuck again then [00:33] Because we need different versions for some build deps on wily and vivid [00:33] Hmmm... [00:33] So, the solution would be to write control such it mentions the least-stringent build dep. [00:34] Then, on wily, when the actual build dep is installed, [00:34] michi: yeah I was just going to say [00:34] it’ll pull the latest anyway. [00:34] different versions of build-deps? [00:34] slangasek: it seems so. [00:34] michi: probably make the official control have >= 1.1 but then when wily generates it it can change to 1.2 later [00:34] why? that's far from what I discussed with robru [00:34] I don’t know right now. [00:34] yeah that does seem weird. [00:34] Need to check. [00:34] I don't know why the same codebase would have different build deps on different releases. [00:35] You’d think they should not differ because we are building the exact same code on vivid and wily [00:35] Exactly [00:35] this is probably some mistake that crept in somewhere. [00:35] right [00:35] michi: just set it to 1.1 then and see what happens. [00:35] Will try that, thanks. [00:35] slangasek: any thoughts on lsb release while you're around? [00:35] robru: what about it? [00:36] guess what? [00:36] I just checked gcc source and it looks for lsb_release -cs [00:36] There are *two* dependency lines for netcpp in control :( [00:36] slangasek: well in the hangout you said something like "lsb_release isn't what I'd use but it's ok" and I don't even know what other options are available [00:36] michi: I don't understand the problem? just set them both to 1.1? [00:36] robru: turns out lsb_release is what everyone's using ;) [00:36] heh [00:37] robru: Well, there should be only one, for starters. [00:37] michi: not necessarily? could have one as a build dep and one as a runtime dep. but I don't really know it well enough to say [00:38] I would have thought it doesn’t make sense to list the same package with different versions in the *same* stanza [00:38] The build deps for unity-scopes-api listed netcpp twice, with different version numbers [00:38] michi: well yeah, that doesn't make sense. why can't you just set them both to 1.1? [00:38] or just only have one... [00:39] I’ve deleted the 1.2 one [00:39] Yes. I mean, if two are better than one, three must be better than two? ;) [00:39] michi: sorry I didn't realize you meant in the same stanza at first [00:40] michi: also maybe use 'wrap-and-sort -a -t', it'll alphabetize your deps so that duplicate ones will be side by side, more obvious. [00:40] Can hardly blame you, because that’s not supposed to happen :) [00:40] You need to tell that to some of colleagues ;) [00:40] michi: I'm a huge fan of wrap-and-sort but a lot of people don't like the noisy diff that you get the first time you use it as it shuffles everything around. [00:41] michi: but since you're making a huge diff anyway, seems like a good time to throw that in there... [00:41] Absolutely! [00:41] michi: lol you should just call wrap-and-sort in your override_dh_auto_clean :-D [00:42] :) [00:42] robru: Just pushed again [00:43] There is one more thing that’s wrong: [00:43] oh? [00:43] The conflicts section needs to list unity-scopes3 for wily, but not for vivid. [00:43] But that’s just more hackery in my script. [00:43] Without this, I think it’ll fail during a dist-upgrade from vivid to wily [00:43] why are you overriding dh_auto_clean? [00:43] slangasek: Because Didier told me to :) [00:43] slangasek: that's how he calls is control-generating script [00:44] hmm [00:44] slangasek: I thought this is what we talked about in the meeting this morning? [00:44] robru: I never said "auto_clean", I said "clean" [00:44] this should probably be directly as part of a clean: target [00:44] slangasek: Can you help me here and tell me what I should be doing instead? I’m a debian virgin :( [00:45] slangasek: the train calls "clean" which calls "dh clean" which calls override_dh_auto_clean [00:45] 'dh clean' calls a lot of things; this doesn't make override_dh_auto_clean the right place to encode this. [00:45] slangasek: also his rules are legit, he passes through to the stock dh_auto_clean [00:45] slangasek: oh I didn't think anything of it [00:46] the meaning of 'dh_auto_clean' is 'run the upstream clean target to clean the source tree of build artifacts" [00:46] s/"/'/ [00:46] slangasek: https://code.launchpad.net/~michihenning/unity-scopes-api/single-tree/+merge/268433 line 2161 of the diff if you want to comment [00:47] michi: https://ci-train.staging.ubuntu.com/job/ubuntu-landing-008-1-build/5/console started [00:47] Thanks. Watching with baited breath... [00:48] slangasek: I'm not sure what other things 'dh clean' runs that would be sooner/more relevant [00:50] well, what I was saying is that I thought it would be better as a clean: target [00:50] however 'dh clean --no-act' tells me I'm wrong [00:50] let's stick with override_dh_auto_clean for now [00:53] ok [00:53] michi: ok well we're past the depwait at least, now the real question is whether or not the builds succeed [00:53] with the right binary package names [00:54] robru: Yes. I know it works on wily at least. [00:54] Not that this helps if it doesn’t on vivid... [00:55] michi: https://launchpadlibrarian.net/215463338/buildlog_ubuntu-vivid-amd64.unity-scopes-api_1.0.1%2B15.04.20150825.1-0ubuntu1_BUILDING.txt.gz build failed in vivid, looks like you're missing a dependency. [00:56] Looking, thanks! [00:56] michi: ok I just went live with the train changes so you should be able to do builds in silo 10 from here on out [00:56] Sweet. [00:56] So, looks like there is another dep on netcpp 1.2 kicking around somewhere. [00:57] Yes. [00:57] Hmmm… I don’t know what happened there. [00:57] How could this have ever built on vivid? [00:57] michi: well, depends what you mean. if the debian/ package was depending on 1.2 it would have depwaited again. the fact that it built but failed implies that you really do need netcpp 1.2 [00:58] But, we’ve been building just fine for vivid and wily for ages. [00:58] michi: I dunno man it's your thing ;-) [00:58] But vivid doesn’t *have* 1.2 [00:58] robru: Too bloody right :) [00:58] It’s not your problem, I know. [00:58] I’ll check our respective trees... [00:58] michi: heh, here to help of course, but yeah I don't know why how it used to work on vivid. [00:59] I’ll check and see where and why this was changed. [00:59] michi: backporting 1.2 to vivid (at least for the overlay PPA) is probably an option if it's too difficult to make your code build with 1.1 [00:59] I think I know. [00:59] Maybe. [01:00] I suspect that 1.2 is only in the overlay ppa [01:00] michi: indeed [01:00] michi: ohhhhhhh [01:00] I see [01:00] Well... [01:00] michi: it's because the staging area doesn't have access to the production overlay ppa, it has a separate overlay ppa that doesn't have any packages in it [01:01] michi: so this should Just Work if you trigger a build in production silo 10. [01:01] Aha. [01:01] Yes [01:01] because the production silos include the overlay ppa [01:01] And I need to change all the 1.1 deps to 1.2 [01:01] right [01:01] yeah [01:01] Phew! [01:01] hehe. "backporting 1.2 is an option", except you already did that ;-) [01:03] OK, about to kick the silo 10 build. Fingers crossed... [01:06] michi: oh, what are you doing with 0replaceme? [01:06] These are new symbols that were added by Pawel just recently. [01:06] In the merge that was missing earlier. [01:06] My branch was behind reality because Pawel merged something recently [01:06] michi: but what is this tools/symbol_diff.in script doing? [01:07] We use that script to help us keep the symbols file up to date. [01:07] Among other things, it does a sort :) [01:07] So the symbols file is in alphabetical order. [01:07] So, if a build reports a problem with the symbols file, you run that tool. [01:07] the tool generates a diff from the old and new symbols file. [01:07] michi: oh ok. sorry I thought you were doing some magic with 0replaceme that was going to conflict with train magic, didn't realize you were *using* the train magic. [01:08] If the diff looks good, you can then just patch the tree with it to bring the symbols file up to date. [01:08] See the instructions in HACKING :) [01:08] Yeah. [01:08] The symbols file is the bane of our lives. [01:08] It’s totally unsuitable for C++, unfortunately. [01:09] It just creates a lot of maintenance work, and gives us very little protection against ABI breaks in return. [01:09] It’s the wrong tool for the job. [01:09] My next job is to come with something to automate abi-compliance-checker, which does a *far* better job at ferreting out ABI breaks. [01:10] michi: ah, interesting. I know remarkably little about C/C++ [01:11] michi: was gonna say "but ABI breaks!", didn't know it doesn't work for C++ [01:11] robru: Even for C, the symbols file is extermely poor assurance against ABI breaks. [01:11] really? [01:11] Yes. [01:11] There are dozens of ways you can break the ABI without it ever showing up the symbols file. [01:11] For example: [01:11] huh [01:11] Change the return type of a function. [01:11] Add a member to a struct. [01:12] Change the order of members of a struct. [01:12] Add an enumerator to an enumeration. [01:12] Or change the order of enumerators. [01:12] Change a #define. [01:12] wow I had no idea [01:12] Change the order of parameters of a function call, or change the type of any of the parameters. [01:12] None of these things (in C) affect the symbols. [01:13] With C++, it’s a little better, because parameter types are part of the mangled symbo. [01:13] symbol. [01:13] slangasek: ^ can you comment on this? why do we fuss over symbols files if they don't guarantee the whole ABI? [01:13] But none of the other things I listed will be caught. [01:13] the point of symbols files isn’t to establish whether the ABI was broken, although that is what people keep saying. [01:13] They are wrong. [01:13] heh [01:14] no, that is exactly the purpose of symbols files [01:14] the point of the symbols file is to record at which revision level each symbol was first added to the library. [01:14] they aren't a complete solution for this [01:14] slangasek: They don’t do it though. [01:14] They miss so many things it’s not funny [01:14] It is a completely and utterly inadequate check for ABI compliance. [01:14] they *catch* so many things it's not funny [01:14] That’s not good enough. [01:15] We shouldn’t be focusing on the things it catches, but the ones it misses. [01:15] abi-compliance-checker makes none of those mistakes. [01:15] michi: your diff doesn't show abi-compliance-checker being used at package build time [01:15] As far as I can see, the point of the symbols file is to record when each symbol was first added. [01:15] is it? [01:15] slangasek: not yet, he said that was the next thing he'd work on fixing [01:16] ok [01:16] Then, by looking at all the symbols that are unresolved in a dependee, I can automatically work out what the minimum revision level is that the dependee needs to link with. [01:16] slangasek: acc reads the headers for the old and new version, and it reads the symbols for the new and old version. [01:16] It then does an in-depth analysis [01:17] It does a lot of very thorough stuff. [01:17] Like analyzing the inheritance tree. [01:17] michi: I'm a little bit concerned about the changelog here: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-010/+sourcepub/5325324/+listing-archive-extra it says it updated the symbols.in file rather than the symbols... [01:17] Checking for reordering of bases with multiple inheritance. [01:17] comparing vtbls [01:17] checking enumerators. [01:17] return types [01:17] etc, etc. [01:17] Looking... [01:17] michi: I'm very familiar with abi-compliance-checker - including how difficult it is to manipulate in practice. If you're going to do the work to integrate it with your packages in a way that ensures ABI breakage will be caught at package build time and fail the build, I have no objections to this [01:18] slangasek: that’s the plan [01:18] Yes, agree, it’s cumbersome to use. [01:18] That’s what I’d like to improve. [01:18] but, if we get that to work, I think it would be a big win. [01:18] we currently require symbols files because it catches *most* ABI breakage; it's a minimum standard, not a mandatory interface [01:18] Much better assurance that we haven’t broken anything, and much less maintenance work. [01:18] * slangasek nods [01:18] Yes. [01:19] But, frankly, we already have done things that technically have broken the ABI, but we’ve snuck it past the symbols file. [01:19] the symbols file just doesn’t make the grade for C++ especially. [01:19] There are *so* many more ways in C++ to break ABI than in C. [01:20] robru: I’m not sure what the train is trying to tell me here. [01:20] michi: yeah I'm just reading through the source code to figure it out myself, one sec [01:21] We don’t have a symbols file anymore. [01:21] Instead, we have symbols.in, which generates the symbols file. [01:21] That’s because the soversion needs be part of the name of the symbols file. [01:22] michi: I think it's ok. That changelog entry means "the train replaced 0replaceme with real version numbers in this file", I was worried it meant that it replaced it in your symbols.in instead of your symbols, but it does this before the package build so your symbols file generation magic should be running happily with the correct version number in the [01:22] symbols.in file... [01:23] Ah, OK [01:23] That’s as it should be. [01:23] michi: I was just surprised to see "symbols.in" in the changelog rather than "symbols" [01:23] Yeah [01:23] What I’m doing is a bit weird, I admit :) [01:23] thank gcc... [01:24] the problem really is that the gcc change has broken process left right and center. [01:24] Nothing we use was ever designed to cope with something like that. [01:24] Not debian, not citrain, not our build process [01:24] yeah this is a pretty massive migration [01:25] michi: what you're supposed to do is just only develop for wily and forget about vivid ;-) [01:25] tell that to someone who needs to support a phone... [01:25] doesn’t need to, I meant... [01:25] michi: phone? what phone? [01:25] michi: you mean like on the wall? ;-) [01:25] Let me check, see if I can find it... [01:26] there was something about a phone... [01:26] nah [01:26] So, I think the replaceme thing is fine. [01:26] debian puts the correct version there. [01:27] Then, when we back-merge from trunk onto devel after release, we pick up the right number. [01:27] michi: well this MP is targetting devel already, so you'll need to merge it to trunk manually at some point [01:27] Yes. [01:27] right. [01:27] Forgot. [01:27] That’s Pawel’s job :) [01:27] michi: can you confirm that these are the correct binary packge names for vivid? https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-010/+build/7833612 [01:27] At least for unity-scopes [01:27] Looking [01:28] :( [01:28] These all should have a 0.6.x version number :( [01:28] michi: and this one for wily? https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-010/+build/7833605 [01:28] no? [01:28] Aargh [01:28] michi: oh yeah, no, i told you the 0.6 thing isn't possible [01:28] just the package names [01:29] Hmmm... [01:29] So, the libunityscopes3 thing is fine. [01:29] Same with qt. [01:29] Now... [01:29] the dev package will have 1.0.1 on wily and vivid. [01:29] michi: seems a bit weird that vivid is "3" while wily is "1.0" [01:29] But the version following that will be different. [01:29] robru: the 3 is historical. [01:29] michi: yeah but it'll be 1.0.1+15.04 for vivid and 1.0.1+15.10 for wil [01:30] I got sick of meaning less sonames [01:30] Because when I see “3”, I have no bloody idea what release that corresponds to. [01:30] So, as of wily, we’ve aligned the soname with the release version. [01:30] ah ok [01:30] So, when I look at an so, I can immediately tell what release it belongs with. [01:31] So, the full package name then is different for vivid and wily. [01:31] because it mangles the release version intot he name [01:31] sweet, this looks really good! [01:31] michi: great [01:31] OK, so I’m not quite done yet with my branch. [01:32] I’ll set it to “QA needed” [01:32] Is that the right one? [01:32] Or QA required? [01:32] Anyway, once I have the final polish on it, I’ll put it through the normal process. [01:32] michi: it depends. what are you trying to do? submit it to qa? [01:32] No, not yet. [01:32] I’m still working on it. [01:32] There are a few more things I need to fix. [01:32] But this has validated the concept. [01:32] michi: then 'qa required'. and when youre ready, set it to 'ready for qa' [01:33] robru, thank you so much again. this was just bloody awesome. There is no way I could have done this without your help! [01:33] michi: you're welcome! glad it's saving you work, that's what I aim for, solve inefficiencies [01:34] I wonder how many other people will want to do something similar. [01:34] It sure as hell will make a big difference. [01:34] unity-api [01:34] unity-scopes-shell, probably [01:34] maybe unity8 [01:34] Basically anything that builds a library that needs to work for both vivid and wily, if the library uses std::string or std::list anywhere in its interface. [01:35] That’s almost every C++ library/module we build [01:35] I think everyone has just gone and split their tree. [02:06] robru, slangasek: The symbols file strikes again [02:06] https://launchpadlibrarian.net/215466145/buildlog_ubuntu-vivid-armhf.unity-scopes-api_1.0.1%2B15.04.20150825-0ubuntu1_BUILDING.txt.gz [02:08] For some reason, only on Armhf [02:53] michi: uh [02:55] michi: i have no idea why that would fail on only armhf [02:57] michi: looks like some confusion between the wily and vivid versions in the symbols file... [02:58] michi: I'm not sure, but the way duals work is they build wily first, then copy the result and build it again for vivid. So probably your symbols script is tripping up on itself when it's run the second time in vivid, you have to make sure it does the right thing in vivid. [02:59] No wait, it copies before building [02:59] But it does copy after the 0replaceme magic happens [04:22] robru: if you are still around... [04:22] it appears the version that is substituted is wrong for some reason [04:23] michi: i wrote some messages an hour ago, did you see them? [04:23] Yes. [04:23] Sorry was AFK for a while [04:23] So, the original version is 1.0.1+15.10.20150825.1 [04:23] All the symbol versions look like that. [04:24] But what it add is this: 1.0.1+15.04.20150825.1-0ubuntu1 [04:24] Why the ubuntu version at the end? [04:24] And only on Arm? [04:24] michi: that makes no sense, there's no code adding Ubuntu to the version only on arm [04:24] Right [04:25] michi: the train will put Ubuntu in the symbols version though, that should be the same on all arches [04:26] Normally, there is no ubuntu version in the symbols file. [04:26] michi: I'm not aware of any arch-specific code on the train, it should do everything the same on all arches [04:26] This is exceedingly strange. [04:26] It’s not Arm generally, only Armhf [04:27] The build succeeded on everything else. [04:28] michi: here is the code that handles merges: http://bazaar.launchpad.net/~cupstream2distro-maintainers/cupstream2distro/trunk/view/head:/citrain/recipes/merge.py#L56 this is what merges the branches and then updates the symbols, etc. [04:28] it's not too magical [04:28]