| wgrant | cjwatson: That seems too easy. | 00:16 |
|---|---|---|
| wgrant | Where does the other 30 seconds go, though? | 00:17 |
| Spads | Mars. | 00:17 |
| Spads | http://www.thirtysecondstomars.com/ <-- Proof. | 00:17 |
| wgrant | Heh | 00:20 |
| cjwatson | wgrant: it spends a fair bit of time in _find_all_versions for each requirement (i.e. the "Collecting" steps) | 09:09 |
| cjwatson | I would like a way to say "don't use setuptools.file_finders no matter what" in LP's setup.py | 09:16 |
| cjwatson | because we care not about the sdist and having to patch setuptools-git is a ridiculous distraction | 09:17 |
| cjwatson | wgrant: But once the virtualenv is populated, it's 3.5 seconds to rerun, which I guess is actually more analogous to buildout's behaviour with a symlink to a populated eggs directory. | 12:18 |
| cjwatson | Oh, indeed, without the eggs symlink, buildout takes 155 seconds vs. the current figure I have for a full virtualenv build with pip of 52 seconds (my previous figure was badly measured, I missed a bit at the start) | 12:23 |
| cjwatson | So performance is not an issue any more and I just need to sort out the large pile of details | 12:24 |
| cjwatson | The pip figure is actually cheating because I had cached wheels; without those it's 164 seconds. But that's sufficiently close to the comparable buildout run that I don't care | 12:31 |
| cjwatson | I found a reasonably horrible but just about tolerable way to cut setuptools-git out of the loop in our setup.py | 12:35 |
| cjwatson | lifeless: How would I go about debugging 'pip install "zope.security[untrustedpython]"' not actually installing the untrustedpython extra in my in-progress pip conversion of LP? | 20:45 |
| cjwatson | (this is, hope you understand, several layers deep in a rabbit hole, but it's where I've got to thus far ...) | 20:45 |
| lifeless | pdb ? | 20:47 |
| lifeless | try pip install -v ... first | 20:47 |
| lifeless | if that doesn't enlighten | 20:48 |
| lifeless | show me a pastebin | 20:48 |
| cjwatson | http://paste.ubuntu.com/12144491/ | 20:48 |
| cjwatson | I was hoping for an answer other than pdb, but I guess I can give that a try ;-) | 20:49 |
| cjwatson | should be installing RestrictedPython, but isn't; works fine if I explicitly request RestrictedPython | 20:50 |
| cjwatson | zope.security's metadata.json looks right, contains {"requires": ["RestrictedPython"], "extra": "untrustedpython"} | 20:51 |
| cjwatson | so if this isn't just something that plain doesn't work yet in pip, I can certainly attack with pdb, just wanted to check first since that's likely to take me a while | 20:52 |
| cjwatson | I know you've been working on various corners of pip's requirement handling | 20:52 |
| lifeless | I'm not at all sure that metadata.json is consuled | 20:52 |
| lifeless | I'd need to check the embedded pkgresources - I think it may still be reading METADATA | 20:53 |
| cjwatson | METADATA has | 20:53 |
| cjwatson | Provides-Extra: untrustedpython | 20:53 |
| cjwatson | Requires-Dist: RestrictedPython; extra == 'untrustedpython' | 20:53 |
| lifeless | so it works for me :) | 20:54 |
| lifeless | Successfully installed RestrictedPython-3.6.0 zope.component-4.2.2 zope.event-4.0.3 zope.i18nmessageid-4.0.3 zope.interface-4.1.2 zope.location-4.0.3 zope.proxy-4.1.6 zope.schema-4.4.2 zope.security-4.0.3 zope.untrustedpython-4.0.0 | 20:54 |
| lifeless | hmm, I wasn't using a constraints file | 20:56 |
| lifeless | whats in your constraints file ? | 20:56 |
| cjwatson | zope.security 4.x goes through a different chain, but it does seem to work with a fresh virtualenv, let's see | 20:56 |
| cjwatson | http://paste.ubuntu.com/12144571/ | 20:57 |
| cjwatson | oh and also an initial run before that with http://paste.ubuntu.com/12144573/ as -r | 20:58 |
| lifeless | cjwatson: are you capturing both stderr and stdout ? | 20:58 |
| cjwatson | yes | 20:58 |
| lifeless | hmm | 20:58 |
| lifeless | I'm not seeing 'collecting' | 20:58 |
| cjwatson | that was in an env that already had lp installed | 20:59 |
| lifeless | -at all- | 20:59 |
| lifeless | yeah, its the constraints file | 21:00 |
| lifeless | workaround - put the [] in the constraints file | 21:00 |
| lifeless | I'll file a bug now | 21:00 |
| lifeless | [I wrote the constraints support for pip :), so was a very good choice to ask] | 21:01 |
| cjwatson | aha, the constraints implies a null set of extras? | 21:01 |
| lifeless | no | 21:01 |
| lifeless | its worse | 21:01 |
| cjwatson | brilliant, thanks - I thought you might be | 21:01 |
| lifeless | pip doesn't merge together extras references from different places in the dependency graph | 21:01 |
| lifeless | which is a subset of the 'does not do resolving at all' | 21:01 |
| lifeless | problem | 21:02 |
| cjwatson | I'm using 7.1.0 FWIW but 7.1.1's changelog doesn't look different here | 21:02 |
| cjwatson | and confirmed that adding the [] in the constraints file fixes it, thanks | 21:04 |
| lifeless | https://github.com/pypa/pip/issues/3046 | 21:06 |
| lifeless | yw | 21:07 |
| cjwatson | now I wonder how to grep for what extras *should* be installed - I guess just compare with the buildout-installed package list | 21:08 |
| lifeless | well, they'll be getting in the dependency list somehow | 21:08 |
| lifeless | so I think a two-phase thing | 21:09 |
| lifeless | phase 1, any extras you define | 21:10 |
| lifeless | phase 2, interrogate the dist-requires of all the things you depend on, transitively | 21:10 |
| lifeless | you could automate phase2 by runnign 'pip install' of each thing you directly dep on separately, and looking in the output: | 21:10 |
| lifeless | perhaps | 21:11 |
| * lifeless shrugs ;) | 21:11 | |
| cjwatson | I just did the stupid thing and did bin/buildout -vc buildout.cfg, mangled output into roughly compatible form, compared with pip list | 21:14 |
| cjwatson | some noise from buildout 1's semi-isolation vs. virtualenv --system-site-packages just allowing everything | 21:14 |
| lifeless | auditing the final output is probably sufficient | 21:15 |
| lifeless | you might end up there coincidentally and be missing a extra for the future | 21:15 |
| cjwatson | yeah, I really only have to do this once | 21:15 |
| cjwatson | true | 21:15 |
| lifeless | but, we're going to get that bug fixed | 21:15 |
| lifeless | so its a time limited window | 21:15 |
| cjwatson | once we're on virtualenv/pip, it's a lot easier to upgrade | 21:15 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!