Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use back up versions in case pip-compile fails to do dependency resolution #394

Closed
wants to merge 1 commit into from

Conversation

milin
Copy link

@milin milin commented Sep 23, 2016

This is a fix for the following issue #366 where two level dependencies in turn have a single dependency that are pinned to different versions respectively. In such a case the following error happens.

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Finding the best candidates:
  found candidate alabaster==0.7.9 (constraint was >=0.7,<0.8)
  found candidate argh==0.26.2 (constraint was >=0.24.1)
  found candidate argparse==1.4.0 (constraint was <any>)
  found candidate asq==1.0 (constraint was ==1.0)
  found candidate astroid==1.0.1 (constraint was ==1.0.1)
  found candidate autopep8==1.1.1 (constraint was ==1.1.1)
  found candidate babel==2.3.4 (constraint was >=1.3,!=2.0)
  found candidate backports.ssl-match-hostname==3.5.0.1 (constraint was <any>)
  found candidate beautifulsoup4==4.4.1 (constraint was ==4.4.1)
  found candidate billiard==3.3.0.23 (constraint was >=3.3.0.21,<3.4)
  found candidate bleach==1.4.2 (constraint was ==1.4.2)
  found candidate blessings==1.6 (constraint was ==1.6)
  found candidate celery==3.1.19 (constraint was ==3.1.19)
  found candidate celery-redis-sentinel==0.3 (constraint was ==0.3)
  found candidate certifi==2016.8.31 (constraint was <any>)
  found candidate cffi==1.7.0 (constraint was ==1.7.0)
  found candidate click==6.3 (constraint was ==6.3)
  found candidate contextlib2==0.5.4 (constraint was <any>)
Could not find a version that matches coverage==3.6,==3.7.1,>=3.4
Tried: 2.8, 2.77, 2.78, 2.80, 2.85, 3.0b3, 3.0, 3.0.1, 3.1b1, 3.1, 3.2b1, 3.2b2, 3.2b3, 3.2b4, 3.2, 3.3, 3.3.1, 3.4b1, 3.4b2, 3.4, 3.5b1, 3.5, 3.5.1b1, 3.5.1,
 3.5.2b1, 3.5.2, 3.5.3, 3.6b1, 3.6b2, 3.6b3, 3.6, 3.7, 3.7.1, 4.0a1, 4.0a2, 4.0a3, 4.0a4, 4.0a5, 4.0a5, 4.0a5, 4.0a6, 4.0a6, 4.0a6, 4.0b1, 4.0b1, 4.0b2, 4.0b2
, 4.0b3, 4.0b3, 4.0, 4.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.0.3, 4.0.3, 4.1b1, 4.1b1, 4.1b2, 4.1b2, 4.1b3, 4.1b3, 4.1, 4.1, 4.2b1, 4.2b1, 4.2, 4.2

In the above example I might only care about coverage==3.6 and not care about other versions. But right now in the above scenario pip-compile just quits on us.

In such a case, this pr gives the option to provide an optional parameter to a file containing json data that has backup versions of your required dependencies, in case pip-compile faces the above error.

For e.g in this case a backup_versions.txt could simply be a file containing a json below.

{
   "coverage": "3.6",
   "click": "6.6"   
}
# Add other backup versions similarly.

Then you would just provide the path of the backup file when doing pip-compile

pip-compile -vv requirements.in --backup-versions backup_versions.txt

@milin milin changed the title Back up versions. Fix for https://github.com/nvie/pip-tools/issues/366 Use back up versions in case pip-compile fails to do dependency resolution Sep 23, 2016
@milin milin force-pushed the back_up_versions branch 5 times, most recently from 0331f26 to 108f7f5 Compare September 23, 2016 21:48
Fix for multiple versions issue.

Add more description

clean things up

Refactor _get_best_candidate.

Refactor.

Fix test

Ignore flake8 max line

Add simplejson

Revert the version bump

Add simplejson as requirement
@nvie
Copy link
Member

nvie commented Sep 26, 2016

Well, pip-compile is a dependency management tool. If you have 2 or more version requirements that are incompatible with each other, the only valid behaviour for pip-compile is to fail. This isn't super user friendly, but at least it's the correct thing to do. After all, it's the entire purpose of pip-tools in the first place to resolve these things and tell you about incompatibilities that will otherwise cause runtime problems.

I've thought about building a flag into pip-compile for this, but it leads to a lot of follow-up questions, like how do you communicate the error, etc? I thought about adding a flag to "simply ignore these", or "pick an arbitrary version and pin that", but then how do you treat secondary dependencies? It opens cans of worms everywhere.

The true fix for this is to reach out to the package maintainers and ask them to fix their dependency declarations. If you need a practical workaround for this, try manually editing the compiled requirements.txt until this problem is resolved.

@nvie nvie closed this Sep 26, 2016
@milin
Copy link
Author

milin commented Sep 26, 2016

It fails during the process so the compiled file is not generated at all. Therefore manually editing the compiled file is not an option. I think if the user knows which version of a particular package pip compile should pin, we should let them have fall back versions, if in case pip-compile fails to do a resolution.

@nvie
Copy link
Member

nvie commented Sep 26, 2016

Agreed, this might be a bit troublesome. I have not had a change to review #388 fully yet, but that might be the way to go. You could --exclude the troublesome package, and either manually pin it after compilation succeeded, or leave it to chance by not adding it at all (which will be the equivalent in practice anyway — it's undefined which version you'll get in production).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants