Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mitmproxy: convert to python3Packages #93701

Merged
merged 2 commits into from Sep 8, 2020
Merged

mitmproxy: convert to python3Packages #93701

merged 2 commits into from Sep 8, 2020

Conversation

veehaitch
Copy link
Member

@veehaitch veehaitch commented Jul 23, 2020

Motivation for this change

For the scripting API of mitmproxy, it is useful to have it available as Python module:

  • Converted mitmproxy to a Python 3 package
  • I've also removed the following dependencies which are apparently not required (any more):
    • cffi
    • h11
    • hpack
    • itsdangerous
    • jinja2
    • markupsafe
    • publicsuffix2
    • pycparser
    • six
    • werkzeug
  • Tests run fine on NixOS
  • mitmproxy v5.2 is already out but apparently it has issues with urwid 2.1: downgrade & pin urwid to 2.0.1 mitmproxy/mitmproxy#4086
  • Updated mitmproxy to v5.2
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@FRidh
Copy link
Member

FRidh commented Jul 29, 2020

It is likely the module will break at times because dependencies cannot be met, which can be overridden as an application. In the past dependencies had to be overridden quite often, so I am a bit skeptical to this change. Not blocking it though.

@veehaitch
Copy link
Member Author

It is likely the module will break at times because dependencies cannot be met, which can be overridden as an application. In the past dependencies had to be overridden quite often, so I am a bit skeptical to this change. Not blocking it though.

I see your point; actually, exactly this is what happened for Mitmproxy v5.2 which downgraded urwid. Guess we have to skip this version and wait for 5.2.1. Still, IMHO, its the better option to have a Python package here.

@veehaitch
Copy link
Member Author

I've updated mitmproxy to v5.2 since upstream introduced support for urwid v2.1.1.

@ofborg ofborg bot requested a review from kamilchm July 30, 2020 13:46
@FRidh
Copy link
Member

FRidh commented Jul 30, 2020

I see your point; actually, exactly this is what happened for Mitmproxy v5.2 which downgraded urwid.

Well, then I take it you accept it that there may be periods of time it is not possible to have a functioning mitmproxy in Nixpkgs.

@veehaitch
Copy link
Member Author

I see your point; actually, exactly this is what happened for Mitmproxy v5.2 which downgraded urwid.

Well, then I take it you accept it that there may be periods of time it is not possible to have a functioning mitmproxy in Nixpkgs.

There might be some quirks, yes. But this also holds true for other Python packages in Nixpkgs, doesn't it?

@jonringer
Copy link
Contributor

@veehaitch that's correct. Due to how python can only import one version of a module, introducing many versions of a given dependency is not feasible, and will likely break some users depending on which packages are introduced.

@risicle
Copy link
Contributor

risicle commented Sep 5, 2020

Due to how python can only import one version of a module, introducing many versions of a given dependency is not feasible

@jonringer I've been thinking about the fundamental "why?"s of this lately, because I realized that it's not as if even c allows you to "import" (link to) more than one version of a library at once. And we don't have this "one version" stipulation with pkgs/development/libraries/. I think the difference really is that with python we expect to be able to compose an arbitrary collection of python packages together into an env and for it to "just work", a common python pattern that doesn't have a straight equivalent in the world of c libraries.

Anyway as far as I'm concerned, this is something that you've removed from my todo list, @veehaitch.

Though I think you might have got the patch's hash wrong (I get 1z5r8izg5nvay01ywl3xc6in1vjfi9f144j057p3k5rzfliv49gg). And it's always nice to give fetchpatch patches a name, e.g. name = "urwid-lt-2.1.0.patch";

And this'll need a rebase so that it survives the big python-packages.nix sort of 2020.

Otherwise WFM macos 10.14. 👍

@veehaitch
Copy link
Member Author

Though I think you might have got the patch's hash wrong (I get 1z5r8izg5nvay01ywl3xc6in1vjfi9f144j057p3k5rzfliv49gg). And it's always nice to give fetchpatch patches a name, e.g. name = "urwid-lt-2.1.0.patch";

Indeed, thanks! Done.

And this'll need a rebase so that it survives the big python-packages.nix sort of 2020.

Done.

Additionally, the derivation now runs all the tests except one. All tests pass on NixOS. Builds and works on macOS 10.15.6.

Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

https://github.com/NixOS/nixpkgs/pull/93701
2 packages built:
mitmproxy python37Packages.mitmproxy

@jonringer jonringer merged commit 9c6c7ac into NixOS:master Sep 8, 2020
@veehaitch veehaitch deleted the mitmproxy-module branch September 8, 2020 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants