New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
credstash: add standalone Python application #51807
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
credstash was only available as a library. Provide it as a standalone application as well. In order for this to work, I needed to remove the copy of the library that's placed in $out/bin and marked executable during the install phase. Other than the patched shebang and executable bit, it's identical to the library that's installed to $out/lib/python3.7/site-packages. Before the postFixup has run `wrapPythonPrograms`, $out/bin contains two Python files -- credstash and credstash.py -- where bin/credstash is the executable you'd expect a user to invoke from the command-line and bin/credstash.py contains the credstash module, which bin/credstash imports. After `wrapPythonPrograms` has run, bin/credstash is a shell wrapper around the bin/.credstash-wrapped python entrypoint, and bin/credstash.py is shell wrapper around bin/.credstash.py-wrapped. Invoking bin/credstash execs bin/.credstash-wrapped, and that python script attempts to import the credstash module from bin/credstash.py, the shell wrapper, rather than either bin/.credstash.py-wrapped or lib/python3.7/site-packages/credstash.py. This leads to an error: $ credstash get mykey Traceback (most recent call last): File "/nix/store/hk6yma716w6141lcdh509d6qyyi7zm0i-python3.7-credstash-1.15.0/bin/.credstash-wrapped", line 8, in <module> from credstash import main File "/nix/store/hk6yma716w6141lcdh509d6qyyi7zm0i-python3.7-credstash-1.15.0/bin/credstash.py", line 2 export PATH='/nix/store/6lm4gi5iv8fbf1b1mm6g3gfnnv63f1gn-python3-3.7.1/bin:/nix/store/hk6yma716w6141lcdh509d6qyyi7zm0i-python3.7-credstash-1.15.0/bin:/nix/store/2n13gf1zdr39ir5dynxlkqndxgy36g08-python3.7-setuptools-40.4.3/bin:/nix/store/mhnqwpa4y1l81zi4cwx989i8h8z9g67l-python3.7-jmespath-0.9.0/bin:/nix/store/qc6q3a2nv4211wyh7q319v6zzd3ab6pc-python3.7-docutils-0.14/bin'${PATH:+':'}$PATH ^ SyntaxError: invalid syntax If we try using `dontWrapPythonPrograms` to resolve this, runtime dependency lookups fail: $ credstash get mykey Traceback (most recent call last): File "/run/current-system/sw/bin/credstash", line 7, in <module> from credstash import main File "/nix/store/8rmldlvlv1z1xl7w02dy7f5qhkzdrg8z-python3.7-credstash-1.15.0/bin/credstash.py", line 26, in <module> import boto3 ModuleNotFoundError: No module named 'boto3' I was able to resolve things by simply removing bin/credstash.py before the postFixup phase has a chance to wrap any executables. Now the executable imports the library correctly: (shell wrapper) bin/credstash │ (python executable) └─> bin/.credstash-wrapped │ (python library) └─> lib/python3.7/site-packages/credstash.py
dotlambda
reviewed
Dec 10, 2018
Make it clear why we're removing this file. For a more in-depth description of the issue it addresses, see the previous commit: 2fc870e
ivanbrennan
force-pushed
the
credstash-application
branch
from
December 10, 2018 13:04
8f94561
to
b85f82c
Compare
@GrahamcOfBorg build credstash |
What is not clear to me is why this is even in |
@dotlambda |
Same here. We have some internal tools that use the library to interact with credstash. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for this change
I want to use the credstash command-line application, but credstash is currently only offered as a Python library.
In order for this to work, I needed to remove the copy of the library that's placed in
$out/bin
and marked executable during the install phase. Other than the patched shebang and executable bit, it's identical to the library that's installed to$out/lib/python3.7/site-packages
.Before postFixup has run
wrapPythonPrograms
, bin/ contains two Python files -- credstash and credstash.py -- wherebin/credstash
is the executable you'd expect a user to invoke from the command-line andbin/credstash.py
contains the credstash module, whichbin/credstash
imports.After
wrapPythonPrograms
has run,bin/credstash
is a shell wrapper around thebin/.credstash-wrapped
python entrypoint, andbin/credstash.py
is shell wrapper around thebin/.credstash.py-wrapped
python module. Invokingbin/credstash
execsbin/.credstash-wrapped
, which attempts to import the credstash module frombin/credstash.py
(now a shell script), rather than eitherbin/.credstash.py-wrapped
orlib/python3.7/site-packages/credstash.py
.This leads to an error:
I was able to resolve this by removing
bin/credstash.py
before the postFixup phase has a chance to wrap anything. Now the executable imports the library correctly:I tried using
dontWrapPythonPrograms
as an alternative solution, but that leads to a different error because runtime dependency lookups fail:Fixes #47751
Things done
sandbox
innix.conf
on non-NixOS)nix-shell -p nox --run "nox-review wip"
./result/bin/
)nix path-info -S
before and after)