mycli: 1.6.0 -> 1.17.0 #44571

kalbasit · 2018-08-06T20:00:24Z

Motivation for this change

I'm getting errors when using mycli 1.6.0, not getting errors after the update.

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/nix/store/7sbvn0wgv7hsnxpss1jba723kx5nz6d4-python-2.7.15/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/nix/store/7sbvn0wgv7hsnxpss1jba723kx5nz6d4-python-2.7.15/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/nix/store/8gs9jrpz8g9hmj4cz1326mr8k39y9nyw-python2.7-prompt_toolkit-1.0.15/lib/python2.7/site-packages/prompt_toolkit/interface.py", line 865, in run
    completions = list(buffer.completer.get_completions(document, complete_event))
  File "/nix/store/rsvy8bnpdninfzm9x3vk8l7a7vsgmzns-mycli-1.6.0/lib/python2.7/site-packages/mycli/sqlcompleter.py", line 255, in get_completions
    suggestions = suggest_type(document.text, document.text_before_cursor)
  File "/nix/store/rsvy8bnpdninfzm9x3vk8l7a7vsgmzns-mycli-1.6.0/lib/python2.7/site-packages/mycli/packages/completion_engine.py", line 85, in suggest_type
    full_text, identifier)
  File "/nix/store/rsvy8bnpdninfzm9x3vk8l7a7vsgmzns-mycli-1.6.0/lib/python2.7/site-packages/mycli/packages/completion_engine.py", line 134, in suggest_based_on_last_token
    token_v = token.value.lower()
AttributeError: 'tuple' object has no attribute 'value'

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

closes kalbasit/shabka#32

dotlambda · 2018-08-06T20:16:21Z

pkgs/top-level/python-packages.nix

@@ -1102,6 +1102,20 @@ in {

  cheroot = callPackage ../development/python-modules/cheroot {};

+  cli-helpers = buildPythonPackage rec {


This needs to be put in pkgs/development/python-modules/cli-helpers/default.nix. The expression should adhere to the newest standards (pname etc.) and adding cli-helpers should be a separate commit.

sounds good, I'll update the PR later today.

dotlambda · 2018-08-06T20:19:39Z

Also note

No tests in archive. Newer versions do include tests

So please enable tests again.

dotlambda · 2018-08-07T06:06:29Z

@kalbasit No need to have the second PR. Please remove the merge commit from this one.

@GrahamcOfBorg build python2.pkgs.cli-helpers python3.pkgs.cli-helpers

GrahamcOfBorg · 2018-08-07T06:09:05Z

Success on aarch64-linux (full log)

Attempted: python2.pkgs.cli-helpers, python3.pkgs.cli-helpers

Partial log (click to expand)

reading manifest template 'MANIFEST.in'
writing manifest file 'cli_helpers.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.001s

OK
/nix/store/4fq6cyqv3x3p44rq5ili3mgsnywfbiwp-python2.7-cli_helpers-1.0.2
/nix/store/wz7ky5g0x4i7w3b5nwh7vj0d2p1jwjxj-python3.6-cli_helpers-1.0.2

dotlambda · 2018-08-07T06:10:01Z

Ran 0 tests in 0.001s

Please provide an appropriate checkPhase or set doCheck = false and add a comment stating the reason.

GrahamcOfBorg · 2018-08-07T06:10:03Z

Success on x86_64-linux (full log)

Attempted: python2.pkgs.cli-helpers, python3.pkgs.cli-helpers

Partial log (click to expand)

reading manifest template 'MANIFEST.in'
writing manifest file 'cli_helpers.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/vy0c0iaz6bhqhzmwbqw5k1spnipxn65a-python2.7-cli_helpers-1.0.2
/nix/store/s60y6x15c2j6azb89bbh14x5hgjmn5sr-python3.6-cli_helpers-1.0.2

kalbasit · 2018-08-07T06:21:23Z

@dotlambda the lack of tests you are talking about is actually in #44578 (you asked for this PR in an earlier comment). I need help on #44578 to figure out the tests part. I'll squash the commits after you merge #44578.

kalbasit · 2018-08-07T06:33:13Z

@dotlambda I edited the commits, the first one is #44578 verbatim and the second one is the primary commit of this PR.

dotlambda · 2018-08-07T06:36:21Z

But now you removed the commit regarding pymysql 😕

you asked for this PR in an earlier comment

No, I just asked for a separate commit, no need to have multiple PRs.

kalbasit · 2018-08-07T06:37:43Z

oh, I see what you mean. Ok give me a second, I will split the last commit and close #44578.

dotlambda · 2018-08-07T06:38:36Z

Judging from tox.ini, it seems like they're using pytest, so the following might be enough

checkInputs = [ pytest ];
checkPhase = ''
  py.test
'';

dotlambda · 2018-08-07T06:44:51Z

@GrahamcOfBorg build python2.pkgs.cli-helpers python3.pkgs.cli-helpers

GrahamcOfBorg · 2018-08-07T06:45:43Z

Success on x86_64-linux (full log)

Attempted: python2.pkgs.cli-helpers, python3.pkgs.cli-helpers

Partial log (click to expand)

tests/tabular_output/test_delimited_output_adapter.py ..                 [ 23%]
tests/tabular_output/test_output_formatter.py ..........                 [ 44%]
tests/tabular_output/test_preprocessors.py .........s.sss.......         [ 89%]
tests/tabular_output/test_tabulate_adapter.py ..                         [ 93%]
tests/tabular_output/test_terminaltables_adapter.py .                    [ 95%]
tests/tabular_output/test_vertical_table_adapter.py ..                   [100%]

===================== 43 passed, 4 skipped in 0.19 seconds =====================
/nix/store/fasxqcn046r34k8rxzc2lmllqn1rkgzm-python2.7-cli_helpers-1.0.2
/nix/store/f69fw507vkv17zw14fhmlrsaqls1z6kv-python3.6-cli_helpers-1.0.2

GrahamcOfBorg · 2018-08-07T06:46:21Z

Success on aarch64-linux (full log)

Attempted: python2.pkgs.cli-helpers, python3.pkgs.cli-helpers

Partial log (click to expand)

tests/tabular_output/test_delimited_output_adapter.py ..                 [ 23%]
tests/tabular_output/test_output_formatter.py ..........                 [ 44%]
tests/tabular_output/test_preprocessors.py .........s.sss.......         [ 89%]
tests/tabular_output/test_tabulate_adapter.py ..                         [ 93%]
tests/tabular_output/test_terminaltables_adapter.py .                    [ 95%]
tests/tabular_output/test_vertical_table_adapter.py ..                   [100%]

===================== 43 passed, 4 skipped in 2.42 seconds =====================
/nix/store/vmbqdprcz3m05q5qrrck03zwbg7k7515-python2.7-cli_helpers-1.0.2
/nix/store/qkmlm9lgb65xch5xnpahvfz1i89vgiw5-python3.6-cli_helpers-1.0.2

kalbasit · 2018-08-07T06:49:47Z

@dotlambda can you also build mycli to make sure it works on borg.

dotlambda · 2018-08-07T06:47:10Z

pkgs/tools/admin/mycli/default.nix

@@ -1,28 +1,31 @@
 { lib
 , python
+, pythonPackages


Drop this line

dotlambda · 2018-08-07T06:47:41Z

pkgs/tools/admin/mycli/default.nix

-  '';
+  patches = [ ./fix-tests.patch ];
+
+  checkInputs = with pythonPackages; [ pytest mock ];


Drop with pythonPackages;. There's with python.pkgs; at the top.

dotlambda · 2018-08-07T06:48:22Z

pkgs/tools/admin/mycli/default.nix

-  doCheck = false;
+  preCheck = ''
+    echo '[main]' > /tmp/myclirc
+    echo 'log_file = /tmp/mycli.test.log' >> /tmp/myclirc


From where would myclirc be read without the patch?

From nowhere, and so the library will end up calling write_default_config which writes the config file at /homeless-shelter/.myclirc and that of course fails. Setting the myclirc stops it from writing the config file. Also, I specified the log_file option because another test invokes logging and that defaults to /homeless-shelter/.mycli.log.

How about setting something like HOME=.?

I'll try that!

This worked! Thanks for the tip.

dotlambda · 2018-08-07T06:49:56Z

pkgs/tools/admin/mycli/default.nix

@@ -1,28 +1,31 @@
 { lib
 , python


If mycli is compatible with python3, please replace this by python3. Otherwise, by python2.

I tried using python3, the tests fail like this:

=================================== FAILURES =================================== ________________________________ test_list_dsn _________________________________ def test_list_dsn(): runner = CliRunner() with NamedTemporaryFile(mode="w") as myclirc: myclirc.write(dedent("""\ [alias_dsn] test = mysql://test/test [main] log_file = /tmp/mycli.test.log """)) myclirc.flush() args = ['--list-dsn', '--myclirc', myclirc.name] result = runner.invoke(cli, args=args) > assert result.output == "test\n" E AssertionError: assert '' == 'test\n' E + test args = ['--list-dsn', '--myclirc', '/build/tmpbldvl9dg'] myclirc = <tempfile._TemporaryFileWrapper object at 0x7fffedb39da0> result = <Result RuntimeError('Click will abort further execution because Python 3 was configured to use ASCII as encoding for ... suitable UTF-8\nlocales were discovered. This most likely requires resolving\nby reconfiguring the locale system.',)> runner = <click.testing.CliRunner object at 0x7fffedb39dd8> test/test_main.py:287: AssertionError ==== 1 failed, 186 passed, 59 skipped, 1 xfailed, 1 xpassed in 5.42 seconds ==== builder for '/nix/store/m4vvpfnpgnlsa265cagv1hq99yfh5fs8-mycli-1.17.0.drv' failed with exit code 1 e

I put python2 there for now, we can revisit in a separate PR.

dotlambda · 2018-08-07T06:51:49Z

@GrahamcOfBorg build mycli

GrahamcOfBorg · 2018-08-07T06:52:53Z

Success on x86_64-linux (full log)

Attempted: mycli

Partial log (click to expand)

test/test_naive_completion.py ....                                       [ 61%]
test/test_parseutils.py .............X.........                          [ 72%]
test/test_prompt_utils.py .                                              [ 72%]
test/test_smart_completion_public_schema_only.py ......................  [ 82%]
test/test_special_iocommands.py ......s...sssss                          [ 88%]
test/test_sqlexecute.py ssssssssssssssssssssssss                         [ 99%]
test/test_tabular_output.py s                                            [100%]

========= 187 passed, 59 skipped, 1 xfailed, 1 xpassed in 5.53 seconds =========
/nix/store/dwg6jmyl8xp0fxq9ai909cfw4j61jyj0-mycli-1.17.0

GrahamcOfBorg · 2018-08-07T06:54:51Z

Success on aarch64-linux (full log)

Attempted: mycli

Partial log (click to expand)

test/test_naive_completion.py ....                                       [ 61%]
test/test_parseutils.py .............X.........                          [ 72%]
test/test_prompt_utils.py .                                              [ 72%]
test/test_smart_completion_public_schema_only.py ......................  [ 82%]
test/test_special_iocommands.py ......s...sssss                          [ 88%]
test/test_sqlexecute.py ssssssssssssssssssssssss                         [ 99%]
test/test_tabular_output.py s                                            [100%]

======== 187 passed, 59 skipped, 1 xfailed, 1 xpassed in 34.64 seconds =========
/nix/store/b7gig4m3vngywdd0mmhyxaarxjz1nb65-mycli-1.17.0

dotlambda · 2018-08-07T07:03:54Z

@GrahamcOfBorg build python2.pkgs.pymysql python3.pkgs.pymysql mycli

GrahamcOfBorg · 2018-08-07T07:04:19Z

Success on x86_64-linux (full log)

Attempted: python2.pkgs.pymysql, python3.pkgs.pymysql, mycli

Partial log (click to expand)

/build/PyMySQL-0.9.2
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/4yb8pfq4kqc9avfk2ymrcc4bv980zavd-python3.6-PyMySQL-0.9.2
strip is /nix/store/1hi76hr87bd1y1q1qjk0lv8nmcjip1c8-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/4yb8pfq4kqc9avfk2ymrcc4bv980zavd-python3.6-PyMySQL-0.9.2/lib
patching script interpreter paths in /nix/store/4yb8pfq4kqc9avfk2ymrcc4bv980zavd-python3.6-PyMySQL-0.9.2
checking for references to /build in /nix/store/4yb8pfq4kqc9avfk2ymrcc4bv980zavd-python3.6-PyMySQL-0.9.2...
/nix/store/bbbkb80zv0fyhni3kiv1hfnv5dnkyyww-python2.7-PyMySQL-0.9.2
/nix/store/4yb8pfq4kqc9avfk2ymrcc4bv980zavd-python3.6-PyMySQL-0.9.2
/nix/store/dy690ly2vx812hrh2v5yvyq3m9r639sx-mycli-1.17.0

GrahamcOfBorg · 2018-08-07T07:07:04Z

Success on aarch64-linux (full log)

Attempted: python2.pkgs.pymysql, python3.pkgs.pymysql, mycli

Partial log (click to expand)

test/test_prompt_utils.py .                                              [ 72%]
test/test_smart_completion_public_schema_only.py ......................  [ 82%]
test/test_special_iocommands.py ......s...sssss                          [ 88%]
test/test_sqlexecute.py ssssssssssssssssssssssss                         [ 99%]
test/test_tabular_output.py s                                            [100%]

======== 187 passed, 59 skipped, 1 xfailed, 1 xpassed in 37.76 seconds =========
/nix/store/lw2mk8kllzm9f95n6fl4i1q4nhq7xzkl-python2.7-PyMySQL-0.9.2
/nix/store/bnvddjchn56ng9wymq2bslsmqd9x9avi-python3.6-PyMySQL-0.9.2
/nix/store/8pp84gs3iqdd6iwv77lid6dqdzf01i2n-mycli-1.17.0

dotlambda · 2018-08-07T07:09:59Z

pkgs/tools/admin/mycli/default.nix

@@ -1,28 +1,29 @@
 { lib
-, python
+, python2


According to https://github.com/dbcli/mycli/blob/19619a4dfa0682654634c33e4ec55263f82475b6/tox.ini#L2, mycli supports python3.

I added a comment about it above, pasting here as well.

I tried using python3, the tests fail like this:

=================================== FAILURES =================================== ________________________________ test_list_dsn _________________________________ def test_list_dsn(): runner = CliRunner() with NamedTemporaryFile(mode="w") as myclirc: myclirc.write(dedent("""\ [alias_dsn] test = mysql://test/test [main] log_file = /tmp/mycli.test.log """)) myclirc.flush() args = ['--list-dsn', '--myclirc', myclirc.name] result = runner.invoke(cli, args=args) > assert result.output == "test\n" E AssertionError: assert '' == 'test\n' E + test args = ['--list-dsn', '--myclirc', '/build/tmpbldvl9dg'] myclirc = <tempfile._TemporaryFileWrapper object at 0x7fffedb39da0> result = <Result RuntimeError('Click will abort further execution because Python 3 was configured to use ASCII as encoding for ... suitable UTF-8\nlocales were discovered. This most likely requires resolving\nby reconfiguring the locale system.',)> runner = <click.testing.CliRunner object at 0x7fffedb39dd8> test/test_main.py:287: AssertionError ==== 1 failed, 186 passed, 59 skipped, 1 xfailed, 1 xpassed in 5.42 seconds ==== builder for '/nix/store/m4vvpfnpgnlsa265cagv1hq99yfh5fs8-mycli-1.17.0.drv' failed with exit code 1 e

I put python2 there for now, we can revisit in a separate PR.

You have to add glibcLocales to checkInputs and add LC_ALL = "en_US.UTF-8"; to the expression.

Nice! it worked!

dotlambda · 2018-08-07T07:10:19Z

pkgs/development/python-modules/pymysql/default.nix

+
+  propagatedBuildInputs = [ cryptography ];
+
+  buildInputs = [ unittest2 ];


I think this is not necessary.

dotlambda · 2018-08-07T07:10:33Z

pkgs/development/python-modules/pymysql/default.nix

+{ lib
+, buildPythonPackage
+, fetchPypi
+, python


Drop this line.

dotlambda · 2018-08-07T07:12:42Z

pkgs/development/python-modules/pymysql/default.nix

+    description = "Pure Python MySQL Client";
+    homepage = https://github.com/PyMySQL/PyMySQL;
+    license = licenses.mit;
+    maintainers = [ maintainers.fridh maintainers.kalbasit ];


Why did you add fridh?

I ran git blame and tracked down the first commit to him, should I remove him?

Yes, he's the main python maintainer but probably does not particularly maintain this package.

kalbasit · 2018-08-07T07:24:14Z

@dotlambda PTAL.

kalbasit · 2018-08-07T07:29:52Z

@dotlambda I simplified mycli even more, there really was no need for the patch, PTAL.

dotlambda · 2018-08-07T08:15:35Z

@GrahamcOfBorg build mycli

GrahamcOfBorg · 2018-08-07T08:16:38Z

Success on x86_64-linux (full log)

Attempted: mycli

Partial log (click to expand)

test/test_naive_completion.py ....                                       [ 61%]
test/test_parseutils.py .............X.........                          [ 72%]
test/test_prompt_utils.py .                                              [ 72%]
test/test_smart_completion_public_schema_only.py ......................  [ 82%]
test/test_special_iocommands.py ......s...sssss                          [ 88%]
test/test_sqlexecute.py ssssssssssssssssssssssss                         [ 99%]
test/test_tabular_output.py s                                            [100%]

========= 187 passed, 59 skipped, 1 xfailed, 1 xpassed in 6.24 seconds =========
/nix/store/wjzgr8ffjvkyvx02mm1hi708ngy906hh-mycli-1.17.0

GrahamcOfBorg · 2018-08-07T08:22:43Z

Success on aarch64-linux (full log)

Attempted: mycli

Partial log (click to expand)

test/test_naive_completion.py ....                                       [ 61%]
test/test_parseutils.py .............X.........                          [ 72%]
test/test_prompt_utils.py .                                              [ 72%]
test/test_smart_completion_public_schema_only.py ......................  [ 82%]
test/test_special_iocommands.py ......s...sssss                          [ 88%]
test/test_sqlexecute.py ssssssssssssssssssssssss                         [ 99%]
test/test_tabular_output.py s                                            [100%]

======== 187 passed, 59 skipped, 1 xfailed, 1 xpassed in 39.26 seconds =========
/nix/store/d3bj0qrl37l1zr95riy79hm4p00gxb7k-mycli-1.17.0

kalbasit requested a review from FRidh as a code owner August 6, 2018 20:00

GrahamcOfBorg added 6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Aug 6, 2018

dotlambda requested changes Aug 6, 2018

View reviewed changes

kalbasit mentioned this pull request Aug 6, 2018

pythonPackages.cli-helpers: init at 1.0.2 #44578

Closed

9 tasks

kalbasit force-pushed the nixpkgs-update-mycli branch from 1254d59 to 4dd71f5 Compare August 6, 2018 23:25

kalbasit changed the title ~~mycli: 1.6.0 -> 1.17.0~~ [WIP] mycli: 1.6.0 -> 1.17.0 Aug 6, 2018

kalbasit force-pushed the nixpkgs-update-mycli branch from 4dd71f5 to 8eb691d Compare August 6, 2018 23:41

kalbasit mentioned this pull request Aug 7, 2018

pythonPackages.pymysql: 0.6.6 -> 0.9.2; move to python-modules. #44582

Closed

9 tasks

kalbasit force-pushed the nixpkgs-update-mycli branch from bbaee4d to 1c52682 Compare August 7, 2018 06:30

kalbasit force-pushed the nixpkgs-update-mycli branch from 1c52682 to b5c6158 Compare August 7, 2018 06:41

kalbasit changed the title ~~[WIP] mycli: 1.6.0 -> 1.17.0~~ mycli: 1.6.0 -> 1.17.0 Aug 7, 2018

pythonPackages.cli-helpers: init at 1.0.2

1faec39

kalbasit force-pushed the nixpkgs-update-mycli branch from b5c6158 to fcacb4f Compare August 7, 2018 06:44

dotlambda requested changes Aug 7, 2018

View reviewed changes

kalbasit force-pushed the nixpkgs-update-mycli branch from fcacb4f to c0d2ac8 Compare August 7, 2018 07:01

dotlambda reviewed Aug 7, 2018

View reviewed changes

kalbasit force-pushed the nixpkgs-update-mycli branch 2 times, most recently from 65273ef to d934f32 Compare August 7, 2018 07:17

pythonPackages.pymysql: 0.6.6 -> 0.9.2; move to python-modules

16cfc2a

kalbasit force-pushed the nixpkgs-update-mycli branch from d934f32 to 1e26e23 Compare August 7, 2018 07:18

mycli: 1.6.0 -> 1.17.0

e9cb147

kalbasit force-pushed the nixpkgs-update-mycli branch from 1e26e23 to e9cb147 Compare August 7, 2018 07:29

dotlambda merged commit 746aae0 into NixOS:master Aug 7, 2018

kalbasit deleted the nixpkgs-update-mycli branch August 7, 2018 15:04

ento mentioned this pull request Sep 28, 2020

zerobin: update to v1.0.5 #98734

Merged

14 tasks

		@@ -1102,6 +1102,20 @@ in {

		cheroot = callPackage ../development/python-modules/cheroot {};

		cli-helpers = buildPythonPackage rec {


		propagatedBuildInputs = [ cryptography ];

		buildInputs = [ unittest2 ];

mycli: 1.6.0 -> 1.17.0 #44571

mycli: 1.6.0 -> 1.17.0 #44571

Conversation

kalbasit commented Aug 6, 2018 • edited

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda commented Aug 6, 2018

dotlambda commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

dotlambda commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

kalbasit commented Aug 7, 2018

kalbasit commented Aug 7, 2018

dotlambda commented Aug 7, 2018

kalbasit commented Aug 7, 2018

dotlambda commented Aug 7, 2018

dotlambda commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

kalbasit commented Aug 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

dotlambda commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kalbasit commented Aug 7, 2018

kalbasit commented Aug 7, 2018

dotlambda commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

GrahamcOfBorg commented Aug 7, 2018

kalbasit commented Aug 6, 2018 •

edited