bpo-29636: Add --indent / --no-indent arguments to json.tool #345

dhimmel · 2017-02-27T15:37:41Z

From #201 which is being split into two pull requests.

Tagging @serhiy-storchaka, @berkerpeksag, @methane who helped review #201.

Closes http://bugs.python.org/issue29636.

https://bugs.python.org/issue29636

dhimmel · 2017-02-27T15:47:42Z

Note to rebase on #346 which should be merged first.

vstinner · 2017-03-02T17:53:22Z

Lib/json/tool.py

+    group.add_argument('--indent', default=4, type=int,
+                       help='Indent level (number of spaces) for '
+                       'pretty-printing. Defaults to 4.')
+    group.add_argument('--no-indent', action='store_const', dest='indent',


I would prefer a --compact option which use indent=None but also separators without spaces: http://bugs.python.org/issue29540

It's not the same than what you propose.

@Haypo I agree the most compact representation would be useful.

One option would be add a third argument (--compact) to the mutually exclusive group. This would provide access to three options:

--indent: pretty (with newlines)

--no-indent: with indent=None in json.dump

--compact: compact, with separators=(',', ':') in json.dump

Alternatively, are you proposing to only support options 1 & 3?

I'm happy to implement this. Let's get a second opinion, so I don't go down the wrong track. Tagging @serhiy-storchaka, @berkerpeksag, @methane.

Is --no-indent differen than --indent=0?

Would it be possible to use a default of 4 spaces when using --indent with no argument?

Is --no-indent differen than --indent=0?

Yes, it is. --indent=0 adds new lines, --no-indent writes all in one line.

Would it be possible to use a default of 4 spaces when using --indent with no argument?

This is the default behavior.

@Haypo the following illustrates the three options

import json obj = [1, 2] print('--indent=4') print(json.dumps(obj, indent=4)) print('--no-indent') print(json.dumps(obj, indent=None)) print('--compact') print(json.dumps(obj, separators=(',', ':')))

which produces the following output

--indent=4 [ 1, 2 ] --no-indent [1, 2] --compact [1,2]

Spaces after comma and semicolon are not just for readability. They make JSON compatible with YAML 1.0.

@serhiy-storchaka great point. I think this is a strong argument for not replacing --no-indent with --compact. However, should we add --compact as a third option? I can see the utility for users who wan't the smallest file sizes.

serhiy-storchaka · 2017-03-03T08:25:55Z

Spaces after comma and semicolon are not just for readability. They make JSON compatible with YAML 1.0.

vstinner · 2017-03-03T11:44:28Z

Yes, it is. --indent=0 adds new lines, --no-indent writes all in one line.

"write all in one line": it sounds like the use case is to produce smaller file, so it seems redundant and less efficient than --compact. I expect that --compact removes all spaces and newlines to produce the smallest file.

serhiy-storchaka · 2017-03-03T17:33:43Z

This is not the right place for the design discussion. Lets continue it on the bug tracker.

dhimmel · 2017-03-03T19:26:58Z

This is not the right place for the design discussion. Lets continue it on the bug tracker.

@serhiy-storchaka and @Haypo I updated bpo-29636 with a summary of the design discussion thus far.

I propose continuing this pull request with only --indent and --no-indent. If we end up reaching consensus to add --compact, that can be added in the future. I think it would fit in nicely alongside --indent and --no-indent. In other words, it's not pressing to decide on --compact right now.

dhimmel · 2017-03-16T12:51:14Z

#346 has been merged and this PR rebased.

@serhiy-storchaka, @berkerpeksag, @methane and @Haypo: open for review. No more outstanding changes on my end.

Originall from python#2720 Set default option for infile and outfile as per python#2720 (review) Use --sort-keys help message based on the json.tool docs at https://docs.python.org/3.6/library/json.html#basic-usage: Remove commens as per https://bugs.python.org/msg298692. Code was descriptive without the comments.

From python#201 which is being split into two pull requests. See http://bugs.python.org/issue29636

methane · 2018-02-22T10:21:27Z

@dhimmel Please add NEWS entry.

dhimmel · 2018-02-22T15:56:05Z

Thanks @methane for ba16891 which merges in the changes from #5315 (dict order guaranteed as of Python 3.7).

I'll get working on the NEWS entry.

Use `blurb add` command to assist with adding the NEWS entry.

dhimmel · 2018-02-22T16:55:33Z

Ready from my end.

As of b8fee34, --help prints the following message:

usage: python -m json.tool [-h] [--sort-keys]
                           [--indent INDENT | --tab | --no-indent | --compact]
                           [infile] [outfile]

A simple command line interface for json module to validate and pretty-print
JSON objects.

positional arguments:
  infile           a JSON file to be validated or pretty-printed
  outfile          write the output of infile to outfile

optional arguments:
  -h, --help       show this help message and exit
  --sort-keys      sort the output of dictionaries by key
  --indent INDENT  separate items with newlines and use this number of spaces
                   for indentation
  --tab            separate items with newlines and use tabs for indentation
  --no-indent      separate items with spaces rather than newlines
  --compact        suppress all whitespace separation (most compact)

serhiy-storchaka · 2018-02-22T17:06:31Z

What if pass conflicting options? --indent=4 and --tab? --indent=4 and --compact?

serhiy-storchaka · 2018-02-22T16:59:30Z

Lib/json/tool.py

@@ -21,23 +21,42 @@ def main():
                   'to validate and pretty-print JSON objects.')
    parser = argparse.ArgumentParser(prog=prog, description=description)
    parser.add_argument('infile', nargs='?', type=argparse.FileType(),
+                        default=sys.stdin,


What is the benefit of setting default to sys.stdin instead of keeping it None?

This way a suggestion from another contributor at #2720 (comment):

it is easier to read when we process all options in one place, and then use the process options.

By adding default=sys.stdin, we remove the following line later on:

infile = options.infile or sys.stdin

serhiy-storchaka · 2018-02-22T17:03:30Z

Lib/json/tool.py

-    parser.add_argument('--sort-keys', action='store_true', default=False,
-                        help='sort the output of dictionaries alphabetically by key')
+    parser.add_argument('--sort-keys', action='store_true',
+                        help='sort the output of dictionaries by key')


Why removed "alphabetically"?

I believe my intention was to make the help message more closely mirror the json.dump documentation:

cpython/Lib/json/__init__.py

Lines 157 to 158 in cfa797c

If *sort_keys* is true (default: ``False``), then the output of

dictionaries will be sorted by key.

Happy to re-add "alphabetically" if that was helpful.

dhimmel · 2018-02-22T18:53:13Z

What if pass conflicting options? --indent=4 and --tab? --indent=4 and --compact?

The intention is to disallow users from passing multiple whitespace options with argparse's add_mutually_exclusive_group. The implementation will, in most cases, detect and raise an error if multiple whitespace arguments are provided. However, there is an argparse issue (bpo-18943) where mutual exclusivity is not enforced when --indent=4 is specified (since 4 is default).

For example, --compact --tab or --indent=3 --compact or --indent=4 --compact --tab will all throw errors of the form:

python -m json.tool: error: argument --tab: not allowed with argument --compact

--indent=4 --compact (or --indent=4 --tab) does not throw an error. In the case that --indent=4 is specified with --compact, --tab, or no-indent, the later will take precedent.

mya12321 · 2018-04-08T09:26:38Z

I hope this feature can be merged ASAP, it's really helpful as a cross-platform command line utility.

dhimmel · 2018-04-08T23:22:15Z

I hope this feature can be merged ASAP

@mya12321 the most effective way for you to support this PR would be to chime in on the issue. At present, I think there is some hesitancy to add additional arguments, so if you explain your use case that may be helpful. But as far as I understand, this discussion should go on the bug tracker and not on GitHub.

methane · 2019-12-01T03:00:12Z

Would you resolve the conflict?

vstinner · 2019-12-04T11:41:49Z

Well done @dhimmel and thanks for your tenacity!

Bumps [gidgethub](https://github.com/brettcannon/gidgethub) from 4.1.0 to 4.1.1. - [Release notes](https://github.com/brettcannon/gidgethub/releases) - [Changelog](https://github.com/brettcannon/gidgethub/blob/master/docs/changelog.rst) - [Commits](gidgethub/gidgethub@v4.1.0...v4.1.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

the-knights-who-say-ni added the CLA signed label Feb 27, 2017

vstinner reviewed Mar 2, 2017

View reviewed changes

vstinner added the type-feature A feature request or enhancement label Mar 2, 2017

serhiy-storchaka mentioned this pull request Mar 3, 2017

Allow setting indent width or character in json.tool #426

Closed

dhimmel mentioned this pull request Mar 10, 2017

bpo-27413: add --no-ensure-ascii argument to json.tool #201

Closed

dhimmel force-pushed the indent branch from 011ae48 to 562f2ef Compare March 16, 2017 12:39

dhimmel force-pushed the indent branch 4 times, most recently from b53a918 to 56a3851 Compare April 13, 2017 18:58

modulitos mentioned this pull request Jun 15, 2017

bpo-30669: add --indent flag to json.tool #2207

Closed

dhimmel mentioned this pull request Jul 15, 2017

bpo-30971: Improve code readability of json.tool #2720

Closed

dhimmel added 4 commits July 20, 2017 11:40

Add --indent / --no-indent arguments to json.tool

3c67514

From python#201 which is being split into two pull requests. See http://bugs.python.org/issue29636

json.tool: --tab identation option

70cc07b

json.tool: --compact identation option

a098b5d

dhimmel force-pushed the indent branch from 56a3851 to a098b5d Compare July 20, 2017 15:51

brettcannon added the awaiting review label Feb 21, 2018

Merge branch 'master' into indent

ba16891

dhimmel added 2 commits February 22, 2018 11:24

Add NEWS entry

3d3477d

Use `blurb add` command to assist with adding the NEWS entry.

Update whitespace option documentation

b8fee34

serhiy-storchaka reviewed Feb 22, 2018

View reviewed changes

wimglenn mentioned this pull request Oct 9, 2018

bpo-29636: improve CLI of json.tool #9765

Closed

Merge master & resolve --json-lines conflicts

292fc1c

methane merged commit 0325794 into python:master Dec 4, 2019

bedevere-bot removed the awaiting review label Dec 4, 2019

jacobneiltaylor pushed a commit to jacobneiltaylor/cpython that referenced this pull request Dec 5, 2019

bpo-29636: Add --(no-)indent arguments to json.tool (pythonGH-345)

3a6bd64

dhimmel mentioned this pull request Dec 6, 2019

bpo-29636: json.tool: Add document for indentation options. #17482

Merged

shihai1991 pushed a commit to shihai1991/cpython that referenced this pull request Jan 31, 2020

bpo-29636: Add --(no-)indent arguments to json.tool (pythonGH-345)

4dd5be6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-29636: Add --indent / --no-indent arguments to json.tool #345

bpo-29636: Add --indent / --no-indent arguments to json.tool #345

dhimmel commented Feb 27, 2017 •

edited by bedevere-bot

dhimmel commented Feb 27, 2017

vstinner Mar 2, 2017

dhimmel Mar 3, 2017 •

edited

vstinner Mar 3, 2017

serhiy-storchaka Mar 3, 2017

dhimmel Mar 3, 2017

dhimmel Mar 3, 2017

serhiy-storchaka commented Mar 3, 2017

vstinner commented Mar 3, 2017 via email

serhiy-storchaka commented Mar 3, 2017

dhimmel commented Mar 3, 2017

dhimmel commented Mar 16, 2017

methane commented Feb 22, 2018

dhimmel commented Feb 22, 2018

dhimmel commented Feb 22, 2018 •

edited

serhiy-storchaka commented Feb 22, 2018

serhiy-storchaka Feb 22, 2018

dhimmel Feb 22, 2018

serhiy-storchaka Feb 22, 2018

dhimmel Feb 22, 2018

dhimmel commented Feb 22, 2018

mya12321 commented Apr 8, 2018

dhimmel commented Apr 8, 2018

methane commented Dec 1, 2019

vstinner commented Dec 4, 2019

	If sort_keys is true (default: ``False``), then the output of
	dictionaries will be sorted by key.

bpo-29636: Add --indent / --no-indent arguments to json.tool #345

bpo-29636: Add --indent / --no-indent arguments to json.tool #345

Conversation

dhimmel commented Feb 27, 2017 • edited by bedevere-bot

dhimmel commented Feb 27, 2017

Choose a reason for hiding this comment

dhimmel Mar 3, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

serhiy-storchaka commented Mar 3, 2017

vstinner commented Mar 3, 2017 via email

serhiy-storchaka commented Mar 3, 2017

dhimmel commented Mar 3, 2017

dhimmel commented Mar 16, 2017

methane commented Feb 22, 2018

dhimmel commented Feb 22, 2018

dhimmel commented Feb 22, 2018 • edited

serhiy-storchaka commented Feb 22, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhimmel commented Feb 22, 2018

mya12321 commented Apr 8, 2018

dhimmel commented Apr 8, 2018

methane commented Dec 1, 2019

vstinner commented Dec 4, 2019

dhimmel commented Feb 27, 2017 •

edited by bedevere-bot

dhimmel Mar 3, 2017 •

edited

dhimmel commented Feb 22, 2018 •

edited