Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0045] Deprecating unquoted URL syntax #45

Merged
merged 12 commits into from Nov 25, 2019

Conversation

7c6f434c
Copy link
Member

No description provided.

Copy link
Member

@grahamc grahamc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm in favor of this RFC.

strings or uniformly unquoted.

Decide to use unquoted URLs for all URLs without special characters or variable
expansion.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind making these two items a list?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely done in the PR branch, I have no idea why GitHub doesn't show this yet…

convert URLs to quoted strings when changing them.

Accept PRs that convert unquoted URLs to quoted strings if such PRs are
submitted.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there is a way ofborg could detect newly introduced URL values?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe your profiling patch to Nix is in the right direction for that…

@joepie91
Copy link
Contributor

Strongly in favour. It's never been clear to me why this was part of the syntax, and more than once I've been left with analysis paralysis because I wasn't sure whether there was a reason to use the special syntax or not, and there appeared to be no documentation on that.

And from a tooling implementor POV: removing this in the long run would reduce the complexity of a Nix (parser) implementation.

Copy link
Member

@peti peti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please remove the special URL syntax! That feature adds very little value (if any) and I've never needed it.

@samueldr
Copy link
Member

Quick thoughts about deprecating language/interpreter features:

As far as detection goes, deprecating features in the interpreter could be made strict with a list of "new" features, e.g. --option strict-eval no-unquoted-urls so that the interpreter gates everything related to the unquoted URLs under conditionals, making it fail. Tracing could additionally be added at the same locations to warn, hopefully with a flag to disable tracing those.

Though, this is all at the cost of adding more conditionals, thus more possibilities for weird behaviours.

Gating it in the main interpreter with an option would also allow end-users to set it globally, allowing new deprecation-free evals to be forced on their systems, hopefully helping weed out the leftover features deprecations.

Copy link
Member

@globin globin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree!

Another thing with unquoted URLs is that most terminals treat the ; as part of the url making it harder to copy/click links with the special syntax.

@globin globin changed the title A small RFC on deprecating URL syntax [RFC 0045] A small RFC on deprecating URL syntax Apr 28, 2019
@globin globin changed the title [RFC 0045] A small RFC on deprecating URL syntax [RFC 0045] Deprecating unquoted URL syntax Apr 28, 2019
@7c6f434c
Copy link
Member Author

7c6f434c commented Apr 28, 2019

Thanks for the suggestion of one more reason, let's have motivation section be the longest one!

Copy link
Member

@zimbatm zimbatm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 happy to see this happening


Add a note in the Nixpkgs manual that the unquoted URL syntax is deprecated,
changes to Nixpkgs should not increase its use, and it is recommended to
convert URLs to quoted strings when changing them.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to be more specific? How would everybody be notified about this new rule?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I think the mere fact of an RFC passing is a noticeable effect event for a large fraction of active contributors. (It is visible in this repository, announced via Discourse, gets into the «weekly», is likely to be discussed a bit on IRC etc.)

For this specific RFC just the announcement «RFC on Deprecation of URL syntax passed» conveys enough information.

  1. It will be noted in two out of three manuals.

  2. Once we have an RFC that something «should not» happen, a request for cosmetic cleanup in a PR (and such requests happen) is more likely to contain a mention of quoting URLs (if relevant for the package in question). This also spreads the knowledge.

I do have an impression that quite a few people mentioned that they are disappointed they cannot refer to a policy that quoted URLs are better, so I expect the review channel of information dissemination to perform well.

Appendix A: future work — maybe the tool gets implemented, then we can open a countdown issue and maybe make ofborg check that PRs do not make things worse).

My plan is indeed just these points. I think that these things do not really need any additions to the text of RFC, and I think these mechanisms will be enough to distribute the information.

If you think that any of these four channels benefit from an addition to RFC text, or that there are other information distribution channels that should be used (and mentioned in the RFC text) please give some details.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't have to be formalized in the RFC but it's good to have an accompanying discussion.

For example you mentioned "they are disappointed they cannot refer to a policy". Would the RFC act as such policy? If not, maybe we should have a list of official policies in place.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having a deprecation notice in the manual (added in accordance with an explicit bullet point in an approved/accepted RFC) is close enough to official policy for the purpose of asking people not to do the deprecated things.

I didn't start the discussion myself, because I thought that (1) is assumed, (2) and (A) are explicitly mentioned, and (3) is a manual variant of (A) in a sense.

rfcs/0045-deprecate-url-syntax.md Outdated Show resolved Hide resolved
rfcs/0045-deprecate-url-syntax.md Outdated Show resolved Hide resolved
start-date: 2019-04-28
author: Michael Raskin
co-authors:
related-issues:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure: the RFC template seems to imply this is for PRs implementing the RFC, not for the changes the RFC intends to undo.

FRidh and others added 2 commits May 1, 2019 06:50
Co-Authored-By: 7c6f434c <7c6f434c@mail.ru>
Co-Authored-By: 7c6f434c <7c6f434c@mail.ru>
@7c6f434c
Copy link
Member Author

7c6f434c commented May 1, 2019

@FRidh thanks for the proofreading.

@edolstra
Copy link
Member

edolstra commented May 1, 2019

We should not break backwards compatibility in such a massive way. I mean, deprecation is only useful if you eventually want to remove it, at which point it becomes impossible for Nix to build old Nix expressions.

In principle, the flakes "epoch" feature would make it possible to make such language changes in a backward compatible way (e.g. unquoted URLs could be deprecated in epoch 2020). I'm not convinced it's actually useful, though. The case of x:x falls under the "just don't do that" header IMHO.

@7c6f434c
Copy link
Member Author

7c6f434c commented May 1, 2019

There are some features that we have chosen not to use in Nixpkgs anymore and we do not recommend to use these features if they can be avoided at all. Additional tooling intended for aiding Nixpkgs development can be useful without supporting such features, and easier creation of tooling is good.

The RFC definitely does not argue for actually dropping the support unless some other breaking changes happen. It will be a pity if an epoch-guarded syntax change for something major like a change in overrides happens without a cleanup of minor annnoyances.

@domenkozar
Copy link
Member

In 2022 we can parse Nix identifier without first parsing a URI with a regex, which will speed up the parser.

@edolstra
Copy link
Member

edolstra commented May 2, 2019

@domenkozar I think URIs, identifiers etc. are all recognized using a single finite state machine, so it shouldn't matter much for performance.

@Mic92 Mic92 added the status: open for nominations Open for shepherding team nominations label May 2, 2019
@ajs124 ajs124 mentioned this pull request Feb 16, 2021
erikarvstedt added a commit to erikarvstedt/nix-bitcoin that referenced this pull request Mar 10, 2021
unquoted urls are deprecated: NixOS/rfcs#45
figsoda added a commit to figsoda/pkgs that referenced this pull request Apr 28, 2021
url literals were deprecated by rfc 45 NixOS/rfcs#45
figsoda added a commit to nix-community/fenix that referenced this pull request Apr 28, 2021
url literals were deprecated by rfc 45 NixOS/rfcs#45
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-2-4-released/15822/1

Ma27 added a commit to Ma27/nix that referenced this pull request Jun 13, 2022
Basically an attempt to resume fixing NixOS#5543 for a breakage introduced
earlier[1]. Basically, when evaluating an older `nixpkgs` with
`nix-shell` the following error occurs:

    λ ma27 [~] → nix-shell -I nixpkgs=channel:nixos-18.03 -p nix
    error: anonymous function at /nix/store/zakqwc529rb6xcj8pwixjsxscvlx9fbi-source/pkgs/top-level/default.nix:20:1 called with unexpected argument 'inNixShell'

           at /nix/store/zakqwc529rb6xcj8pwixjsxscvlx9fbi-source/pkgs/top-level/impure.nix:82:1:

               81|
               82| import ./. (builtins.removeAttrs args [ "system" "platform" ] // {
                 | ^
               83|   inherit config overlays crossSystem;

This is a problem because one of the main selling points of Nix is that
you can evaluate any old Nix expression and still get the same result
(which also means that it *still evaluates*). In fact we're deprecating,
but not removing a lot of stuff for that reason such as unquoted URLs[2]
or `builtins.toPath`. However this property was essentially thrown away
here.

The change is rather simple: check if `inNixShell` is specified in the
formals of an auto-called function. This means that

    { inNixShell ? false }:
    builtins.trace inNixShell
      (with import <nixpkgs> { }; makeShell { name = "foo"; })

will show `trace: true` while

    args@{ ... }:
    builtins.trace args.inNixShell
      (with import <nixpkgs> { }; makeShell { name = "foo"; })

will throw the following error:

    error: attribute 'inNixShell' missing

This is explicitly needed because the function in
`pkgs/top-level/impure.nix` of e.g. NixOS 18.03 has an ellipsis[3], but
passes the attribute-set on to another lambda with formals that doesn't
have an ellipsis anymore (hence the error from above). This was perhaps
a mistake, but we can't fix it anymore. This also means that there's
AFAICS no proper way to check if the attr-set that's passed to the Nix
code via `EvalState::autoCallFunction` is eventually passed to a lambda
with formals where `inNixShell` is missing.

However, this fix comes with a certain price. Essentially every
`shell.nix` that assumes `inNixShell` to be passed to the formals even
without explicitly specifying it would break with this[4]. However I think
that this is ugly, but preferable:

* Nix 2.3 was declared stable by NixOS up until recently (well, it still
  is as long as 21.11 is alive), so most people might not have even
  noticed that feature.

* We're talking about a way shorter time-span with this change being
  in the wild, so the fallout should be smaller IMHO.

[1] NixOS@9d612c3
[2] NixOS/rfcs#45 (comment)
[3] https://github.com/NixOS/nixpkgs/blob/release-18.03/pkgs/top-level/impure.nix#L75
[4] See e.g. the second expression in this commit-message or the changes
    for `tests/ca/nix-shell.sh`.
Ma27 added a commit to Ma27/nix that referenced this pull request Jun 13, 2022
Basically an attempt to resume fixing NixOS#5543 for a breakage introduced
earlier[1]. Basically, when evaluating an older `nixpkgs` with
`nix-shell` the following error occurs:

    λ ma27 [~] → nix-shell -I nixpkgs=channel:nixos-18.03 -p nix
    error: anonymous function at /nix/store/zakqwc529rb6xcj8pwixjsxscvlx9fbi-source/pkgs/top-level/default.nix:20:1 called with unexpected argument 'inNixShell'

           at /nix/store/zakqwc529rb6xcj8pwixjsxscvlx9fbi-source/pkgs/top-level/impure.nix:82:1:

               81|
               82| import ./. (builtins.removeAttrs args [ "system" "platform" ] // {
                 | ^
               83|   inherit config overlays crossSystem;

This is a problem because one of the main selling points of Nix is that
you can evaluate any old Nix expression and still get the same result
(which also means that it *still evaluates*). In fact we're deprecating,
but not removing a lot of stuff for that reason such as unquoted URLs[2]
or `builtins.toPath`. However this property was essentially thrown away
here.

The change is rather simple: check if `inNixShell` is specified in the
formals of an auto-called function. This means that

    { inNixShell ? false }:
    builtins.trace inNixShell
      (with import <nixpkgs> { }; makeShell { name = "foo"; })

will show `trace: true` while

    args@{ ... }:
    builtins.trace args.inNixShell
      (with import <nixpkgs> { }; makeShell { name = "foo"; })

will throw the following error:

    error: attribute 'inNixShell' missing

This is explicitly needed because the function in
`pkgs/top-level/impure.nix` of e.g. NixOS 18.03 has an ellipsis[3], but
passes the attribute-set on to another lambda with formals that doesn't
have an ellipsis anymore (hence the error from above). This was perhaps
a mistake, but we can't fix it anymore. This also means that there's
AFAICS no proper way to check if the attr-set that's passed to the Nix
code via `EvalState::autoCallFunction` is eventually passed to a lambda
with formals where `inNixShell` is missing.

However, this fix comes with a certain price. Essentially every
`shell.nix` that assumes `inNixShell` to be passed to the formals even
without explicitly specifying it would break with this[4]. However I think
that this is ugly, but preferable:

* Nix 2.3 was declared stable by NixOS up until recently (well, it still
  is as long as 21.11 is alive), so most people might not have even
  noticed that feature.

* We're talking about a way shorter time-span with this change being
  in the wild, so the fallout should be smaller IMHO.

[1] NixOS@9d612c3
[2] NixOS/rfcs#45 (comment)
[3] https://github.com/NixOS/nixpkgs/blob/release-18.03/pkgs/top-level/impure.nix#L75
[4] See e.g. the second expression in this commit-message or the changes
    for `tests/ca/nix-shell.sh`.
ursi added a commit to ursi/smos that referenced this pull request Feb 12, 2023
Unquoted URLs are deprecated NixOS/rfcs#45
NorfairKing pushed a commit to NorfairKing/smos that referenced this pull request Feb 13, 2023
ajs124 added a commit to helsinki-systems/nixpkgs that referenced this pull request Feb 20, 2023
otherwise, eval fails when the experimental no-url-literals feature is activated

unquoted urls are discouraged after NixOS/rfcs#45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet