I think this would be very dangerous, because the port numbers depend on the order of evaluation. How do you intend to deal with this problem? (my_identifier could be assigned the port 5000 in one evaluation and the port 5001 in the next one, because it may be evaluated before or after port "webserver1")

@bennofs your observation is correct. maybe it could be solved by using the options system: read from config (after it has been evaluated) and then translate ports into numbers using builtins.port, where builtins.port is a hidden interface?

Thoughts:

1. It doesn't feel like Nix the build system/package manager should know anything about ports, from a separation of concerns angle
2. Most times ports are invoked you'll want them both inside Nix and outside of it, because stuff outside your system is going to want to talk to things inside your system
3. This seems like it could be generalized somewhat to be pure and not specific to ports, by shoving some of the work outside the system. In many ways it seems like this wants to be akin to a "symbol", but with a numeric representation constrained somehow. It seems that by thinking more broadly, you could create a namespace of function-specific symbols that can have external behavior. Evaluation does seem like it'd get more complicated though...

@copumpkin
(1) True, that is why this is titled RFC. A more generalized version of builtins.port could be called builtins.uniqueID "identifier" for instance.
(2) Don't understand outside and inside. When doing deployment with nixops or disnix it is hard distinguish between different machines in regards to how nix, the proramming language, sees them.
(3) Not quite sure what you want to say, can you please explain me?

For users.users.uid there exists a UID-allocator written in perl. It generates additional entries in /etc/passwd. However, adapting this to ports is not possible as there is not a central place as /etc/passwd holding all the ports, but instead each daemon is configured individually.

@bennofs

builtins.port

input1

{
  a={ 
    bar=(builtins.port "aa12");
    foo=(builtins.port "aa1"); 
  };
}

[nix-shell:~/nix/inst]$ bin/nix-instantiate --eval --strict z.nix 
{ a = { bar = 50000; foo = 50001; }; }

input2

{
  a={ 
    foo=(builtins.port "aa1"); 
    bar=(builtins.port "aa12");
  };
}

[nix-shell:~/nix/inst]$ bin/nix-instantiate --eval --strict z.nix 
{ a = { bar = 50001; foo = 50000; }; }

udev

however, the same is true for udev as well:

a.nix

{...}: 

{

  config = {
  services.udev.extraRules = ''
    ATTRS{idVendor}=="054c", ATTRS{idProduct}=="0268", SUBSYSTEMS=="usb", ACTION=="add", MODE="0660", GROUP="users"
  '';

  };
}

b.nix

{...}: 

{
  config = {
  services.udev.extraRules = ''
    ATTRS{idVendor}=="054c", ATTRS{idProduct}=="0268", SUBSYSTEMS=="usb", ACTION=="add", MODE="0661", GROUP="users"
  '';

  };
}

configuration.nix

...
 imports =
    [ # Include the results of the hardware scan.
      ./a.nix
      ./b.nix
...

when doing nixos-rebuild build it builds the udev stuff:

these derivations will be built:
  /nix/store/3a51qlzvydcnkijh57aln31ha8lrnyxc-extra-udev-rules.drv
  /nix/store/0a678m0ppcznkdzsggzyynxhqwbhry6v-udev-rules.drv
  /nix/store/x2hawf0jyrfmb7niiddx68zb7aixh5rp-unit-systemd-udevd.service.drv

and when changing the imports order of a.nix and b.nix it will also rebuild it:

...
 imports =
    [ # Include the results of the hardware scan.
      ./b.nix
      ./a.nix
...

https://github.com/NixOS/nixos/blob/master/modules/services/hardware/udev.nix#L149

conclusion (so far)

the generated files of builtins.port as well as those of udev are small in size and are neither would be built by hydra (it is basically stuff which would go in /etc - simple config files). i still think something like builtins.port is a valuable function.

@shlevy what do you think about this? i wonder if builtins.port could be implemented in nixpkgs/lib but i don't see how. any pointers how to go on?

Can you say a bit more about the use case? It seems to me that dynamically allocated ports (unlike uids/gids) are not generally all that useful because you need the port number to access the service. So it would only be useful for services that are internal to the configuration (e.g. a backend server that sits behind a reverse proxy on the same machine).

As others have said, having a global counter is problematic in a purely functional language. This is not necessarily a fatal problem since Nix is a DSL for describing packages and system configurations, not an exercise in purely functional language design. However, it would be problematic if port assignments can change on every nixos-rebuild.

The typical functional approach would be to thread a counter through the module system computation (i.e. every function gets a type like Counter -> (Counter, a)). Then we could have a option type types.port that increments the counter by one. This could slow down the module system by a non-trivial amount though.

BTW, it actually would be possible to allocate ports dynamically and persistently at runtime in a way analogous to how we manage uids/gids, namely by making /etc/services dynamically generated at runtime.

@edolstra

use case

the use-case is automatic reverse proxy configuration:

    services.nixcloud.dokuwiki.dok1 = {
      enable = true;
      proxyOptions = {
        port   = 39998;
        path   = "/dokuwiki";
        domain = "nix.lt";
      };
    };
    
    services.nixcloud.filesender.fs1 = {
      enable = true;
      proxyOptions = {
        port   = 39999;
        path   = "/filesender";
        domain = "nix.lt";
      };
    };

we are currently building a webserver/webservice abstraction which configures a reverse proxy automatically. a brief introduction is this: https://github.com/nixcloud/minimal-example but it does not contain the reverse proxy at the moment. i plan to merge this code into nixpkgs as soon as possible but there are still problems to be solved before i want anyone to see it.

nix is a DSL

port numbers only change if one changes the configuration or the order of how configurations are evaluated. builtins.port is reproducible already. the main setback is that it depends on the evaluation order, similar to the module system, as shown by the udev example.

module system counter

i don't understand how one would implement that. if it is a requirement to have it that way, i will consider it. can you point out which files in nixpkgs to look at?

/etc/services

i always thoguht /etc/services is only used by nmap and other network tools to show a string instead of a port number. if i generate a nginx.conf or httpd.conf file, how would i reference the port to be from /etc/services? i don't want the port to have a fancy name in the nmap sense.

I'm very very skeptical that this is something we want. It doesn't seem to solve a particularly pressing issue and involves some completely non-functional semantics. If you can find a way to make the output solely dependent on the input, I'd be much more inclined to take a loo.

@shlevy how would you solve the problem? this issue is pressing for us! we have to support a 'dynamic' ammount of webservers.

i think i can't make the output soley dependent on the input since this a mutual dependency. i'm quite depressed now as i don't see a way forward. if there is someone who understands the problem more than me, please speak up!

maybe using unix domain sockets could be an alternative as we don't have to collapse a name into a port name and therefore don't have the problem which would be solved by builtins.port "identifier".

however, using unix domain sockets for webservers is so 'not standard' that users will have a hard time to get into it.

One option is to dynamically generate your nix expressions, rather than having your nix expression do dynamic allocation.

Perhaps you can use the new builtins.exec to hack something together?

i think i can't make the output soley dependent on the input since this a mutual dependency.

I don't understand what this means.

I'm not actually sure I understand your use case completely.

@shlevy using builtins.exec is an interesting thought!

i was wrong with builtins.port since i really need to track state. say a user has 10 webservices, he removes the one which happens to be evaluated first, then 9 webservices not only get a different port but also get restarted! that is a no-go scenario!

with using builtins.exec we could write a program, which keeps track of service names and tries to reassign the port previously used. on the other side, this makes the evaluation stateful and different systems will have different port numbers.

how to use builtins.exec btw?

nix-instantiate --option enableNativeCode true --eval --strict z.nix  

with

cat z.nix
{
  a = builtins.exec "/nix/store/5lfwsx1nkrcqp7p24qr8z4wiwfxx5idv-coreutils-8.26/bin/cat /etc/fstab";
}

didn't work ;P

@qknight the udev example does not depend on evaluation order of nix expressions, but it depends on the order of composition of modules. This is a big difference, as the evaluation order can potentially change with each nix release (or does nix have a guaranteed evaluation order?)

does nix have a guaranteed evaluation order?

No it doesn't

thanks everyone for the discussion!

Doesn't disnix have some sort of construct for this sort of thing? We've talked about it before in other contexts

The Dynamic Disnix framework (https://github.com/svanderburg/dydisnix) has a port assigner tool that can be used to automatically assign unique port numbers to services in the Disnix services model and facilitates reuse across upgrades.

The idea is basically that each service takes a metadata property that indicates whether it needs a port assignment and whether the port assignment needs to be unique to the machine or to the environment.

The port assigner tool reads the services model (internally converting it into an XML representation) and composes a Nix expression that contains the port reservations. The services expression can import this ports Nix expression to retrieve the port assignments and do whatever they want with them.

To make this stuff work, I did not extend the Nix language -- I just simply read an XML representation of the Disnix service model and I compose a Nix expression from that.

One of the limitations of the port assigner tool is that it has been made for Disnix specifically -- it knows how to parse Disnix service models, but it cannot be used, for example, for NixOS configuration expressions.

Some time ago, I wrote a blog about the port assignment problem and the solution I implemented:

http://sandervanderburg.blogspot.com/2015/07/assigning-port-numbers-to-microservices.html

RFC: builtins.uniqueID

uniqueID, a builtin to statefully manage ID mappings.

the current implementation draft is here:

nixcloud@957f6cf

concept

in NixOS we have two domains where unique IDs (integers) are required:

• UID/GID
• port management for micro services (as sander calls them)

both of these domains have a 'dynamic' property which is hard to modulate in nix.

UID/GID scenario

if you create a systemd service with a new user/group (apache-a, apache-a) then NixOS will allocate a new ID () using an activation script:
https://github.com/NixOS/nixpkgs/blob/release-17.03/nixos/modules/config/users-groups.nix#L524

/etc/passwd: apache-a:x:1002:498::/var/empty:/run/current-system/sw/bin/nologin
/etc/groups: apache-a:x:498:

if one later removes that user/group (apache-a, apache-a) and adds a (foo-a, foo-a) user/group NixOS, via the activation script, will reuse the integer representation:

/etc/passwd: foo-a:x:1002:498::/var/empty:/run/current-system/sw/bin/nologin
/etc/groups: foo-a:x:498:

a side-effect is that one has files with user/group IDs which don't belong to anyone for some time:

drwxr-xr-x  4                   1006                    494 4,0K 31. Mär 17:31 trac-trac1
drwxr-x---  4                   1004                    496 4,0K 31. Mär 17:31 phpmyadmin-admin1
drwxr-x---  4                   1005                    495 4,0K 31. Mär 17:31 phppgadmin-admin2
drwxr-x---  4 apache-a               apache-a               4,0K 31. Mär 17:33 mediawiki-test1
drwxr-x---  5 nixcloud-reverse-proxy nixcloud-reverse-proxy 4,0K 31. Mär 19:53 apache-myapache
drwxr-x---  4                   1004                    496 4,0K 31. Mär 19:53 lighttpd-l
drwxr-x---  4                   1005                    495 4,0K 31. Mär 19:53 nginx-n
drwxr-x---  7 nixcloud-reverse-proxy nixcloud-reverse-proxy 4,0K  4. Apr 10:38 nextcloud-nc
drwxr-x---  5 apache-a               apache-a               4,0K  4. Apr 18:22 apache-a

but after that user/group ID is reused, the files from a service previously belonging to some other user/group (service) are now incorrectly owned by the newly added service shown for folder mediawiki-test1 above.

port management

at nixcloud we are heading towards modular micro service management. for that we have a reverse-proxy and a webserver per webservice. each webservice has a ${uniqueName} which also represents the user/group and each webservice requires a port which is shared between reverse proxy and webservice (50000-51000). see https://github.com/nixcloud/minimal-example as an example.

/* IDPool: for each string a unique integer numbers is returned
* 
*  motivation:
*    say you want to connect two inet services, which are using port numbers. this abstraction handles the ID number management.
*    one can easily write a library function: `lib.getUniquePort "identifier"` and it will return a integer `50000`
* 
*  example:
*    given nix code:                                               after evaluation
*      port1 = ${lib.getUniquePort "myNginxInstance"}                port1 = 50000
*      port2 = ${lib.getUniquePort "myNginxInstance"}                port2 = 50000
*      port3 = ${lib.getUniquePort "bar"}                            port3 = 50001
* 
*  http://rapidjson.org/classrapidjson_1_1_generic_value.html#ad290f179591025e871bedbbac89ac276
*/

problem

one problem, as also pointed out by sander in https://sandervanderburg.blogspot.de/2015/07/assigning-port-numbers-to-microservices.html, is that if one does not map the previous port configuration to the new configuration a worst-case scenario would be that all webservers would have to be restarted because all the port number had been changed.

but lets see a usage example:

usage example

in the framework, we currently work on at nixcloud, we can instantiate a webservice multiple times using services.nixcloud.nginx.<name?> and it looks like this:

services.nixcloud.nginx.n = rec {
  enable = true;
    # user = "joachim";
    # group = "users";
  proxyOptions = {
    port   = 33333; # want to remove this
    #port   = builtins.port "${uniqueName}"; # want to use this
    path   = "/foo";
    domain = "nix.n";
  };
};

currently the user has to do the port management manually, which is a burden. a nice helper is that we check all proxyOption records for port collisions.

nixos-rebuild switch
building Nix...
building the system configuration...
error: evaluation aborted with the following error message: ‘port '33333' is not unique in 'nix.a'’
(use ‘--show-trace’ to show detailed location information)

automated port resolving

z.nix

let
  lib = {
    # emulate a lib implementation
    port = builtins.uniqueID;
  };
in
{
  a = { 
    bar=(lib.port "aa12");
    foo=(lib.port "aa14"); 
    foo1=(lib.port "aa14"); 
    foo12=(lib.port "aa141");
  };
}

nix-instantiate --eval --strict z.nix

{ a = { bar = 50000; foo = 50003; foo1 = 50003; foo12 = 50005; }; }

open questions

during development, the port history was stored in:

/tmp/port-history.json

but for practical use it should go somewhere else. i'd like to be able to configure the uid/gid/port history storage location.

thus, i propose to use either of these:

• /etc/nixos/idPool/
• /var/something?
• /nix/var/nix/idPool

if the evaluation had the pointer to the previous OS build: /run/current-system/ we could also leverage this with that concept: write the idPool mappings with active/inactive mappings into a file and import it from there.

what do you think?

unix domain sockets

for a while i tried to interconnect the webservices to the reverse-proxy using 'unix domain sockets' which sounds great since it avoids ports but apache does not have support for this.
nginx seems to support it, but i didn't get it running. there are lots of other webserver implementations as nodejs, go, python and so on which probably won't work with
'unix domain sockets' out of the box either.

on the other hand: if one uses nixops for multiple machine deployments one can't use unix domain sockets.

using builtins.exec

we've been playing with builtins.exec but we can't realize all requirements with this feature. the idPool only sees incoming requests (similar to builtins.port "myident") but never gets the whole picture. except when the evaluation finished and the ~idPool() destructor is called. i copied this from Qt where i've first seen this in QMutex. only at program termination we can decided which "string" -> int mappings have been used/reused. we use the destructor as a kind of session handler. i don't see how to implement such a 'session' thing for builtins.exec.

note: with the current implementation i don't differentiate between active bindings / inactive bindings. to come...

updates

eelco mentioned:

BTW, it actually would be possible to allocate ports dynamically and persistently at runtime in a way analogous to how we manage uids/gids, namely by making /etc/services dynamically generated at runtime.

and aszlig convinced me we should alter the UID/GID activation script and create a PORTs activation script which would generate a custom /etc/services. this sounds so much better than to maintain state within nix!

@edolstra
i didn't understand at first what you meant by generating /etc/services

thanks everyone, closing as this will get a PR for nixpkgs most likely ;-)

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/detect-port-conflicts-in-nixos-services/61589/9