Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iso-image: normalize volumeID #83551

Merged
merged 3 commits into from Apr 9, 2020
Merged

Conversation

worldofpeace
Copy link
Contributor

@worldofpeace worldofpeace commented Mar 28, 2020

Motivation for this change

The volumeID of nixos images will now be in the format of:

nixos-$EDITION-$RELEASE-$ARCH
nixos-minimal-20.09-x86-64-linux

We're trying to get nixos into osinfo-db, see https://gitlab.com/libosinfo/osinfo-db/-/merge_requests/107 and https://gitlab.com/libosinfo/osinfo-db/-/merge_requests/107#note_313094852 as motivation.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Copy link
Member

@mmilata mmilata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good:+1:

Are we be able to distinguish unstable from release? If I understand correctly, current unstable has version == 20.09.

Would be nice if someone (@wkennington?) checked whether #2838 is still a problem. Looking at the other ISOs in osinfo-db it probably shouldn't be.

@worldofpeace
Copy link
Contributor Author

Would be nice if someone (@wkennington?) checked whether #2838 is still a problem. Looking at the other ISOs in osinfo-db it probably shouldn't be.

I think the issue with windows will be fixed with using the joliet extension.

The volumeID will now be in the format of:
nixos-$EDITON-$RELEASE-$ARCH

an example for the minimal image would look like:
nixos-minimal-20.09-x86-64-linux
@mmilata
Copy link
Member

mmilata commented Mar 28, 2020

Installation in virt-manager working fine:)

@worldofpeace
Copy link
Contributor Author

Installation in virt-manager working fine:)

I made a vm in gnome-boxes. I'll see if I can install nixos to a usb and windows and start it eventually, since there was the issue in #2838.

@jtojnar
Copy link
Contributor

jtojnar commented Mar 30, 2020

Looking at the wiki page Volume Identifier is strD type, which only supports A-Z, 0-9 and _ characters.

@worldofpeace
Copy link
Contributor Author

worldofpeace commented Mar 31, 2020

Looking at the wiki page Volume Identifier is strD type, which only supports A-Z, 0-9 and _ characters.

I guess a regex would work for this. Does it not support the dash?
Edit: it must if I was able to boot and build the iso.

@jtojnar
Copy link
Contributor

jtojnar commented Mar 31, 2020

No it does not: https://wiki.osdev.org/ISO_9660#String_format

Maybe implementations follow Postel’s law but if the wiki is correct, we can only use [A-Z0-9_]{1,32}.

@worldofpeace
Copy link
Contributor Author

Perhaps https://wiki.osdev.org/ISO_9660#Rock_Ridge_and_Joliet has something to do with it. I see there's various improvements and extensions. For that matter, I see fedora has volume id's with dashes https://gitlab.com/libosinfo/osinfo-db/-/blob/master/data/os/fedoraproject.org/fedora-31.xml.in#L36. So perhaps there's something we're missing here.

@worldofpeace
Copy link
Contributor Author

It seems ubuntu even has spaces https://gitlab.com/libosinfo/osinfo-db/-/blob/master/data/os/ubuntu.com/ubuntu-19.04.xml.in#L37 so maybe that documentation isn't an account of improvements

@jtojnar
Copy link
Contributor

jtojnar commented Mar 31, 2020

Briefly looking at the ECMA document, it looks like the Joliet format only adds enhanced volume descriptor, which mainly loosens the file and directory identifiers but not the volume identifier.

@mmilata
Copy link
Member

mmilata commented Mar 31, 2020

Not sure if this part of the standard is mostly ignored or Joliet/Rockridge relaxes the constraint but it seems minority of the ISOs found in osinfo-db conform:

$ ack -ch "<volume-id>.*</volume-id>"
2721
ack -ch "<volume-id>[A-Z0-9_]*</volume-id>"
188

(Ok, the content of the tag can be regex, but still only a minority are actual strD.)

Also from man xorriso:

   -volid text
         Specify the volume ID, which most operating systems will consider to be the volume name of the image or medium.
         xorriso accepts any text up to 32 characters, but according to rarely obeyed specs stricter rules apply:
         ECMA-119 demands ASCII characters out of [A-Z0-9_]. Like:
           "IMAGE_23"
         Joliet allows 16 UCS-2 characters. Like:
           "Windows name"
         Be aware that the volume id might get used automatically as the name of the mount point when the medium is inserted into a playful computer system.

@jtojnar
Copy link
Contributor

jtojnar commented Mar 31, 2020

Actually, I was reading the ECMA doc wrong. The Volume Identifier described in 8.4.6 is only relevant for Primary Volume Descriptor. Joliet’s Enhanced Volume Descriptor has its Volume Identifier described in section 8.5.5, which specifies it as d1-characters:

7.4.2 c-characters

The characters of the coded graphic character sets identified by the escape sequences in a Supplementary Volume Descriptor are referred to as c-characters.

7.4.2.1 a1-characters

A subset of the c-characters will be referred to as a1-characters. This subset shall be subject to agreement between the originator and the recipient of the volume.

7.4.2.2 d1-characters

A subset of the a1-characters will be referred to as d1-characters. This subset shall be subject to agreement between the originator and the recipient of the volume.

So if I am reading it right, we are indeed not limited by character range in Joliet format, only by the 32 byte size of the field.

@worldofpeace
Copy link
Contributor Author

worldofpeace commented Mar 31, 2020

@jtojnar Great, thank you for the research. This kind of PR is one that could go wrong in the worst ways without it.

@worldofpeace worldofpeace merged commit 8583d99 into NixOS:master Apr 9, 2020
@worldofpeace worldofpeace deleted the iso-id branch April 9, 2020 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants