Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openmoji-black,openmoji-color: init at 12.0.0 #70177

Merged
merged 10 commits into from Aug 13, 2021

Conversation

fgaz
Copy link
Member

@fgaz fgaz commented Oct 1, 2019

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Copy link
Member

@dtzWill dtzWill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bash reimplementation needs some attention, resulting files don't work for me but that may be an issue locally. See my comment for details.

pkgs/data/fonts/openmoji/default.nix Outdated Show resolved Hide resolved
@fgaz fgaz requested a review from dtzWill October 22, 2019 14:49
@fgaz
Copy link
Member Author

fgaz commented Dec 7, 2019

Ping 1F60A

@fgaz
Copy link
Member Author

fgaz commented Jan 7, 2020

Updated to 12.1.0

@stale
Copy link

stale bot commented Jul 5, 2020

Hello, I'm a bot and I thank you in the name of the community for your contributions.

Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do:

If you received an approval by an unpriviledged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using Git blame on the relevant files, or via GitHub's web interface. You can see if someone's a member of the nixpkgs-committers team, by hovering with the mouse over their username on the web interface, or by searching them directly on the list.

If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use Git blame on the relevant files, or GitHub's web interface to find someone who touched the relevant files in the past.

If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always officially request them for a review, or just @ mention them and say you've addressed their comments.

Lastly, you can always ask for help at our Discourse Forum, or more specifically, at this thread or at #nixos' IRC channel.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 5, 2020
@fgaz
Copy link
Member Author

fgaz commented Jul 6, 2020

Not stale, stale-bot. Unfortunately though 12.3.0 segfaults scfbuild, so I cannot update this pr to that version.

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 6, 2020
@fgaz
Copy link
Member Author

fgaz commented Jul 29, 2020

Ugggh, 12.4.0 is segfaulting scfbuild too, maybe I should just fetch the prebuilt font

@OPNA2608
Copy link
Contributor

OPNA2608 commented Sep 15, 2020

I would love to finally have these fonts in stable, can you check the state of this again please? I checked out this PR locally, bumped scfbuild to 2.0.0 (13rac1/scfbuild@6d84339), openmoji to 12.4.0 & dropped the patch in openmoji's postPatch phase and FWIW both the black and color builds succeed.

@fgaz
Copy link
Member Author

fgaz commented Sep 16, 2020

I just tried but it doesn't seem to build for me. Maybe I missed something though. Let's see what ofborg says

@OPNA2608
Copy link
Contributor

OPNA2608 commented Sep 22, 2020

I bumped those versions without rebasing before, but now I see the crash you were mentioning. I've debugged this down to a problem in fontforge:

malloc_consolidate(): invalid chunk size

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6af63c0 in SplineSetStroke () from /nix/store/1k5kvnxjfrvpqgwbadmisjpygnbbamfh-fontforge-20200314/lib/libfontforge.so.4

Upstream issue fontforge/fontforge#4315 seems related, should we chime in there with this "real world" example of that crash?

In the meantime, we could either try downgrading fontforge for this font's scfbuild or admit defeat and use the prebuilt TTFs.

];

buildPhase = ''
# Bash reimplementation of helpers/export-svg-font.js
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upstream no longer has helpers/export-svg-font.js https://github.com/hfg-gmuend/openmoji/tree/12.4.0/helpers. Also, if possible, I recommend patching source instead of copying-pasting script contents. That is less brittle.

It looks like we are now supposed to run npm run generate-font: https://github.com/hfg-gmuend/openmoji/tree/12.4.0/helpers#big-helpers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't copypaste, I reimplemented it entirely. The reason why I did that instead of using their scripts is that then we'd have to first package the node stuff too (and I see there's some recursive node-bash-docker calls :-/ ), which is a pain.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upstream no longer has helpers/export-svg-font.js

It was moved to helpers/generate-font-glyphs.js.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fgaz I believe we can install node dependencies with node2nix and patch out the docker calls, either with sed or patches.

Copy link
Contributor

@justinlovinger justinlovinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the build should be refactored to use the upstream npm run generate-font. It looks like we are currently building in an unsupported way. That is probably why the build fails for us, but presumably succeeds upstream.

{ stdenv
, fetchFromGitHub
, scfbuild
, python3Packages
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This argument is not used (python3Packages).

@fgaz
Copy link
Member Author

fgaz commented Oct 13, 2020

admit defeat and use the prebuilt TTFs

🏳️ 🏳️ 🏳️
I'll switch this to the prebuilts soon

@OPNA2608
Copy link
Contributor

That is probably why the build fails for us, but presumably succeeds upstream.

Afaict upstream builds with a Docker image based on Ubuntu 18.04 (fontforge 2017something) while we have the latest fontforge release packaged instead (fontforge 202003something). With an older nixpkgs revision I was able to build this package without problems, since ~ the bump of fontforge it started crashing as I coarsely bisected earlier.

I wasn't sure where exactly to report this back then and was abit stressed with tasks. It's obviously a crash within fontforge as noted earlier, so I'll visit the linked issue later and add our crash example, assuming this is the same issue. Though their report implies that it's due to bad input and will cause visual problems even if it's fixed, so informing the openmoji team of this problem would prolly be a good idea too.

Given that the input is strange and could/should have been a closed contour, I think the fix can focus on avoiding the crash and need not return an ideal result.

I have a hunch they'll tell us to use their handy docker file or prebuilt font files though 😛.

@justinlovinger
Copy link
Contributor

@fgaz it looks like both Noto and Twemoji use prebuilt TTFs. I guess there is precident. It would be nice to build from source, but I'm not sure what the practical benefits are.

@fgaz
Copy link
Member Author

fgaz commented Aug 11, 2021

Ugh of course it's a mass rebuild... I guess I'll split the scfbuild fontforge update into a separate pr.

I'd like to see the reimplemented script abandoned in favour of getting upstream's version working

If someone else does and documents it, I'll try to maintain it, but my preference is to stay clear of node stuff.

Thanks @OPNA2608 and @kevincox for the contributions!

@@ -30,6 +30,12 @@ stdenv.mkDerivation rec {
url = "https://salsa.debian.org/fonts-team/fontforge/raw/76bffe6ccf8ab20a0c81476a80a87ad245e2fd1c/debian/patches/0001-add-extra-cmake-install-rules.patch";
sha256 = "u3D9od2xLECNEHhZ+8dkuv9818tPkdP6y/Tvd9CADJg=";
})
# Fix segmentation fault with some fonts.
# This is merged and should be present in the next release.
(fetchpatch {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 for the patch.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I submit it separately?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is causing the mass rebuild than it would be good we can do a PR that targets staging with it and wait until it is in master.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch has been upstreamed. We can drop this commit from the PR.

@kevincox
Copy link
Contributor

I looked into the build process on their end and it does the follow steps:

  1. npm run generate-font
  2. https://github.com/hfg-gmuend/openmoji/blob/3811fc0a9518db8f53453a8e7f341b7bd29dd152/helpers/generate-fonts.sh
  3. https://github.com/hfg-gmuend/openmoji/blob/3811fc0a9518db8f53453a8e7f341b7bd29dd152/helpers/generate-font-glyphs.js

2 seems like it causes most of the problems and provides the least value.

3 Seems helpful and I think we can avoid re-implementing it.

  • Only two NPM dependencies
  • It is fairly simple but does encode a lot of the logic for how they organize files and would be nice to avoid duplicating that logic in nixpkgs.

So overall I don't mind re-implementing the script but ideally it seems like we would reuse 3. However that wouldn't prevent me from merging this PR. I would appreciate any opposing views on this.

@fgaz
Copy link
Member Author

fgaz commented Aug 11, 2021

Only two NPM dependencies

direct dependencies :)

@OPNA2608
Copy link
Contributor

Those two npm packages, once added according to https://nixos.org/manual/nixpkgs/stable/#node.js, are enough to get everything running with their JS script.

  nativeBuildInputs = [
    scfbuild
    nodejs
    nodePackages.glob
    nodePackages.lodash
  ];
  
  buildPhase = ''
    runHook preBuild
    
    node helpers/generate-font-glyphs.js
    
    cd font
    scfbuild -c scfbuild-${variant}.yml
    
    runHook postBuild
  '';

kevincox added a commit to kevincox/nixpkgs that referenced this pull request Aug 11, 2021
This segfault appears when trying to build some fonts such as openmoji. See NixOS#70177 for discussion.
erictapen pushed a commit that referenced this pull request Aug 11, 2021
This segfault appears when trying to build some fonts such as openmoji. See #70177 for discussion.
@fgaz
Copy link
Member Author

fgaz commented Aug 12, 2021

Oh nooo a conflict... it took literally multiple hours to regenerate node-packages.nix D:

@fgaz
Copy link
Member Author

fgaz commented Aug 12, 2021

Oh, and i guess we should wait for @kevincox's patch to land in master before merging this?

@kevincox
Copy link
Contributor

Alternatively I think it is fine to target staging.

@kevincox
Copy link
Contributor

@fgaz do you mind if I merge this into staging?

@fgaz
Copy link
Member Author

fgaz commented Aug 13, 2021

@kevincox ok. could you preserve the intermediate commit with the bash implementation?

@kevincox
Copy link
Contributor

I was just going to merge the commits as is. (And re-gen the node-packages)

@kevincox kevincox changed the base branch from master to staging August 13, 2021 20:38
@kevincox kevincox merged commit f5e552e into NixOS:staging Aug 13, 2021
@fgaz fgaz deleted the openmoji-color-font/init branch August 15, 2021 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants