Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chromium: Add ungoogled flag to enable ungoogled-chromium patches #51195

Closed
wants to merge 1 commit into from
Closed

chromium: Add ungoogled flag to enable ungoogled-chromium patches #51195

wants to merge 1 commit into from

Conversation

rht
Copy link
Member

@rht rht commented Nov 28, 2018

Motivation for this change

Apply patches from https://github.com/Eloston/ungoogled-chromium.
(Only one of the patches is applied for now for testing purpose)

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@rht
Copy link
Member Author

rht commented Nov 29, 2018

Is there a succinct way to fetch a list of patch urls? There are ~167 patches.

@worldofpeace
Copy link
Contributor

Is there a succinct way to fetch a list of patch urls? There are ~167 patches.

You could maybe adapt
https://github.com/NixOS/nixpkgs/blob/8a2bf1319ddc50a1234d48c6040ed2d85c7ba934/pkgs/tools/misc/grub/grub1.patches.sh
https://github.com/NixOS/nixpkgs/blob/8a2bf1319ddc50a1234d48c6040ed2d85c7ba934/pkgs/tools/misc/grub/grub1.patches.nix

++ map fetchurl (import ./grub1.patches.nix)

Or an inspired idea. There could be better examples in nixpkgs btw.

@Ekleog
Copy link
Member

Ekleog commented Nov 29, 2018

It's also possible to fetch the patch directory at once, and override patchPhase to use all the patches in the directory… if the default patchPhase doesn't already recurse in directories when a directory is passed in patches.

@rht
Copy link
Member Author

rht commented Nov 29, 2018

I grepped with patches = fetch and found that some of them are in the form of fetchurl, e.g.

patches = fetchurl {
. The extracted tarball of https://cdimage.parrotsec.org/mirrors/parrot/pool/main/x/xloadimage/xloadimage_4.1-25.debian.tar.xz doesn't contain a flat list of *.patch files, rather, they are located in a debian/patches/ dir. So I suppose patches does automatically traverse through the subdirs.

Though in this case, some of the patches are specific to ubuntu/debian/opensuse (https://github.com/Eloston/ungoogled-chromium/tree/master/patches), where not all the patches should be applied.

@@ -16120,6 +16120,8 @@ with pkgs;

chromiumDev = lowPrio (chromium.override { channel = "dev"; });

chromiumUngoogled = lowPrio (chromium.override { ungoogled = true; });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should let hydra build yet another chromium. The build takes quiet long.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I haven't tested building it though, which will take a very long time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should before we merge a feature like this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you intend to keep this up-to-date?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if I add chromiumUnstable back to all-packages.nix, let Borg build it once (or if Borg could build with an override), then remove it. Since the patches are fetched instead of vendored/hardcoded, to keep them up-to-date I just have to update the git rev and sha256.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ofborg has a time limit of one hour. Chromium takes 10 hours or more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I will bite the bullet and compile chromium + ungoogled.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option to prevent massive 10 hour build is to take the binary from https://github.com/Eloston/ungoogled-chromium#software-repositories (AUR) and do patchELF.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might be an option.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option to prevent massive 10 hour build is to take the binary from https://github.com/Eloston/ungoogled-chromium#software-repositories (AUR) and do patchELF.

I feel like I saw @srhb discuss this somewhere (apologies in advance if not)

) ++ optional ungoogled
(fetchgit {
url = "https://github.com/Eloston/ungoogled-chromium.git";
rev = "46bbd9f6686dd1ee2fa67ac94ddd618d0ae56d3b";
Copy link
Member

@Mic92 Mic92 Nov 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to use release tags from here? https://github.com/Eloston/ungoogled-chromium/releases

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@matthewbauer
Copy link
Member

IMO this requires too much maintenance in the long term. Chromium updates a lot and each option has to be rebuilt with new updates.

This also should not be too hard to do with an overlay:

self: super: {
  ungoogledChromium = super.chromium.overrideAttrs (old: {
    patches = old.patches ++ [(fetchgit {
                url    = "https://github.com/Eloston/ungoogled-chromium.git";
                rev    = "70.0.3538.110-2";
                sha256 = "1mj7b2wm87s5vv0d4s00570drvfz8yf2hdbgqhrh5hfdjbp7sfll";
                postFetch = ''
                  rm -r $out/patches/debian_buster \
                        $out/patches/debian_stretch \
                        $out/patches/opensuse \
                        $out/patches/ubuntu
                  # sort according to https://github.com/Eloston/ungoogled-chromium/blob/master/config_bundles/common/patch_order.list
                  mv $out/patches/inox-patchset $out/patches/1inox-patchset
                  mv $out/patches/iridium-browser $out/patches/2iridium-browser
                  mv $out/patches/ungoogled-chromium $out/patches/3ungoogled-chromium
                  mv $out/patches/bromite $out/patches/4bromite
                '';
              })];
  });
}

@rht
Copy link
Member Author

rht commented Nov 29, 2018

(I have a WIP binary version which should require far less maintenance, will push once tested)

@rht
Copy link
Member Author

rht commented Nov 30, 2018

Superseded by #51266 (maybe I will put the source version on NUR later)

@rht rht closed this Nov 30, 2018
@bendlas
Copy link
Contributor

bendlas commented Dec 9, 2018

I'd like to resume work on this approach. It seems to me, that ungoogled-chromium is pretty maintainable (and actively maintained, e.g. we probably could already build current stable with this PR ungoogled-software/ungoogled-chromium#618)

Long term, I'd even consider building Nix' chromium with the ungoogled patch set by default, because I imagine that most people running chromium instead of chrome, do so, to get rid of google's supervision.

Hopefully the maintenance burden could be reduced to a minimum by integrating ungoogled-chromium into the auto-updater script pkgs/applications/networking/browsers/chromium/update.sh:

Additionally, config_bundles from ungoogled should be reused: https://github.com/Eloston/ungoogled-chromium/blob/3c33f1c35170e21a2adb5b5fb48620f4fccc017b/packaging/archlinux/PKGBUILD.ungoogin#L73-L78

@bendlas
Copy link
Contributor

bendlas commented Dec 9, 2018

@rht I can do chromium builds in 1-2 h, so if you want me to try a build, feel free to reopen this.

@rht
Copy link
Member Author

rht commented Dec 9, 2018

Long term, I'd even consider building Nix' chromium with the ungoogled patch set by default, because I imagine that most people running chromium instead of chrome, do so, to get rid of google's supervision.

There are other variants, such as the inox-patchset, bromite, iridium, I'd say choosing ungoogled-chromium by default over the other requires a consensus.
Some people might still want the safe browsing to prevent the homoglyph attack (although there is a patch that adds a flag to enable punycode). The extensions are all disabled by default, so, no flash (in 2018?) and no pdf plugins.

@rht rht reopened this Dec 9, 2018
@bendlas
Copy link
Contributor

bendlas commented Dec 10, 2018

On my working branch, master...bendlas:chromium-ungoogled, I have added ungoogled-chromium sources to the auto-updater, integrated ungoogled-chromium into the build process and worked through a couple of fixes, some of which should be interesting for the regular chromium build as well. Got it working pretty far:

[14865/18450] CXX obj/third_party/blink/renderer/platform/platform/platform_jumbo_6.o
FAILED: obj/third_party/blink/renderer/platform/platform/platform_jumbo_6.o

@rht
Copy link
Member Author

rht commented Dec 10, 2018

I found https://aur.archlinux.org/packages/ungoogled-chromium/?comments=all&O=0&PP=10 at the comment

@matoro: Arch's harfbuzz is too new. Here is a patch for the PKGBUILD:

to the

FAILED: obj/third_party/blink/renderer/platform/platform/platform_jumbo_6.o

error message.

@bendlas
Copy link
Contributor

bendlas commented Dec 11, 2018

Added some more commits. Now the build fails at

[18469/18469] LINK ./chrome
FAILED: chrome
python "../../build/toolchain/gcc_link_wrapper.py" --output="./chrome" -- clang++ -pie -Wl,--version-script=../../build/linux/chrome.map -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -fuse-ld=gold -B/nix/store/z7fadhim7jrdz5fbqqssbsk50bl3iv2x-clang-wrapper-7.0.0/bin -m64 -Wl,-O2 -Wl,--gc-sections -rdynamic -Wl,-rpath-link=. -Wl,--disable-new-dtags -L/nix/store/74jjpddizvlay59v5dswxnplb0fgj6xl-glib-2.58.1/lib -L/nix/store/7q7hglda3r28lk30pqpl4fjzqbmcsy7k-icu4c-59.1/lib -L/nix/store/3q4i78kg7sm1nbws944vs3f7nq8czmw1-nspr-4.20/lib -L/nix/store/givkbf32y4516cfga4zxrpa10xghvgnp-nss-3.40.1/lib64 -L/nix/store/8wxmwa36pri3v0mvgzxzxg62dy1vpjq3-libva-2.3.0/lib -L/nix/store/d8105y30rlr10l6g27xml6d86qzgd6yn-libxml2-2.9.8/lib -L/nix/store/wkby3nkhwx75dbydc9hqjvndr7nsa7vi-dbus-1.12.10-lib/lib -L/nix/store/6hsjkkbpfk79hbdb3ffm4f640sgs5abm-libvpx-1.7.0/lib -L/nix/store/mva8acq88gxkyp47n07fn68k2w2aqhav-libopus-1.3/lib -L/nix/store/f3l96d1br38kdlif8b5031wc6lal3n38-libpng-apng-1.6.35/lib -L/nix/store/ma27f3n6h01ppxbl6h1igdywhvy4cmr1-zlib-1.2.11/lib -L/nix/store/i7bp1pxj9l49y5m7z3hbmp0lnygk1r2h-libwebp-1.0.1/lib -L/nix/store/vi5w51dnngnfh82afxr14vp5dasfhqxb-freetype-2.9/lib -L/nix/store/5dxcisxnk8xk783wgc0irf4mgb1pnzw1-harfbuzz-2.1.0/lib -L/nix/store/4fm4nw0wdasn6jyk80w732dpaccjmi9i-libdrm-2.4.96/lib -L/nix/store/q8pfgaiv5ba7gahcbc3r29z5psqq0wvn-libpulseaudio-12.2/lib -L/nix/store/hb8pfd6cvzsmzh66cs9cgh182q4rszla-ffmpeg-4.0.2/lib -L/nix/store/bigi6na0g7pfsg59dw0aqspwqqrpkh1r-cairo-1.16.0/lib -L/nix/store/sb9k455qjghsfznbx23ss0w7s6j7vml1-pango-1.42.4/lib -L/nix/store/hgq6336b1h68gjnnisqd52jw3qd6smz1-atk-2.30.0/lib -L/nix/store/zp10w11mk5nragsn8j4cdxfhlkb698cr-at-spi2-atk-2.30.0/lib -L/nix/store/6qmc93llzj5x6md057bvmld0y9yk20vh-flac-1.3.2/lib -L/nix/store/s5r2l2b2g4rzdfm679jbw9svjqfjhsz6-gdk-pixbuf-2.38.0/lib -L/nix/store/49926y8zcfra58dksmhjvv0ffrhv27g4-gtk+3-3.24.1/lib -L/nix/store/xs52mrqff9pjhvrjxf9ln0zd2d97kv8c-libxslt-1.1.32/lib -L/nix/store/4avwa2a566ihd7m3v478a77kmg9jrhzm-lcms2-2.9/lib -o "./chrome" -Wl,--start-group @"./chrome.rsp"  -Wl,--end-group   -latomic -ldl -lpthread -lrt -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender -lXtst -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lglib-2.0 -licui18n -licuuc -licudata -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lva -lcups -lz -lm -lcrypt -lxml2 -lexpat -lfontconfig -ldbus-1 -levent -lresolv -ljpeg -lvpx -lopus -lXss -lpng16 -lwebpdemux -lwebpmux -lwebp -lfreetype -lharfbuzz -ldrm -lXrandr -lasound -lpulse -lavcodec -lavformat -lavutil -lsnappy -lpangocairo-1.0 -lpango-1.0 -lcairo -lpci -latk-1.0 -latk-bridge-2.0 -lFLAC -lminizip -lgtk-3 -lgdk-3 -lcairo-gobject -lgdk_pixbuf-2.0 -lgio-2.0 -lxslt -llcms2
/nix/store/8rnh2n15w4wlz14wlg41b7mh8xjwjmsm-binutils-2.30/bin/ld.gold: error: /nix/store/10sp329j4mg8jbvmxckczz2zxy55l544-gcc-7.3.0/lib64/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libatomic.a(glfree.o): requires unsupported dynamic reloc 11; recompile with -fPIC
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)

I suspect, this is due to linking with gold. ungoogled-chromium would normally be built with lld, unfortunately, this also produces errors:

ld.lld: error: can't create dynamic relocation R_X86_64_64 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /nix/store/10sp329j4mg8jbvmxckczz2zxy55l544-gcc-7.3.0/lib64/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libatomic.a(gload.o)
>>> referenced by gload.o:(.rodata+0x80) in archive /nix/store/10sp329j4mg8jbvmxckczz2zxy55l544-gcc-7.3.0/lib64/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libatomic.a

passing -z,notext leads to

[80/121] ACTION //v8:run_torque(//build/toolchain/linux/unbundle:default)
FAILED: gen/v8/torque-generated/builtin-definitions-from-dsl.h gen/v8/torque-generated/builtins-base-from-dsl-gen.cc gen/v8/torque-generated/builtins-base-from-dsl-gen.h gen/v8/torque-generated/builtins-array-from-dsl-gen.cc gen/v8/torque-generated/builtins-array-from-dsl-gen.h gen/v8/torque-generated/builtins-typed-array-from-dsl-gen.cc gen/v8/torque-generated/builtins-typed-array-from-dsl-gen.h gen/v8/torque-generated/builtins-data-view-from-dsl-gen.cc gen/v8/torque-generated/builtins-data-view-from-dsl-gen.h gen/v8/torque-generated/builtins-test-from-dsl-gen.cc gen/v8/torque-generated/builtins-test-from-dsl-gen.h
python ../../v8/tools/run.py ./torque -o gen/v8/torque-generated ../../v8/src/builtins/base.tq ../../v8/src/builtins/array.tq ../../v8/src/builtins/array-copywithin.tq ../../v8/src/builtins/array-foreach.tq ../../v8/src/builtins/array-reverse.tq ../../v8/src/builtins/typed-array.tq ../../v8/src/builtins/data-view.tq ../../v8/test/torque/test-torque.tq ../../v8/third_party/v8/builtins/array-sort.tq

passing is_official_build = false gets rid of those early warnings, but produces more errors during the build (odd, because this is the default for nixos chromium)

I'll give it a final shot to compile with gold and -z,notext because those errors looked pretty similar.

@bendlas
Copy link
Contributor

bendlas commented Dec 12, 2018

no dice :-(

@rht
Copy link
Member Author

rht commented Dec 14, 2018

@bendlas what are the remaining possible permutations of the build flags x dependencies? (Excluding harfbuzz and freetype since they are vendored in the source anyway).

@bendlas
Copy link
Contributor

bendlas commented Dec 14, 2018

what are the remaining possible permutations of the build flags x dependencies? (Excluding harfbuzz and freetype since they are vendored in the source anyway).

Haven't kept track, sorry ... I just kept fixing build errors, but I don't know exactly what to do with the lld error. Not a C++ expert and I won't have much time in the next couple of weeks ...

Btw, if you want to work on this, I recommend building it with --keep-failed and then trying quick rebuilds with something like: sudo nix-shell . -A chromiumUngoogled.browser --run 'eval "$configurePhase"; eval "$buildPhase"'

@laalf
Copy link

laalf commented Apr 7, 2019

Is there still work being done to this? I am still very interested in ungoogled-chromium.

@bendlas
Copy link
Contributor

bendlas commented Apr 9, 2019

Since you asked: I've tried rebasing my branch and updating to the latest ungoogled release: master...bendlas:chromium-ungoogled

Still got early linker errors with lld. If you want take a look, asking around with chromium folks might be helpful. Here is the build log, building with lld, similar to arch: https://gist.github.com/bendlas/9b5d5444dcb57cf6f1b95469a9763a0f

bendlas added a commit to bendlas/nixpkgs that referenced this pull request Apr 12, 2019
bendlas added a commit to bendlas/nixpkgs that referenced this pull request Apr 12, 2019
bendlas added a commit to bendlas/nixpkgs that referenced this pull request Apr 12, 2019
see NixOS#51195

fix a wrong include location for nspr with an ad-hoc link farm
fix freetype private header include
@bendlas
Copy link
Contributor

bendlas commented Apr 12, 2019

I actually got the branch to run now master...bendlas:chromium-ungoogled
There seem to be problems with it, as it randomly freezes and is slow on input, but it's actually running. If somebody could help find out, what's going on there, that would be awesome. Besides that issue, there is some more cleanup work to be done, and maybe even a new variant of icu to be used.

In the meantime, if you want to have a look, I put the build here https://bendlas.cachix.org/

@doronbehar
Copy link
Contributor

The ungoogled-chromium is available in Nixpkgs. Thanks to @squalus. I'm closing this.

@doronbehar doronbehar closed this May 16, 2020
@rht rht deleted the ungoogled-chromium branch May 16, 2020 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants