Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

apache-kafka: replace ParNew to G1 YoungGeneration #100115

Closed
wants to merge 2 commits into from
Closed

apache-kafka: replace ParNew to G1 YoungGeneration #100115

wants to merge 2 commits into from

Conversation

GTrunSec
Copy link
Contributor

@GTrunSec GTrunSec commented Oct 9, 2020 •
edited

Motivation for this change
  1. the flag of UseParNewGC has been deprecated in JDK 9 and removed in JDK 10.

you can find info from here:

  1. remove CMS flags
    Dropping support for CMS and then removing the CMS code, or at least more thoroughly segregating it, will reduce the maintenance burden of the GC code base and accelerate new development. The G1 garbage collector is intended, in the long term, to be a replacement for most uses of CMS.

the issue output will be like this:

Unrecognized VM option 'UseParNewGC'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Things done
  • test flags
 more /nix/store/bq7qd8mmm63rwn1kzbw9rsfprnpa7q10-run-kafka/nix/store/y853yiia1ky7aipp73fv08xd2n6y3jyd-openjdk-14.0.2-ga/bin/java \
  -cp "/nix/store/kvq4jw9d9kzigdik4n2wrx1q9mzgapc2-apache-kafka-2.13-2.5.0/libs/*" \
  -Dlog4j.configuration=file:/nix/store/7qq35w8373pvbn9av0xk9c2rmwbas20m-log4j.properties \
  -server -Xmx1G -Xms1G -XX:+UseCompressedOops -XX:+UseG1GC -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSScavengeBeforeR
emark -XX:+DisableExplicitGC -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true \
  kafka.Kafka \
  /nix/store/ys10abnr79brsqh9ys1mxzx19jwmcd59-server.properties
  • output
    NOTICE: the ignoring warning will be sloved in second commit.
/nix/store/2jysm3dfsgby5sw5jgj43qjrb5v79ms9-bash-4.4-p23/bin/bash /nix/store/bq7qd8mmm63rwn1kzbw9rsfprnpa7q10-run-kafka
OpenJDK 64-Bit Server VM warning: Ignoring option UseConcMarkSweepGC; support was removed in 14.0
OpenJDK 64-Bit Server VM warning: Ignoring option CMSClassUnloadingEnabled; support was removed in 14.0
OpenJDK 64-Bit Server VM warning: Ignoring option CMSScavengeBeforeRemark; support was removed in 14.0
[2020-10-10 07:01:56,191] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2020-10-10 07:01:56,444] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2020-10-10 07:01:56,491] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2020-10-10 07:01:56,495] INFO starting (kafka.server.KafkaServer)
[2020-10-10 07:01:56,496] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2020-10-10 07:01:56,517] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2020-10-10 07:01:56,525] INFO Client environment:zookeeper.version=3.5.7-f0fdd52973d373ffd9c86b81d99842dc2c7f660e, built on 02/10/2020 11:30 GMT (org.apache.zookeeper.ZooKeeper)
[2020-10-10 07:01:56,525] INFO Client environment:host.name=gtcompile (org.apache.zookeeper.ZooKeeper)
[2020-10-10 07:01:56,525] INFO Client environment:java.version=14.0.2-internal (org.apache.zookeeper.ZooK
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@GTrunSec GTrunSec mentioned this pull request Oct 12, 2020
Merged
10 tasks
asbachb
asbachb reviewed Oct 12, 2020
View reviewed changes
nixos/modules/services/misc/apache-kafka.nix
"-XX:+UseConcMarkSweepGC"
"-XX:+CMSClassUnloadingEnabled"
"-XX:+CMSScavengeBeforeRemark"
"-XX:+UseG1GC"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there's a need to be explicit since G1 should be default GC since java 9.

Copy link
Contributor Author

@GTrunSec GTrunSec Oct 12, 2020 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/apache/kafka/blob/aba9036eb681597911343510e246de4fc547a479/bin/kafka-run-class.sh#L257. I think, the option of default should be matched with this line of the shell. otherwise, I can remove the default flags to list of example such as XX:+DisableExplicitGC was replaced by -XX:+ExplicitGCInvokesConcurrent in the default JVM settings. may this will be a good way to comment for users. it does not matter the flag is default setting or not. Or set the default list to empty(the example list as explain option).

@GTrunSec GTrunSec closed this Oct 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@asbachb asbachb asbachb left review comments

Assignees
No one assigned
Labels
6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 10.rebuild-linux: 1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@GTrunSec @asbachb