-
-
Notifications
You must be signed in to change notification settings - Fork 14.6k
Comparing changes
Open a pull request
base repository: NixOS/nixpkgs
base: ed41afe60a75
head repository: NixOS/nixpkgs
compare: 19c8562dbc10
Commits on Oct 26, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 4deb3a5 - Browse repository at this point
Copy the full SHA 4deb3a5View commit details
Commits on Oct 30, 2020
-
Configuration menu - View commit details
-
Copy full SHA for c766921 - Browse repository at this point
Copy the full SHA c766921View commit details -
Configuration menu - View commit details
-
Copy full SHA for f5994c2 - Browse repository at this point
Copy the full SHA f5994c2View commit details -
nixos ec2/create-amis.sh: shellcheck: SC2155: Declare and assign sepa…
…rately to avoid masking return values.
Configuration menu - View commit details
-
Copy full SHA for baf7ed3 - Browse repository at this point
Copy the full SHA baf7ed3View commit details -
Configuration menu - View commit details
-
Copy full SHA for a66a22c - Browse repository at this point
Copy the full SHA a66a22cView commit details -
nixos ec2/create-amis.sh: shellcheck: explicitly make the additions t…
…o block_device_mappings single strings
Configuration menu - View commit details
-
Copy full SHA for 7dac847 - Browse repository at this point
Copy the full SHA 7dac847View commit details -
Configuration menu - View commit details
-
Copy full SHA for f92a883 - Browse repository at this point
Copy the full SHA f92a883View commit details -
Configuration menu - View commit details
-
Copy full SHA for e253de8 - Browse repository at this point
Copy the full SHA e253de8View commit details -
create-amis: allow customizing the service role name
The complete setup on the AWS end can be configured with the following Terraform configuration. It generates a ./credentials.sh which I just copy/pasted in to the create-amis.sh script near the top. Note: the entire stack of users and bucket can be destroyed at the end of the import. variable "region" { type = string } variable "availability_zone" { type = string } provider "aws" { region = var.region } resource "aws_s3_bucket" "nixos-amis" { bucket_prefix = "nixos-amis-" lifecycle_rule { enabled = true abort_incomplete_multipart_upload_days = 1 expiration { days = 7 } } } resource "local_file" "credential-file" { file_permission = "0700" filename = "${path.module}/credentials.sh" sensitive_content = <<SCRIPT export service_role_name="${aws_iam_role.vmimport.name}" export bucket="${aws_s3_bucket.nixos-amis.bucket}" export AWS_ACCESS_KEY_ID="${aws_iam_access_key.uploader.id}" export AWS_SECRET_ACCESS_KEY="${aws_iam_access_key.uploader.secret}" SCRIPT } # The following resources are for the *uploader* resource "aws_iam_user" "uploader" { name = "nixos-amis-uploader" } resource "aws_iam_access_key" "uploader" { user = aws_iam_user.uploader.name } resource "aws_iam_user_policy" "upload-to-nixos-amis" { user = aws_iam_user.uploader.name policy = data.aws_iam_policy_document.upload-policy-document.json } data "aws_iam_policy_document" "upload-policy-document" { statement { effect = "Allow" actions = [ "s3:ListBucket", "s3:GetBucketLocation", ] resources = [ aws_s3_bucket.nixos-amis.arn ] } statement { effect = "Allow" actions = [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject", ] resources = [ "${aws_s3_bucket.nixos-amis.arn}/*" ] } statement { effect = "Allow" actions = [ "ec2:ImportSnapshot", "ec2:DescribeImportSnapshotTasks", "ec2:DescribeImportSnapshotTasks", "ec2:RegisterImage", "ec2:DescribeImages" ] resources = [ "*" ] } } # The following resources are for the *vmimport service user* # See: https://docs.aws.amazon.com/vm-import/latest/userguide/vmie_prereqs.html#vmimport-role resource "aws_iam_role" "vmimport" { assume_role_policy = data.aws_iam_policy_document.vmimport-trust.json } resource "aws_iam_role_policy" "vmimport-access" { role = aws_iam_role.vmimport.id policy = data.aws_iam_policy_document.vmimport-access.json } data "aws_iam_policy_document" "vmimport-access" { statement { effect = "Allow" actions = [ "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket", ] resources = [ aws_s3_bucket.nixos-amis.arn, "${aws_s3_bucket.nixos-amis.arn}/*" ] } statement { effect = "Allow" actions = [ "ec2:ModifySnapshotAttribute", "ec2:CopySnapshot", "ec2:RegisterImage", "ec2:Describe*" ] resources = [ "*" ] } } data "aws_iam_policy_document" "vmimport-trust" { statement { effect = "Allow" principals { type = "Service" identifiers = [ "vmie.amazonaws.com" ] } actions = [ "sts:AssumeRole" ] condition { test = "StringEquals" variable = "sts:ExternalId" values = [ "vmimport" ] } } }
Configuration menu - View commit details
-
Copy full SHA for 2bf1fc0 - Browse repository at this point
Copy the full SHA 2bf1fc0View commit details -
stage-1: add datestamps to logs
When the stage-1 logs get imported in to the journal, they all get loaded with the same timestamp. This makes it difficult to identify what might be taking a long time in early boot.
Configuration menu - View commit details
-
Copy full SHA for a179781 - Browse repository at this point
Copy the full SHA a179781View commit details -
stage-1: modprobe ext{2,3,4} before resizing
I noticed booting a system with an ext4 root which expanded to 5T took quite a long time (12 minutes in some cases, 43(!) in others.) I changed stage-1 to run `resize2fs -d 62` for extra debug output and timing information. It revealed the adjust_superblock step taking almost all of the time: [Fri Oct 30 11:10:15 UTC 2020] zero_high_bits_in_metadata: Memory used: 132k/0k (63k/70k), time: 0.00/ 0.00/ 0.00 [Fri Oct 30 11:21:09 UTC 2020] adjust_superblock: Memory used: 396k/4556k (295k/102k), time: 654.21/ 0.59/ 5.13 but when I ran resize2fs on a disk with the identical content growing to the identical target size, it would only take about 30 seconds. I looked at what happened between those two steps in the fast case with strace and found: ``` 235 getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=1795}, ru_stime={tv_sec=0, tv_usec=3590}, ...}) = 0 236 write(1, "zero_high_bits_in_metadata: Memo"..., 84zero_high_bits_in_metadata: Memory used: 132k/0k (72k/61k), time: 0.00/ 0.00/ 0.00 237 ) = 84 238 gettimeofday({tv_sec=1604061278, tv_usec=480147}, NULL) = 0 239 getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=1802}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0 240 gettimeofday({tv_sec=1604061278, tv_usec=480192}, NULL) = 0 241 mmap(NULL, 2564096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa3c7355000 242 access("/sys/fs/ext4/features/lazy_itable_init", F_OK) = 0 243 brk(0xf85000) = 0xf85000 244 brk(0xfa6000) = 0xfa6000 245 gettimeofday({tv_sec=1604061278, tv_usec=538828}, NULL) = 0 246 getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=58720}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0 247 write(1, "adjust_superblock: Memory used: "..., 79adjust_superblock: Memory used: 396k/2504k (305k/92k), time: 0.06/ 0.06/ 0.00 248 ) = 79 249 gettimeofday({tv_sec=1604061278, tv_usec=539119}, NULL) = 0 250 getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=58812}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0 251 gettimeofday({tv_sec=1604061279, tv_usec=939}, NULL) = 0 252 getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=520411}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0 253 write(1, "fix_uninit_block_bitmaps 2: Memo"..., 88fix_uninit_block_bitmaps 2: Memory used: 396k/2504k (305k/92k), time: 0.46/ 0.46/ 0.00 254 ) = 88 ``` In particular the access to /sys/fs seemed interesting. Looking at the source of resize2fs: ``` [root@ip-172-31-22-182:~/e2fsprogs-1.45.5]# rg -B2 -A1 /sys/fs/ext4/features/lazy_itable_init . ./resize/resize2fs.c 923- if (getenv("RESIZE2FS_FORCE_LAZY_ITABLE_INIT") || 924- (!getenv("RESIZE2FS_FORCE_ITABLE_INIT") && 925: access("/sys/fs/ext4/features/lazy_itable_init", F_OK) == 0)) 926- lazy_itable_init = 1; ``` I confirmed /sys is mounted, and then found a bug suggesting the ext4 module is maybe not loaded: https://bugzilla.redhat.com/show_bug.cgi?id=1071909 My home server doesn't have ext4 loaded and had 3T to play with, so I tried (and succeeded with) replicating the issue locally: ``` [root@kif:/scratch]# lsmod | grep -i ext [root@kif:/scratch]# zfs create -V 3G rpool/scratch/ext4 [root@kif:/scratch]# time mkfs.ext4 /dev/zvol/rpool/scratch/ext4 mke2fs 1.45.5 (07-Jan-2020) Discarding device blocks: done Creating filesystem with 786432 4k blocks and 196608 inodes Filesystem UUID: 560a4a8f-93dc-40cc-97a5-f10049bf801f Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done real 0m2.261s user 0m0.000s sys 0m0.025s [root@kif:/scratch]# zfs set volsize=3T rpool/scratch/ext4 [root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4 resize2fs 1.45.5 (07-Jan-2020) fs has 11 inodes, 1 groups required. fs requires 16390 data blocks. With 1 group(s), we have 22234 blocks available. Last group's overhead is 10534 Need 16390 data blocks in last group Final size of last group is 26924 Estimated blocks needed: 26924 Extents safety margin: 49 Resizing the filesystem on /dev/zvol/rpool/scratch/ext4 to 805306368 (4k) blocks. read_bitmaps: Memory used: 132k/0k (63k/70k), time: 0.00/ 0.00/ 0.00 read_bitmaps: I/O read: 1MB, write: 0MB, rate: 3802.28MB/s fix_uninit_block_bitmaps 1: Memory used: 132k/0k (63k/70k), time: 0.00/ 0.00/ 0.00 resize_group_descriptors: Memory used: 132k/0k (68k/65k), time: 0.00/ 0.00/ 0.00 move_bg_metadata: Memory used: 132k/0k (68k/65k), time: 0.00/ 0.00/ 0.00 zero_high_bits_in_metadata: Memory used: 132k/0k (68k/65k), time: 0.00/ 0.00/ 0.00 ``` here it got stuck for quite some time ... straceing this 20 minutes in revealed this in a tight loop: ``` getuid() = 0 geteuid() = 0 getgid() = 0 getegid() = 0 prctl(PR_GET_DUMPABLE) = 1 (SUID_DUMP_USER) fallocate(3, FALLOC_FL_ZERO_RANGE, 2222649901056, 2097152) = 0 fsync(3) = 0 ``` it finally ended 43(!) minutes later: ``` adjust_superblock: Memory used: 264k/3592k (210k/55k), time: 2554.03/ 0.16/15.07 fix_uninit_block_bitmaps 2: Memory used: 264k/3592k (210k/55k), time: 0.16/ 0.16/ 0.00 blocks_to_move: Memory used: 264k/3592k (211k/54k), time: 0.00/ 0.00/ 0.00 Number of free blocks: 755396/780023556, Needed: 0 block_mover: Memory used: 264k/3592k (216k/49k), time: 0.05/ 0.01/ 0.00 block_mover: I/O read: 1MB, write: 0MB, rate: 18.68MB/s inode_scan_and_fix: Memory used: 264k/3592k (216k/49k), time: 0.00/ 0.00/ 0.00 inode_ref_fix: Memory used: 264k/3592k (216k/49k), time: 0.00/ 0.00/ 0.00 move_itables: Memory used: 264k/3592k (216k/49k), time: 0.00/ 0.00/ 0.00 calculate_summary_stats: Memory used: 264k/3592k (216k/49k), time: 16.35/16.35/ 0.00 fix_resize_inode: Memory used: 264k/3592k (222k/43k), time: 0.04/ 0.00/ 0.00 fix_resize_inode: I/O read: 1MB, write: 0MB, rate: 22.80MB/s fix_sb_journal_backup: Memory used: 264k/3592k (222k/43k), time: 0.00/ 0.00/ 0.00 overall resize2fs: Memory used: 264k/3592k (222k/43k), time: 2570.90/16.68/15.07 overall resize2fs: I/O read: 1MB, write: 1MB, rate: 0.00MB/s The filesystem on /dev/zvol/rpool/scratch/ext4 is now 805306368 (4k) blocks long. real 43m1.943s user 0m16.761s sys 0m15.069s ``` I then cleaned up and recreated the zvol, loaded the ext4 module, created the ext4 fs, resized the volume, and resize2fs'd and it went quite quickly: ``` [root@kif:/scratch]# zfs destroy rpool/scratch/ext4 [root@kif:/scratch]# zfs create -V 3G rpool/scratch/ext4 [root@kif:/scratch]# modprobe ext4 [root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4 [root@kif:/scratch]# time mkfs.ext4 /dev/zvol/rpool/scratch/ext4 mke2fs 1.45.5 (07-Jan-2020) Discarding device blocks: done Creating filesystem with 786432 4k blocks and 196608 inodes Filesystem UUID: 5b415f2f-a8c4-4ba0-ac1d-78860de77610 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done real 0m1.013s user 0m0.001s sys 0m0.023s [root@kif:/scratch]# zfs set volsize=3T rpool/scratch/ext4 [root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4 resize2fs 1.45.5 (07-Jan-2020) fs has 11 inodes, 1 groups required. fs requires 16390 data blocks. With 1 group(s), we have 22234 blocks available. Last group's overhead is 10534 Need 16390 data blocks in last group Final size of last group is 26924 Estimated blocks needed: 26924 Extents safety margin: 49 Resizing the filesystem on /dev/zvol/rpool/scratch/ext4 to 805306368 (4k) blocks. read_bitmaps: Memory used: 132k/0k (63k/70k), time: 0.00/ 0.00/ 0.00 read_bitmaps: I/O read: 1MB, write: 0MB, rate: 3389.83MB/s fix_uninit_block_bitmaps 1: Memory used: 132k/0k (63k/70k), time: 0.00/ 0.00/ 0.00 resize_group_descriptors: Memory used: 132k/0k (68k/65k), time: 0.00/ 0.00/ 0.00 move_bg_metadata: Memory used: 132k/0k (68k/65k), time: 0.00/ 0.00/ 0.00 zero_high_bits_in_metadata: Memory used: 132k/0k (68k/65k), time: 0.00/ 0.00/ 0.00 adjust_superblock: Memory used: 264k/1540k (210k/55k), time: 0.02/ 0.02/ 0.00 fix_uninit_block_bitmaps 2: Memory used: 264k/1540k (210k/55k), time: 0.15/ 0.15/ 0.00 blocks_to_move: Memory used: 264k/1540k (211k/54k), time: 0.00/ 0.00/ 0.00 Number of free blocks: 755396/780023556, Needed: 0 block_mover: Memory used: 264k/3592k (216k/49k), time: 0.01/ 0.01/ 0.00 block_mover: I/O read: 1MB, write: 0MB, rate: 157.11MB/s inode_scan_and_fix: Memory used: 264k/3592k (216k/49k), time: 0.00/ 0.00/ 0.00 inode_ref_fix: Memory used: 264k/3592k (216k/49k), time: 0.00/ 0.00/ 0.00 move_itables: Memory used: 264k/3592k (216k/49k), time: 0.00/ 0.00/ 0.00 calculate_summary_stats: Memory used: 264k/3592k (216k/49k), time: 16.20/16.20/ 0.00 fix_resize_inode: Memory used: 264k/3592k (222k/43k), time: 0.00/ 0.00/ 0.00 fix_resize_inode: I/O read: 1MB, write: 0MB, rate: 5319.15MB/s fix_sb_journal_backup: Memory used: 264k/3592k (222k/43k), time: 0.00/ 0.00/ 0.00 overall resize2fs: Memory used: 264k/3592k (222k/43k), time: 16.45/16.38/ 0.00 overall resize2fs: I/O read: 1MB, write: 1MB, rate: 0.06MB/s The filesystem on /dev/zvol/rpool/scratch/ext4 is now 805306368 (4k) blocks long. real 0m17.908s user 0m16.386s sys 0m0.079s ``` Success!
Configuration menu - View commit details
-
Copy full SHA for ece5c0f - Browse repository at this point
Copy the full SHA ece5c0fView commit details -
create-amis: improve wording around the service name's IAM role
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
Configuration menu - View commit details
-
Copy full SHA for 74a577b - Browse repository at this point
Copy the full SHA 74a577bView commit details -
amazon-glacier-cmd-interface: remove package
Last real upstream activity appears to be ~6 years ago and depends on python2 packages that will soon be vulnerable Helps #101964
Configuration menu - View commit details
-
Copy full SHA for b5faf8e - Browse repository at this point
Copy the full SHA b5faf8eView commit details -
amazon-image: random.trust_cpu=on to cut 10s from boot
Ubuntu and other distros already have this set via kernel config.
Configuration menu - View commit details
-
Copy full SHA for c851030 - Browse repository at this point
Copy the full SHA c851030View commit details -
Configuration menu - View commit details
-
Copy full SHA for d78aa08 - Browse repository at this point
Copy the full SHA d78aa08View commit details -
Configuration menu - View commit details
-
Copy full SHA for d77ddf2 - Browse repository at this point
Copy the full SHA d77ddf2View commit details -
Merge pull request #102171 from grahamc/faster-ext-resize
stage-1: modprobe ext{2,3,4} before resizing (so resizing takes less than 45 minutes)
Configuration menu - View commit details
-
Copy full SHA for b34cf36 - Browse repository at this point
Copy the full SHA b34cf36View commit details -
Merge pull request #102172 from grahamc/stage-1-datestamps
stage-1: add datestamps to logs
Configuration menu - View commit details
-
Copy full SHA for 82578fc - Browse repository at this point
Copy the full SHA 82578fcView commit details -
Merge pull request #102173 from grahamc/create-amis
create-amis.sh: fixup shellcheck issues, improve error logging, and add configurable service names
Configuration menu - View commit details
-
Copy full SHA for c06b971 - Browse repository at this point
Copy the full SHA c06b971View commit details -
Merge pull request #102175 from grahamc/ami-random
amazon-image: random.trust_cpu=on to cut 10s from boot
Configuration menu - View commit details
-
Copy full SHA for 860a3a2 - Browse repository at this point
Copy the full SHA 860a3a2View commit details -
Merge pull request #102174 from grahamc/ami-root-use-gpt
AMI root partition table: use GPT to support >2T partitions
Configuration menu - View commit details
-
Copy full SHA for 38a394b - Browse repository at this point
Copy the full SHA 38a394bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 76cd544 - Browse repository at this point
Copy the full SHA 76cd544View commit details -
Merge pull request #101783 from elseym/plex
plex: 1.20.2.3402 -> 1.20.3.3483
Configuration menu - View commit details
-
Copy full SHA for 76dc3ea - Browse repository at this point
Copy the full SHA 76dc3eaView commit details -
Merge pull request #101384 from eyJhb/among-sus
among-sus: init at 2020-10-22
Configuration menu - View commit details
-
Copy full SHA for 8faab3d - Browse repository at this point
Copy the full SHA 8faab3dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 822edc5 - Browse repository at this point
Copy the full SHA 822edc5View commit details -
Merge pull request #102163 from mkenigs/rm-glacier-cmd
amazon-glacier-cmd-interface: remove package
Configuration menu - View commit details
-
Copy full SHA for 3d37920 - Browse repository at this point
Copy the full SHA 3d37920View commit details -
Merge pull request #101829 from NixOS/haskell-updates
Update Haskell package set to LTS 16.20 (plus other fixes)
Configuration menu - View commit details
-
Copy full SHA for 19c8562 - Browse repository at this point
Copy the full SHA 19c8562View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff ed41afe60a75...19c8562dbc10