Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

borgbackup: 1.1.10 → 1.1.11 #82052

Merged
merged 1 commit into from Mar 8, 2020
Merged

Conversation

mweinelt
Copy link
Member

@mweinelt mweinelt commented Mar 8, 2020

Motivation for this change

New upstream release. https://github.com/borgbackup/borg/blob/1.1.11/docs/changes.rst#version-1111-2020-03-08

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Version 1.1.11 (2020-03-08)

Compatibility notes:

    When upgrading from borg 1.0.x to 1.1.x, please note:
        read all the compatibility notes for 1.1.0*, starting from 1.1.0b1.
        borg upgrade: you do not need to and you also should not run it.
        borg might ask some security-related questions once after upgrading. You can answer them either manually or via environment variable. One known case is if you use unencrypted repositories, then it will ask about a unknown unencrypted repository one time.
        your first backup with 1.1.x might be significantly slower (it might completely read, chunk, hash a lot files) - this is due to the --files-cache mode change (and happens every time you change mode). You can avoid the one-time slowdown by using the pre-1.1.0rc4-compatible mode (but that is less safe for detecting changed files than the default). See the --files-cache docs for details.
    1.1.11 removes WSL autodetection (Windows 10 Subsystem for Linux). If WSL still has a problem with sync_file_range, you need to set BORG_WORKAROUNDS=basesyncfile in the borg process environment to work around the WSL issue.

Fixes:

    fixed potential index corruption / data loss issue due to bug in hashindex_set, NixOS#4829 Please read and follow the more detailled notes close to the top of this document.
    upgrade bundled xxhash to 0.7.3, NixOS#4891 0.7.2 is the minimum requirement for correct operations on ARMv6 in non-fixup mode, where unaligned memory accesses cause bus errors. 0.7.3 adds some speedups and libxxhash 0.7.3 even has a pkg-config file now.
    upgrade bundled lz4 to 1.9.2
    upgrade bundled zstd to 1.4.4
    fix crash when upgrading erroneous hints file, NixOS#4922
    extract:
        fix KeyError for "partial" extraction, NixOS#4607
        fix "partial" extract for hardlinked contentless file types, NixOS#4725
        fix preloading for old (0.xx) remote servers, NixOS#4652
        fix confusing output of borg extract --list --strip-components, NixOS#4934
    delete: after double-force delete, warn about necessary repair, NixOS#4704
    create: give invalid repo error msg if repo config not found, NixOS#4411
    mount: fix FUSE mount missing st_birthtime, NixOS#4763 NixOS#4767
    check: do not stumble over invalid item key, NixOS#4845
    info: if the archive doesn't exist, print a pretty message, NixOS#4793
    SecurityManager.known(): check all files, NixOS#4614
    Repository.open: use stat() to check for repo dir, NixOS#4695
    Repository.check_can_create_repository: use stat() to check, NixOS#4695
    fix invalid archive error message
    fix optional/non-optional location arg, NixOS#4541
    commit-time free space calc: ignore bad compact map entries, NixOS#4796
    ignore EACCES (errno 13) when hardlinking the old config, NixOS#4730
    --prefix / -P: fix processing, avoid argparse issue, NixOS#4769

New features:

    enable placeholder usage in all extra archive arguments
    new BORG_WORKAROUNDS mechanism, basesyncfile, NixOS#4710
    recreate: support --timestamp option, NixOS#4745
    support platforms without os.link (e.g. Android with Termux), NixOS#4901 if we don't have os.link, we just extract another copy instead of making a hardlink.
    support linux platforms without sync_file_range (e.g. Android 7 with Termux), NixOS#4905

Other:

    ignore --stats when given with --dry-run, but continue, NixOS#4373
    add some ProgressIndicator msgids to code / fix docs, NixOS#4935
    elaborate on "Calculating size" message
    argparser: always use REPOSITORY in metavar, also use more consistent help phrasing.
    check: improve error output for matching index size, see NixOS#4829
    docs:
        changelog: add advisory about hashindex_set bug NixOS#4829
        better describe BORG_SECURITY_DIR, BORG_CACHE_DIR, NixOS#4919
        infos about cache security assumptions, NixOS#4900
        add FAQ describing difference between a local repo vs. repo on a server.
        document how to test exclusion patterns without performing an actual backup
        timestamps in the files cache are now usually ctime, NixOS#4583
        fix bad reference to borg compact (does not exist in 1.1), NixOS#4660
        create: borg 1.1 is not future any more
        extract: document limitation "needs empty destination", NixOS#4598
        how to supply a passphrase, use crypto devices, NixOS#4549
        fix osxfuse github link in installation docs
        add example of exclude-norecurse rule in help patterns
        update macOS Brew link
        add note about software for automating backups, NixOS#4581
        AUTHORS: mention copyright+license for bundled msgpack
        fix various code blocks in the docs, NixOS#4708
        updated docs to cover use of temp directory on remote, NixOS#4545
        add restore docs, NixOS#4670
        add a pull backup / push restore how-to, NixOS#1552
        add FAQ how to retain original paths, NixOS#4532
        explain difference between --exclude and --pattern, NixOS#4118
        add FAQs for SSH connection issues, NixOS#3866
        improve password FAQ, NixOS#4591
        reiterate that 'file cache names are absolute' in FAQ
    tests:
        cope with ANY error when importing pytest into borg.testsuite, NixOS#4652
        fix broken test that relied on improper zlib assumptions
        test_fuse: filter out selinux xattrs, NixOS#4574
    travis / vagrant:
        misc python versions removed / changed (due to openssl 1.1 compatibility) or added (3.7 and 3.8, for better borg compatibility testing)
        binary building is on python 3.5.9 now
    vagrant:
        add new boxes: ubuntu 18.04 and 20.04, debian 10
        update boxes: openindiana, darwin, netbsd
        remove old boxes: centos 6
        darwin: updated osxfuse to 3.10.4
        use debian/ubuntu pip/virtualenv packages
        rather use python 3.6.2 than 3.6.0, fixes coverage/sqlite3 issue
        use requirements.d/development.lock.txt to avoid compat issues
    travis:
        darwin: backport some install code / order from master
        remove deprecated keyword "sudo" from travis config
        allow osx builds to fail, NixOS#4955 this is due to travis-ci frequently being so slow that the OS X builds just fail because they exceed 50 minutes and get killed by travis.
@mweinelt
Copy link
Member Author

mweinelt commented Mar 8, 2020

Poking @flokli @globin @dotlambda

Copy link
Member

@lsix lsix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Built and tested.

All good for me,

Thanks!

@flokli flokli merged commit a2aacb6 into NixOS:master Mar 8, 2020
@@ -58,7 +58,7 @@ python3.pkgs.buildPythonApplication rec {
HOME=$(mktemp -d) py.test --pyargs borg.testsuite
'';

# 63 failures, needs pytest-benchmark
# 64 failures, needs pytest-benchmark
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have pytest-benchmark since eb6e376 (and it got bumped in the meantime as well). Can you take a look if we can run some of the tests?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lot's of PermissionError exceptions and it's also complaining about fuse not working.

https://gist.github.com/mweinelt/d5c1dbfc78cdbd02ee46a688f7f019f1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This look like the test suite simply tries to use the current working directory (which is read-only), instead of setting up a temporary scratchdir.

You could try a cd $(mktemp -d) before invoking the tests, and if that works, ask upstream to use https://pytest.readthedocs.io/en/2.7.3/tmpdir.html instead of just assuming the folder is write-able.

@mweinelt mweinelt deleted the pr/borgbackup/1.1.11 branch March 8, 2020 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants