Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

notmuch: 0.25.3 -> 0.26 #33680

Merged
merged 3 commits into from Jan 10, 2018
Merged

notmuch: 0.25.3 -> 0.26 #33680

merged 3 commits into from Jan 10, 2018

Conversation

flokli
Copy link
Contributor

@flokli flokli commented Jan 10, 2018

Motivation for this change

This bumps notmuch from version 0.25.3 to 0.26, which brings a ton of new features,
including support for cleartext indexing of encrypted emails and fixes for encoding-related crashes in Python 3.

Notmuch 0.26 (2018-01-09)

Command Line Interface

Support for re-indexing existing messages

There is a new subcommand, notmuch reindex, which re-indexes all
messages matching supplied search terms. This permits users to
change the way specific messages are indexed.

Note that for messages with multiple variants in the message
archive, the recorded Subject: of may change upon reindexing,
depending on the order in which the variants are indexed.

Improved error reporting in notmuch new

Give more details when reporting certain Xapian exceptions.

Support maildir synced tags in new.tags

Tags draft, flagged, passed, and replied are now supported
in new.tags. The tag unread is still special in the presence of
maildir syncing, and will be added for files in new/ regardless of
the setting of new.tags.

Support /regex/ in new.ignore

Files and directories may be ignored based on regular expressions.

Allow notmuch insert --folder=""

This inserts into the top level folder.

Strip trailing '/' from folder path for notmuch insert

This prevents a potential problem with duplicated database records.

New option --output=address for notmuch address

Make notmuch show more robust against deleting duplicate files

The option --decrypt now takes an explicit argument

The --decrypt option to notmuch show and notmuch reply now takes
an explicit argument. If you were used to invoking notmuch show --decrypt, you should switch to notmuch show --decrypt=true.

Boolean and keyword arguments now take a --no- prefix

Encrypted Mail

Indexing cleartext of encrypted e-mails

It's now possible to include the cleartext of encrypted e-mails in
the notmuch index. This makes it possible to search your encrypted
e-mails with the same ease as searching cleartext. This can be done
on a per-message basis by passing --decrypt=true to indexing
commands (new, insert, reindex), or by default by running "notmuch
config set index.decrypt true".

Encrypted messages whose cleartext is indexed will typically also
have their session keys stashed as properties associated with the
message. Stashed session keys permit rapid rendering of long
encrypted threads, and disposal of expired encryption-capable keys.
If for some reason you want cleartext indexing without stashed
session keys, use --decrypt=nostash for your indexing commands (or
run "notmuch config set index.decrypt nostash"). See index.decrypt
in notmuch-config(1) for more details.

Note that stashed session keys permit reconstruction of the
cleartext of the encrypted message itself, and the contents of the
index are roughly equivalent to the cleartext as well. DO NOT USE
this feature without considering the security of your index.

Emacs

Guard against concurrent searches in notmuch-tree

Use make-process when available

This allows newer Emacs to separate stdout and stderr from the
notmuch command without using temporary files.

Library Changes

Indexing files with duplicate message-id

Files with duplicate message-id's are now indexed, and searchable
via terms and phrases. There are known issues related to
presentation of results and regular-expression search, but in
principle no mail file should be completely unsearchable now.

New functions to count files

Two new functions in the libnotmuch API:
notmuch_message_count_files, and notmuch_thread_get_total_files.

New function to remove properties

A new function was added to the libnotmuch API to make it easier to
drop all properties with a common pattern:
notmuch_message_remove_all_properties_with_prefix

Change of return value of notmuch_thread_get_authors

In certain corner cases, notmuch_thread_get_authors previously
returned NULL. This has been replaced by an empty string, since the
possibility of NULL was not documented.

Transition notmuch_database_add_message to notmuch_database_index_file

When indexing an e-mail message, the new
notmuch_database_index_file function is the preferred form, and
the old notmuch_database_add_message is deprecated. The new form
allows passing a set of options to the indexing engine, which the
operator may decide to change from message to message.

Test Suite

Out-of-tree builds

The test suite now works properly with out-of-tree builds, i.e. with
separate source and build directories. The --root option to tests
has been dropped. The same can now be achieved more reliably using
out-of-tree builds.

Python Bindings

Python bindings specific Debian packaging is removed

The bindings have been build by the top level Debian packaging for a
long time, and bindings/python/debian has bit-rotted.

Open mail files in binary mode when using Python 3

This avoids certain encoding related crashes under Python 3.

Add python bindings for notmuch_database_{get,set}_config*

Optional decrypt_policy flag is available for notmuch.database().index_file()

nmbug

nmbug's internal version increases to 0.3 in this notmuch release.
User-facing changes with this notmuch release:

  • Accept failures to unset core.worktree in clone, which allows
    nmbug to be used with Git 2.11.0 and later.
  • Auto-checkout in clone if it wouldn't clobber existing content,
    which makes the initial clone more convenient.
  • Only error for invalid diff lines in tags/, which allows for
    READMEs and similar in nmbug repositories.

Documentation

New man page: notmuch-properties(7)

This new page to the manual describes common conventions for how
properties are used by libnotmuch, the CLI, and associated programs.
External projects that use properties are encouraged to claim their
properties and conventions here to avoid collisions.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

This was fixed upstream in f0d06c8ca3fce15c9e264445485bd93a6de8d25e, by
using epg-gpg-program instead of `(call-process epg-gpg-program …)` as
before.
@oxij
Copy link
Member

oxij commented Jan 10, 2018 via email

@andir
Copy link
Member

andir commented Jan 10, 2018

@GrahamcOfBorg build notmuch python3Packages.notmuch python2Packages.notmuch

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failure for system: x86_64-darwin

0  0:00:04
  0:00:04
--:--:-
- 8924
k

cannot build derivation ‘/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/33y1fi8p4bmzrb2j830nli25q41z405w-python2.7-python-notmuch-0.26.drv’: 2 dependencies couldn't be built
cannot build derivation ‘/nix/store/10xmmfr0z4jw34z7fhigi6zzagmyalz4-python3.6-python-notmuch-0.26.drv’: 2 dependencies couldn't be built
error: build of ‘/nix/store/10xmmfr0z4jw34z7fhigi6zzagmyalz4-python3.6-python-notmuch-0.26.drv’, ‘/nix/store/33y1fi8p4bmzrb2j830nli25q41z405w-python2.7-python-notmuch-0.26.drv’, ‘/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv’ failed

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Success for system: x86_64-linux

writing manifest file 'notmuch.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/g241wf6hjhaxz7v53y3cqpv4kqk2x7py-notmuch-0.26
/nix/store/l8v9s1vbf5w8p550p95mc6akdl7vpybb-python3.6-python-notmuch-0.26
/nix/store/hy5p8693ljxvdwlgpn7c3fvw4vd5lil4-python2.7-python-notmuch-0.26

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Success for system: aarch64-linux

writing manifest file 'notmuch.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/9sjv4bzgfzlm0amqp0g0qzip0r3fmgc8-notmuch-0.26
/nix/store/063d0mvgzdgdjyg2bcdz7fjw7ni7jiyq-python3.6-python-notmuch-0.26
/nix/store/x0b33spqmjg4n32711s41lj35ld4lrr4-python2.7-python-notmuch-0.26

@andir
Copy link
Member

andir commented Jan 10, 2018

@GrahamcOfBorg build notmuch python3Packages.notmuch python2Packages.notmuch

retrying since the darwin build failure looks funny

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failure for system: x86_64-darwin

  /nix/store/10xmmfr0z4jw34z7fhigi6zzagmyalz4-python3.6-python-notmuch-0.26.drv
  /nix/store/33y1fi8p4bmzrb2j830nli25q41z405w-python2.7-python-notmuch-0.26.drv
building path(s) ‘/nix/store/acn4c93dk40zmpwrkvd1jq00d39ii1cx-notmuch-0.26.tar.gz’
/nix/store/zsn4rc9p48hsq6a53357fmk314b0ahg4-stdenv-darwin/setup: fork: Operation not permitted
/nix/store/zsn4rc9p48hsq6a53357fmk314b0ahg4-stdenv-darwin/setup: fork: Operation not permitted
builder for ‘/nix/store/ilyjm12vyjs6z6s3r3hiy5zngrpx0sfv-notmuch-0.26.tar.gz.drv’ failed with exit code 254
cannot build derivation ‘/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/33y1fi8p4bmzrb2j830nli25q41z405w-python2.7-python-notmuch-0.26.drv’: 2 dependencies couldn't be built
cannot build derivation ‘/nix/store/10xmmfr0z4jw34z7fhigi6zzagmyalz4-python3.6-python-notmuch-0.26.drv’: 2 dependencies couldn't be built
error: build of ‘/nix/store/10xmmfr0z4jw34z7fhigi6zzagmyalz4-python3.6-python-notmuch-0.26.drv’, ‘/nix/store/33y1fi8p4bmzrb2j830nli25q41z405w-python2.7-python-notmuch-0.26.drv’, ‘/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv’ failed

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Success for system: aarch64-linux

/nix/store/9sjv4bzgfzlm0amqp0g0qzip0r3fmgc8-notmuch-0.26
/nix/store/063d0mvgzdgdjyg2bcdz7fjw7ni7jiyq-python3.6-python-notmuch-0.26
/nix/store/x0b33spqmjg4n32711s41lj35ld4lrr4-python2.7-python-notmuch-0.26

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Success for system: x86_64-linux

writing manifest file 'notmuch.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/g241wf6hjhaxz7v53y3cqpv4kqk2x7py-notmuch-0.26
/nix/store/l8v9s1vbf5w8p550p95mc6akdl7vpybb-python3.6-python-notmuch-0.26
/nix/store/hy5p8693ljxvdwlgpn7c3fvw4vd5lil4-python2.7-python-notmuch-0.26

@grahamc
Copy link
Member

grahamc commented Jan 10, 2018

@GrahamcOfBorg build notmuch python3Packages.notmuch python2Packages.notmuch

sorry for the noise, testing a darwin fix to OfBorg.

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Success for system: aarch64-linux

/nix/store/9sjv4bzgfzlm0amqp0g0qzip0r3fmgc8-notmuch-0.26
/nix/store/063d0mvgzdgdjyg2bcdz7fjw7ni7jiyq-python3.6-python-notmuch-0.26
/nix/store/x0b33spqmjg4n32711s41lj35ld4lrr4-python2.7-python-notmuch-0.26

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failure for system: x86_64-darwin

clang-4.0: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
ld: file not found: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error: Initial sanity checking of environment failed.  Please try
running configure in a clean environment, and if the problem persists,
report a bug.
builder for '/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv' failed with exit code 1
cannot build derivation '/nix/store/33y1fi8p4bmzrb2j830nli25q41z405w-python2.7-python-notmuch-0.26.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/10xmmfr0z4jw34z7fhigi6zzagmyalz4-python3.6-python-notmuch-0.26.drv': 1 dependencies couldn't be built
error: build of '/nix/store/10xmmfr0z4jw34z7fhigi6zzagmyalz4-python3.6-python-notmuch-0.26.drv', '/nix/store/33y1fi8p4bmzrb2j830nli25q41z405w-python2.7-python-notmuch-0.26.drv', '/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv' failed

Copy link

@GrahamcOfBorg GrahamcOfBorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Success for system: x86_64-linux

writing manifest file 'notmuch.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/g241wf6hjhaxz7v53y3cqpv4kqk2x7py-notmuch-0.26
/nix/store/l8v9s1vbf5w8p550p95mc6akdl7vpybb-python3.6-python-notmuch-0.26
/nix/store/hy5p8693ljxvdwlgpn7c3fvw4vd5lil4-python2.7-python-notmuch-0.26

@grahamc
Copy link
Member

grahamc commented Jan 10, 2018

Full build log:

these derivations will be built:
  /nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv
building '/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv'...
unpacking sources
unpacking source archive /nix/store/acn4c93dk40zmpwrkvd1jq00d39ii1cx-notmuch-0.26.tar.gz
source root is notmuch-0.26
setting SOURCE_DATE_EPOCH to timestamp 1515497301 of file notmuch-0.26/vim/syntax/notmuch-show.vim
patching sources
configuring
configure flags: --disable-dependency-tracking --prefix=/nix/store/isxj6ca18mwc76kd9yypmpc5v0m88l5l-notmuch-0.26
Welcome to Notmuch, a system for indexing, searching and tagging your email.

We hope that the process of building and installing notmuch is quick
and smooth so that you can soon be reading and processing your email
more efficiently than ever.

If anything goes wrong in the configure process, you can override any
decisions it makes by manually editing the Makefile.config file that
it creates. Also please do as much as you can to figure out what could
be different on your machine compared to those of the notmuch
developers. Then, please email those details to the Notmuch list
(notmuch@notmuchmail.org) so that we can hopefully make future
versions of notmuch easier for you to use.

We'll now investigate your system to verify that all required
dependencies are available:

Sanity checking C compilation environment... Fail.
Executed: clang -g -O2 minimal.c -o minimal
ld: file not found: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)
Sanity checking C++ compilation environment... Fail.
Executed: clang++ -g -O2 minimal.c -o minimal
clang-4.0: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
ld: file not found: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error: Initial sanity checking of environment failed.  Please try
running configure in a clean environment, and if the problem persists,
report a bug.
builder for '/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv' failed with exit code 1
error: build of '/nix/store/wxwxrzwsbz239jvpk02x18cmc57ay40h-notmuch-0.26.drv' failed

@grahamc
Copy link
Member

grahamc commented Jan 10, 2018

Looks like the issue is due to purity problems on Darwin.

I tried:

diff --git a/pkgs/applications/networking/mailreaders/notmuch/default.nix b/pkgs/applications/networking/mailreaders/notmuch/default.nix
index 0c0f55e6337..f21e4a3f51b 100644
--- a/pkgs/applications/networking/mailreaders/notmuch/default.nix
+++ b/pkgs/applications/networking/mailreaders/notmuch/default.nix
@@ -7,6 +7,7 @@
 , emacs
 , ruby
 , which, dtach, openssl, bash, gdb, man
+, darwin
 }:
 
 with stdenv.lib;
@@ -36,7 +37,7 @@ stdenv.mkDerivation rec {
     ruby  # (optional) ruby bindings
     which dtach openssl bash  # test dependencies
   ]
-  ++ optional stdenv.isDarwin fixDarwinDylibNames
+  ++ optionals stdenv.isDarwin [ fixDarwinDylibNames darwin.apple_sdk.frameworks.CoreFoundation ]
   ++ optionals (!stdenv.isDarwin) [ gdb man ]; # test dependencies
 
   postPatch = ''

but didn't seem to help. I think we can merge as-is.

@andir andir merged commit f77a5ba into NixOS:master Jan 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants