OK, after 3 hours I’ve concluded that py27 support for check is either a) unfixable or b) beyond my ability to fix. Too many Python internals are hardcoded to be ascii only. Anything that traverses the filesystem is a nightmare. I can cobble together something that runs on py27 excluding py3 and everything that isn’t Linux. py3 works fine as-is now.

Would it be okay ti throw an error and die if a user tries using check with non-ascii characters in anything older than py32? "It’s cool that you want to use UTF-8 paths, but you can’t have nice things in py2. Please upgrade to Python 3 to check your files and links."

Everything is awesome. Should work*. Does it work for you, @masayuko?

* OSX may still explode because of it’s unique NFD preference, seems to work fine, but really it shouldn’t have.

We wouldn’t mind not supporting Python 2 here.

This is ready to merge with py27 and py3 support. Someone want to sanity check it? @Kwpolska? @masayuko?

Sorry for late reply. I am checking it now. I'll report back later.
It seems that NFKC is better than NFD.
(refer to nameprep rfc3491 stringprep rfc3454)

OK. I have just tested in several ways. It works for me.
As mentioned above, it should use NFKC to normalize in util.encodelink().
I have encountered a issue that fail to create sitemap. When the words of author, category, tag, etc end with period(.), It will be occured due to follow line:

path = (path.replace(os.sep, '/') + '/').replace('./', '')

in nikola/plugins/task/sitemap/init.py.
It was fixed in my PR code #2005.

NFKC is destructive where as NFC is not. NFKC offers no advantages here as far as I know. Emailing discussions on W3C’s list recommended NFC over NFKC, so I’m sticking with that. (I couldn’t be bothered to dig up the link again.) NFD and NFKD is out of the question because they can’t be used reliably to produce the same output. Besides, everyone but Apple is using NFC everywhere for everything.

Regarding the issue with periods in the end of paths, could you make a separate issue about that?

The netloc should follow nameprep RFC3491, RFC3454. so prefer NFKC.
Examples of japanese idn include converting half-width kana between full-width kana.
In actual operation, half-width kana cannot be used as idn because half-width kana convert to full-width kana by NFKC.
Kana is the major character in Japanese and It has several expressions in Unicode.

ﾊ (U+FF8A) half-width
ハ (U+30CF) full-width
ﾊ (U+FF8A) -> NFC -> ﾊ (U+FF8A)　<-- disallow in idn
ﾊ (U+FF8A) -> NFKC -> ハ (U+30CF)

ﾊﾟ (U+FF8A U+FF9F) half-width and two code points
パ (U+30D1) full-width
ﾊﾟ (U+FF8A U+FF9F) -> NFC -> ﾊﾟ (U+FF8A U+FF9F) <-- disallow in idn
ﾊﾟ (U+FF8A U+FF9F) -> NFKC -> パ (U+30D1)

On the other hand, the path get used as the location name on a filesystem. so prefer NFC.

I have just checked myself.

print('パ'.encode('idna'))
b'xn--odk'
print('ﾊﾟ'.encode('idna'))
b'xn--odk'

NFC is no problem. Please forget the above said.