Complete URL translation #2502

felixfontein · 2016-09-19T19:36:52Z

Test implementation of my idea in #2116, to allow to override the relative path to the post (relatively to the source folder specified in POSTS/PAGES) using post metadata.

fixes #2116

felixfontein · 2016-09-19T21:31:53Z

I modified the implementation so that you can use a utils.TranslatableSetting as Post's destination_base. The goal is to make the second argument to PAGES and POSTS a TranslatableSetting. I haven't touched the options yet since I'm not sure if these are used somewhere else as well.

felixfontein · 2016-09-25T01:14:16Z

@ralsina: any thoughts on this?

Kwpolska · 2016-09-25T18:22:18Z

nikola/post.py

        """Destination path for this post, relative to output/.

        If lang is not specified, it's the current language.
        Extension is used in the path if specified.
        """
        if lang is None:
            lang = nikola.utils.LocaleBorg().current_lang
+        if _force_source:
+            folder = self.folder


This sounds like putting source files in /posts/foo.rst and contents in /whatever/path/says.html. 👎

Ah. I think I misunderstood something about Post.source_link. Should be fixed now.

Kwpolska · 2016-09-25T18:23:14Z

nikola/post.py

@@ -843,7 +859,12 @@ def permalink(self, lang=None, absolute=False, extension='.html', query=None):
            extension = self.compiler.extension()

        pieces = self.translations[lang].split(os.sep)
-        pieces += self.folder.split(os.sep)
+        folder = self.meta[lang].get('path', self.folder_relative)


Make this an instance variable instead of copy-pasting this around and potentially running it multiple times.

Ok, that should be fixed now, too.

ralsina · 2016-09-26T17:48:24Z

I am concerned this makes it almost impossible to understand where a post will end up.

So, it would be something like

[translation prefix]/[localized dest folder]/[path metadata]/[slug] ?

ralsina · 2016-09-26T17:46:48Z

nikola/post.py

+        elif self.folder_base is not None:
+            self.folders = {lang: os.path.normpath(os.path.join(self.folder_base(lang), folder)) for lang, folder in self.folders.items()}
+        self.folder = self.folders[self.default_lang]
+


So, let me see if I understand this...

Post.destination_path will be calculated from the "path" metadata and the destination folder in POSTS which will be translatable.

If there is no path metadata, this will be "self.folder_relative". What is that? Does that name make sense?

Yes, that's the current behavior.

Whether the name makes sense: I don't know. This is just a test implementation for the idea in #2116.

ralsina · 2016-09-26T17:51:04Z

In general, I would prefer if features start as a description of the intended experience, and not as a code dump for everyone else to figure out :-P

Kwpolska · 2016-09-26T17:51:42Z

I’m 👎 for the entire idea. This complicates code and usage. What is the final path? Why is it like this?

And how are users going to find out about this anyway? How many people need this, and where are those translations even involved? This feels like a huge generalisation.

felixfontein · 2016-09-26T20:38:23Z

@ralsina: as mentioned in the PR text, this is a test implementation of the idea described in #2116.

@Kwpolska: there is some interest in having paths be translatable, see #2116. This is just one idea on how to actually implement it. Do you have a better idea?

Kwpolska · 2016-09-27T14:19:17Z

I’m afraid I don’t. But I don’t think this really solves the problem it claims it solves. The original issue was “make post paths (sections, mainly) translatable”. This makes (parts of?) the path customizable. Which might solve the problem, but in a round-about way, if the user manually changes the path to the correct section for every post. A real solution would make the section mechanism less dependent on exact output paths (and therefore less fragile).

(I don’t think anyone really cares about translating source URLs.)

felixfontein · 2016-09-30T12:07:16Z

It doesn't solve the problem completely. If you focus on sections and blog posts, and you actually use sections, then yes, it doesn't really solve it well. If you don't use sections, and don't have pages in a URL hierarchy, you will only need the translatable destination in POSTS and PAGES, but not .. path: xxx.

But if you care about static sites (i.e. pages and not posts), the story is completely different. Think for example about a static page which should have pages like /about/, /about/project/, /about/team/ in another language. In german, you'd like the URLs to be /de/ueber/, /de/ueber/projekt/, /de/ueber/mitarbeiter/ or something like that. That's currently impossible to do with Nikola if you want automatic links to the translated pages. If there's only one layer, like /about/, you can use .. slug: ueber in the german metadata to let it end up in /de/ueber/ instead of /de/about/, but for /about/team/, you can only change the team part with .. slug: mitarbeiter. The resulting URL will be /de/about/mitarbeiter/. The patch in this PR allows you to specify .. path: ueber as well to put the result to /de/ueber/mitarbeiter. Not the best way, but at least it results in good URLs.

(About source URLs: since the code generating source URLs is very similar to the one generating the post/page URL it is simpler to also translate the source URL than not to do that.)

Kwpolska · 2016-09-30T15:14:44Z

So, this really only solves a corner case: when the user wants to change the part between the slug and whatever comes from the language and PAGES settings.

A better solution would be one that’s useful for more things. Especially because the name path is terribly misleading here.

felixfontein · 2016-10-02T12:21:15Z

Well, that's not really true. This also allows to actually solve the problem in a better way for post scanning plugins. The path metadata could also be set by the post scanning plugin after creating the Post object; or it could provide a more specific destination_base for every post which specifies the final path.

Every better solution of this problem MUST be implemented in the post scanning plugin anyway: the post object shouldn't know about such things, and only the post scanning plugin can see more than one post to figure out a potential translated hierarchy.

But without a mechanism to actually allow the post scanning plugin to specify a path based on the language, there's no way to ever solve this.

So moving path out of metadata into something the post scanning plugin provides, or something the post scanning plugin can modify after creating the Post object (for example, by taking it out of the metadata itself) should be there.

felixfontein · 2016-10-09T10:19:38Z

How about just having support for localized dest folders in PAGES and POSTS? Would that be acceptable?

Kwpolska · 2016-10-09T10:20:55Z

Any example of how this would look?

felixfontein · 2016-10-09T10:21:34Z

(That would allow to play around with post scanning plugins which can place the posts depending on locale whereever they want. And it would remove the need for hacks as mentioned in #2116 which try to emulate this.)

felixfontein · 2016-10-09T10:24:51Z

You mean in the config? Something like:

POSTS = (
    ("posts/*.rst", {"en": "posts", "de": "beitraege"}, "post.tmpl"),
    ("posts/*.md", {"en": "posts", "de": "beitraege"}, "post.tmpl"),
)

The interface to Post's constructor would be similar to the one in this PR, mainly except that you cannot override anything with .. path: xx.

Kwpolska · 2016-10-09T10:35:26Z

That sounds better to me. As for the implementation, lines 166-172 in post.py look scary to me.

And perhaps PostScanner plugins should be more concerned with the paths?

felixfontein · 2016-10-09T10:44:28Z

I'll try to make it less scary, then :)

felixfontein · 2016-10-09T11:09:47Z

Ok. I hope it's now less scary! I've tested it with a modified demo page, it seems to work.

felixfontein · 2016-10-09T11:11:43Z

(Anyone minds if I do a rebase to get rid of the conflicts and to squash the commits, and then force-push?)

Kwpolska · 2016-10-09T11:13:16Z

Go ahead, just make sure to specify the branch (git push --force origin translated-paths).

Also allowing post scanning plugins to place posts depending on languages.

felixfontein · 2016-10-09T11:24:18Z

Now it looks kind of better :)

Signed-off-by: Chris Warrick <kwpolska@gmail.com>

Kwpolska

Not documented enough. It could also be slightly simpler.

Kwpolska · 2016-10-14T10:51:16Z

nikola/conf.py.in

@@ -119,6 +119,9 @@ THEME_COLOR = '#5670d4'
 # to feeds, indexes, tag lists and archives and are considered part
 # of a blog, while PAGES are just independent HTML pages.
 #
+# Finally, note that destination can be translated, i.e. you can
+# specify a different translation folder per language.


Needs an example here and in manual.

Kwpolska · 2016-10-14T10:53:55Z

nikola/post.py

@@ -845,7 +859,8 @@ def permalink(self, lang=None, absolute=False, extension='.html', query=None):
            extension = self.compiler.extension()

        pieces = self.translations[lang].split(os.sep)
-        pieces += self.folder.split(os.sep)
+        folder = self.folders[lang]
+        pieces += folder.split(os.sep)


Can be merged into two lines.

Do you mean "merge into one line"?

Yes, of course.

Kwpolska · 2016-10-14T10:57:23Z

nikola/post.py

@@ -161,6 +166,14 @@ def __init__(
            for lang in sorted(self.translated_to):
                default_metadata.update(self.meta[lang])

+        # Compose paths


Use an if/else here, perhaps a regular (simpler) for loop instead of convoluted comprehensions with partial items (re-using the first self.folders is just a waste of memory and time)

For example:

if self.folder_base is not None: # Use translatable destination folders self.folders = {} for lang in self.config['TRANSLATIONS'].keys(): self.folders[lang] = os.path.normpath(os.path.join( self.folder_base(lang), self.folder_relative)) else: # Old behavior (non-translatable destination path, normalized by scanner) self.folders = {lang: self.folder_relative for lang in self.config['TRANSLATIONS'].keys()}

Kwpolska · 2016-10-14T18:55:58Z

👍 from me as long as you add some more documentation.

felixfontein · 2016-10-14T18:58:18Z

Do you think that's enough documentation?

Kwpolska · 2016-10-14T19:00:59Z

artikel isn’t a direct translation of pages, is it now?

And perhaps add it to the Multilingual posts section of manual.txt. Merge when you’re done.

felixfontein · 2016-10-14T19:04:45Z

Yes, it's not a direct translation, but what I would use; I'd use articles in english instead of pages as well. Would you prefer a direct translation?

Kwpolska · 2016-10-14T19:13:28Z

Do as you please. In English, articles would primarily mean text in a newspaper which corresponds to posts on blogs (and this is what happens on your favorite newspaper’s website: the index page contains articles) — but that’s still a digression.

…ltilingual posts in manual.

felixfontein · 2016-10-14T19:42:59Z

I added a somewhat more general paragraph to Multilingual posts.

felixfontein mentioned this pull request Sep 19, 2016

Complete URL translation #2116

Closed

felixfontein added the in progress label Sep 19, 2016

Kwpolska requested changes Sep 25, 2016

View reviewed changes

ralsina reviewed Sep 26, 2016

View reviewed changes

Making the destination in POSTS and PAGES translatable.

b1e30fc

Also allowing post scanning plugins to place posts depending on languages.

felixfontein force-pushed the translated-paths branch from edeaf5c to b1e30fc Compare October 9, 2016 11:22

Kwpolska added 2 commits October 14, 2016 12:50

Merge branch 'master' into translated-paths

59cd58f

Update changelog

10e3094

Signed-off-by: Chris Warrick <kwpolska@gmail.com>

Kwpolska reviewed Oct 14, 2016

View reviewed changes

Simplifying code.

b854621

Kwpolska approved these changes Oct 14, 2016

View reviewed changes

Improved documenation.

1707992

felixfontein added 3 commits October 14, 2016 21:33

Using literal translation of 'pages'.

6cae40e

Marking PAGES and POSTS as 'translatable'. Adding new paragraph to mu…

2cf24f3

…ltilingual posts in manual.

Added some examples.

1e14a14

felixfontein merged commit 3fae233 into master Oct 14, 2016

felixfontein deleted the translated-paths branch October 14, 2016 20:10

felixfontein mentioned this pull request Oct 14, 2016

Added hierarchical_pages plugin. getnikola/plugins#178

Merged

Kwpolska added this to the v7.8.2 milestone Oct 15, 2016

Kwpolska removed the in progress label Oct 15, 2016

Kwpolska mentioned this pull request Oct 15, 2016

Improving hierarchical_pages. getnikola/plugins#179

Merged

Complete URL translation #2502

Complete URL translation #2502

Conversation

felixfontein commented Sep 19, 2016

felixfontein commented Sep 19, 2016

felixfontein commented Sep 25, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ralsina commented Sep 26, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ralsina commented Sep 26, 2016

Kwpolska commented Sep 26, 2016

felixfontein commented Sep 26, 2016

Kwpolska commented Sep 27, 2016

felixfontein commented Sep 30, 2016

Kwpolska commented Sep 30, 2016

felixfontein commented Oct 2, 2016

felixfontein commented Oct 9, 2016

Kwpolska commented Oct 9, 2016

felixfontein commented Oct 9, 2016

felixfontein commented Oct 9, 2016

Kwpolska commented Oct 9, 2016

felixfontein commented Oct 9, 2016

felixfontein commented Oct 9, 2016

felixfontein commented Oct 9, 2016

Kwpolska commented Oct 9, 2016

felixfontein commented Oct 9, 2016

Kwpolska left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kwpolska commented Oct 14, 2016

felixfontein commented Oct 14, 2016

Kwpolska commented Oct 14, 2016

felixfontein commented Oct 14, 2016

Kwpolska commented Oct 14, 2016

felixfontein commented Oct 14, 2016