Fixing dependencies #2547

felixfontein · 2016-10-25T19:36:37Z

Implementation of #2536.

The first part ensures that the .dep file is always written and specified as a second target of the render_post task, so that the .dep file will be created if it isn't there yet. (Otherwise changing dependencies of the post might be ignored when the .dep file is gone.)

…get for render_posts plugin.

Kwpolska · 2016-10-26T14:45:34Z

I’d rather not create empty .dep files. Are there any issues with the current approach?

felixfontein · 2016-10-26T16:04:24Z

Yes: if the .dep file was removed for some reasons, and a file which should have been mentioned there was modified, the post won't be recompiled because Nikola cannot distinguish between an empty .dep file and the .dep file not being there.

felixfontein · 2016-10-28T21:55:04Z

How about letting the compiler decide whether .dep must be created (and let it insert the .dep target manually via the new function)?

Kwpolska · 2016-10-29T07:25:04Z

Are compilers the only things that can provide .dep files? If yes, we could do that.

felixfontein · 2016-10-29T10:42:50Z

Well, of course any other plugin could also create .dep files for whatever it wants :) To my knowledge, .dep files are only generated by compilers. Or more precisely, nowadays they are created by the Post object.

A problem is that sometimes compilers also create .dep files with a different meaning; for example the WordPress compiler. So always writing .dep files is probably not a good idea at all. How about: the plugin can indicate that the Post object should always write the .dep file, and add it to the list of targets. This will then be used by the ReST plugin and which else of the standard plugins uses the .dep mechanism.

I'll try that out.

… be always created and added as a target.

felixfontein · 2016-10-29T10:59:31Z

Ah. There's actually a bug in the .dep system: .dep files are written for different languages, but only the .dep file for the default language is ever read.

ralsina · 2016-11-18T14:24:56Z

nikola/plugin_categories.py

@@ -250,6 +250,7 @@ class PageCompiler(BasePlugin):
    friendly_name = ''
    demote_headers = False
    supports_onefile = True
+    use_dep_file = False  # If set to true, the .dep file is always written and added as a target


Is there a case when we want this to be False? If not, then let's lose the option and simplify.
Other than that, LGTM.

Well, some compilers might use .dep files for something else. Actually, the WordPress compiler does: it uses the .dep file to store a JSON-encoded file which not only contains file dependencies, but potentially other dependencies. (I added that feature when .dep files were completely handled by the compilers themselves.) I don't know if any other plugin not shipped with Nikola directly uses .dep like this as well.

Or perhaps we could make it so .dep always means file dependencies separated by newlines, and the WordPress compiler creates its custom .wpdep format?

And I think the default should be True.

felixfontein · 2016-11-20T15:06:22Z

Hmm, there seem to be a LOT of failing stuff in the tests which are not caused by this PR. Like all the flake8 warnings. I was only able to get these warnings after upgrading flake8. Also, it seems like the baseline is (again) out-of-date, as the generated thumbnails differ.

Kwpolska · 2016-11-20T16:21:52Z

nikola/plugin_categories.py

@@ -250,6 +250,7 @@ class PageCompiler(BasePlugin):
    friendly_name = ''
    demote_headers = False
    supports_onefile = True
+    use_dep_file = False  # If set to true, the .dep file is always written and added as a target


Or perhaps we could make it so .dep always means file dependencies separated by newlines, and the WordPress compiler creates its custom .wpdep format?

And I think the default should be True.

Kwpolska · 2016-11-20T16:23:40Z

nikola/plugin_categories.py

@@ -273,7 +278,18 @@ def _read_extra_deps(self, post):

    def register_extra_dependencies(self, post):
        """Add dependency to post object to check .dep file."""
-        post.add_dependency(lambda: self._read_extra_deps(post), 'fragment')
+        def create_lambda(lang):


Add a comment for future readers to explain this:

# We create a lambda like this so we can pass `lang` to it, because if we didn’t add that function, `lang` would always be the last language in TRANSLATIONS.

Fixed that.

@.dep file name: Yes, we could do that, too. I'll change it.

Signed-off-by: Chris Warrick <kwpolska@gmail.com>

felixfontein · 2016-11-20T18:21:47Z

I'll wait until the checks are done, and then I'll merge.

Fixing .dep filename, and adjusting to getnikola/nikola#2547.

Making sure that .dep file is always generated, and listing it as tar…

6470755

…get for render_posts plugin.

felixfontein added the in progress label Oct 25, 2016

felixfontein added 2 commits October 25, 2016 21:39

Allowing compilers to specify additional targets.

9e80b59

Making pydoctest happy.

6953d3d

felixfontein added 2 commits October 29, 2016 12:53

Compilers must explicitly state that they use the .dep file for it to…

022dbec

… be always created and added as a target.

Fixing bug.

3c5edb3

felixfontein added 2 commits October 29, 2016 13:12

Fixing wrong .dep filename bug.

83ffc8b

Merge branch 'master' into fixing-post-dependencies

7904138

felixfontein mentioned this pull request Oct 29, 2016

`.dep files: only dependencies for default language are ever used #2550

Closed

felixfontein removed the in progress label Oct 30, 2016

ralsina reviewed Nov 20, 2016

View reviewed changes

felixfontein added 2 commits November 20, 2016 15:38

Forgot lang argument.

8899b24

Merge branch 'master' into fixing-post-dependencies

8502aa7

felixfontein added a commit to getnikola/plugins that referenced this pull request Nov 20, 2016

Fixing .dep filename, and adjusting to getnikola/nikola#2547.

95b7bcb

felixfontein mentioned this pull request Nov 20, 2016

Fixing .dep filename, and adjusting to getnikola/nikola#2547. getnikola/plugins#189

Merged

Kwpolska reviewed Nov 20, 2016

View reviewed changes

felixfontein and others added 4 commits November 20, 2016 18:53

Merge branch 'master' into fixing-post-dependencies

044dda1

Added comment explaning inner function.

c7da1f3

Changing default to true.

a6a459c

Remove redundant use_dep_file lines

d59359a

Signed-off-by: Chris Warrick <kwpolska@gmail.com>

Kwpolska approved these changes Nov 20, 2016

View reviewed changes

felixfontein merged commit d6dde70 into master Nov 20, 2016

felixfontein deleted the fixing-post-dependencies branch November 20, 2016 18:52

felixfontein mentioned this pull request Nov 20, 2016

Let compiler specify additional targets #2536

Closed

Kwpolska added a commit to getnikola/plugins that referenced this pull request Nov 20, 2016

Merge pull request #189 from getnikola/fix-wordpress-plugin-dep-writing

4200935

Fixing .dep filename, and adjusting to getnikola/nikola#2547.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixing dependencies #2547

Fixing dependencies #2547

felixfontein commented Oct 25, 2016

Kwpolska commented Oct 26, 2016

felixfontein commented Oct 26, 2016

felixfontein commented Oct 28, 2016

Kwpolska commented Oct 29, 2016

felixfontein commented Oct 29, 2016

felixfontein commented Oct 29, 2016

ralsina Nov 18, 2016

felixfontein Nov 20, 2016

Kwpolska Nov 20, 2016

felixfontein commented Nov 20, 2016

Kwpolska Nov 20, 2016

Kwpolska Nov 20, 2016

felixfontein Nov 20, 2016

felixfontein Nov 20, 2016

felixfontein commented Nov 20, 2016

Fixing dependencies #2547

Fixing dependencies #2547

Conversation

felixfontein commented Oct 25, 2016

Kwpolska commented Oct 26, 2016

felixfontein commented Oct 26, 2016

felixfontein commented Oct 28, 2016

Kwpolska commented Oct 29, 2016

felixfontein commented Oct 29, 2016

felixfontein commented Oct 29, 2016

ralsina Nov 18, 2016

Choose a reason for hiding this comment

felixfontein Nov 20, 2016

Choose a reason for hiding this comment

Kwpolska Nov 20, 2016

Choose a reason for hiding this comment

felixfontein commented Nov 20, 2016

Kwpolska Nov 20, 2016

Choose a reason for hiding this comment

Kwpolska Nov 20, 2016

Choose a reason for hiding this comment

felixfontein Nov 20, 2016

Choose a reason for hiding this comment

felixfontein Nov 20, 2016

Choose a reason for hiding this comment

felixfontein commented Nov 20, 2016