WordPress compiler plugin. #91

felixfontein · 2015-06-27T12:56:44Z

This plugin allows to compile WordPress posts (i.e. posts directly copied out of a WordPress blog, without any modifications).

It has support for shortcodes. By default, it only supports the [code] shortcode, but plugins for other shortcodes can be added via a plugin interface. See wordpress/plugins/code.py for an example.

Improved some spacing.

ralsina · 2015-07-02T00:26:51Z

v7/wordpress_compiler/wordpress/__init__.py

+__all__ = ['CompileWordpress', 'WordPressPlugin']
+
+
+class WordPressPlugin(IPlugin):


Why not make this a PageCompiler? Look at https://github.com/getnikola/nikola/blob/master/nikola/plugins/compile/pandoc.py#L43 for an example.

This is a class which shortcode plugins for the WordPress compiler should derive from. This is not the class for the WordPress page compiler itself. The WordPress page compiler is CompileWordpress and defined in wordpress.py.

In any case, inherit at least nikola.plugin_categories.BasePlugin or you will be missing stuff.

Why? The plugin handling for .wpplugin plugins (which inherit from WordPressPlugin) is done completely by the WordPress page compiler plugin. Nikola won't see these plugins.

Oh, it will. Nikola's plugin loader will find them, unless you put them in a different place? Then they will be loaded twice.
Also, there's no reason for the compiler to load plugins, just let Nikola load them and get them from self.site.plugin_manager. Saves you a chunk of code, too.

How about a "plugin plugin" category? The base class for that category would have a field for the plugin name this plugin is for, and the Nikola main object would get a function which allows to retrieve all plugins for a given plugin.
I can add something like that to Nikola if you want. The problem is that this won't be part of Nikola anytime soon (7.6.0 was just released), and so the WordPress plugin won't be useable until a newer Nikola version is released -- and in particular, it won't be useable for almost all v7 versions.

Re registering plugin categories from within plugins: that would be really cumbersome, since we have to specify the plugin categories before any plugin is loaded. Except of course if we load plugins twice, but I don't really like that idea.

class CompilerExtension(BasePlugin): compiler_name = '' class RestExtension(CompilerExtension): compiler_name = 'rest' class MarkdownExtension(CompilerExtension) compiler_name = 'markdown'

Your plugins would be CompilerExtensions with compiler_name = 'wordpress', and would filter for that.

We could make your plugin v8-only (and we really need to get our act together and land that release) to make compatibility easier.

Yes, that's what I was thinking. Also very good idea to include RestExtension and MarkdownExtension into this scheme!

Apropos v8: is there a timeline when v8 should be done? For the earlytask branch, the main problem is that it is waiting for some features to appear in official doit before it an be merged.

Well, it will take a while for v8 to become a thing, because there isn’t that much to change yet. However, I just implemented CompilerExtensions in getnikola/nikola@c65be6e. You could make your compiler v7.6.1+ and use them.

ralsina · 2015-07-02T13:52:48Z

Review status: 0 of 10 files reviewed at latest revision, 5 unresolved discussions, all commit checks successful.

^{v7/wordpress_compiler/README.md, line 12 [r1] (raw file):}
If you put this in conf.py.sample it will be automatically shown here, and also shown as a suggestion to the user when he installs the plugin. For example here's what happens when you install markmin:

$ nikola plugin -i markmin
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): plugins.getnikola.com
[2015-07-02T13:49:30Z] INFO: plugin: Downloading: https://plugins.getnikola.com/v7/markmin.zip
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): plugins.getnikola.com
[2015-07-02T13:49:31Z] INFO: plugin: Extracting: markmin into plugins/
[2015-07-02T13:49:31Z] NOTICE: plugin: This plugin has a sample config file.  Integrate it with yours in order to make this plugin work!
Contents of the conf.py.sample file:

    # Add the markmin compiler to your COMPILERS dict.
    COMPILERS["markmin"] = ('.mm',)

    # Add markmin files to your POSTS, PAGES
    POSTS = POSTS + (("posts/*.mm", "posts", "post.tmpl"),)
    PAGES = PAGES + (("stories/*.mm", "posts", "post.tmpl"),)

^{v7/wordpress_compiler/wordpress/plugins/code.py, line 3 [r1] (raw file):}
Since you wrote it, you shoud use your name in it :-)

^{v7/wordpress_compiler/wordpress/plugins/code.wpplugin, line 2 [r1] (raw file):}
this name is waaaaay too generic and bound to conflict with something.

Comments from the review on Reviewable.io

felixfontein · 2015-07-02T21:05:30Z

Since you wrote it, you shoud use your name in it :-)

Well, I patched it together, but I took some parts from other people -- like you ;-)

this name is waaaaay too generic and bound to conflict with something.

Isn't this always local to the .wpplugin file?

ralsina · 2015-07-02T21:07:59Z

I try to keep the names consistent, same name in the metadata plugin file, the plugin itself, task names, etc. You never know what someone is going to try in the future :-)

ralsina · 2015-07-03T15:39:37Z

Ok, let's merge this whenever you want, and let's revisit it in a few months.

felixfontein · 2015-07-03T20:44:04Z

I'll add an remark for potential shortcode plugin authors that the plugin interface will be changed for v8, so they should be aware that their plugins have to be changed a bit.
I'll probably manage that by tomorrow.

felixfontein · 2015-07-05T12:25:36Z

The new version now uses the CompilerExtension interface of Nikola 7.6.1.

felixfontein · 2015-07-05T12:46:12Z

I guess the tests will fail until Nikola 7.6.1 is released.

Kwpolska · 2015-07-05T12:48:01Z

v7/wordpress_compiler/wordpress/__init__.py

+from .wordpress import CompileWordpress
+
+
+__all__ = ['CompileWordpress', 'WordPressPlugin']


should be a tuple, not a list

where is WordPressPlugin?

Thanks! I moved WordPressPlugin to plugin_interface, otherwise yapsy would (at least for me) load that definition as the plugin and ignore the page compiler...

Kwpolska · 2015-07-05T12:52:47Z

Replace by the contributors with , Felix Fontein.
Add a # -*- coding: utf-8 -*- declaration.
To fix the tests, you must duplicate even more code, the plugin test suite uses its own base.py file.
Get rid of pickles
Get rid of UnformattedLexer

felixfontein · 2015-07-05T13:13:45Z

Ok, there's a big problem with the new CompilerExtensions interface. How do I now import something from the original WordPress plugin? When it was using its own plugin system, I would just write import wordpress.xxx and everything would work fine. Now, I can't anymore.

Kwpolska · 2015-07-05T13:14:10Z

v7/wordpress_compiler/wordpress/plugins/wordpress_shortcode_code.py

+import pygments.token
+
+
+class UnformattedLexer(pygments.lexer.RegexLexer):


Do you really need that? There is a text lexer available.

Must have missed that one.

Kwpolska · 2015-07-05T13:17:30Z

You could cheat and use import plugins.wordpress.plugin_interface, though this will make user-wide plugin installs impossible.

felixfontein · 2015-07-05T13:26:17Z

Doesn't work for me, either, since I'm including the wordpress plugin via EXTRA_PLUGINS_DIRS.
I'll Maybe I can expose the modules dynamically in the register method. Will be somewhat ugly, but should work. I'll play with that later...

felixfontein · 2015-07-05T13:31:19Z

Hmm, that won't work either, since the first thing I need to do is import wordpress.plugin_interface to be able to use wordpress.plugin_interface.WordPressPlugin. That sucks bigtime.

Kwpolska · 2015-07-05T13:42:25Z

Hack idea 2: use the plain CompilerExtension class and set compiler_name = 'wordpress'.

felixfontein · 2015-07-05T13:43:32Z

And define register() myself, and never be able to define any helper functionality in the plugin object itself. Very, very hacky. But probably the only thing that works... :(

Kwpolska · 2015-07-05T13:53:45Z

You could write some functions (≠methods) and pass them to every extension in your compiler, when you register them.

felixfontein · 2015-07-05T14:04:42Z

I'm now providing the modules. I'll do some more testing by porting my other plugins to this way...

felixfontein · 2015-07-05T14:15:04Z

Seems to work this way.

Kwpolska · 2015-07-05T14:33:49Z

Please complete the checklist a few posts above.

felixfontein · 2015-07-05T14:40:37Z

I now also adjusted the copyright lines (though I didn't do precisely the replacement you wanted to do). Is that ok?

felixfontein · 2015-07-05T14:44:00Z

Yeah, just noticed that too :)

WordPress compiler plugin.

felixfontein · 2015-07-05T18:36:46Z

Thanks!

felixfontein added 3 commits June 27, 2015 14:54

Added WordPress compiler plugin.

74781ba

Removed one flake8 warning.

e5cca7f

Improved some spacing.

Making flake8 completely happy.

fde7993

ralsina reviewed Jul 2, 2015
View reviewed changes

Added conf.py.sample, and also mentioning POSTS and PAGES in README.md.

030396b

Using new Nikola 7.6.1 (not released yet) compiler extension interface.

fdb84cc

Changed exception (thanks to Kwpolska!).

8912105

Kwpolska reviewed Jul 5, 2015
View reviewed changes

felixfontein added 3 commits July 5, 2015 14:54

Fixed __all__.

30eeea4

Fixed two little things.

960777f

Using utf-8 marker.

54d81a4

Kwpolska reviewed Jul 5, 2015
View reviewed changes

Now using JSON for serializing dependency data.

fe430ae

Using TextLexer.

08e23a5

Removed superfluous import.

fbb24e8

felixfontein added 4 commits July 5, 2015 16:02

Fixed text lexer usage.

d13b3e4

Adding more logging output.

585df00

Providing wordpress modules on register() call.

36ae511

Directly using Nikola's CompilerExtension.

4c29f33

Added name for plugin.

7221926

felixfontein added 2 commits July 5, 2015 16:36

More code duplication (from nikola/tests/base.py).

44f7c53

Adjusted copyright lines.

d2057a4

Forgot one line.

dfbfebf

Kwpolska added a commit that referenced this pull request Jul 5, 2015

Merge pull request #91 from felixfontein/master

ea80c63

WordPress compiler plugin.

Kwpolska merged commit ea80c63 into getnikola:master Jul 5, 2015

		__all__ = ['CompileWordpress', 'WordPressPlugin']


		class WordPressPlugin(IPlugin):

		from .wordpress import CompileWordpress


		__all__ = ['CompileWordpress', 'WordPressPlugin']

		import pygments.token


		class UnformattedLexer(pygments.lexer.RegexLexer):

WordPress compiler plugin. #91

WordPress compiler plugin. #91

Conversation

felixfontein commented Jun 27, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ralsina commented Jul 2, 2015

felixfontein commented Jul 2, 2015

ralsina commented Jul 2, 2015

ralsina commented Jul 3, 2015

felixfontein commented Jul 3, 2015

felixfontein commented Jul 5, 2015

felixfontein commented Jul 5, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kwpolska commented Jul 5, 2015

felixfontein commented Jul 5, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kwpolska commented Jul 5, 2015

felixfontein commented Jul 5, 2015

felixfontein commented Jul 5, 2015

Kwpolska commented Jul 5, 2015

felixfontein commented Jul 5, 2015

Kwpolska commented Jul 5, 2015

felixfontein commented Jul 5, 2015

felixfontein commented Jul 5, 2015

Kwpolska commented Jul 5, 2015

felixfontein commented Jul 5, 2015

felixfontein commented Jul 5, 2015

felixfontein commented Jul 5, 2015