Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Item13822: compatibility with legacy Foswiki
also:

   *  fixed backwards compatibility with pre-unicode Foswiki
   *  bring back =solr::queryfields= in SolrSearchBaseTemplate
   *  fixed language facet to properly match language tags to their name
   *  improved layout of search results as well as autosuggestion widget
   *  removed workflow facet from default search
   *  fixed icon mapping for topics that don't come with an icon defined in their TopicType
   *  don't try to encode html entities without a code point in utf8
   *  don't remove all macros from topic text, just some
   *  removed dependency on MimeIconsPlugin as we are using fontawesome now
   *  improved formula for sorting results by reference
   *  fixed sorting in ajax-solr
   *  fixed exposing/hiding parameters in ajax-solr
   *  improved findability of content; i.e. when containing stop words only in the title
   *  removed unused =/browse= search handler from solr config
  • Loading branch information
MichaelDaum committed Oct 18, 2015
1 parent e7d936c commit 573f0bc
Show file tree
Hide file tree
Showing 22 changed files with 339 additions and 293 deletions.
28 changes: 21 additions & 7 deletions data/System/SolrPlugin.txt
@@ -1,4 +1,4 @@
%META:TOPICINFO{author="ProjectContributor" date="1443691651" format="1.1" version="1"}%
%META:TOPICINFO{author="ProjectContributor" date="1445162450" format="1.1" version="1"}%
---+ Solr Plugin
%FORMFIELD{"Description"}%

Expand Down Expand Up @@ -34,9 +34,8 @@ The current plugin requires *Solr 5.0.0* or later. Download it from the [[http:
%JQREQUIRE{"chili"}%

<verbatim class="bash">
tar xzf solr-5.x.x.tgz solr-5.x.x/bin/install_solr_service.sh
cd solr-5.x.x/bin/
./install_solr_service.sh ../../solr-5.x.x.tgz
tar xzf solr-5.x.x.tgz solr-5.x.x/bin/install_solr_service.sh --strip-components 2
./install_solr_service.sh ./solr-5.x.x.tgz
service solr stop
</verbatim>

Expand Down Expand Up @@ -138,7 +137,7 @@ cd <foswiki-dir>/tools
For instance to empty your index completely use:

<verbatim class="bash">
./soldelete *:*
./solrdelete *:*
</verbatim>

---+++ solrjob
Expand Down Expand Up @@ -762,6 +761,7 @@ Similarly topic preferences are indexed using a =preference_*= prefix.
| *_d | tdouble | | stored | fields with a _f suffix are indexed as a double precision float |
| *_b | boolean | | stored | true, false |
| *_s | string | | stored | dynamic field for unanalyzed text |
| *_std | string | not stored | dynamic field for standard analysis, i.e. stopwords not being removed |
| *_t | text_generic | | stored | generic text |
| *_dt | tdate | | stored | a dateTime value |
| *_lst | string | multivalued | stored | this field is used for any multi-valued formfield in !DataForms like, select, radio, checkbox, textboxlist |
Expand Down Expand Up @@ -847,6 +847,20 @@ specific search applications. The destination fields are then analysed using the

---++ Change History
%TABLE{columnwidths="7em" tablewidth="100%"}%
| 18 Oct 2015: | fixed backwards compatibility with pre-unicode Foswiki; \
bring back =solr::queryfields= in <nop>SolrSearchBaseTemplate; \
fixed language facet to properly match language tags to their name; \
improved layout of search results as well as autosuggestion widget; \
removed workflow facet from default search; \
fixed icon mapping for topics that don't come with an icon defined in their <nop>TopicType; \
don't try to encode html entities without a code point in utf8; \
don't remove all macros from topic text, just some; \
removed dependency on <nop>MimeIconsPlugin as we are using fontawesome now; \
improved formula for sorting results by reference; \
fixed sorting in ajax-solr; \
fixed exposing/hiding parameters in ajax-solr; \
improved findability of content; i.e. when containing stop words only in the title; \
removed unused =/browse= search handler from solr config |
| 01 Oct 2015: | improve default layout of search results; moved unsafe inline-javascript into a js file of its own |
| 21 Sep 2015: | cache stringified attachments using Cache::FileCache now and added api to purge/clear cache regularly; \
removed =IndexExtensions= config parameter to let the stringifier decide on supported file formats; \
Expand Down Expand Up @@ -933,8 +947,8 @@ specific search applications. The destination fields are then analysed using the
| 22 Nov 2010: | fixes integration with pattern skin |
| 18 Nov 2010: | initial public release |

%META:FILEATTACHMENT{name="SolrPluginSnap1.png" attr="" comment="" date="1443691649" size="93552" user="ProjectContributor" version="1"}%
%META:FILEATTACHMENT{name="SolrPluginSnap2.png" attr="" comment="" date="1443691649" size="158013" user="ProjectContributor" version="1"}%
%META:FILEATTACHMENT{name="SolrPluginSnap1.png" attr="" comment="" date="1445162454" size="93552" user="ProjectContributor" version="1"}%
%META:FILEATTACHMENT{name="SolrPluginSnap2.png" attr="" comment="" date="1445162454" size="158013" user="ProjectContributor" version="1"}%
%META:FORM{name="PackageForm"}%
%META:FIELD{name="Author" title="Author" value="Michael Daum"}%
%META:FIELD{name="Copyright" title="Copyright" value="&copy; 2009-2015, Michael Daum http://michaeldaumconsulting.com"}%
Expand Down
86 changes: 50 additions & 36 deletions data/System/SolrSearchBaseTemplate.txt
Expand Up @@ -18,26 +18,18 @@
%TMPL:DEF{"solr::defaultsort"}%score desc%TMPL:END%
%TMPL:DEF{"solr::dateformat"}%dddd, Do MMMM YYYY, HH:mm%TMPL:END%
%TMPL:DEF{"solr::fields"}%%TMPL:END%
%TMPL:DEF{"solr::extrafilter"}%%IF{
"defined 'SOLR_EXTRAFILTER'"
then="%SOLR_EXTRAFILTER%"
}% %IF{
"not $USERNAME ingroup 'AdminGroup'"
then="-web_search:Applications"
}%%TMPL:END%
%TMPL:DEF{"solr::queryfields"}%%TMPL:END%
%TMPL:DEF{"solr::extrafilter"}%%SOLR_EXTRAFILTER{default=""}% %IF{ "not $USERNAME ingroup 'AdminGroup'" then="-web_search:Applications" }%%TMPL:END%
%TMPL:DEF{"solr::facet::include::web"}%%TMPL:END%
%TMPL:DEF{"solr::facet::exclude::web"}%^(_.*%IF{
"not $USERNAME ingroup 'AdminGroup'"
then="|Application.*"
}%)$%TMPL:END%
%TMPL:DEF{"solr::facet::exclude::web"}%^(_.*%IF{ "not $USERNAME ingroup 'AdminGroup'" then="|Application.*" }%)$%TMPL:END%
%{</verbatim>}%

%{<verbatim class="tml">}%
%{ ###########################################################################
content: main definition that puts together all pieces
}%
%TMPL:DEF{"content"}%<noautolink>
<div id='solrSearch' style='display:none' data-more-fields='%TMPL:P{"solr::fields"}%' data-solr-url='%SCRIPTURL{"rest"}%/SolrPlugin/proxy' data-solr-params='{"topic": "%WEB%.%TOPIC%", "qt":"%TMPL:P{"solr::defaultquerytype"}%", "rows": %TMPL:P{"solr::defaultrows"}%}' data-extra-filter='%TMPL:P{"solr::extrafilter"}%'>
<div id='solrSearch' style='display:none' data-more-fields='%TMPL:P{"solr::fields"}%' %FORMATLIST{"%TMPL:P{"solr::queryfields"}%" header="data-query-fields='" footer="'" separator=" " split=" "}% data-solr-url='%SCRIPTURL{"rest"}%/SolrPlugin/proxy' data-solr-params='{"topic": "%WEB%.%TOPIC%", "qt":"%TMPL:P{"solr::defaultquerytype"}%", "rows": %TMPL:P{"solr::defaultrows"}%}' data-extra-filter='%TMPL:P{"solr::extrafilter"}%'>
%TMPL:P{"solr::title"}%<!-- -->
%TMPL:P{"solr::header"}%<!-- -->
%TMPL:P{"solr::hits"}%<!-- -->
Expand Down Expand Up @@ -91,13 +83,43 @@
"language": "%MAKETEXT{"language"}%",
"keyword": "%MAKETEXT{"keyword"}%",
"tag": "%MAKETEXT{"tag"}%",
"webcat": "%MAKETEXT{"Category"}%"%IF{"context MultiLingualPluginEnabled"
then="$percntLANGUAGES{
header=\",$n\"
format=\" \\"$code\\":\\"$language\\"\"
separator=\",$n\"
}$percnt"
}%
"webcat": "%MAKETEXT{"Category"}%",
"ar":"Arabic",
"bg":"Bulgarian",
"ca":"Catalan",
"ckb":"Kurdish",
"cjk":"Japanese",
"cz":"Czech",
"da":"Danish",
"de":"German",
"el":"Greek",
"en":"English",
"es":"Spanish",
"fa":"Persian",
"fi":"Finnish",
"fr":"French",
"ga":"Irish",
"gl":"Galician",
"hi":"Hindi",
"hu":"Hungarian",
"hy":"Armenian",
"id":"Indonesian",
"it":"Italian",
"ja":"Japanese",
"ko":"Korean",
"lv":"Latvian",
"nl":"Dutch",
"no":"Norwegian",
"pl":"Polish",
"pt-br":"Portuguese",
"pt":"Portuguese",
"ro":"Romanian",
"ru":"Russian",
"sv":"Swedish",
"tr":"Turkish",
"uk":"Ukrainian",
"zh-cn":"Chinese",
"zh-tw":"Chinese"
}
</div>
</literal>%TMPL:END%
Expand Down Expand Up @@ -151,9 +173,8 @@
%{<verbatim class="tml">}%
%TMPL:DEF{"solr::templates::hits::ClassifiedTopic"}%<script id="solrHitTemplate_ClassifiedTopic" type="text/x-jsrender">
<div class='solrSearchHit solrTopicHit clearfix'>
%TMPL:P{"solr::thumbnail"}%
<h3>
<img src='{{if icon}}{{:icon}}{{else}}%PUBURLPATH%/%SYSTEMWEB%/FamFamFamSilkIcons/page_white.png{{/if}}' width='16' height='16' class='solrHitIcon' />
<i class='solrHitIcon foswikiIcon jqIcon fa {{if icon}}{{:icon}}{{else}}fa-file-o{{/if}}'></i>
<a href='{{:url}}'>{{:title}}</a>
<span class=' solrContainerLink foswikiGrayText foswikiNormal'>
%MAKETEXT{"in [_1]" args="<a href='{{:container_url}}'>{{:container_title}}</a>"}%
Expand All @@ -162,6 +183,7 @@
<div class='solrRevision'>
{{:~formatDate(date,'%TMPL:P{"solr::dateformat"}%')}}, {{:author}}
</div>
%TMPL:P{"solr::thumbnail"}%
<div class='solrHilite'>
{{:~getHilite(id)}}
</div>
Expand All @@ -176,9 +198,8 @@
%{<verbatim class="tml">}%
%TMPL:DEF{"solr::templates::hits::topic"}%<script id="solrHitTemplate_topic" type="text/x-jsrender">
<div class='solrSearchHit solrTopicHit clearfix'>
%TMPL:P{"solr::thumbnail"}%
<h3>
<img src='{{if icon}}{{:icon}}{{else}}%PUBURLPATH%/%SYSTEMWEB%/FamFamFamSilkIcons/page_white.png{{/if}}' width='16' height='16' class='solrHitIcon' />
<i class='solrHitIcon foswikiIcon jqIcon fa {{if icon}}{{:icon}}{{else}}fa-file-o{{/if}}'></i>
<a href='{{:url}}'>{{:title}}</a>
<span class='foswikiGrayText foswikiNormal solrContainerLink'>
%MAKETEXT{"in [_1]" args="<a href='{{:container_url}}'>{{:container_title}}</a>"}%
Expand All @@ -187,6 +208,7 @@
<div class='solrRevision'>
{{:~formatDate(date,'%TMPL:P{"solr::dateformat"}%')}}, {{:author}}
</div>
%TMPL:P{"solr::thumbnail"}%
<div class='solrHilite'>
{{:~getHilite(id)}}
</div>
Expand All @@ -203,7 +225,7 @@
</a>
</div>
<h3>
<img src='{{if icon}}{{:icon}}{{else}}%PUBURLPATH%/%SYSTEMWEB%/FamFamFamSilkIcons/page_white.png{{/if}}' width='16' height='16' class='solrHitIcon' />
<i class='solrHitIcon foswikiIcon jqIcon fa {{if icon}}{{:icon}}{{else}}fa-file-o{{/if}}'></i>
<a href='{{:url}}'>{{:name}}</a>
<span class='foswikiGrayText foswikiNormal solrContainerLink'>
%MAKETEXT{"in [_1]" args="<a href='{{:container_url}}'>{{:container_title}}</a>"}%
Expand All @@ -220,7 +242,7 @@
%TMPL:DEF{"solr::templates::hits::comment"}%<script id="solrHitTemplate_comment" type="text/x-jsrender">
<div class='solrSearchHit solrCommentHit'>
<h3>
<img src='%PUBURLPATH%/%SYSTEMWEB%/FamFamFamSilkIcons/comment.png' width='16' height='16' class='solrHitIcon' />
<i class='solrHitIcon foswikiIcon jqIcon fa {{if icon}}{{:icon}}{{else}}fa-file-o{{/if}}'></i>
<a href='{{:url}}'>{{:title}}</a>
<span class='foswikiGrayText foswikiNormal solrContainerLink'>
%MAKETEXT{"in [_1]" args="<a href='{{:container_url}}'>{{:container_title}}</a>"}%
Expand All @@ -240,7 +262,7 @@
%TMPL:DEF{"solr::templates::hits::misc"}%<script id="solrHitTemplate_misc" type="text/x-jsrender">
<div class='solrSearchHit solrAttachmentHit'>
<h3>
<img src='{{if icon}}{{:icon}}{{else}}%PUBURLPATH%/%SYSTEMWEB%/FamFamFamSilkIcons/page_white.png{{/if}}' width='16' height='16' class='solrHitIcon' />
<i class='solrHitIcon foswikiIcon jqIcon fa {{if icon}}{{:icon}}{{else}}fa-file-o{{/if}}'></i>
<a href='{{:url}}'>{{:name}}</a>
<span class='foswikiGrayText foswikiNormal solrContainerLink' >
%MAKETEXT{"in [_1]" args="<a href='{{:container_url}}'>{{:container_title}}</a>"}%
Expand Down Expand Up @@ -311,7 +333,7 @@

%{<verbatim class="tml">}%
%TMPL:DEF{"solr::templates::hierarchy"}%<script id="solrHierarchyTemplate" type="text/x-jsrender">
<li><img src='{{:icon}}' width='16' height='16' /> <a href='#' class='solrFacetFalue cat_{{:id}}' data-value='{{:id}}'>{{:title}}</a> {{:~renderFacetCount(id)}}
<li><i class='solrHitIcon foswikiIcon jqIcon fa {{:icon}}'></i><a href='#' class='solrFacetFalue cat_{{:id}}' data-value='{{:id}}'>{{:title}}</a> {{:~renderFacetCount(id)}}
<div class='solrHierarchyChildren'>
{{for ~getChildren}}
{{if index < 5}}
Expand Down Expand Up @@ -494,18 +516,10 @@
}%
%TMPL:P{"solr::facet"
id="field_TopicType_lst"
exclude="ApplicationLicense|ApplicationTopic|DataForm|DocuTopic|TopicFunction|TopicStub|TopicTemplate|TopicType|TopicView|WebTool|WikiApplication|WikiTopic|TaggedTopic|CategorizedTopic"
exclude="ApplicationLicense|ApplicationTopic|DataForm|DocuTopic|TopicStub|TopicTemplate|WebTool|WikiApplication|WikiTopic|TaggedTopic|CategorizedTopic"
sort="title"
title="%MAKETEXT{"Topic type"}%"
}%
%TMPL:P{"solr::facet"
id="state"
title="%MAKETEXT{"Workflow State"}%"
}%
%TMPL:P{"solr::facet"
id="preference_WORKFLOW_s"
title="%MAKETEXT{"Workflow"}%"
}%
%TMPL:P{"solr::facet"
id="language"
title="%MAKETEXT{"Language"}%"
Expand Down
4 changes: 2 additions & 2 deletions data/System/SolrSideBar.txt
@@ -1,4 +1,4 @@
%META:TOPICINFO{author="ProjectContributor" comment="" date="1442835433" format="1.1" version="1"}%
%META:TOPICINFO{author="daum" comment="" date="1445009153" format="1.1" version="4"}%
---+!! %TOPIC%

This is a set of widgets to be used in a sidebar navigation. This is used like this:
Expand Down Expand Up @@ -36,7 +36,7 @@ part of the %SYSTEMWEB%.SolrPlugin documentation.
<verbatim class="tml">
%STARTSECTION{"solrsimilar"}%<!-- -->
%SOLRSIMILAR{"id:%BASEWEB%.%BASETOPIC%"
filter="web:%BASEWEB% type:topic %IF{"defined FILTER" then="%FILTER%"}% -preference_BASETRANSLATION_s:%BASETOPIC% -webtopic:%BASEWEB%.%BASETRANSLATION%"
filter="web:%BASEWEB% type:topic %IF{"defined FILTER" then="%FILTER%"}% -preference_BASETRANSLATION_s:%BASETOPIC% -webtopic:%BASEWEB%.%BASETRANSLATION{default=""}%"
like="%SOLRSIMILAR_LIKE{default="field_Category_flat_lst^5,tag"}%"
fields="web,topic,title,score,summary"
rows="5"
Expand Down
4 changes: 2 additions & 2 deletions lib/Foswiki/Plugins/SolrPlugin.pm
Expand Up @@ -30,8 +30,8 @@ BEGIN {
}
}

our $VERSION = '4.20';
our $RELEASE = '01 Oct 2015';
our $VERSION = '4.30';
our $RELEASE = '18 Oct 2015';
our $SHORTDESCRIPTION = 'Enterprise Search Engine for Foswiki based on [[http://lucene.apache.org/solr/][Solr]]';
our $NO_PREFS_IN_TOPIC = 1;
our %searcher;
Expand Down

0 comments on commit 573f0bc

Please sign in to comment.