Giter Club home page Giter Club logo

metalsmith-lunr's Introduction

metalsmith-lunr

A Metalsmith plugin that integrates the Lunr.js client side search engine.

Builds a searchable JSON index based on Metalsmith metadata.

Installation

$ npm install metalsmith-lunr

Usage

Include lunr: true in file metadata to include it in the search index.

---
lunr: true
title: My Article
tags: maybe some tags for indexing
---

My article contents...

Metalsmith-lunr can be used without options:

var lunr = require('metalsmith-lunr');

metalsmith.use(lunr()).

Use file metadata as fields for the search and assign weight for each field. The content field refers Metalsmith's internal record of the files contents and should not be included in the file metadata.

var lunr = require('metalsmith-lunr');
var lunr_ = require('lunr');
require('lunr-languages/lunr.stemmer.support')(lunr_);
require('lunr-languages/lunr.no')(lunr_);

metalsmith.use(lunr({
  ref: 'title',
  indexPath: 'index.json',
  fields: {
      contents: 1,
      tags: 10
  },
  pipelineFunctions: [
    lunr_.trimmer,
    lunr_.no.stopWordFilter,
    lunr_.no.stemmer
  ],
  preprocess: function(content) {
    // Replace all occurrences of __title__ with the current file's title metadata.
    return content.replace(/__title__/g, this.title);
  }
}));

Optional Parameters

  • fields: {metadata search field: search weight}
  • ref: metadata search reference for document
  • indexPath: path for JSON index file
  • pipelineFunctions: [lunr pipeline functions] Functions will be called in order by lunr, see lunr doc for more information.
  • preprocess: a callback function that can pre-process the content of each file before it is indexed. (For example stripping HTML tags). This will not affect the content of the files themselves. The callback is passed the content as a string to it's first argument. The metadata (including the raw content buffer) can be access with this. The callback must return a string.

Default Parameter Values

  • fields: {contents: 1}
  • ref: filePath
  • indexPath: searchIndex.json

##Client Side Search

Metalsmith-lunr will generate searchIndex.json. Include lunr.js in your javascript source files. Client side search example can be found here.

Once the JSON file has been parsed into javascript, simply run the following:

//index is the parsed JSON file
idx = lunr.Index.load(index)
var results = idx.search("Your Search Terms Here");

CLI Usage

{
  "plugins": {
    "metalsmith-lunr": {
      "fields": {
        "tags": 10,
        "contents": 1
      }
    }
  }
}

Tests

npm test

License

MIT

metalsmith-lunr's People

Contributors

arve0 avatar cmclay avatar sukima avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

metalsmith-lunr's Issues

Proposal for adoption by Metalsmith org

Hi there I would like to propose a repository & NPM transfer of this repo to the @metalsmith org.
This plugin would be refactored with a common setup for core plugins & renamed to @metalsmith/search-index.

Your name would remain in the LICENSE file, and a new line would be added in front (c) webketje.
I would add you as a member to the metalsmith org (temporarily) so you can initiate a Transfer of the repo.
You would need to deprecate the NPM package instructing users to use @metalsmith/search-index instead.

If you do not further maintain this package, or respond to this request in a month's time, I will fork the project and move ahead as I did with @metalsmith/postcss

Adding more properties to store items

Nice plugin! Is it possible to add multiples refs in the store items (or something along those lines)? For example, if you index by 'filePath', and then use lunr.js to search through searchIndex.json, how would you grab the title? Is there a way to make 'title' and 'filePath' easily accessible at the same time, using the same json file? I'm trying to use this to make a search-suggest sort of thing. Any tips appreciated. Thanks in advance.

Uncaught TypeError: Cannot read property 'tf' of undefined

I'm not sure if this is a bug for you, or for Lunr, but seeing as it has to do with the index, and this is what I'm using to build the index, I thought I would start here.

So I'm getting the error Uncaught TypeError: Cannot read property 'tf' of undefined. It seems to have to do with parsing vector similarities. The code seems to be looking for tf on a document in the tokenStore.
https://cl.ly/1B3f1m0B391b

My lunr configs look like:

      metalsmithLunr({
        ref: 'id',
        indexPath: 'api/search-index.json',
        fields: {
          headline: 5,
          subheadline: 3,
          category: 5,
          tags: 3,
          contents: 1,
        },
        preprocess: function(content) {
          const tr = (str) => {
            const map = {"а":"a","б":"b",/*  truncated for issue  */"І":"I","і":"i"};
            let new_str = "", char, substitute, n = str.length;
            for(let i = 0; i < n; i++) {
                char = str[i];
                substitute = map[char];
                new_str += substitute ? substitute : char;
            }
            return new_str;
          };
          return tr(
            content.replace(/<[^>]+>/g, ' ') // Strip HTML
          ) // Transliterate foreign characters
            .replace(/[^\w]/g, ' ') // Strip Punctuation
          ;
        },
      }),

And the client side code looks like:

    let ip = new Promise((resolve, reject) => {
      $.getJSON('/api/search-index.json', data => {
        console.log('SEARCH DATA', data);
        resolve(lunr.Index.load(data));
      });
    });

and then later:

      if (keywords) {
        SearchService.results = SearchService.index.search(keywords);
      }

I can provide the index data or the serialized index object if you would like.

editing lunr pipeline

It would be nice if one could edit the lunr pipeline through the options object. E.g., to support stemming of other languages than english.

Does not work with new version of Lunr

I tried using this plugin with v2.0.0 of Lunr and I am getting this error :

my_static_site_ blog-_google_chrome_2017-04-10_10-38-22

Once I switched back to an older version : 0.5.12. It seemed to work properly.

How to strip html tags before generating searchIndex.json?

This is my lunr snippet from the build file:

.use(lunr({
        preprocess: function(content) {
        // Replace all occurrences of __title__ with the current file's title metadata.
        return content.replace(/__title__/g, this.title);
        }
 }))

How do I strip HTML tags ??

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.