Giter Club home page Giter Club logo

elasticsearch-analysis-kuromoji-ipadic-neologd's Introduction

Elasticsearch Analysis Kuromoji IPADic Neologd

Overview

Elasticsearch Analysis Neologd Plugin provides Tokenizer/CharFilter/TokenFilter for Kuromoji with Neologd.

Version

Versions in Maven Repository Old(<= 6.5.1) Repository

Issues/Questions

Please file an issue. (Japanese forum is here.)

Installation

$ $ES_HOME/bin/elasticsearch-plugin install org.codelibs:elasticsearch-analysis-kuromoji-ipadic-neologd:7.1.0

References

Analyzer, Tokenizer, TokenFilter, CharFilter

The plugin includes these analyzer and tokenizer, tokenfilter.

name type
kuromoji_ipadic_neologd_iteration_mark charfilter
kuromoji_ipadic_neologd analyzer
kuromoji_ipadic_neologd_tokenizer tokenizer
kuromoji_ipadic_neologd_baseform tokenfilter
kuromoji_ipadic_neologd_part_of_speech tokenfilter
kuromoji_ipadic_neologd_readingform tokenfilter
kuromoji_ipadic_neologd_stemmer tokenfilter

Usage

See Elasticsearch Kuromoji.

Update Kuromoji Jar File

If you want to replace with the latest Lucene Neologd jar file, download it from https://maven.codelibs.org/org/codelibs/lucene-analyzers-kuromoji-ipadic-neologd/ and then replace old file in $ES_HOME/plugins/analysis-kuromoji-ipadic-neologd.

What is NEologd

See mecab-ipadic-NEologd.

Use Lucene Kuromoji for Neologd

If you want to use Lucene Kuromoji for Neologd in your application other than elasticsearch, you can use lucene-analyzers-kuromoji-ipadic-neologd jar file, not this plugin. To use the jar file, put the following settings into your pom.xml.

...
<repositories>
    <repository>
        <id>codelibs.org</id>
        <name>CodeLibs Repository</name>
        <url>https://maven.codelibs.org/</url>
    </repository>
</repositories>
...
<dependencies>
    <dependency>
        <groupId>org.codelibs</groupId>
        <artifactId>lucene-analyzers-kuromoji-ipadic-neologd</artifactId>
        <version>6.4.0-20180927</version>
        <!-- https://maven.codelibs.org/org/codelibs/lucene-analyzers-kuromoji-ipadic-neologd/ --->
    </dependency>
...

See CodeLibs Maven Repository.

elasticsearch-analysis-kuromoji-ipadic-neologd's People

Contributors

deka0106 avatar dependabot[bot] avatar keiichiw avatar marevol avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

s-wool cstoku ikeikeikeike daisy9516 mengyouhan dot-deamon takuti tool-recommender-bot wangkai2014 sugita-q takahi-i kato-ryo0 ryook iyah35 bibio kdl-albert bestjex johtani odd12258053 todesking mage95 krrrr38 r4-keisuke faithnh

elasticsearch-analysis-kuromoji-ipadic-neologd's Issues

Tokenization fails when some kind of katakana appeared followed by (株)

Hi,

Firstly, I want to tell you thank you for this great product.

I recently noticed that this plugin fails to tokenize text on some context.

For example, following texts are failing to tokenize.

  • (株)サイゼリヤ
  • (株)アクセス
  • (株)ライン
  • (株)ライト

It seems that the plugin fails when KATAKANA word appears just after (株) but not all KATAKANA phrase fails.
Would you like to check this issue?

Thanks in advance,

Reproducing Procedure

[Step 1] Create Dockerfile

FROM elasticsearch:6.5.1
  
RUN elasticsearch-plugin install -b org.codelibs:elasticsearch-analysis-kuromoji-neologd:6.5.1

[Step 2] Build docker image

docker build -t es6 .

[Step 3] Run elasticsearch

docker run -p 9200:9200 es6

[Step 4] Query with this text

% curl -s -H 'Content-Type:application/json' -XPOST http://localhost:9200/_analyze -d '{"tokenizer": "kuromoji_neologd_tokenizer", "text": "(株)サイゼリヤ", "explain": true}' | jq .
{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "kuromoji_neologd_tokenizer",
      "tokens": []
    },
    "tokenfilters": []
  }
}

Would you please release 6.3.2?

Hello.

I want to use elasticsearch-analysis-kuromoji-neologd with Elasticsearch 6.3.2
However, elasticsearch-analysis-kuromoji-neologd 6.3.2 has not been released.

Would you please release it?

Remove the intermediate "elasticsearch" directory within the plugin zip

I got a error on installing elasticsearch-analysis-kuromoji-neologd to 6.3 cluster.
As error message said, this error is caused by plugin directory structure.

./elasticsearch-plugin install org.codelibs:elasticsearch-analysis-kuromoji-neologd:6.3.0
-> Downloading org.codelibs:elasticsearch-analysis-kuromoji-neologd:6.3.0 from maven central
[=================================================] 100%   
Warning: sha512 not found, falling back to sha1. This behavior is deprecated and will be removed in a future release. Please update the plugin to use a sha512 checksum.
ERROR: This plugin was built with an older plugin structure. Contact the plugin author to remove the intermediate "elasticsearch" directory within the plugin zip.

ES2.2.1 start failed

elasticsearch cannot start by AccessControlException error.

java version "1.8.0_66"
MacOS10.11.3

[article] IndexCreationException[failed to create index]; nested: IllegalArgumentException[Failed to load fields.]; nested: AccessControlException[access denied ("java.lang.reflect.ReflectPermission" "suppressAccessChecks")];
    at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:360)
    at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyNewIndices(IndicesClusterStateService.java:294)
    at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:163)
    at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:600)
    at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:762)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Failed to load fields.
    at org.codelibs.elasticsearch.kuromoji.neologd.index.analysis.ReloadableKuromojiTokenizerFactory.<init>(ReloadableKuromojiTokenizerFactory.java:118)
    at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:50)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.FactoryProxy.get(FactoryProxy.java:54)
    at org.elasticsearch.common.inject.InjectorImpl$5$1.call(InjectorImpl.java:828)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:887)
    at org.elasticsearch.common.inject.InjectorImpl$5.get(InjectorImpl.java:823)
    at org.elasticsearch.common.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:236)
    at com.sun.proxy.$Proxy17.create(Unknown Source)
    at org.elasticsearch.index.analysis.AnalysisService.<init>(AnalysisService.java:91)
    at org.elasticsearch.index.analysis.AnalysisService.<init>(AnalysisService.java:66)
    at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:50)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:887)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:887)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:887)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:880)
    at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
    at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
    at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(InjectorImpl.java:159)
    at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(ModulesBuilder.java:55)
    at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:358)
    ... 9 more
Caused by: java.security.AccessControlException: access denied ("java.lang.reflect.ReflectPermission" "suppressAccessChecks")
    at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
    at java.security.AccessController.checkPermission(AccessController.java:884)
    at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
    at java.lang.reflect.AccessibleObject.setAccessible(AccessibleObject.java:128)
    at org.codelibs.elasticsearch.kuromoji.neologd.index.analysis.ReloadableKuromojiTokenizerFactory.<init>(ReloadableKuromojiTokenizerFactory.java:105)
    ... 61 more

Exception in thread "main" java.io.FileNotFoundException

Hi, there~

My Elasticsearch Version is 6.5.1，When I run the install command, I always get this error message.

sh-4.2# ./bin/elasticsearch-plugin install org.codelibs:elasticsearch-analysis-kuromoji-ipadic-neologd:6.5.1
-> Downloading org.codelibs:elasticsearch-analysis-kuromoji-ipadic-neologd:6.5.1 from maven central
Exception in thread "main" java.io.FileNotFoundException: https://repo1.maven.org/maven2/org/codelibs/elasticsearch-analysis-kuromoji-ipadic-neologd/6.5.1/elasticsearch-analysis-kuromoji-ipadic-neologd-6.5.1.zip
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1963)
        at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1958)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1957)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1525)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:245)
        at org.elasticsearch.plugins.InstallPluginCommand.downloadZip(InstallPluginCommand.java:380)
        at org.elasticsearch.plugins.InstallPluginCommand.downloadAndValidate(InstallPluginCommand.java:453)
        at org.elasticsearch.plugins.InstallPluginCommand.download(InstallPluginCommand.java:264)
        at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:229)
        at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:216)
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
        at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:77)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
        at org.elasticsearch.cli.Command.main(Command.java:90)
        at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)
Caused by: java.io.FileNotFoundException: https://repo1.maven.org/maven2/org/codelibs/elasticsearch-analysis-kuromoji-ipadic-neologd/6.5.1/elasticsearch-analysis-kuromoji-ipadic-neologd-6.5.1.zip
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1909)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3084)
        at java.base/java.net.URLConnection.getHeaderFieldLong(URLConnection.java:636)
        at java.base/java.net.URLConnection.getContentLengthLong(URLConnection.java:508)
        at java.base/java.net.URLConnection.getContentLength(URLConnection.java:492)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getContentLength(HttpsURLConnectionImpl.java:389)
        at org.elasticsearch.plugins.InstallPluginCommand.downloadZip(InstallPluginCommand.java:379)
        ... 10 more

What should I do?
Thanks in advance.

Test failed.

Is anyone have same issue as the test failed?
Maybe the code is too old need be updated?

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   PosConcatenationFilterFactoryTest.test_basic » MasterNotDiscovered
[ERROR]   PosConcatenationFilterFactoryTest.test_basic2 » MasterNotDiscovered
[INFO]
[ERROR] Tests run: 3, Failures: 0, Errors: 2, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------

java.lang.NoSuchMethodError: 'void org.elasticsearch.index.analysis.AbstractTokenizerFactory

Elastic search

Version: 7.13.1, Build: default/tar/9a7758028e4ea59bcab41c12004603c5a7dd84a9/2021-05-28T17:40:59.346932922Z, JVM: 16

java.lang.NoSuchMethodError: 'void org.elasticsearch.index.analysis.AbstractTokenizerFactory.<init>(org.elasticsearch.index.IndexSettings, org.elasticsearch.common.settings.Settings)'
	at org.codelibs.elasticsearch.kuromoji.ipadic.neologd.index.analysis.KuromojiTokenizerFactory.<init>(KuromojiTokenizerFactory.java:50) ~[?:?]
	at org.elasticsearch.index.analysis.AnalysisRegistry.buildMapping(AnalysisRegistry.java:433) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.index.analysis.AnalysisRegistry.buildTokenizerFactories(AnalysisRegistry.java:275) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.index.analysis.AnalysisRegistry.build(AnalysisRegistry.java:203) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.index.IndexModule.newIndexService(IndexModule.java:431) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.indices.IndicesService.createIndexService(IndicesService.java:663) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:566) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:170) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.indices.cluster.IndicesClusterStateService.createIndices(IndicesClusterStateService.java:468) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:228) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:499) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:489) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:460) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:407) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.cluster.service.ClusterApplierService.access$000(ClusterApplierService.java:57) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:151) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:241) ~[elasticsearch-7.13.1.jar:7.13.1]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:204) ~[elasticsearch-7.13.1.jar:7.13.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[?:?]
	at java.lang.Thread.run(Thread.java:831) [?:?]
fatal error in thread [elasticsearch[Mymacbook][clusterApplierService#updateTask][T#1]], exiting
java.lang.NoSuchMethodError: 'void org.elasticsearch.index.analysis.AbstractTokenizerFactory.<init>(org.elasticsearch.index.IndexSettings, org.elasticsearch.common.settings.Settings)'
	at org.codelibs.elasticsearch.kuromoji.ipadic.neologd.index.analysis.KuromojiTokenizerFactory.<init>(KuromojiTokenizerFactory.java:50)
	at org.elasticsearch.index.analysis.AnalysisRegistry.buildMapping(AnalysisRegistry.java:433)
	at org.elasticsearch.index.analysis.AnalysisRegistry.buildTokenizerFactories(AnalysisRegistry.java:275)
	at org.elasticsearch.index.analysis.AnalysisRegistry.build(AnalysisRegistry.java:203)
	at org.elasticsearch.index.IndexModule.newIndexService(IndexModule.java:431)
	at org.elasticsearch.indices.IndicesService.createIndexService(IndicesService.java:663)
	at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:566)
	at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:170)
	at org.elasticsearch.indices.cluster.IndicesClusterStateService.createIndices(IndicesClusterStateService.java:468)
	at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:228)
	at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:499)
	at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:489)
	at org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:460)
	at org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:407)
	at org.elasticsearch.cluster.service.ClusterApplierService.access$000(ClusterApplierService.java:57)
	at org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:151)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673)
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:241)
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:204)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:831)

elasticsearch-analysis-kuromoji-neologd 7.7.x release schedule

Hello.

I want to use elasticsearch-analysis-kuromoji-neologd with Elasticsearch 7.7.x
However, I haven't found elasticsearch-analysis-kuromoji-neologd 7.7.x plugin

When is it scheduled to be released ?

Or, Can I use elasticsearch-analysis-kuromoji-neologd 7.2.0 instead ?

got illegal_argument_exception when use kuromoji_neologd_tokenizer with synonym_graph

Environment

What is the problem?

Expected behavior

return HTTP Status 200.

Actual behavior

return HTTP Status 400.

Steps to reproduce the behavior

Use kuromoji_neologd_tokenizer with synonym_graph

request:

$ curl -X PUT  "localhost:9210/es63-reproduce-index?pretty" -H 'Content-Type: application/json' [email protected]

response:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "failed to build synonyms"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "failed to build synonyms",
    "caused_by" : {
      "type" : "parse_exception",
      "reason" : "Invalid synonym rule at line 1",
      "caused_by" : {
        "type" : "illegal_argument_exception",
        "reason" : "term: 高等学校 analyzed to a token (高等学校) with position increment != 1 (got: 0)"
      }
    }
  },
  "status" : 400
}

reproduce.json:

{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "synonym": {
            "type": "synonym_graph",
            "synonyms": ["高校,高等学校"]
          }
        },
        "tokenizer": {
          "ja_text_tokenizer": {
            "type": "kuromoji_neologd_tokenizer",
            "mode": "search"
          }
        },
        "analyzer": {
          "ja_text_analyzer": {
            "tokenizer": "ja_text_tokenizer",
            "type": "custom",
            "filter": ["synonym"]
          }
        }
      }
    }
  },
  "mappings": {
    "_doc": {
      "dynamic": "strict",
      "properties": {
        "name": {
          "fielddata": true,
          "type": "text",
          "analyzer": "ja_text_analyzer"
        }
      }
    }
  }
}

Use kuromoji_tokenizer with synonym_graph

request:

$ curl -X PUT  "localhost:9210/es63-reproduce-index?pretty" -H 'Content-Type: application/json' [email protected]

response:

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "es63-reproduce-index"
}

ok.json:

{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "synonym": {
            "type": "synonym_graph",
            "synonyms": ["高校,高等学校"]
          }
        },
        "tokenizer": {
          "ja_text_tokenizer": {
            "type": "kuromoji_tokenizer",
            "mode": "search"
          }
        },
        "analyzer": {
          "ja_text_analyzer": {
            "tokenizer": "ja_text_tokenizer",
            "type": "custom",
            "filter": ["synonym"]
          }
        }
      }
    }
  },
  "mappings": {
    "_doc": {
      "dynamic": "strict",
      "properties": {
        "name": {
          "fielddata": true,
          "type": "text",
          "analyzer": "ja_text_analyzer"
        }
      }
    }
  }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.