Giter Club home page Giter Club logo

solr-scala-client's Introduction

solr-scala-client Scala CI solr-scala-client Scala version support License

The simple Apache Solr client for Scala. This is based on the SolrJ and provides optimal interface for Scala.

Add the following dependency into your build.sbt to use solr-scala-client.

libraryDependencies += "com.github.takezoe" %% "solr-scala-client" % "0.0.27"

If you want to test SNAPSHOT version, add the following dependency instead of above:

resolvers += "sonatype-oss-snapshot" at "https://oss.sonatype.org/content/repositories/snapshots"

libraryDependencies += "com.github.takezoe" %% "solr-scala-client" % "x.x.x-SNAPSHOT"

This is a simplest example to show usage of solr-scala-client.

import com.github.takezoe.solr.scala._

val client = new SolrClient("http://localhost:8983/solr")

// register
client
  .add(Map("id"->"001", "manu" -> "Lenovo", "name" -> "ThinkPad X201s"))
  .add(Map("id"->"002", "manu" -> "Lenovo", "name" -> "ThinkPad X220"))
  .add(Map("id"->"003", "manu" -> "Lenovo", "name" -> "ThinkPad X121e"))
  .commit

// query
val result = client.query("name: %name%")
  .fields("id", "manu", "name")
  .sortBy("id", Order.asc)
  .getResultAsMap(Map("name" -> "ThinkPad"))

result.documents.foreach { doc: Map[String, Any] =>
  println("id: " + doc("id"))
  println("  manu: " + doc("manu"))
  println("  name: " + doc("name"))
}

It's also possible to use the case class as the search result and parameters instead of Map.

// query
val result = client.query("name: %name%")
  .fields("id", "manu", "name")
  .sortBy("id", Order.asc)
  .getResultAs[Product](Param(name = "ThinkPad"))

result.documents.foreach { product: Product =>
  println("id: " + product.id)
  println("  manu: " + product.manu)
  println("  name: " + product.name)
}

Query Syntax

Following notations are available to embed variables to the query:

  • %VAR_NAME% : place holder to set a single word (parameter would be escaped)
  • ?VAR_NAME? : place holder to set an expression (&, | and ! are available in an expression)
  • $VAR_NAME$ : string replacement (parameter would be not escaped)

See examples of parameterized queries and assembled Solr queries.

// %VAR_NAME% (Single keyword)
client.query("name: %name%").getResultAsMap(Map("name" -> "ThinkPad X201s"))
  // => name:"ThinkPad X201s"

// $VAR_NAME$ (String replacement)
client.query("name: $name$").getResultAsMap(Map("name" -> "ThinkPad AND X201s"))
  // => name:ThinkPad AND X201s

// ?VAR_NAME? (Expression)
client.query("name: ?name?").getResultAsMap(Map("name" -> "ThinkPad & X201s"))
  // => name:("ThinkPad" AND "X201s")

Highlight

Configure the query to return the highlighted content by QueryBuilder#highlight(). The highlighted content is set as the "highlight" property to the Map or the case class.

val result = client.query("content: Scala")
  // NOTE: unique key field is required.
  .fields("id")
  // Specify the highlighted field, prefix and postfix (prefix and postfix are optional).
  .highlight("content", "<strong>", "</strong>")
  .getResultAsMap()

result.documents.foreach { doc: Map[String, Any] =>
  println("id: " + doc("id"))
  println(doc("highlight")) // highlighted content is set as the "highlight" property
}

solr-scala-client expects that the unique key is "id". If your schema has the different field as the unique key, you can specify the unique key name as following:

val result = client.query("content: Scala")
  .id("documentId") // Specify the unique key name
  .fields("documentId")
  .highlight("content", "<strong>", "</strong>")
  .getResultAsMap()

Asynchronous API

solr-scala-client has also asynchronous API based on AsyncHttpCleint.

val client = new AsyncSolrClient("http://localhost:8983/solr")

// Register
client
  .register(Map("id" -> "005", "name" -> "ThinkPad X1 Carbon", "manu" -> "Lenovo"))
  .onComplete{
    case Success(x) => println("registered!")
    case Failure(t) => t.printStackTrace()
  }

// Query
client.query("name:%name%")
  .fields("id", "manu", "name")
  .facetFields("manu")
  .sortBy("id", Order.asc)
  .getResultAsMap(Map("name" -> "ThinkPad X201s"))
  .onComplete {
    case Success(x) => println(x)
    case Failure(t) => t.printStackTrace()
  }

See more example at AsyncSolrClientSample.scala.

Release Notes

0.0.27 - 5 Oct 2021

  • Add sortBy overload for multiple sorting fields

0.0.26 - 14 Sep 2021

  • Fix scala-parser-combinators dependency

0.0.25 - 6 Sep 2021

  • Support Scala 3

0.0.24 - 10 Mar 2020

  • Include facetpivot in mapquery result

0.0.23 - 24 Feb 2020

  • Facet Pivot Fields support
  • Update dependent libraries

0.0.22 - 11 Dec 2019

  • Spatial parameters support

0.0.21 - 22 Jun 2019

  • Scala 2.13.0 support

0.0.20 - 13 Jan 2019

  • Scala 2.13.0-M5 support
  • Support specifying collection with transaction

0.0.19 - 4 Jun 2018

  • Add support for grouping and qTime in the response
  • Allow batch processing with a specific collection

0.0.18 - 15 Feb 2018

  • Fix response leaking bug

0.0.17 - 5 Dec 2017

  • Upgrade to SolrJ-7.1.0
  • Switch backend to OkHttp from async-http-client
  • Allow specifying collection name when building query
  • Add implementation of CloudSolrClient with and without authentication

0.0.16 - 18 Oct 2017

  • Upgrade Scala and async-http-client

0.0.15 - 22 Nov 2016

  • Scala 2.12 support and library updating

0.0.14 - 14 Aug 2016

  • Small refactoring

0.0.13 - 13 Aug 2016

  • Upgrade to SolrJ-6.1.0
  • Change group id and package name to com.github.takezoe
  • Publish to the Maven central repository

0.0.12 - 7 Feb 2015

  • Add QueryBuilderBase#fq()
  • Add QueryBuilderBase#setRequestHandler()
  • Add date facet
  • Support for streaming results

0.0.11 - 29 Mar 2014

  • Add SolrClient#shutdown()
  • QueryBuilder became immutable
  • Upgrade solrj version to 4.5.1

0.0.10 - 08 Feb 2014

  • Fix escaping in string literal.

0.0.9 - 18 Dec 2013

  • Bug fix

0.0.8 - 2 Aug 2013

  • Added recommendation search support.
  • Added rollback and withTransaction to SolrScalaClient.
  • Added Asynchronous API.

0.0.7 - 4 Apr 2013

  • Add build for Scala 2.10
  • Upgrade to SolrJ 4.2.0
  • Support highlighting

0.0.6 - 22 Jan 2013

  • Fixed some ExpressionParser bugs.

0.0.5 - 20 Nov 2012

  • ExpressionParser became pluggable and added GoogleExpressionParser as an optional implementation of ExpressionParser.
  • Converts the full-width space to the half-width space in the expression before calling ExpressionParser.
  • Introduced the SolrServer factory. Auth.basic moved to SolrServerFactory.basicAuth and SolrServerFactory.dummy for unit testing.

0.0.4 - 16 Sep 2012

  • Expanding expression to the Solr query by ?VAR_NAME? in SolrClient#query().
  • Bug fix

0.0.3 - 16 Aug 2012

  • Added case class support in update operations.
  • Added commit() method to SolrClient.

0.0.2 - 27 May 2012

  • Added initializer which configures SolrClient.
  • Added basic authentication support as initializer.
  • Added facet search support.
  • Added case class support as query results and query parameters.

0.0.1 - 4 May 2012

  • Initial public release.

solr-scala-client's People

Contributors

adamsar avatar caneroj1 avatar firemuzzy avatar jeffdyke avatar magro avatar maqdev avatar masahitojp avatar nhooey avatar palak21 avatar rtyley avatar rvegas avatar schmmd avatar takezoe avatar tcak76 avatar tsuyoshizawa avatar ukjain avatar xuwei-k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

solr-scala-client's Issues

Asynchronous Solr calls

First of all thank you for your great Scala-Solr library. :-)

Are there any plans to make the client asynchronous (f.ex. as Play2 module)?

Clarify license

You haven't specified the license. This is important to us so that we can use your project. If you're open to suggestions: Apache 2 would be great! :-)

add and addToCollection throw NPE

I have noticed in a newer version of the ApacheSolrClient it seems to want the collection being written to as an argument and also one for commiting, otherwise an NPE, or 404 is thrown depending on the way the client is instantiated. I have a simple schema that conforms to this case class:

case class Test(caption: String, categoryGroup: String, categoryName: String, id: String)

The following are my results with unchanged solr-scala-client (running sbt console):

import com.github.takezoe.solr.scala.{BatchRegister, SolrClient}
case class Test(caption: String, categoryGroup: String, categoryName: String, id: String)
val solrTest = Test("here is a caption", "here is a group", "here is name", "media-6000")
val test = new SolrClient("http://localhost:8983/solr")
test.add(solrTest)
---
java.lang.NullPointerException
  at com.github.takezoe.solr.scala.BatchRegister.$anonfun$add$1(BatchRegister.scala:17)
---

Using addToCollection, same imports/data as above

test.addToCollection("testing", solrTest)
res9: com.github.takezoe.solr.scala.BatchRegister = com.github.takezoe.solr.scala.BatchRegister@4f86b106
res9.commit
---
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/update. Reason:
<pre>    Not Found</pre></p>
</body>
</html>
---

With modified code seen here:
master...jeffdyke:master

add and addToCollection will return properly, when supplied a collection, but you can't send them the collection as part of the URL, that will result in a 404 as it tries to append it.

I'm happy to open an official PR, but would like to get your thoughts on this. If i were to change it, i would make it a breaking change, or you could infer the collection from the URL, but i don't like that idea to much.

Don't squelch exceptions in map2case

I don't see a point in squelching all exceptions in map2case. Maybe there is a particular bothersome exception? I didn't realize that multivalued fields would return an ArrayList and it took a while to track down the problem. Maybe solr-scala-client should convert these to a Seq?

Simple Question

I need to use this library form scala but I could not prepare sbt file properly. I have this in sbt.
But it could not find solr client. Any help to resolve this?

scalaVersion := "2.11.12"
libraryDependencies in ThisBuild ++= Seq(
"org.apache.spark" %% "spark-core" % "2.3.0",
"org.apache.spark" %% "spark-streaming" % "2.3.0",
"org.apache.spark" %% "spark-sql" % "2.3.0",
"com.couchbase.client" %% "spark-connector" % "2.3.0",
"com.github.takezoe" %% "solr-scala-client" % "0.0.21"
)

QueryBulder should be immutable

The current QueryBuilder has internal state so it's mutable, but it should be immutable.

The reason of this state is QueryBuilder wraps org.apache.solr.client.solrj.SolrQuery. QueryBuilder should have only configurations instead of an SolrQuery instance. It should assemble SolrQuery before it runs query actually.

I want to contribute with you

Dear Team,

Kindly note that this library needs extra functions for building Solr Query like

we need to add bf,qf,mm, ...

so we need to add this function

/**

  • Sets parameter field names.
    *
  • @param field name
  • @param value
    */
    def setParameter(field:String,value:String ): Repr = {
    val ret = copy()
    ret.solrQuery.setParam(field, value)
    ret
    }

and there is other modifications. So, Can I help you to improve this library ?

Project Owner to Fix Build

Naoki,
There seems to be an issue with build set up - sbt-launcher v0.12 is getting pulled which results in a version incompatibility exception:
wget -O /tmp/sbt-launch.jar http://typesafe.artifactoryonline.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/**0.12.0**/sbt-launch.jar Exception: Error during sbt execution: java.lang.IncompatibleClassChangeError: JLine incompatibility detected. Check that the sbt launcher is version 0.13.x or later.

Can you add me?

Thanks,
Artem

EmbeddedSolrServer

I'm interested in using this library with an embedded server. It should be simple enough for me to add it.

Does anyone foresee any difficulties with this?

I need you help

I am new in Scala, and I want to use Solr with Scala, and your library is awesome, but please can you help to add your library in my Play/Scala framework ?

I am trying to convert your lib to jar, but no avail.

Please advice

Is there a reason SolrClient.server has to be private?

Being that it is private means you have to write extra code to get simple things like server.ping
While I find this client very useful for my needs as they are quite simple, being able to access the underlying client would be helpful and stop you from writing additional code or interacting with both this library and then the raw java library.

Oh, i'm happy to open a PR, but only if it will be accepted.
Thanks

NPE when using groupBy

Hi there,

in version 0.0.16, when using a .groupBy(..) clause on a query, getResultAsMap fails with a NullPointerException at com.github.takezoe.solr.scala.QueryBuilderBase.responseToMap(QueryBuilderBase.scala:229)

I don't have any experience with Solr, but maybe QueryResponse::getResults returns null, and the path through QueryResponse::getGroupResponse...getGroups...getResults etc. has to be taken in this case?

Or am I missing something?

Thanks for publishing the library though!

Query Solr did not work well

I have tried your example for regular client and Async but it did not work for query.
I have created a repo so that so can try it.
https://github.com/hendisantika/Coba-Solr

Here are my notes :

  1. Coba.scala --> It add to solr repo but the query result is not working
  2. AsyncSolrClientSample.scala --> Can not add and query.
  3. SolrClientSample.scala --> Can not add and query.

Please give me an advice and solution.

Thanks

Leaking response bodies from OkHttp

Under heavy load I'm getting exceptions

okhttp3.OkHttpClient                A connection to ... was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE: Logger.getLogger(OkHttpClient.class.getName()).setLevel(Level.FINE);

I belive the problem is that Response from OkHttp is Closeable and should be released ASAP.

How do we use pagination

Hi @takezoe,
I want to ask more :
How do we use pagination in solr-scala-client?
Finally I use your regular client not Async for a while.

val result1 = client.query("name:%name%")
    .fields("id", "manu", "name")
    .sortBy("id", Order.asc)
    .getResultAsMap(Map("name" -> "ThinkPad X220"))

  println("-- matched documents 1--")
  result1.documents.foreach { doc =>
    println("id: " + doc("id"))
    println("  manu: " + doc.get("manu").getOrElse("<NULL>"))
    println("  name: " + doc("name"))
  }

Thanks

.facetFields() must be called, or getResultAsMap throws exception

See title. Here's my query code:
val result = solrClient
.query("content:%term% description:%term% tags:%term%")
.fields("id")
.facetFields()
.sortBy("created", Order.desc)
.getResultAsMap(Map("term" -> term))

If I remove .facetFields(), then getResultAsMap throws a null-pointer exception.

case classes with empty field

Thank you for this useful work !

I'm running into a problem when using case class to describe documents.
If for one document a text field value is an empty string (""), the document can be stored by Solr but the specific field will not be. Then at query time, the SolrClient seems unable to rebuild the document when I try to getResultsAs[MyCaseClass](...)

It seems to work with Option[String] (with Some("") becoming None though)

want to stop dependency on Netty 4.0

I use playframework 2.6 + NettyServer. It depends on netty 4.1.

https://www.playframework.com/documentation/2.6.x/Highlights26#Akka-HTTP-Server-Backend

solr-scala-client(Asynchronous API) depends on async-http-client 2.0. It depends on Netty 4.0 (by the way, 2.1-alpha uses netty 4.1)

https://github.com/AsyncHttpClient/async-http-client
AsyncHttpClient/async-http-client#1161

Suggestion

  • Synchronous API and Asynchronous API libraries are separated
  • update to async-http-client 2.1-alpha

Everyone What do you think?

0.0.21 is not on maven central?

Thanks for the great lib! Let me report the issue I found. Could you check this? ๐Ÿ™

I tried this from the README.

libraryDependencies += "com.github.takezoe" %% "solr-scala-client" % "0.0.21"

I got an error.

sbt:solr-test> test
[info] Updating 
[info] Resolved  dependencies
[warn] 
[warn] 	Note: Unresolved dependencies path:
[error] stack trace is suppressed; run last update for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading com.github.takezoe:solr-scala-client_2.12:0.0.21
[error]   Not found
[error]   Not found
[error]   not found: /Users/horie/.ivy2/local/com.github.takezoe/solr-scala-client_2.12/0.0.21/ivys/ivy.xml
[error]   not found: https://repo1.maven.org/maven2/com/github/takezoe/solr-scala-client_2.12/0.0.21/solr-scala-client_2.12-0.0.21.pom
[error] Total time: 1 s, completed Oct 18, 2019, 10:01:48 AM

I checked on this page and I couldn't find 0.0.21. Is it actually published?

https://mvnrepository.com/artifact/com.github.takezoe/solr-scala-client

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.