Comments (7)
mosuka
commented on May 18, 2024
Hi @Peipeilvcm ,
Thank you for your report.
Unfortunately, it could not occur in my environment.
I tried the following command. Can you share your commands?
Start indexer1:
$ ./bin/blast indexer start \
--grpc-address=:5000 \
--grpc-gateway-address=:6000 \
--http-address=:8000 \
--node-id=indexer1 \
--node-address=:2000 \
--data-dir=/tmp/blast/indexer1 \
--raft-storage-type=boltdb \
--index-mapping-file=./example/wiki_index_mapping.json \
--index-type=upside_down \
--index-storage-type=boltdbStart indexer2:
$ ./bin/blast indexer start \
--peer-grpc-address=:5000 \
--grpc-address=:5010 \
--grpc-gateway-address=:6010 \
--http-address=:8010 \
--node-id=indexer2 \
--node-address=:2010 \
--data-dir=/tmp/blast/indexer2 \
--raft-storage-type=boltdbStart indexer3:
$ ./bin/blast indexer start \
--peer-grpc-address=:5000 \
--grpc-address=:5020 \
--grpc-gateway-address=:6020 \
--http-address=:8020 \
--node-id=indexer3 \
--node-address=:2020 \
--data-dir=/tmp/blast/indexer3 \
--raft-storage-type=boltdb
You can see cluster info as following:
$ ./bin/blast indexer cluster info --grpc-address=:5000 | jq .
{
"cluster": {
"nodes": {
"indexer1": {
"id": "indexer1",
"bind_address": ":2000",
"state": 3,
"metadata": {
"grpc_address": ":5000",
"grpc_gateway_address": ":6000",
"http_address": ":8000"
}
},
"indexer2": {
"id": "indexer2",
"bind_address": ":2010",
"state": 1,
"metadata": {
"grpc_address": ":5010",
"grpc_gateway_address": ":6010",
"http_address": ":8010"
}
},
"indexer3": {
"id": "indexer3",
"bind_address": ":2020",
"state": 1,
"metadata": {
"grpc_address": ":5020",
"grpc_gateway_address": ":6020",
"http_address": ":8020"
}
}
}
}
}
Stop indexer1, the leader, and check the cluster information using the following command:
$ ./bin/blast indexer cluster info --grpc-address=:5010 | jq .
{
"cluster": {
"nodes": {
"indexer1": {
"state": 4,
"metadata": {}
},
"indexer2": {
"id": "indexer2",
"bind_address": ":2010",
"state": 3,
"metadata": {
"grpc_address": ":5010",
"grpc_gateway_address": ":6010",
"http_address": ":8010"
}
},
"indexer3": {
"id": "indexer3",
"bind_address": ":2020",
"state": 1,
"metadata": {
"grpc_address": ":5020",
"grpc_gateway_address": ":6020",
"http_address": ":8020"
}
}
}
}
}
You can see one of the followers has become a leader ("state": 3) and indexer1 has gone ("state": 4) .
Restart indexer1 as follower:
$ ./bin/blast indexer start \
--peer-grpc-address=:5010 \
--grpc-address=:5000 \
--grpc-gateway-address=:6000 \
--http-address=:8000 \
--node-id=indexer1 \
--node-address=:2000 \
--data-dir=/tmp/blast/indexer1 \
--raft-storage-type=boltdbThen, check the cluster information using the following command:
$ ./bin/blast indexer cluster info --grpc-address=:5010 | jq .
{
"cluster": {
"nodes": {
"indexer1": {
"id": "indexer1",
"bind_address": ":2000",
"state": 1,
"metadata": {
"grpc_address": ":5000",
"grpc_gateway_address": ":6000",
"http_address": ":8000"
}
},
"indexer2": {
"id": "indexer2",
"bind_address": ":2010",
"state": 3,
"metadata": {
"grpc_address": ":5010",
"grpc_gateway_address": ":6010",
"http_address": ":8010"
}
},
"indexer3": {
"id": "indexer3",
"bind_address": ":2020",
"state": 1,
"metadata": {
"grpc_address": ":5020",
"grpc_gateway_address": ":6020",
"http_address": ":8020"
}
}
}
}
}
You can see indexer1 has become a follower ("state": 1).
from blast.
Peipeilvcm
commented on May 18, 2024
start cluster as followings:
sudo docker run -d --name blast-indexer1 --net=host -v $PWD/example:/opt/blast/example \
mosuka/blast:latest blast indexer start \
--grpc-address=:5001 \
--grpc-gateway-address=:6000 \
--http-address=:8000 \
--node-id=blast-indexer1 \
--node-address=:2000 \
--data-dir=/tmp/blast/indexer1 \
--raft-storage-type=boltdb \
--index-mapping-file=/opt/blast/example/A_mapping_for_logging.json \
--index-type=scorch \
--index-storage-type=scorch
sudo docker run -d --name blast-indexer2 --net=host \
mosuka/blast:latest blast indexer start \
--peer-grpc-address=:5001 \
--grpc-address=:5010 \
--grpc-gateway-address=:6010 \
--http-address=:8010 \
--node-id=indexer2 \
--node-address=:2010 \
--data-dir=/tmp/blast/indexer2 \
--raft-storage-type=boltdb
sudo docker run -d --name blast-indexer3 --net=host \
mosuka/blast:latest blast indexer start \
--peer-grpc-address=:5001 \
--grpc-address=:5020 \
--grpc-gateway-address=:6020 \
--http-address=:8020 \
--node-id=indexer3 \
--node-address=:2020 \
--data-dir=/tmp/blast/indexer3 \
--raft-storage-type=boltdb
kill indexer1:
sudo docker rm blast-indexer1
sudo docker stop blast-indexer2
Indexing a document to indexer2 by curl
curl -X PUT 'http://127.0.0.1:6010/v1/documents/enwiki_1' -H 'Content-Type: application/json' --data-binary '
{
"fields": {
"title_en": "Search engine (computing)",
"text_en": "A search engine is an information retrieval system designed to help find information stored on a computer system. The search results are usually presented in a list and are commonly called hits. Search engines help to minimize the time required to find information and the amount of information which must be consulted, akin to other techniques for managing information overload. The most public, visible form of a search engine is a Web search engine which searches for information on the World Wide Web.",
"timestamp": "2018-07-04T05:41:00Z",
"_type": "enwiki"
}
}
'
The document can be got from indexer2 & indexer3
$ curl -X GET 'http://127.0.0.1:6020/v1/documents/enwiki_1' -H 'Content-Type: application/json' | jq .
{
"fields": {
"_type": "enwiki",
"text_en": "A search engine is an information retrieval system designed to help find information stored on a computer system. The search results are usually presented in a list and are commonly called hits. Search engines help to minimize the time required to find information and the amount of information which must be consulted, akin to other techniques for managing information overload. The most public, visible form of a search engine is a Web search engine which searches for information on the World Wide Web.",
"timestamp": "2018-07-04T05:41:00Z",
"title_en": "Search engine (computing)"
}
}
indexing a document to indexer3 by curl
curl -X PUT 'http://127.0.0.1:6010/v1/documents/enwiki_2' -H 'Content-Type: application/json' --data-binary '
{
"fields": {
"title_en": "Search engine (computing)",
"text_en": "A saasascafa",
"timestamp": "2018-07-04T05:41:00Z",
"_type": "enwiki"
}
}
'
It works well, indexer2 & indexer3 can be both writen.
Thanks.
from blast.
Peipeilvcm
commented on May 18, 2024
@mosuka no reply?
from blast.
mosuka
commented on May 18, 2024
Hi @Peipeilvcm ,
Sorry for the late reply.
Data replication was completed by restarting indexer1 as follows:
sudo docker run -d --name blast-indexer1 --net=host -v $PWD/example:/opt/blast/example \
mosuka/blast:latest blast indexer start \
--peer-grpc-address=:5010 \
--grpc-address=:5001 \
--grpc-gateway-address=:6000 \
--http-address=:8000 \
--node-id=blast-indexer1 \
--node-address=:2000 \
--data-dir=/tmp/blast/indexer1 \
--raft-storage-type=boltdb
It may take some time to elect a leader.
Can you check it?
from blast.
mosuka
commented on May 18, 2024
I added a docker-compose.yml. If you use Docker to build a cluster, please refer to following as well:
https://github.com/mosuka/blast#running-cluster-on-docker-compose
from blast.
Peipeilvcm
commented on May 18, 2024
Thanks for your replying. But there is an issue as following:
@mosuka , can indexing happen on follower nodes? In my operations above, I can index a document enwiki_2(or maybe enwiki_1) on follower node by curl.
from blast.
mosuka
commented on May 18, 2024
In Blast, update requests received by followers are forwarded to the leader once. The followers do not update the data directly.
from blast.
Related Issues (20)
- The storage backend only supports scorch
- re-index / update an individual document HOT 2
- Comparison with other systems like Toshi/Tantivy HOT 5
- Syncing or integrating with existing PostgreSQL database HOT 4
- Error getting document by ID over rest API HOT 3
- Is there a way to create new mapping definitions using the REST API? HOT 1
- Delete the experimentally implemented feature for distributed search
- Implement CORS
- docker no http beyond root HOT 2
- How to search documents with id prefix? HOT 3
- leadership lost in docker-compose HOT 3
- Snapshotting and Chaos Testing HOT 2
- How to use simple analyzer? HOT 2
- Some questions about queries HOT 1
- bleve v2 and blast HOT 2
- https://github.com/blevesearch/blevex HOT 2
- Dockerfile image not being built HOT 1
- is dead
- I have got a SIGSEGV: segmentation violation HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from blast.