senssei / mongo-cluster-docker Goto Github PK
View Code? Open in Web Editor NEWDocker compose config for mongodb cluster
License: MIT License
Docker compose config for mongodb cluster
License: MIT License
You are using the curl
to setup the containers properly in synchronized from the scripts/*. But I can't find the curl
in the recent official mongo images. The belows are what I tested.
mongo latest 5bc602c0b7fe 6 days ago 360 MB
mongo 3.4.1 0dffc7177b06 2 months ago 402 MB
mongo 3.3 3abda4f46443 4 months ago 401 MB
mongo 3.3.15 3abda4f46443 4 months ago 401 MB
mongo 3.2.10 fe9198c04d62 4 months ago 342 MB
And
$ docker run -it --rm mongo:3.3.15 curl
/entrypoint.sh: line 21: exec: curl: not found
Any advice?
Command ping
was removed from mongo:lastest
image, so all variables in script will not work.
How to mount data volume to replication set.
There are missing chmod +x
in git index for init_shard.sh
, setup_cnf.sh
and setup.sh
Really painful on Linux boxes.
Just reading on the documentation, It seems that there are no shards added after completing the instructions.
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("587d306454828b89adaca524")
}
shards:
active mongoses:
"3.4.1" : 1
balancer:
Currently enabled: yes
Currently running: yes
Balancer lock taken at Mon Jan 16 2017 22:18:53 GMT+0100 by ConfigServer:Balancer
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
Isn't the line shards:
supposed to contain the added shards?
Is this setup include automatic adding of shards?
total 32
drwxr-xr-x 6 1155169105 263744041 204 Apr 3 10:08 .
drwxr-xr-x 12 1155169105 263744041 408 Apr 3 10:08 ..
-rw-r--r-- 1 1155169105 263744041 18 Apr 3 10:08 .gitattributes
-rw-r--r-- 1 1155169105 263744041 1070 Apr 3 10:08 init-shard.sh
-rw-r--r-- 1 1155169105 263744041 1020 Apr 3 10:08 setup-cnf.sh
-rwxr-xr-x 1 1155169105 263744041 991 Apr 3 10:08 setup.sh
Currently only setup.sh
has the permission.
Add sample sharding with index.
During run of mongo-cnf-x
it seems to be invalid rest option.
Error parsing command line: unrecognised option '--rest'
try 'mongod --help' for more information
Update docs with latest mongo and upcoming 3.6
MongoDB 4.0, you cannot specify --nojournal in replica set on wiredTiger engine
mongo-2-1 | 2018-08-21T08:18:34.782+0000 I STORAGE [initandlisten] Running wiredTiger without journaling in a replica set is not supported. Make sure you are not using --nojournal and that storage.journal.enabled is not set to 'false'.
Replication protocol update
{
"operationTime": Timestamp(1534837629,1),
"ok": 0,
"errmsg": "Support for replication protocol version 0 was removed in MongoDB 4.0. Downgrade to MongoDB version 3.6 and upgrade your protocol version to 1 before upgrading your MongoDB version",
"code": 103,
"codeName": "NewReplicaSetConfigurationIncompatible",
"$gleStats": {
"lastOpTime": Timestamp(1534837629,1),
"electionId": ObjectId("000000000000000000000000")
},
"lastCommittedOpTime": Timestamp(0,0),
"$clusterTime": {
"clusterTime": Timestamp(1534837629,1),
"signature": {
"hash": BinData(0,
"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId": NumberLong(0)
}
}
I have a bit of a problem.
I use your code, and I have used three replicas, so I have a problem that I do not understand.
ERROR: Service 'mongo-router' depends on service 'mongo-rs3-setup' which is undefined.
Create additional files. "docker-compose.3.yml"
version: '2'
services:
mongo-3-2:
container_name: "mongo-3-2"
image: mongo
ports:
- "30032:27017"
command: mongod --replSet rs3 --shardsvr --port 27017 --nojournal --oplogSize 16 --noprealloc --smallfiles
restart: always
mongo-3-3:
container_name: "mongo-3-3"
image: mongo
ports:
- "30033:27017"
command: mongod --replSet rs3 --shardsvr --port 27017 --nojournal --oplogSize 16 --noprealloc --smallfiles
restart: always
mongo-3-1:
container_name: "mongo-3-1"
image: mongo
ports:
- "30031:27017"
command: mongod --replSet rs3 --shardsvr --port 27017 --nojournal --oplogSize 16 --noprealloc --smallfiles
links:
- mongo-3-2:mongo-3-2
- mongo-3-3:mongo-3-3
restart: always
mongo-rs3-setup:
container_name: "mongo-rs3-setup"
image: mongo
depends_on:
- "mongo-3-1"
- "mongo-3-2"
- "mongo-3-3"
links:
- mongo-3-1:mongo-3-1
- mongo-3-2:mongo-3-2
- mongo-3-3:mongo-3-3
volumes:
- ./scripts:/scripts
environment:
- MONGO1=mongo-3-1
- MONGO2=mongo-3-2
- MONGO3=mongo-3-3
- RS=rs3
entrypoint: [ "/scripts/setup.sh" ]
Modify files. "docker-compose.shard.yml" (ports and depends_on, links,environment)
version: '2'
services:
mongo-router:
container_name: "mongo-router"
image: mongo
ports:
- "37031:27017"
depends_on:
- "mongo-rs1-setup"
- "mongo-rs2-setup"
- "mongo-rs3-setup"
- "mongo-cnf-setup"
links:
- mongo-cnf-1:mongo-cnf-1
- mongo-cnf-2:mongo-cnf-2
- mongo-cnf-3:mongo-cnf-3
- mongo-1-1:mongo-1-1
- mongo-1-2:mongo-1-2
- mongo-1-3:mongo-1-3
- mongo-2-1:mongo-2-1
- mongo-2-2:mongo-2-2
- mongo-2-3:mongo-2-3
- mongo-3-1:mongo-3-1
- mongo-3-2:mongo-3-2
- mongo-3-3:mongo-3-3
command: mongos --configdb cnf-serv/mongo-cnf-1:27017,mongo-cnf-2:27017,mongo-cnf-3:27017
restart: always
mongo-shard-setup:
container_name: "mongo-shard-setup"
image: mongo
depends_on:
- "mongo-router"
links:
- mongo-router:mongo-router
volumes:
- ./scripts:/scripts
environment:
- MONGOS=mongo-router
- MONGO11=mongo-1-1
- MONGO12=mongo-1-2
- MONGO13=mongo-1-3
- MONGO21=mongo-2-1
- MONGO22=mongo-2-2
- MONGO23=mongo-2-3
- MONGO31=mongo-3-1
- MONGO32=mongo-3-2
- MONGO33=mongo-3-3
- RS1=rs1
- RS2=rs2
- RS3=rs3
- PORT=27017
- PORT1=27017
- PORT2=27017
- PORT3=27017
entrypoint: [ "/scripts/init-shard.sh" ]
restart: on-failure:20
And...
Modify init-shard.sh file
#!/bin/bash
mongodb1=`ping -c 1 ${MONGOS} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb11=`ping -c 1 ${MONGO11} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb12=`ping -c 1 ${MONGO12} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb13=`ping -c 1 ${MONGO13} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb21=`ping -c 1 ${MONGO21} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb22=`ping -c 1 ${MONGO22} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb23=`ping -c 1 ${MONGO23} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb31=`ping -c 1 ${MONGO31} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb32=`ping -c 1 ${MONGO32} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongodb33=`ping -c 1 ${MONGO33} | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
port=${PORT:-27017}
echo "Waiting for startup.."
until mongo --host ${mongodb1}:${port} --eval 'quit(db.runCommand({ ping: 1 }).ok ? 0 : 2)' &>/dev/null; do
printf '.'
sleep 1
done
echo "Started.."
echo init-shard.sh time now: `date +"%T" `
mongo --host ${mongodb1}:${port} <<EOF
sh.addShard( "${RS1}/${mongodb11}:${PORT1},${mongodb12}:${PORT2},${mongodb13}:${PORT3}" );
sh.addShard( "${RS2}/${mongodb21}:${PORT1},${mongodb22}:${PORT2},${mongodb23}:${PORT3}" );
sh.addShard( "${RS3}/${mongodb31}:${PORT1},${mongodb32}:${PORT2},${mongodb33}:${PORT3}" );
sh.status();
EOF
And....
Execution proceeded in the following order.
docker-compose -f docker-compose.1.yml up -d
docker-compose -f docker-compose.2.yml up -d
docker-compose -f docker-compose.3.yml up -d
docker-compose -f docker-compose.cnf.yml up -d
docker-compose -f docker-compose.shard.yml up -d
The container is also running well (maybe ..)
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fc2fdca63f7c mongo "/scripts/setup-cn..." 15 minutes ago Up 15 minutes 27017/tcp mongo-cnf-setup
456ad9a0d02c mongo "docker-entrypoint..." 15 minutes ago Up 15 minutes 0.0.0.0:30001->27017/tcp mongo-cnf-1
40f02b4b73e5 mongo "docker-entrypoint..." 15 minutes ago Up 15 minutes 0.0.0.0:30003->27017/tcp mongo-cnf-3
61211e609ea1 mongo "docker-entrypoint..." 15 minutes ago Up 15 minutes 0.0.0.0:30002->27017/tcp mongo-cnf-2
1da83345cf14 mongo "/scripts/setup.sh" 16 minutes ago Up 16 minutes 27017/tcp mongo-rs3-setup
50504749f187 mongo "docker-entrypoint..." 16 minutes ago Up 16 minutes 0.0.0.0:30031->27017/tcp mongo-3-1
7517e293bf64 mongo "docker-entrypoint..." 16 minutes ago Up 16 minutes 0.0.0.0:30033->27017/tcp mongo-3-3
99a0241d44d0 mongo "docker-entrypoint..." 16 minutes ago Up 16 minutes 0.0.0.0:30032->27017/tcp mongo-3-2
5eb4f41e9cc0 mongo "/scripts/setup.sh" 17 minutes ago Up 17 minutes 27017/tcp mongo-rs2-setup
9e4a739f8dbc mongo "docker-entrypoint..." 17 minutes ago Up 17 minutes 0.0.0.0:30021->27017/tcp mongo-2-1
fe570b7269bc mongo "docker-entrypoint..." 17 minutes ago Up 17 minutes 0.0.0.0:30022->27017/tcp mongo-2-2
20b2412a1b78 mongo "docker-entrypoint..." 17 minutes ago Up 17 minutes 0.0.0.0:30023->27017/tcp mongo-2-3
783fadb516c9 mongo "/scripts/setup.sh" 19 minutes ago Up 19 minutes 27017/tcp mongo-rs1-setup
46a5fca17104 mongo "docker-entrypoint..." 19 minutes ago Up 19 minutes 0.0.0.0:30011->27017/tcp mongo-1-1
1949685b824a mongo "docker-entrypoint..." 19 minutes ago Up 19 minutes 0.0.0.0:30013->27017/tcp mongo-1-3
9597d0035946 mongo "docker-entrypoint..." 19 minutes ago Up 19 minutes 0.0.0.0:30012->27017/tcp mongo-1-2
I keep getting the error even though I killed the instances as you suggested. Here's the console dump:
I'm using Windows 10 with Docker version 1.13.1 build 092cba3
Any idea why this keep happening?
Thanks!
Hi,
This code working like charm on local machine, but i unable to connect to from my other machine.
(Local Machine and Host Machine on Same network)
I tried to connect from command line using following command
mongo --host testreplica/192.168.24.5:30001,192.168.24.5:30002,192.168.24.5:30003 test_db -u test_user -p testpassword
Its shows something like,
ongoDB shell version v3.4.13
connecting to: mongodb://192.168.24.5:30001,192.168.24.5:30002,192.168.24.5:30003/test_db?replicaSet=testreplica
2018-04-26T13:09:41.521+0530 I NETWORK [thread1] Starting new replica set monitor for testreplica/192.168.24.5:30001,192.168.24.5:30002,192.168.24.5:30003
2018-04-26T13:09:41.524+0530 I NETWORK [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to 192.168.24.5:30001 (1 connections now open to 192.168.24.5:30001 with a 5 second timeout)
2018-04-26T13:09:41.524+0530 I NETWORK [thread1] Successfully connected to 192.168.24.5:30002 (1 connections now open to 192.168.24.5:30002with a 5 second timeout)
2018-04-26T13:09:41.525+0530 I NETWORK [ReplicaSetMonitor-TaskExecutor-0] changing hosts to testreplica/172.19.0.2:27017,172.19.0.3:27017,172.19.0.4:27017 from testreplica/192.168.24.5:30001,192.168.24.5:30002,192.168.24.5:30003
2018-04-26T13:09:44.531+0530 W NETWORK [thread1] Failed to connect to 172.19.0.4:27017, in(checking socket for error after poll), reason: No route to host
2018-04-26T13:09:44.531+0530 W NETWORK [ReplicaSetMonitor-TaskExecutor-0] Failed to connect to 172.19.0.3:27017, in(checking socket for error after poll), reason: No route to host
2018-04-26T13:09:47.537+0530 W NETWORK [thread1] Failed to connect to 172.19.0.2:27017, in(checking socket for error after poll), reason: No route to host
2018-04-26T13:09:51.043+0530 W NETWORK [thread1] Failed to connect to 172.19.0.4:27017, in(checking socket for error after poll), reason: No route to host
2018-04-26T13:09:54.049+0530 W NETWORK [thread1] Failed to connect to 172.19.0.2:27017, in(checking socket for error after poll), reason: No route to host
2018-04-26T13:09:57.055+0530 W NETWORK [thread1] Failed to connect to 172.19.0.3:27017, in(checking socket for error after poll), reason: No route to host
2018-04-26T13:09:57.055+0530 W NETWORK [thread1] No primary detected for set testreplica
2018-04-26T13:09:57.055+0530 I NETWORK [thread1] All nodes for set testreplica are down. This has happened for 1 checks in a row.
2018-04-26T13:09:57.055+0530 E QUERY [thread1] Error: connect failed to replica set testreplica/192.168.24.5:30001,192.168.24.5:30002,192.168.24.5:30003 :
connect@src/mongo/shell/mongo.js:240:13
@(connect):1:6
exception: connect failed
Also i tried this option from
https://docs.docker.com/network/bridge/#enable-forwarding-from-docker-containers-to-the-outside-world
$ sysctl net.ipv4.conf.all.forwarding=1
sudo iptables -P FORWARD ACCEPT
Describe the bug
MongoDB shell version v4.0.1
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 4.0.1
{
"info" : "mongos",
"ok" : 0,
"errmsg" : "replSetGetStatus is not supported through mongos",
"operationTime" : Timestamp(1559956781, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1559956781, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}```
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
Cluster console:
```mongo-cnf-1 | 2019-06-08T00:58:47.141+0000 I SHARDING [ConfigServerCatalogCacheLoader-1] Refresh for collection config.system.sessions took 0 ms and found version 1|0||5cfaf570d2e4fff3a7510339
mongo-cnf-2 | 2019-06-08T00:58:51.367+0000 W SHARDING [replSetDistLockPinger] Failed to unlock lock with ts: 5cfaf56d3f588dd62feaf80d and _id: config :: caused by :: NotMaster: Not primary while running findAndModify command on collection config.locks
mongo-cnf-3 | 2019-06-08T00:58:52.291+0000 W SHARDING [replSetDistLockPinger] Failed to unlock lock with ts: 5cfaf56ecae5f095192e595f and _id: config :: caused by :: NotMaster: Not primary while running findAndModify command on collection config.locks
mongo-cnf-1 | 2019-06-08T00:58:57.149+0000 I SHARDING [Balancer] Refreshing chunks for collection config.system.sessions based on version 1|0||5cfaf570d2e4fff3a7510339
m```
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [NAME="Ubuntu"]
- Browser [e.g. chrome, safari]
- Version [VERSION="18.04.2 LTS (Bionic Beaver)"]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
```essh@kubernetes-master:~/mongo-cluster-docker$ docker --version
Docker version 18.09.6, build 481bc77```
Thanks. The architecture is very well.
Describe the bug
sudo docker stack deploy -c docker-compose.1.yml -c docker-compose.cnf.yml -c docker-compose.shard.yml mongodb-cluster
sudo docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
5jv0eqanzrc2 mongodb-cluster_mongo-1-1 replicated 1/1 mongo:4.0.1 *:30011->27017/tcp
ijg65m9f7dw0 mongodb-cluster_mongo-1-2 replicated 1/1 mongo:4.0.1 *:30012->27017/tcp
cm7eyuj5nifg mongodb-cluster_mongo-1-3 replicated 1/1 mongo:4.0.1 *:30013->27017/tcp
o4zq7d2hm8zs mongodb-cluster_mongo-cnf-1 replicated 1/1 mongo:4.0.1 *:30101->27017/tcp
sst80l6u2610 mongodb-cluster_mongo-cnf-2 replicated 1/1 mongo:4.0.1 *:30102->27017/tcp
il8a22h436vw mongodb-cluster_mongo-cnf-3 replicated 1/1 mongo:4.0.1 *:30103->27017/tcp
sdqx3ritf9js mongodb-cluster_mongo-cnf-setup replicated 0/1 mongo:4.0.1
eyjr71ilz42n mongodb-cluster_mongo-router replicated 1/1 mongo:4.0.1 *:30001->27017/tcp
2tct3og66lfv mongodb-cluster_mongo-rs1-setup replicated 1/1 mongo:4.0.1
z3p31zdni4m9 mongodb-cluster_mongo-shard-setup replicated 1/1 mongo:4.0.1
essh@kubernetes-master:~/mongo-cluster-docker$ sudo docker logs mongodb-cluster_mongo-cnf-setup.1.dw4g8c0zjxnba3nnlvhxipelk
Waiting for startup..
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
To Reproduce
Steps to reproduce the behavior:
Expected behavior
sudo docker ps -a | grep mongodb-cluster_mongo-cnf-setup
f0a831081042 mongo:4.0.1 "/scripts/setup-cnf.…" Less than a second ago Created mongodb-cluster_mongo-cnf-setup.1.ny9rfgwx7g25il0hth8oeovfz
16e8597b575d mongo:4.0.1 "/scripts/setup-cnf.…" 7 seconds ago Exited (0) 1 second ago mongodb-cluster_mongo-cnf-setup.1.x4wyvan8mr4rz1g6dut89ytye
2dd4f3adaa85 mongo:4.0.1 "/scripts/setup-cnf.…" 13 seconds ago Exited (0) 7 seconds ago mongodb-cluster_mongo-cnf-setup.1.g22oq5g56bzn1mcw1hgmkh5c1
d844c55ad2d8 mongo:4.0.1 "/scripts/setup-cnf.…" 20 seconds ago Exited (0) 14 seconds ago mongodb-cluster_mongo-cnf-setup.1.3noxlc2ewbnyodrkm5i7gers1
45ae7057fc17 mongo:4.0.1 "/scripts/setup-cnf.…" 27 seconds ago Exited (0) 21 seconds ago mongodb-cluster_mongo-cnf-setup.1.7va4kk7g5cve1jxlh31pl777f
e8de43f5ec8f mongo:4.0.1 "/scripts/setup-cnf.…" 34 seconds ago Exited (0) 28 seconds ago mongodb-cluster_mongo-cnf-setup.1.ocahhp9tqn67a5mbx6f35f4rm
887012201c6d mongo:4.0.1 "/scripts/setup-cnf.…" 2 hours ago Exited (137) 4 minutes ago mongodb-cluster_mongo-cnf-setup.1.dw4g8c0zjxnba3nnlvhxipelk
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
Command "sudo docker-compose .... up" work. Command "docker deploy ..." didn't work. I think, mongodb nod't supported a overley network. Am I right?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.