Mongodb副本集
创建副本集需要三台机器,已经提前安装好mongodb了,本次使用的版本是mongodb3.2.10
。
- 环境
节点1
[root@mongodb-node1 ~]# cat /etc/redhat-release
CentOS release 6.3 (Final)
[root@mongodb-node1 ~]# uname -r
2.6.32-279.el6.x86_64
[root@mongodb-node1 ~]# hostname -I
192.168.56.4
节点2
[root@mongodb-node2 ~]# cat /etc/redhat-release
CentOS release 6.3 (Final)
[root@mongodb-node2 ~]# uname -r
2.6.32-279.el6.x86_64
[root@mongodb-node2 ~]# hostname -I
192.168.56.5
节点3
[root@mongodb-node3 ~]# cat /etc/redhat-release
CentOS release 6.3 (Final)
[root@mongodb-node3 ~]# uname -r
2.6.32-279.el6.x86_64
[root@mongodb-node3 ~]# hostname -I
192.168.56.6
- 创建key
处于安全考虑,开启认证(auth)功能,所以需要在副本集同步的时候需要指定key,权限是600。
[root@mongodb-node1 ~]# openssl rand -base64 753 >/data/mongo/conf/key
[root@mongodb-node1 ~]# chmod 600 /data/mongo/conf/key
分配key文件,在分配过程中权限不会变的。
[root@mongodb-node1 ~]# scp -P 22 key root@192.168.56.5:/data/mongo/conf/
[root@mongodb-node1 ~]# scp -P 22 key root@192.168.56.6:/data/mongo/conf/
- 配置文件
在副本集创建完成,前不要把key认证开启,auth认证也不要开启。
节点1配置文件
[root@mongodb-node1 ~]# cat /data/mongo/conf/mongodb.conf
dbpath=/data/mongo/data
port=20000
bind_ip=192.168.56.4
logpath=/data/mongo/logs/mongodb.log
logappend=true
fork=true
maxConns=5000
noauth=true
httpinterface=true
rest=false
oplogSize=10
replSet = repl
#keyFile = /data/mongo/conf/key
剩下的基本上都是一样的就是端口和IP不一样。
- 启动登录mongod
[root@mongodb-node1 ~]# /usr/local/mongodb-3.2.10/bin/mongod -f /data/mongo/conf/mongodb.conf
about to fork child process, waiting until server is ready for connections.
forked process: 13539
child process started successfully, parent exiting
- 配置mongodb副本集
[root@mongodb-node1 ~]# /usr/local/mongodb-3.2.10/bin/mongo 192.168.56.4:20000
MongoDB shell version: 3.2.10
connecting to: 192.168.56.4:20000/test
Server has startup warnings:
2016-10-25T15:31:18.193+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2016-10-25T15:31:18.193+0800 I CONTROL [initandlisten]
> repl_cfg = { _id:"repl", members:[
... {_id:0,host:"192.168.56.4:20000",priority:2},
... {_id:1,host:"192.168.56.5:20001",priority:1},
... {_id:2,host:"192.168.56.6:20002",priority:0}]
... }
{
"_id" : "repl",
"members" : [
{
"_id" : 0,
"host" : "192.168.56.4:20000",
"priority" : 2
},
{
"_id" : 1,
"host" : "192.168.56.5:20001",
"priority" : 1
},
{
"_id" : 2,
"host" : "192.168.56.6:20002",
"priority" : 0
}
]
}
> rs.initiate(repl_cfg)
{ "ok" : 1 }
repl:OTHER>
repl:PRIMARY> rs.status()
{
"set" : "repl",
"date" : ISODate("2016-10-25T07:33:24.932Z"),
"myState" : 1,
"term" : NumberLong(1),
"heartbeatIntervalMillis" : NumberLong(2000),
"members" : [
{
"_id" : 0,
"name" : "192.168.56.4:20000",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 127,
"optime" : {
"ts" : Timestamp(1477380795, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2016-10-25T07:33:15Z"),
"infoMessage" : "could not find member to sync from",
"electionTime" : Timestamp(1477380794, 1),
"electionDate" : ISODate("2016-10-25T07:33:14Z"),
"configVersion" : 1,
"self" : true
},
{
"_id" : 1,
"name" : "192.168.56.5:20001",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 21,
"optime" : {
"ts" : Timestamp(1477380795, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2016-10-25T07:33:15Z"),
"lastHeartbeat" : ISODate("2016-10-25T07:33:24.700Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T07:33:23.275Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.56.4:20000",
"configVersion" : 1
},
{
"_id" : 2,
"name" : "192.168.56.6:20002",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 21,
"optime" : {
"ts" : Timestamp(1477380795, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2016-10-25T07:33:15Z"),
"lastHeartbeat" : ISODate("2016-10-25T07:33:24.700Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T07:33:23.269Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.56.4:20000",
"configVersion" : 1
}
],
"ok" : 1
}
配置完成,另外两个副本集会自动加入。priority
意思是优先值的问题,数值越大,优先值越高。
优先值高的写入数据
repl:PRIMARY> show dbs;
local 0.000GB
repl:PRIMARY> use test
switched to db test
repl:PRIMARY> db.test.insert({'a':'1'})
WriteResult({ "nInserted" : 1 })
repl:PRIMARY> db.test.find()
{ "_id" : ObjectId("580f0b3afdcbdd0ab706f021"), "a" : "1" }
去另一个副本查看。
[root@mongodb-node2 ~]# /usr/local/mongodb-3.2.10/bin/mongo 192.168.56.5:20001
MongoDB shell version: 3.2.10
connecting to: 192.168.56.5:20001/test
Server has startup warnings:
2016-10-23T21:08:16.565+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2016-10-23T21:08:16.565+0800 I CONTROL [initandlisten]
>
repl:SECONDARY> use test
switched to db test
repl:SECONDARY> db.test.find()
{ "_id" : ObjectId("580f0b3afdcbdd0ab706f021"), "a" : "1" }
如果报错not master and slaveOk=false
,如下是解决方法
repl:SECONDARY> db.test.find()
Error: error: { "ok" : 0, "errmsg" : "not master and slaveOk=false", "code" : 13435 }
repl:SECONDARY> rs.slaveOk()
只有优先值最高的副本才能写入数据。
repl:SECONDARY> db.test.insert({'b':'2'})
WriteResult({ "writeError" : { "code" : 10107, "errmsg" : "not master" } })
- 开启认证
在开启认证前记得把用户给创建好。
> use admin
switched to db admin
>db.createUser({
user: "admin",
pwd: "admin",
roles: [ { role: "userAdminAnyDatabase",db: "admin" },{ role: "readAnyDatabase",db: "admin"},
{ role: "readWriteAnyDatabase",db: "admin"},{ role: "dbAdminAnyDatabase",db: "admin"}]
})
先关闭mongodb,修改配置文件
[root@mongodb-node1 ~]# /usr/local/mongodb-3.2.10/bin/mongod --shutdown --dbpath /data/mongo/data/
killing process with pid: 13717
[root@mongodb-node1 ~]# vim /data/mongo/conf/mongodb.conf
[root@mongodb-node1 ~]# cat /data/mongo/conf/mongodb.conf
dbpath=/data/mongo/data
port=20000
bind_ip=192.168.56.4
logpath=/data/mongo/logs/mongodb.log
logappend=true
fork=true
maxConns=5000
auth=true
httpinterface=true
rest=false
oplogSize=10
replSet = repl
keyFile = /data/mongo/conf/key
[root@mongodb-node1 ~]# /usr/local/mongodb-3.2.10/bin/mongod -f /data/mongo/conf/mongodb.conf
about to fork child process, waiting until server is ready for connections.
forked process: 13819
child process started successfully, parent exiting
三个节点都需要这么做。
登录mongodb进行查看,写如数据进行测试。
[root@mongodb-node1 ~]# /usr/local/mongodb-3.2.10/bin/mongo 192.168.56.4:20000
MongoDB shell version: 3.2.10
connecting to: 192.168.56.4:20000/test
repl:PRIMARY> show dbs;
2016-10-25T15:49:45.697+0800 E QUERY [thread1] Error: listDatabases failed:{
"ok" : 0,
"errmsg" : "not authorized on admin to execute command { listDatabases: 1.0 }",
"code" : 13
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:62:1
shellHelper.show@src/mongo/shell/utils.js:761:19
shellHelper@src/mongo/shell/utils.js:651:15
@(shellhelp2):1:1
repl:PRIMARY> use admin
switched to db admin
repl:PRIMARY> db.auth('admin','admin')
1
repl:PRIMARY> show dbs;
admin 0.000GB
local 0.000GB
test 0.000GB
repl:PRIMARY> use test
switched to db test
repl:PRIMARY> db.test.insert({'c':'3'})
WriteResult({ "nInserted" : 1 })
repl:PRIMARY> db.test.find()
{ "_id" : ObjectId("580f0b3afdcbdd0ab706f021"), "a" : "1" }
{ "_id" : ObjectId("580f0d59fdcbdd0ab706f022"), "b" : "2" }
{ "_id" : ObjectId("580f0ebe5447a2d906013ca9"), "c" : "3" }
进入另一个副本进行查看
[root@mongodb-node2 ~]# /usr/local/mongodb-3.2.10/bin/mongo 192.168.56.5:20001
MongoDB shell version: 3.2.10
connecting to: 192.168.56.5:20001/test
repl:SECONDARY> show dbs;
2016-10-23T21:28:38.644+0800 E QUERY [thread1] Error: listDatabases failed:{
"ok" : 0,
"errmsg" : "not authorized on admin to execute command { listDatabases: 1.0 }",
"code" : 13
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:62:1
shellHelper.show@src/mongo/shell/utils.js:761:19
shellHelper@src/mongo/shell/utils.js:651:15
@(shellhelp2):1:1
repl:SECONDARY> use admin
switched to db admin
repl:SECONDARY> db.auth('admin','admin')
1
repl:SECONDARY> show dbs;
2016-10-23T21:28:56.429+0800 E QUERY [thread1] Error: listDatabases failed:{ "ok" : 0, "errmsg" : "not master and slaveOk=false", "code" : 13435 } :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:62:1
shellHelper.show@src/mongo/shell/utils.js:761:19
shellHelper@src/mongo/shell/utils.js:651:15
@(shellhelp2):1:1
repl:SECONDARY> rs.slaveOk()
repl:SECONDARY> show dbs;
admin 0.000GB
local 0.000GB
test 0.000GB
repl:SECONDARY> use test
switched to db test
repl:SECONDARY> db.test.find()
{ "_id" : ObjectId("580f0b3afdcbdd0ab706f021"), "a" : "1" }
{ "_id" : ObjectId("580f0d59fdcbdd0ab706f022"), "b" : "2" }
{ "_id" : ObjectId("580f0ebe5447a2d906013ca9"), "c" : "3" }
要进行认证才可以查看,要执行rs.slaveOk()
才会恢复正的副本复制。
- 创建用户
如果不认证这个用户,会查询不了副本集的状态
repl:PRIMARY> rs.status()
{
"ok" : 0,
"errmsg" : "not authorized on admin to execute command { replSetGetStatus: 1.0 }",
"code" : 13
}
repl:PRIMARY> use admin
switched to db admin
repl:PRIMARY> db.createUser(
... {
... user: "root",
... pwd: "root",
... roles: [ { role: "root", db: "admin" } ]
... }
... );
Successfully added user: {
"user" : "root",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
repl:PRIMARY> db.auth('root','root')
1
repl:PRIMARY> rs.status()
{
"set" : "repl",
"date" : ISODate("2016-10-25T08:15:38.217Z"),
"myState" : 1,
"term" : NumberLong(4),
"heartbeatIntervalMillis" : NumberLong(2000),
"members" : [
{
"_id" : 0,
"name" : "192.168.56.4:20000",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 1651,
"optime" : {
"ts" : Timestamp(1477383316, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:15:16Z"),
"electionTime" : Timestamp(1477381758, 1),
"electionDate" : ISODate("2016-10-25T07:49:18Z"),
"configVersion" : 1,
"self" : true
},
{
"_id" : 1,
"name" : "192.168.56.5:20001",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1600,
"optime" : {
"ts" : Timestamp(1477383316, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:15:16Z"),
"lastHeartbeat" : ISODate("2016-10-25T08:15:36.637Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T08:15:37.451Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.56.6:20002",
"configVersion" : 1
},
{
"_id" : 2,
"name" : "192.168.56.6:20002",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1589,
"optime" : {
"ts" : Timestamp(1477383316, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:15:16Z"),
"lastHeartbeat" : ISODate("2016-10-25T08:15:36.623Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T08:15:37.586Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.56.4:20000",
"configVersion" : 1
}
],
"ok" : 1
}
- 移除和增加节点
删除节点后写入数据
repl:PRIMARY> rs.remove('192.168.56.5:20001')
{ "ok" : 1 }
repl:PRIMARY> rs.status()
{
"set" : "repl",
"date" : ISODate("2016-10-25T08:19:25.481Z"),
"myState" : 1,
"term" : NumberLong(4),
"heartbeatIntervalMillis" : NumberLong(2000),
"members" : [
{
"_id" : 0,
"name" : "192.168.56.4:20000",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 1878,
"optime" : {
"ts" : Timestamp(1477383563, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:19:23Z"),
"electionTime" : Timestamp(1477381758, 1),
"electionDate" : ISODate("2016-10-25T07:49:18Z"),
"configVersion" : 2,
"self" : true
},
{
"_id" : 2,
"name" : "192.168.56.6:20002",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1816,
"optime" : {
"ts" : Timestamp(1477383316, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:15:16Z"),
"lastHeartbeat" : ISODate("2016-10-25T08:19:23.596Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T08:19:23.622Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.56.4:20000",
"configVersion" : 1
}
],
"ok" : 1
}
repl:PRIMARY> use test
switched to db test
repl:PRIMARY> db.test.insert({'d':'4'})
WriteResult({ "nInserted" : 1 })
repl:PRIMARY> db.test.find()
{ "_id" : ObjectId("580f0b3afdcbdd0ab706f021"), "a" : "1" }
{ "_id" : ObjectId("580f0d59fdcbdd0ab706f022"), "b" : "2" }
{ "_id" : ObjectId("580f0ebe5447a2d906013ca9"), "c" : "3" }
{ "_id" : ObjectId("580f12185447a2d906013caa"), "c" : "3" }
{ "_id" : ObjectId("580f16085447a2d906013cab"), "d" : "4" }
查看那个节点状态
repl:SECONDARY>
> show dbs;
2016-10-23T21:57:10.004+0800 I NETWORK [thread1] trying reconnect to 192.168.56.5:20001 (192.168.56.5) failed
2016-10-23T21:57:10.023+0800 I NETWORK [thread1] reconnect 192.168.56.5:20001 (192.168.56.5) ok
admin 0.000GB
local 0.000GB
test 0.000GB
> use test
switched to db test
> db.test.find()
Error: error: { "ok" : 0, "errmsg" : "node is recovering", "code" : 13436 }
增加节点
repl:PRIMARY> rs.add('192.168.56.5:20001')
{ "ok" : 1 }
repl:PRIMARY> rs.status()
{
"set" : "repl",
"date" : ISODate("2016-10-25T08:23:31.316Z"),
"myState" : 1,
"term" : NumberLong(4),
"heartbeatIntervalMillis" : NumberLong(2000),
"members" : [
{
"_id" : 0,
"name" : "192.168.56.4:20000",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2124,
"optime" : {
"ts" : Timestamp(1477383807, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:23:27Z"),
"electionTime" : Timestamp(1477381758, 1),
"electionDate" : ISODate("2016-10-25T07:49:18Z"),
"configVersion" : 3,
"self" : true
},
{
"_id" : 2,
"name" : "192.168.56.6:20002",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 2062,
"optime" : {
"ts" : Timestamp(1477383807, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:23:27Z"),
"lastHeartbeat" : ISODate("2016-10-25T08:23:31.254Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T08:23:31.313Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.56.4:20000",
"configVersion" : 3
},
{
"_id" : 3,
"name" : "192.168.56.5:20001",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 2,
"optime" : {
"ts" : Timestamp(1477383807, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:23:27Z"),
"lastHeartbeat" : ISODate("2016-10-25T08:23:31.304Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T08:23:29.256Z"),
"pingMs" : NumberLong(0),
"configVersion" : 3
}
],
"ok" : 1
}
查看新增加的节点数据
repl:SECONDARY> use test
switched to db test
repl:SECONDARY> db.test.find()
{ "_id" : ObjectId("580f0b3afdcbdd0ab706f021"), "a" : "1" }
{ "_id" : ObjectId("580f0d59fdcbdd0ab706f022"), "b" : "2" }
{ "_id" : ObjectId("580f0ebe5447a2d906013ca9"), "c" : "3" }
{ "_id" : ObjectId("580f12185447a2d906013caa"), "c" : "3" }
{ "_id" : ObjectId("580f16085447a2d906013cab"), "d" : "4" }
在增加节点的时候不要执行priority
,如果执行了,节点会变成仲裁节点。
repl:PRIMARY> rs.add('192.168.56.5:20001','priority:1')
{ "ok" : 1 }
repl:PRIMARY> rs.status()
{
"set" : "repl",
"date" : ISODate("2016-10-25T08:39:58.509Z"),
"myState" : 1,
"term" : NumberLong(4),
"heartbeatIntervalMillis" : NumberLong(2000),
"members" : [
{
"_id" : 0,
"name" : "192.168.56.4:20000",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 3111,
"optime" : {
"ts" : Timestamp(1477384788, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:39:48Z"),
"electionTime" : Timestamp(1477381758, 1),
"electionDate" : ISODate("2016-10-25T07:49:18Z"),
"configVersion" : 15,
"self" : true
},
{
"_id" : 4,
"name" : "192.168.56.6:20002",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 309,
"optime" : {
"ts" : Timestamp(1477384788, 1),
"t" : NumberLong(4)
},
"optimeDate" : ISODate("2016-10-25T08:39:48Z"),
"lastHeartbeat" : ISODate("2016-10-25T08:39:58.231Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T08:39:58.292Z"),
"pingMs" : NumberLong(1),
"syncingTo" : "192.168.56.4:20000",
"configVersion" : 15
},
{
"_id" : 5,
"name" : "192.168.56.5:20001",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 8,
"lastHeartbeat" : ISODate("2016-10-25T08:39:58.290Z"),
"lastHeartbeatRecv" : ISODate("2016-10-25T08:39:58.326Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "192.168.56.6:20002",
"configVersion" : 15
}
],
"ok" : 1
}
repl:SECONDARY>
> use test;
switched to db test
> db.test.find()
2016-10-23T22:17:37.106+0800 I NETWORK [thread1] trying reconnect to 192.168.56.5:20001 (192.168.56.5) failed
2016-10-23T22:17:37.106+0800 I NETWORK [thread1] reconnect 192.168.56.5:20001 (192.168.56.5) ok
Error: error: { "ok" : 0, "errmsg" : "node is recovering", "code" : 13436 }
> db.test.find()
Error: error: { "ok" : 0, "errmsg" : "node is recovering", "code" : 13436 }
尽量不要使用仲裁节点,仲裁节点的作用是,如果当两个优先值一样的时候,仲裁节点进行选举,快速选择出主节点。