利用 snapshot 进行集群迁移

pczhy
发布于 2022-9-22 11:05
浏览
0收藏

利用 snapshot 进行集群迁移-鸿蒙开发者社区

Nebula Graph 提供快照(snapshot)功能,用于保存集群当前时间点的数据状态,当出现数据丢失或误操作时,可以通过快照恢复数据。

那是否可以通过 snapshot 进行数据迁移呢?本文就为大家带来利用 snapshot 进行集群迁移的实践。

>>>>

前置条件

  • 源集群与新集群同构(节点数量,磁盘数量,数据目录一致)
  • 新集群 Nebula 已部署(无需启动服务)

本次实操以三节点集群为例:源集群(192.168.8.191,192.168.8.192,192.168.8.193)、新集群(192.168.8.91,192.168.8.92,192.168.8.93)。

>>>>

操作步骤

在源集群创建 snapshot

(root@nebula) [(none)]> CREATE SNAPSHOT;
Execution succeeded (Time spent: 5.1133/5.11391 s)

Tue Nov 30 04:05:08 2021

(root@nebula) [(none)]> SHOW SNAPSHOTS;
=========================================================================================================
| Name                         | Status | Hosts                                                         |
=========================================================================================================
| SNAPSHOT_2021_11_30_04_05_03 | VALID  | 192.168.8.191:44500, 192.168.8.192:44500, 192.168.8.193:44500 |
---------------------------------------------------------------------------------------------------------
Got 1 rows (Time spent: 1.24/1.875 ms)

Tue Nov 30 04:05:34 2021




拷贝文件

# 拷贝 meta 文件
-- ⚠️snapshot 只会在 meta leader 节点生成 checkpoints 目录,这里是 192.168.8.191,需要拷贝至新集群所有 meta 节点
-- 在 192.168.8.91,92,93 执行以下命令
$ cd /usr/local/nebula/data/meta/nebula/0/
$ sudo scp -r vesoft@192.168.8.191:/usr/local/nebula/data/meta/nebula/0/checkpoints/SNAPSHOT_2021_11_30_04_05_03/* ./

# 拷贝 storage 文件
-- ⚠️目录中 14 为对应 space 的 ID,192.168.8.191,192,193 -> 192.168.8.91,92,93 一对一拷贝
-- 192.168.8.91
$ cd /usr/local/nebula/data/storage/nebula/
$ sudo mkdir 14
$ cd 14
$ sudo scp -r vesoft@192.168.8.191:/usr/local/nebula/data/storage/nebula/14/checkpoints/SNAPSHOT_2021_11_30_04_05_03/* ./

-- 192.168.8.92
$ cd /usr/local/nebula/data/storage/nebula/
$ sudo mkdir 14
$ cd 14
$ sudo scp -r vesoft@192.168.8.192:/usr/local/nebula/data/storage/nebula/14/checkpoints/SNAPSHOT_2021_11_30_04_05_03/* ./

-- 192.168.8.93
$ cd /usr/local/nebula/data/storage/nebula/
$ sudo mkdir 14
$ cd 14
$ sudo scp -r vesoft@192.168.8.193:/usr/local/nebula/data/storage/nebula/14/checkpoints/SNAPSHOT_2021_11_30_04_05_03/* ./


新集群启动 meta 和 graph 服务

$ sudo /usr/local/nebula/scripts/nebula.service start metad
$ sudo /usr/local/nebula/scripts/nebula.service start graphd


查看新集群 meta leader 节点

# 查看 meta 日志,如果看到“The partition is elected as the leader”说明改节点为 leader
$ grep "leader" /usr/local/nebula/logs/nebula-metad.INFO
I1130 08:45:48.413197 26299 RaftPart.cpp:1179] [Port: 45501, Space: 0, Part: 0] The partition is elected as the leader


修改 meta 信息

# 192.168.8.194 为上一步查到的 meta leader 节点,from 和 to 参数分别对应源 IP 和目标 IP。如果返回“Replace Host successfully”即为成功。
-- ⚠️这里使用的是 1.2 版本,http 端口为 11000,如果是 2.x 版本端口为 19559
curl -G "http://192.168.8.194:11000/replace?from=192.168.8.191&to=192.168.8.91"
curl -G "http://192.168.8.194:11000/replace?from=192.168.8.192&to=192.168.8.92"
curl -G "http://192.168.8.194:11000/replace?from=192.168.8.193&to=192.168.8.93"


新集群启动 storage 服务

$ sudo /usr/local/nebula/scripts/nebula.service start storaged


检查服务是否正常

# show hosts 查看服务和 partition 分布
(root@nebula) [basketballplayer]> show hosts;
==================================================================================================
| Ip            | Port  | Status  | Leader count | Leader distribution  | Partition distribution |
==================================================================================================
| 192.168.8.91  | 44500 | online  | 30           | basketballplayer: 30 | basketballplayer: 30   |
--------------------------------------------------------------------------------------------------
| 192.168.8.92  | 44500 | online  | 0            |                      | basketballplayer: 30   |
--------------------------------------------------------------------------------------------------
| 192.168.8.93  | 44500 | online  | 0            |                      | basketballplayer: 30   |
--------------------------------------------------------------------------------------------------
| Total         |       |         | 30           | basketballplayer: 30 | basketballplayer: 90   |
--------------------------------------------------------------------------------------------------
Got 8 rows (Time spent: 1.298/2.199 ms)

Tue Nov 30 09:09:58 2021

# 如果 leader 分布不均匀,手工 balance
(root@nebula) [basketballplayer]> balance leader;
Execution succeeded (Time spent: 5.02013/5.0213 s)

Tue Nov 30 09:13:31 2021

# 检查业务数据
(root@nebula) [basketballplayer]> fetch prop on player 100;
=======================================
| VertexID | player.name | player.age |
=======================================
| 100      | Tim Duncan  | 42         |
---------------------------------------
Got 1 rows (Time spent: 9.689/10.488 ms)

Tue Nov 30 09:09:55 2021

以上便是实操的全部过程。最后提醒下各位,Nebula Graph 的身份认证功能默认是关闭的,此时任何用户都能使用快照功能。如果身份认证开启,仅 God 角色用户可以使用快照功能。

分类
已于2022-9-22 11:05:23修改
收藏
回复
举报
回复
    相关推荐