Redis 7 RC1发布了！

虎年除夕(2022-01-31)，Redis 7.0 RC1发布，这是新的Redis团队开发的第2个版本( 6.2和7.0)，该版本提供了很多新的特性、做了很多改进(尤其是在性能和内存优化上，关键字Significant)，当然也修复了很多小bug，总结下重点如下：

 

• Redis Function：新的服务脚本编程模式
• 内存和性能优化：非用户功能，但对性能和内存优化做了重大改进（搜关键字Significant）
• 新的AOF模式：切片式AOF，减少AOF重写的问题以及提供一些新的功能。
• 使用新的数据结构listpack完全代替ziplist。
• 新的RDB版本：version 10，注意该版本和之前版本不兼容。
• 添加很多新的用户命令和命令选项
• 添加一些info统计信息：尤其是memory方面
• 安全方面在ACL做了一些优化和增强

一、新的特性、演进、变化

1.Function：相比于lua(since 2.8)、module(since 4.0)，Redis Gears(one module)它是一种新的服务端脚本编程方式，解决了以往的一些痛点。(#8693)

 

2.新的AOF模式：将AOF进行切片(譬如按照大小)，主要解决AOF重写对可用性干扰，同时还在日志中添加了元信息（时间戳）可以进行定点恢复(point-in-time recovery )。(#9788)(#9326)

 

这个功能是阿里云团队提供的，在之前开源的ApsaraCache已经引入该特性（此处有误，是百度团队贡献的）

 

3.maxmemory-client配置: 用来限制全部客户端的总内存（防止溢出），此前版本中只能针对单个客户端的输入输出缓冲区进行限制。(#8687)

4.Redis Cluster: 支持hostname配置(之前版本只支持IP)、较好支持pubsub(之前会有消息扩散)、在cluster bus链接管理做了优化防止内存泄露。(#9530) (#8621) (#9774)

5.ACL：更细粒度的基于key的权限控制、基于selectors的命令权限控制。(#9974)

 

6.使用新的数据结构listpack完全代替ziplist：包含encoding返回类型、hash/set/zset内部实现方式和配置等(#8887, #9366, #9740)

 

7.repl-diskless-sync：无盘复制默认为yes，这个需要关注。(#10092)

 

8.config set maxmemory执行后立即返回，不等evict完成，这个对于运维同学需要关注，可能是个坑。(#10019)

 

9.jemalloc升级到5.2.1 (#9623)

二、新的命令

用户命令：

 

• zmpop、bzmpop：按排序批量弹出zset的元素
• lmpop、blmpop:   从左侧批量从list弹出元素
• sintercard、zintercard: 计算交集的集合元素个数
• expiretime、pexpiretime: 返回过期的秒级或者毫秒级时间戳
• spublish、ssubscribe、sunsbuscribe、pubsub shardchannels/shardnumsub：集群相关的pubsub命令
• expire：支持nx/xx/gt/lt选项
• set: 支持nx和get组合
• bitops、bitcount：支持bit/byte选项
• shutdown：添加now、force、abort选项
• config get/set: 支持多个配置
• function相关命令：FUNCTION *, FCALL, FCALL_RO - https://redis.io/commands/function-load
• docs、command-tips:  命令文档和命令tips命令
• quit添加到正式命令中
• eval_ro、evalsha_ro：只读版本的eval、evalsha，在只读从节点执行
• sort_ro：只读版本sort，在只读从节点执行
• xadd: 通过"-*"实现自动序号

三、性能和内存优化

1.集群模式性能和内存优化：使用新的数据结构代替radix tree保存key和slot映射(#9356)   (Significant)

 

2.优化dict内部结构(privdata)实现内存优化：对整体key-value、hash key、zset key效果明显   (Significant)

 

3.设计共享复制缓冲区：减少多个slave复制造成的内存开销 (#9166)

来源：百度的Redis内核同学，Redis 7.0 共享复制缓冲区的设计与实现

 

4.优化copy-on-write内存开销(#8974)  (Significant)

 

5.使用listpack替代ziplist (#8887, #9366, #9740)

 

6.集群模式：优化send buffer中无用的内存开销 (#9255)

 

7.支持list大小超过4GB(#9357)

 

8.更充分的使用cliengt reply buffers空间(#8968)

 

9.去掉multi-bulk limit的hard code，使用INT_MAX代替 (#9528)

 

10.优化fsync相关避免对磁盘产生大量写入。(#9545)

四、新的配置

1.appenddirname：定义分片AOF的目录名 (#9872)

2.shutdown-timeout: 定义shutdown的超时时间，默认10秒 (#9872)

3.maxmemory-clients：上文已经介绍

4.cluster-port：自定义cluster通信端口，默认是+10000 (#9389)

5.list-max-listpack-, hash-max-listpack-, zset-max-listpack-* ：代替ziplist (#8887, #9366, #9740)

6.busy-reply-threshold：lua-time-limit的别名  (#9963)

7.bind-source-addr：针对outgoing connections(outgoing connections (from replica to master, from Sentinel to instances, cluster bus, etc)的bind (#9142)

8.repl-diskless-sync-max-replicas: 控制在full repl-diskless-sync-delay前是否开始复制 (#10092)

9.latency-tracking：是否开启latency追踪，默认是yes，如开启会展示latencystats section (#9462)

10.cluster-announce-hostname and cluster-preferred-endpoint-type：redis cluster支持hostname，默认是ip(#9530)

11.cluster-allow-pubsublocal-when-down：redis cluster是down state时候，是否允许使用pubsub，默认是yes (#8621)

12.cluster-link-sendbuf-limit：限制cluster bus link's send buffer，默认是0，即不限制，可以通过cluster link查询相关状态 (#9774)

 

五、新的Info统计

1.latencystats section (#9462)：展示latency信息

2.total_active_defrag_time and current_active_defrag_time (#9377)：展示碎片整理相关耗时

3.total_eviction_exceeded_time and current_eviction_exceeded_time (#9031)：展示逐出数据相关

4.INFO: evicted_clients：被maxmemory-clients干掉的客户端连接数 (#8687)

5.INFO: mem_cluster_links, total_cluster_links_buffer_limit_exceeded：cluster link's send buffer相关内存 (#9774)

6.current_cow_peak：展示copy-on-write消耗峰值 (#8974)
7.移除aof_rewrite_buffer_length选项 (#9788)
8.MEMORY STATS: 展示集群模式下key slots映射关系数据结构用到的内存量 (#10017)
9.INFO MEMORY: 拆分Functions and EVAL内存统计 (#9780)
10.INFO MEMORY: 添加mem_total_replication_buffers, 改变mem_clients_slaves含义 (#9166)
11.CLIENT LIST: tot-mem, multi-mem，展示总内存 (#8687)
12.CLIENT LIST, INFO: 展示RESP版本 (#9508)
13.SENTINEL INFO: tilt_mode_since：sentinel在tilt的持续时间 (#9000)
14.LATENCY: 展示module-acquire-GIL latency (#9608)

六、一些changes和bug fix

一些changes和bug fix，可能会对当前客户端、周边生态(平台、相关工具)、线上生产环境产生一些问题，由于条目众多只对其中比较重要的进行描述：

 

比较重要的changes:

 

1.repl-diskless-sync：无盘复制默认为yes，这个需要关注。
2.INFO commandstats：支持子命令
3.LPOP/RPOP count在空list执行返回空
4.CONFIG SET maxmemory执行后立即返回，不等evict完成

 

比较重要的bug修复：

 

1.memory usage准确度有进一步提升 (#9095)
2.修复了list-compress-depth的相关bug (#9849, #9779)
3.redis-cli: Fix SCAN sleep interval for --bigkeys, --memkeys, --hotkeys

 

其他changes:

 

• Modifying the bind parameter to a non-default value will no longer implicitly disable protected-mode (#9034)
• Remove EVAL script verbatim replication, propagation, and deterministic execution logic (#9812) This has been deprecated and off by default since Redis 6 and is no longer supported.
• ACL: pub/sub channels are blocked by default (acl-pubsub-default=resetchannels) (#10181)
• SCRIPT LOAD and SCRIPT FLUSH are no longer propagated to replicas / AOF (#9812)
• ACL: Declarations of duplicate ACL users in startup files and command line arguments will result in an error, whereas previously the last declaration would overwrite the others. (#9330)
• Replication: TTLs are always replicated as absolute (not relative) millisecond timestamps (#8474)
• Fixes in handling multi-key commands with expired keys on writable replicas (#9572)
• CONFIG SET maxmemory returns before starting eviction (#10019)
• AOF: The new Multi-Part mechanism stores data as a set of multiple files in a designated folder (#9788)
• Remove STRALGO command, preserve LCS a standalone command which only works on keys (#9799)
• Remove gopher protocol support (#9057)
• MODULE and DEBUG commands disabled (protected) by default, for better security (#9920)
• Snapshot-creating and other admin commands in MULTI/EXEC transactions are now rejected (#10015)
• PING is now rejected with -MASTERDOWN when replica-serve-stale-data=no (#9757)
• ACL GETUSER reply now uses ACL syntax for keys and channels (#9974)
• COMMAND reply drops random and sort-for-scripts flags, which are now part of command tips (#10104)
• LPOP/RPOP with count against non-existing list return null array (#10095)
• INFO commandstats now shows the stats per sub-command (#9504)
• ZPOPMIN/ZPOPMAX used to produce wrong replies when count is 0 with non-zset (#9711)
• LPOP/RPOP used to produce wrong replies when count is 0 (#9692)
• CONFIG GET bind now returns the current value in effect, even if the implicit default is in use (#9034)
• CONFIG REWRITE now rewrites the list of modules to load (#4848)
• Config: repl-diskless-sync is now set to yes by default (#10092)
• When shutting down, Redis can optionally wait for replicas to catch up on the replication link (#9872)
• Most CONFIG SET, REWRITE, RESETSTAT commands are now allowed during loading (#9878)
• READONLY and READWRITE commands are now allowed when loading and on stale replicas (#7425)
• Fix ACL category for SELECT, WAIT, ROLE, LASTSAVE, READONLY, READWRITE, ASKING (#9208)
• RESET is now allowed even when on unauthenticated connections (#9798)
• SCRIPT LOAD is now allowed on stale replicas (#10126)

除此之外还有一些bug修复，就不逐一说明了：

 

• Fix COMMAND GETKEYS on EVAL without keys (#9733)
• Improve MEMORY USAGE with allocator overheads (#9095)
• Unpause clients after manual failover ends instead of waiting for timed (#9676)
• Lua: fix crash on a script call with many arguments, a regression in v6.2.6 (#9809)
• Lua: Use all characters to calculate string hash to prevent hash collisions (#9449)
• Prevent LCS from allocating temp memory over proto-max-bulk-len (#9817)
• Tracking: Make invalidation messages always after command's reply (#9422)
• Cluster: Hide empty replicas from CLUSTER SLOTS responses (#9287)
• CLIENT KILL killed all clients when used with ID of 0 (#9853)
• Fix bugs around lists with list-compress-depth (#9849, #9779)
• Fix one in a blue moon LRU bug in RESTORE, RDB loading, and module API (#9279)
• Reset lazyfreed_objects info field with RESETSTAT, test for stream lazyfree (#8934)
• Fix RDB and list node compression for handling values larger than 4GB (#9776)
• Fix a crash when adding elements larger than 2GB to a Set or Hash (#9916)
• Diskless replication could not count as a change and skip next database SAVE (#9323)
• Fix excessive stream trimming due to an overflow (#10068)
• Safe and organized exit when receiving SIGTERM while loading (#10003)
• Improve EXPIRE TTL overflow detection (#9839)
• Add missed error counting for INFO errorstats (#9646)
• DECRBY LLONG_MIN caused negation overflow (#9577)
• Delay discarding cached master when full synchronization (#9398)
• Fix Stream keyspace notification and persistence triggers in consumer creation and deletion (#9263)
• Fix rank overflow in zset with more than 2B entries (#9249)
• Avoid starting in check-aof / check-rdb / sentinel modes if only the folder name contains that name (#9215, #9176)
• create the log file only after done parsing the entire config file (#6741)
• redis-cli: Fix SCAN sleep interval for --bigkeys, --memkeys, --hotkeys (#9624)
• redis-cli: Fix prompt to show the right DB num and transaction state after RESET (#9096)
• Module API: fix possible propagation bugs in case a module calls CONFIG SET maxmemory outside a command (#10019, #9890)
• Module API: carry through client RESP version to module blocked clients (#9634)
• Module API: release clients blocked on module commands in cluster resharding and down state (#9483)
• Sentinel: Fix availability after master reboot (#9438)
• Sentinel: Fix memory leak with TLS (#9753)
• Sentinel: Fix possible failover due to duplicate zero-port (#9240)
• Sentinel: Fix issues with hostname support (#10146)
• Sentinel: Fix election failures on certain container environments (#10197)

 

七、结束

整体看，对用户侧比较重要的是一些新的命令，对于Redis管理侧，需要关注的是若干个  (Significant)优化、一些不兼容(譬如RDB)、新的AOF兼容性、一些新的配置的最佳实践。总之，个人觉得Redis 7值得期待，按照套路至少等上几个RC以及几个小版本后，使用更加靠谱。

 

后面针对Redis 7会做更详细的介绍：

 

1.内存优化详解
2.新的命令
3.Function
4.新的AOF
5.ziplist的替代者listpack
6.新的管理命令和统计信息
7.新的配置最佳实践

 

文章转自公众号：Redis开发运维实战