-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The slave sync thread model is not reasonable #2637
Comments
Description: Currently, Pika's thread model for consuming Binlog from nodes is as follows:
The issues are: Regarding point 2, using the
|
Is this a regression?
No
Description
当前,Pika从节点消费Binlog部分的线程模型是:
1 取conf文件中的sync-thread-num值,产生 sync-thread-num * 2数量的worker线程,这批worker的前一半会被选取来Apply Binlog,后一半Worker用于Apply DB
2 消费Binlog时,为了保整消费顺序,每个DB的Binlog都确保是同一个worker处理的,此时的worker选取策略是对db_name做hash来得到worker index,从worker vector的前一半中取一个固定的worker
3 某个worker完成Apply Binlog以后,会使用key做hash来取得index,从worker数组的后一半中取得一个worker,提交异步的WriteDB任务
问题在于:
针对1, 用户并不知情pika内部对sync-thread-num乘以了2,是否不太合适,而且这样一来其实用户无法精确控制具体的线程数:比如为了保证WriteDB部分线程数不会太少,该配置项的默认值是6,那么Pika内部就一共有12个Worker,其中前6个用于写Binlog,后6个用于写DB,而在单DB的情况下,前6个worker中有5个是闲置且永远不会被使用的。
针对2,使用db_name做hash来取得index,存在倾斜问题,经过实测,在DB数量为8,且sync-thread-num为8的情况下,根据hash映射:
DB 1,4,7会都绑定到worker 3上;
DB 3,6绑定到worker 0上;
DB0会绑定到worker2;
DB2会绑定到worker7上;
DB5会绑定到worker6上;
(这里说的绑定是指:DB只向该worker线程委托(Schedule)写Binlog的任务)
而worker1,4,5完全闲置,但此时有8个DB,也有8个worker也是专门用于写Binlog的,完全可以每个DB使用一个worker。
这种倾斜不仅会带来资源浪费,更重要的是如果某个DB暂时因为WriteStall阻塞了,这种阻塞可能会被放大(因为共用worker的缘故,会把其他DB的WriteBinlog任务也给阻塞了)
Please provide a link to a minimal reproduction of the bug
No response
Screenshots or videos
No response
Please provide the version you discovered this bug in (check about page for version information)
No response
Anything else?
No response
The text was updated successfully, but these errors were encountered: