[curvefs] Optimize the implementation of curvefs rename to improve rename performance

[cw123]

In the current rename implementation of CurveFS, we guarantee the atomicity of this operation, but there are still the following two problems:

In the multi-mount scenario, in order to ensure the correctness of txid, we add a distributed lock to the MDS to ensure that all transactions are executed serially, which seriously affects the performance of the rename interface, that is, transactions cannot be concurrent implement.

Similarly, in order to ensure that other operation interfaces (such as GetDentry, DeleteDentry, etc.) can get the correct version of dentry, it is necessary to go to the MDS to obtain the latest txid before these interfaces are executed. overall metadata performance

在目前 CurveFS 的 rename 实现中，我们保证了该操作的原子性，但是仍然存在以下 2 个问题：

在多挂载的场景下，我们为了保证 txid 的正确性，在 MDS 中加了一把分布式锁来保证所有的事务都是串行执行，这严重影响了 rename 接口的性能，即事务不能并发执行。

同样地，为了保证其他操作接口（如 GetDentry、DeleteDentry 等）能拿到正确版本的 dentry，在这些接口执行前都要去 MDS 获取最新的 txid，对于这些接口来说，多了一次 RPC 请求，降低了整体的元数据性能

Solution:
Through research, the Google Percolator transaction model and existing open source implementations can provide us with a reference to solve the above two problems.

解决方法：
通过调研， Google Percolator 事务模型以及现有的开源实现可以为我们解决上述两个问题提供参考。

Large-scale Incremental Processing Using Distributed Transactions and Notifications
Google Percolator 的事务模型

We have done research here and wrote a design document, but it has not yet been implemented. You can refer to the design of this design document.

我们这里经过调研，写了一篇设计文档，但是还未来得及实现，可以参考这篇设计文档的设计。

curvefs_improve_rename_design.md

Optimization point 1: Concurrent Prewrite

The prewrite in the paper is performed in sequence, and we can push all keys to be performed concurrently to improve performance

Optimization point 2: Remove the process of obtaining timestamps for each read operation

Each acquisition in the paper also needs to acquire a Tso, which will increase an RPC time for our implementation. In fact, this Tso is redundant. We only need to acquire the latest data of the current key without lock, and there are If the lock is required, the previous transaction needs to be pushed.

优化点 1：并发 Prewrite

论文中的 prewrite 是序列进行的，而我们可以推动所有的键并发进行，提高性能

优化点 2：去除每次读取操作获取时间戳的流程

论文中的每一个获取也需要获取一个 Tso，这对于我们的实现来说会增加一个 RPC 时间，而其实这个 Tso 是多余的，我们只需要在无锁获取当前 key 的最新数据即可，而有锁的话则需要推动前一个事务。

[WyattJia]

assign to me

[wuhongsong]

@WyattJia @Wine93 Are you having some trouble? If so, please let us know so that we can communicate together

[wuhongsong]

@WyattJia @Wine93

[Tom-CaoZH]

Hello, I want to ask if this issue still needs help? If so, maybe I can have a try.

[ilixiaocui]

Hello, I want to ask if this issue still needs help? If so, maybe I can have a try.

cc @Wine93

[Wine93]

Hi @Tom-CaoZH , have you joined our WeChat group?

[Tom-CaoZH]

I have joined the user WeChat group but not the contributor group.

[Tom-CaoZH]

Hi, in the last two days, I have done two things. First, I read the related code about rename and found that maybe I need to modify the code in FuseClient::FuseOpRename function. Second, I read the design doc and combine with the code, I think maybe I need to add serveral functions(maybe the functions in the design doc) in RenameOperator to implement this feature. I want to verify whether my direction is right? If so, next I will begin to write the code. BTW, I am a little busy this week, so this process maybe a little slow, but I will try my best to do it and update timely with your guys.

[Wine93]

Hi, @Tom-CaoZH，glad you're interested in it and took action on it.

I think maybe I need to add serveral functions(maybe the functions in the design doc) in RenameOperator to implement this feature. I want to verify whether my direction is right?

yep, reading source code and design doc is essential.

If so, next I will begin to write the code.

I think you can simulate each handler step for rename() in you mind or write it on your paper before writing code, especially various abnormal scenarios. our solution is a 2PC implementation for transaction, if you simulate each various abnormal scenarios and can still keep the rename operation atomic, do it :)

BTW, I am a little busy this week, so this process maybe a little slow, but I will try my best to do it and update timely with your guys

NO PROBLEM! if you need any help, please tell us :)

[Cyber-SiKu]

@Tom-CaoZH Is this still going on?

[Tom-CaoZH]

I am sorry, due to my final exam, I need to take a break from the project for a little while(strill needs one week). I am committed to completing my tasks as soon as I am able to return. Also, if this issue is urgent and another contributor is interested in working on this issue, they are more than welcome to complete it when I was leaving.

[caoxianfei1]

@Tom-CaoZH feel free to continue but the quicker the better. 💯