You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While crossbeam_skiplist::SkipMap and std::collections::BTreeMap are solid choices for a general-purpose ordered map, they may not be ideal for a database environment. This is primarily due to its reliance on dynamic memory allocations for each entry, leading to non-contiguous memory layouts. In a database scenario, where performance and efficient memory usage are critical, this can become a bottleneck.
The lack of contiguous memory means that pointer chasing occurs frequently during lookups, increasing cache misses and search times. This added overhead can significantly impact performance when performing large-scale or frequent queries.
In contrast, skl::SkipMap provides a more suitable approach for database environments because it stores entries in a continuous memory buffer, reducing the need for frequent allocations. This leads to better cache locality and faster search operations, making skl::SkipMap a more performant choice in cases where minimizing memory overhead and maximizing search efficiency are crucial, such as in database systems.
Hence, in 0.5, the sidecar of the write-ahead log will be changed to skl::SkipMap.
The text was updated successfully, but these errors were encountered:
In #22, which supports using different memtables, either based on crossbeam-skiplist or skl. However, the performance regression problem on skl has not been solved.
al8n
changed the title
Use skl::SkipMap as the sidecar lookup map instead of crossbeam_skiplist::SkipMap and std::collections::BTreeMap
Support different memtable implementation
Nov 2, 2024
While
crossbeam_skiplist::SkipMap
andstd::collections::BTreeMap
are solid choices for a general-purpose ordered map, they may not be ideal for a database environment. This is primarily due to its reliance on dynamic memory allocations for each entry, leading to non-contiguous memory layouts. In a database scenario, where performance and efficient memory usage are critical, this can become a bottleneck.The lack of contiguous memory means that pointer chasing occurs frequently during lookups, increasing cache misses and search times. This added overhead can significantly impact performance when performing large-scale or frequent queries.
In contrast,
skl::SkipMap
provides a more suitable approach for database environments because it stores entries in a continuous memory buffer, reducing the need for frequent allocations. This leads to better cache locality and faster search operations, makingskl::SkipMap
a more performant choice in cases where minimizing memory overhead and maximizing search efficiency are crucial, such as in database systems.Hence, in
0.5
, the sidecar of the write-ahead log will be changed toskl::SkipMap
.The text was updated successfully, but these errors were encountered: