A new version of search key and metadata encoding

[PragmaTwice]

Search before asking

• I had searched in the issues and found no similar issues.

Motivation

AS IS: (InernalKey encoding)
(key is omitted, actually it's InternalKey namespace | (slot num) | index name)

search data type metadata:

key -> metadata flag | expire | version | size | on_data_type (HASH or JSON)

prefixes encoding:

key | version | PREFIXES -> prefix num | prefix1 prefix2 ...

tag field metadata encoding:

key | version | TAG_FIELD_META | field name -> field flag | separator | case sensitive

tag field index encoding:

key | version | TAG_FIELD | field name | tag | key -> (nil)

numeric field metadata encoding:

key | version | NUM_FIELD_META | field name -> field flag

numeric field index encoding:

key | version | NUM_FIELD | field name | ordered floating number | key -> (nil)

field flag:

|              8 bit              |
|---------------------------------|
|  noindex: 1bit | reserved: 7bit |

TO BE: (new encoding)

search data type metadata:

ns | INDEX_META | index name ->  index flag | on_data_type (HASH or JSON)

prefixes encoding:

ns | PREFIXES | index name -> prefix num | prefix1 prefix2 ...

field alias encoding:

ns | FIELD_ALIAS | index name | alias name -> field name

tag field metadata encoding:

ns | FIELD_META | index name | field name -> field flag | separator | case sensitive

numeric field metadata encoding:

ns | FIELD_META | index name | field name -> field flag

tag field index encoding:

ns | FIELD | index name | field name | tag | key -> (nil)

numeric field index encoding:

ns | FIELD | index name | field name | ordered floating number | key -> (nil)

field flag:

|              8 bit                                 |
|----------------------------------------------------|
|  noindex: 1bit | field type: 4bit | reserved: 3bit |

field type:
TAG     1
NUMERIC 2

benefits:

• we can iterate all index metadata via a rocksdb iterator
• remove useless fields in index metadata
• field name is unique regardless of field type
• prefix compression can work better
• field type stored in metadata instead of key

Solution

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!