Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stdlibs): add encoding, encoding/{base32,binary,csv} #1290

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
a58c9e0
feat: add encoding stdlib
notJoon Oct 25, 2023
a96c6ff
csv and varint
notJoon Oct 25, 2023
8a33d7f
create base32 and tests
notJoon Oct 26, 2023
7415d31
remove `fallthrough`
notJoon Oct 26, 2023
6a4d328
rewrite test for `varint`
notJoon Oct 27, 2023
7355668
update test and add some en/decode cases
notJoon Oct 31, 2023
1de0a69
porting base32 fin
notJoon Oct 31, 2023
febc76f
fix
notJoon Nov 12, 2023
ed78d3d
fix test
notJoon Nov 12, 2023
5d55e29
specifying package scope
notJoon Nov 12, 2023
e863269
type
notJoon Nov 12, 2023
d1ff5f1
another typo
notJoon Nov 12, 2023
de17d0e
nopadding
notJoon Nov 12, 2023
7b8c192
fix
notJoon Nov 12, 2023
eef4d2c
fix
notJoon Nov 12, 2023
62bb9c4
wip
notJoon Nov 12, 2023
62b297c
remove reflect on csv test
notJoon Nov 12, 2023
d69298a
fix
notJoon Nov 12, 2023
6abba1a
fix
notJoon Nov 13, 2023
97aa8a0
fix
notJoon Nov 13, 2023
30325f6
add simple reader test
notJoon Nov 13, 2023
0b63206
fmt
notJoon Nov 13, 2023
7ea23dc
Merge branch 'master' into add-encoding
notJoon Nov 13, 2023
ef756d6
package bufio -> bufio_test
thehowl Nov 16, 2023
d861226
fixes to pkg base32
thehowl Nov 16, 2023
ae61fca
rollback some changes
thehowl Nov 16, 2023
8b61830
revert chagnes in varint
thehowl Nov 16, 2023
bac1c37
csv fix-up
thehowl Nov 16, 2023
8efeb0d
fmt
thehowl Nov 16, 2023
2080c5d
Merge branch 'master' of github.com:gnolang/gno into add-encoding
thehowl Dec 8, 2024
a93e4c0
attempt reverting base32 to old state; side-quested on different issue
thehowl Dec 8, 2024
abddb96
fix(gnolang): use strconv.UnquoteChar to parse rune literals
thehowl Dec 8, 2024
25687c8
Merge branch 'dev/morgan/no-strconv-unquote' into add-encoding
thehowl Dec 8, 2024
f2ff51f
fixup
thehowl Dec 8, 2024
1fb863e
fix(gnovm): in op_binary, return typed booleans where appropriate
thehowl Dec 8, 2024
e92c78c
Merge branch 'dev/morgan/runtime-typed-bools' into add-encoding
thehowl Dec 8, 2024
183a1c1
fixup
thehowl Dec 8, 2024
9c28fbe
fmt
thehowl Dec 8, 2024
49a90df
update generated.go
thehowl Dec 8, 2024
fd5565d
Merge branch 'master' into add-encoding
thehowl Dec 8, 2024
0b20f19
Merge branch 'master' into add-encoding
thehowl Dec 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
586 changes: 586 additions & 0 deletions gnovm/stdlibs/encoding/base32/base32.gno
thehowl marked this conversation as resolved.
Show resolved Hide resolved

Large diffs are not rendered by default.

833 changes: 833 additions & 0 deletions gnovm/stdlibs/encoding/base32/base32_test.gno
thehowl marked this conversation as resolved.
Show resolved Hide resolved

Large diffs are not rendered by default.

166 changes: 166 additions & 0 deletions gnovm/stdlibs/encoding/binary/varint.gno
thehowl marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package binary

// This file implements "varint" encoding of 64-bit integers.
// The encoding is:
// - unsigned integers are serialized 7 bits at a time, starting with the
// least significant bits
// - the most significant bit (msb) in each output byte indicates if there
// is a continuation byte (msb = 1)
// - signed integers are mapped to unsigned integers using "zig-zag"
// encoding: Positive values x are written as 2*x + 0, negative values
// are written as 2*(^x) + 1; that is, negative numbers are complemented
// and whether to complement is encoded in bit 0.
//
// Design note:
// At most 10 bytes are needed for 64-bit values. The encoding could
// be more dense: a full 64-bit value needs an extra byte just to hold bit 63.
// Instead, the msb of the previous byte could be used to hold bit 63 since we
// know there can't be more than 64 bits. This is a trivial improvement and
// would reduce the maximum encoding length to 9 bytes. However, it breaks the
// invariant that the msb is always the "continuation bit" and thus makes the
// format incompatible with a varint encoding for larger numbers (say 128-bit).

import (
"errors"
"io"
)

// MaxVarintLenN is the maximum length of a varint-encoded N-bit integer.
const (
MaxVarintLen16 = 3
MaxVarintLen32 = 5
MaxVarintLen64 = 10
)

// AppendUvarint appends the varint-encoded form of x,
// as generated by PutUvarint, to buf and returns the extended buffer.
func AppendUvarint(buf []byte, x uint64) []byte {
for x >= 0x80 {
buf = append(buf, byte(x)|0x80)
x >>= 7
}
return append(buf, byte(x))
}

// PutUvarint encodes a uint64 into buf and returns the number of bytes written.
// If the buffer is too small, PutUvarint will panic.
func PutUvarint(buf []byte, x uint64) int {
i := 0
for x >= 0x80 {
buf[i] = byte(x) | 0x80
x >>= 7
i++
}
buf[i] = byte(x)
return i + 1
}

// Uvarint decodes a uint64 from buf and returns that value and the
// number of bytes read (> 0). If an error occurred, the value is 0
// and the number of bytes n is <= 0 meaning:
//
// n == 0: buf too small
// n < 0: value larger than 64 bits (overflow)
// and -n is the number of bytes read
func Uvarint(buf []byte) (uint64, int) {
var x uint64
var s uint
for i, b := range buf {
if i == MaxVarintLen64 {
// Catch byte reads past MaxVarintLen64.
// See issue https://golang.org/issues/41185
return 0, -(i + 1) // overflow
}
if b < 0x80 {
if i == MaxVarintLen64-1 && b > 1 {
return 0, -(i + 1) // overflow
}
return x | uint64(b)<<s, i + 1
}
x |= uint64(b&0x7f) << s
s += 7
}
return 0, 0
}

// AppendVarint appends the varint-encoded form of x,
// as generated by PutVarint, to buf and returns the extended buffer.
func AppendVarint(buf []byte, x int64) []byte {
ux := uint64(x) << 1
if x < 0 {
ux = ^ux
}
return AppendUvarint(buf, ux)
}

// PutVarint encodes an int64 into buf and returns the number of bytes written.
// If the buffer is too small, PutVarint will panic.
func PutVarint(buf []byte, x int64) int {
ux := uint64(x) << 1
if x < 0 {
ux = ^ux
}
return PutUvarint(buf, ux)
}

// Varint decodes an int64 from buf and returns that value and the
// number of bytes read (> 0). If an error occurred, the value is 0
// and the number of bytes n is <= 0 with the following meaning:
//
// n == 0: buf too small
// n < 0: value larger than 64 bits (overflow)
// and -n is the number of bytes read
func Varint(buf []byte) (int64, int) {
ux, n := Uvarint(buf) // ok to continue in presence of error
x := int64(ux >> 1)
if ux&1 != 0 {
x = ^x
}
return x, n
}

var errOverflow = errors.New("binary: varint overflows a 64-bit integer")

// ReadUvarint reads an encoded unsigned integer from r and returns it as a uint64.
// The error is EOF only if no bytes were read.
// If an EOF happens after reading some but not all the bytes,
// ReadUvarint returns io.ErrUnexpectedEOF.
func ReadUvarint(r io.ByteReader) (uint64, error) {
var x uint64
var s uint
for i := 0; i < MaxVarintLen64; i++ {
b, err := r.ReadByte()
if err != nil {
if i > 0 && err == io.EOF {
err = io.ErrUnexpectedEOF
}
return x, err
}
if b < 0x80 {
if i == MaxVarintLen64-1 && b > 1 {
return x, errOverflow
}
return x | uint64(b)<<s, nil
}
x |= uint64(b&0x7f) << s
s += 7
}
return x, errOverflow
}

// ReadVarint reads an encoded signed integer from r and returns it as an int64.
// The error is EOF only if no bytes were read.
// If an EOF happens after reading some but not all the bytes,
// ReadVarint returns io.ErrUnexpectedEOF.
func ReadVarint(r io.ByteReader) (int64, error) {
ux, err := ReadUvarint(r) // ok to continue in presence of error
x := int64(ux >> 1)
if ux&1 != 0 {
x = ^x
}
return x, err
}
Loading
Loading