Skip to content

perf: speed up generated encode paths #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shamaton merged 3 commits into from
Jun 7, 2026
Merged

perf: speed up generated encode paths #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1e1f83c
perf: speed up generated encode paths
shamaton Jun 5, 2026
71deae4
chore: update README
shamaton Jun 7, 2026
f9f3867
ci: add go 1.26 to test targets
shamaton Jun 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,14 @@ env:
WORKSPACE: ${{ github.workspace }}/src/github.com/${{ github.repository }}

jobs:

test:
defaults:
run:
working-directory: ${{ env.WORKSPACE }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
go: [1.23, 1.24, 1.25]
go: [1.23, 1.24, 1.25, 1.26]
name: ${{ matrix.os }} @ Go ${{ matrix.go }}
runs-on: ${{ matrix.os }}
steps:
Expand Down Expand Up @@ -44,7 +43,7 @@ jobs:
run: sh testdata/test.sh

- name: Upload coverage to Codecov
if: success() && matrix.go == 1.25 && matrix.os == 'windows-latest'
if: success() && matrix.go == 1.26 && matrix.os == 'windows-latest'
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
with:
token:
Expand Down
88 changes: 46 additions & 42 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@

**msgpackgen** provides a code generation tool and serialization library for [MessagePack](http://msgpack.org/).

* 🚀 Extremely Fast
* ♻️ Easy Maintenance
* 💯 Compliant with [specifications](https://github.com/msgpack/msgpack/blob/master/spec.md)
* 🦺 No use `unsafe`
- 🚀 Extremely Fast
- ♻️ Easy Maintenance
- 💯 Compliant with [specifications](https://github.com/msgpack/msgpack/blob/master/spec.md)
- 🦺 No use `unsafe`

## Quickstart

Expand Down Expand Up @@ -95,17 +95,17 @@ struct: `time.Time` and structures you defined

Renaming or omitting are available.

* Renaming fields via `msgpack:"field_name"`
* Omitting fields via `msgpack:"-"`
* Omitting zero-value fields from map encoding via `msgpack:",omitempty"` or `msgpack:"field_name,omitempty"`
- Renaming fields via `msgpack:"field_name"`
- Omitting fields via `msgpack:"-"`
- Omitting zero-value fields from map encoding via `msgpack:",omitempty"` or `msgpack:"field_name,omitempty"`

For example:

```go
type Example struct {
ID int
Name string `msgpack:"name,omitempty"`
Extra string `msgpack:",omitempty"`
ID int
Name string `msgpack:"name,omitempty"`
Extra string `msgpack:",omitempty"`
}
```

Expand All @@ -132,9 +132,9 @@ All you have to delete is one generated `.go` file.
Code is generated by recursively searching directories,
but some directories and files are ignored.

* Prefix `_` and `.` directory.
* `testdata` and `vendor` directory
* `_test.go` file
- Prefix `_` and `.` directory.
- `testdata` and `vendor` directory
- `_test.go` file

If you use `-input-file` option, it will work without considering the above conditions.

Expand All @@ -144,9 +144,9 @@ Compatible with various import rules.

```go
import (
"example.com/user/a/b"
d "example.com/user/a/c"
. "example.com/user/a/e"
"example.com/user/a/b"
d "example.com/user/a/c"
. "example.com/user/a/e"
)
```

Expand All @@ -157,31 +157,31 @@ Not generated in the following cases:
```go
// ex. a/example.go
type Example struct {
// unsupported types
Interface any
Uintptr uintptr
Error error
Chan chan
Func func()
// unsupported types
Interface any
Uintptr uintptr
Error error
Chan chan
Func func()

// nested struct is also unsupported
NestedStruct struct {}
// nested struct is also unsupported
NestedStruct struct {}

// because b.Example is not generated
B b.Example
// because b.Example is not generated
B b.Example

// because bytes.Buffer is in outside package
Buf bytes.Buffer
// because bytes.Buffer is in outside package
Buf bytes.Buffer
}

func (e Example) F() {
// unsupported struct defined in func
type InFunction struct {}
// unsupported struct defined in func
type InFunction struct {}
}

// ex a/b/example.go
type Example struct {
Interface any
Interface any
}
```

Expand Down Expand Up @@ -225,14 +225,18 @@ Our package results show in `ShamatonGen`.

![Benchmark Decode][benchmark-decode-image]

[go-reference-badge]:https://pkg.go.dev/badge/github.com/shamaton/msgpackgen.svg
[go-reference]:https://pkg.go.dev/github.com/shamaton/msgpackgen
[test-badge]:https://github.com/shamaton/msgpackgen/workflows/test/badge.svg
[go-report-badge]:https://goreportcard.com/badge/github.com/shamaton/msgpackgen
[go-report]:https://goreportcard.com/report/github.com/shamaton/msgpackgen
[codecov-badge]:https://codecov.io/gh/shamaton/msgpackgen/branch/main/graph/badge.svg?token=K7M3778X7C
[codecov]:https://codecov.io/gh/shamaton/msgpackgen
[fossa-badge]:https://app.fossa.com/api/projects/git%2Bgithub.com%2Fshamaton%2Fmsgpackgen.svg?type=shield
[fossa]:https://app.fossa.com/projects/git%2Bgithub.com%2Fshamaton%2Fmsgpackgen?ref=badge_shield
[benchmark-encode-image]:https://raw.githubusercontent.com/shamaton/msgpack_bench/refs/heads/main/docs/benchmarks/compare-encode.svg
[benchmark-decode-image]:https://raw.githubusercontent.com/shamaton/msgpack_bench/refs/heads/main/docs/benchmarks/compare-decode.svg
## License

This library is under the MIT License. See the [LICENSE](./LICENSE) file for details.

[go-reference-badge]: https://pkg.go.dev/badge/github.com/shamaton/msgpackgen.svg
[go-reference]: https://pkg.go.dev/github.com/shamaton/msgpackgen
[test-badge]: https://github.com/shamaton/msgpackgen/workflows/test/badge.svg
[go-report-badge]: https://goreportcard.com/badge/github.com/shamaton/msgpackgen
[go-report]: https://goreportcard.com/report/github.com/shamaton/msgpackgen
[codecov-badge]: https://codecov.io/gh/shamaton/msgpackgen/branch/main/graph/badge.svg?token=K7M3778X7C
[codecov]: https://codecov.io/gh/shamaton/msgpackgen
[fossa-badge]: https://app.fossa.com/api/projects/git%2Bgithub.com%2Fshamaton%2Fmsgpackgen.svg?type=shield
[fossa]: https://app.fossa.com/projects/git%2Bgithub.com%2Fshamaton%2Fmsgpackgen?ref=badge_shield
[benchmark-encode-image]: https://raw.githubusercontent.com/shamaton/msgpack_bench/refs/heads/main/docs/benchmarks/compare-encode.svg
[benchmark-decode-image]: https://raw.githubusercontent.com/shamaton/msgpack_bench/refs/heads/main/docs/benchmarks/compare-decode.svg
12 changes: 8 additions & 4 deletions internal/generator/gen.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -492,8 +492,10 @@ func (g *generator) publicEncodeReturn(privateFuncName, fallbackFuncName string)
}
if !g.strict {
code = append(code,
If(Qual("errors", "Is").Call(Err(), Qual(ptn.PkTop, "ErrUndefinedType"))).Block(
Return(Qual(ptn.PkFallback, fallbackFuncName).Call(Id("v"))),
If(Err().Op("!=").Nil()).Block(
If(Qual("errors", "Is").Call(Err(), Qual(ptn.PkTop, "ErrUndefinedType"))).Block(
Return(Qual(ptn.PkFallback, fallbackFuncName).Call(Id("v"))),
),
),
)
}
Expand All @@ -506,8 +508,10 @@ func (g *generator) publicDecodeReturn(privateFuncName, fallbackFuncName string)
}
if !g.strict {
code = append(code,
If(Qual("errors", "Is").Call(Err(), Qual(ptn.PkTop, "ErrUndefinedType"))).Block(
Return(Qual(ptn.PkFallback, fallbackFuncName).Call(Id("data"), Id("v"))),
If(Err().Op("!=").Nil()).Block(
If(Qual("errors", "Is").Call(Err(), Qual(ptn.PkTop, "ErrUndefinedType"))).Block(
Return(Qual(ptn.PkFallback, fallbackFuncName).Call(Id("data"), Id("v"))),
),
),
)
}
Expand Down
17 changes: 15 additions & 2 deletions internal/generator/structure/code_array.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,11 @@ func (st *Structure) createArrayCode(node *Node, encodeFieldName, decodeFieldNam

func (g arrayCodeGen) createCalcCode(fieldName, childName string, isChildByte, passChildPointer bool, elmCodes []Code) []Code {
blockCodes := createAddSizeErrCheckCode("CalcSliceLength", Len(Id(fieldName)), Lit(isChildByte))
blockCodes = append(blockCodes, createSequenceRangeCode(fieldName, childName, passChildPointer, elmCodes))
if isChildByte {
blockCodes = append(blockCodes, Id("size").Op("+=").Len(Id(fieldName)))
} else {
blockCodes = append(blockCodes, createSequenceRangeCode(fieldName, childName, passChildPointer, elmCodes))
}

codes := make([]Code, 0)
codes = append(codes, Block(
Expand All @@ -58,7 +62,16 @@ func (g arrayCodeGen) createEncCode(fieldName, childName string, isChildByte, pa

blockCodes := make([]Code, 0)
blockCodes = append(blockCodes, Id("offset").Op("=").Qual(ptn.PkEnc, "WriteSliceLength").Call(Id("buf"), Len(Id(fieldName)), Id("offset"), Lit(isChildByte)))
blockCodes = append(blockCodes, createSequenceRangeCode(fieldName, childName, passChildPointer, elmCodes))
if isChildByte {
blockCodes = append(blockCodes,
Id("offset").Op("+=").Id("copy").Call(
Id("buf").Index(Id("offset").Op(":").Id("offset").Op("+").Len(Id(fieldName))),
Id(fieldName).Index(Op(":")),
),
)
} else {
blockCodes = append(blockCodes, createSequenceRangeCode(fieldName, childName, passChildPointer, elmCodes))
}

codes := make([]Code, 0)
codes = append(codes, Block(
Expand Down
17 changes: 15 additions & 2 deletions internal/generator/structure/code_slice.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,11 @@ func (st *Structure) createSliceCode(node *Node, encodeFieldName, decodeFieldNam
func (g sliceCodeGen) createCalcCode(fieldName, childName string, isChildTypeByte, passChildPointer bool, elmCodes []Code) []Code {

blockCodes := createAddSizeErrCheckCode("CalcSliceLength", Len(Id(fieldName)), Lit(isChildTypeByte))
blockCodes = append(blockCodes, createSequenceRangeCode(fieldName, childName, passChildPointer, elmCodes))
if isChildTypeByte {
blockCodes = append(blockCodes, Id("size").Op("+=").Len(Id(fieldName)))
} else {
blockCodes = append(blockCodes, createSequenceRangeCode(fieldName, childName, passChildPointer, elmCodes))
}

codes := make([]Code, 0)
codes = append(codes, If(Id(fieldName).Op("!=").Nil()).Block(
Expand All @@ -57,7 +61,16 @@ func (g sliceCodeGen) createEncCode(fieldName, childName string, isChildTypeByte

blockCodes := make([]Code, 0)
blockCodes = append(blockCodes, Id("offset").Op("=").Qual(ptn.PkEnc, "WriteSliceLength").Call(Id("buf"), Len(Id(fieldName)), Id("offset"), Lit(isChildTypeByte)))
blockCodes = append(blockCodes, createSequenceRangeCode(fieldName, childName, passChildPointer, elmCodes))
if isChildTypeByte {
blockCodes = append(blockCodes,
Id("offset").Op("+=").Id("copy").Call(
Id("buf").Index(Id("offset").Op(":").Id("offset").Op("+").Len(Id(fieldName))),
Id(fieldName),
),
)
} else {
blockCodes = append(blockCodes, createSequenceRangeCode(fieldName, childName, passChildPointer, elmCodes))
}

codes := make([]Code, 0)
codes = append(codes, If(Id(fieldName).Op("!=").Nil()).Block(
Expand Down
Loading