Skip to content

Commit

Permalink
Remove fixtures references (#532)
Browse files Browse the repository at this point in the history
  • Loading branch information
smaye81 authored May 4, 2023
1 parent 21e2865 commit ded8791
Show file tree
Hide file tree
Showing 177 changed files with 1,038 additions and 2,607 deletions.
22 changes: 11 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@ Building and testing this project requires `node`, `npm`, `protoc`,
`make`, `git`, and `go`.

- `go` is required for the Twirp transport client compatibility test suite
- `protoc` is required for test fixtures
- `protoc` is required for compiling .proto files.

The entire project can be built by running `make`, but if you do not have all required
tooling installed, you can still build all of the TypeScript packages.


Not all packages are public, or even JavaScript projects. There are also examples,
benchmarks and fixtures in the `packages/` directory. They can be built with their
benchmarks and Protobuf files in the `packages/` directory. They can be built with their
respective Makefiles.


Expand Down Expand Up @@ -92,21 +92,21 @@ run the plugin during testing without invoking protoc.
All protobuf plugins work with `CodeGeneratorRequest` and `CodeGeneratorResponse`
messages (defined in `google/protobuf/compiler/plugin.proto`).

The protocol buffer compiler parses .proto files and creates a `CodeGeneratorRequest`,
then passes it to a plugin. Because invoking protoc during testing is difficult,
we generate a `FileDescriptorSet` in `packages/test-fixtures/all.descriptorset`
ahead of time. The `FileDescriptorSet` contains all information necessary to
create the `CodeGeneratorRequest`s we need for testing the plugin.
A protocol buffer compiler parses .proto files and creates a `CodeGeneratorRequest`,
then passes it to a plugin. Because invoking a compiler during testing is difficult,
we generate a `FileDescriptorSet` ahead of time. The `FileDescriptorSet` contains all
information necessary to create the `CodeGeneratorRequest`s we need for testing
the plugin.

`packages/plugin` and `packages/plugin-framework` are both tested using the
`FileDescriptorSet`. In `spec/helpers.ts`, the function `getFixtureCodeGeneratorRequest`
can be used to create a `CodeGeneratorRequest` from the fixture file descriptors.
`FileDescriptorSet`. In `spec/helpers.ts`, the function `getCodeGeneratorRequest`
can be used to create a `CodeGeneratorRequest` from the file descriptors.

The plugin itself has only very basic test coverage. We generate TypeScript
(in memory) for all .proto files in `packages/test-fixture` and compile the
(in memory) for all .proto files in `packages/proto` and compile the
generated code using the TypeScript Compiler API, checking for static errors.

Adding a feature or fixing a bug in the plugin can cumbersome. Instead of
Adding a feature or fixing a bug in the plugin can be cumbersome. Instead of
building the plugin and running it with protoc, you can let a test case spit
out the generated code for you. `spec/protobufts-plugin.spec.ts` contains the
necessary code to do this (commented out). If you enable the code, you can simply
Expand Down
4 changes: 2 additions & 2 deletions MANUAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -1034,7 +1034,7 @@ operations.
and methods and will add them to the reflection information.
For example, consider the following service definition in
[service-annotated.proto](./packages/test-fixtures/service-annotated.proto):
[service-annotated.proto](./packages/proto/service-annotated.proto):
```proto
// import the proto that extends google.protobuf.MethodOptions
Expand Down Expand Up @@ -1078,7 +1078,7 @@ console.log(AnnotatedService.options);
```
Because the option "google.api.http" is actually a message
(see [annotations.proto](./packages/test-fixtures/google/api/annotations.proto)),
(see [annotations.proto](./packages/proto/google/api/annotations.proto)),
you can parse the message with this convenience method:
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ official [language guide](https://developers.google.com/protocol-buffers/docs/ov
- `npm install @protobuf-ts/plugin`
> installs the plugin and the compiler "protoc"
- download the example file [msg-readme.proto](https://raw.githubusercontent.com/timostamm/protobuf-ts/master/packages/test-fixtures/msg-readme.proto) and place it into a `protos/` directory
- download the example file [msg-readme.proto](https://raw.githubusercontent.com/timostamm/protobuf-ts/main/packages/proto/msg-readme.proto) and place it into a `protos/` directory

- `npx protoc --ts_out . --proto_path protos protos/msg-readme.proto`
> generates msg-readme.ts
Expand Down Expand Up @@ -95,7 +95,7 @@ Read the [MANUAL](MANUAL.md) to learn more.
- The files [plugin.ts](./packages/plugin-framework/src/google/protobuf/compiler/plugin.ts) and [descriptor.ts](./packages/plugin-framework/src/google/protobuf/descriptor.ts) are Copyright 2008 Google Inc., licensed under BSD-3-Clause
- The [gRPC status codes](./packages/grpcweb-transport/src/goog-grpc-status-code.ts) are Copyright 2016 gRPC authors, licensed under Apache-2.0.
- The [Twirp error codes](./packages/twirp-transport/src/twitch-twirp-error-code.ts) are Copyright 2018 Twitch Interactive, Inc., licensed under Apache-2.0.
- The proto files in [test-fixtures/google](./packages/test-fixtures/google) and [test-conformance/proto](./packages/test-conformance/proto) are Copyright Google Inc. / Google LLC, licensed under Apache-2.0 / BSD-3-Clause.
- The proto files in [proto/google](./packages/proto/google) and [test-conformance/proto](./packages/test-conformance/proto) are Copyright Google Inc. / Google LLC, licensed under Apache-2.0 / BSD-3-Clause.
- All other files are licensed under Apache-2.0, see [LICENSE](./LICENSE).


Expand Down
2 changes: 1 addition & 1 deletion packages/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,5 @@
- [example-node-grpc-transport-client](./example-node-grpc-transport-client) - a gRPC client using @protobuf-ts/grpc-transport, running in Node.JS
- [example-node-twirp-transport-client](./example-node-twirp-transport-client) - a Twirp client using @protobuf-ts/twirp-transport, running in Node.JS
- [example-browser-grpcweb-transport-client](example-browser-grpcweb-transport-client) - a gRPC-web client using @protobuf-ts/grpcweb-transport, running in the web browser
- [test-fixtures](./test-fixtures) - example .proto files and test fixtures
- [proto](./proto) - example .proto files
- [benchmarks](./benchmarks) - code size and performance benchmarks
2 changes: 1 addition & 1 deletion packages/benchmarks/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: default perf code-size testees-generate testees-webpack

PROTO_PATH = ../test-fixtures
PROTO_PATH = ../proto
PROTOS := $(shell find ${PROTO_PATH} -name '*.proto')
TESTEES := $(shell find ./testees -type d -maxdepth 1 -mindepth 1)

Expand Down
2 changes: 1 addition & 1 deletion packages/benchmarks/testees/google-protobuf/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate webpack tsc

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := "import_style=commonjs,binary"

Expand Down
2 changes: 1 addition & 1 deletion packages/benchmarks/testees/pbf/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate webpack

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := ""

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate tsc webpack

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := "force_optimize_code_size,long_type_bigint"

Expand Down
2 changes: 1 addition & 1 deletion packages/benchmarks/testees/protobuf-ts.size/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate tsc webpack

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := "force_optimize_code_size,long_type_string"

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate tsc webpack

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := "force_optimize_speed,long_type_bigint"

Expand Down
2 changes: 1 addition & 1 deletion packages/benchmarks/testees/protobuf-ts.speed/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate tsc webpack

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := "force_optimize_speed,long_type_string"

Expand Down
2 changes: 1 addition & 1 deletion packages/benchmarks/testees/protobufjs/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate webpack

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := ""

Expand Down
2 changes: 1 addition & 1 deletion packages/benchmarks/testees/ts-proto.default/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate tsc webpack

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := "forceLong=string"

Expand Down
2 changes: 1 addition & 1 deletion packages/benchmarks/testees/ts-proto.no-json/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: generate tsc webpack

PROTO_PATH = ../../../test-fixtures
PROTO_PATH = ../../../proto
PROTOS := ${PROTO_PATH}/google/protobuf/descriptor.proto
OPTIONS := "outputJsonMethods=false,forceLong=string"

Expand Down
2 changes: 1 addition & 1 deletion packages/example-go-twirp-server/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.PHONY: server install generate
PROTO_PATH = ../test-fixtures
PROTO_PATH = ../proto


server:
Expand Down
2 changes: 1 addition & 1 deletion packages/example-node-twirp-transport-client/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.PHONY: default npm-install generate run-client
PROTO_PATH = ../test-fixtures
PROTO_PATH = ../proto


default: npm-install generate run-client
Expand Down
1 change: 1 addition & 0 deletions packages/plugin-framework/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@
/build/**/*.js
/build/**/*.ts.map
/build/**/*.js.map
*.codegenreq
19 changes: 4 additions & 15 deletions packages/plugin-framework/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.PHONY: default build clean test
PROTO_PATH = ../test-fixtures
PROTO_PATH = ../proto
PROTOS := $(shell find ${PROTO_PATH} -name '*.proto')
SPECS := $(shell find spec -name '*.spec.ts')

Expand All @@ -16,10 +16,10 @@ build:
clean:
@find build \( -name '*.js' -or -name '*.map' -or -name '*.ts' \) -delete;
@find build -type d ! -path build -delete
@find ../test-fixtures \( -name '*.codegenreq' -or -name '*.descriptorset' \) -delete;
@rm -f all.codegenreq
@echo "'${@}' done"

test: ${PROTO_PATH}/all.descriptorset
test:
@echo "'${@}' ..."
@./node_modules/.bin/ts-node \
--project tsconfig.test.json \
Expand All @@ -38,18 +38,7 @@ bootstrap:
google/protobuf/descriptor.proto \
google/protobuf/compiler/plugin.proto

${PROTO_PATH}/all.descriptorset: $(PROTOS)
@protoc \
--descriptor_set_out=${@} \
--proto_path=${PROTO_PATH} \
--proto_path ../plugin/ \
--include_source_info \
--include_imports \
--experimental_allow_proto3_optional \
$^
@echo "'${@}' done"

${PROTO_PATH}/all.codegenreq: $(PROTOS)
all.codegenreq: $(PROTOS)
@protoc \
--dump_out=. \
--plugin=./bin/protoc-gen-dump \
Expand Down
1 change: 0 additions & 1 deletion packages/plugin-framework/karma.conf.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ module.exports = function (config) {
files: [
"spec/**/*.ts",
"src/**/*.ts",
"../test-fixtures/*.ts",
],
exclude: [
"spec/support/reporter.ts",
Expand Down
3 changes: 3 additions & 0 deletions packages/plugin/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,6 @@
/build/**/*.js
/build/**/*.ts.map
/build/**/*.js.map
*.descriptorset
*.codegenreq

11 changes: 6 additions & 5 deletions packages/plugin/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.PHONY: default build clean test
PROTO_PATH = ../test-fixtures
PROTO_PATH = ../proto
PROTOS := $(shell find ${PROTO_PATH} -name '*.proto')
SPECS := $(shell find spec -name '*.spec.ts')

Expand All @@ -13,17 +13,18 @@ build:
clean:
@find build \( -name '*.js' -or -name '*.map' -or -name '*.ts' \) -delete;
@find build -type d ! -path build -delete
@find ../test-fixtures \( -name '*.codegenreq' -or -name '*.descriptorset' \) -delete;
@rm -f all.descriptorset
@rm -f all.codegenreq
@echo "'${@}' done"

test: ${PROTO_PATH}/all.descriptorset
test: all.descriptorset
@./node_modules/.bin/ts-node \
--project tsconfig.test.json \
--require tsconfig-paths/register \
./node_modules/.bin/jasmine --helper="spec/support/reporter.ts" \
$(SPECS)

${PROTO_PATH}/all.descriptorset: $(PROTOS)
all.descriptorset: $(PROTOS)
@protoc \
--descriptor_set_out ${@} \
--proto_path ${PROTO_PATH} \
Expand All @@ -34,7 +35,7 @@ ${PROTO_PATH}/all.descriptorset: $(PROTOS)
$^
@echo "'${@}' done"

${PROTO_PATH}/all.codegenreq: $(PROTOS)
all.codegenreq: $(PROTOS)
@protoc \
--dump_out=. \
--plugin=./bin/protoc-gen-dump \
Expand Down
4 changes: 2 additions & 2 deletions packages/plugin/spec/interpreter.spec.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import {getFixtureFileDescriptor} from "./support/helpers";
import {getFileDescriptor} from "./support/helpers";
import {DescriptorRegistry} from "@protobuf-ts/plugin-framework";
import {Interpreter} from "../src/interpreter";
import * as rt from "@protobuf-ts/runtime";
Expand All @@ -8,7 +8,7 @@ describe('interpreter', function () {
it('recognizes field option jstype', function () {
[rt.LongType.NUMBER, rt.LongType.STRING, rt.LongType.BIGINT].forEach(normalLongType => {

const registry = DescriptorRegistry.createFrom(getFixtureFileDescriptor("msg-longs.proto"));
const registry = DescriptorRegistry.createFrom(getFileDescriptor("msg-longs.proto"));
const interpreter = new Interpreter(registry, {
normalLongType,
synthesizeEnumZeroValue: 'UNSPECIFIED$',
Expand Down
18 changes: 9 additions & 9 deletions packages/plugin/spec/protobufts-plugin-long.spec.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import {getFixtureCodeGeneratorRequest} from "./support/helpers";
import {getCodeGeneratorRequest} from "./support/helpers";
import {ProtobuftsPlugin} from "../src/protobufts-plugin";
import {GeneratedFile} from "@protobuf-ts/plugin-framework/src";

Expand All @@ -14,7 +14,7 @@ const stringSnippets = [
// BinaryWriter
'/* fixed64 fixed64_field_min_str = 11 [jstype = JS_STRING]; */',
'if (message.fixed64FieldMinStr !== "0")',
];
];

const bigintSnippets = [
// Message
Expand Down Expand Up @@ -47,7 +47,7 @@ const numberSnippets = [
describe('Generated code for long type', function () {
describe('Default to string', () => {
let file: string;

beforeAll(() => {
file = generateTypescript('long_type_string').getContent();
});
Expand Down Expand Up @@ -77,7 +77,7 @@ describe('Generated code for long type', function () {

describe('Default to bigint', () => {
let file: string;

beforeAll(() => {
file = generateTypescript('long_type_bigint').getContent();
});
Expand All @@ -103,11 +103,11 @@ describe('Generated code for long type', function () {
expectContainAll(file, bigintSnippets);
expectContainAll(file, numberSnippets);
});
});
});

describe('Default to number', () => {
let file: string;

beforeAll(() => {
file = generateTypescript('long_type_number').getContent();
});
Expand All @@ -133,14 +133,14 @@ describe('Generated code for long type', function () {
expectContainAll(file, bigintSnippets);
expectContainAll(file, numberSnippets);
});
});
});
});

// Generate typescript code for msg-longs.proto.
// Generate typescript code for msg-longs.proto.
// The `parameter` is forwarded to the request.
function generateTypescript(parameter: string): GeneratedFile {
let plugin = new ProtobuftsPlugin('test');
let request = getFixtureCodeGeneratorRequest({
let request = getCodeGeneratorRequest({
parameter,
includeFiles: [ 'msg-longs.proto' ]
});
Expand Down
10 changes: 5 additions & 5 deletions packages/plugin/spec/protobufts-plugin.spec.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import {getFixtureCodeGeneratorRequest} from "./support/helpers";
import {getCodeGeneratorRequest} from "./support/helpers";
import {ProtobuftsPlugin} from "../src/protobufts-plugin";
import * as ts from "typescript";
import {setupCompiler} from "@protobuf-ts/plugin-framework";
Expand All @@ -7,7 +7,7 @@ import {OutFile} from "../src/out-file";

describe('protobuftsPlugin', function () {
let plugin = new ProtobuftsPlugin('test');
let request = getFixtureCodeGeneratorRequest({
let request = getCodeGeneratorRequest({
parameter: 'long_type_string',
// fileToGenerate: [
// 'msg-annotated.proto',
Expand Down Expand Up @@ -52,11 +52,11 @@ describe('protobuftsPlugin', function () {
// }


describe('generates valid typescript for every fixture .proto', function () {
describe('generates valid typescript for every .proto', function () {

if (generatedFiles.length === 0) {
it('Fixture .protos available', function () {
pending('No fixture .proto available!');
it('.protos available', function () {
pending('No .proto available!');
});
return;
}
Expand Down
Loading

0 comments on commit ded8791

Please sign in to comment.