CORE_RULES.md

Build Rules

The core of stackb/rules_proto contains two build rules:

Rule	Description
proto_compile	Executes the protoc tool.
proto_plugin	Provides static protoc plugin-specific configuration.

proto_compile

Example:

load("@rules_proto//proto:defs.bzl", "proto_library")
load("@build_stack_rules_proto//rules:proto_compile.bzl", "proto_compile")

proto_library(
    name = "thing_proto",
    srcs = ["thing.proto"],
    deps = ["@com_google_protobuf//:timestamp_proto"],
)

proto_plugin(name = "cpp")

proto_compile(
    name = "person_cpp_compile",
    outputs = [
        "person.pb.cc",
        "person.pb.h",
    ],
    plugins = [":cpp"],
    proto = "person_proto",
)

Takeaways:

• A proto_library rule forms the basis for other language-specific derived rules.
• proto_library is provided by bazelbuild/rules_proto.
• A proto_compile rule references a single proto_library target.
• The plugins attribute is a list of labels to proto_plugin targets.
• The outputs attribute names the files that will be generated by the protoc invocation.
• The proto extension provided by [bazel-gazelle] is responsible for generating proto_library.

proto_plugin

proto_plugin primarily provides the plugin tool executable. The example seen above is the simplest case where the plugin is builtin to protoc itself; no separate plugin tool is required. In this case the proto_plugin rule degenerates into just a name.

It is possible to add additional plugin-specific name = "foo", options = ["bar"] on the proto_plugin rule, but the use-case for this is narrow. Generally it is preferred to say # gazelle:proto_plugin foo option bar such that the option can be interpreted during a gazelle run.

proto_compiled_sources

proto_compiled_sources is used when you prefer to check the generated files into source control. This may be necessary for legacy reasons, during an initial Bazel migration, or to support better IDE integration.

The shape of a proto_compiled_sources rule is essentially identical to proto_compile with one exception: generated source are named in the srcs attribute rather than outputs.

For example, a proto_compiled_sources named //example/thing:proto_go_sources is a macro that generates three rules:

1. bazel build //example/thing:proto_go_sources emits the generated files.
2. bazel run //example/thing:proto_go_sources.update copies the generated files back into the source package.
3. bazel test //example/thing:proto_go_sources_test asserts the source files are identical to generated files.

In this scenario, 2. is used to build the generated files (in the bazel-bin/ output tree) and copy the example/thing/thing.pb.go back into place where it will be committed under source control. 3. is used to prevent drift: if a developer modifies thing.proto and neglects to run the .update the test will fail in CI.

proto_compile_assets

The macro proto_compile_assets aggregates a list of dependencies (which provide ProtoCompileInfo) into a single runnable target that copies files in bulk.

For example, bazel run //proto:assets will copy all the generated .pb.go files back into the source tree:

load("@build_stack_rules_proto//rules:proto_compile_assets.bzl", "proto_compile_assets")

proto_compile_assets(
    name = "assets",
    deps = [,
      "//proto/api/v1:proto_go_compile",
      "//proto/api/v2:proto_go_compile",
      "//proto/api/v3:proto_go_compile",
    ],
)

The output_mappings attribute

Consider the following rule within the package example/thing:

proto_compile(
    name = "thing_go_compile",
    output_mappings = ["thing.pb.go=github.com/stackb/rules_proto/v4/example/thing/thing.pb.go"],
    outputs = ["thing.pb.go"],
    plugins = ["@build_stack_rules_proto//plugin/golang/protobuf:protoc-gen-go"],
    proto = "thing_proto",
)

This rule is declaring that a file bazel-bin/example/thing/thing.pb.go will be output when the action is run. When we bazel build //example/thing:thing_go_compile, the file is indeed created.

Let's temporarily comment out the output_mappings attribute and rebuild:

proto_compile(
    name = "thing_go_compile",
    # output_mappings = ["thing.pb.go=github.com/stackb/rules_proto/v4/example/thing/thing.pb.go"],
    outputs = ["thing.pb.go"],
    plugins = ["@build_stack_rules_proto//plugin/golang/protobuf:protoc-gen-go"],
    proto = "thing_proto",
)

$ bazel build //example/thing:thing_go_compile
ERROR: /github.com/stackb/rules_proto/v4/example/thing/BUILD.bazel:54:14: output 'example/thing/thing.pb.go' was not created

What happened? Let's add a debugging attribute verbose = True on the rule: this will print debugging information and show the bazel sandbox before and after the protoc tool is invoked:

proto_compile(
    name = "thing_go_compile",
    # output_mappings = ["thing.pb.go=github.com/stackb/rules_proto/v4/example/thing/thing.pb.go"],
    outputs = ["thing.pb.go"],
    plugins = ["@build_stack_rules_proto//plugin/golang/protobuf:protoc-gen-go"],
    proto = "thing_proto",
    verbose = True,
)

$ bazel build //example/thing:thing_go_compile
##### SANDBOX BEFORE RUNNING PROTOC
./bazel-out/host/bin/external/com_google_protobuf/protoc
./bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/com_github_golang_protobuf/protoc-gen-go/protoc-gen-go_/protoc-gen-go
./bazel-out/darwin-fastbuild/bin/example/thing/thing_proto-descriptor-set.proto.bin
./bazel-out/darwin-fastbuild/bin/external/com_google_protobuf/timestamp_proto-descriptor-set.proto.bin

##### SANDBOX AFTER RUNNING PROTOC
./bazel-out/darwin-fastbuild/bin/github.com/stackb/rules_proto/v4/example/thing/thing.pb.go

So, the file was created, but not in the location we wanted. In this case the protoc-gen-go plugin is not "playing nice" with Bazel. Because this thing.proto has option go_package = "github.com/stackb/rules_proto/v4/example/thing;thing";, the output location is no longer based on the package. This is a problem, because Bazel semantics disallow declaring a File outside its package boundary. As a result, we need to do a mv ./bazel-out/darwin-fastbuild/bin/github.com/stackb/rules_proto/v4/example/thing/thing.pb.go ./bazel-out/darwin-fastbuild/bin/example/thing/thing.pb.go to relocate the file into its expected location before the action terminates.

Therefore, the output_mappings attribute is a list of entries that map file locations want=got relative to the action execution root. It is required when the actual output location does not match the desired location. This can occur if the proto package statement does not match the Bazel package path, or in special circumstances specific to the plugin itself (like go_package).

proto_gazelle

proto_gazelle is not a repository rule: it's just like the typical gazelle rule, but with extra deps resolution superpowers. But, we discuss it here since it works in conjunction with proto_repository:

load("@build_stack_rules_proto//rules:proto_gazelle.bzl", "DEFAULT_LANGUAGES", "proto_gazelle")

proto_gazelle(
    name = "gazelle",
    cfgs = ["//proto:config.yaml"],
    command = "update",
    gazelle = ":gazelle-protobuf",
    imports = [
        "@bazelapis//:imports.csv",
        "@googleapis//:imports.csv",
        "@protobufapis//:imports.csv",
        "@remoteapis//:imports.csv",
    ],
)

In this example, we are again setting the base gazelle config using the YAML file (the same one used in for the proto_repository rules). We are also now importing resolve information from four external sources.

With this setup, we can simply place an import statement like import "src/main/java/com/google/devtools/build/lib/buildeventstream/proto/build_event_stream.proto"; in a foo.proto file in the default workspace, and gazelle will automagically figure out the import dependency tree spanning @bazelapis, @remoteapis, @googleapis, and the well-known types from @protobufapis.

This works for any proto_language, with any set of custom protoc plugins.

golden_filegroup

golden_filegroup is a utility macro for golden file testing. It works like a native filegroup, but adds .update and .test targets. Example:

load("@build_stack_rules_proto//rules:golden_filegroup.bzl", "golden_filegroup")

# golden_filegroup asserts that generated files named in 'srcs' are
# identical to the ones checked into source control.
#
# Usage:
#
# $ bazel build :golden        # not particularly useful, just a regular filegroup
#
# $ bazel test  :golden.test   # checks that generated files are identical to
# ones in git (for CI)
#
# $ bazel run   :golden.update # copies the generated files into source tree
# (then 'git add' to your PR if it looks good)
golden_filegroup(
    name = "golden",
    srcs = [
        ":some_generated_file1.json",
        ":some_generated_file2.json",
    ],
)