Protocol Buffers for the Rest of Us
Motivation
protoletariat has one goal: fixing the broken imports for the Python code generated by protoc.
Usage
Here's an example of how to use the tool, called protol:
- Create a few protobuf files
// thing1.proto
syntax = "proto3";
import "thing2.proto";
package things;
message Thing1 {
Thing2 thing2 = 1;
}
// thing2.proto
syntax = "proto3";
package things;
message Thing2 {
string data = 1;
}
- Run
protocon those files
$ mkdir out
$ protoc --python_out=out --proto_path=directory/containing/protos thing1.proto thing2.proto
- Run
protolon the generated code
$ protol --create-init --overwrite -g out --proto-path=directory/containing/protos thing1.proto thing2.proto
The out/thing1_pb2.py file should show a diff containing at least these lines:
-import thing2_pb2 as thing2__pb2
-
+from . import thing2_pb2 as thing2__pb2
How it works
At a high level, protoletariat converts absolute imports to relative imports.
However, it doesn't just convert any absolute import to a relative import.
The protol tool will only convert imports that were generated from .proto files. It does this by inspecting FileDescriptorProtos from the protobuf files.
The core mechanism is implemented using a simplified form of pattern matching, that looks at the Python AST, and if an import pattern is matched and corresponding rewrite rule is fired.
Help
$ protol --help
Usage: protol [OPTIONS] PROTO_FILES...
Rewrite protoc-generated imports for use by the proletariat.
Options:
-g, --generated-python-dir DIRECTORY
Directory containing generated Python code
[required]
-p, --proto-path DIRECTORY Protobuf file search path(s). Accepts
multiple values. [required]
--overwrite / --no-overwrite Overwrite all generated Python files with
modified imports
--create-init / --dont-create-init
Create an __init__.py file under the
`generated-python-dir` directory
--help Show this message and exit.
1 Jan 27, 2022
6 Feb 17, 2022
1 Nov 03, 2021
1 Oct 24, 2021
5 Jan 12, 2022
5 Jan 03, 2022
8 Oct 22, 2022
2 Nov 23, 2021
1.3k Dec 26, 2022
108 Jan 04, 2023
574 Dec 26, 2022
2 Oct 14, 2021
64 Dec 23, 2022
25 Dec 15, 2022
8 Feb 27, 2022
54 Dec 23, 2022
7 Sep 18, 2022
114 Oct 12, 2022
4 May 14, 2022
46 Dec 09, 2022