Protocol Buffers for the Rest of Us
Motivation
protoletariat has one goal: fixing the broken imports for the Python code generated by protoc.
Usage
Here's an example of how to use the tool, called protol:
- Create a few protobuf files
// thing1.proto
syntax = "proto3";
import "thing2.proto";
package things;
message Thing1 {
Thing2 thing2 = 1;
}
// thing2.proto
syntax = "proto3";
package things;
message Thing2 {
string data = 1;
}
- Run
protocon those files
$ mkdir out
$ protoc --python_out=out --proto_path=directory/containing/protos thing1.proto thing2.proto
- Run
protolon the generated code
$ protol --create-init --overwrite -g out --proto-path=directory/containing/protos thing1.proto thing2.proto
The out/thing1_pb2.py file should show a diff containing at least these lines:
-import thing2_pb2 as thing2__pb2
-
+from . import thing2_pb2 as thing2__pb2
How it works
At a high level, protoletariat converts absolute imports to relative imports.
However, it doesn't just convert any absolute import to a relative import.
The protol tool will only convert imports that were generated from .proto files. It does this by inspecting FileDescriptorProtos from the protobuf files.
The core mechanism is implemented using a simplified form of pattern matching, that looks at the Python AST, and if an import pattern is matched and corresponding rewrite rule is fired.
Help
$ protol --help
Usage: protol [OPTIONS] PROTO_FILES...
Rewrite protoc-generated imports for use by the proletariat.
Options:
-g, --generated-python-dir DIRECTORY
Directory containing generated Python code
[required]
-p, --proto-path DIRECTORY Protobuf file search path(s). Accepts
multiple values. [required]
--overwrite / --no-overwrite Overwrite all generated Python files with
modified imports
--create-init / --dont-create-init
Create an __init__.py file under the
`generated-python-dir` directory
--help Show this message and exit.
2 May 17, 2022
2 Mar 19, 2022
44 Oct 29, 2022
13 Nov 27, 2022
22 Mar 17, 2022
1 Dec 13, 2021
4 Oct 27, 2022
1 Jan 18, 2022
116 Nov 25, 2022
2 Jan 11, 2022
2 Jan 17, 2022
2 Mar 20, 2022
1 Jan 19, 2022
8 Jan 30, 2022
201 Dec 30, 2022
4 Mar 23, 2022
11 May 27, 2022
16 Aug 13, 2022
3 Feb 11, 2022
3 Feb 02, 2022