Protocol Buffers for the Rest of Us
Motivation
protoletariat has one goal: fixing the broken imports for the Python code generated by protoc.
Usage
Here's an example of how to use the tool, called protol:
- Create a few protobuf files
// thing1.proto
syntax = "proto3";
import "thing2.proto";
package things;
message Thing1 {
Thing2 thing2 = 1;
}
// thing2.proto
syntax = "proto3";
package things;
message Thing2 {
string data = 1;
}
- Run
protocon those files
$ mkdir out
$ protoc --python_out=out --proto_path=directory/containing/protos thing1.proto thing2.proto
- Run
protolon the generated code
$ protol --create-init --overwrite -g out --proto-path=directory/containing/protos thing1.proto thing2.proto
The out/thing1_pb2.py file should show a diff containing at least these lines:
-import thing2_pb2 as thing2__pb2
-
+from . import thing2_pb2 as thing2__pb2
How it works
At a high level, protoletariat converts absolute imports to relative imports.
However, it doesn't just convert any absolute import to a relative import.
The protol tool will only convert imports that were generated from .proto files. It does this by inspecting FileDescriptorProtos from the protobuf files.
The core mechanism is implemented using a simplified form of pattern matching, that looks at the Python AST, and if an import pattern is matched and corresponding rewrite rule is fired.
Help
$ protol --help
Usage: protol [OPTIONS] PROTO_FILES...
Rewrite protoc-generated imports for use by the proletariat.
Options:
-g, --generated-python-dir DIRECTORY
Directory containing generated Python code
[required]
-p, --proto-path DIRECTORY Protobuf file search path(s). Accepts
multiple values. [required]
--overwrite / --no-overwrite Overwrite all generated Python files with
modified imports
--create-init / --dont-create-init
Create an __init__.py file under the
`generated-python-dir` directory
--help Show this message and exit.
43 Dec 26, 2022
2 Dec 27, 2021
3 Jan 27, 2022
1 Jan 12, 2022
0 Dec 13, 2021
1 Jan 06, 2022
6 Jun 25, 2022
1 Jan 30, 2022
4 Jan 19, 2022
6 Dec 11, 2022
1 Jan 08, 2022
6 Jul 19, 2022
90 Dec 14, 2022
2.7k Jan 03, 2023
3 Nov 17, 2022
2 Feb 18, 2022
25 Dec 05, 2022
3 Sep 25, 2022
1 Jan 10, 2022
2 Nov 13, 2022