Protocol Buffers for the Rest of Us
Motivation
protoletariat has one goal: fixing the broken imports for the Python code generated by protoc.
Usage
Here's an example of how to use the tool, called protol:
- Create a few protobuf files
// thing1.proto
syntax = "proto3";
import "thing2.proto";
package things;
message Thing1 {
Thing2 thing2 = 1;
}
// thing2.proto
syntax = "proto3";
package things;
message Thing2 {
string data = 1;
}
- Run
protocon those files
$ mkdir out
$ protoc --python_out=out --proto_path=directory/containing/protos thing1.proto thing2.proto
- Run
protolon the generated code
$ protol --create-init --overwrite -g out --proto-path=directory/containing/protos thing1.proto thing2.proto
The out/thing1_pb2.py file should show a diff containing at least these lines:
-import thing2_pb2 as thing2__pb2
-
+from . import thing2_pb2 as thing2__pb2
How it works
At a high level, protoletariat converts absolute imports to relative imports.
However, it doesn't just convert any absolute import to a relative import.
The protol tool will only convert imports that were generated from .proto files. It does this by inspecting FileDescriptorProtos from the protobuf files.
The core mechanism is implemented using a simplified form of pattern matching, that looks at the Python AST, and if an import pattern is matched and corresponding rewrite rule is fired.
Help
$ protol --help
Usage: protol [OPTIONS] PROTO_FILES...
Rewrite protoc-generated imports for use by the proletariat.
Options:
-g, --generated-python-dir DIRECTORY
Directory containing generated Python code
[required]
-p, --proto-path DIRECTORY Protobuf file search path(s). Accepts
multiple values. [required]
--overwrite / --no-overwrite Overwrite all generated Python files with
modified imports
--create-init / --dont-create-init
Create an __init__.py file under the
`generated-python-dir` directory
--help Show this message and exit.
9 Feb 06, 2022
8 Mar 11, 2022
73 Dec 14, 2022
4 Nov 10, 2021
34 Nov 28, 2022
72 Oct 09, 2022
162 Dec 07, 2022
4 Dec 01, 2022
47 Oct 08, 2022
1 Jan 12, 2022
32 Dec 24, 2022
1 Dec 15, 2021
0 Dec 07, 2021
49 Dec 22, 2022
2 Dec 31, 2022
14 Aug 18, 2022
1 Nov 10, 2021
1 Feb 12, 2022
1 Oct 29, 2021
3 Aug 09, 2021