A python tool for synchronizing the messages from different threads, processes, or hosts.

Last update: Aug 11, 2021

Overview

Sync-stream

This project is designed for providing the synchoronization of the stdout / stderr among different threads, processes, devices or hosts. The package could be used for the following cases:

Use syncstream.LineBuffer: Multiple threads are created. The messages (stdout) from different threads are required to be collected.
Use syncstream.LineProcBuffer in the main process, and syncstream.LineProcMirror in the sub-processes: Multiple sub-processes are created on the same device. The stdout / stderr of each process is redirected to a LineProcMirror, and the results are collected by LineProcBuffer.
Use syncstream.LineFileBuffer: Multiple processes are created. These processes may be deployed on different devices (even with different platforms), but all devices could get accessed to the same shared disk. In this case, the message could be shared by locked files. Each process would hold an independent LineFileBuffer pointing to the same log files.
Use syncstream.LineHostBuffer on the server side, and syncstream.LineHostMirror on the client side: Multiple processes are deployed on different devices, and they could not share the same disk. In this case, the message are synchronized by the web service. Each process would hold a LineHostMirror, and the results would be collected by LineHostBuffer.

The basic package would not contain the file and host modules. To install the package, please use the following options:

pip install syncstream[option1,option2...]

Option	Supports
`file`	Install dependencies for the `file` module. The module provides `syncstream.LineFileBuffer`.
`host`	Install dependencies for the `host` module. The module provides `syncstream.LineHostBuffer`, and `syncstream.LineHostMirror`.

Documentation

View the documentation here: 📘 https://cainmagi.github.io/sync-stream/

Update reports

0.3.3 @ 6/29/2021

Fix small typos.
Bump the dependencies to the newest versions.

0.3.2 @ 6/14/2021

Fix a bug caused by stopping the mirrors.
Format the meta-data defined in setup.py.
Add the documentation. Currently only the tutorial is finished.

0.3.0 @ 6/4/2021

Support the stop signal for mproc and host modules.
Fix some bugs in the testing script.
Fix typos.

0.2.2 @ 5/25/2021

Add clear() methods for all buffers and mirrors.
Fix typos in the package setup and info file.
Fix a bug caused by writing data to the host in the testing scripts for Linux.

0.2.1 @ 5/24/2021

Add the PyPI publish workflow.

0.2.0 @ 5/24/2021

Finish the synchronization based on the file lock package fasteners.
Finish the synchronization based on the web service packages flask, flask-restful and urllib3.
Fix the compatibility of the testing scripts for py36, py37.

0.1.0 @ 5/22/2021

Finish the synchronization based on the stdlib.
Create this project.

Provide error messages for Python exceptions, even if the original message is empty

errortext is a Python package to provide error messages for Python exceptions, even if the original message is empty.

0 Dec 7, 2021

Telegram bot to remove the forwarded tag from messages.

Anonymous Sender Bot @AnonySendBot Telegram bot to remove the forwarded tag from messages. Table of Contents Usage Deploy To Heroku Local Deploying En

26 Nov 24, 2022

Automatically remove user join messages when the user leaves the server.

CleanLeave Automatically remove user join messages when the user leaves the server. Installation You will need to install poetry to run this bot local

11 Sep 19, 2022

Simple script with AminoLab to send ghost messages

1 Nov 22, 2021

A Github Action for sending messages to a Matrix Room.

matrix-commit A Github Action for sending messages to a Matrix Room. Screenshot: Example Usage: # .github/workflows/matrix-commit.yml on: push:

3 Sep 11, 2022

ViberExport - Export messages from Viber messenger using viber.db file

📲 ViberExport Export messages from Viber messenger using viber.db file ⚡ Usage:

7 Nov 23, 2022

Telop - Encode and decode messages using an interpretation of the telegraphic code devised by José María Mathé

telop Telop (TELégrafoÓPtico) - Utilidad para codificar y descodificar mensajes de texto empleando una interpretación del código telegráfico ideado po

4 Nov 1, 2022

Automatically unpin old messages so you can always pin more!

PinRotate Automatically unpin old messages so you can always pin more! Installation You will need to install poetry to run this bot locally for develo

3 Sep 18, 2022

A tool to flash .ofp files in bootloader mode without needing MSM Tool, an alternative to official realme tool

Oppo/Realme Flash .OFP File on Bootloader A tool to flash .ofp files in bootloader mode without needing MSM Tool, an alternative to official realme to

70 Jan 2, 2023

Releases(0.3.3)

0.3.3(Jun 29, 2021)
Introduction

Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

Updating reports

0.3.3 @ 6/29/2021

Fix small typos.

Bump the dependencies to the newest versions.

Source code(tar.gz)
Source code(zip)
0.3.2(Jun 14, 2021)
Introduction

Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

Updating reports

0.3.2 @ 6/14/2021

Fix a bug caused by stopping the mirrors.

Format the meta-data defined in setup.py.

Add the documentation. Currently, only the tutorial is finished.

Source code(tar.gz)
Source code(zip)
0.3.0(Jun 4, 2021)
Introduction

Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

Updating reports

0.3.0 @ 6/4/2021

Support the stop signal for mproc and host modules.

Fix some bugs in the testing script.

Fix typos.

Source code(tar.gz)
Source code(zip)
0.2.2(May 25, 2021)
Introduction

Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

Updating reports

0.2.2 @ 5/25/2021

Add clear() methods for all buffers and mirrors.

Fix typos in the package setup and info file.

Fix a bug caused by writing data to the host in the testing scripts for Linux.

Source code(tar.gz)
Source code(zip)
0.2.1(May 24, 2021)
Introduction

This is the first release of this project. Sync-stream is a pure python tool for synchronizing the messages (including stdout, stderr, and raised exceptions or warnings) among threads, processes, devices, and hosts. This powerful tool is useful for capturing the updating logs from the sub-processes in some projects.

Updating reports

0.2.1 @ 5/24/2021

Add the PyPI publish workflow.

0.2.0 @ 5/24/2021

Finish the synchronization based on the file lock package fasteners.

Finish the synchronization based on the web service packages flask, flask-restful and urllib3.

Fix the compatibility of the testing scripts for py36, py37.

0.1.0 @ 5/22/2021

Finish the synchronization based on the stdlib.

Create this project.

Source code(tar.gz)
Source code(zip)

Owner

Yuchen Jin

A Ph.D student in University of Houston (UH). Interested area includes: machine learning, programming and religion.

GitHub Repository https://cainmagi.github.io/sync-stream/

This program can calculate the Aerial Distance between two cities.

Aerial_Distance_Calculator This program can calculate the Aerial Distance between two cities. This repository include both Jupyter notebook and Python

1 Apr 08, 2022

A conda-smithy repository for boost-histogram.

The official Boost.Histogram Python bindings. Provides fast, efficient histogramming with a variety of different storages combined with dozens of composable axes. Part of the Scikit-HEP family.

0 Dec 17, 2021

Start and stop your NiceHash miners using this script.

NiceHash Mining Scheduler Use this script to schedule your NiceHash Miner(s). Electricity costs between 4-9pm are high in my area and I want NiceHash

2 Sep 30, 2022

poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions

poetry2nix poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions. It does so by parsing pyproject.t

405 Dec 29, 2022

Parametric Bottle in CADQuery

Parametric Bottle using CADQuery The proposed code makes it possible to generate different types and sizes of 3D bottles in order to train Pixel2mesh

1 May 22, 2022

A curses based mpd client with basic functionality and album art.

Miniplayer A curses based mpd client with basic functionality and album art. After installation, the player can be opened from the terminal with minip

102 Dec 24, 2022

Control System Packer is a lightweight, low-level program to transform energy equations into the compact libraries for control systems.

Control System Packer is a lightweight, low-level program to transform energy equations into the compact libraries for control systems. Packer supports Python 🐍 , C 💻 and C++ 💻 libraries.

31 Sep 15, 2022

A python tool for synchronizing the messages from different threads, processes, or hosts.

Related tags

Overview

Sync-stream

Documentation

Update reports

0.3.3 @ 6/29/2021

0.3.2 @ 6/14/2021

0.3.0 @ 6/4/2021

0.2.2 @ 5/25/2021

0.2.1 @ 5/24/2021

0.2.0 @ 5/24/2021

0.1.0 @ 5/22/2021

You might also like...

Provide error messages for Python exceptions, even if the original message is empty

Telegram bot to remove the forwarded tag from messages.

Automatically remove user join messages when the user leaves the server.

Simple script with AminoLab to send ghost messages

A Github Action for sending messages to a Matrix Room.

ViberExport - Export messages from Viber messenger using viber.db file

Telop - Encode and decode messages using an interpretation of the telegraphic code devised by José María Mathé

Automatically unpin old messages so you can always pin more!

A tool to flash .ofp files in bootloader mode without needing MSM Tool, an alternative to official realme tool

Releases(0.3.3)

0.3.3(Jun 29, 2021)

Introduction

Updating reports

0.3.3 @ 6/29/2021

0.3.2(Jun 14, 2021)

Introduction

Updating reports

0.3.2 @ 6/14/2021

0.3.0(Jun 4, 2021)

Introduction

Updating reports

0.3.0 @ 6/4/2021

0.2.2(May 25, 2021)

Introduction

Updating reports

0.2.2 @ 5/25/2021

0.2.1(May 24, 2021)

Introduction

Updating reports

0.2.1 @ 5/24/2021

0.2.0 @ 5/24/2021

0.1.0 @ 5/22/2021

Owner

Yuchen Jin

This program can calculate the Aerial Distance between two cities.

A conda-smithy repository for boost-histogram.

Start and stop your NiceHash miners using this script.

poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions

Parametric Bottle in CADQuery

A curses based mpd client with basic functionality and album art.

Control System Packer is a lightweight, low-level program to transform energy equations into the compact libraries for control systems.

This scrypt for auto brightness control

A turtlebot auto controller allows robot to autonomously explore environment.

🚀 emojimash 🚀 is a programming language with ALL THE EMOJI

AIO solution for SSIS students

Run the Tianxunet software on the Xiaoyao Android simulator

A function decorator for enforcing function signatures

i3wm helper tool for workspaces on multiple monitors

Use `forge` and `cast` commands in Python scripts

Release for Improved Denoising Diffusion Probabilistic Models

Organize seu linux - organize your linux

Pre-crisis Risk Management for Personal Finance

We want to check several batch of web URLs (1~100 K) and find the phishing website/URL among them.

tagls is a language server based on gtags.