Yaml - Loggers are like print() statements

Overview

Upgrade your print statements

print statement meme

Loggers are like print() statements

except they also include loads of other metadata:

  • timestamp
  • msg (same as print!)
  • args (values or variables put into functions)
  • function name
  • level (e.g. DEBUG)
  • line number (e.g line 42)
  • module (which python script it came from)
  • processes (getting fancy)
  • threads (getting very fancy)

I want it all WITHOUT CONFIGURATION!

loguru logo

No problems. Check out the Loguru repo!
You can pretty much run with minimal config and loads of features. Loguru has all the features of this repo and more, along with a badass logo.

Where do I start?

This repo comes bundled with tidy logging configuration files saved in src/logconfig/.
There is also a timing decorator in src/utils/ so you can optimize your code by simply decorating your function with @timing above it. There are many online articles but very few clearly explain how to configure complex loggers with filters and yaml files.
To see loggers in action, setup a virtual environment and then run src/mainmodule.py

Requirements

  • Python 3.6+

(Feel free to submit earlier versions that work)

Quickstart - Git clone and virtual env setup

Quickstart Instructions

Windows using powershell or CMD

cd to clone directory. Create virtual env with pip + venv:

git clone https://github.com/izzley/loggerexamples
cd loggerexamples\
py -0p # Optional: check your version and python path
py -m venv .venv
.venv\Scripts\activate
pip install --upgrade pip
pip install -r requirements.txt

# run main script
.\src\main.py

Linux/Mac

cd /to/clone/location
git clone https://github.com/izzley/loggerexamples
cd loggerexamples/
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

If your relative imports aren't working, create `.pth' and add the parent folder/s to the file:

$ echo $(pwd) >> .venv/lib/python3.8/site-packages/my_p_ext.pth

Play with the logger

Walkthroughs

@TODO add list of suggestions to inspect

Try changing the main logger level from main_module.py

The main logger is first setup from main_module.py. Notice how the level is set to DEBUG?

logger level change


  • Run main_module.py with this DEBUG setting and take a look at src/logconfig/root.log.
  • Now change the level to WARNING and spot the difference in src/logconfig/root.log. Remember this is the __main__ logger. Notice any changes?

logger level change


  • Notice everything from __main__ is greater or equal to WARNING? The logs running from other_module.py are still showing because we haven't changed its log level. Go ahead and change the loglevel in other_module.py to CRITICAL and spot the difference.

Whats in the YAML file??

YAML parts

In short, the conf.YAML file contains all of the instructions for how the logger should behave. Below breaks down the conf yaml file into its parts:

formatters

formatters:
    standard:
        format: "%(asctime)s %(levelname)s - [%(filename)s: line %(lineno)s] - %(funcName)s - %(message)s"

Take this logger for example:

def funccalc(n):
    logger.debug("something executed")
    for _ in range(n):
        i = 0
    return

The output reflects the yaml file format settings:

2021-11-21 15:43:47,689 DEBUG - [module01.py: line 17] - funccalc - something executed

loggers

@TODO describe root loggers and their inheritance

root:
  level: DEBUG
  handlers: [console, debug_file_handler, info_file_handler, warn_file_handler, error_file_handler, critical_file_handler, root_file_handler]

root logger yaml

handlers

@TODO describe handlers and how they redirect bytes

filters

@TODO describe how filters only allow bytes to handlers if a condition is true. reference filter classes in logconfig.py

References

Owner
isaac peterson
Open source || Awesome sauce
isaac peterson
Rich is a Python library for rich text and beautiful formatting in the terminal.

Rich 中文 readme • lengua española readme • Läs på svenska Rich is a Python library for rich text and beautiful formatting in the terminal. The Rich API

Will McGugan 41.5k Jan 07, 2023
Key Logger - Key Logger using Python

Key_Logger Key Logger using Python This is the basic Keylogger that i have made

Mudit Sinha 2 Jan 15, 2022
Log4j alternative for Python

Log4p Log4p is the most secure logging library ever created in this and all other universes. Usage: import log4p log4p.log('"Wow, this library is sec

Isaak Uchakaev 15 Dec 16, 2022
Log processor for nginx or apache that extracts user and user sessions and calculates other types of useful data for bot detection or traffic analysis

Log processor for nginx or apache that extracts user and user sessions and calculates other types of useful data for bot detection or traffic analysis

David Puerta Martín 1 Nov 11, 2021
Ultimate Logger - A Discord bot that logs lots of events in a channel written in python

Ultimate Logger - A Discord bot that logs lots of events in a channel written in python

Luca 2 Mar 27, 2022
蓝鲸日志平台(BK-LOG)是为解决分布式架构下日志收集、查询困难的一款日志产品,基于业界主流的全文检索引擎

蓝鲸日志平台(BK-LOG)是为解决分布式架构下日志收集、查询困难的一款日志产品,基于业界主流的全文检索引擎,通过蓝鲸智云的专属 Agent 进行日志采集,提供多种场景化的采集、查询功能。

腾讯蓝鲸 102 Dec 22, 2022
A very basic esp32-based logic analyzer capable of sampling digital signals at up to ~3.2MHz.

A very basic esp32-based logic analyzer capable of sampling digital signals at up to ~3.2MHz.

Davide Della Giustina 43 Dec 27, 2022
APT-Hunter is Threat Hunting tool for windows event logs

APT-Hunter is Threat Hunting tool for windows event logs which made by purple team mindset to provide detect APT movements hidden in the sea of windows event logs to decrease the time to uncover susp

824 Jan 08, 2023
Python logging package for easy reproducible experimenting in research

smilelogging Python logging package for easy reproducible experimenting in research. Why you may need this package This project is meant to provide an

Huan Wang 20 Dec 23, 2022
A Python package which supports global logfmt formatted logging.

Python Logfmter A Python package which supports global logfmt formatted logging. Install $ pip install logfmter Usage Before integrating this library,

Joshua Taylor Eppinette 15 Dec 29, 2022
Structured Logging for Python

structlog makes logging in Python faster, less painful, and more powerful by adding structure to your log entries. It's up to you whether you want str

Hynek Schlawack 2.3k Jan 05, 2023
A simple, transparent, open-source key logger, written in Python, for tracking your own key-usage statistics.

A simple, transparent, open-source key logger, written in Python, for tracking your own key-usage statistics, originally intended for keyboard layout optimization.

Ga68 56 Jan 03, 2023
loghandler allows you to easily log messages to multiple endpoints.

loghandler loghandler allows you to easily log messages to multiple endpoints. Using Install loghandler via pip pip install loghandler In your code im

Mathias V. Nielsen 2 Dec 04, 2021
Python bindings for g3log

g3logPython Python bindings for g3log This library provides python3 bindings for g3log + g3sinks (currently logrotate, syslog, and a color-terminal ou

4 May 21, 2021
The easy way to send notifications

See changelog for recent changes Got an app or service and you want to enable your users to use notifications with their provider of choice? Working o

Or Carmi 2.4k Dec 25, 2022
Token Logger with python

Oxy Token Stealer Features Grabs discord tokens Grabs chrome passwords Grabs edge passwords Nothing else, I don't feel like releasing full on malware

oxy 1 Feb 12, 2022
👻 - Simple Keylloger with Socket

Keyllogs 👻 - Simple Keylloger with Socket Keyllogs 🎲 - Run Keyllogs

Bidouffe 3 Mar 28, 2022
Greppin' Logs: Leveling Up Log Analysis

This repo contains sample code and example datasets from Jon Stewart and Noah Rubin's presentation at the 2021 SANS DFIR Summit titled Greppin' Logs. The talk was centered around the idea that Forens

Stroz Friedberg 20 Sep 14, 2022
Command-line tool that instantly fetches Stack Overflow results when an exception is thrown

rebound Rebound is a command-line tool that instantly fetches Stack Overflow results when an exception is thrown. Just use the rebound command to exec

Jonathan Shobrook 3.9k Jan 03, 2023
changedetection.io - The best and simplest self-hosted website change detection monitoring service

changedetection.io - The best and simplest self-hosted website change detection monitoring service. An alternative to Visualping, Watchtower etc. Designed for simplicity - the main goal is to simply

7.3k Jan 01, 2023