Command line parser for common log format (Nginx default).

Related tags

Command-line Toolspythonnginxcommonlogging
Overview

Logparser

Command line parser for common log format (Nginx default).

Usage

It counts most important data: referrers, operating systems, browsers and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

Install and update PIP packages.

pip install -U -r requirements.txt

Speed

  • logparser 24,249/s
  • GoAccess 6,234/s

Outputs

  • HTML output is based on Jinja2 templates. It can be improved as you see fit.
  • Console output for Subreply for a fews days in Sep, 2021:
----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week
Owner
Lucian Marin
Python developer and all things design.
Lucian Marin
GitHub Repository
Library and command-line utility for rendering projects templates.

A library for rendering project templates. Works with local paths and git URLs. Your project can include any file and Copier can dynamically replace v

808 Jan 04, 2023
Lets you view, edit and execute Jupyter Notebooks in the terminal.

Lets you view, edit and execute Jupyter Notebooks in the terminal.

David Brochart 684 Dec 28, 2022
Tablicate - Python library for easy table creation and output to terminal

Tablicate Tablicate - Python library for easy table creation and output to terminal Features Column-wise justification alignment (left, right, center)

3 Dec 14, 2022
A CLI tool for searching and watching videos on youtube with no spyware and MPV and yt-dlp

A CLI tool for searching and watching videos on youtube with no spyware and MPV and yt-dlp

TruncatedDinosour 3 Feb 22, 2022
🐍The nx-python plugin allows users to create a basic python application using nx commands.

🐍 NxPy: Nx Python plugin This project was generated using Nx. The nx-python plugin allows users to create a basic python application using nx command

StandUP Communications 74 Aug 31, 2022
🎄 Advent of Code command-line tool.

🎄 advent-cli advent-cli is a command-line tool for interacting with Advent of Code, specifically geared toward writing solutions in Python. It can be

Christian Ferguson 6 Dec 01, 2022
👻 Ghoul is an easy to use information service, allowing you to get/add information on someone or something directly from your terminal.

👻 Ghoul is an easy to use information service, allowing you to get/add information on someone or something directly from your terminal. It c

Billy 11 Nov 10, 2021
Command-line tool for looking up colors and palettes.

Colorpedia Colorpedia is a command-line tool for looking up colors, shades and palettes. Supported color models: HEX, RGB, HSL, HSV, CMYK. Requirement

Joohwan Oh 282 Dec 27, 2022
A simple python implementation of a reverse shell

llehs A python implementation of a reverse shell. Note for contributors The project is open for contributions and is hacktoberfest registered! llehs u

Archisman Ghosh 2 Jul 05, 2022
Low-Cost Open Source Ventilator or PAPR

Last updated 2020/04/19 Low-Cost Open-Source Ventilator-ish Device or PAPR NOTE: This is currently an independent project not affiliated with any comm

Johnny Lee 1.7k Dec 21, 2022
A command-line tool to upload local files and pastebin pastes to your mega account, without signing in anywhere

A command-line tool to upload local files and pastebin pastes to your mega account, without signing in anywhere

ADI 4 Nov 17, 2022
Python library & console tool for controlling Xiaomi smart appliances

python-miio This library (and its accompanying cli tool) can be used to interface with devices using Xiaomi's miIO and MIoT protocols. Getting started

Teemu R. 2.4k Jan 02, 2023
CLI tool that helps manage shell libraries.

shmgr CLI tool that helps manage shell libraries. Badges 📛 project status badges: version badges: tools / frameworks used by test suite (i.e. used by

Bryan Bugyi 0 Dec 15, 2021
Euporie is a text-based user interface for running and editing Jupyter notebooks

Euporie is a text-based user interface for running and editing Jupyter notebooks

781 Jan 01, 2023
A simple CLI tool for getting region-specific status of Logz.io components.

About A simple CLI tool for checking the current status of Logz.io components per region. Built With Python 3 The following packeges (see requirements

Yotam Bernaz 1 Dec 11, 2021
CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.

jc JSONifies the output of many CLI tools and file-types for easier parsing in scripts

Kelly Brazil 5.8k Jan 03, 2023
PyArmor is a command line tool used to obfuscate python scripts

PyArmor is a command line tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts.

Dashingsoft 2k Jan 07, 2023
Program Command Line Interface (CLI) Sederhana: Pemesanan Nasi Goreng Hekel

Program ini merupakan aplikasi yang berjalan di dalam command line (terminal). Program ini menggunakan built-in library python yaitu argparse yang dapat menerima parameter saat program ini dijalankan

Habib Abdurrasyid 5 Nov 19, 2021
OneDriveExplorer - A command line and GUI based application for reconstructing the folder structure of OneDrive from the UserCid.dat file

OneDriveExplorer - A command line and GUI based application for reconstructing the folder structure of OneDrive from the UserCid.dat file

Brian Maloney 100 Dec 13, 2022
A simple note taker CLI program written in python

note-taker A simple note taker program written in python This allows you to snip your todo's, notes, and your tasks easily without extra charges Requi

marcusz 4 Nov 02, 2021