Taking the fight to the establishment.

Overview

Throwdown

Taking the fight to the establishment.

Wat?

I wanted a simple markdown interpreter in python and/or javascript to output html for my website. Python does not have a bug-free official distribution, javascript only has things you install through npm and I don't want to have anything to do with node and the 100 MB of dependencies you end up uploading to your FTP server in order to do the most basic tasks.

So writing my own parser it is then eh? I tried to trudge through the commonmark markdown spec and had a heart attack at the complexity. 24722 words and 181 pages of complicated language explaining features I absolutely don't need.

I just want minimal, well defined, syntactical elements with maximum payoff, so here is throwdown. Taking the fight to the establishment to have a stupidly minimal markup language in both definition and capability.

Goals

Keep it a subset for markdown so we can use existing IDEs & plugins. Support HTML tags in line with text. Write a well defined language spec in the manual to ease creating new interpreters for it.

The spec

Tokenization

Given a piece of text we tokenize the following concepts (these are regular expressions using DOTALL and MULTILINE modifiers):

blank_line(s): (\r | \n | \r\n){2,}
html tag: <.*?>
code: ```.*?```
unescaped italic: ^_|(?

any characters inbetween matching tokens are flagged content.

Content itself gets an additional treatment where we replace this regex

\\(?)

for escaped characters, with whatever was in the matched group. I currently do this in the generation stage but it could move to any stage.

I am not sure if in the UTF8 2/3/4 byte characters any of these elements may match, so make sure to perform these single-characetr checks per unicode char, not per byte.

Parsing

We then have a parsing pass that tries to group matching tokens:

In this example:

This *word* is bold but this* is wrong.

We have the following tokens:

content, bold, content, bold, content, bold, content

The parser simply finds any content block surrounded by matching code|italic|bold neighbours, and then 'consumes' these neighbours so they can not be picked up more than once. Reading from left to right this means we get (note we search outwards from content recursively to support *_content_* notations, instead of holding on to the boundary tokens as soon as we encounter them):

content group content bold content

Then, any token outside of a group gets merged into it's content, any consecutive content gets merged into 1 content. The first step reduced the bold into it's left neighbour:

content group content content

The next step reduces the two content blocks into one:

content group content

The above step should include html tags.

A final step is to remove the blank line tokens, but first we must make sure to merge consecutive group and content blocks, because after this any consecutive content and/or group tokens are known unique paragraphs (or headers) so the blank lines are no longer necessary to imply this separation.

Generation

Then there is the generation step. We simply walk the resulting tokens and output a html document.

  • If a content group is preceded by a heading, the node gets wrapped into tags where n is the number of #.
  • Every other content node gets wrapped into

    tags.

  • Every group gets wrapped based on the first and last tokens (which are identical).
    • italic becomes In this case the wrapping is recursive, a bold group in an intalic group may exist.
    • bold becomes In this case the wrapping is recursive, an italic group in a bold group may exist.
    • code becomes

Write
to insert single line breaks manually.

TODO:

Consider bullet points and numbered lists, though the html is not super invasive.

Owner
Trevor van Hoof
I write tools & shaders TropicalTrevor in the Demoscene
Trevor van Hoof
Python: Wrangled and unpivoted gaming datasets. Tableau: created dashboards - Market Beacon and Player’s Shopping Guide.

Created two information products for GameStop. Using Python, wrangled and unpivoted datasets, and created Tableau dashboards.

Zinaida Dvoskina 2 Jan 29, 2022
Fiber implements an proof-of-concept Python decorator that rewrites a function

Fiber implements an proof-of-concept Python decorator that rewrites a function so that it can be paused and resumed (by moving stack variables to a heap frame and adding if statements to simulate jum

Tyler Hou 225 Dec 13, 2022
Hashcrack - A non-object oriented open source, Software for Windows/Linux made in Python 3

Multi Force This project is a non-object oriented open source, Software for Wind

Radiationbolt 3 Jan 02, 2023
Predicting Global Crop Yield for World Hunger

Crop Yield And Global Famine - The fifth project I created during my time at General Assembly. I completed this project with three other classmates in the span of three weeks. Most of my work was dir

Adam Muhammad Klesc 2 Jun 19, 2022
Code needed for hybrid land cover change analysis for NASA IDS project

Documentation for the NASA IDS change analysis Poley 10/21/2021 Required python packages: whitebox numpy rasterio rasterio.mask os glob math itertools

Andrew Poley 2 Nov 12, 2021
Building an Investment Portfolio for Day Trade with Python

Montando um Portfólio de Investimentos para Day Trade com Python Instruções: Para reproduzir o projeto no Google Colab, faça o download do repositório

Paula Campigotto 9 Oct 26, 2021
Just some information about this nerd.

Greetings, mates, I am ErrorDIM - aka ErrorDimension 👋 🧬 Programming Languages I Can Use: 🥇 Top Starred Repositories: # Name Stars Size Major Langu

ErrorDIM 3 Jan 11, 2022
【幼盾】个性化图片徽章服务!

【幼盾】个性化图片徽章服务! 你对方形的徽章感到无聊了吗?想要定制属于自己的开源项目徽章了吗? 快来使用unv-shield吧! unv-shield提供包含自定义图片的徽章服务,可以让你的项目主页更加个性化!

黄巍 130 Dec 23, 2022
Project aims to map out common user behavior on the computer

User-Behavior-Mapping-Tool Project aims to map out common user behavior on the computer. Most of the code is based on the research by kacos2000 found

trustedsec 136 Dec 23, 2022
用于红队成员初步快速攻击的全自动化工具。

关于 Author:m0sway Mail:[email protected] Github:https://www.github.com/m0sway/Jud JuD是

m0sway 46 Jul 21, 2022
Web站点选优工具 - 优化GitHub的打开速度、高效Clone

QWebSiteOptimizer - Web站点速度选优工具 在访问GitHub等网站时,DNS解析到的IP地址可能并不是最快,过慢的节点会严重影响我们的访问情况,故制作出这样的工具来进一步优化网络质量。 由于该方案并非为VPN等方式进行的速度优化,以下几点需要您注意: 后续访问对应网站时仍可能需

QPT Family 15 May 01, 2022
Expense-manager - Expense manager with python

Expense_manager TO-DO Source extractor: Credit Card, Wallet Destination extracto

1 Feb 13, 2022
AlexaUsingPython - Alexa will pay attention to your order, as: Hello Alexa, play music, Hello Alexa

AlexaUsingPython - Alexa will pay attention to your order, as: Hello Alexa, play music, Hello Alexa, what's the time? Alexa will pay attention to your order, get it, and afterward do some activity as

Abubakar Sattar 10 Aug 18, 2022
Personal Assistant Tessa

Personal Assistant Tessa Introducing our all new personal assistant Tessa..... An intelligent virtual assistant (IVA) or intelligent personal assistan

Anusha Joseph 4 Mar 08, 2022
Run Python code right in your Telegram messages

Run Python code right in your Telegram messages Made with Telethon library, TGPy is a tool for evaluating expressions and Telegram API scripts. Instal

29 Nov 22, 2022
PhD document for navlab

PhD_document_for_navlab The project contains the relative software documents which I developped or used during my PhD period. It includes: FLVIS. A st

ZOU YAJING 9 Feb 21, 2022
Anki Cards for the HSK vocabulary Chinese-German

Anki-HanyuShuipingKaoshi Anki Cards for the HSK vocabulary Chinese-German Das Deck baut auf folgenden Quellen auf: China Endecken Wortschatz von wohok

1 Jan 07, 2022
Opensource Desktop application for kenobi.

Kenobi-Server WIP Opensource desktop application for Kenobi. Download the apple watch app to get started. What is this repo? It's repo for the opensou

Aayush 9 Oct 08, 2022
A shim for the typeshed changes in mypy 0.900

types-all A shim for the typeshed changes in mypy 0.900 installation pip install types-all why --install-types is annoying, this installs all the thin

Anthony Sottile 28 Oct 20, 2022
Age of Empires II recorded game parsing and summarization in Python 3.

mgz Age of Empires II recorded game parsing and summarization in Python 3. Supported Versions Age of Kings (.mgl) The Conquerors (.mgx) Userpatch 1.4

148 Dec 11, 2022