YAML metadata extension for Python-Markdown

Overview

YAML metadata extension for Python-Markdown

Build Status Coverage Status Code style: black Python versions PyPi

This extension adds YAML meta data handling to markdown with all YAML features.

As in the original, metadata is parsed but not used in processing.

Metadata parsed as is by PyYaml and without additional transformations, so this plugin is not compatible with original Meta-Data extension.

Basic Usage

Lorem Ipsum is simply dummy text.

' md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}">
import markdown


text = """---
title: What is Lorem Ipsum?
categories:
  - Lorem Ipsum
  - Stupid content
...

Lorem Ipsum is simply dummy text.
"""

md = markdown.Markdown(extensions=['full_yaml_metadata']})
md.convert(text) == '

Lorem Ipsum is simply dummy text.

'
md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}

Specify a custom YAML loader

By default the full YAML loader is used for parsing, which is insecure when used with untrusted user data. In such cases, you may want to specify a different loader such as yaml.SafeLoader using the extension_configs keyword argument:

import markdown
import yaml

md = markdown.Markdown(extensions=['full_yaml_metadata']}, extension_configs={
        "full_yaml_metadata": {
            "yaml_loader": yaml.SafeLoader,
        },
    },
)

Development and contribution

  • install project dependencies
python setup.py develop
  • install linting, formatting and testing tools
pip install -r requirements.txt
  • run tests
pytest
  • run linters
flake8
mypy ./
black --check ./
  • feel free to contribute!
Comments
  • Move setup_requires to pyproject.yaml

    Move setup_requires to pyproject.yaml

    When build system (setuptools) requirements are specified in setup.py, they end up being installed by distutils, even when pip installing. Because distutils is bit-rotting, it doesn't work with system installed openssl.

    Locally, for me, that means distutils doesn't know about some SSL CA certs, and as such, a pip install markdown-full-yaml-metadata will fail when trying to install setuptools_markdown due to being unable to validate the SSL cert (I am behind a coorporate proxy which MITMs all traffic and resigns with a local cert).

    Moving the setup_requires deps to pyproject.toml fixes this - I suggest something like this:

    [build-system]
    # Minimum requirements for the build system to execute.
    requires = ["setuptools>=36.6", "setuptools_markdown", "wheel",]
    build-backend = "setuptools.build_meta"
    
    opened by jonathanunderwood 16
  • Fixes9+10

    Fixes9+10

    change to README.md on how extension is invoked in python interactive shell closes #9 change to full_yaml_metadata.py to makeExtension parameters closes #10

    opened by philbarker 9
  • RFE: don't pin dependencies exactly

    RFE: don't pin dependencies exactly

    This extension pins every dependency exactly. That's a fine strategy for an end-user application, but for a library it's not really the right thing to do, as you inevitably end up with conflicting dependencies in a project that consumes the library. Please would you consider having more liberal dependencies (eg. markdown>3.0, rather than markdown==3.0.1) for example.

    Thanks for a great extension, by the way!

    opened by jonathanunderwood 4
  • Allow specifing custom loader using configuration option

    Allow specifing custom loader using configuration option

    Currently there is no way to use the yaml.BaseLoader for example. The full loader is unsafe for arbitrary user data and also converts strings like 2020-01-01 10:00:00 to datetime.datetime objects, which might be undesired.

    opened by Holzhaus 1
  • url for pypi at top of page is wrong

    url for pypi at top of page is wrong

    Website link in project description for pypi is wrong https://pypi.python.org/pypi/makrdown… ('makr', should be 'mark') -- I guess this worked before fixing #4

    opened by philbarker 1
  • KeyError: 'configs' on running

    KeyError: 'configs' on running

    After getting extension to load properly, got error:

    >>> import markdown
    >>> md = markdown.Markdown(extensions=['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 100, in __init__
        configs=kwargs.get('extension_configs', {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 126, in registerExtensions
        ext = self.build_extension(ext, configs.get(ext, {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 181, in build_extension
        return module.makeExtension(**configs)
      File "/home/phil/Share/Projects/full-yaml-test/python-markdown-full-yaml-metadata/full_yaml_metadata.py", line 45, in makeExtension
        return FullYamlMetadataExtension(configs=configs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 42, in __init__
        self.setConfigs(kwargs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 73, in setConfigs
        self.setConfig(key, value)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 61, in setConfig
        if isinstance(self.config[key][0], bool):
    KeyError: 'configs'
    

    I fixed by changing

    def makeExtension(configs: dict={}):
        return FullYamlMetadataExtension(configs=configs)
    

    to

    def makeExtension(*args, **kwargs):
        return FullYamlMetadataExtension(*args, **kwargs)
    

    Hope this helps.

    opened by philbarker 0
  • md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    Basic usage on pypi invokes extension with md = markdown.Markdown(['full_yaml_metadata'])

    I got the error:

    >>> md = markdown.Markdown(['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: __init__() takes 1 positional argument but 2 were given
    

    But this did work (or at least gave another error) md = markdown.Markdown(extensions=['full_yaml_metadata'])

    opened by philbarker 0
  • Support space after metadata delimitation

    Support space after metadata delimitation

    Hello ! I propose to add the support of spaces after metadata delimiting. Currently if a markdown file contains metadata and a space after the metadata delimiter, the markdown parser crashes. Example :

    ---
    title: What is Lorem Ipsum?
    ---        
    Lorem Ipsum is simply dummy text.
    

    There are spaces after the end of metadata declaration (the third line is equal to '--- '). If we try to parse the example above with the 'full_yaml_metadata' plugin we get the following error :

    E           yaml.composer.ComposerError: expected a single document in the stream
    E             in "<unicode string>", line 1, column 1:
    E               title: What is Lorem Ipsum?
    E               ^
    E           but found another document
    E             in "<unicode string>", line 2, column 1:
    E               ---        
    E               ^
    

    So I propose this little correction, hoping that you will accept it. If you have any remark on the code or on the test don't hesitate to sharing them with me, I would be glad to correct it.

    opened by Ricardaux 2
  • Using pre-commit hooks

    Using pre-commit hooks

    I noticed that some parts of the code don't pass your linter checks. This is common for projects with only one main developer where you sometimes push directly instead of opening a PR. I suggest to use pre-commit, which checks that all linters pass before committing. It also takes care dev dependency installation in a venv (so you can remove dev dependencies from requirements.txt.

    opened by Holzhaus 0
  • Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    This might work for very simple metadata, like:

    title: Foo bar
    date: 2020-01-01 10:00:00
    
    This is text
    

    If the metadata delimiters are not found and the option is enabled, the plugin could do something like this:

    meta, sep, content = text.partition("\n\n")
    try:
        metadata = yaml.load(meta, Loader=...)
    except yaml.error.YAMLError:
        content = text
        metadata = {}
    else:
        # Prevent false-positives if the text does not begin with a metadata block
        if isinstance(metadata, str):
            metadata = {}
            content = text
    
    self.md.Meta = metadata
    return content
    opened by Holzhaus 3
Releases(2.1.0)
Owner
Nikita Sivakov
New World Disorder
Nikita Sivakov
the project for the most brutal and effective language learning technique

- "The project for the most brutal and effective language learning technique" (c) Alex Kay The langflow project was created especially for language le

Alexander Kaigorodov 7 Dec 26, 2021
Service for visualisation of high dimensional for hydrosphere

hydro-visualization Service for visualization of high dimensional for hydrosphere DEPENDENCIES DEBUG_ENV = bool(os.getenv("DEBUG_ENV", False)) APP_POR

hydrosphere.io 1 Nov 12, 2021
CoderByte | Practice, Tutorials & Interview Preparation Solutions|

CoderByte | Practice, Tutorials & Interview Preparation Solutions This repository consists of solutions to CoderByte practice, tutorials, and intervie

Eda AYDIN 6 Aug 09, 2022
204-python-string-21BCA90 created by GitHub Classroom

204-Python This repository is created for subject "204 Programming Skill" Python Programming. This Repository contain list of programs of python progr

VIDYABHARTI TRUST COLLEGE OF BCA 6 Mar 31, 2022
Generate modern Python clients from OpenAPI

openapi-python-client Generate modern Python clients from OpenAPI 3.x documents. This generator does not support OpenAPI 2.x FKA Swagger. If you need

555 Jan 02, 2023
Type hints support for the Sphinx autodoc extension

sphinx-autodoc-typehints This extension allows you to use Python 3 annotations for documenting acceptable argument types and return value types of fun

Alex Grönholm 462 Dec 29, 2022
📖 Generate markdown API documentation from Google-style Python docstring. The lazy alternative to Sphinx.

lazydocs Generate markdown API documentation for Google-style Python docstring. Getting Started • Features • Documentation • Support • Contribution •

Machine Learning Tooling 118 Dec 31, 2022
layout-parser 3.4k Dec 30, 2022
Main repository for the Sphinx documentation builder

Sphinx Sphinx is a tool that makes it easy to create intelligent and beautiful documentation for Python projects (or other documents consisting of mul

5.1k Jan 04, 2023
A repository of links with advice related to grad school applications, research, phd etc

A repository of links with advice related to grad school applications, research, phd etc

Shaily Bhatt 946 Dec 30, 2022
NetBox plugin for BGP related objects documentation

Netbox BGP Plugin Netbox plugin for BGP related objects documentation. Compatibility This plugin in compatible with NetBox 2.10 and later. Installatio

Nikolay Yuzefovich 133 Dec 27, 2022
More detailed upload statistics for Nicotine+

More Upload Statistics A small plugin for Nicotine+ 3.1+ to create more detailed upload statistics. ⚠ No data previous to enabling this plugin will be

Nick 1 Dec 17, 2021
swagger-codegen contains a template-driven engine to generate documentation, API clients and server stubs in different languages by parsing your OpenAPI / Swagger definition.

Master (2.4.25-SNAPSHOT): 3.0.31-SNAPSHOT: Maven Central ⭐ ⭐ ⭐ If you would like to contribute, please refer to guidelines and a list of open tasks. ⭐

Swagger 15.2k Dec 31, 2022
Hasköy is an open-source variable sans-serif typeface family

Hasköy Hasköy is an open-source variable sans-serif typeface family. Designed with powerful opentype features and each weight includes latin-extended

67 Jan 04, 2023
The OpenAPI Specification Repository

The OpenAPI Specification The OpenAPI Specification is a community-driven open specification within the OpenAPI Initiative, a Linux Foundation Collabo

OpenAPI Initiative 25.5k Dec 29, 2022
ReStructuredText and Sphinx bridge to Doxygen

Breathe Packagers: PGP signing key changes for Breathe = v4.23.0. https://github.com/michaeljones/breathe/issues/591 This is an extension to reStruct

Michael Jones 643 Dec 31, 2022
Reproducible Data Science at Scale!

Pachyderm: The Data Foundation for Machine Learning Pachyderm provides the data layer that allows machine learning teams to productionize and scale th

Pachyderm 5.7k Dec 29, 2022
[Unofficial] Python PEP in EPUB format

PEPs in EPUB format This is a unofficial repository where I stock all valid PEPs in the EPUB format. Repository Cloning git clone --recursive Mickaël Schoentgen 9 Oct 12, 2022

A tutorial for people to run synthetic data replica's from source healthcare datasets

Synthetic-Data-Replica-for-Healthcare Description What is this? A tailored hands-on tutorial showing how to use Python to create synthetic data replic

11 Mar 22, 2022
Python solutions to solve practical business problems.

Python Business Analytics Also instead of "watching" you can join the link-letter, it's already being sent out to about 90 people and you are free to

Derek Snow 357 Dec 26, 2022