Cached file system for online resources in Python

Overview

Minato

Actions Status Python version License pypi version

Cache & file system for online resources in Python

Features

Minato enables you to:

  • Download & cache online recsources
    • minato supports the following protocols: HTTP(S) / AWS S3 / Google Cloud Storage
    • You can manage cached files via command line interface
  • Automatically update cached files based on ETag
    • minato downloads new versions if available when you access cached files
  • Open online files super easily
    • By using minato.open, you can read/write online resources like the built-in open method

Installation

pip install minato[all]

Usage

Python

import minato

# Read / write files on online storage
with minato.open("s3://your_bucket/path/to/file", "w") as f:
    f.write("Create a new file on AWS S3!")

# Cache & manage online resources in local storage
local_filename = minato.cached_path("http://example.com/path/to/archive.zip!inner/path/to/file")

CLI

❯ poetry run minato --help
usage: minato

positional arguments:
  {cache,list,remove,update}
    cache               cache remote file and return cached local file path
    list                show list of cached files
    remove              remove cached files
    update              update cached files

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
You might also like...
Exploit Discord's cache system to remote upload payloads on Discord users machines
Exploit Discord's cache system to remote upload payloads on Discord users machines

Exploit Discord's cache system to hide payloads PoC Remote upload embedded payload from image using EOF to Discord users machines through cache. Depen

A generic system for filtering Django QuerySets based on user selections

Django Filter Django-filter is a reusable Django application allowing users to declaratively add dynamic QuerySet filtering from URL parameters. Full

A Django based shop system

django-SHOP Django-SHOP aims to be a the easy, fun and fast e-commerce counterpart to django-CMS. Here you can find the full documentation for django-

Login System Django

Login-System-Django Login System Using Django Tech Used Django Python Html Run Locally Clone project git clone https://link-to-project Get project for

A Blog Management System Built with django
A Blog Management System Built with django

Blog Management System Backend use: Django Features Enhanced Ui

This repository contains django library management system project.

Library Management System Django ** INSTALLATION** First of all install python on your system. Then run pip install -r requirements.txt to required se

Djang Referral System
Djang Referral System

Djang Referral System About | Features | Technologies | Requirements | Starting | License | Author 🎯 About I created django referral system and I wan

Advanced school management system written in Django :)
Advanced school management system written in Django :)

Advanced school management system written in Django :) ⚙️ Config the project First you should make venv for this project. So in the main root of proje

A visual indicator of what environment/system you're using in django

A visual indicator of what environment/system you're using in django

Comments
  • Bump protobuf from 4.21.5 to 4.21.6

    Bump protobuf from 4.21.5 to 4.21.6

    Bumps protobuf from 4.21.5 to 4.21.6.

    Release notes

    Sourced from protobuf's releases.

    Protocol Buffers v3.0.0-beta-4

    Version 3.0.0-beta-4

    General

    • Added a deterministic serialization API for C++. The deterministic serialization guarantees that given a binary, equal messages will be serialized to the same bytes. This allows applications like MapReduce to group equal messages based on the serialized bytes. The deterministic serialization is, however, NOT canonical across languages; it is also unstable across different builds with schema changes due to unknown fields. Users who need canonical serialization, e.g. persistent storage in a canonical form, fingerprinting, etc, should define their own canonicalization specification and implement the serializer using reflection APIs rather than relying on this API.

    • Added OneofOptions. You can now define custom options for oneof groups.

      import "google/protobuf/descriptor.proto";
      extend google.protobuf.OneofOptions {
        optional int32 my_oneof_extension = 12345;
      }
      message Foo {
        oneof oneof_group {
          (my_oneof_extension) = 54321;
          ...
        }
      }
      

    C++ (beta)

    • Introduced a deterministic serialization API in CodedOutputStream::SetSerializationDeterministic(bool). See the notes about deterministic serialization in the General section.
    • Added google::protobuf::Map::swap() to swap two map fields.
    • Fixed a memory leak when calling Reflection::ReleaseMessage() on a message allocated on arena.
    • Improved error reporting when parsing text format protos.
    • JSON
      • Added a new parser option to ignore unknown fields when parsing JSON.
      • Added convenient methods for message to/from JSON conversion.
    • Various performance optimizations.

    Java (beta)

    • File option "java_generate_equals_and_hash" is now deprecated. equals() and hashCode() methods are generated by default.
    • Added a new JSON printer option "omittingInsignificantWhitespace" to produce a more compact JSON output. The printer will pretty-print by default.
    • Updated Java runtime to be compatible with 2.5.0/2.6.1 generated protos.

    Python (beta)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump ujson from 5.3.0 to 5.4.0

    Bump ujson from 5.3.0 to 5.4.0

    Bumps ujson from 5.3.0 to 5.4.0.

    Release notes

    Sourced from ujson's releases.

    5.4.0

    Added

    Fixed

    Commits
    • 9c20de0 Merge pull request from GHSA-fm67-cv37-96ff
    • b21da40 Fix double free on string decoding if realloc fails
    • 67ec071 Merge pull request #555 from JustAnotherArchivist/fix-decode-surrogates-2
    • bc7bdff Replace wchar_t string decoding implementation with a uint32_t-based one
    • cc70119 Merge pull request #548 from JustAnotherArchivist/arbitrary-ints
    • 4b5cccc Merge pull request #553 from bwoodsend/pypy-ci
    • abe26fc Merge pull request #551 from bwoodsend/bye-bye-travis
    • 3efb5cc Delete old TravisCI workflow and references.
    • 404de1a xfail test_decode_surrogate_characters() on Windows PyPy.
    • f7e66dc Switch to musl docker base images.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 0
Releases(v0.10.0)
  • v0.10.0(Aug 11, 2022)

    What's Changed

    • Bump ujson from 5.3.0 to 5.4.0 by @dependabot in https://github.com/altescy/minato/pull/36
    • Table supports full-width characters by @altescy in https://github.com/altescy/minato/pull/37
    • Modify auto-format settings by @altescy in https://github.com/altescy/minato/pull/38
    • Add progress & remove tqdm by @altescy in https://github.com/altescy/minato/pull/39
    • Add disable-progressbar option by @altescy in https://github.com/altescy/minato/pull/40
    • Add interactive selector by @altescy in https://github.com/altescy/minato/pull/41
    • Bump version number to v0.10.0 by @altescy in https://github.com/altescy/minato/pull/42

    New Contributors

    • @dependabot made their first contribution in https://github.com/altescy/minato/pull/36

    Full Changelog: https://github.com/altescy/minato/compare/v0.9.0...v0.10.0

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Jul 2, 2022)

    What's Changed

    • Use options for open_file function by @altescy in https://github.com/altescy/minato/pull/33
    • Support Python 3.9 and 3.10 by @altescy in https://github.com/altescy/minato/pull/34
    • Bump version number to v0.9.0 by @altescy in https://github.com/altescy/minato/pull/35

    Full Changelog: https://github.com/altescy/minato/compare/v0.8.2...v0.9.0

    Source code(tar.gz)
    Source code(zip)
  • v0.8.2(Feb 24, 2022)

    What's Changed

    • Modify log level by @altescy in https://github.com/altescy/minato/pull/30
    • Upgrade dependent packages by @altescy in https://github.com/altescy/minato/pull/31
    • Bump version number to v0.8.2 by @altescy in https://github.com/altescy/minato/pull/32

    Full Changelog: https://github.com/altescy/minato/compare/v0.8.1...v0.8.2

    Source code(tar.gz)
    Source code(zip)
  • v0.8.1(Feb 11, 2022)

    What's Changed

    • Fix s3 progress by @altescy in https://github.com/altescy/minato/pull/27
    • Add all command to Makefile by @altescy in https://github.com/altescy/minato/pull/28
    • Bump version number to v0.8.1 by @altescy in https://github.com/altescy/minato/pull/29

    Full Changelog: https://github.com/altescy/minato/compare/v0.8.0...v0.8.1

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Feb 8, 2022)

    What's Changed

    • Modify tqdm progress by @altescy in https://github.com/altescy/minato/pull/22
    • Upgrade dependent packages by @altescy in https://github.com/altescy/minato/pull/23
    • Split github workflows into ci and pypi-publish by @altescy in https://github.com/altescy/minato/pull/24
    • Modify type annotations by @altescy in https://github.com/altescy/minato/pull/25
    • Bump version number to v0.8.0 by @altescy in https://github.com/altescy/minato/pull/26

    Full Changelog: https://github.com/altescy/minato/compare/v0.7.0...v0.8.0

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Nov 19, 2021)

    • Shrink table column width automatically (#16)
    • Modify S3 download (#17)
    • Move boto3-stubs to dev-dependencies (#18)
    • Add upload method to S3FileSystem (#19)
    • Add upload method to GCSFileSystem (#20)
    • Bump version number to v0.7.0 (#21)
    Source code(tar.gz)
    Source code(zip)
  • v0.6.2(Oct 23, 2021)

  • v0.6.1(Oct 1, 2021)

  • v0.6.0(Sep 30, 2021)

    • Remove quotations of S3 Etags (#2)
    • Modify CI settings (#3)
    • Update dependent packages (#4)
    • Make filesystems optional (#5)
    • Add opening options (#6)
    • Add helps to each command line options (#7)
    • Add feature description to readme (#8)
    • Bump version number to v0.6.0 (#9)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Jul 31, 2021)

  • v0.4.0(Jun 25, 2021)

    • Support file-level expiration
    • Enable automatic cache updating based on HTTP ETag
    • Show download progress of S3 and GCS
    • Deprecate upload/download commands
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Jun 23, 2021)

Owner
Yasuhiro Yamaguchi
Research Engineer / NLP / ML
Yasuhiro Yamaguchi
A simple REST API to manage postal addresses, written in Python/Django.

A simple REST API to manage postal addresses, written in Python/Django.

Attila Bagossy 2 Feb 14, 2022
Cookiecutter Django is a framework for jumpstarting production-ready Django projects quickly.

Cookiecutter Django Powered by Cookiecutter, Cookiecutter Django is a framework for jumpstarting production-ready Django projects quickly. Documentati

10.1k Jan 08, 2023
Django Course Project - TextCorrector

Django-TextUtils Django Course Project A tool for analyzing text data in Django backend. It is a project where you can do some of the things with you

1 Oct 29, 2021
A simple app that provides django integration for RQ (Redis Queue)

Django-RQ Django integration with RQ, a Redis based Python queuing library. Django-RQ is a simple app that allows you to configure your queues in djan

RQ 1.6k Jan 06, 2023
Django app for building dashboards using raw SQL queries

django-sql-dashboard Django app for building dashboards using raw SQL queries Brings a useful subset of Datasette to Django. Currently only works with

Simon Willison 383 Jan 06, 2023
A fresh approach to autocomplete implementations, specially for Django. Status: v3 stable, 2.x.x stable, 1.x.x deprecated. Please DO regularely ping us with your link at #yourlabs IRC channel

Features Python 2.7, 3.4, Django 2.0+ support (Django 1.11 (LTS), is supported until django-autocomplete-light-3.2.10), Django (multiple) choice suppo

YourLabs 1.7k Jan 01, 2023
Simple tagging for django

django-taggit This is a Jazzband project. By contributing you agree to abide by the Contributor Code of Conduct and follow the guidelines. django-tagg

Jazzband 3k Jan 02, 2023
Django Query Capture can check the query situation at a glance, notice slow queries, and notice where N+1 occurs.

django-query-capture Overview Django Query Capture can check the query situation at a glance, notice slow queries, and notice where N+1 occurs. Some r

GilYoung Song 80 Nov 22, 2022
PostgreSQL with Docker + Portainer + pgAdmin + Django local

django-postgresql-docker Running PostgreSQL with Docker + Portainer + pgAdmin + Django local for development. This project was done with: Python 3.9.8

Regis Santos 4 Jun 12, 2022
☄️ Google Forms autofill script

lazrr 'Destroy Them With Lazers' - Knife Party, 2011 Google Forms autofill script Installation: pip3 install -r requirements.txt Usage: python3 lazrr.

Serezha Rakhmanov 12 Jun 04, 2022
Strawberry-django-plus - Enhanced Strawberry GraphQL integration with Django

strawberry-django-plus Enhanced Strawberry integration with Django. Built on top

BLB Ventures 138 Dec 28, 2022
Keep track of failed login attempts in Django-powered sites.

django-axes Axes is a Django plugin for keeping track of suspicious login attempts for your Django based website and implementing simple brute-force a

Jazzband 1.1k Dec 30, 2022
Source files for a free pyRevit toolbar.

pyRoovit (WIP) What is this? PyRoovit is/will be a toolbar for the use with pyRevit built by Gavin Crump (aka Aussie BIM Guru). Having used and taught

Gavin Crump 11 Nov 10, 2022
A Powerful HTML white space remover for Django

HTML Whitespace remover for Django Introduction : A powerful tool to optimize Django rendered templates Why use "django_stip_whitespace" ? Adds line b

3 Jan 01, 2022
It's the assignment 1 from the Python 2 course, that requires a ToDoApp with authentication using Django

It's the assignment 1 from the Python 2 course, that requires a ToDoApp with authentication using Django

0 Jan 20, 2022
Analytics services for Django projects

django-analytical The django-analytical application integrates analytics services into a Django project. Using an analytics service with a Django proj

Jazzband 1.1k Dec 31, 2022
Djangoblog - A blogging platform built on Django and Python.

djangoblog 👨‍💻 A blogging platform built on Django and Python

Lewis Gentle 1 Jan 10, 2022
The friendly PIL fork (Python Imaging Library)

Pillow Python Imaging Library (Fork) Pillow is the friendly PIL fork by Alex Clark and Contributors. PIL is the Python Imaging Library by Fredrik Lund

Pillow 10.4k Jan 03, 2023
Template for Django Project Using Docker

You want a Django project who use Docker and Docker-compose for Development and for Production ? It's for you !

1 Dec 17, 2021
Book search Django web project that uses requests python library and openlibrary API.

Book Search API Developer: Vladimir Vojtenko Book search Django web project that uses requests python library and openlibrary API. #requests #openlibr

1 Dec 08, 2021