A pretty quick and simple interface to paramiko SFTP

Overview

sftpretty

A pretty quick and simple interface to paramiko SFTP. Provides multi-threaded routines with progress notifications for reliable, asynchronous transfers. This is a Python3 optimized fork of pysftp with additional features & improvements.

  • Built-in retry decorator
  • Hash function for integrity checking
  • Improved local & remote directory mapping
  • Improved logging mechanism
  • More tests
  • Multi-threaded directory transfers
  • Progress notifications
  • Support for digests & kex connection options
  • Support for ED25519 & ECDSA keys
  • Support for private key passwords
  • Thread-safe connection manager

Example

from sftpretty import Connection


# Basic

with Connection('hostname', username='me', password='secret') as sftp:
    # Temporarily chdir to public/.
    with sftp.cd('public'):
        # Upload file to public/ on remote.
        sftp.put('/my/local/filename')
        # Download a remote file from public/.
        sftp.get('remote_file')


with Connection('hostname', private_key='~/.ssh/id_ed25519',
                            private_key_pass='secret') as sftp:
    # Upload local directory to remote_directory.
    sftp.put_d('/my/local', '/remote_directory')

    # Recursively download a remote_directory and save it to /tmp locally.
    sftp.get_r('remote_directory', '/tmp')


# Advanced

with Connection('hostname', username='me', password='secret') as sftp:
    # Upload local directory to remote_directory. On occurance of any
    # exception or child of, passed in the tuple, retry the operation.
    # Between each attempt increment a pause equal to backoff * delay.
    # Run a total of tries (six) times including the first attempt.
    sftp.put_d('/my/local', '/remote_directory',
               exceptions=(NoValidConnectionsError,
                           socket.timeout,
                           SSHException),
               tries=6, backoff=2, delay=1)


with Connection('hostname', private_key='~/.ssh/id_ed25519',
                            private_key_pass='secret') as sftp:
    # Recursively download a remote_directory and save it to /tmp locally.
    # Don't confirm files, useful in a scenario where the server removes
    # the remote file immediately after download. Preserve remote mtime on
    # local copy
    sftp.get_r('remote_directory', '/tmp', confirm=False,
               preserve_mtime=True)

Additional Information

Requirements

paramiko >= 1.17.0

Supports

Tested on Python 3.6, 3.7, 3.8, 3.9

https://travis-ci.org/byteskeptical/sftpretty.svg?branch=master
Comments
  • put_d() unexpected remote path handling and issue with absolute Windows paths

    put_d() unexpected remote path handling and issue with absolute Windows paths

    I found two issues with the put_d() function.

    1. It looks like it doesn't accept absolute Windows paths. I get the error:

    ValueError: 'C:\\Users\\johnm\\PycharmProjects\\project\\.other\\folder\\file' does not start with '\\'

    1. When I call put_d("local_folder", "folder"), I get the following error:

    File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\sftpretty\__init__.py", line 563, in put confirm=confirm, preserve_mtime=preserve_mtime) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\sftpretty\__init__.py", line 554, in _put confirm=confirm) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 759, in put return self.putfo(fl, remotepath, file_size, callback, confirm) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 714, in putfo with self.file(remotepath, "wb") as fr: File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 372, in open t, msg = self._request(CMD_OPEN, filename, imode, attrblock) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 822, in _request return self._read_response(num) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 874, in _read_response self._convert_status(msg) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 903, in _convert_status raise IOError(errno.ENOENT, text) FileNotFoundError: [Errno 2] Path not found.

    As I understand, the path it refers to is the remote path. When I debug it, in put_d(), the paths it creates are the following: image As you can see, the remotedir "folder/local_folder/file" is probably the issue. In my remote location, it creates the folder "folder". This of course makes sense as it successfully calls self.mkdir_p(remotedir) in line 599. Do you think this has to do with the location of the self.mkdir_p(remotedir) call? Should there be code that also creates the subfolders of the "local_folder" in the remote location(here the "folder/local_folder" subfolder)?

    Thank you for your time.

    bug 
    opened by Johnmaras 9
  • Add support to use a single channel for all commands. Some servers do…

    Add support to use a single channel for all commands. Some servers do…

    Certain SFTP server do not allow to open a channel for every command. The original pysftp used a single channel for all actions. I've added a flag to allow us to work in a similar way, while by default keeping the new behavior.

    Those servers will simply close the connection on the second command issued and it makes sftpretty not very useful with these old(er) SFTP servers.

    opened by erans 5
  • Typo in argument name - tires/tries

    Typo in argument name - tires/tries

    Hi,

    In sftpretty/init.py:716 there is a typo in the tries argument which causes the call to putfo() to raise an exception

    retry() got an unexpected keyword argument 'tires'

    bug 
    opened by Johnmaras 2
  • Fix potential `_channel` `UnboundLocalError`

    Fix potential `_channel` `UnboundLocalError`

    Sometimes when assigning to _channel, an exception can occur in SFTPClient.from_transport(). Because _channel doesn't exist if that happens, it causes a second exception when trying to call close() in the finally block. Assign to the variable outside the try block initially so that this exception can't happen in the future. Also make sure that _channel is not None before we try to close it to account for it being sometimes None.

    bug 
    opened by RobertCochran 1
  • calling basicConfig at this level for a library that should be embedded changes all dowstream logs

    calling basicConfig at this level for a library that should be embedded changes all dowstream logs

    Calling basicConfig on the top of the file for a library that is embedded into other projects can cause issue. I suggest you drop it completely specifically because there are no change to it other than setting the default level to INFO.

    https://github.com/byteskeptical/sftpretty/blob/ca8e2414ac156276059fcd091cc91e489d856e7d/sftpretty/init.py#L22

    In our case it added ":" to the start of every log line.

    enhancement 
    opened by erans 1
Releases(1.0.5)
Extended refactoring capabilities for Python LSP Server using Rope.

pylsp-rope Extended refactoring capabilities for Python LSP Server using Rope. This is a plugin for Python LSP Server, so you also need to have it ins

36 Dec 24, 2022
nettrace is a powerful tool to trace network packet and diagnose network problem inside kernel.

nettrace nettrace is is a powerful tool to trace network packet and diagnose network problem inside kernel on TencentOS. It make use of eBPF and BCC.

84 Jan 01, 2023
No-dependency, single file NNTP server library for developing modern, rfc3977-compliant (bridge) NNTP servers.

nntpserver.py No-dependency, single file NNTP server library for developing modern, rfc3977-compliant (bridge) NNTP servers for python =3.7. Develope

Manos Pitsidianakis 44 Nov 14, 2022
An improved version of the original AutoDD

AutoDD = Automatically does the "due diligence" for you. If you want to know what stocks people are talking about on reddit, this little program might help you.

Steven Zhu 169 Oct 05, 2022
ProtOSINT is a Python script that helps you investigate Protonmail accounts and ProtonVPN IP addresses

ProtOSINT ProtOSINT is a Python script that helps you investigate ProtonMail accounts and ProtonVPN IP addresses. Description This tool can help you i

pixelbubble 249 Dec 23, 2022
Use Fast Redirect to easily redirect your domains.

Fast Redirect Use Fast Redirect to easily redirect your domains. Fast Redirects expects a JSON 'database'. This JSON 'database' contains the domains t

Cyberfusion 1 Dec 20, 2021
An API for controlling Wi-Fi connections on Balena devices.

Description An API for controlling Wi-Fi connections on Balena devices. It does not contain an interface, instead it provides API endpoints to send re

8 Dec 25, 2022
A simple chat room using socket and threading for handle multiple connections.

• Socket Chat Room was a little project for socket study. It works with a server handling the incoming connections from the clients. Clients send encoded messages while waiting for others clients mes

Guilherme de Oliveira 2 Mar 03, 2022
A gRPC-Web implementation for Python

Sonora Sonora is a Python-first implementation of gRPC-Web built on top of standard Python APIs like WSGI and ASGI for easy integration. Why? Regular

Alex Stapleton 216 Dec 30, 2022
Client library for relay - a service for relaying server side messages to the client side browsers via websockets.

Client library for relay - a service for relaying server side messages to the client side browsers via websockets.

getme 1 Nov 10, 2021
Multi-vendor library to simplify CLI connections to network devices

Netmiko Multi-vendor library to simplify CLI connections to network devices Why Netmiko? Network automation to screen-scraping devices is primarily co

Kirk Byers 3k Jan 01, 2023
Minimal, self-hosted, 0-config alternative to ngrok. Caddy+OpenSSH+50 lines of Python.

If you have a webserver running on one computer (say your development laptop), and you want to expose it securely (ie HTTPS) via a public URL, SirTunnel allows you to easily do that.

Anders Pitman 423 Jan 02, 2023
Program can control your server via discord bot

GTPS Controller Program can control your server via discord bot Require Python How To Use Download This Source Extract The Zip File Paste gtps.py to y

Lamp 2 Mar 15, 2022
Py script to aid in setting up the boot chime in OpenCore.

BootChime Py script to aid in setting up the boot chime in OpenCore. It does so by helping you locate your IOHDACodecDevices, IOHDACodecAddress values

CorpNewt 7 Sep 19, 2022
snappi-trex is a snappi plugin that allows executing scripts written using snappi with Cisco's TRex Traffic Generator

snappi-trex snappi-trex is a snappi plugin that allows executing scripts written using snappi with Cisco's TRex Traffic Generator Design snappi-trex c

Open Traffic Generator 14 Sep 07, 2022
Out-of-box Python RPC framework

typed-jsonrpc Out-of-box Python RPC framework. WIP. Make LSP easy for everyone. The conception of final usage: from typed_jsonrpc import * ls = Langu

Taine Zhao 4 Dec 28, 2021
Learn how modern web applications and microservice architecture work as you complete a creative assignment

Micro-service Создание микросервиса Цель работы Познакомиться с механизмом работы современных веб-приложений и микросервисной архитектуры в процессе в

Григорий Верховский 1 Dec 19, 2021
Building a Robust IOT device which is customizable, encrypted, secure and user friendly

Building a Robust IOT device which is customizable, encrypted, secure and user friendly, which uses a single GPIO pin to extract multiple sensor values

1 Jan 03, 2022
The can package provides controller area network support for Python developers

python-can The Controller Area Network is a bus standard designed to allow microcontrollers and devices to communicate with each other. It has priorit

Brian Thorne 904 Dec 29, 2022
A python socket.io client for Roboteur

Roboteur Client Example TODO Basic setup Install the requirements: $ pip install -r requirements.txt Run the application: $ python -m roboteur_client

Barry Buck 1 Oct 13, 2021