A pretty quick and simple interface to paramiko SFTP

Overview

sftpretty

A pretty quick and simple interface to paramiko SFTP. Provides multi-threaded routines with progress notifications for reliable, asynchronous transfers. This is a Python3 optimized fork of pysftp with additional features & improvements.

  • Built-in retry decorator
  • Hash function for integrity checking
  • Improved local & remote directory mapping
  • Improved logging mechanism
  • More tests
  • Multi-threaded directory transfers
  • Progress notifications
  • Support for digests & kex connection options
  • Support for ED25519 & ECDSA keys
  • Support for private key passwords
  • Thread-safe connection manager

Example

from sftpretty import Connection


# Basic

with Connection('hostname', username='me', password='secret') as sftp:
    # Temporarily chdir to public/.
    with sftp.cd('public'):
        # Upload file to public/ on remote.
        sftp.put('/my/local/filename')
        # Download a remote file from public/.
        sftp.get('remote_file')


with Connection('hostname', private_key='~/.ssh/id_ed25519',
                            private_key_pass='secret') as sftp:
    # Upload local directory to remote_directory.
    sftp.put_d('/my/local', '/remote_directory')

    # Recursively download a remote_directory and save it to /tmp locally.
    sftp.get_r('remote_directory', '/tmp')


# Advanced

with Connection('hostname', username='me', password='secret') as sftp:
    # Upload local directory to remote_directory. On occurance of any
    # exception or child of, passed in the tuple, retry the operation.
    # Between each attempt increment a pause equal to backoff * delay.
    # Run a total of tries (six) times including the first attempt.
    sftp.put_d('/my/local', '/remote_directory',
               exceptions=(NoValidConnectionsError,
                           socket.timeout,
                           SSHException),
               tries=6, backoff=2, delay=1)


with Connection('hostname', private_key='~/.ssh/id_ed25519',
                            private_key_pass='secret') as sftp:
    # Recursively download a remote_directory and save it to /tmp locally.
    # Don't confirm files, useful in a scenario where the server removes
    # the remote file immediately after download. Preserve remote mtime on
    # local copy
    sftp.get_r('remote_directory', '/tmp', confirm=False,
               preserve_mtime=True)

Additional Information

Requirements

paramiko >= 1.17.0

Supports

Tested on Python 3.6, 3.7, 3.8, 3.9

https://travis-ci.org/byteskeptical/sftpretty.svg?branch=master
Comments
  • put_d() unexpected remote path handling and issue with absolute Windows paths

    put_d() unexpected remote path handling and issue with absolute Windows paths

    I found two issues with the put_d() function.

    1. It looks like it doesn't accept absolute Windows paths. I get the error:

    ValueError: 'C:\\Users\\johnm\\PycharmProjects\\project\\.other\\folder\\file' does not start with '\\'

    1. When I call put_d("local_folder", "folder"), I get the following error:

    File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\sftpretty\__init__.py", line 563, in put confirm=confirm, preserve_mtime=preserve_mtime) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\sftpretty\__init__.py", line 554, in _put confirm=confirm) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 759, in put return self.putfo(fl, remotepath, file_size, callback, confirm) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 714, in putfo with self.file(remotepath, "wb") as fr: File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 372, in open t, msg = self._request(CMD_OPEN, filename, imode, attrblock) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 822, in _request return self._read_response(num) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 874, in _read_response self._convert_status(msg) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 903, in _convert_status raise IOError(errno.ENOENT, text) FileNotFoundError: [Errno 2] Path not found.

    As I understand, the path it refers to is the remote path. When I debug it, in put_d(), the paths it creates are the following: image As you can see, the remotedir "folder/local_folder/file" is probably the issue. In my remote location, it creates the folder "folder". This of course makes sense as it successfully calls self.mkdir_p(remotedir) in line 599. Do you think this has to do with the location of the self.mkdir_p(remotedir) call? Should there be code that also creates the subfolders of the "local_folder" in the remote location(here the "folder/local_folder" subfolder)?

    Thank you for your time.

    bug 
    opened by Johnmaras 9
  • Add support to use a single channel for all commands. Some servers do…

    Add support to use a single channel for all commands. Some servers do…

    Certain SFTP server do not allow to open a channel for every command. The original pysftp used a single channel for all actions. I've added a flag to allow us to work in a similar way, while by default keeping the new behavior.

    Those servers will simply close the connection on the second command issued and it makes sftpretty not very useful with these old(er) SFTP servers.

    opened by erans 5
  • Typo in argument name - tires/tries

    Typo in argument name - tires/tries

    Hi,

    In sftpretty/init.py:716 there is a typo in the tries argument which causes the call to putfo() to raise an exception

    retry() got an unexpected keyword argument 'tires'

    bug 
    opened by Johnmaras 2
  • Fix potential `_channel` `UnboundLocalError`

    Fix potential `_channel` `UnboundLocalError`

    Sometimes when assigning to _channel, an exception can occur in SFTPClient.from_transport(). Because _channel doesn't exist if that happens, it causes a second exception when trying to call close() in the finally block. Assign to the variable outside the try block initially so that this exception can't happen in the future. Also make sure that _channel is not None before we try to close it to account for it being sometimes None.

    bug 
    opened by RobertCochran 1
  • calling basicConfig at this level for a library that should be embedded changes all dowstream logs

    calling basicConfig at this level for a library that should be embedded changes all dowstream logs

    Calling basicConfig on the top of the file for a library that is embedded into other projects can cause issue. I suggest you drop it completely specifically because there are no change to it other than setting the default level to INFO.

    https://github.com/byteskeptical/sftpretty/blob/ca8e2414ac156276059fcd091cc91e489d856e7d/sftpretty/init.py#L22

    In our case it added ":" to the start of every log line.

    enhancement 
    opened by erans 1
Releases(1.0.5)
OpenNeoMC:an Open-source Tool for Particle Transport Optimization that Combining OpenMC with NEORL

OpenNeoMC:an Open-source Tool for Particle Transport Optimization that Combining OpenMC with NEORL OpenMC is a community-developed Monte Carlo neutron

7 Aug 17, 2022
mitm6 is a pentesting tool that exploits the default configuration of Windows to take over the default DNS server.

mitm6 is a pentesting tool that exploits the default configuration of Windows to take over the default DNS server.

Fox-IT 1.3k Jan 05, 2023
A Python framework for interacting with Solana's Pyth network.

Pyth Network A basic Python framework for reading and decoding data regarding the Pyth network

1 Nov 29, 2021
Process incoming JSON-RPC requests in Python

August 16, 2021: Version 5 has been released. Read about the changes in version 5, or read the full documentation. Version 5 is for Python 3.8+ only.

Exploding Labs 156 Dec 31, 2022
🐛 SSH self spreading worm written in python3 to propagate a botnet.

Mirkat SSH self spreading worm written in python3 to propagate a botnet. Install tutorial. cd ./script && sh setup.sh Support me. ⚠️ If this reposito

Ѵιcнч 58 Nov 01, 2022
Timeouts for popular Python packages

Python Timeouts An unresponsive service can be worse than a down one. It can tie up your entire system if not handled properly. All network requests s

Andrew Kane 11 Nov 22, 2022
A non-custodial oracle and escrow system for the lightning network. Make LN contracts more expressive.

Hodl contracts A non-custodial oracle and escrow system for the lightning network. Make LN contracts more expressive. If you fire it up, be aware: (1)

31 Nov 30, 2022
the objective of this project is to create a Node Js server with a Python client

Socket.io-Server-client Objective The objective of this project is to send data real time ,we use socket.io(server, client) on this project Server Nod

Reda Ennakouri 5 Mar 07, 2022
The AKS cluster provisioner provisions AKS clusters :-)

Overview The AKS cluster provisioner provisions AKS clusters :-) It uses the Azure CLI to configure VNet and subnets before creating the cluster itsel

Gigi Sayfan 1 Nov 10, 2021
A python 3 library which helps in using nmap port scanner.

A python 3 library which helps in using nmap port scanner. This is done by converting each nmap command into a callable python3 method or function. System administrators can now automatic nmap scans

Nmmapper 179 Dec 19, 2022
An API for controlling Wi-Fi connections on Balena devices.

Description An API for controlling Wi-Fi connections on Balena devices. It does not contain an interface, instead it provides API endpoints to send re

8 Dec 25, 2022
Network Dynaimcs Simulation

A Final Year Project in CUHK, Autumn 2021 Network Dynaimcs Simulation Files param.h edit all the variables & settings here simulate.c the main program

Likchun 0 Mar 28, 2022
Extended refactoring capabilities for Python LSP Server using Rope.

pylsp-rope Extended refactoring capabilities for Python LSP Server using Rope. This is a plugin for Python LSP Server, so you also need to have it ins

36 Dec 24, 2022
A library of functions that can be used to manage the download of claims from the LBRY network.

lbrytools A library of functions that can be used to manage the download of claims from the LBRY network. It includes methods to download claims by UR

13 Dec 03, 2022
School Project using Python Sockets and Personal Encryption Method.

Python-Secure-File-Transfer School Project using Python Sockets and Personal Encryption Method. Installation Must have python3 installed on your syste

1 Dec 03, 2021
TunnelProxy 是一个本地隧道代理,可以从fofa爬取免费的socks代理,然后构建代理池,如果一个代理失效,会自动切换

TunnelProxy 是一个本地隧道代理,可以从fofa爬取免费的socks代理,然后构建代理池,如果一个代理失效,会自动切换。 应用场景 渗透测试需要访问某些国内网站(比如edu的),想要隐藏自己,但是国外代理不能访问,也没有稳定的可用代理的时候。 之后,可能我会增加国外代理,实现白嫖科学上网。

urdr-gungnir 45 Nov 17, 2022
Wallc - Calculate the layout on the wall to hang up pictures

wallc Calculate the layout on the wall to hang up pictures. Installation pip install git+https://github.com/trbznk/wallc.git Getting Started Currently

Alex Trbznk 68 Sep 09, 2022
tradingview socket api for fetching real time prices.

tradingView-API tradingview socket api for fetching real time prices. How to run git clone https://github.com/mohamadkhalaj/tradingView-API.git cd tra

MohammadKhalaj 35 Dec 31, 2022
Network monitoring tool

netmeter If you are looking for a tool to monitor your network interfaces, here you are. See netmeter-exporter to export Prometheus metrics. Installat

Saeid Bostandoust 97 Dec 03, 2022
PcapXray - A Network Forensics Tool - To visualize a Packet Capture offline as a Network Diagram

PcapXray - A Network Forensics Tool - To visualize a Packet Capture offline as a Network Diagram including device identification, highlight important communication and file extraction

Srinivas P G 1.4k Dec 28, 2022