A lobby boy will create a VPS server when you need one, and destroy it after using it.

Overview

Lobbyboy

What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipates the client's needs before the needs are needed. A lobby boy is, above all, discreet to a fault.

--The Grand Budapest Hotel

Test codecov PyPI version Python version

This project is still under testing, it worked but may have bugs.

What is lobbyboy?

Well, lobbyboy is a ssh server. Yes, like sshd. But instead of spawn a new shell on the server like sshd, when you ssh to lobbyboy, lobbyboy will create a new server(VPS) from available providers(meaning to say, DigitalOcean, AWS, GCP, Vultr, etc), then redirect you to the newly created servers. Of course, if lobbyboy finds any servers available already, he will just ask if you want to enter the existing server, or still want to create a new one.

Key Features

  • talks in SSH2 protocol, no need to install any software of configs for client-side, just ssh to lobbyboy!
  • extendable provider: just implement 3 methods, then lobbyboy can work with any provider!
  • destroy the server when you no longer needed.
  • manage ssh keys for you

Installation

Install libkrb5-dev first, this is a dependency for gssapi support.

apt install libkrb5-dev

Install via pip:

pip install lobbyboy

Run server

First, generate a config file:

lobbyboy-config-example > config.toml
# Edit your config before running!

Run the server with:

lobbyboy-server -c config.toml

You can ssh to Lobbyboy now, if you keep the default user Gustave in default config. You can ssh to Lobbyboy via:

ssh [email protected] -p 12200
# Enter the default password "Fiennes"(without quotes)
Welcome to Lobbyboy 0.2.2!
There are 1 available servers:
  0 - Create a new server...
  1 - Enter vagrant lobbyboy-41 127.0.0.1 (0 active sessions)
Please input your choice (number):

You may want to change the password in config.toml or use a public key for authentication. The latter is recommended in a production environment.

Generate a key pair for authentication

Generate a key pair:

ssh-keygen -f lobbyboy_key

Add the content of lobbyboy_key.pub to the end of authorized_keys under [user.Gustave] table. Now you can ssh to the lobbyboy server via:

ssh [email protected] -i lobbyboy_key

Deployment

Lobbyboy is supposed to be a server daemon, so you can manage it by systemd/supervisord or put it into a docker.

Systemd Example

[Unit]
Description=Lobbyboy Server

[Service]
User=me
Group=me
ExecStart=/path/to/lobbyboy-server -c /path/to/lobbyboy/config.toml
Restart=on-failure
WorkingDirectory=/path/to/lobbyboy/

[Install]
WantedBy=multi-user.target

Run in Docker

# Generate a config file
docker run --rm ghcr.io/lobbyboy-ssh/lobbyboy lobbyboy-config-example > lobbyboy_config.toml
# Run the docker container
docker run -v `pwd`/lobbyboy_config.toml:/app/config.toml -p "12200:12200" -d ghcr.io/lobbyboy-ssh/lobbyboy

The lobbyboy server should be active on 12200 port and you can connect to it with

ssh [email protected] -p 12200

The default password for user Gustave is Fiennes. Please change it when you deployed it into production, and consider use ssh key to auth instead of password.

Providers

// TBD

Builtin Providers

Lobbyboy current support multiple Providers:

  • Vagrant (Need vagrant and virtualbox to be installed)
  • Footlosse, in another words, containers (Need footloose and docker to be installed)
  • DigitalOcean
  • Linode

Different Providers support different configs, please see the example config for more detail.

Vagrant Provider

Vagrant Provider won't cost you any money, vagrant is a software runs on your computer along with virtual machine providers, vagrant can provision and control your VM.

This provider can help you to create a new Vagrant instance when you login to Lobbyboy, and destroy the server when you no longer use it.

Supported Features:

  • Create new Vagrant instances
  • You can configure your VM via vagrantfile config (see the config example).

Footloose Provider

footloose can make your docker containers(or Firecracker with ignite) act like virtual machine, so you can ssh to it.

Supported feature:

  • Configurable base image
  • Create a docker container and redirect you in

DigitalOcean Provider

This Provider will create Droplet from DigitalOcean.

Supported Features:

  • Create a new ssh key every time create a droplet.
  • Ask user to input region/droplet size/image when creating.
  • User can save favorite Droplet region/size/image in configs to quick create.
  • Destroy droplet when it is not in use.

Linode Provider

This Provider will create Node from Linode.

Supported Features:

  • Create a new ssh key every time create a droplet.
  • Ask user to input region/node type/image when creating.
  • User can save favorite node region/type/image in configs to quick create.
  • Destroy node when it is not in use.

Please see configs to check available options.

Ignite(Firecracker) Provider

Supported Features:

  • Create a new Firecracker virtual machine
  • Destroy node when it is not in use.

Write Your Own Providers

Providers are VPS vendors, by writing new providers, lobbyboy can work with any VPS vendors.

To make a new Provider work, you need to extend base class `lobbyboy.provider.BaseProvider``, implement 2 methods:

    def create_server(self, channel: Channel) -> LBServerMeta:
        """
        Args:
            channel: paramiko channel

        Returns:
            LBServerMeta: server meta info
        """
        ...


    def destroy_server(self, meta: LBServerMeta, channel: Channel = None) -> bool:
        """
        Args:
            meta: LBServerMeta, we use this to locate one server then destroy it.
            channel: Note that the channel can be None.
                     If called from server_killer, channel will be None.
                     if called when user logout from server, channel is active.

        Returns:
            bool: True if destroy successfully, False if not.
        """
        ...

Then add your Provider to your config file.

Those 3 configs are obligatory, as lobbyboy has to know when should he destroy your spare servers. You can add more configs, and read them from self.provider_config from code, just remember to add docs about it :)

[provider.<your provider name>]
load_module = "lobbyboy.contrib.provider.<your provider module name>::<Provider Class>"
min_life_to_live = "1h"
bill_time_unit = "1h"

Publish Your Own Providers

// TBD

FAQ

Q: Can I use lobbyboy as a proxy, like adding it to my ProxyCommand in ssh config?

A: No. Lobbyboy works like a reverse proxy, meaning to say, for ssh client, it just like a ssh server(sshd maybe), ssh client get a shell from lobbyboy, and doesn't know if it is local shell or it is a nested shell which runs another ssh. (but you know it, right? :D )

I Want to Know More!

Comments
  • LBConfigProvider is inflexible

    LBConfigProvider is inflexible

    My original idea for the provider is that:

    A provider can be defined in a single file, a provider's exceptions, implementations, and config definitions, all can be defined in a single file.

    So that it would be easy to maintain. And also, a provider can be implemented outside lobbyboy's codebase. (let's say, one can upload his provider to pypi, and other users can install them via pip install lobbyboy-aws-ec2-provider, then lobbyboy can load it.

    And one can add custom configs to his provider.

    The current problem is, we have to update the LBConfigProvider if one wants to add more fields to his provider config. It is not possible to do so without updating lobbyboy's source code.

    opened by laixintao 7
  • UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    CRI [20211208-07:42:34.995] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.                                                                                                                                                                                                                          Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 247, in run
        self.user_using(server, proxy_subprocess)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 188, in user_using
        send_to_channel(self.channel, os.read(master_fd, 10240).decode(), suffix="")
    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data
    DEB [20211208-07:42:35.003] thr=140672300570368 paramiko.transport:1819: EOF in transport thread
    
    opened by messense 5
  • Initialization Problems.

    Initialization Problems.

    I am really interested in your idea of lobbyboy. But I know nothing about python.

    So here is my problems during initialization, can you help me to figure it out?

    1. I configure my own data_dir but service start with an error. obbyboy.utils: Error when reading available_servers.json, [Errno 2] No such file or directory: '~/Software/lobbyboy/data/available_servers.json' Even I create the available_servers.json file, it does not work. Until I write {} to the file. But I'm not sure if this is ok for the lobbyboy service
    2. When I ssh with ssh [email protected] -p 12200, the logs shows an error below:
    ERR [20211119-17:40:41.664] thr=123145413386240 paramiko.transport: Unknown exception: too many values to unpack (expected 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: Traceback (most recent call last):
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/transport.py", line 2109, in run
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     handler(self.auth_handler, m)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/auth_handler.py", line 525, in _parse_userauth_request
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     result = self.transport.server_object.check_auth_publickey(
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/lobbyboy/server.py", line 103, in check_auth_publickey
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     _, _key_pub = line.split(" ", 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: ValueError: too many values to unpack (expected 2)
    

    I did some research but it's really difficult for me to understand python.

    So if you can give me a hand, that's would be nice.

    Thank you.

    opened by kimichen13 4
  • Use pre-commit as the linter

    Use pre-commit as the linter

    Close #48

    Install the commit hooks in dev:

    poetry run pre-commit install
    

    Run in CI: I recommend using pre-commit ci, it can upgrade the hooks and commit linter changes automatically. It is also blazing fast, e2e run time is < 10s

    BTW, the code is changed by the linter so this is better to merge after #55, otherwise there will be lots of conflicts

    opened by frostming 3
  • Recommended use pre-commit to make code style consistency

    Recommended use pre-commit to make code style consistency

    @laixintao @messense @frostming

    Recommended use pre-commit to make code style consistency, especially in the case of multi-person cooperation, what do you think?

    opened by luxiaba 3
  • Bump paramiko from 2.8.1 to 2.10.1

    Bump paramiko from 2.8.1 to 2.10.1

    Bumps paramiko from 2.8.1 to 2.10.1.

    Commits
    • 286bd9f Cut 2.10.1
    • 4c491e2 Fix CVE re: PKey.write_private_key chmod race
    • aa3cc6f Cut 2.10.0
    • e50e19f Fix up changelog entry with real links
    • 02ad67e Helps to actually leverage your mocked system calls
    • 29d7bf4 Clearly our agent stuff is not fully tested yet...
    • 5fcb8da OpenSSH docs state %C should also work in IdentityFile and Match exec
    • 1bf3dce Changelog enhancement
    • f6342fc Prettify, add %C as acceptable controlpath token, mock gethostname
    • 3f3451f Add to changelog
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • OSError: [Errno 9] Bad file descriptor

    OSError: [Errno 9] Bad file descriptor

    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.utils:127: user choose 1 for option There are 1 available servers:
    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.socket_handle:79: user choose server input=1.
    DEB [20211208-07:44:51.057] thr=140672309044992 lobbyboy.contrib.provider.ignite:86: get ssh to server command for ignite: ['cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739']
    INF [20211208-07:44:51.057] thr=140672309044992 lobbyboy.socket_handle:103: ssh to server 2021-12-08-0739 127.0.0.1: cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739
    INF [20211208-07:44:51.062] thr=140672309044992 lobbyboy.socket_handle:176: proxy subprocess created, pid=71892
    CRI [20211208-07:44:51.987] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.
    Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 252, in run
        self.cleanup(t, meta=lb_server, check_destroy=True)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 194, in cleanup
        self.remove_server_session(t, meta.server_name)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in remove_server_session
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in <lambda>
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/.cache/pypoetry/virtualenvs/lobbyboy-P-fHd6h2-py3.8/lib/python3.8/site-packages/paramiko/transport.py", line 1787, in getpeername
        return gp()
    OSError: [Errno 9] Bad file descriptor
    
    opened by messense 1
  • tests for need_destroy

    tests for need_destroy

    somehow some of my instance didn't get destroyed, I suspect there are some bugs in https://github.com/lobbyboy-ssh/lobbyboy/blob/41bba07e856e6ce31f7164232723e7440a469f18/lobbyboy/server_killer.py#L42

    but I didn't find any, let me add test cases later.

    opened by laixintao 3
  • Security check when lobbyboy server starts.

    Security check when lobbyboy server starts.

    Need to check:

    • user started lobbyboy with default username/password?
    • with the ssh host key https://github.com/lobbyboy-ssh/lobbyboy/blob/main/dev_datadir/ssh_host_rsa_key in git repo (middle man attack)

    If yes, then need to print warning information to stderr.

    related discussion: https://github.com/lobbyboy-ssh/lobbyboy/pull/32

    opened by laixintao 0
  • vultr provider support

    vultr provider support

    i want to add vultr provider and i found python-vultr, but seems that it hasn't been updated for a long time, and there are fatal bugs that can't be used normally, i write a new one pyvultr, can we use this to support vultr provider?

    opened by luxiaba 2
Releases(v0.4.0)
  • v0.4.0(Dec 11, 2021)

    What's Changed

    • fix typo of exmaple config file name. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/25
    • Fix typos and add codespell check to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • Fix the path of example config.toml by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/28
    • Support docker container (based on footloose) by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/30
    • ci: fix unittest running branch master-> main by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/36
    • Add instructions to run in docker by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/32
    • Add ignite provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/37
    • Revamp provider config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/41
    • Fix send_to_channel by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/44
    • Add support for enable/disable provider in config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/45
    • Add a test case for load_config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/49
    • Add a test case for load_providers by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/50
    • Add coverage report to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/51
    • update readme: new architecture picture, update new providers. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/46
    • Define a default provider if there is only one provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/47

    New Contributors

    • @messense made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • @frostming made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/28

    Full Changelog: https://github.com/lobbyboy-ssh/lobbyboy/compare/v0.3.0...v0.4.0

    Source code(tar.gz)
    Source code(zip)
framework providing automatic constructions of vulnerable infrastructures

中文 | English 1 Introduction Metarget = meta- + target, a framework providing automatic constructions of vulnerable infrastructures, used to deploy sim

rambolized 685 Dec 28, 2022
Oracle Cloud Infrastructure Object Storage fsspec implementation

Oracle Cloud Infrastructure Object Storage fsspec implementation The Oracle Cloud Infrastructure Object Storage service is an internet-scale, high-per

Oracle 9 Dec 18, 2022
Jenkins-AWS-CICD - Implement Jenkins CI/CD with AWS CodeBuild and AWS CodeDeploy, build a python flask web application.

Jenkins-AWS-CICD - Implement Jenkins CI/CD with AWS CodeBuild and AWS CodeDeploy, build a python flask web application.

Ning 1 Jan 01, 2022
IP address management (IPAM) and data center infrastructure management (DCIM) tool.

NetBox is an IP address management (IPAM) and data center infrastructure management (DCIM) tool. Initially conceived by the network engineering team a

NetBox Community 11.8k Jan 07, 2023
Ralph is the CMDB / Asset Management system for data center and back office hardware.

Ralph Ralph is full-featured Asset Management, DCIM and CMDB system for data centers and back offices. Features: keep track of assets purchases and th

Allegro Tech 1.9k Jan 01, 2023
A colony of interacting processes

NColony Infrastructure for running "colonies" of processes. Hacking $ tox Should DTRT -- if it passes, it means unit tests are passing, and 100% cover

23 Apr 04, 2022
🐳 Docker templates for various languages.

Docker Deployment Templates One Stop repository for Docker Compose and Docker Templates for Deployment. Features Python (FastAPI, Flask) Screenshots D

CodeChef-VIT 6 Aug 28, 2022
A tool to clone efficiently all the repos in an organization

cloner A tool to clone efficiently all the repos in an organization Installation MacOS (not yet tested) python3 -m venv .venv pip3 install virtualenv

Ramon 6 Apr 15, 2022
Create pinned requirements.txt inside a Docker image using pip-tools

Pin your Python dependencies! pin-requirements.py is a script that lets you pin your Python dependencies inside a Docker container. Pinning your depen

4 Aug 18, 2022
Azure plugins for Feast (FEAture STore)

Feast on Azure This project provides resources to enable running a feast feature store on Azure. Feast Azure Provider The Feast Azure provider acts li

Microsoft Azure 70 Dec 31, 2022
Run your clouds in RAID.

UniKlaud Run your clouds in RAID Table of Contents About The Project Built With Getting Started Installation Usage Roadmap Contributing License Contac

3 Jan 16, 2022
Ansible for DevOps examples.

Ansible for DevOps Examples This repository contains Ansible examples developed to support different sections of Ansible for DevOps, a book on Ansible

Jeff Geerling 6.6k Jan 08, 2023
This repository contains useful docker-swarm-tools.

docker-swarm-tools This repository contains useful docker-swarm-tools. swarm-guardian This Docker image is intended to be used in a multihost docker e

NeuroForge GmbH & Co. KG 4 Jan 12, 2022
Hubble - Network, Service & Security Observability for Kubernetes using eBPF

Network, Service & Security Observability for Kubernetes What is Hubble? Getting Started Features Service Dependency Graph Metrics & Monitoring Flow V

Cilium 2.4k Jan 04, 2023
This projects provides the documentation and the automation(code) for the Oracle EMEA WLA COA Demo UseCase.

COA DevOps Training UseCase This projects provides the documentation and the automation(code) for the Oracle EMEA WLA COA Demo UseCase. Demo environme

Cosmin Tudor 1 Jan 28, 2022
Quick & dirty controller to schedule Kubernetes Jobs later (once)

K8s Jobber Operator Quickly implemented Kubernetes controller to enable scheduling of Jobs at a later time. Usage: To schedule a Job later, Set .spec.

Jukka Väisänen 2 Feb 11, 2022
A cron monitoring tool written in Python & Django

Healthchecks Healthchecks is a cron job monitoring service. It listens for HTTP requests and email messages ("pings") from your cron jobs and schedule

Healthchecks 5.8k Jan 02, 2023
Utilitaire de contrôle de Kubernetes

Utilitaire de contrôle de Kubernetes ** What is this ??? ** Every time we use a word in English our manager tells us to use the French translation of

Théophane Vié 9 Dec 03, 2022
MLops tools review for execution on multiple cluster types: slurm, kubernetes, dask...

MLops tools review focused on execution using multiple cluster types: slurm, kubernetes, dask...

4 Nov 30, 2022
🎡 Build Python wheels for all the platforms on CI with minimal configuration.

cibuildwheel Documentation Python wheels are great. Building them across Mac, Linux, Windows, on multiple versions of Python, is not. cibuildwheel is

Python Packaging Authority 1.3k Jan 02, 2023