Custom SLURM wrapper scripts to make finding job histories and system resource usage more easily accessible

Overview

SLURM Wrappers

Executables

job-history

A simple wrapper for grabbing data for completed and running jobs.

nodes-busy

Developed for the HPC systems at University of Arizona. High memory nodes are differentiated from standard nodes with AvailableFeatures=hi_mem.

nodes-busy.mp4
You might also like...
Projeto job insights - Projeto avaliativo da Trybe do Bloco 32: Introdução à Python

Termos e acordos Ao iniciar este projeto, você concorda com as diretrizes do Código de Ética e Conduta e do Manual da Pessoa Estudante da Trybe. Boas

NUM Alert - A work focus aid created for the Hack the Job hackathon

Contributors: Uladzislau Kaparykha, Amanda Hahn, Nicholas Waller Hackathon Team Name: N.U.M General Purpose: The general purpose of this program is to

Standalone PyQGIS application for executing custom scripts without a QGIS GUI.

PyQGIS Standalone Script Executer Standalone PyQGIS application that is able to run a custom script, in this case Proximity.py without the need of a G

This is a Poetry plugin that will make it possible to build projects using custom TOML files

Poetry Multiproject Plugin This is a Poetry plugin that will make it possible to build projects using custom TOML files. This is especially useful whe

Coinloggr - A learning resource and social platform for the coin collecting community
Coinloggr - A learning resource and social platform for the coin collecting community

Coinloggr A learning resource and social platform for the coin collecting commun

python scripts - mostly automation scripts

python python scripts - mostly automation scripts You can set your environment in various ways bash #!/bin/bash python - locally on remote host #!/bi

A simple bot that will help you in your learning and make it more fun.

hyperskill-SimpleChattyBot-python A simple bot that will help you in your learning and make it more fun. Syntax bot.py Stages Stage #1: Zuhura Bot we

Make after-work Mending More flexible In Python

Mending Make after-work Mending More flexible In Python A Lite Package focuses on making project's after-post mending pythonic and flexible. Certainly

Comments
  • KeyError: GPUAlloc error when running nodes-busy

    KeyError: GPUAlloc error when running nodes-busy

    Howdy,

    Blake J pointed us (UAB Research Computing) to your nice slurm-wrapper repo :-)

    While job-history and system-busy work great on our cluster, nodes-busy is crashing on our cluster with the following:

    ❯ ./bin/nodes-busy
    
    nodes-busy: visualize live system resource usage.
    Trouble seeing the output? Try 'nodes-busy --ascii'
    
    Traceback (most recent call last):
      File "./bin/nodes-busy", line 766, in <module>
        merged = merge(job_data, node_data)
      File "./bin/nodes-busy", line 330, in merge
        nodes_dictionary[node]["JOBS"][job] = {"CPUs":cpus, "GPUs":jobs_dictionary[job]["GPUAlloc"],"EndTime":jobs_dictionary[job]["EndTime"],"Partition":jobs_dictionary[job]["Partition"],"Restarts":jobs_dictionary[job]["Restarts"]}
    KeyError: 'GPUAlloc'
    

    I'm digging through the code, but figured I'd post this for tracking purposes.

    Thanks, Mike

    opened by flakrat 7
  • nodes-busy

    nodes-busy "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 53196: invalid start byte"

    Howdy, noticed the following error today when running nodes-busy on our cluster.

    > nodes-busy
    
    Traceback (most recent call last):
      File "/share/apps/rc/bin/nodes-busy", line 1177, in <module>
        job_data = get_scontrol_job_data()
      File "/share/apps/rc/bin/nodes-busy", line 370, in get_scontrol_job_data
        output = out.decode('utf-8').split("\n")
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 53196: invalid start byte
    

    I have no idea what's in position 53196, but I fixed the issue with the following addition to output.decode to ignore errors:

    diff --git a/bin/nodes-busy b/bin/nodes-busy
    index 6fb1744..aee728f 100755
    --- a/bin/nodes-busy
    +++ b/bin/nodes-busy
    @@ -367,7 +367,7 @@ def get_scontrol_job_data(target_job = None):
             sys.exit(1)
    
         # Split up space-delimited output into a job dictionary
    -    output = out.decode('utf-8').split("\n")
    +    output = out.decode('utf-8', 'ignore').split("\n")
         for job in output:
             details = job.split(' ')
             for i in details:
    
    opened by flakrat 3
  • Feature added user arg to past-jobs

    Feature added user arg to past-jobs

    Added new argument to past-jobs to support selecting jobs by someone other than the user running the script:

    $ past-jobs -d 30 -u flakrat
    
                             Jobs submitted by user flakrat in last 30 days.
    
    JobID    Start       User            JobName         Partition  Account    State       ExitCode
    ------------------------------------------------------------------------------------------------
    14364561 2022-06-02  flakrat         hostname        express    flakrat    CANCELLED        0:0
    14385956 2022-06-05  flakrat         hostname        express    flakrat    COMPLETED        0:0
    14850816 2022-06-12  flakrat         hostname        express    flakrat    COMPLETED        0:0
    
    opened by flakrat 0
Releases(v0.2.0)
  • v0.2.0(May 11, 2022)

  • v0.1.0(Dec 9, 2021)

    The scripts in this repository have some bits and pieces that are specific to the HPC setup at University of Arizona. These parts are being cleaned up.

    Source code(tar.gz)
    Source code(zip)
Owner
Sara
HPC Consultant at University of Arizona, UITS
Sara
Aero is an open source airplane intelligence tool. Aero supports more than 13,000 airlines and 250 countries. Any flight worldwide at your fingertips.

Aero Aero supports more than 13,000 airlines and 250 countries. Any flight worldwide at your fingertips. Features Main : Flight lookup Aircraft lookup

Vickey 비키 4 Oct 27, 2021
Replit theme sync; Github theme sync but in Replit.

This is a Replit theme sync, basically meaning that it keeps track of the current time (which may need to be edited later on), and if the time passes morning, afternoon, etc, the theme switches. The

Glitch 8 Jun 25, 2022
This an Anki add on that automatically converts Notion notes into Anki flash cards. Currently in development!

NotionFlash This is an Anki add on in development that will allow automatically convert your Notion study notes into Anki flash cards. The Anki deck c

Neeraj Patel 10 Oct 07, 2022
An addin for Autodesk Fusion 360 that lets you view your design in a Looking Glass Portrait 3D display

An addin for Autodesk Fusion 360 that lets you view your design in a Looking Glass Portrait 3D display

Brian Peiris 12 Nov 02, 2022
Mpis-ex7 - Implementation of tasks 1, 2, 3 for Metody Probabilistyczne i Statystyka Lista 7

Implementations of task 1, 2 and 3 from here Author: Maciej Bazela Index: 261743 Each task was implemented in Python 3. I've used Cython to speed up e

Maciej Bazela 1 Feb 27, 2022
A collection of Workflows samples for various use cases

Workflows Samples Workflows allow you to orchestrate and automate Google Cloud and HTTP-based API services with serverless workflows.

Google Cloud Platform 76 Jan 07, 2023
PyPI package for scaffolding out code for decision tree models that can learn to find relationships between the attributes of an object.

Decision Tree Writer This package allows you to train a binary classification decision tree on a list of labeled dictionaries or class instances, and

2 Apr 23, 2022
PyScaffold is a project generator for bootstrapping high quality Python packages

PyScaffold is a project generator for bootstrapping high quality Python packages, ready to be shared on PyPI and installable via pip. It is easy to use and encourages the adoption of the best tools a

PyScaffold 1.7k Jan 03, 2023
An example of Connecting a MySQL Database with Python Code

An example of Connecting a MySQL Database with Python Code And How to install Table of contents General info Technologies Setup General info In this p

Mohammad Hosseinzadeh 1 Nov 23, 2021
LibreMind is a free meditation app made in under 24 hours. It has various meditation, breathwork, and visualization exercises.

libreMind Meditation exercises What is it? LibreMind is a free meditation app made in under 24 hours. It has various meditation, breathwork, and visua

1 May 24, 2022
Path of Exile Vendor Recipe Tracker (Chaos/Regal orb)

Path of Exile Vendor Trade Tracker Are you tired of manually keeping track of collected and missing items for farming Chaos or Regal Orbs in PoE? Me t

1 Nov 09, 2021
A simple desktop application to scan and export Genshin Impact Artifacts.

「天目」 -- Amenoma 简体中文 | English 「天目流的诀窍就是滴水穿石的耐心和全力以赴的意志」 扫描背包中的圣遗物,并导出至 json 格式。之后可导入圣遗物分析工具( 莫娜占卜铺 、 MingyuLab 、 Genshin Optimizer 进行计算与规划等。 已支持 原神2.

夏至 475 Dec 30, 2022
An implementation to rank your favourite songs from World of Walker

World-Of-Walker-Elo An implementation to rank your favourite songs from Alan Walker's 2021 album World of Walker. Uses the Elo rating system, which is

1 Nov 26, 2021
Fisherman is a free open source fishing bot written in python.

Fisherman is a free open source fishing bot written in python.

Pure | Cody 33 Jan 29, 2022
Simple control of Thorlabs Elliptec devices from Python.

Elliptec Simple control of Thorlabs Elliptec devices. No docs yet » Get started · Report a bug · Request a feature About The Project ThorLabs Elliptec

David Roesel 8 Sep 22, 2022
It converts ING BANK account historic into a csv file you can import in HomeBank application.

ing2homebank It converts your ING Bank account historic csv file into another csv file you can import in HomeBank application

1 Feb 14, 2022
Nfog - Scriptable Database-Driven NFO Generator for Movies and TV

nfog Scriptable Database-Driven NFO Generator for Movies and TV. Installation pi

6 Oct 08, 2022
kurwa deska ADB

kurwa-deska-ADB kurwa-deska Запуск Linux -- python3 kurwa_deska.py Termux -- python3 kurwa_deska.py Встановлення cd kurwa_deska ADB і зразу запуск pyt

1 Jan 21, 2022
CoreSE - basic of social Engineering tool

Core Social Engineering basic of social Engineering tool. just for fun :) About First of all, I must say that I wrote such a project because of my int

Hamed Mohammadvand 7 Jun 10, 2022
basic tool for NFT. let's spam, this is the easiest way to generate a hell lotta image

NFT generator this is the easiest way to generate a hell lotta image buckle up and follow me! how to first have your image in .png (transparent backgr

34 Nov 18, 2022