Avoid PyCharm Overloading
=========================
This speeds up PyCharm's package index processes and avoids CPU & memory
overloading.
On my machine (4c/32GB) with 5 conda environments and 10 projects, this reduces
PyCharm's package scanning from 15 minutes to 45 seconds.
Why?
----
When PyCharm has multiple conda environments, it will sometimes launch concurrent
package scanning across those environments. To do so, it uses the `conda` cli,
and a tool called conda_packaging_tool.py, available from the JetBrains
intellij-community github repository (see below).
Unfortunately, these tools do not behave well when launched in parallel, as
they tend to overload CPU & memory in this case.
How?
----
To avoid stalling these processes, we modify their sources as follows:
1. modify the conda cli
$ conda activate base
$ pip install runfast
$ nano `which conda`
Modify the main section:
if __name__ == '__main__':
from conda.cli import main
from runfast import cached
cached(main)
2. modify the conda_packaging_tool.py
if __name__ == '__main__':
from runfast import cached
cached(main)
How does this work?
-------------------
Two simple steps:
1. When called in parallel, only one process is allowed to proceed.
2. `runfast.cached` caches the output of these tools (stdout, stderr) for 1 minute,
given the same command line parameters
That is, when PyCharm launches 5 package scanning (`conda list`) commands, only
one of them will run immediately. If some of the scans are for the same environment,
only one of them will actually run, while the others simply return the same output.
How to clear the cache or avoid caching at all?
$ export RUNFAST_NOCACHE=1
Testing
-------
a) conda packaging tool
# first time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m13.271s
user 0m9.631s
sys 0m1.936s
# second time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m3.159s
user 0m0.139s
sys 0m1.298s
b) conda cli
# first time
$ time conda list -p /path/to/env
real 0m7.985s
user 0m7.883s
sys 0m0.087s
# second time
real 0m0.152s
user 0m0.116s
sys 0m0.032s
References
----------
* https://github.com/JetBrains/intellij-community/blob/master/python/helpers/conda_packaging_tool.py
* https://github.com/conda/conda/blob/33a142c16530fcdada6c377486f1c1a385738a96/conda/core/index.py#L53
This speeds up PyCharm's package index processes and avoids CPU & memory overloading
Overview
Simple, high-school-leveled sequence library written in Python / 간단한 고등학교 수준 수열 라이브러리 (Python)
Simple, high-school-leveled sequence library written in Python
Push Prometheus metrics to VictoriaMetrics or other exporters
Push metrics from your periodic long-running jobs to existing Prometheus/VictoriaMetrics monitoring system.
Program Input Data Mahasiswa Oop
PROGRAM INPUT NILAI MAHASISWA MENGGUNAKAN OOP PENGERTIAN OOP object-oriented-programing/OOP adalah paradigma pemrograman berdasarkan konsep "objek", y
Semantic Data Management - Property Graphs 📈
SDM - Lab 1 @ UPC 👨🏻💻 Table of contents Introduction Property Graph Dataset 1. Introduction This repo is all about what we have done in SDM lab 1
The Great Autoencoder Bake Off
The Great Autoencoder Bake Off The companion repository to a post on my blog. It contains all you need to reproduce the results. Features Currently fe
Release for Improved Denoising Diffusion Probabilistic Models
improved-diffusion This is the codebase for Improved Denoising Diffusion Probabilistic Models. Usage This section of the README walks through how to t
Wordle Solver
Wordle Solver Installation Install the following onto your computer: Python 3.10.x Download Page Run pip install -r requirements.txt Instructions To r
A pure-Python codified rant aspiring to a world where numbers and types can work together.
Copyright and other protections apply. Please see the accompanying LICENSE file for rights and restrictions governing use of this software. All rights
Custom python interface to xstan (a modified (cmd)stan)
Custom python interface to xstan (a modified (cmd)stan) Use at your own risk, currently everything is very brittle and will probably be changed in the
Blender addon - Breakdown in object mode
Breakdowner Breakdown in object mode Download latest Demo Youtube Description Same breakdown shortcut as in armature mode in object mode Currently onl
An osu! cheat made in c++ rewritten in python and currently undetected.
megumi-python An osu! cheat made in c++ rewritten in python and currently undetected. Installation Guide Download python 3.9 from https://python.org C
PBN Obfuscator: A overpowered obfuscator for python, which will help you protect your source code
PBN Obfuscator PBN Obfuscator is a overpowered obfuscator for python, which will
Just another sentiment wrapper.
sentimany Just a simple sentiment tool. It just grabs a set of pre-made sentiment models that you can quickly use to attach sentiment scores to text.
This is an implementation of PEP 557, Data Classes.
This is an implementation of PEP 557, Data Classes. It is a backport for Python 3.6. Because dataclasses will be included in Python 3.7, any discussio
Your missing PO formatter and linter
pofmt Your missing PO formatter and linter Features Wrap msgid and msgstr with a constant max width. Can act as a pre-commit hook. Display lint errors
A Python version of Canvacord
A copy of canvacord made in python! Table of contents Installation Examples Creating Images Links Downloads Installation Run any of these commands in
Basic Hspice runner with Python
HSpicePy Bilgisayarınıza PATH değişkenlerine eklediğiniz HSPICE programını python ile çalıştırmanızı sağlayan basit bir araç. A simple tool that allow
Excel cell checker with python
excel-cell-checker Description This tool checks a given .xlsx file has the struc
This is a Fava extension to display a grouped portfolio view in Fava for a set of Beancount accounts.
Fava Portfolio Summary This is a Fava extension to display a grouped portfolio view in Fava for a set of Beancount accounts. It can also calculate MWR
Cool Bioinformatics Scripts
Cool Bioinformatics Scripts qqplot You can use this script in two ways read tons of millions of P values from stdin # python zcat pval.txt.gz | qqplo