Avoid PyCharm Overloading
=========================
This speeds up PyCharm's package index processes and avoids CPU & memory
overloading.
On my machine (4c/32GB) with 5 conda environments and 10 projects, this reduces
PyCharm's package scanning from 15 minutes to 45 seconds.
Why?
----
When PyCharm has multiple conda environments, it will sometimes launch concurrent
package scanning across those environments. To do so, it uses the `conda` cli,
and a tool called conda_packaging_tool.py, available from the JetBrains
intellij-community github repository (see below).
Unfortunately, these tools do not behave well when launched in parallel, as
they tend to overload CPU & memory in this case.
How?
----
To avoid stalling these processes, we modify their sources as follows:
1. modify the conda cli
$ conda activate base
$ pip install runfast
$ nano `which conda`
Modify the main section:
if __name__ == '__main__':
from conda.cli import main
from runfast import cached
cached(main)
2. modify the conda_packaging_tool.py
if __name__ == '__main__':
from runfast import cached
cached(main)
How does this work?
-------------------
Two simple steps:
1. When called in parallel, only one process is allowed to proceed.
2. `runfast.cached` caches the output of these tools (stdout, stderr) for 1 minute,
given the same command line parameters
That is, when PyCharm launches 5 package scanning (`conda list`) commands, only
one of them will run immediately. If some of the scans are for the same environment,
only one of them will actually run, while the others simply return the same output.
How to clear the cache or avoid caching at all?
$ export RUNFAST_NOCACHE=1
Testing
-------
a) conda packaging tool
# first time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m13.271s
user 0m9.631s
sys 0m1.936s
# second time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m3.159s
user 0m0.139s
sys 0m1.298s
b) conda cli
# first time
$ time conda list -p /path/to/env
real 0m7.985s
user 0m7.883s
sys 0m0.087s
# second time
real 0m0.152s
user 0m0.116s
sys 0m0.032s
References
----------
* https://github.com/JetBrains/intellij-community/blob/master/python/helpers/conda_packaging_tool.py
* https://github.com/conda/conda/blob/33a142c16530fcdada6c377486f1c1a385738a96/conda/core/index.py#L53
This speeds up PyCharm's package index processes and avoids CPU & memory overloading
Overview
A Python package to request and process seismic waveform data from Hi-net.
HinetPy is a Python package to simplify tedious data request, download and format conversion tasks related to NIED Hi-net. NIED Hi-net | Source Code |
A compilation of useful scripts to automate common tasks
Scripts-To-Automate-This A compilation of useful scripts for common tasks Name What it does Type Add file extensions Adds ".png" to a list of file nam
Simple module with some functions such as generate password (get_random_string)
Simple module with some functions such as generate password (get_random_string), fix unicode strings, size converter, dynamic console, read/write speed checker, etc.
Sentiment Based Product Recommendation System
Sentiment Based Product Recommendation System The e-commerce business is quite p
Programmatic interface to Synapse services for Python
A Python client for Sage Bionetworks' Synapse, a collaborative, open-source research platform that allows teams to share data, track analyses, and collaborate
Structured, dependable legos for starknet development.
Structured, dependable legos for starknet development.
This is a far more in-depth and advanced version of "Write user interface to a file API Sample"
Fusion360-Write-UserInterface This is a far more in-depth and advanced version of "Write user interface to a file API Sample" from https://help.autode
Reverse the infix string. Note that while reversing the string you must interchange left and right parentheses
Reverse the infix string. Note that while reversing the string you must interchange left and right parentheses. Obtain the postfix expression of the infix expression Step 1.Reverse the postfix expres
Runtime fault injection platform by Daniele Rizzieri (2021)
GDBitflip [v1.04] Runtime fault injection platform by Daniele Rizzieri (2021) This platform executes N times a binary and during each execution it inj
Block when attacker want to bypass the limit of request
Block when attacker want to bypass the limit of request
Shai-Hulud - A qtile configuration for the (spice) masses
Shai-Hulud - A qtile configuration for the (spice) masses Installation Notes These dotfiles are set up to use GNU stow for installation. To install, f
Enhanced version of blender's bvh add-on with more settings supported. The bvh's rest pose should have the same handedness as the armature while could use a different up/forward definiton.
Enhanced bvh add-on (importer/exporter) for blender Enhanced bvh add-on (importer/exporter) for blender Enhanced bvh importer Enhanced bvh exporter Ho
原神抽卡记录导出
原神抽卡记录导出 抽卡记录分析工具 from @笑沐泽 抽卡记录导出工具js版,含油猴脚本可在浏览器导出 注意:我的是python版,带饼图的是隔壁electron版,功能类似 Wik
We want to check several batch of web URLs (1~100 K) and find the phishing website/URL among them.
We want to check several batch of web URLs (1~100 K) and find the phishing website/URL among them. This module is designed to do the URL/web attestation by using the API from NUS-Phishperida-Project.
Simple dependency injection framework for Python
A simple, strictly typed dependency injection library.
An esoteric programming language that supports concurrency, regex, and web requests.
The Hofstadter Esoteric Programming Language Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's La
On this repo, you'll find every codes I made during my NSI classes (informatical courses)
👨💻 👩💻 school-codes On this repo, you'll find every codes I made during my NSI classes (informatical courses) French for now since this repo is d
Tuple-sum-filter - Library to play with filtering numeric sequences by sums of their pairs, triplets, etc. With a bonus CLI demo
Tuple Sum Filter A library to play with filtering numeric sequences by sums of t
Py-Parser est un parser de code python en python encore en plien dévlopement.
PY - PARSER Py-Parser est un parser de code python en python encore en plien dévlopement. Une fois achevé, il servira a de nombreux projets comme glad
A plugin for managing mod installers in Mod Organizer 2
Reinstaller v1.0.* Introduction Reinstaller allows you to conveninetly backup mod installers to re-run later, without risk of them cluttering up your