Avoid PyCharm Overloading
=========================
This speeds up PyCharm's package index processes and avoids CPU & memory
overloading.
On my machine (4c/32GB) with 5 conda environments and 10 projects, this reduces
PyCharm's package scanning from 15 minutes to 45 seconds.
Why?
----
When PyCharm has multiple conda environments, it will sometimes launch concurrent
package scanning across those environments. To do so, it uses the `conda` cli,
and a tool called conda_packaging_tool.py, available from the JetBrains
intellij-community github repository (see below).
Unfortunately, these tools do not behave well when launched in parallel, as
they tend to overload CPU & memory in this case.
How?
----
To avoid stalling these processes, we modify their sources as follows:
1. modify the conda cli
$ conda activate base
$ pip install runfast
$ nano `which conda`
Modify the main section:
if __name__ == '__main__':
from conda.cli import main
from runfast import cached
cached(main)
2. modify the conda_packaging_tool.py
if __name__ == '__main__':
from runfast import cached
cached(main)
How does this work?
-------------------
Two simple steps:
1. When called in parallel, only one process is allowed to proceed.
2. `runfast.cached` caches the output of these tools (stdout, stderr) for 1 minute,
given the same command line parameters
That is, when PyCharm launches 5 package scanning (`conda list`) commands, only
one of them will run immediately. If some of the scans are for the same environment,
only one of them will actually run, while the others simply return the same output.
How to clear the cache or avoid caching at all?
$ export RUNFAST_NOCACHE=1
Testing
-------
a) conda packaging tool
# first time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m13.271s
user 0m9.631s
sys 0m1.936s
# second time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m3.159s
user 0m0.139s
sys 0m1.298s
b) conda cli
# first time
$ time conda list -p /path/to/env
real 0m7.985s
user 0m7.883s
sys 0m0.087s
# second time
real 0m0.152s
user 0m0.116s
sys 0m0.032s
References
----------
* https://github.com/JetBrains/intellij-community/blob/master/python/helpers/conda_packaging_tool.py
* https://github.com/conda/conda/blob/33a142c16530fcdada6c377486f1c1a385738a96/conda/core/index.py#L53
This speeds up PyCharm's package index processes and avoids CPU & memory overloading
Overview
A visidata plugin for parsing f5 ltm/gtm/audit logs
F5 Log Visidata Plugin This plugin supports the default log format for: /var/log/ltm* /var/log/gtm* /var/log/apm* /var/log/audit* It extracts common l
This project intends to take the user's CEP (brazilian adress code) and return the local in which the CEP is placed.
This project aims to simply return the CEP's (the brazilian resident adress code) User of the application. The project uses a request and passes on to
pybicyclewheel calulates the required spoke length for bicycle wheels
pybicyclewheel pybicyclewheel calulates the required spoke length for bicycle wheels. (under construcion) - homepage further readings wikipedia bicyc
RFDesign - Protein hallucination and inpainting with RoseTTAFold
RFDesign: Protein hallucination and inpainting with RoseTTAFold Jue Wang (juewan
一个Graia-Saya的插件仓库
一个Graia-Saya的插件仓库 这是一个存储基于 Graia-Saya 的插件的仓库 如果您有这类项目
A very simple boarding app with DRF
CRUD project with DRF A very simple boarding app with DRF. About The Project 유저 정보를 갖고 게시판을 다루는 프로젝트 입니다. Version Python: 3.9 DB: PostgreSQL 13 Django
A project to work with databases in 4 worksheets, insert, update, select, delete using Python and MySqI
A project to work with databases in 4 worksheets, insert, update, select, delete using Python and MySqI As a small project for school or college hope it is useful
Experimental proxy for dumping the unencrypted packet data from Brawl Stars (WIP)
Brawl Stars Proxy Experimental proxy for version 39.99 of Brawl Stars. It allows you to capture the packets being sent between the Brawl Stars client
Add your recently blog and douban states in your GitHub Profile
Add your recently blog and douban states in your GitHub Profile
These are the scripts used for the project of ‘Assembly of a pan-genome for global cattle reveals missing sequence and novel structural variation, providing new insights into their diversity and evolution history’
script-SV-genotyping These are the scripts used for the project of ‘Assembly of a pan-genome for global cattle reveals missing sequence and novel stru
Random Programming Language Project
Crastle Random Programming Language Project Freedom of expression Are you a fan of curly brace languages? Then use curly braces! Not a fan of curly br
Plock : A stack based programming language
Plock : A stack based programming language
Python plugin/extra to load data files from an external source (such as AWS S3) to a local directory
Data Loader Plugin - Python Table of Content (ToC) Data Loader Plugin - Python Table of Content (ToC) Overview References Python module Python virtual
TallerStereoVision Convencion Python Chile 2021
TallerStereoVision Convencion Python Chile 2021 Taller Stereo Vision & Python PyCon.cl 2021 Instalación Se recomienta utilizar Virtual Environment pyt
Advanced Variable Manager {AVM} [0.8.0]
Advanced Variable Manager {AVM} [0.8.0] By Grosse pastèque#6705 WARNING : This modules need some typing modifications ! If you try to run it without t
Scrapper For Paste.pics
PrntScScrapper Scrapper for Paste.pics If you are bored you can find some random screenshots from prnt.sc Features Saving screenshots Open in Browser
Collection of system-wide scripts that I use on my Gentoo
linux-scripts Collection of scripts that I use on my Gentoo machine. I tend to put all scripts in /scripts directory. It is not likely that you would
Free and open source qualitative research tool
Taguette A spin on the phrase "tag it!", Taguette is a free and open source qualitative research tool that allows users to: Import PDFs, Word Docs (.d
A simple project which is a ecm to found a good way to provide a path to img_dir in gooey
ECM to find a good way for img_dir Path in Gooey This code is just an ECM to find a good way to indicate a path of image in image_dir variable. We loo
Gba-free-fonts - Free font resources for GBA game development
gba-free-fonts Free font resources for GBA game development This repo contains m