Avoid PyCharm Overloading
=========================
This speeds up PyCharm's package index processes and avoids CPU & memory
overloading.
On my machine (4c/32GB) with 5 conda environments and 10 projects, this reduces
PyCharm's package scanning from 15 minutes to 45 seconds.
Why?
----
When PyCharm has multiple conda environments, it will sometimes launch concurrent
package scanning across those environments. To do so, it uses the `conda` cli,
and a tool called conda_packaging_tool.py, available from the JetBrains
intellij-community github repository (see below).
Unfortunately, these tools do not behave well when launched in parallel, as
they tend to overload CPU & memory in this case.
How?
----
To avoid stalling these processes, we modify their sources as follows:
1. modify the conda cli
$ conda activate base
$ pip install runfast
$ nano `which conda`
Modify the main section:
if __name__ == '__main__':
from conda.cli import main
from runfast import cached
cached(main)
2. modify the conda_packaging_tool.py
if __name__ == '__main__':
from runfast import cached
cached(main)
How does this work?
-------------------
Two simple steps:
1. When called in parallel, only one process is allowed to proceed.
2. `runfast.cached` caches the output of these tools (stdout, stderr) for 1 minute,
given the same command line parameters
That is, when PyCharm launches 5 package scanning (`conda list`) commands, only
one of them will run immediately. If some of the scans are for the same environment,
only one of them will actually run, while the others simply return the same output.
How to clear the cache or avoid caching at all?
$ export RUNFAST_NOCACHE=1
Testing
-------
a) conda packaging tool
# first time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m13.271s
user 0m9.631s
sys 0m1.936s
# second time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m3.159s
user 0m0.139s
sys 0m1.298s
b) conda cli
# first time
$ time conda list -p /path/to/env
real 0m7.985s
user 0m7.883s
sys 0m0.087s
# second time
real 0m0.152s
user 0m0.116s
sys 0m0.032s
References
----------
* https://github.com/JetBrains/intellij-community/blob/master/python/helpers/conda_packaging_tool.py
* https://github.com/conda/conda/blob/33a142c16530fcdada6c377486f1c1a385738a96/conda/core/index.py#L53
This speeds up PyCharm's package index processes and avoids CPU & memory overloading
Overview
Medical appointments No-Show classifier
Medical Appointments No-shows Why do 20% of patients miss their scheduled appointments? A person makes a doctor appointment, receives all the instruct
A simple program to run through inputs for a 3n+1 problem
Author Tyler Windemuth Collatz_Conjecture A simple program to run through inputs for a 3n+1 problem Purpose: doesn't really have a purpose, did this t
This is a simple quizz which can ask user for login/register session, then consult to the Quiz interface.
SIMPLE-QUIZ- This is a simple quizz which can ask user for login/register session, then consult to the Quiz interface. By CHAKFI Ahmed MASTER SYSTEMES
An implementation to rank your favourite songs from World of Walker
World-Of-Walker-Elo An implementation to rank your favourite songs from Alan Walker's 2021 album World of Walker. Uses the Elo rating system, which is
Demo repository for Saltconf21 talk - Testing strategies for Salt states
Saltconf21 testing strategies Demonstration repository for my Saltconf21 talk "Strategies for testing Salt states" Talk recording Slides and demos Get
This is a Saleae Logic custom high level analyzer that allows you to search and mark specific packets.
SaleaePacketParser This is a Saleae Logic custom high level analyzer that allows you to search and mark specific packets. Field "Search For" is used f
Basic-Killfeed - A simple DayZ Console Killfeed
Basic-Killfeed A simple DayZ Console Killfeed. Setup Install Python Version 3.10
Dot Browser is a privacy-conscious web browser with smarts built-in for protection against trackers and advertisments online.
🌍 Take back your privacy with Dot Browser, the privacy-conscious web browser that protects you from being tracked and monitored online.
tool to automate exploitation of android degubg bridge vulnerability
DISCLAIMER DISCLAIMER: ANY MALICIOUS USE OF THE CONTENTS FROM THIS ARTICLE WILL NOT HOLD THE AUTHOR RESPONSIBLE HE CONTENTS ARE SOLELY FOR EDUCATIONAL
Helps compare between New and Old Tax Regime.
Income-Tax-Calculator Helps compare between New and Old Tax Regime. Sample Console Input/Output
BloodCheck enables Red and Blue Teams to manage multiple Neo4j databases and run Cypher queries against a BloodHound dataset.
BloodCheck BloodCheck enables Red and Blue Teams to manage multiple Neo4j databases and run Cypher queries against a BloodHound dataset. Installation
Powering up Apache JMeter with Streamlit and opening the door for machine learning.
Powering up Apache JMeter with Streamlit Overview Apache JMeter is an open source load testing tool written in 100% pure Java. JMeter supports umpteen
An useful scripts for Misskey
misskey-scripts This place storing useful scripts which made by me. icon-repair Repair broken remote user's icon.
30DaysOfCode-PhoenixClub - Solution of everyday coding problem given in 30DaysofCode contest held on Hackerrank
30DaysOfCode-PhoenixClub 👨💻 Every day problems solution given in 30DaysOfCode
Objetivo: de forma colaborativa pasar de nodos de Dynamo a Python.
ITTI_Ed01_De-nodos-a-python ITTI. EXPERT TRAINING EN AUTOMATIZACIÓN DE PROCESOS BIM: OFFICIAL DE AUTODESK. Edición 1 Enlace al Master Enunciado: Traba
Drug Discovery App Using Lipinski's Rule-of-Five.
Drug Discovery App A Drug Discovery App Using Lipinski's Rule-of-Five. TAPIWA CHAMBOKO 🚀 About Me I'm a full stack developer experienced in deploying
This is the Halloween edition of my Flask Greeting App - HAPPY HALLOWEEEN EVERYONE! :)
HalloweenGreetingApp HAPPY HALLOWEEN EVERYONE! :) This is the Halloween Edition of my Flask Greeting App! Please note, this application is mean to be
Data on Free Food at MIT
MIT Free Food Timing Procrastinating research by plotting data on how long it takes emails on the free-food at mit edu mailing list to go through. Dat
this is a basic python project that I made using python
this is a basic python project that I made using python. This project is only for practice because my python skills are still newbie.
Modelling and Implementation of Cable Driven Parallel Manipulator System with Tension Control
Cable Driven Parallel Robots (CDPR) is also known as Cable-Suspended Robots are the emerging and flexible end effector manipulation system. Cable-driven parallel robots (CDPRs) are categorized as a t