Add main topic Concurrency

[madmanRE]

Roadmap URL

https://roadmap.sh/python

Suggestions

Concurrency

Concurrency in Python allows multiple tasks to be executed simultaneously using different approaches. GIL (Global Interpreter Lock) limits thread execution, making multithreading less efficient for computational tasks, but suitable for I/O. Multiprocessing, using the multiprocessing module, allows multiple cores to be utilized, providing true parallelism. Asynchrony via asyncio is optimal for I/O operations, allowing thousands of connections to be processed simultaneously without blocking. The choice of approach depends on the nature of the task.

Relevant links

• https://realpython.com/python-concurrency/

(subtopics below)

GIL

GIL is a mechanism that allows only one thread to execute Python code at a time. This limitation is related to memory management in CPython and can reduce the efficiency of multithreaded applications on multi-core systems.

Relevant links

• https://realpython.com/python-gil/

Threading

Multithreading allows multiple threads within a single process. However, because of GIL, threads cannot run in parallel on different cores, which makes multithreading suitable for I/O tasks (e.g., network requests) but not for computational tasks.

Relevant links

• https://realpython.com/intro-to-python-threading/
• https://docs.python.org/3/library/threading.html

Multiprocessing

Multiprocessing utilizes multiple processes, each with its own GIL. This allows full utilization of multiple processor cores, which is effective for computationally intensive tasks. Python's multiprocessing module supports creating processes and exchanging data between them.

Relevant links

• https://docs.python.org/3/library/multiprocessing.html
• https://www.digitalocean.com/community/tutorials/python-multiprocessing-example

Asynchrony

Asynchronous programming, supported by asyncio, allows code to be executed without blocking, using async and await. This is especially useful for I/O tasks such as networking or file manipulation, allowing thousands of connections to be handled without blocking the main thread.

Relevant links

• https://realpython.com/async-io-python/
• https://docs.python.org/3/library/asyncio.html