Metadata-Version: 2.1 Name: multiprocess Version: 0.70.15 Summary: better multiprocessing and multithreading in Python Home-page: https://github.com/uqfoundation/multiprocess Download-URL: https://pypi.org/project/multiprocess/#files Author: Mike McKerns Author-email: mmckerns@uqfoundation.org Maintainer: Mike McKerns Maintainer-email: mmckerns@uqfoundation.org License: BSD-3-Clause Project-URL: Documentation, http://multiprocess.rtfd.io Project-URL: Source Code, https://github.com/uqfoundation/multiprocess Project-URL: Bug Tracker, https://github.com/uqfoundation/multiprocess/issues Platform: Linux Platform: Windows Platform: Mac Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: Intended Audience :: Science/Research Classifier: License :: OSI Approved :: BSD License Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: 3.9 Classifier: Programming Language :: Python :: 3.10 Classifier: Programming Language :: Python :: 3.11 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Classifier: Topic :: Scientific/Engineering Classifier: Topic :: Software Development Requires-Python: >=3.7 License-File: LICENSE License-File: COPYING Requires-Dist: dill (>=0.3.7) ----------------------------------------------------------------- multiprocess: better multiprocessing and multithreading in Python ----------------------------------------------------------------- About Multiprocess ================== ``multiprocess`` is a fork of ``multiprocessing``. ``multiprocess`` extends ``multiprocessing`` to provide enhanced serialization, using `dill`. ``multiprocess`` leverages ``multiprocessing`` to support the spawning of processes using the API of the Python standard library's ``threading`` module. ``multiprocessing`` has been distributed as part of the standard library since Python 2.6. ``multiprocess`` is part of ``pathos``, a Python framework for heterogeneous computing. ``multiprocess`` is in active development, so any user feedback, bug reports, comments, or suggestions are highly appreciated. A list of issues is located at https://github.com/uqfoundation/multiprocess/issues, with a legacy list maintained at https://uqfoundation.github.io/project/pathos/query. Major Features ============== ``multiprocess`` enables: - objects to be transferred between processes using pipes or multi-producer/multi-consumer queues - objects to be shared between processes using a server process or (for simple data) shared memory ``multiprocess`` provides: - equivalents of all the synchronization primitives in ``threading`` - a ``Pool`` class to facilitate submitting tasks to worker processes - enhanced serialization, using ``dill`` Current Release =============== The latest released version of ``multiprocess`` is available from: https://pypi.org/project/multiprocess ``multiprocess`` is distributed under a 3-clause BSD license, and is a fork of ``multiprocessing``. Development Version =================== You can get the latest development version with all the shiny new features at: https://github.com/uqfoundation If you have a new contribution, please submit a pull request. Installation ============ ``multiprocess`` can be installed with ``pip``:: $ pip install multiprocess For Python 2, a C compiler is required to build the included extension module from source. Python 3 and binary installs do not require a C compiler. Requirements ============ ``multiprocess`` requires: - ``python`` (or ``pypy``), **>=3.7** - ``setuptools``, **>=42** - ``dill``, **>=0.3.7** Basic Usage =========== The ``multiprocess.Process`` class follows the API of ``threading.Thread``. For example :: from multiprocess import Process, Queue def f(q): q.put('hello world') if __name__ == '__main__': q = Queue() p = Process(target=f, args=[q]) p.start() print (q.get()) p.join() Synchronization primitives like locks, semaphores and conditions are available, for example :: >>> from multiprocess import Condition >>> c = Condition() >>> print (c) ), 0> >>> c.acquire() True >>> print (c) ), 0> One can also use a manager to create shared objects either in shared memory or in a server process, for example :: >>> from multiprocess import Manager >>> manager = Manager() >>> l = manager.list(range(10)) >>> l.reverse() >>> print (l) [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] >>> print (repr(l)) Tasks can be offloaded to a pool of worker processes in various ways, for example :: >>> from multiprocess import Pool >>> def f(x): return x*x ... >>> p = Pool(4) >>> result = p.map_async(f, range(10)) >>> print (result.get(timeout=1)) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] When ``dill`` is installed, serialization is extended to most objects, for example :: >>> from multiprocess import Pool >>> p = Pool(4) >>> print (p.map(lambda x: (lambda y:y**2)(x) + x, xrange(10))) [0, 2, 6, 12, 20, 30, 42, 56, 72, 90] More Information ================ Probably the best way to get started is to look at the documentation at http://multiprocess.rtfd.io. Also see ``multiprocess.tests`` for scripts that demonstrate how ``multiprocess`` can be used to leverge multiple processes to execute Python in parallel. You can run the test suite with ``python -m multiprocess.tests``. As ``multiprocess`` conforms to the ``multiprocessing`` interface, the examples and documentation found at http://docs.python.org/library/multiprocessing.html also apply to ``multiprocess`` if one will ``import multiprocessing as multiprocess``. See https://github.com/uqfoundation/multiprocess/tree/master/py3.11/examples for a set of examples that demonstrate some basic use cases and benchmarking for running Python code in parallel. Please feel free to submit a ticket on github, or ask a question on stackoverflow (**@Mike McKerns**). If you would like to share how you use ``multiprocess`` in your work, please send an email (to **mmckerns at uqfoundation dot org**). Citation ======== If you use ``multiprocess`` to do research that leads to publication, we ask that you acknowledge use of ``multiprocess`` by citing the following in your publication:: M.M. McKerns, L. Strand, T. Sullivan, A. Fang, M.A.G. Aivazis, "Building a framework for predictive science", Proceedings of the 10th Python in Science Conference, 2011; http://arxiv.org/pdf/1202.1056 Michael McKerns and Michael Aivazis, "pathos: a framework for heterogeneous computing", 2010- ; https://uqfoundation.github.io/project/pathos Please see https://uqfoundation.github.io/project/pathos or http://arxiv.org/pdf/1202.1056 for further information.