Multiprocessing in python

The word multiprocessing is used to compute the operation in which more than two processors run simultaneously with more than one central processor. The computer's performance increases gradually. Multiprocessing is the ability of a system to run more than two processors simultaneously. The main application in multiprocessing will split into small threads and will run independently. These threads are allocated to the processors to increase the computer's performance.

Why do we Use multiprocessing?

Let’s consider a single processor in a computer system. When we assign multiple processes simultaneously, it takes more time, and it will disturb each task and move to others, which will be repeated until every process is completed.

This matches a housewife working at the workplace as well as at home. She has to look after both office work and household work.

The problem is that it is more critical to track all of them when you assign more processes to a single computer, and each process must run on time; this becomes more difficult. So within a single computer, multiprocessing uses more than two CPUs. In the operating system, every process is allocated to the processor.

1. Multiprocessing has more than one central processor, which is represented by a computer.
2. The multicore processor has a single computing component with two or more independent units.

How to Perform Multiprocessing in Python

To perform multiple tasks in a single system, python provides a multiprocessing module. It provides API and is user-friendly to work with multiprocessing.

EXAMPLE

# Import  multiprocessing module
import multiprocessing


def print_sum(num):
    """
    function to print the sum of given num
    """
    print ("sum: {}". format (num + num ))


def print_mul(num):
    """
    function to print mul of given num
    """


Print ("mul: {}". format (num * num))


if __name__ == "__main__":


    p1 = multiprocessing. Process (target=print_ sum, args = (15,))
    p2 = multiprocessing. Process (target=print_ mul, args= (15,))




    p1. start ()


    p2. start ()




    p1. join ()


    p2. join ()


    # Both processes finished
    print("Done!")
Square: 1

Output

Done!

Explanation

1. To import the module of multiprocessing, we use import multiprocessing.
2. For creating a process, we must create an object of a process class. It contains the following arguments:
   • Target: This is the function that the process will execute.
   • Args:  In this, the arguments will be passed to the target function.
   • From the above example, we have created two processes, namely p1 and p2.
   • p1 = multiprocessing. Process (target=print _sum, args=(10,))
   • p2 = multiprocessing. Process (target=print_ mul, args=(10,))
     
3. We use the start method to start a process.
   • P1. start ()
   • P2. start ()
     
4. The present program will execute only when the program starts. Using the join method stops the execution of the current program until and unless the process is completed.
   • P1. join ()
   • P2. join ()
5. From the above-written program, until the completion of p1 and p2 current program will stop, and the following statements of the present program are executed once they are finished.

Communication between Processes

Communication is nothing but exchanging information between two sources. Without communication between each other or each resource, it isn't easy to carry out a task. Here in our topic, the data must be exchanged between processes to develop the parallel application. Processes use queues mostly to pass messages back and forth to other processes.

The below flowchart shows different communications mechanisms for the synchronization between multiple subprocesses. There are two types of multiprocessing, such as:

1. Asymmetric multiprocessing
2. Symmetric multiprocessing

Asymmetric Multiprocessing

In asymmetric multiprocessing, there will be the use of two or more processors that only one master processor handles. The CPUs are connected but are not self-scheduling. This is used to schedule a particular task to the CPU based on its importance and priority of tasks.

The processor's precedence is not equal. It is simpler to design and cheaper to implement. The processors are not treated equally in asymmetric multiprocessing. The main purpose of the master processor is to complete tasks of an operating system. 

Symmetric Multiprocessing

Symmetric multiprocessing uses two or more self-scheduling processors that share only a single and common memory space. Here every processor has access to memory devices and input and output. Symmetric multiprocessing uses two or more CPUs for a task and should complete it faster and more parallelly.

The processors are equally treated in symmetric multiprocessing. For communication between processors, shared memory will be used. Each processor is responsible for tasks of the operating system.

Different Communication Mechanisms

Queues, pipes, managers, and Ctypes are the different communication mechanisms. Firstly, we will learn about Queues.

1. QUEUES
   
   Multi-process programs use a queue communication mechanism for processing data between processes. API is used in multiprocessing because the queue class of the multiprocessing module is similar to the queue. As we know, the queue follows FIFO (First-in, first-out) mechanism, meaning the data element added first will be removed first, and the data element added last will be removed last.
   
   For communicating between processes, the queue provides a thread and also the process-safe FIFO mechanism for communication between processes.

For example,

# Importing Queue Class 


from multiprocessing import Queue  


cars = ['Benz', 'Audi', 'Kia', 'Tata', 'Skoda']  
count = 1  
# Creating a queue object  
queue = Queue ()  
print ('pushing items to the queue:')  
for c in cars:  
   print ('item no: ', count, ' ', 
   queue. put(c)  
   count += 1  


print ('\npopping items from the queue:')  
count = 0  
while not queue.empty(): 
   print ('item no: ', count, ' ', queue.get())  
   count += 1  

Output

The above program shows we get to push and pop operations using a multiprocessing queue. We have imported the class of queue and a list of car names, and also, we had assigned count to 1; this count variable will count the total number of elements in the list.

Next, we created a queue object using the Queue () method. In for loop, we have inserted elements in order so that it performs operations in the queue.

• PIPES
  
  Pipes are data structures that communicate between processes like multi-process programs. The function named pipe () will return a pair of connection objects which are connected by a pipe that has two ways called send () and receive ().
  
  For communicating between processes, a pipe acts as a medium of communication. A pipe can be within communication between parent and child processes or one process.
  
  In other words, communication can be hierarchical, like between grandparents, parents, children, etc. Communication between processes can be achieved by writing one process into the pipe and reading another process from the pipe. It works the same as a pipe system call that is one file to write and another to read from that file.
  
  Let us take a general scenario, such as plating trees by using a pipe here, plants will absorb water that is coming from the pipe, and we are pouring water with a pipe the same as above.

• MANAGER
  The manager is a class of multiprocessing modules that provides a way to process the information shared between all users. The object is a manager that controls shared objects and allows other processes to duplicate them. We can also say that manager is a communication module used to create data that share between processes of various types. 
  There are different properties of manager object; they are:
  • To control the server process, the manager manages the shared objects.
  • It has an important property that updates shared objects when a process modifies them.

• C-TYPES Arrays and Values
  A c-type module provides tools for working with c datatypes. These c-type modules allow python to read and write with data using standard datatypes.
  
  The multiprocessing concept provides Array and Value objects for storing data in a shared memory map. Here Array is a c-type array allocated from shared memory, and Value is a c-type object allocated from shared memory.