Bash in Python Connected Components in Python Cosine Similarity in Python Cows and Bulls Game in Python Create a Directory in Python How to Calculate CRC32 with Python? How to Check Whether a Number is Prime or Not in Python? Iris Dataset in Python islice in Python Reading Binary Files in Python The has_key Method in Python Alternatives to Python Min Max Algorithm In Python Ways to Add Rows in Dataframe Apriori Algorithm in Python Difference Between Series, Dataframe, and Panel Data Structure in Pandas Graph Data Structure in Python List the frequency of elements in Python Glass Box Testing

Python Generator
2020-12-17 08:30:13

Python Generator

Python Generator: A Function is said to be a Python Generator that produces or generates a sequence of results. A Python Generator maintains its native state to work so that whenever a function is called subsequent times, it can resume again exactly where it left off.

We can use the keyword known as yield to maintain the state of the function. The following is the syntax for the keyword yield:

yield [expression]

Python keyword yield works much similar to return; however, there are few significant differences between them that we will discuss in the following section.

Along with the yield statement, Generators were introduced in PEP 255. The Generators have been accessible since Python version 2.2.

Understanding the working of Generators

Let’s see the following example to understand the working of Generators:

def num_Generator(i):
      num = 0
      while num < i:
          yield num
          num += 1
 my_Generator = num_Generator(4)
 print(next(my_Generator))
 print(next(my_Generator))
 print(next(my_Generator))
 print(next(my_Generator))

In the above snippet of code, we have defined a generator, namely num_Generator. This generator is taking a value i as an argument, which is defined and used in a while loop as the limit value. Moreover, we have also defined a variable, namely num and assigned its value to zero.

We have used the next() method to call the “instantiated” generator (my_Generator) and runs the code until the yield statement returns 1 in the first case.

Even once the value is returned to us, the function still stores the variable num value for the next iteration or for the next time when the function is called and increases its value by one.

Now, when we called the function one more time, it will return the next number as per the sequence, and in this case, we have called it three more times. Thus, the function will provide the next three numbers, as shown below:

0

1

2

3

This functionality of generators helps to create iterable dynamically on the fly. The program will return an array of numbers (such as [0, 1, 2, 3]) instead of a generator object if we wrap my_Generator with the list(). Furthermore, it is a bit easier to work within some applications.

Difference between yield and return

The keyword yield helps in maintaining the state between the calls of the function. It resumes the function from where it left off whenever the next() method is called.

On the other hand, the keyword return helps to return a value from a function and loses its native state once it returns the function. Thus, the function starts over from its first statement whenever we call that function for the next time.

Hence, if we call the keyword yield in the generator, we will pick right back up after the preceding yield statement the next time we call the same generator.

The Use of return in a Generator

We can use a return statement in a generator; however, the one possible way is to use it without returning any value, as shown below:

return statement

This helps the generator proceeding as in any other function return when it finds the return statement.

Let’s see the following example based on this concept:

def num_Generator(i):
   if i < 25:
      num = 0
      while num < i:
          yield num
          num += 1
   else:
      return
 print(list(num_Generator(35)))

Output:

[]

Explanation -

As we can observe, we have made some initial changes in our previous program by adding an if-else condition, which will discriminate against the numbers higher than 25. However, we have passed the argument like 35, which is higher than 25. Thus, the generator won’t yield any values; instead, it will return an empty array. Hence, we can conclude that the return statement works similar to a break statement in the following case.

The Use of next() in Generator

We can use the next() method to parse the values that are yielded by a generator. This method lets the generator to return the subsequent iterable value only.

Let’s see the following example demonstrating the same:

def num_Generator(i):
      num = 0
      while num < i:
          yield num
          num += 1
 gen = num_Generator(15)
 cntr = 0
 while cntr < 15:
     print(next(gen))
     cntr += 1

Output:

0
 1
 2
 3
 4
 5 
 6
 7
 8
 9
 10
 11
 12
 13
 14

Explanation -

As we can observe, the above snippet of code is similar to the previous one. However, the generator yielded each value using the next() method. We have instantiated generator gen, which works as a variable holding the state of the generator.

We have called the next() method, and the Python generator executed the function until it finds a yield statement. Then, the yielded value is returned to the caller and save the generator state for further use.

What is Generator Expression?

The Generator expressions are similar to the list comprehensions. However, the generator expression returns a generator instead of a list. Introduced in PEP 289, they became a part of Python since version 2.4.

They have a similar overall syntax to that of list comprehensions. However, they use parenthesis instead of square brackets.

Let’s see the following example to understand the use of generator expressions:

gen = (i for i in range(15))
print(list(gen))

From the above snippet of code, we would have the following output:

$ python generator.py
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]

Generator expressions play a significant role while using the reduction functions like min(), max(), or sum(), because they help in reducing the code to a single line and much shorter to type than a full function.

Let’s see one more example shown below demonstrating the sum of the first 15 numbers:

gen = (i for i in range(15))
print(sum(gen))

From the above snippet of code, we would have the following output:

105

Exceptions Handling

It is noteworthy that the keyword yield is not allowed in the try section but in the finally clause of the try/finally statements. But it can appear in the try and except clause of the try/except statements. So, we should be cautious about allocating the resources of the generator.

An example is shown below, demonstrating the same:

def num_Generator(i):
   try:
      num = 0
      while num < i:
          yield num
          num += 1
   finally:
      yield i
 print(list(num_Generator(15)))

Output:

 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]

Explanation -

In the above snippet of code, we have included 15 numbers in the finally clause, and as a result, a list of numbers from 0 to 15 will be printed. However, this should not be happening as we have provided a condition as num < i.

The Concept of Connecting Generators

A new feature is added in Python version 3.3 that allows the user to interconnect generators and delegate to a sub-generator.

The syntax for defined new expression in PEP 380 is as follows:

yield from <the_expression>

In the above syntax, <the_expression> is an iterable evaluated expression used to define the delegating generator.

An example is shown below, demonstrating the same:

Example -

def my_Gen1(i):
     for x in range(i):
         yield x
 def my_Gen2(i, j):
     for y in range(i, j):
         yield y
 def my_Gen3(i, j):
     yield from my_Gen1(i)
     yield from my_Gen2(i, j)
     yield from my_Gen2(j, j+6)
 print(list(my_Gen1(15)))
 print(list(my_Gen2(6, 15)))
 print(list(my_Gen3(0, 15)))

Output:

 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
 [6, 7, 8, 9, 10, 11, 12, 13, 14]
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

Explanation -

In the above snippet of code, we have defined three distinct generators. The first, namely my_Gen1, has a parameter as input for specifying the limit in a range.

The second, namely my_Gen2, is similar to the last one but with two parameters as input for specifying the two limits allowed in the number range. And the last one, namely my_Gen3 that calls my_Gen1 and my_Gen2 for yielding their values.

The last three syntax helps print the lists generated from each of the generators defined.

Some Benefits of Generators

  1. Simplified Code: The generator helps in simplifying the code in a very well-designed manner. This simplification and elegance can be observed in the previous examples throughout the topic. Moreover, the generator expressions provide much more elegance and simplifications, where an entire code block can be replaced with a single line of code.
  2. Better Performance: The generator also provides better performance as it works on on-demand value generation. Thus, it helps in reducing memory consumption.

When should we use Generators?

The Python Generator is an advanced tool that increases the efficiency in various programming cases. Some of these programming cases are as follows:

  1. The generator provides calculation lazy, also known as on-demand evaluation that helps collect a large amount of data. This technique also helps in stream processing.
  2. The generator can also be useful in concurrency generation (simulation).
  3. We can use the stacked generators as Pipes, in a similar manner as Unix pipes.