Python Generator: A Function is said to be a Python Generator that produces or generates a sequence of results. A Python Generator maintains its native state to work so that whenever a function is called subsequent times, it can resume again exactly where it left off.

We can use the keyword known as yield to maintain the state of the function. The following is the syntax for the keyword yield:

Python keyword yield works much similar to return; however, there are few significant differences between them that we will discuss in the following section.

Along with the yield statement, Generators were introduced in PEP 255. The Generators have been accessible since Python version 2.2.

Understanding the working of Generators

Let’s see the following example to understand the working of Generators:

In the above snippet of code, we have defined a generator, namely num_Generator. This generator is taking a value i as an argument, which is defined and used in a while loop as the limit value. Moreover, we have also defined a variable, namely num and assigned its value to zero.

We have used the next() method to call the “instantiated” generator (my_Generator) and runs the code until the yield statement returns 1 in the first case.

Even once the value is returned to us, the function still stores the variable num value for the next iteration or for the next time when the function is called and increases its value by one.

Now, when we called the function one more time, it will return the next number as per the sequence, and in this case, we have called it three more times. Thus, the function will provide the next three numbers, as shown below:





This functionality of generators helps to create iterable dynamically on the fly. The program will return an array of numbers (such as [0, 1, 2, 3]) instead of a generator object if we wrap my_Generator with the list(). Furthermore, it is a bit easier to work within some applications.

Difference between yield and return

The keyword yield helps in maintaining the state between the calls of the function. It resumes the function from where it left off whenever the next() method is called.

On the other hand, the keyword return helps to return a value from a function and loses its native state once it returns the function. Thus, the function starts over from its first statement whenever we call that function for the next time.

Hence, if we call the keyword yield in the generator, we will pick right back up after the preceding yield statement the next time we call the same generator.

The Use of return in a Generator

We can use a return statement in a generator; however, the one possible way is to use it without returning any value, as shown below:

This helps the generator proceeding as in any other function return when it finds the return statement.

Let’s see the following example based on this concept:


Explanation –

As we can observe, we have made some initial changes in our previous program by adding an if-else condition, which will discriminate against the numbers higher than 25. However, we have passed the argument like 35, which is higher than 25. Thus, the generator won’t yield any values; instead, it will return an empty array. Hence, we can conclude that the return statement works similar to a break statement in the following case.

The Use of next() in Generator

We can use the next() method to parse the values that are yielded by a generator. This method lets the generator to return the subsequent iterable value only.

Let’s see the following example demonstrating the same:


Explanation –

As we can observe, the above snippet of code is similar to the previous one. However, the generator yielded each value using the next() method. We have instantiated generator gen, which works as a variable holding the state of the generator.

We have called the next() method, and the Python generator executed the function until it finds a yield statement. Then, the yielded value is returned to the caller and save the generator state for further use.

What is Generator Expression?

The Generator expressions are similar to the list comprehensions. However, the generator expression returns a generator instead of a list. Introduced in PEP 289, they became a part of Python since version 2.4.

They have a similar overall syntax to that of list comprehensions. However, they use parenthesis instead of square brackets.

Let’s see the following example to understand the use of generator expressions:

From the above snippet of code, we would have the following output:

Generator expressions play a significant role while using the reduction functions like min(), max(), or sum(), because they help in reducing the code to a single line and much shorter to type than a full function.

Let’s see one more example shown below demonstrating the sum of the first 15 numbers:

From the above snippet of code, we would have the following output:

Exceptions Handling

It is noteworthy that the keyword yield is not allowed in the try section but in the finally clause of the try/finally statements. But it can appear in the try and except clause of the try/except statements. So, we should be cautious about allocating the resources of the generator.

An example is shown below, demonstrating the same:


Explanation –

In the above snippet of code, we have included 15 numbers in the finally clause, and as a result, a list of numbers from 0 to 15 will be printed. However, this should not be happening as we have provided a condition as num < i.

The Concept of Connecting Generators

A new feature is added in Python version 3.3 that allows the user to interconnect generators and delegate to a sub-generator.

The syntax for defined new expression in PEP 380 is as follows:

In the above syntax, <the_expression> is an iterable evaluated expression used to define the delegating generator.

An example is shown below, demonstrating the same:

Example –


Explanation –

In the above snippet of code, we have defined three distinct generators. The first, namely my_Gen1, has a parameter as input for specifying the limit in a range.

The second, namely my_Gen2, is similar to the last one but with two parameters as input for specifying the two limits allowed in the number range. And the last one, namely my_Gen3 that calls my_Gen1 and my_Gen2 for yielding their values.

The last three syntax helps print the lists generated from each of the generators defined.

Some Benefits of Generators

  1. Simplified Code: The generator helps in simplifying the code in a very well-designed manner. This simplification and elegance can be observed in the previous examples throughout the topic. Moreover, the generator expressions provide much more elegance and simplifications, where an entire code block can be replaced with a single line of code.
  2. Better Performance: The generator also provides better performance as it works on on-demand value generation. Thus, it helps in reducing memory consumption.

When should we use Generators?

The Python Generator is an advanced tool that increases the efficiency in various programming cases. Some of these programming cases are as follows:

  1. The generator provides calculation lazy, also known as on-demand evaluation that helps collect a large amount of data. This technique also helps in stream processing.
  2. The generator can also be useful in concurrency generation (simulation).
  3. We can use the stacked generators as Pipes, in a similar manner as Unix pipes.

Pin It on Pinterest

Share This