Difference between iterators and generators in Python

Table of contents

Introduction

Today, we are going to discuss a little bit confusing topic in Python programming language. We are going to clarify the difference between iterators and generators. Before we start, I advise beginners to ready slowly until the confusion goes away. First, we will start with basic concepts then build on top of that. At the end of the article, if you still think it is not clear you may start over while taking notes. This topic is not straight forward but not that hard. Just keep track of what is being explained.

Let us get started…

Python is a scripting language but it is also object oriented so I assume you are familiar with the basics of object oriented programming concepts such as classes, objects and methods. To refresh your memory, a class is simply an object type or definition and an object is an instance of a class. Classes (therefore objects) encapsulate data members and methods to operate on. That is all what we need to recall from OOP for now.

What is an iterator?

In plain English, an iterator is an object that keeps track of some state and returns the next state. For example, an odd number iterator currently at 7 returns the next odd number of 9. Yes, it is that simple! Programmatically speaking, it is a class that implements a next method which in turn operates on a state data member regardless of how the next value is calculated.

Take a look at the following example…

Iterator example

As an example, we are going to implement the odd number iterator we just mentioned earlier. Here is the code in Python 3 syntax:

As you can see, the odd iterator is a Python class named OddIterator. The state data member is the variable curr which keeps track of the current odd number. The next method returns the next odd number by adding 2 to the current value. If you run the code above, you should get the following output:

What is an iterable?

Now we know what an iterator is, what is an iterable then? An iterable is an object that returns an iterator for the purpose of returning all of its elements. An iterable implements the iter method and returns an iterator object. The returned iterator can be an instance of a separate class or we can use a single class for both the iterable and the iterator it returns.

Take a look at the following examples…

Iterable example – separate class

Here is the code in Python 3 syntax:

If you run the code above, you should get the following output:

Iterable example – single class

For convenience, we can have a single class that implements both methods (iter and next). This makes the class both an iterable and its own iterator. Let us see how can we do that. Here is the code in Python 3 syntax:

When you run a for loop against an iterable, the iter method is called which returns a iterator to print all elements. The output of the above code should be the following:

Python containers

As we are talking about iterables, let us quickly talk about a relevant topic i.e. Python containers. Python supports built in data types called containers such as lists, sets and dictionaries to mention a few. These data types are typically iterable. This means you can easily run a for loop against these types and get access to their elements. You may need to check this article for more information on how to iterate through containers using a Python for loop.

What are generators

After we have explained what an iterator and iterable are, we can now define what a Python generator is. In short, a generator is a special kind of iterator that is implemented in an elegant way. It is a powerful programming construct that enables us to write iterators without the need to use classes or implement the iter and next methods.

In Python, we can write generators in two different ways…

Generator types

  1. Generator function: in the form of a regular function but the difference in syntax between a generator function and a regular function is that a yield statement is used instead of a return statement. A regular function returns a value but a generator function returns a generator object. Calling a generator function does not change the generator’s state.
  2. Generator expression: in the form of a list comprehension but instead of using square brackets [ ] we use parenthesis ( ). For more information about list comprehension check the following article:

Let us look at some examples…

Generator function example

We want to implement the odd iterator using a generator function. Here is the code:

In the code above, the generator function OddNumber returns an OBJECT not a value. This is what you need to keep in mind. Think of it, it is not any different from the iterator class that we implemented earlier. The only difference is the syntax which is more elegant.

Let us now implement it using the generator comprehension…

Generator comprehension example

That is all for today. I hope I was able to help you understand the difference between iterators and generators in Python. In the next section, I will end by mentioning some advantages of using generators followed by a summary.

Generator advantages

We can definitely achieve the same results using regular Python code as opposed to iterators and generators however using generators has some advantages.

  • The code using generators is compact and elegant with fewer intermediate data structures
  • The code runs faster because generators are memory and CPU efficient as they run lazily without the need to buffer large memory space (ex. reading a large file)
  • Generators work just fine with infinite streams as in the case of reading from a network socket
  • End user decides how he or she wants to use the result meaning end user decides whether he wants to put the result in a list, set or dictionary

Summary

  • An iterator is an object that keeps track of some state and returns the next state
  • An iterable implements the iter method and returns an iterator object
  • We can have a single class that implements both methods (iter and next). This makes the class both an iterable and its own iterator
  • A generator is a programming construct that enables us to write iterators without the need to use classes or implement the iter and next methods.
  • We can write generators in two different ways. Using generator function or generator comprehension
  • Using generators we can write compact and fast code
  • To get an idea why iterators and generators are handy. An example application of iterators and generators is to read a file line by line without buffering the entire file in memory using few lines of code.

References

  1. anandology.com
  2. nvie.com
  3. wiki.python.org
  4. hackerearth.com
  5. jeffknupp.com
  6. codementor.io
  7. intermediatepythonista.com
  8. blog.pythonlibrary.org
  9. programiz.com
  10. medium.freecodecamp.org

Thanks for reading. For questions and feedback, please use the comments section below.

Tags:

Add a Comment

Your email address will not be published. Required fields are marked *