Python Generators and Iterators: Lazy Evaluation Explained

This is the sixth and final post in our Python Basics series. We have covered functions, error handling, and the standard library. Together with the earlier posts on basic syntax and OOP, you now have a solid foundation in Python.

Generators are the topic I have been looking forward to most. They are one of those features that, once you understand them, you start seeing everywhere — and you wonder how you ever managed without them. The idea is simple and beautiful: instead of computing an entire sequence up front and storing it in memory, you compute one value at a time, only when it is actually needed.

Lazy birds, in other words. They do not arrive all at once. They come one by one, when conditions are right.

What Is an Iterator?

Before generators, let us understand iterators, because a generator is a particular kind of iterator.

An iterator is any object that implements two methods: __iter__() (which returns the iterator itself) and __next__() (which returns the next value, or raises StopIteration when there are no more values).

You use iterators constantly without realising it. Every for loop in Python works by calling __next__() on an iterator:

birds = ["Eagle", "Pigeon", "Stork"]

# What a for loop actually does, manually
iterator = iter(birds)
print(next(iterator))  # Eagle
print(next(iterator))  # Pigeon
print(next(iterator))  # Stork

try:
    print(next(iterator))  # raises StopIteration
except StopIteration:
    print("No more birds.")

Eagle
Pigeon
Stork
No more birds.

The for loop does all of this for you invisibly. But knowing the mechanism matters when you write your own iterators and generators.

You can write a full iterator class with __iter__ and __next__ methods. It works, but it is verbose. Generators give you the same behaviour with a fraction of the code.

Generators with yield

🔒 Subscribe to keep reading.

Why Generators Save Memory

🔒 Subscribe to keep reading.

Generator Expressions

🔒 Subscribe to keep reading.

Creating Infinite Sequences with Python Generators

🔒 Subscribe to keep reading.

itertools: Generator Combinators

🔒 Subscribe to keep reading.

Generator Pipeline Pattern: Chaining Multiple Generators

🔒 Subscribe to keep reading.

Generators vs Lists: Key Takeaways

🔒 Subscribe to keep reading.

References

🔒 Subscribe to keep reading.