Generators and Iterators with Simple Examples

Posted on:October 14, 2023 at 05:00 PM

generators-iterators-simple-examples Photo by Paul Pastourmatzis Unsplash

Table Of Contents

Open Table Of Contents

Generators

Generators and Iterators may help to produce clean code.

With the help of the generators, we can create an iterable sequence of values on the fly.

Generators may be helpful when we need to work with large data or when we want to generate values only as needed. (i.e. lazily)

Some characteristics of generators include:

Generators are also used in asynchronous programming, coroutines, etc.

Example generator in Javascript:

function* exampleGenerator(initial = 2) {
  let n = initial;
  while (n < 100) {
    yield n * 2;
    n = n * 2;
  }
}
let gen = exampleGenerator(5);
console.log(gen.next());  // { value: 10, done: false }
console.log(gen.next()); // { value: 20, done: false }
console.log(gen.next()); // { value: 40, done: false }
console.log(gen.next()); // { value: 80, done: false }
console.log(gen.next()); // { value: 160, done: false }
console.log(gen.next()); // { value: undefined, done: true }

Example generator in Python:

def test_gen():
	yield 1
	yield 2
gen = test()
next(gen) # 1
next(gen) # 2
next(gen) # Exception: StopIteration

We can send parameters to generators.

For example, in Python we can use the send() function for this purpose:

1def test_gen():
    val = yield 1 # wait for gen.send
    print("val: ", val) # val: None
    yield 2
    yield 3

# Scenario 1 (sending after value yielded)
gen = test_gen()
print(next(gen)) # 1
print(next(gen)) #  2
print(gen.send("abc")) # 3
# Output of scenario 1:
# 1 | val: None | 2 | 3

# Scenario 2 (sending before value yielded)
gen2 = test_gen()
print(next(gen2)) # 1
print(gen2.send("abc")) # val: abc | 2
print(next(gen2)) #  3
# Output of scenario 2:
# 1 | val: abc | 2 | 3

For Javascript, we can put parameters to the next function:

function *generator(){
    let a = yield "First value";
    yield "another " + a;
}


// Scenerio 1 
const gen1 = generator();
console.log(gen1.next()); // { value: 'First value', done: false }
console.log(gen1.next());// { value: 'another undefined', done: false }
console.log(gen1.next()); // { value: undefined, done: true }

// Scenerio 2
const gen2 = generator();
console.log(gen2.next()); // { value: 'First value', done: false }
console.log(gen2.next("example")); // { value: 'another example', done: false }
console.log(gen2.next()); // { value: undefined, done: true }

We can throw exceptions from generators.

function *generator(){
    yield "First value";
    throw new Error("Unexpected error");
    yield "Someone else will fix it";
}

try {
    for (const item of generator()){
        console.log(item);
    }
} catch( err ){
    console.log(err);
}

// First value
// ERROR!: Unexpected error

Iterators

Iterators allow to traverse/iterate over a group of items. For example elements in an array.

Iterators are commonly used to access and process each item in a collection one at a time, without exposing the underlying data structure’s implementation details.

In languages that support iterators, there is often an established iteration protocol or interface.

For example in the Javascript world iterable objects should implement [Symbol.iterator]() function.

interface Iterator {
  next() : IteratorResult;
}
interface IteratorResult {
  value: any;
  done: boolean;
}
interface Iterable {
  [Symbol.iterator]() : Iterator;
}

Example synchronous iterable object in Javascript:

const syncIterable = {
    [Symbol.iterator]() {
        return {
            i: 0,
            next() {
                if (this.i < 6) {
                    return { value: this.i++ * 2, done: false};
                }
                return { done: true }; // iteration is done
            }
        };
    }
};

// We can use `of` operator with iterabls in Javascript
for(const num of syncIterable) {
    console.log(num);
}
// Output: 0 2 4 6 8 10

Example synchronous iterable object in Python:

class MyIterator:
    def __init__(self, data):
        self.data = data
        self.index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.index < len(self.data):
            value = self.data[self.index]
            self.index += 1
            return value
        else:
            raise StopIteration

# Create an iterable object
my_iterable = MyIterator([1, 2, 3, 4, 5])

# Use a for loop to iterate over the elements
for item in my_iterable:
    print(item)

In the C# we can use IEnumerable<> generic for iterables.

Synchronous generators help to create synchronous iterators and asynchronous generators help to create async iterators.

Asynchronous Generators & Iterators

Basic async generator in Javascript:

async function someLongRunningFunc() {
	// Time-bound task... 
    return 2;
}

async function *generator() {
  yield 1;
  // We can use await here
  const result = await someLongRunningFunc();
  yield result;
}

const g = generator();

(async function () {
  console.log(await g.next());// { value: 1, done: false }
  console.log(await g.next());// { value: 2, done: false }
  console.log(await g.next());// { value: undefined, done: true }
})();

Basic async generator in Python:

import asyncio

async def async_generator():
    for i in range(5):
        await asyncio.sleep(1)  # Simulate an asynchronous operation
        yield i

# Using the async generator in an asynchronous loop
async def main():
    async for value in async_generator():
        print(value)

asyncio.run(main())
# 0 .. 1 .. 2 .. 3 .. 4

We can use for await loop with async generators and iterables in Javascript.

async function* gen() {
  yield 1;
  yield 2;
}
 
(async function () {
    for await (const value of gen()) {
        console.log(value);
    }
})()
// Output:
// 1
// 2

We can use yield* keyword for delegation to other generators in Javascript.

async function* gen1() {
  yield 1;
  yield 2;
  return 3; // DONE!
}

async function *gen2(){
    const result = yield* gen1();
    console.log(result); // 3 
}

(async function () {
  for await (const value of gen2()) {
    console.log(value); // 1, 2
  }
})();
// Output:
// 1
// 2
// 3

We can use yield from keyword for delegation in Python.

Async Iterables in Javascript/Typescript

interface AsyncIterable {
  [Symbol.asyncIterator]() : AsyncIterator;
}
interface AsyncIterator {
  next() : Promise<IteratorResult>;
}
interface IteratorResult {
  value: any;
  done: boolean;
}
const asyncIterable = {
  [Symbol.asyncIterator]() {
    return {
      i: 0,
      async next() { // We are returning promise
        if (this.i < 3) {
          return { value: this.i++, done: false };
        }
        return { done: true };
      }
    };
  }
};

(async function() {
  for await (const num of asyncIterable) {
    console.log(num);
  }
})();
// 0
// 1
// 2

We can easily convert synchronous iterator to async iterator with async generators.

1async function* createAsyncIterable(syncIterable) {
  for (const elem of syncIterable) {
    yield elem;
  }
}


(async function() {
    const asyncGen = createAsyncIterable([1, 2]);
    console.log( await asyncGen.next()); // { value: 1, done: false }
    console.log( await asyncGen.next()); // { value: 2, done: false }
    // ...
})();

Practical Example: Image Loading

In the below Javascript example we are loading 10 images lazily every time the user clicks the button.

const domImages = document.getElementById('images');
const button = document.getElementById('button');

async function* getImages() {
  let images = [];
  for (let i = 1; i < 100; i++) {
    const response = await fetch(`https://cdn.mydomain.com/img/${i}`);
    const json = await response.json();
    images.push(json.img);
    
    if (i % 10 === 0) { // Every 10 image
      yield images;
      images = [];
    }
  }
}

// Single instance
const imgGenerator = getImage();

const addImagesToDom = async () => {
  const result = await imgGenerator.next();
  const images = result.value;

  images.forEach(image => {
    domImages.innerHTML += `<img src="${image}"></img>`;
  });
}

button.addEventListener('click', () => {
  addImagesToDom();
});

Thanks for reading.

References