{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "61d929bd",
   "metadata": {},
   "source": [
    "# Generators"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91a51b21",
   "metadata": {},
   "source": [
    "**Learning goals** — By the end of this section you will be able to:\n",
    "\n",
    "- Write **generator functions** using `yield` to produce sequences lazily\n",
    "- Explain why generators are memory-efficient compared to lists\n",
    "- Create **generator expressions** as a one-line alternative to generator functions\n",
    "- Use generators for **infinite sequences** (e.g., Fibonacci) and large-file processing\n",
    "- Chain generators into efficient **data pipelines**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "197a2fe2",
   "metadata": {},
   "source": [
    "## Generator Functions\n",
    "\n",
    "A **generator function** looks like a regular function but uses `yield` instead of `return`. Each call to `next()` runs the function body until the next `yield`, then pauses — preserving all local variables.\n",
    "\n",
    "```python\n",
    "def my_gen():\n",
    "    yield 1   # pauses here on first next()\n",
    "    yield 2   # pauses here on second next()\n",
    "    yield 3   # pauses here on third next()\n",
    "    # function returns → StopIteration is raised automatically\n",
    "```\n",
    "\n",
    "Key differences from a regular function:\n",
    "- Calling a generator function returns a **generator object** — it does not execute the body\n",
    "- Each `next()` resumes from where it left off\n",
    "- Values are produced **lazily** — only when requested"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9be6a20c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_up_to(max_count):\n",
    "    \"\"\"Yields integers 1, 2, ..., max_count.\"\"\"\n",
    "    n = 1\n",
    "    while n <= max_count:\n",
    "        yield n\n",
    "        n += 1\n",
    "\n",
    "# Calling the function returns a generator object — nothing executes yet\n",
    "gen = count_up_to(5)\n",
    "print(type(gen))          # <class 'generator'>\n",
    "\n",
    "# Values are produced one at a time\n",
    "print(next(gen))          # 1\n",
    "print(next(gen))          # 2\n",
    "\n",
    "# for loops consume the remaining values\n",
    "for val in gen:\n",
    "    print(val, end=' ')   # 3 4 5\n",
    "print()\n",
    "\n",
    "# A fresh generator starts from the beginning\n",
    "print(list(count_up_to(5)))   # [1, 2, 3, 4, 5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62414b74",
   "metadata": {},
   "source": [
    "## Generator Expressions\n",
    "\n",
    "A **generator expression** has the same syntax as a list comprehension, but uses `()` instead of `[]`. The result is a generator, not a list — values are computed on demand.\n",
    "\n",
    "```python\n",
    "# List comprehension — builds the entire list in memory immediately\n",
    "squares_list = [x ** 2 for x in range(1_000_000)]   # uses ~8 MB\n",
    "\n",
    "# Generator expression — produces values one at a time\n",
    "squares_gen  = (x ** 2 for x in range(1_000_000))   # uses ~200 bytes\n",
    "```\n",
    "\n",
    "Use a generator expression whenever you only iterate through the result once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d402d082",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "# Compare memory: list vs generator\n",
    "n = 100_000\n",
    "squares_list = [x ** 2 for x in range(n)]\n",
    "squares_gen  = (x ** 2 for x in range(n))\n",
    "\n",
    "print(f\"List size:      {sys.getsizeof(squares_list):,} bytes\")\n",
    "print(f\"Generator size: {sys.getsizeof(squares_gen):,} bytes\")\n",
    "\n",
    "# Generator expressions compose well with sum(), max(), any(), etc.\n",
    "total      = sum(x ** 2 for x in range(1001))             # no intermediate list\n",
    "first_even = next(x for x in range(1, 100) if x % 7 == 0) # 7\n",
    "print(f\"Sum of squares 0-1000: {total}\")\n",
    "print(f\"First multiple of 7:   {first_even}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63336546",
   "metadata": {},
   "source": [
    "## Practical Use Cases\n",
    "\n",
    "### Infinite sequences\n",
    "\n",
    "Generators can represent sequences that have no end — you just take as many values as you need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4151dc45",
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "\n",
    "def fibonacci():\n",
    "    \"\"\"Infinite Fibonacci sequence generator.\"\"\"\n",
    "    a, b = 0, 1\n",
    "    while True:\n",
    "        yield a\n",
    "        a, b = b, a + b\n",
    "\n",
    "# Take only the values you need — never computes the rest\n",
    "fib = fibonacci()\n",
    "first_10 = list(itertools.islice(fib, 10))\n",
    "print(\"First 10 Fibonacci:\", first_10)  # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]\n",
    "\n",
    "# Find the first Fibonacci number > 1000\n",
    "fib = fibonacci()\n",
    "big = next(f for f in fib if f > 1000)\n",
    "print(\"First Fibonacci > 1000:\", big)   # 1597"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a0e49f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Exercise: Running Total Generator\n",
    "#   1. Write a generator function `running_total(numbers)` that yields the\n",
    "#      cumulative sum after each element.\n",
    "#      Example: running_total([1, 4, 2, 8]) yields 1, 5, 7, 15.\n",
    "#   2. Write a generator expression `even_squares_gen` that yields the squares\n",
    "#      of even numbers from 0 to 19 (i.e., 0, 4, 16, 36, 64, 100, ..., 324).\n",
    "### Your code starts here.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "### Your code ends here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "93fafd97",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Solution\n",
    "def running_total(numbers):\n",
    "    total = 0\n",
    "    for n in numbers:\n",
    "        total += n\n",
    "        yield total\n",
    "\n",
    "print(list(running_total([1, 4, 2, 8])))   # [1, 5, 7, 15]\n",
    "print(list(running_total([10, -3, 7])))    # [10, 7, 14]\n",
    "\n",
    "even_squares_gen = (x ** 2 for x in range(20) if x % 2 == 0)\n",
    "print(list(even_squares_gen))   # [0, 4, 16, 36, 64, 100, 144, 196, 256, 324]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e7c6d3b",
   "metadata": {},
   "source": [
    "## `yield from`\n",
    "\n",
    "`yield from` delegates yielding to another iterable (including another generator).\n",
    "It is essentially a shorthand for `for item in sub: yield item`, but also\n",
    "properly forwards `send()` and `throw()` calls to the sub-generator.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bbf9a04f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def flatten(nested):\n",
    "    \"\"\"Recursively flatten a nested list structure.\"\"\"\n",
    "    for item in nested:\n",
    "        if isinstance(item, list):\n",
    "            yield from flatten(item)  # delegate to recursive call\n",
    "        else:\n",
    "            yield item\n",
    "\n",
    "data = [1, [2, 3], [4, [5, 6]], 7]\n",
    "print(list(flatten(data)))   # [1, 2, 3, 4, 5, 6, 7]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "266e59ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# yield from with multiple sub-generators\n",
    "def combined(*iterables):\n",
    "    for it in iterables:\n",
    "        yield from it\n",
    "\n",
    "result = list(combined([1, 2], 'ab', range(3)))\n",
    "print(result)   # [1, 2, 'a', 'b', 0, 1, 2]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64b94d17",
   "metadata": {},
   "source": [
    "## More `itertools`\n",
    "\n",
    "The `itertools` module provides iterator building-blocks. Beyond `islice` and\n",
    "`chain` (already covered), three more are especially useful in data processing.\n",
    "\n",
    "| Function | Description | Example |\n",
    "|---|---|---|\n",
    "| `count(start, step)` | Infinite counter | `count(1, 2)` → 1, 3, 5, … |\n",
    "| `cycle(iterable)` | Repeats iterable endlessly | `cycle('AB')` → A, B, A, B, … |\n",
    "| `combinations(it, r)` | All r-length combos (no repeat) | `combinations('ABC', 2)` → AB AC BC |\n",
    "| `permutations(it, r)` | All ordered r-length arrangements | `permutations('AB', 2)` → AB BA |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29b5d7c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "\n",
    "# count: infinite arithmetic sequence\n",
    "evens = itertools.islice(itertools.count(0, 2), 5)\n",
    "print('First 5 even numbers:', list(evens))  # [0, 2, 4, 6, 8]\n",
    "\n",
    "# cycle: round-robin scheduling\n",
    "servers = ['A', 'B', 'C']\n",
    "requests = list(itertools.islice(itertools.cycle(servers), 7))\n",
    "print('Request routing:', requests)   # ['A', 'B', 'C', 'A', 'B', 'C', 'A']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe4c6b66",
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "\n",
    "# combinations: choose without caring about order\n",
    "teams = list(itertools.combinations(['Alice', 'Bob', 'Carol'], 2))\n",
    "print('Possible pairs:', teams)\n",
    "# [('Alice', 'Bob'), ('Alice', 'Carol'), ('Bob', 'Carol')]\n",
    "\n",
    "# permutations: order matters (e.g. race positions)\n",
    "podium = list(itertools.permutations(['Gold', 'Silver', 'Bronze'], 2))\n",
    "print(f'Top-2 orderings: {len(podium)} total')\n",
    "print(podium[:3])   # first 3 arrangements\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b836bec3",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Exercise: itertools\n",
    "#   1. Use itertools.count() and islice() to generate the first 8\n",
    "#      multiples of 7 (7, 14, 21, ..., 56).\n",
    "#   2. Use itertools.cycle() and islice() to simulate dealing cards\n",
    "#      to 4 players, 3 cards each (total 12 cards from an infinite deck).\n",
    "#      Players are 'North', 'East', 'South', 'West'.\n",
    "#   3. Use itertools.combinations() to find all 3-letter combinations\n",
    "#      from 'ABCDE' and count them.\n",
    "import itertools\n",
    "\n",
    "### Your code starts here.\n",
    "\n",
    "\n",
    "\n",
    "### Your code ends here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e4648a52",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Solution\n",
    "import itertools\n",
    "\n",
    "# 1. First 8 multiples of 7\n",
    "multiples = list(itertools.islice(itertools.count(7, 7), 8))\n",
    "print(multiples)   # [7, 14, 21, 28, 35, 42, 49, 56]\n",
    "\n",
    "# 2. Deal 3 cards each to 4 players\n",
    "players = itertools.cycle(['North', 'East', 'South', 'West'])\n",
    "cards   = range(1, 13)  # card values 1-12\n",
    "deal    = list(zip(itertools.islice(players, 12), cards))\n",
    "print(deal)\n",
    "# [('North', 1), ('East', 2), ('South', 3), ('West', 4),\n",
    "#  ('North', 5), ('East', 6), ...]\n",
    "\n",
    "# 3. Count 3-letter combinations from 'ABCDE'\n",
    "combos = list(itertools.combinations('ABCDE', 3))\n",
    "print(f'{len(combos)} combinations')   # 10\n",
    "print(combos)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0bcd8836",
   "metadata": {},
   "source": [
    "## Bridge: Async Iteration\n",
    "\n",
    "Synchronous generators pause at `yield` and resume when `next()` is called — all within a single thread. **Async generators** do the same thing, but they can also pause to *wait for I/O* (a network response, a database query, a file read) without blocking other work.\n",
    "\n",
    "The pattern mirrors what you've seen, but uses `async def` with `yield`, and requires `async for` to consume:\n",
    "\n",
    "```python\n",
    "import asyncio\n",
    "\n",
    "async def async_count(n):\n",
    "    \"\"\"Yields 0..n-1, simulating a pause between each.\"\"\"\n",
    "    for i in range(n):\n",
    "        await asyncio.sleep(0)   # yields control to the event loop\n",
    "        yield i\n",
    "\n",
    "async def main():\n",
    "    async for value in async_count(5):\n",
    "        print(value)\n",
    "\n",
    "asyncio.run(main())   # 0 1 2 3 4\n",
    "```\n",
    "\n",
    "**Key vocabulary for future study:**\n",
    "\n",
    "| Term | Meaning |\n",
    "|---|---|\n",
    "| `async def` | Declares a coroutine (or async generator if it uses `yield`) |\n",
    "| `await` | Pauses the coroutine until the awaited task is ready |\n",
    "| `async for` | Iterates over an async iterable; releases control between items |\n",
    "| Event loop | The scheduler that runs coroutines concurrently |\n",
    "\n",
    "You don't need to master `asyncio` now. The takeaway: the iterator protocol you learned here — `__iter__`/`__next__`, generators, `yield` — is exactly the foundation Python's async system is built on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8597e41e",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "| Concept | Key idea |\n",
    "|---|---|\n",
    "| Generator function | Uses `yield`; returns a generator object; body runs lazily |\n",
    "| `yield` | Pauses function, returns a value; resumes on next `next()` call |\n",
    "| Generator expression | `(expr for item in seq if cond)` — lazy, memory-efficient |\n",
    "| Infinite generator | Uses `while True: yield ...`; take values with `itertools.islice` or `next()` |\n",
    "| Memory efficiency | Generators produce one value at a time — constant memory regardless of sequence length |\n",
    "| `itertools.islice` | Takes the first N values from any iterator |"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}