Chapter 14

Algorithms

15.0 Intro · 15.1 Algorithm Concepts · 15.2 Searching · 15.3 Sorting

← → or Space to navigate · F for fullscreen

14.1 Algorithm Concepts

Big O · time & space complexity · benchmarking

Big O Notation

How an algorithm's runtime scales with input size n:

Notation Name Example
O(1) Constant Dict lookup, list append
O(log n) Logarithmic Binary search
O(n) Linear List scan, linear search
O(n log n) Linearithmic Merge sort, Timsort
O(n²) Quadratic Bubble sort, nested loops

Always prefer O(1) and O(log n) over O(n) and O(n²) when working with large data.

Benchmarking

import time

# time.perf_counter — wall clock, high precision
start = time.perf_counter()
result = sum(range(1_000_000))
elapsed = time.perf_counter() - start
print(f"{elapsed:.6f}s")
import timeit

# timeit — eliminates startup noise, repeats automatically
t = timeit.timeit(
    "sum(range(1_000_000))",
    number=10
)
print(f"avg: {t/10:.6f}s")

Best / worst / average case

Case Meaning
Best Most favourable input
Worst Most unfavourable input
Average Expected over all inputs

Big O usually refers to the worst case.

Benchmark with realistic data sizes — microbenchmarks on tiny inputs often mislead.

14.2 Searching

Linear · binary · bisect · hash-based

Linear search — O(n)

def linear_search(lst, target):
    for i, item in enumerate(lst):
        if item == target:
            return i
    return -1

Works on any list. Scans every element in the worst case.

Binary search — O(log n)

Requires a sorted list.

def binary_search(lst, target):
    lo, hi = 0, len(lst) - 1
    while lo <= hi:
        mid = (lo + hi) // 2
        if lst[mid] == target:
            return mid
        elif lst[mid] < target:
            lo = mid + 1
        else:
            hi = mid - 1
    return -1

bisect Module & Hash-Based Lookup

bisect — maintain sorted order

import bisect

data = [1, 3, 5, 7, 9]

# Find insertion point
i = bisect.bisect_left(data, 6)   # 3

# Insert maintaining sort order
bisect.insort(data, 6)
print(data)   # [1, 3, 5, 6, 7, 9]

# Lookup: is x in the sorted list?
def sorted_contains(lst, x):
    i = bisect.bisect_left(lst, x)
    return i < len(lst) and lst[i] == x

Hash-based — O(1) average

# Preprocess once
lookup = set(data)     # or dict for key→value

# O(1) membership check
print(6 in lookup)     # True

# Summary
Method Time Requirement
Linear O(n) None
Binary O(log n) Sorted
bisect O(log n) Sorted
Hash O(1) avg Hashable elements

14.3 Sorting

Built-in · insertion · bubble · merge · quicksort

Built-in Sort & Simple Algorithms

Python's Timsort — O(n log n)

lst = [3, 1, 4, 1, 5, 9, 2, 6]
lst.sort()                        # in-place
s = sorted(lst)                   # new list
s = sorted(lst, reverse=True)     # descending
s = sorted(lst, key=abs)          # custom key

Insertion sort — O(n²) worst

def insertion_sort(lst):
    for i in range(1, len(lst)):
        key = lst[i]
        j = i - 1
        while j >= 0 and lst[j] > key:
            lst[j+1] = lst[j]
            j -= 1
        lst[j+1] = key

Merge sort — O(n log n)

def merge_sort(lst):
    if len(lst) <= 1:
        return lst
    mid = len(lst) // 2
    left  = merge_sort(lst[:mid])
    right = merge_sort(lst[mid:])
    return merge(left, right)

def merge(left, right):
    result = []
    i = j = 0
    while i < len(left) and j < len(right):
        if left[i] <= right[j]:
            result.append(left[i]); i += 1
        else:
            result.append(right[j]); j += 1
    return result + left[i:] + right[j:]

Sorting Algorithm Comparison

Algorithm Best Average Worst Space Stable
Timsort (Python) O(n) O(n log n) O(n log n) O(n)
Merge sort O(n log n) O(n log n) O(n log n) O(n)
Quick sort O(n log n) O(n log n) O(n²) O(log n)
Insertion sort O(n) O(n²) O(n²) O(1)
Bubble sort O(n) O(n²) O(n²) O(1)

Use Python's built-in sorted() / .sort() in practice — Timsort is exceptionally well-optimized.

Chapter 14 — Quick Reference

Concept Key syntax / notes
Big O O(1) < O(log n) < O(n) < O(n log n) < O(n²)
Benchmark time.perf_counter(), timeit.timeit()
Linear search O(n) — works on any list
Binary search O(log n) — requires sorted list
bisect bisect_left, bisect_right, insort — O(log n)
Hash lookup x in set_or_dict — O(1) average
Built-in sort lst.sort() / sorted(lst) — Timsort O(n log n)
Stable sort Equal elements keep original order

End of Chapter 14

Congratulations — you've reached the end of ThinkPy!

algorithms · data structures · Python programming