Interpreter in the Wild: Python's ast Module

Python parses 1 + 2 * 3 into BinOp and Constant nodes, the same tree drawn by hand.

Python represents its own source code as a tree of node classes, one per rule of its grammar, which is the representation half of the Interpreter pattern. The ast module exposes that tree. This card prints the tree Python builds for 1 + 2 * 3, and maps its nodes onto the pattern.s participants.

import ast
import sys

tree = ast.parse("1 + 2 * 3", mode="eval")
print("Python", sys.version.split()[0])
print(ast.dump(tree.body, indent=2))
print("evaluates to", eval(compile(tree, "<rule>", "eval")))

OutputRan with python:3.12-slim (Python 3.12.13)

Python 3.12.13
BinOp(
  left=Constant(value=1),
  op=Add(),
  right=BinOp(
    left=Constant(value=2),
    op=Mult(),
    right=Constant(value=3)))
evaluates to 7

The node class names and the fields shown depend on the Python version; this ran on 3.12.13.

The tree has the shape a hand-written interpreter would build. BinOp, a binary operation, is the non-terminal expression, holding a left part, an operator and a right part; Constant is the terminal. The multiplication is the inner BinOp, below the addition, because it is evaluated first.

Two things differ from a hand-written interpreter. The nodes have no interpret method: Python compiles the tree to bytecode, instructions for its own virtual machine, and runs that, which is far faster than walking the tree. And ast.parse turns text into the tree, a step the pattern itself leaves out. The output also depends on the version: Python 3.8 replaced the older Num node with Constant, so code that walks these trees names the version it expects.

← → move through sections and cards