Clever Geek Handbook

Does CPython include the mentioned optimizations from PEP 380?

PEP 380 mentions that the syntax yield from expr

can be optimized in Python.

PEP 380 - Optimization

The use of specialized syntax opens up opportunities for optimization when there is a long chain of generators. Such chains can arise, for example, during the recursive movement of a tree structure. The overhead of passing calls __next__()

and values ​​given down and up the chain can cause what should be an O (n) operation to become, in the worst case, O (n ** 2).

A possible strategy is to add a slot to generator objects to allow for generator delegation. When calls to __next__()

or are made in the generator send()

, this slot is first checked, and if it is not empty, the generator it referenced is resumed instead. If it rises StopIteration

, the slot is cleared and the main generator resumes.

This will reduce the redundancy of delegation to chaining C function calls that do not involve executing Python code. A possible reinforcement would be to loop through the entire generator chain and directly resume at the end, although the processing is StopIteration

more complicated.

Does CPython use this optimization?

+3


source to share


1 answer


Does not look like. As of Python 3.6, the structure of a generator structure does not have a suggested field, and the code path for yield from

through the chain of generators always goes through the process of resuming their Python stack frames individually. (This code path goes from the opcode through / to , and from there to and on to the next generator in the chain.) YIELD_FROM

_PyGen_Send

gen_iternext

gen_send_ex

PyEval_EvalFrameEx

YIELD_FROM



+2


source






More articles:


All Articles