PythoC: A new way to generate C code from Python

PythoC: A New Way to Generate C Code from Python

Python and C share more than it might seem. The reference version of the Python interpreter is written in C, and many of the third-party libraries written for Python wrap C code. It’s also possible to generate C code from Python.

Generating C code with Python has typically involved libraries like Cython, which use type-annotated Python code to generate C extension modules for Python.

A new project, PythoC, takes a different approach. It uses type-hinted Python to programmatically generate C code—but chiefly for standalone use, and with many more compile-time code generation features than Cython has.

PythoC’s makers use the phrase “C level runtime, Python powered compile time” to describe their approach. The project is still in its early phases, but there are enough working features to make it worth a look.

A Basic PythoC Program

Here’s a simple program adapted from PythoC’s examples:

from pythoc import compile, i32

@compile
def add(x: i32, y: i32) -> i32:
    return x + y

if __name__ == "__main__":
    print(add(10, 20))

To indicate which functions in a module to compile to C, you use PythoC’s @compile decorator, supplying type hints for the result and each parameter. Note that you also need to import PythoC’s own i32 hint, instead of using Python’s native int. This means you’re using machine-native integers and not Python’s arbitrary-size ints.

When you run this program, you’ll get 30 as the output, after a delay. The C code is compiled on the fly each time you execute the program, hence the delay. PythoC doesn’t yet have a mechanism for re-using compiled code when it’s called from Python, the way Cython does.

Generating Standalone C Programs

Here’s a new version of the same program, with different behaviors.

from pythoc import compile, i32, ptr, i8
from pythoc.libc.stdio import printf

@compile
def add(x: i32, y: i32) -> i32:
    return x + y

@compile
def main(argc: i32, argv: ptr[ptr[i8]]) -> i32:
    printf("%un", add(10, 20))

if __name__ == "__main__":
    from pythoc import compile_to_executable
    compile_to_executable()

PythoC’s Emulation of C Features

With only a few exceptions, PythoC can generate code that fully utilizes C’s feature set and runtime.

You’ve already seen how to use type annotations to indicate primitive data types. You can likewise use the ptr[T] annotation to describe a pointer (also shown above), and use array[T, N] for N-dimensional arrays of type T. You can make structs, unions, and enums by decorating Python classes, and all the usual operators and control-flow operations (except for goto) will work. For switch/case, just use match/case, although fall-through cases aren’t available.

Compile-Time Code Generation

It’s possible to use Cython for compile-time code generation, which means you can produce different kinds of C code, or even fall back to Python code, depending on what happens at compile time. But PythoC’s compile-time code generation has abilities Cython lacks.

Here’s an example from PythoC’s documentation:

from pythoc import compile, struct, i32, f64

def make_point(T):
    @struct(suffix=T)
    class Point:
        x: T
        y: T

    @compile(suffix=T)
    def add_points(p1: Point, p2: Point) -> Point:
        result: Point = Point()
        result.x = p1.x + p2.x
        result.y = p1.y + p2.y
        return result

    return Point, add_points

Point_i32, add_i32 = make_point(i32)
Point_f64, add_f64 = make_point(f64)

Memory Safety Features

The bugs that can spring from C’s manual memory management are gloomily familiar to anyone who uses the language. Cython has memory safety features to address this, but PythoC offers unique type-based features in this vein.

One is a feature called linear types. The linear import lets you generate a “proof,” usually to accompany a memory allocation, that has to be “consumed” when the same memory is deallocated. If you don’t have a matching consume(prf) for each prf=linear(), the PythoC compiler will generate a compile-time error.

Possible Future Directions for PythoC

PythoC is still quite new, so its future development is relatively open ended. One possibility is that it could integrate more closely with Python at runtime. For instance, a @cached decorator could compile modules once, ahead of time, and then re-use the compiled modules when they’re called from within Python, instead of being recompiled at each run.

The potential for PythoC to revolutionize the way we think about C code generation is enormous. Whether or not it achieves its goal of becoming the standard tool for generating C code from Python remains to be seen, but it’s certainly an exciting development to watch.

For those interested in exploring PythoC further, its documentation and GitHub repository are available online. As with any new tool, there are likely to be kinks to work out along the way, but the potential benefits of using PythoC make it well worth considering for any project that requires the generation of C code.

Learn more about PythoC’s unique approach to C code generation.