Python has established itself as the dominant programming language of the modern era, ranking #1 on the TIOBE index as of December 2025 with a commanding 26% market share – more than double its nearest competitor. Simple to use and powerful, Python has become the standard language for data science and many other applications. However, Python has limitations, particularly when it comes to low and ultra-low latency performance for applications like high-frequency trading, real-time AI inference, or industrial control. Usually the solution to these limitations is to build a separate stack in C++ or some other lower level language to handle the ultra-low latency use cases. This results in architectural fragmentation with multiple codebases written in different languages, i.e. a complex C++/Java/Rust stack separate from the high-velocity, productive Python stack. But this is often far from ideal – this duality introduces bugs, deployment friction, and significant maintenance overhead, often requiring two teams with two different skill sets to develop.
Which is why we developed Wingfoil-Python. Wingfoil is an ultra-low latency streaming framework built in Rust. Wingfoil-Python is a Python module that allows you to deliver the blazingly fast, deterministic performance of a native Rust stream processing engine, directly within your familiar Python environment. In other words with Wingfoil-Python, you can still develop in Python, but get all the ultra-low latency benefits of Rust.
To understand how Wingfoil-Python works, it’s also important to understand a little more about Wingfoil, the broader streaming framework.
Wingfoil – some background
Wingfoil is an ultra-low latency, highly-scalable stream processing framework used to receive, process and distribute streaming data. Built in Rust, it’s open source, user friendly and seamlessly supports both real-time and historical workloads. Wingfoil uses a Directed Acyclical Graph (DAG) to efficiently coordinate the execution of its nodes, so application developers implement logic to wire up their graph of calculations which Wingfoil’s graph engine then executes. This helps Wingfoil deal with high scalability of throughput and remain ultra-low latency. Wingfoil comes with near zero-cost abstraction
meaning that performance approaches that of Asynchronous Streams – the
native Rust method for streaming data.
How Wingfoil-Python overcomes Python’s issues with latency
Pure Python applications struggle to meet stringent latency requirements as a consequence of a number of built-in constraints. Wingfoil-Python’s design specifically targets each of these issues:
1. The parallelism bottleneck – the Global Interpreter Lock (GIL)
The GIL restricts all CPU-bound Python threads to executing one at a time. This severely limits throughput and scalability, causing core-starvation in multi-core environments.
Solution: Wingfoil’s core graph execution and stream processing logic are offloaded to its native, multi-threaded Rust engine. This mitigates GIL contention for the most latency-critical workloads, enabling true parallelism and superior throughput.
2. The determinism crisis – unpredictable GC Jitters
The GC process causes execution to reclaim memory, introducing sudden, severe latency spikes, known as “jitters”, that can last for milliseconds. For systems requiring sub-millisecond guarantees, these pauses are catastrophic.
Solution: By leveraging Rust’s deterministic memory management within the high-speed core, Wingfoil is effective at resolving GC-induced latency spikes, ensuring highly predictable and ultra-low latency performance.
3. The efficiency drain – inefficient graph evaluation
Many popular reactive stream processing frameworks (like Rx) rely on a depth-first execution approach. While simple, this can be computationally expensive for complex Data Flow Graphs (DFGs), leading to poor cache locality and an explosive increase in time complexity for many real-world use cases.
Solution: Wingfoil utilises a highly efficient DAG-based engine designed for optimal execution. Its breadth-first execution strategy is demonstrably more efficient and cache-friendly, ensuring a much higher throughput and predictable performance profile compared to common depth-first paradigms.
Wingfoil-Python – features and benefits
Wingfoil-Python has several key features and associated benefits.
Performance and velocity in one stack: developers can build with the velocity of Python and performance of Rust. This means developers can define complex strategies, utilise Python’s vast ML ecosystem, and conduct rapid prototyping and backtesting directly in Python without having to swap to different languages or ecosystems.
Historical and realtime modes: Wingfoil-Python allows developers to seamlessly switch between processing live data and replaying historical data for rigorous backtesting and optimisation.
Simple and obvious API: define complex stream logic with a clear, concise, and easy-to-read API.
Clear upgrade path: Wingfoil provides a clear, supported migration and learning path for Python developers to adopt Rust. By enforcing standardised data flow patterns, Wingfoil encourages the re-use of core graph components during the transition, allowing Python prototypes to be systematically migrated to Rust for ultimate performance without abandoning the initial architectural design.
Use cases: who might use Wingfoil-Python and why?
Wingfoil-Python has a wide range of general use cases for Data Scientists and ML Engineers working in real-time environments where prototype models are built in Python (using libraries like PyTorch or Scikit-learn) but are difficult to deploy into live latency-critical production systems, such as fraud detection pipelines or real-time recommendation engines. As we’ve illustrated, Wingfoil-Python allows developers to define their logic in Python while the heavy lifting of data streaming and graph execution happens in Rust.
More specifically Wingfoil-Python can be used in Electronic Training, by quant developers and engineers to develop complex risk modeling, strategy backtesting and risk models in Python while benefitting from ultra-low latency and high throughput processing of Wingfoil’s Rust streaming framework.
IOT and Real-Time AI/ML Engineers can benefit from the production deployment of streaming models, data pre-processing pipelines. Where Wingfoil delivers fast, deterministic data ingestion and minimal jitter during feature engineering and inference. There are many more potential applications.
Learn more about Wingfoil and Wingfoil python
To learn more or try out Wingfoil and Wingfoil-Python, go to:
• Source Code (GitHub): https://github.com/wingfoil-io/wingfoil/
• Python Module (PyPI): https://pypi.org/project/wingfoil/
• Core Rust Crate: https://crates.io/crates/wingfoil/
Get involved!
Wingfoil and Wingfoil-Python are open source projects and we are always looking for new contributors. If you want to get involved, take a look at Github, and our list of existing issues. We’re just finishing work on a public roadmap that we’ll publish in the New Year, but analytics, monitoring and adapters for Apache Kafka and Zero MQ, and other relevant services are high on the list. And feel free to suggest or create your own tools and solutions.
On a final note, we’ve just released Wingfoil and Wingfoil-Python, so there might be some minor issues, for example, we’re aware that while the underlying Rust Wingfoil crate supports native multithreading for production execution, the direct exposure of this capability through Python bindings is currently Work In Progress. Let us know about any other issues and any other feedback.
