We present FastFlow-Python, a framework that brings parallelism to Python for stream-processing applications. FastFlow-Python enables developers to build high-throughput, low-latency data-flow networks by instantiating high-level, ready-to-use parallel building blocks. Built on the C++ FastFlow library, it leverages Python bindings via the C/Python API to efficiently manage parallel execution using both subinterpreters and multiprocessing, all abstracted by the framework. We demonstrate the performance benefits of FastFlow-Python through a comparative analysis with a pure Python stream-processing implementation, highlighting its effectiveness in overcoming the limitations imposed by the Global Interpreter Lock (GIL). Experimental results show almost linear scalability when increasing the number of workers.

FastFlow-Python: parallel building blocks in Python through FastFlow integration

Massa J.;
2025

Abstract

We present FastFlow-Python, a framework that brings parallelism to Python for stream-processing applications. FastFlow-Python enables developers to build high-throughput, low-latency data-flow networks by instantiating high-level, ready-to-use parallel building blocks. Built on the C++ FastFlow library, it leverages Python bindings via the C/Python API to efficiently manage parallel execution using both subinterpreters and multiprocessing, all abstracted by the framework. We demonstrate the performance benefits of FastFlow-Python through a comparative analysis with a pure Python stream-processing implementation, highlighting its effectiveness in overcoming the limitations imposed by the Global Interpreter Lock (GIL). Experimental results show almost linear scalability when increasing the number of workers.
2025
Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" - ISTI
979-8-3315-3465-3
Algorithmic skeletons
Parallel programming
Stream processing
File in questo prodotto:
File Dimensione Formato  
DellaBartola et al_FastFlow-Python_Parallel_IC2E_2025.pdf

solo utenti autorizzati

Descrizione: FastFlow-Python: Parallel Building Blocks in Python Through FastFlow Integration
Tipologia: Versione Editoriale (PDF)
Licenza: NON PUBBLICO - Accesso privato/ristretto
Dimensione 584.32 kB
Formato Adobe PDF
584.32 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.14243/563284
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact