This technical report documents the design, implementation and deployment of a live speech-to- speech translation service for in-person and hybrid events, running on the H2IOSC Kubernetes cluster of ICAR-CNR [1]. A speaker, authenticated by password, talks in a source language (Italian by default); attendees join without any login, select a target language and receive, in near real time, both the translated text and the synthesized audio rendered with a native voice of the chosen language. The service is built as a three-stage cascade – automatic speech recognition (Whisper large-v3 [2]), text translation (Qwen3.6-27B [4]) and text-to-speech (Kokoro-82M [6]) – served through vLLM [3] and glued together by a Node.js/Socket.io application. An end-to-end alternative based on Qwen3- Omni [5] was evaluated and discarded because of GPU memory exhaustion, non-native accents and high latency. The cascade achieves a measured processing latency of roughly 0.5–1 s per speech segment, supports seven languages (IT, EN, FR, ES, PT, JA, ZH) and its computational cost scales with the number of *distinct active languages* rather than with the number of listeners. The service is exposed on the internet through a static MetalLB address [15] behind an HAProxy reverse proxy with an HARICA certificate [16, 17]. Only 5–7 GB of additional GPU memory are needed by the speech components, since the translation model is a shared, pre-existing service of the cluster. The document is organized as follows: Section 2 states the objectives and requirements; Section 3 describes the architecture and the rationale of the cascade; Section 4 lists the technology stack; Section 5 details the Kubernetes deployment and network exposure; Section 6 reports all configurations applied; Section 7 describes operation and access; Section 8 collects the problems encountered and their solutions; Section 9 estimates resource consumption; Section 10 provides concluding remarks and future developments.

Live Speech-to-Speech Translation Service on the H2IOSC Cluster

Luigi Barbato
Primo
Membro del Collaboration Group
;
Francesco Gargiulo
Ultimo
Membro del Collaboration Group
2026

Abstract

This technical report documents the design, implementation and deployment of a live speech-to- speech translation service for in-person and hybrid events, running on the H2IOSC Kubernetes cluster of ICAR-CNR [1]. A speaker, authenticated by password, talks in a source language (Italian by default); attendees join without any login, select a target language and receive, in near real time, both the translated text and the synthesized audio rendered with a native voice of the chosen language. The service is built as a three-stage cascade – automatic speech recognition (Whisper large-v3 [2]), text translation (Qwen3.6-27B [4]) and text-to-speech (Kokoro-82M [6]) – served through vLLM [3] and glued together by a Node.js/Socket.io application. An end-to-end alternative based on Qwen3- Omni [5] was evaluated and discarded because of GPU memory exhaustion, non-native accents and high latency. The cascade achieves a measured processing latency of roughly 0.5–1 s per speech segment, supports seven languages (IT, EN, FR, ES, PT, JA, ZH) and its computational cost scales with the number of *distinct active languages* rather than with the number of listeners. The service is exposed on the internet through a static MetalLB address [15] behind an HAProxy reverse proxy with an HARICA certificate [16, 17]. Only 5–7 GB of additional GPU memory are needed by the speech components, since the translation model is a shared, pre-existing service of the cluster. The document is organized as follows: Section 2 states the objectives and requirements; Section 3 describes the architecture and the rationale of the cascade; Section 4 lists the technology stack; Section 5 details the Kubernetes deployment and network exposure; Section 6 reports all configurations applied; Section 7 describes operation and access; Section 8 collects the problems encountered and their solutions; Section 9 estimates resource consumption; Section 10 provides concluding remarks and future developments.
2026
Istituto di Calcolo e Reti ad Alte Prestazioni - ICAR - Sede Secondaria Napoli
Speech-to-speech translation, live translation, automatic speech recognition, machine translation, text-to-speech, Whisper, Qwen, Kokoro, vLLM, Kubernetes, H2IOSC, GPU cluster, open-weight models, multilingual accessibility, WebSocket
File in questo prodotto:
File Dimensione Formato  
Speech-Translation.pdf

accesso aperto

Tipologia: Documento in Post-print
Licenza: Creative commons
Dimensione 791.21 kB
Formato Adobe PDF
791.21 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.14243/608697
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact