Abstract

Large Language Models (LLMs) have shown strong performance in low-resource offline translation; however, extending them to simultaneous speech-to-speech translation (Simul-S2ST) remains challenging due to lack of causally aligned, speaker-preserving data. In addition, existing approaches rely on fixed translation policy or confidence heuristics, leading to suboptimal quality and higher latency. We propose a causality-aware Simul-S2ST framework with a novel data pipeline that generates high-fidelity, causally aligned segments with voice preservation. The framework further proposes (i) a factorized S2ST architecture (FAST), (ii) a causality-aware adaptive policy (CAP), and (iii) alignment-aware latency metric. Experiments on CVSS (Spanish, German, and French) show that FAST improves translation by up to +9.6 ASR-BLEU over shared codec-based representations, while CAP gains up to +5.5 ASR-BLEU and reduces latency by 26% compared to fixed policy.

Approach

Spanish–English Samples

CVSS-T Source CVSS-T Target Pipeline Target (Ours) FAST-CAP
Es la única de sus obras conservada en su contexto original.
It is the only one of his writings preserved in its original context.
It is the only one of his writings preserved in its original context.
It is the only one of his work preserved in its original content.
Actualmente los antibióticos se han vuelto muy sofisticados.
Presently antibiotics have become very sophisticated.
Presently antibiotics have become very sophisticated.
Currently the antibiotics become highly sophisticated.
Pero ya no puedo estar con un hombre que me ha mentido.
But i cannot be with a man who cheated on me.
But i cannot be with a man who cheated on me.
But i can't be with a man who has lied to me.
A enfermo, niño o anciano, hay que tenderles la mano.
Sick kid or elder you have to lend a hand.
Sick kid or elder you have to lend a hand.
The patient child or old man should hold his hand.

German–English Samples

CVSS-T Source CVSS-T Target Pipeline Target (Ours) FAST-CAP
Anschließend versah er seinen Dienst in verschiedenen Dienststellungen der sowjetischen Seekriegsflotte.
Afterwards he did his duty in different official positions within the soviet navy.
Afterwards he did his duty in different official positions within the soviet navy.
He then provided service in various positions in the soviet naval war fleet.
Daneben entstanden Theaterstücke und andere Werke von regionaler Bedeutung.
Besides that theater plays and other works of regional importance were created.
Besides that theater plays and other works of regional importance were created.
In addition they were created plays and other works of regional importance.
Er ist der einzige Leuchtturm auf Langeneß und auch als Leuchtturm Langeneß bekannt.
It is the only lighthouse on langeness and also known as the lighthouse langeness.
It is the only lighthouse on langeness and also known as the lighthouse langeness.
It is the lighthouse on langen and also known as the langeness lighthouse
Das Museum zeigt Exponate aus hundert Jahren Hochspannungstechnik.
The museum shows exhibits from one hundred years of high voltage technology.
The museum shows exhibits from one hundred years of high voltage technology.
The museum displayed its exhibits from one hundred years high voltage technology.

French–English Samples

CVSS-T Source CVSS-T Target Pipeline Target (Ours) FAST-CAP
Quand il fut devant son cabinet, il ouvrit la porte toute grande.
When he was in front of his cabinet he opened the door wide.
When he was in front of his cabinet he opened the door wide.
When he was in front of his cabinet he opened the door wide.
Actuellement, la présidence est assurée par Jordan Pasco et l'entraîneur général est Laurent Peltier.
The current chairperson is jordan pasco and the general trainer is laurent peltier.
The current chairperson is jordan pasco and the general trainer is laurent peltier.
The president currently provided by jordan pasquale and the coach general is lauren pelletty.
Brunei, alors sous protectorat britannique, l'Indonésie et les Philippines participent pour la première fois.
Brunei at that time under british protectorate as well as indonesia and the philippines took part for the first time.
Brunei at that time under british protectorate as well as indonesia and the philippines took part for the first time.
Brunette was then under british protectorate indonesia and the philippines he participated for the first time.
Une superficie de est sous gestion publique, où le prélèvement est strictement contrôlé.
An area is under public management of which area sampling is strictly controlled.
An area is under public management of which area sampling is strictly controlled.
A surface area is under public management where the supplement is strictly controlled.