[ KUULA ]

[ Focus ]

Reliable conversion of Estonian speech into text, local use, low computational cost and a clear interface to other systems.

[ Status ]

The project is in development. The work ahead is focused on putting the training data in order, choosing and training a base model, the evaluation methodology, and finding an architecture suited to local use.

[ The beginning ]

KUULA grows out of the work on HÄÄL.

HÄÄL solves the output side of the system — turning text into speech. The aim of KUULA is to solve the input side of the same system: to turn Estonian speech into text in a way that can be used both in a standalone application and as the input of the next processing layer.

Together, the two projects form the two main parts of a speech interface:

A language model or other application logic can be placed between them, according to the particular use case.

[ The Estonian language ]

Besides the architecture of the model, the quality of speech recognition depends heavily on the language and on the training data.

For Estonian, the model has to cope with case and verb forms, compound words, names, numbers, loanwords, different speaking rates and pronunciation variants.

The aim is therefore not simply to take a general-purpose speech-to-text model and label it Estonian. During development it has to be assessed which tokenizer, which structure of the training material and which model size give Estonian the best balance between quality and computational cost.

[ Data ]

One of the central parts of KUULA's development is preparing an Estonian speech dataset.

The training material has to contain different speakers, speaking rates, recording environments and speaking styles. Besides the audio, it needs transcriptions that are as accurate as possible and a clear correspondence between audio and text.

The corpora used have to be made consistent before training:

• audio format;
• sample rate;
• the format of the transcriptions;
• punctuation;
• the form of numbers and abbreviations;
• removal of faulty or incomplete segments;
• detection of audio that is too quiet, noisy or broken.

The same dataset is not automatically suitable for training, validation and evaluation alike. Separate splits are created for these.

[ The base model ]

The first technical task is to build a reliable Estonian base model against which further experiments can be compared.

At this stage, using existing speech recognition architectures, adapting them, and where needed training a smaller specialised model are compared.

The choice is not made on transcription accuracy alone. It also has to take into account:

model size, memory requirements, speed, local usability and the scope for further adaptation.

The base model remains the reference point for later changes, in the same way as the larger training model does for HÄÄL.

[ Speech is not only text ]

The output of a speech-to-text model is not always text that is ready to use as it is.

Speech recognition has to distinguish or restore, for example:

• sentence boundaries;
• punctuation;
• upper and lower case;
• numbers;
• dates;
• times;
• names;
• technical terms;
• abbreviations.

In some cases it makes sense to let the acoustic model concentrate on recognising the speech, and to do part of the textual tidying in the next processing layer.

For KUULA, speech recognition itself and the normalisation of its output are therefore treated separately.

[ Longer audio ]

Transcribing individual short audio clips is not enough for a practical system.

KUULA also has to be able to process longer recordings, interviews, meetings and continuous speech.

That requires solving:

• segmenting the audio;
• distinguishing silence from speech;
• timestamps for the segments;
• processing a long file incrementally;
• joining the results in the right order.

Processing longer audio has to work without the model needing to load the whole file into memory at once.

[ Real-time speech ]

A separate line of research is processing speech incrementally, that is, streaming.

In that kind of use the system does not wait for the speech to end: it has to start producing text while the person is still speaking.

This sets different requirements from transcribing a recorded file: what matters is not only the accuracy of the final result, but also the latency and how stably a partial transcription turns into final text.

Real-time operation is needed, for example, in conversational interfaces, in assistive technology, and in systems where KUULA works together with HÄÄL or with a language model.

[ Noise and the real world ]

Clean studio audio in the training data does not describe real use in full.

Within certain limits at least, speech recognition also has to cope with:

• background noise;
• different microphones;
• quiet speech;
• different distances from the microphone;
• uneven loudness;
• ordinary recording faults.

The model is therefore evaluated separately on clean audio and on more difficult audio.

The aim is not to hide a weak input behind aggressive audio processing, but to know under which conditions the model works reliably and where additional pre-processing is needed.

[ A smaller model ]

One of KUULA's aims is to avoid a situation where practical Estonian speech recognition always requires a large GPU or an external cloud service.

Once a base model of good quality has been reached, the scope for reducing the model is assessed.

Depending on the architecture, that can be done through distillation, quantisation, or training a smaller model separately.

The comparison considers at least four figures:

transcription accuracy, speed, memory requirements and model size.

The aim is to find the smallest model whose quality is sufficient for the particular use.

[ Evaluation ]

The quality of KUULA cannot be judged simply by whether some individual sentence was written down correctly.

A fixed test set is created for the project and is not used in training.

The test material has to contain, among other things:

• ordinary speech;
• different speakers;
• numbers and dates;
• names;
• loanwords;
• technical terms;
• different speaking rates;
• longer sentences;
• noisier audio.

The main technical measures are the error rates appropriate to speech recognition, supplemented by listening tests and practical use tests.

The purpose of the measurements is to make different model versions comparable on the same basis.

[ Data and rights ]

As with HÄÄL, the origin of the data and the rights to use it are part of the technical work in KUULA as well.

For every corpus used, the following has to be documented:

• the data source;
• the licence;
• the permitted use;
• the terms for using it in training;
• any restrictions on publishing the model or on commercial use.

Alongside public corpora, it is important to increase the share of our own checked material.

That makes it possible to control audio quality, the accuracy of the transcriptions and the composition of the test material, and reduces dependence on data sources whose terms of use may restrict later use of the model.

[ The connecting layer ]

KUULA and HÄÄL do not have to be tied firmly to one particular system.

A clear connecting layer is being developed between them, whose task is to put the text obtained from speech into a form suitable for the next component, and to take that component's output for HÄÄL.

Through it, the middle component can be, for example:

• a local language model;
• an organisation's own model;
• an external language model service;
• a rule-based system;
• other application logic.

TERNARY is not developing a general-purpose LLM of its own in this project. The connecting layer has to make it possible to use whichever model suits the task.

This keeps the STT, language model and TTS layers technically separate from one another.

[ KUULA → LLM → HÄÄL ]

The longer-term goal is for the projects to form a complete speech interface:

KUULA's task here is the input, not the decision logic of the whole system.

This structure allows speech recognition and speech synthesis to be used separately as well. The same KUULA API can, for instance, produce a transcript of a meeting without HÄÄL or an LLM, and HÄÄL can still work as an independent TTS service.

They are joined only where an application needs it.

[ API ]

For practical use, KUULA has to be more than the model.

A stable interface has to be defined for the system, through which it is possible to send:

• an audio file;
• an audio stream;
• language and processing settings;
• additional context where needed.

The output has to be machine-readable and contain at least the recognised text. Where needed, timestamps, segments and other metadata the model produces reliably can be added to it.

The same API has to make the model usable both in a local application and as a server service, without rebuilding the whole system.

[ Where it stands ]

KUULA is the development stage that follows HÄÄL.

The current work is focused on preparing the project's data, architecture and evaluation basis. The next technical stages are:

auditing and preparing the training data; choosing and training a base model; building a fixed Estonian test set; a solution for long audio and segmentation; normalising the output text; measuring local performance; reducing and quantising the model; streaming experiments; the API and the KUULA–LLM–HÄÄL connecting layer.

The aim of KUULA is not only to build one model that writes an audio file out as text.

The aim is to take Estonian speech recognition to a documented and measurable technology layer that can be used on its own and connected to other models and applications.