Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate language detection into a standalone method & Convert test framework into unittest from pytest. #906

Closed
wants to merge 3 commits into from

Conversation

zh-plus
Copy link
Contributor

@zh-plus zh-plus commented Jul 9, 2024

Separate language detection into a standalone method

After the latest update (v1.0.3) to Silero-Vad V5, I noticed that the VAD threshold value is highly dependent on the language. For English, the default threshold=0.5 works well. However, for Asian languages like Chinese and Japanese, a threshold<0.2 should be set; otherwise, VAD may filter out 90% of the audio content.

Thus, I plan to add language detection before transcription, such as:

whisper_model = WhisperModel(model_name, device, compute_type=compute_type)

# Set VAD threshold here according to the detected language
language = ...
vad_options = get_vad_options(language)

seg_gen, info = whisper_model.transcribe(str(audio_path), language=language,
                                         vad_filter=True, vad_parameters=vad_options)

I found that faster-whisper can already detect languages during the whisper_model.transcribe process. However, this feature is embedded and not directly accessible to users. So, I submitted a PR to separate it from the transcribe method, which also reduces its complexity (from 176 to 120 lines of code).

Now we can perform the language detection before transcription by:

whisper_model = WhisperModel(model_name, device, compute_type=compute_type)
lang, lang_prob, all_lang_probs, _ = whisper_model.detect_language(AUDIO_PATH)

Convert test framework into unittest from pytest

While adding related tests, I noticed we are using the pytest framework, which introduces extra dependencies. Thus, I converted all test suites to unittest and updated the dependency requirements.

@zh-plus zh-plus closed this Jul 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant