So OpenAI researchers created a speech recognition tool called Whisper. It could transcribe the audio from YouTube videos, yielding new conversational text that would make an A.I. system smarter.

Combining audio, images, and text helps the model better understand speech context. To improve its performance, we fine-tune a strong language model by blending unsupervised learning with multimodal ...