Building Voice-Based Agentic Systems

Key Takeaways

• Apply foundational speech processing and recognition techniques to convert spoken audio into accurate textual representations, and interpret system outputs to evaluate performance, limitations, and real-world robustness.
• Design and implement speech generation pipelines with awareness of practical considerations related to naturalness, latency, multilingual support, and computational efficiency.
• Integrate speech, language, and visual understanding components, such as automatic speech recognition, machine translation, optical character recognition (OCR), and large language models, to construct integrated voice-based agent systems capable of handling multimodal inputs.
• Leverage large language models and multimodal models to design and implement spoken and multimodal dialogue systems, while evaluating architectural choices and their implications on scalability, reliability, and user experience.
• Develop integrated voice-based agent applications through hands-on case studies by translating real-world problem statements into executable multimodal agent pipelines, while identifying and mitigating component-specific risks such as recognition errors, latency, and system integration failures.

Who Should Attend

• AI engineers and developers building speech-enabled or conversational systems
• Solution architects and technical leads responsible for voice or multimodal AI applications
• Conversational AI practitioners seeking to extend from text-based assistants into voice workflows
• Technically inclined professionals interested in building real-world voice-enabled assistants for enterprise or service use cases

Pre-requisites

• Able to program and debug independently using Python
• Well-versed with Anaconda, Jupyter Notebooks, Google Colab and Github
• Basic knowledge of machine learning concepts (at the level of Problem Solving using Pattern Recognition course offered by NUS-ISS)
• Completion of Text Processing Using Deep Learning and Building Multimodal Virtual Assistants, or equivalent background in language AI, agents, and assistant systems, is strongly recommended

What to Bring

No printed copies of course materials are issued.
Participants must bring their internet-enabled computing device (laptops, tablet etc) with power charger to access and download course materials.

If you are bringing a laptop, please see below for the tech specs:

What Will Be Covered

• Speech processing basics
• Speech recognition and machine translation
• Integrating speech recognition with language processing
• Speech synthesis
• Voice conversion and audio generation
• OCR for voice-based agents
• Spoken and multimodal dialogue systems
• Building integrated voice-agent systems

Fees & Subsidies

1. SSG Funding is available to qualified individuals, subject to meeting the attendance requirement and passing of assessment.
2. All self-sponsored Singaporeans aged 25 and above can use their SkillsFuture Credit to pay for course fees. For more information about SkillsFuture Credit, click here.
3. SME fees are applicable only to participants who are sponsored by small and medium enterprises.
4. SSG funding is subjected to availability.

Certificate

Preparing for Your Course

NUS-ISS Course Registration Terms and Conditions

Find out more.

NUS-ISS and Learner’s Commitment and Responsibilities

Find out more.

WIFI Access

WIFI access will be made available to participants.

Venue

NUS-ISS
25 Heng Mui Keng Terrace
Singapore 119615

Click HERE for directions to NUS-ISS

In the event of a change of venue, participants are advised to refer to the acceptance email sent one week prior to the commencement date.

Course Confirmation

All classes are subject to confirmation and NUS-ISS will send an acceptance email to participants one week prior to the commencement date. Confirmed registrants are to attend and complete all lectures, class exercises, workshops and assessments (where applicable). Additionally, all responses to feedbacks and surveys conducted by NUS-ISS and its partners must be submitted. All training and assessments will be delivered as described in the course webpage.

General Enquiry

Please feel free to write to ask-iss@nus.edu.sg if you have any enquiry or feedback.