Building Multimodal Virtual Assistants

Key Takeaways

• Understand the fundamentals of modern virtual assistants - the key concepts, types, applications and issues shaping their development
• Build single-turn and retrieval-augmented question-answering systems that form the foundation of virtual assistants
• Develop multi-turn conversational assistants with effective memory and dialogue management
• Integrate language and vision models to create multimodal assistants capable of interpreting text and images for richer contextual interactions
• Innovate by combining advanced multimodal techniques to design and prototype intelligent multimodal virtual assistants through hands-on workshops and projects

Who Should Attend

• Software developers and GenAI application developers building assistant-based solutions
• Solution architects and technical leads responsible for conversational or multimodal AI systems
• AI practitioners and product technologists interested in retrieval, dialogue, and multimodal workflows
• Technically inclined professionals seeking to build enterprise assistants for knowledge access, customer support, or operational automation

Pre-requisites

• Able to program and debug independently using Python.
• Well-versed with Anaconda, Jupyter Notebooks, Google Colab and Github
• Basic knowledge of machine learning concepts (at the level of Problem Solving using Pattern Recognition course offered by NUS-ISS)
• Completion of Text Processing Using Deep Learning, or equivalent background in NLP/LLM fundamentals, is strongly recommended

What to Bring

No printed copies of course materials are issued.
Participants must bring their internet-enabled computing device (laptops, tablet etc) with power charger to access and download course materials.

If you are bringing a laptop, please see below for the tech specs:

What Will Be Covered

• Introduction to virtual assistants
• Single-turn question answering and retrieval-augmented generation (RAG)
• Multi-turn conversational assistants with memory and dialogue management
• Vision understanding for virtual assistants
• Multi-modality in practice
• Hands-on multimodal virtual assistant project

Fees & Subsidies

1. All fees and subsidies are valid from January 2024, unless otherwise advised.
2. All self-sponsored Singaporeans aged 25 and above can use their SkillsFuture Credit to pay for course fees. For more information about SkillsFuture Credit, click here.
3. From 1st January 2024, the GST will be increased to 9%.

Certificate

Preparing for Your Course

NUS-ISS Course Registration Terms and Conditions

Find out more.

NUS-ISS and Learner’s Commitment and Responsibilities

Find out more.

WIFI Access

WIFI access will be made available to participants.

Venue

NUS-ISS
25 Heng Mui Keng Terrace
Singapore 119615

Click HERE for directions to NUS-ISS

In the event of a change of venue, participants are advised to refer to the acceptance email sent one week prior to the commencement date.

Course Confirmation

All classes are subject to confirmation and NUS-ISS will send an acceptance email to participants one week prior to the commencement date. Confirmed registrants are to attend and complete all lectures, class exercises, workshops and assessments (where applicable). Additionally, all responses to feedbacks and surveys conducted by NUS-ISS and its partners must be submitted. All training and assessments will be delivered as described in the course webpage.

General Enquiry

Please feel free to write to ask-iss@nus.edu.sg if you have any enquiry or feedback.