Complete first implementation
This commit is contained in:
10
.idea/.gitignore
generated
vendored
10
.idea/.gitignore
generated
vendored
@@ -1,10 +0,0 @@
|
|||||||
# Default ignored files
|
|
||||||
/shelf/
|
|
||||||
/workspace.xml
|
|
||||||
# Ignored default folder with query files
|
|
||||||
/queries/
|
|
||||||
# Datasource local storage ignored files
|
|
||||||
/dataSources/
|
|
||||||
/dataSources.local.xml
|
|
||||||
# Editor-based HTTP Client requests
|
|
||||||
/httpRequests/
|
|
||||||
17
.idea/MockInterviewer.iml
generated
17
.idea/MockInterviewer.iml
generated
@@ -1,17 +0,0 @@
|
|||||||
<?xml version="1.0" encoding="UTF-8"?>
|
|
||||||
<module type="PYTHON_MODULE" version="4">
|
|
||||||
<component name="NewModuleRootManager">
|
|
||||||
<content url="file://$MODULE_DIR$">
|
|
||||||
<sourceFolder url="file://$MODULE_DIR$/interview-agent/src" isTestSource="false" />
|
|
||||||
</content>
|
|
||||||
<orderEntry type="jdk" jdkName="mock-interviewer WSL (Ubuntu): (null)" jdkType="Python SDK" />
|
|
||||||
<orderEntry type="sourceFolder" forTests="false" />
|
|
||||||
</component>
|
|
||||||
<component name="PyDocumentationSettings">
|
|
||||||
<option name="format" value="PLAIN" />
|
|
||||||
<option name="myDocStringFormat" value="Plain" />
|
|
||||||
</component>
|
|
||||||
<component name="TestRunnerService">
|
|
||||||
<option name="PROJECT_TEST_RUNNER" value="py.test" />
|
|
||||||
</component>
|
|
||||||
</module>
|
|
||||||
6
.idea/inspectionProfiles/profiles_settings.xml
generated
6
.idea/inspectionProfiles/profiles_settings.xml
generated
@@ -1,6 +0,0 @@
|
|||||||
<component name="InspectionProjectProfileManager">
|
|
||||||
<settings>
|
|
||||||
<option name="USE_PROJECT_PROFILE" value="false" />
|
|
||||||
<version value="1.0" />
|
|
||||||
</settings>
|
|
||||||
</component>
|
|
||||||
7
.idea/misc.xml
generated
7
.idea/misc.xml
generated
@@ -1,7 +0,0 @@
|
|||||||
<?xml version="1.0" encoding="UTF-8"?>
|
|
||||||
<project version="4">
|
|
||||||
<component name="Black">
|
|
||||||
<option name="sdkName" value="mock-interviewer WSL (Ubuntu): (null)" />
|
|
||||||
</component>
|
|
||||||
<component name="ProjectRootManager" version="2" project-jdk-name="mock-interviewer WSL (Ubuntu): (null)" project-jdk-type="Python SDK" />
|
|
||||||
</project>
|
|
||||||
8
.idea/modules.xml
generated
8
.idea/modules.xml
generated
@@ -1,8 +0,0 @@
|
|||||||
<?xml version="1.0" encoding="UTF-8"?>
|
|
||||||
<project version="4">
|
|
||||||
<component name="ProjectModuleManager">
|
|
||||||
<modules>
|
|
||||||
<module fileurl="file://$PROJECT_DIR$/.idea/MockInterviewer.iml" filepath="$PROJECT_DIR$/.idea/MockInterviewer.iml" />
|
|
||||||
</modules>
|
|
||||||
</component>
|
|
||||||
</project>
|
|
||||||
7
.idea/vcs.xml
generated
7
.idea/vcs.xml
generated
@@ -1,7 +0,0 @@
|
|||||||
<?xml version="1.0" encoding="UTF-8"?>
|
|
||||||
<project version="4">
|
|
||||||
<component name="VcsDirectoryMappings">
|
|
||||||
<mapping directory="" vcs="Git" />
|
|
||||||
<mapping directory="$PROJECT_DIR$/agents-playground" vcs="Git" />
|
|
||||||
</component>
|
|
||||||
</project>
|
|
||||||
Submodule agents-playground deleted from 2a2d1dbe33
@@ -1,285 +0,0 @@
|
|||||||
# AI Mock Interview Agent Walkthrough
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This project turns the LiveKit Agents Python starter into a realtime voice AI mock interviewer. The agent plays the interviewer, named Jordan, and guides the candidate through two foundational interview stages:
|
|
||||||
|
|
||||||
1. Self-introduction
|
|
||||||
2. Past experience
|
|
||||||
|
|
||||||
The implementation is centered in `src/agent.py`. It uses LiveKit's multi-agent workflow model so each interview stage has its own focused agent, prompt, entry behavior, and transition rules.
|
|
||||||
|
|
||||||
## Architecture at a Glance
|
|
||||||
|
|
||||||
The runtime is one LiveKit `AgentSession[InterviewData]` with shared speech, voice, turn-taking, and LLM configuration. Stage-specific behavior lives in two `Agent` subclasses:
|
|
||||||
|
|
||||||
- `SelfIntroAgent`: opens the interview and asks for the candidate's self-introduction.
|
|
||||||
- `PastExperienceAgent`: asks targeted questions about the candidate's prior work or projects.
|
|
||||||
|
|
||||||
The shared session state is intentionally small:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@dataclass
|
|
||||||
class InterviewData:
|
|
||||||
"""Session-level state for the mock interview."""
|
|
||||||
|
|
||||||
intro_was_given: bool = field(default=False)
|
|
||||||
```
|
|
||||||
|
|
||||||
The production session uses LiveKit Inference for models, prewarmed Silero VAD for speech detection, multilingual turn detection, adaptive interruption handling, preemptive generation, and AI-coustics noise cancellation:
|
|
||||||
|
|
||||||
```python
|
|
||||||
session = AgentSession[InterviewData](
|
|
||||||
llm=inference.LLM(model="openai/gpt-4.1-mini"),
|
|
||||||
stt=inference.STT(model="deepgram/nova-3", language="en"),
|
|
||||||
tts=inference.TTS(
|
|
||||||
model="cartesia/sonic-3",
|
|
||||||
voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc",
|
|
||||||
),
|
|
||||||
vad=ctx.proc.userdata["vad"],
|
|
||||||
turn_handling=TurnHandlingOptions(
|
|
||||||
turn_detection=MultilingualModel(),
|
|
||||||
endpointing={"mode": "dynamic", "min_delay": 0.5, "max_delay": 3.0},
|
|
||||||
interruption={
|
|
||||||
"mode": "adaptive",
|
|
||||||
"min_words": 2,
|
|
||||||
"resume_false_interruption": True,
|
|
||||||
},
|
|
||||||
preemptive_generation={"enabled": True, "preemptive_tts": True},
|
|
||||||
),
|
|
||||||
userdata=InterviewData(),
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Goal One: Two Interview Stages with Clear Logic
|
|
||||||
|
|
||||||
The project avoids one large, general-purpose interviewer prompt. Instead, each stage has a dedicated agent with only the instructions needed for that phase.
|
|
||||||
|
|
||||||
`SelfIntroAgent` is responsible for exactly one job: greet the candidate and collect a self-introduction. Its instructions explicitly tell the model to call `intro_complete` as soon as the candidate gives any substantive introduction:
|
|
||||||
|
|
||||||
```python
|
|
||||||
class SelfIntroAgent(Agent):
|
|
||||||
def __init__(self, *, silence_budget: float = 20.0, stage_budget: float = 60.0) -> None:
|
|
||||||
super().__init__(
|
|
||||||
instructions=textwrap.dedent(
|
|
||||||
"""\
|
|
||||||
You are Jordan, a senior hiring manager...
|
|
||||||
You are conducting the opening stage of a mock job interview, and your
|
|
||||||
sole goal at this stage is to hear the candidate's self-introduction.
|
|
||||||
|
|
||||||
...Once they have shared a description of themselves
|
|
||||||
and their experience — even briefly — call `intro_complete` immediately
|
|
||||||
to advance the interview. Do not ask follow-up questions...
|
|
||||||
"""
|
|
||||||
)
|
|
||||||
+ "\n"
|
|
||||||
+ _VOICE_RULES,
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
When the self-introduction stage begins, the agent generates a short greeting and invitation, then starts both fallback timers:
|
|
||||||
|
|
||||||
```python
|
|
||||||
async def on_enter(self) -> None:
|
|
||||||
await self.session.generate_reply(
|
|
||||||
instructions="Greet the candidate warmly and invite them to introduce themselves."
|
|
||||||
)
|
|
||||||
self._silence_task = asyncio.create_task(self._silence_watchdog())
|
|
||||||
self._stage_task = asyncio.create_task(self._stage_watchdog())
|
|
||||||
```
|
|
||||||
|
|
||||||
`PastExperienceAgent` owns the second stage. It assumes the introduction has already happened and focuses on one meaningful experience at a time:
|
|
||||||
|
|
||||||
```python
|
|
||||||
class PastExperienceAgent(Agent):
|
|
||||||
def __init__(self, chat_ctx: ChatContext | None = None) -> None:
|
|
||||||
super().__init__(
|
|
||||||
instructions=textwrap.dedent(
|
|
||||||
"""\
|
|
||||||
You are Jordan, the same senior hiring manager from the opening stage...
|
|
||||||
You are now conducting the past-experience stage of a mock job interview.
|
|
||||||
|
|
||||||
Pick the most relevant experience the candidate mentioned and ask them
|
|
||||||
to walk you through it: what the challenge was, their specific
|
|
||||||
contribution, the outcome, and what they learned.
|
|
||||||
"""
|
|
||||||
)
|
|
||||||
+ "\n"
|
|
||||||
+ _VOICE_RULES,
|
|
||||||
chat_ctx=chat_ctx,
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
This split keeps each prompt short, reduces unnecessary context, and makes the conversation easier to test.
|
|
||||||
|
|
||||||
## Goal Two: Smooth Stage Transition Without Repetition
|
|
||||||
|
|
||||||
The normal transition is handled by the `intro_complete` function tool. It is called by `SelfIntroAgent` after the candidate gives a self-introduction.
|
|
||||||
|
|
||||||
```python
|
|
||||||
@function_tool
|
|
||||||
async def intro_complete(self, context: RunContext[InterviewData]):
|
|
||||||
"""Call this tool as soon as the candidate has described themselves and
|
|
||||||
their work experience."""
|
|
||||||
context.userdata.intro_was_given = True
|
|
||||||
return (
|
|
||||||
PastExperienceAgent(chat_ctx=self.chat_ctx.copy(exclude_instructions=True)),
|
|
||||||
"Thanks for that introduction. Now I'd like to explore your past experience.",
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
There are three important details here:
|
|
||||||
|
|
||||||
- `context.userdata.intro_was_given = True` records that the normal introduction happened.
|
|
||||||
- `PastExperienceAgent(chat_ctx=self.chat_ctx.copy(exclude_instructions=True))` preserves conversation history without carrying old system instructions into the next agent.
|
|
||||||
- The tool returns only the next agent and one transition line, avoiding competing replies during the handoff.
|
|
||||||
|
|
||||||
When `PastExperienceAgent` enters, it checks the shared session state. If the self-introduction happened normally, it immediately asks about the most relevant experience without reintroducing Jordan or restating the transition:
|
|
||||||
|
|
||||||
```python
|
|
||||||
async def on_enter(self) -> None:
|
|
||||||
if self.session.userdata.intro_was_given:
|
|
||||||
instructions = (
|
|
||||||
"Ask the candidate to walk you through their most relevant past experience "
|
|
||||||
"or project. Do not re-introduce yourself or re-explain the transition — "
|
|
||||||
"go straight to the question."
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
instructions = (
|
|
||||||
"Start directly: ask the candidate to walk you through their most recent "
|
|
||||||
"role or a project they've worked on. Do not reference any prior "
|
|
||||||
"introduction — go straight to the question."
|
|
||||||
)
|
|
||||||
await self.session.generate_reply(instructions=instructions)
|
|
||||||
```
|
|
||||||
|
|
||||||
That fallback branch matters because forced transitions can happen even if the candidate never completed a normal introduction.
|
|
||||||
|
|
||||||
## Goal Three: Time-Based Fallback for Continuous Progress
|
|
||||||
|
|
||||||
The normal path depends on the LLM calling `intro_complete`. To guarantee the interview still moves forward, `SelfIntroAgent` starts two independent watchdogs after its opening prompt:
|
|
||||||
|
|
||||||
- Silence watchdog: moves on after continuous candidate silence.
|
|
||||||
- Stage watchdog: moves on after the total stage time budget is reached.
|
|
||||||
|
|
||||||
The default budgets are:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def __init__(self, *, silence_budget: float = 20.0, stage_budget: float = 60.0) -> None:
|
|
||||||
```
|
|
||||||
|
|
||||||
The silence watchdog resets while LiveKit reports that the user is speaking:
|
|
||||||
|
|
||||||
```python
|
|
||||||
while True:
|
|
||||||
if self.session.user_state == "speaking":
|
|
||||||
last_speech_time = time.monotonic()
|
|
||||||
silence_elapsed = time.monotonic() - last_speech_time
|
|
||||||
if silence_elapsed >= self._silence_budget:
|
|
||||||
break
|
|
||||||
await asyncio.sleep(min(self._silence_budget - silence_elapsed, poll))
|
|
||||||
```
|
|
||||||
|
|
||||||
When a fallback fires, it uses a deterministic spoken transition and then updates the active agent:
|
|
||||||
|
|
||||||
```python
|
|
||||||
if not isinstance(self.session.current_agent, SelfIntroAgent):
|
|
||||||
return
|
|
||||||
if self._transitioning:
|
|
||||||
return
|
|
||||||
self._transitioning = True
|
|
||||||
|
|
||||||
await self.session.say(
|
|
||||||
"Let's go ahead and move on to your background and experience."
|
|
||||||
)
|
|
||||||
self.session.update_agent(
|
|
||||||
PastExperienceAgent(chat_ctx=self.chat_ctx.copy(exclude_instructions=True))
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
The fallback also protects against double transitions. Each watchdog checks whether `SelfIntroAgent` is still active, waits for any in-flight speech to finish, checks again, and then uses `_transitioning` as a final guard.
|
|
||||||
|
|
||||||
On normal handoff or fallback handoff, `on_exit` cancels both watchdog tasks:
|
|
||||||
|
|
||||||
```python
|
|
||||||
async def on_exit(self) -> None:
|
|
||||||
for task in (self._silence_task, self._stage_task):
|
|
||||||
if task is not None:
|
|
||||||
task.cancel()
|
|
||||||
self._silence_task = None
|
|
||||||
self._stage_task = None
|
|
||||||
```
|
|
||||||
|
|
||||||
This prevents the old stage from firing a second transition after control has already moved to `PastExperienceAgent`.
|
|
||||||
|
|
||||||
## Options Considered but Not Used
|
|
||||||
|
|
||||||
| Option | Why We Did Not Use It |
|
|
||||||
| --- | --- |
|
|
||||||
| One monolithic interviewer agent | It would mix stage instructions, increase prompt scope, and make transitions less reliable. |
|
|
||||||
| LLM-only transition logic | If the model fails to call the handoff tool, the interview can stall. The watchdog fallback guarantees progress. |
|
|
||||||
| `generate_reply` for forced transitions | Fallback movement needs deterministic wording. `session.say(...)` avoids wandering or repeated explanations. |
|
|
||||||
| Dropping chat history on handoff | The past-experience stage needs the candidate's introduction. Without copied `chat_ctx`, the candidate may have to repeat themselves. |
|
|
||||||
| Direct provider plugins for every model | LiveKit Inference keeps model routing under LiveKit Cloud credentials and avoids separate provider-key setup for the demo. |
|
|
||||||
| Extra interview stages now | The current roadmap limits scope to the foundational two-stage demo. More stages can be added later with the same handoff pattern. |
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
The test suite in `tests/test_agent.py` covers the core behaviors:
|
|
||||||
|
|
||||||
- `test_self_intro_greeting`: verifies the opening agent greets and asks for an introduction.
|
|
||||||
- `test_intro_complete_handoff`: verifies a complete introduction calls `intro_complete` and transfers to `PastExperienceAgent`.
|
|
||||||
- `test_no_premature_handoff`: verifies a bare greeting does not trigger the handoff.
|
|
||||||
- `test_watchdog_cancelled_on_normal_handoff`: verifies fallback timers are cancelled after normal transition.
|
|
||||||
- `test_fallback_handoff`: verifies the silence fallback moves to `PastExperienceAgent`.
|
|
||||||
- `test_silence_reset_on_speech`: verifies the silence timer resets while the candidate is speaking.
|
|
||||||
|
|
||||||
Latest local verification:
|
|
||||||
|
|
||||||
```console
|
|
||||||
$ uv run pytest -q
|
|
||||||
...... [100%]
|
|
||||||
6 passed in 14.60s
|
|
||||||
```
|
|
||||||
|
|
||||||
## Three-to-Five Minute Video Transcript
|
|
||||||
|
|
||||||
**Opening**
|
|
||||||
|
|
||||||
This project is an AI mock interview agent built on LiveKit's multi-agent framework. The goal is to create a realtime voice interviewer that can guide a candidate through a self-introduction and then move naturally into past-experience questions.
|
|
||||||
|
|
||||||
**Architecture**
|
|
||||||
|
|
||||||
The core implementation lives in `src/agent.py`. We use one LiveKit `AgentSession` to hold the shared LLM, speech-to-text, text-to-speech, voice activity detection, turn detection, and session state. The session state is a small `InterviewData` dataclass that records whether the candidate gave a self-introduction.
|
|
||||||
|
|
||||||
Instead of writing one large interviewer prompt, we split the workflow into two agents. `SelfIntroAgent` handles only the opening stage. `PastExperienceAgent` handles only the experience stage. This keeps each agent focused and helps reduce latency and prompt confusion in a live voice conversation.
|
|
||||||
|
|
||||||
**Stage One**
|
|
||||||
|
|
||||||
When `SelfIntroAgent` enters, it greets the candidate and asks them to introduce themselves. Its prompt tells the model that one real introduction is enough. As soon as the candidate describes who they are and their background or experience, the agent should call the `intro_complete` tool.
|
|
||||||
|
|
||||||
**Normal Transition**
|
|
||||||
|
|
||||||
The `intro_complete` tool is the normal handoff mechanism. It marks `intro_was_given` as true, creates a `PastExperienceAgent`, and passes a copy of the chat context into the next agent. That context transfer is important because the second stage can use the candidate's introduction without asking them to repeat it.
|
|
||||||
|
|
||||||
The tool returns only two things: the next agent and one transition sentence. That keeps the handoff clean and avoids multiple assistant responses competing in the same turn.
|
|
||||||
|
|
||||||
**Stage Two**
|
|
||||||
|
|
||||||
When `PastExperienceAgent` starts, it checks whether the introduction happened normally. If it did, it goes straight into a past-experience question without reintroducing itself. It asks the candidate to walk through a relevant project or role, focusing on the challenge, their contribution, the outcome, and what they learned.
|
|
||||||
|
|
||||||
**Fallback Mechanism**
|
|
||||||
|
|
||||||
The project also includes a time-based fallback so the interview keeps moving even if the LLM does not trigger the normal handoff. After the opening greeting, `SelfIntroAgent` starts two watchdog tasks. One watches for continuous silence. The other enforces a total stage budget.
|
|
||||||
|
|
||||||
If either watchdog fires, the agent waits for any current speech to finish, says a deterministic transition line, and then calls `session.update_agent(...)` to move to `PastExperienceAgent`. The transition passes the same chat context copy, so the second stage still has whatever context is available.
|
|
||||||
|
|
||||||
The watchdogs also include guard checks and a `_transitioning` flag. These prevent duplicate transitions if the normal tool call and fallback timer happen close together. When the self-introduction agent exits, it cancels both watchdog tasks so the old stage cannot interfere with the new one.
|
|
||||||
|
|
||||||
**Verification**
|
|
||||||
|
|
||||||
We verify this behavior with six tests. They cover the greeting, the normal self-introduction handoff, preventing premature handoff, cancelling watchdogs after normal transition, fallback handoff, and resetting the silence timer during speech. The latest test run passed with six out of six tests.
|
|
||||||
|
|
||||||
**Closing**
|
|
||||||
|
|
||||||
In short, the agent achieves the demo goals by combining LiveKit's realtime voice pipeline with a focused multi-agent interview workflow. The self-introduction and past-experience stages are separated cleanly, the normal handoff is explicit and context-preserving, and the fallback timers guarantee that the interview continues even when the LLM-driven transition does not fire.
|
|
||||||
@@ -1,109 +0,0 @@
|
|||||||
Hi, this is Daniel, and I'll walk you through my implementation of the AI Mock Interview demo using the Livekit's multi-agent framework.
|
|
||||||
|
|
||||||
## Introduction
|
|
||||||
|
|
||||||
In this demo, the AI acts as a mock interviewer. The goal is not to build a full interview system yet, but to build a strong foundation for a real-time voice interview flow.
|
|
||||||
|
|
||||||
The current version focuses on two interview stages. The first stage is the self-introduction. The second stage is past experience. The AI interviewer starts by asking the candidate to introduce themselves, and then it moves into questions about the candidate's previous work, projects, or role experience.
|
|
||||||
|
|
||||||
The core code is in `src/agent.py`. This file defines the agents, the stage transition logic, the fallback logic, and the LiveKit session setup.
|
|
||||||
|
|
||||||
## Goal One: Build the Two Interview Stages
|
|
||||||
|
|
||||||
The first goal is to implement the self-introduction stage and the past-experience stage using LiveKit's multi-agent design.
|
|
||||||
|
|
||||||
Instead of using one big prompt for the whole interview, I split the interview into two agents.
|
|
||||||
|
|
||||||
The first one is `SelfIntroAgent`. Its job is very simple. It greets the candidate and asks them to introduce themselves. It does not ask follow-up questions. It does not jump into other topics. Once the candidate gives a real self-introduction, this stage is done.
|
|
||||||
|
|
||||||
The second one is `PastExperienceAgent`. This agent starts after the self-introduction. Its job is to ask about the candidate's most relevant past experience. For example, it may ask the candidate to walk through a project, explain their contribution, describe the result, and share what they learned.
|
|
||||||
|
|
||||||
This multi-agent structure makes the logic much cleaner. Each agent only needs to understand one part of the interview. That also helps with real-time voice performance, because the model does not need to carry one large instruction prompt for every possible stage.
|
|
||||||
|
|
||||||
The session also uses shared LiveKit components, including speech-to-text, text-to-speech, VAD, turn detection, and the LLM. In this implementation, the LLM is set to `gpt-4.1-mini` through LiveKit Inference. The voice pipeline also uses Deepgram for STT, Cartesia for TTS, Silero VAD, LiveKit's multilingual turn detector, and AI-coustics noise cancellation.
|
|
||||||
|
|
||||||
So for the first goal, the key idea is: one shared session, two focused agents, and clear responsibility for each stage.
|
|
||||||
|
|
||||||
## Goal Two: Make the Transition Smooth and Natural
|
|
||||||
|
|
||||||
The second goal is to move from self-introduction to past experience without interruptions, conflicts, or repeated prompts.
|
|
||||||
|
|
||||||
For the normal transition, I use a LiveKit function tool called `intro_complete`. The self-introduction agent is instructed to call this tool as soon as the candidate gives a meaningful introduction.
|
|
||||||
|
|
||||||
When `intro_complete` is called, it does three important things.
|
|
||||||
|
|
||||||
First, it updates the shared session state and marks that the introduction was given.
|
|
||||||
|
|
||||||
Second, it creates the next agent, which is `PastExperienceAgent`.
|
|
||||||
|
|
||||||
Third, it copies the chat context into the next agent. This part is very important. If we do not pass the chat context, the new agent starts fresh and may ask the candidate to repeat information they already shared. By copying the context, the past-experience agent can use the candidate's introduction naturally.
|
|
||||||
|
|
||||||
The handoff also returns only one transition sentence, something like: "Thanks for that introduction. Now I'd like to explore your past experience."
|
|
||||||
|
|
||||||
This keeps the transition clean. There is no extra reply fighting with the handoff. There is no repeated introduction from the interviewer. The next agent simply starts from the right point and asks the next interview question.
|
|
||||||
|
|
||||||
In the `PastExperienceAgent`, the `on_enter` method also checks whether the introduction was completed normally. If it was, the agent goes straight into the past-experience question. If it was not, it still starts directly with a general past-experience question, so the interview can continue.
|
|
||||||
|
|
||||||
So for the second goal, the key idea is: use a clear tool-based handoff, preserve chat context, and make the next agent start without repeating the previous stage.
|
|
||||||
|
|
||||||
## Goal Three: Add a Time-Based Fallback
|
|
||||||
|
|
||||||
The third goal is to make sure the interview always moves forward, even if the normal LLM-based transition does not happen.
|
|
||||||
|
|
||||||
In a voice agent, we cannot fully rely on the model to always call the right tool at the right time. Sometimes the user may stay silent. Sometimes the model may not trigger the handoff. So I added a time-based fallback inside `SelfIntroAgent`.
|
|
||||||
|
|
||||||
There are two watchdog timers.
|
|
||||||
|
|
||||||
The first one is the silence watchdog. It checks how long the candidate has stayed silent. By default, if the user is silent for twenty seconds, the interview moves forward.
|
|
||||||
|
|
||||||
The second one is the stage watchdog. It is a hard time limit for the whole self-introduction stage. By default, after sixty seconds, the interview moves forward even if the normal transition did not fire.
|
|
||||||
|
|
||||||
The silence watchdog is not just a simple countdown. It checks `session.user_state`. If LiveKit detects that the user is speaking, the silence timer resets. This prevents the agent from interrupting while the candidate is talking.
|
|
||||||
|
|
||||||
When a fallback transition happens, the agent uses `session.say(...)` to speak a fixed transition line. I chose this instead of asking the LLM to generate a reply, because the fallback should be predictable. After that, it calls `session.update_agent(...)` and switches to `PastExperienceAgent`.
|
|
||||||
|
|
||||||
There are also guard checks to prevent duplicate transitions. For example, if the normal handoff and a timer happen very close together, the code checks whether `SelfIntroAgent` is still the current agent. It also uses a `_transitioning` flag, so only one transition can win.
|
|
||||||
|
|
||||||
Finally, when `SelfIntroAgent` exits, it cancels both watchdog tasks. This is important because the old timers should not keep running after the interview has already moved to the next stage.
|
|
||||||
|
|
||||||
So for the third goal, the key idea is: normal handoff first, but fallback timers make sure the interview never gets stuck.
|
|
||||||
|
|
||||||
## Real-Time Demo
|
|
||||||
|
|
||||||
Now I will show the real-time demo.
|
|
||||||
|
|
||||||
First, I start the agent from the terminal. The app loads the LiveKit session and waits for a user to join.
|
|
||||||
|
|
||||||
When the candidate connects, the interviewer starts the self-introduction stage. The agent greets the candidate and asks them to introduce themselves.
|
|
||||||
|
|
||||||
For the normal path, I answer with a short self-introduction. For example, I might say: "Hi, I'm Daniel. I have experience building AI applications and backend systems, and recently I worked on a voice AI interview project."
|
|
||||||
|
|
||||||
After that, the agent should call `intro_complete`. It then moves to the second stage and asks about my past experience. The transition should feel natural. It should not ask me to introduce myself again.
|
|
||||||
|
|
||||||
Next, I can also test the fallback path. In the self-introduction stage, I stay silent. After the silence budget is reached, the agent says a fixed transition line and moves into the past-experience stage anyway.
|
|
||||||
|
|
||||||
This shows that the workflow does not depend only on the LLM. If the user is silent, or if the normal tool call does not happen, the interview can still continue.
|
|
||||||
|
|
||||||
I also verified the behavior with tests. The test suite checks the greeting, the normal handoff, no early handoff, watchdog cancellation, fallback handoff, and silence timer reset. The latest run passed with all six tests.
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
There are two main next steps I want to work on.
|
|
||||||
|
|
||||||
The first one is to revise the prompts for smaller models like `gpt-4.1-mini`. Right now, the interviewer has a role name, Jordan. With a smaller model and limited context, this can sometimes be confusing. For example, if the user stays silent during the introduction stage, the model may accidentally treat "Jordan" as the user's name, even though Jordan is the interviewer. So I want to make the prompt more explicit and safer for small models.
|
|
||||||
|
|
||||||
The second next step is to extract important hyperparameters and prompts from `agent.py`. Right now, things like the agent persona, stage timing, and interview style are written directly in the code. In the future, I want to make them configurable.
|
|
||||||
|
|
||||||
That would let us support different interview modes. For a resume-based interview, the interviewer can be more patient and friendly, and the questions can follow the candidate's resume more closely. For a stress interview, the interviewer can be less patient and ask sharper questions in a shorter time.
|
|
||||||
|
|
||||||
This would make the project easier to extend without rewriting the core agent logic.
|
|
||||||
|
|
||||||
## Closing
|
|
||||||
|
|
||||||
To summarize, this demo uses LiveKit's multi-agent framework to build a two-stage AI mock interview.
|
|
||||||
|
|
||||||
The first stage collects the candidate's self-introduction. The second stage asks about past experience. The normal transition uses a tool-based handoff and keeps the chat context, so the conversation feels smooth. The fallback timers make sure the interview keeps moving even if the normal transition does not happen.
|
|
||||||
|
|
||||||
This gives us a stable base for a larger interview system. From here, we can improve prompts, support different interviewer personas, add resume-based interviews, and build more interview types on top of the same workflow.
|
|
||||||
|
|
||||||
That's the walkthrough of my AI Mock Interview demo. Thank you for watching.
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
Hi, this is Danyang, and I'll walk you through my implementation of the AI Mock Interview demo using the LiveKit's multi-agent framework.
|
|
||||||
|
|
||||||
This demo covers two stages: self-introduction and past experience. The core implementation is inside the agent.py file. I'll briefly explain the architecture, the transition and fallback logic. Then I'll show a live demo.
|
|
||||||
|
|
||||||
For the first goal, I used a multi-agent structure instead of one large prompt. The first agent is SelfIntroAgent. Its only job is to greet the candidate and ask for a self-introduction. It does not ask follow-up questions or move into other topics. Once the user gives a real introduction, it passes control to the next agent. The second agent is PastExperienceAgent. This agent asks about the candidate's previous work experience, such as projects, role experience, or contributions mentioned in the first stage.
|
|
||||||
|
|
||||||
This structure keeps the logic clean. Each agent focuses on one stage, so the model does not need to manage a long instruction set. It also helps real-time voice performance because the prompts are smaller and more focused. The whole system still runs in one shared LiveKit session, with LiveKit Inference, gpt-4.1-mini, speech-to-text, text-to-speech, VAD, turn detection, and shared interview state.
|
|
||||||
|
|
||||||
One important detail is that VAD and dynamic endpointing do not decide whether the first stage is complete. They help detect speech and turn boundaries. The actual stage completion is decided by the intro_complete tool. The fallback timers are only there to guarantee progress if the normal transition does not happen.
|
|
||||||
|
|
||||||
For the second goal, I focused on a smooth transition between stages. The normal transition uses a LiveKit function tool called intro_complete. When the candidate finishes their introduction, SelfIntroAgent calls this tool.
|
|
||||||
|
|
||||||
The tool does three things. First, it marks the introduction as complete in the session state. It then creates the PastExperienceAgent. And finally copies the chat context into the next agent. By passing the context, the second agent knows what the candidate already said, so it does not ask the candidate to repeat the same information again. The handoff also returns only one transition sentence. This keeps the transition clean and avoids overlapping responses.
|
|
||||||
|
|
||||||
For the third goal, I added a time-based fallback inside SelfIntroAgent. This handles cases where the user stays silent, or the model does not call the transition tool.
|
|
||||||
|
|
||||||
There are two watchdog timers. The silence watchdog moves forward if the user stays silent for twenty seconds. The stage watchdog moves forward if the self-introduction stage reaches sixty seconds. The silence timer checks LiveKit's user_state, so if the candidate starts speaking, the timer resets and the agent does not interrupt in the middle of an answer.
|
|
||||||
|
|
||||||
When a fallback triggers, the agent uses a fixed transition phrase, switches to PastExperienceAgent, and cancels the old timers. I also added guard checks so if the normal tool call and a timer happen almost at the same time, only one transition happens.
|
|
||||||
|
|
||||||
Now let's see the demo. There are three cases.
|
|
||||||
|
|
||||||
First, I stay silent during the self-introduction stage. After about twenty seconds, the silence fallback moves the interview forward.
|
|
||||||
|
|
||||||
Second, I give a normal self-introduction. The agent calls intro_complete and hands off to PastExperienceAgent without asking me to repeat myself.
|
|
||||||
|
|
||||||
Third, I give a long answer for more than sixty seconds. The stage watchdog should still move the interview forward, but it waits briefly so the agent does not cut in immediately while I am talking.
|
|
||||||
|
|
||||||
[demo playing for about thirty seconds]
|
|
||||||
|
|
||||||
Looking ahead, I have four next steps.
|
|
||||||
|
|
||||||
First, I want to refine the prompts for smaller models like gpt-4.1-mini. Right now, the interviewer has the name Jordan. If the user stays silent, the model may confuse Jordan as the user's name, so the prompt should make the interviewer identity more explicit.
|
|
||||||
|
|
||||||
Second, I want to extract important hyperparameters and prompts from agent.py. This will make it easier to configure different interview personas and interview types, like a friendly resume-based interview or a more stressful interview.
|
|
||||||
|
|
||||||
Third, I want to test LiveKit's newer audio turn detector and compare it with the current multilingual turn detector setup.
|
|
||||||
|
|
||||||
Finally, I want to finish the configuration for PastExperienceAgent, because right now it can keep asking follow-up questions without a clear stopping point.
|
|
||||||
|
|
||||||
To wrap up, this multi-agent setup gives us a stable foundation for an AI mock interviewer. It separates the interview into clear stages, uses a clean tool-based handoff, and adds fallback timers to keep the conversation moving. From here, we can extend the same structure to more interview types and more complete workflows. And thanks for watching.
|
|
||||||
Reference in New Issue
Block a user