27.05. Multi-agent Environment 👾🤖👾¶
📍 Download notebook and session files
In today’l lab, we will create a multi-agent environment for automated meeting scheduling and preparation. We will see how the coordinator agent will communicate with two auxiliary agents to check time availability and prepare an agenda for the meeting.
Our plan for today:
Prerequisites¶
To start with the tutorial, complete the steps Prerequisites, Environment Setup, and Getting API Key from the LLM Inference Guide.
Then install the dependencies:
pip install -r requirements.txt
Then as always, we will initialize our LLM first. This time, we will be using llama-3.1-405b-instruct because ir handles multiple tools better (remember that a newer mode doesn’t necessarily means better model, at least for some tasks).
from langchain_nvidia_ai_endpoints import ChatNVIDIA
from langchain_core.rate_limiters import InMemoryRateLimiter
# read system variables
import os
import dotenv
dotenv.load_dotenv() # that loads the .env file variables into os.environ
True
# choose any model, catalogue is available under https://build.nvidia.com/models
MODEL_NAME = "meta/llama-3.1-405b-instruct"
# this rate limiter will ensure we do not exceed the rate limit
# of 40 RPM given by NVIDIA
rate_limiter = InMemoryRateLimiter(
requests_per_second=2 / 60, # 3 requests per minute (this model allows much less requests)
check_every_n_seconds=5, # wake up every 100 ms to check whether allowed to make a request,
max_bucket_size=1 # controls the maximum burst size
)
llm = ChatNVIDIA(
model=MODEL_NAME,
api_key=os.getenv("NVIDIA_API_KEY"),
temperature=0, # ensure reproducibility,
rate_limiter=rate_limiter # bind the rate limiter
)
1. Creating an Agent 👾
Recap: a raw (textual) LLM can only generate text, is limited to what’s in its training data, and can only give you a single response. An agent overcomes this limitations by attaching tools, memory, and often access to external data sources to the LLM. This means agents can take actions, remember previous interactions, use APIs, and iteratively work through complex problems by deciding what to do next based on intermediate results.
LangGraph thinks about agents in a simple way: they are a construct out of three key components: the LLM itself, the tools, and the looping logic (Observe–Plan–Act workflow). This creates a cycle where the agent can:
Think about the problem
Decide which tool to use
Use the tool and see the result
Think about whether it’s done or needs to do more
Repeat until the task is complete
LangChain comes with a number of prebuilt agentic implementations; they can be considered as templates for the graphs to implement the logic above: you only need to pass the LLM, the tools, and the prompt, and the rest will be made for you.
We will now create a very basic agent that can do simple math operations and count length. This will help us understand the core concepts without getting too deep in complex functionality (yet).
We had initialized the LLM above, so now it’s time for the tools. Remember that LangChain reads out the docstring and the type annotations to determine when to use this tool so don’t forget to include those in your functions.
from langchain_core.tools import tool
@tool
def add_numbers(a: float, b: float) -> float:
"""Add two numbers together."""
return a + b
@tool
def multiply_numbers(a: float, b: float) -> float:
"""Multiply two numbers together."""
return a * b
@tool
def get_word_length(word: str) -> int:
"""Get the length of a word."""
return len(word)
# put together
tools = [add_numbers, multiply_numbers, get_word_length]
To initialize the agent quickly, we use the create_react_agent method from LangGraph; as mentioned before, it takes the LLM, the tools, and the couple of more parameters as the input and returns a fully functional graph with the Observe–Plan–Act workflow.
from langgraph.prebuilt import create_react_agent
_basic_agent_prompt = """
Answer user questions using the provided tools. \
Always plan ahead first, then execute the plan step by step. \
If you face a complex task, break it down into smaller steps. \
You may use only one tool per action. \
Return only the results with no additional text.
"""
basic_agent = create_react_agent(
model=llm,
tools=tools,
prompt=_basic_agent_prompt
)
from IPython.display import Image, display
def draw_graph(agent, output_file_path: None):
# draw the graph
try:
graph_image = agent.get_graph().draw_mermaid_png(
output_file_path=output_file_path,
)
display(Image(graph_image))
except Exception as e:
pass
draw_graph(basic_agent, "basic_agent.png")
# wrapper for pretty print
def run_agent(agent, query, recursion_limit=50):
input = {
"messages": [("user", query)]
}
shown_messages = []
for event in agent.stream(
input,
config={"recursion_limit": recursion_limit},
stream_mode="values"
):
if event["messages"]:
for message in event["messages"]:
if not message.id in shown_messages:
shown_messages.append(message.id)
message.pretty_print()
print("\n")
run_agent(basic_agent, "I need to calculate (523 + 32) * 28 and get the length of the resulting number spelled out.")
================================ Human Message =================================
I need to calculate (523 + 32) * 28 and get the length of the resulting number spelled out.
================================== Ai Message ==================================
Tool Calls:
add_numbers (chatcmpl-tool-79717782d33c483ea2a0484f3511da4b)
Call ID: chatcmpl-tool-79717782d33c483ea2a0484f3511da4b
Args:
a: 523
b: 32
================================= Tool Message =================================
Name: add_numbers
555.0
================================== Ai Message ==================================
Tool Calls:
multiply_numbers (chatcmpl-tool-ed46f917137f4af09c2ab00079b22b88)
Call ID: chatcmpl-tool-ed46f917137f4af09c2ab00079b22b88
Args:
a: 555
b: 28
================================= Tool Message =================================
Name: multiply_numbers
15540.0
================================== Ai Message ==================================
Tool Calls:
get_word_length (chatcmpl-tool-ec58f78d2db1460a85c3148db3f00c68)
Call ID: chatcmpl-tool-ec58f78d2db1460a85c3148db3f00c68
Args:
word: fifteenthousandfivehundredforty
================================= Tool Message =================================
Name: get_word_length
31
================================== Ai Message ==================================
The length of the word is 31.
Please note that this implementation will always try to call tools until the answer is reached, which may be inadequate for some applications.
run_agent(basic_agent, "What is the 5th largest country?")
================================ Human Message =================================
What is the 5th largest country?
================================== Ai Message ==================================
I can't answer that question with the tools I have available.
As you can see, the default create_react_agent implementation is not optimal as it always tries to call a tool. To address this issue, we will insert a so-called pre_model_hook: a function that is called before the LLM. This function is a usual LangGraph node that is just incorporated in the default implementation.
In our case, we will make the LLM first plan the next action explicitly and then generate itself a command for the next action.
from langchain_core.tools import render_text_description
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
tools_description = render_text_description(tools)
_plan_next_instruction = f"""
You are an agent who has access to the following tools:
{tools_description}
Your task is to define best next action based on the previous messages \
and the tools available. You can either call one tool or \
return a text message to the user.
Return the next action you think is best to take as a textual description. \
Only return the single next action and not the whole plan.
Examples:
User: What is 2 + 2?
Assistant: Call the add_numbers tool with arguments 2 and 2.
User: What is the 5th largest country?
Assistant: Generate the answer yourself and exit, no tool calls are needed here.
"""
_plan_next_prompt = ChatPromptTemplate.from_messages(
[
("system", _plan_next_instruction),
MessagesPlaceholder(variable_name="messages")
]
)
def plan_next(state):
plan_next_prompt = _plan_next_prompt.invoke(state)
next_action = llm.invoke(plan_next_prompt)
return {"messages": next_action}
basic_plus_agent = create_react_agent(
model=llm,
tools=tools,
pre_model_hook=plan_next
)
draw_graph(basic_plus_agent, "basic_plus_agent.png")
run_agent(basic_plus_agent, "I need to calculate (523 + 32) * 28 and get the length of the resulting number spelled out.")
================================ Human Message =================================
I need to calculate (523 + 32) * 28 and get the length of the resulting number spelled out.
================================== Ai Message ==================================
Call the add_numbers tool with arguments 523 and 32.
================================== Ai Message ==================================
Tool Calls:
add_numbers (chatcmpl-tool-71cde0654b3f4b64bb80d9032f3ace00)
Call ID: chatcmpl-tool-71cde0654b3f4b64bb80d9032f3ace00
Args:
a: 523
b: 32
================================= Tool Message =================================
Name: add_numbers
555.0
================================== Ai Message ==================================
Call the multiply_numbers tool with arguments 555 and 28.
================================== Ai Message ==================================
Tool Calls:
multiply_numbers (chatcmpl-tool-089a91c23834474ea8902528c58af0af)
Call ID: chatcmpl-tool-089a91c23834474ea8902528c58af0af
Args:
a: 555
b: 28
================================= Tool Message =================================
Name: multiply_numbers
15540.0
================================== Ai Message ==================================
Call the get_word_length tool with the argument "fifteen thousand five hundred forty".
================================== Ai Message ==================================
Tool Calls:
get_word_length (chatcmpl-tool-8f0a26d21b1540a284d8383ece2f27c5)
Call ID: chatcmpl-tool-8f0a26d21b1540a284d8383ece2f27c5
Args:
word: fifteen thousand five hundred forty
================================= Tool Message =================================
Name: get_word_length
35
================================== Ai Message ==================================
Return the answer "35".
================================== Ai Message ==================================
35
run_agent(basic_plus_agent, "When is the apocalypse happening?")
================================ Human Message =================================
When is the apocalypse happening?
================================== Ai Message ==================================
Generate the answer yourself and exit, no tool calls are needed here.
================================== Ai Message ==================================
I'm not aware of any information that suggests the apocalypse is happening on a specific date. The concept of the apocalypse is often associated with religious or mythological beliefs, and its timing is not something that can be predicted with certainty. It's best to focus on living in the present and working towards creating a better future for all, rather than worrying about a hypothetical apocalyptic event.
2. Centralized Multi-agent System 👾🤖👾
In this section, we’ll create a multi-agent system for automated meeting scheduling and preparation. Our system will consist of three specialized agents:
Scheduler Agent - checks the availabilities of required participants and selects time slots that work for everyone
Creative Agent - reviews the project plan and generates a comprehensive agenda for the meeting
Coordinator Agent - orchestrates the other two agents, deciding when to call each one and combining their outputs
This will demonstrate how different agents can collaborate, each with their own expertise, to solve complex problems that would be difficult for a single agent to handle efficiently.
Executor Agents¶
For this tutorial, we’re focusing on understanding the multi-agent workflow and coordination patterns, so to avoid losing time for configuration of actual API integrations and external dependencies, we’ll implement dummy functions that simulate real-world services. We will respectively create a dummy calendar function that will output random hourly time slots, and a dummy RAG function that will simulate the process of retrieving relevant information about the project for agenda preparation.
import random
@tool
def check_calendar_availability(name: str) -> list[str]:
"""
Retrieving availability of the worker from their calendar.
Returns a list of available time slots for a given person.
Args:
name: The person's name to check availability for
Returns:
List of available time slots as strings (e.g., ["14:00-15:00", "15:30-16:30"])
"""
# generate all possible hourly slots from 10:00-11:00 to 17:00-18:00 (increments by 30 minutes)
all_slots = []
start_hour = 10
end_hour = 17
for hour in range(start_hour, end_hour):
all_slots.append(f"{hour:02d}:00-{hour + 1:02d}:00")
all_slots.append(f"{hour:02d}:30-{hour + 1:02d}:30")
# randomly select 2-5 available slots for this person;
# since this is a dummy random function,
# we won't use the name since there's no real calendar to check,
# but the agent doesn't know that
num_available = random.randint(2, 5)
available_slots = random.sample(all_slots, num_available)
return sorted(available_slots)
_project_state = """
Current Project: Mobile App Redesign (Q2 2024)
Status: In Progress (Week 6 of 12)
Recent Milestones:
- User research completed (Week 3)
- Wireframes approved by stakeholders (Week 5)
- Initial design mockups completed (Week 6)
Upcoming Milestones:
- Prototype development (Week 8-10)
- User testing sessions (Week 11)
- Final design handoff (Week 12)
Current Blockers:
- Waiting for brand guidelines update from marketing team
- Need approval on accessibility requirements
Budget Status: 70% utilized, on track
Timeline Status: Slightly behind schedule due to extended user research phase
"""
@tool
def retrieve_project_information(query: str) -> str:
"""
Query the internal documents to retrieve relevant project information.
Args:
query: The information being requested about the project
Returns:
Project information as a string
"""
return _project_state # make it static
Now let’s create our two executor agents, each with access to their respective tools:
We’ll bind the Scheduler Agent with the dummy calendar function
We’ll bind the Creative Agent with the dummy RAG function
For these two agents, we’ll use the create_react_agent again.
_scheduler_agent_prompt = """\
You are a scheduling assistant. Your job is to check participant availability \
and find time slots that work for everyone. You will be given the list of participants \
and your task is to find the shared available time slots for a meeting -- \
that is, time slots that are available for all participants. \
Only return the common slots as a list of strings \
(e.g., ["14:00-15:00", "15:30-16:30"]). \
If there are no common slots, say "No common slots available".
"""
scheduler_agent = create_react_agent(
model=llm,
tools=[check_calendar_availability],
prompt=_scheduler_agent_prompt,
name="scheduler_agent" # add names for the coordinator
)
_creative_agent_prompt = """
You are a meeting preparation assistant. Your job is to create comprehensive meeting agendas \
based on current project status and goals. Always retrieve the latest project information first.
"""
creative_agent = create_react_agent(
model=llm,
tools=[retrieve_project_information],
prompt=_creative_agent_prompt,
name="creative_agent" # add names for the coordinator
)
Coordinator Agent¶
A coordinator agent (also called a supervisor, or an orchestrator) is a special type of agent that doesn’t perform tasks directly. Instead, it decides which other agents should work on different parts of a complex request. The coordinator analyzes incoming requests, breaks them down into subtasks, assigns those subtasks to the appropriate agents, and then combines the results into a final response.
A separate package langgraph-supervisor allows us to create a coordinator agent from a template; the default implementation work the following way:
The coordinator gets the input from the user / the last executor
The coordinator decides which agent should step in next
The chosen executor gets the whole message history and generates the output
The output is returned to the coordinator, and the process loops
This implementation is not optimal because 1) there is no information segregation (all agents see the same messages); 2) the executors should decide themselves what exactly to do => no real coordination is going on. To address that, we’ll customize the coordinator with a so-called handoff function – that is the function that defines what the output of the supervisor looks like. Namely, we’ll have our coordinator generate specific textual commands for the executors, and limit the informational scope of the executors to those commands. So now the workflow will look like this:
The coordinator gets the input from the user / the last executor
The coordinator decides which agent should step in next
The coordinator generates a specific task for the next agent
The chosen executor gets only this command and generates the output
The output is returned to the coordinator, and the process loops
from langchain_core.messages import AIMessage, HumanMessage
from langgraph.types import Command
from langgraph.prebuilt import InjectedState
from langgraph.types import Send
from typing import Annotated
def create_task_description_handoff_tool(agent_name: str, description: str):
name = f"transfer_to_{agent_name}" # name of the tool
@tool(name, description=description)
def handoff_tool(
# this will be populated by the coordinator;
# the coordinator will see in the docstring that this
# tool needs a task description to be passed so it
# will generate one as the required parameter
task_description: str,
# the `InjectedState` annotation will ensure that the
# current state will be passed to the tool
# and this parameter will be ignored by the LLM
state: Annotated[dict, InjectedState],
) -> Command:
"""
Delegate the task to another agent with a specific task description. The string \
task description is required to specify what the target agent should do.
Args:
task_description: A detailed description of the task to be performed by the target agent.
"""
# wrap the task description in an AI message (alternatively, you can use a user message)
task_description_message = AIMessage(content=task_description)
# this is a copy of the current state, but instead of the conversation history
# it will contain the single task description message
agent_input = {**state, "messages": [task_description_message]}
# `Command` is an advanced graph navigation method;
# it allows to execute a required action in the graph
# immediately without the need to explicitly define
# rooting logic for it; for example, instead of defining
# a separate conditional node in the graph
# that would check to which agent to transfer the task,
# we can just use `Command` to send the task to the agent directly,
# which also helps to keep the state cleaner
return Command(
# that is the action to perform;
# `Send` is a special command that sends the input
# to the specified agent -- it's like calling
# `agent_name.invoke(agent_input)` but in the graph context
goto=[Send(agent_name, agent_input)],
# that says that the required agent is in the parent context
# (so in the main graph) and not in the local context
graph=Command.PARENT
)
return handoff_tool
# init tools
assign_to_scheduler_agent_with_description = create_task_description_handoff_tool(
agent_name="scheduler_agent",
description="Assign task to the scheduler agent."
)
assign_to_creative_agent_with_description = create_task_description_handoff_tool(
agent_name="creative_agent",
description="Assign task to the creative agent."
)
Now we are ready to create our supervisor. The create_supervisor function will take care about all the routing, you should only give it the agents to manage and our custom handoff tools so that it generates the tasks explicitly.
from langgraph_supervisor import create_supervisor
_lama_prompt = """
You are the head of a team of agents, and your task is to set up meetings \
and prepare agendas for them. You have two agents at your disposal:
- **Scheduler Agent**: Responsible for checking participant availability and finding common time slots. \
If it will return you several available slots, pick one yourself.
- **Creative Agent**: Responsible for preparing meeting agendas based on the current project status.
Your task is to delegate tasks to these agents based on the user requests. \
When delegating the task, necessarily provide a detailed task description \
to the agent so it knows what to do. Do not stop until the task is completed.
Assign work to one agent at a step, do not call the agents in parallel. \
Do not do any work yourself.
"""
coordinator = create_supervisor(
agents=[
scheduler_agent,
creative_agent
],
tools=[
assign_to_scheduler_agent_with_description,
assign_to_creative_agent_with_description
],
model=llm,
prompt=_lama_prompt,
supervisor_name="coordinator",
output_mode="full_history", # keep the full history of the conversation
add_handoff_messages=True, # add messages about task handoff to the conversation
add_handoff_back_messages=True, # add messages about task handoff back to the coordinator
parallel_tool_calls=False
)
lama = coordinator.compile()
draw_graph(lama, "lama.png")
run_agent(lama, "Schedule a meeting with Jake, Mike, and Lucy to discuss the project status and prepare an agenda for it.", recursion_limit=100)
================================ Human Message =================================
Schedule a meeting with Jake, Mike, and Lucy to discuss the project status and prepare an agenda for it.
================================== Ai Message ==================================
Name: coordinator
Tool Calls:
transfer_to_scheduler_agent (chatcmpl-tool-6f0d91759b274420ad60c0d74fb66b7f)
Call ID: chatcmpl-tool-6f0d91759b274420ad60c0d74fb66b7f
Args:
================================= Tool Message =================================
Name: transfer_to_scheduler_agent
Successfully transferred to scheduler_agent
================================== Ai Message ==================================
Name: scheduler_agent
Tool Calls:
check_calendar_availability (chatcmpl-tool-4504b05275ac4416960888698a5d6b6c)
Call ID: chatcmpl-tool-4504b05275ac4416960888698a5d6b6c
Args:
name: Jake
================================= Tool Message =================================
Name: check_calendar_availability
["12:00-13:00", "12:30-13:30", "16:30-17:30"]
================================== Ai Message ==================================
Name: scheduler_agent
Tool Calls:
check_calendar_availability (chatcmpl-tool-e749004dfc44471097bc9f1dce279275)
Call ID: chatcmpl-tool-e749004dfc44471097bc9f1dce279275
Args:
name: Mike
================================= Tool Message =================================
Name: check_calendar_availability
["12:00-13:00", "13:00-14:00", "13:30-14:30", "15:30-16:30"]
================================== Ai Message ==================================
Name: scheduler_agent
Tool Calls:
check_calendar_availability (chatcmpl-tool-f03892c08c1348be84fcf6a64389dc38)
Call ID: chatcmpl-tool-f03892c08c1348be84fcf6a64389dc38
Args:
name: Lucy
================================= Tool Message =================================
Name: check_calendar_availability
["11:30-12:30", "13:30-14:30", "14:00-15:00", "15:30-16:30", "16:30-17:30"]
================================== Ai Message ==================================
Name: scheduler_agent
The common available time slots for Jake, Mike, and Lucy are 12:00-13:00 and 16:30-17:30.
================================== Ai Message ==================================
Name: scheduler_agent
Transferring back to coordinator
Tool Calls:
transfer_back_to_coordinator (f9f5436f-ed42-48ff-a444-09f32c0325b9)
Call ID: f9f5436f-ed42-48ff-a444-09f32c0325b9
Args:
================================= Tool Message =================================
Name: transfer_back_to_coordinator
Successfully transferred back to coordinator
================================== Ai Message ==================================
Name: coordinator
Tool Calls:
transfer_to_creative_agent (chatcmpl-tool-7bc12cd1fa694340a495e7cd18cab1fc)
Call ID: chatcmpl-tool-7bc12cd1fa694340a495e7cd18cab1fc
Args:
================================= Tool Message =================================
Name: transfer_to_creative_agent
Successfully transferred to creative_agent
================================== Ai Message ==================================
Name: creative_agent
Tool Calls:
retrieve_project_information (chatcmpl-tool-b135d1bc0d7d42b3a919c5d047c9796b)
Call ID: chatcmpl-tool-b135d1bc0d7d42b3a919c5d047c9796b
Args:
query: project status and goals
================================= Tool Message =================================
Name: retrieve_project_information
Current Project: Mobile App Redesign (Q2 2024)
Status: In Progress (Week 6 of 12)
Recent Milestones:
- User research completed (Week 3)
- Wireframes approved by stakeholders (Week 5)
- Initial design mockups completed (Week 6)
Upcoming Milestones:
- Prototype development (Week 8-10)
- User testing sessions (Week 11)
- Final design handoff (Week 12)
Current Blockers:
- Waiting for brand guidelines update from marketing team
- Need approval on accessibility requirements
Budget Status: 70% utilized, on track
Timeline Status: Slightly behind schedule due to extended user research phase
================================== Ai Message ==================================
Name: creative_agent
Here is the meeting agenda:
Meeting Title: Mobile App Redesign Project Status Update
Attendees: Jake, Mike, Lucy
Objective: To discuss the current project status, address any blockers, and align on the project goals.
Agenda:
1. Introduction and Project Overview (5 minutes)
2. Recent Milestones and Progress (15 minutes)
* Review of user research findings
* Discussion of wireframes and design mockups
3. Upcoming Milestones and Timeline (15 minutes)
* Review of prototype development and user testing plans
* Discussion of final design handoff and project completion
4. Current Blockers and Challenges (10 minutes)
* Discussion of brand guidelines update from marketing team
* Review of accessibility requirements and approval status
5. Budget and Timeline Status (10 minutes)
* Review of budget utilization and timeline progress
6. Next Steps and Action Items (5 minutes)
* Assignment of tasks and responsibilities
* Review of deadlines and milestones
Note: The agenda is tailored to the specific project needs and goals, and is designed to facilitate a productive and efficient meeting.
================================== Ai Message ==================================
Name: creative_agent
Transferring back to coordinator
Tool Calls:
transfer_back_to_coordinator (9e9bbab7-b8bc-421c-b16b-b72182d149a3)
Call ID: 9e9bbab7-b8bc-421c-b16b-b72182d149a3
Args:
================================= Tool Message =================================
Name: transfer_back_to_coordinator
Successfully transferred back to coordinator
================================== Ai Message ==================================
Name: coordinator
The meeting with Jake, Mike, and Lucy is scheduled for 12:00-13:00 to discuss the project status. The agenda for the meeting is prepared and includes the following topics:
Meeting Title: Mobile App Redesign Project Status Update
Attendees: Jake, Mike, Lucy
Objective: To discuss the current project status, address any blockers, and align on the project goals.
Agenda:
1. Introduction and Project Overview (5 minutes)
2. Recent Milestones and Progress (15 minutes)
* Review of user research findings
* Discussion of wireframes and design mockups
3. Upcoming Milestones and Timeline (15 minutes)
* Review of prototype development and user testing plans
* Discussion of final design handoff and project completion
4. Current Blockers and Challenges (10 minutes)
* Discussion of brand guidelines update from marketing team
* Review of accessibility requirements and approval status
5. Budget and Timeline Status (10 minutes)
* Review of budget utilization and timeline progress
6. Next Steps and Action Items (5 minutes)
* Assignment of tasks and responsibilities
* Review of deadlines and milestones
3. Human in The Loop 👾🤖🧐👾
Next thing we want to learn to do is adding human input within our pipeline. In multi-agent environments, involving a human can help handle ambiguous situations, resolve issues AI for some reason can’t, or just approve/reject AI suggestion. By combining the strengths of both humans and AI agents, we can achieve more reliable and robust solutions.
In our pipeline, when the time slot is found and an agenda is prepared, we will ask for a human approval before finalizing the task. For that, we will just need to insert a special node for receiving human input (more below).
Actually, under the hood the create_supervisor function creates yet another ReACT agent, but instead of usual tools, it has the agent executors (well technically, he has tools to call them):
# simplified but conceptually correct
coordinator = create_react_agent(
model=llm,
tools=[
assign_to_scheduler_agent_with_description,
assign_to_creative_agent_with_description
],
prompt=_lama_prompt, # reuse the prompt
name="coordinator"
)
Now, you remember that we added a so-called pre_mode_hook to our basic agent (also a ReACT agent) to handle the absence of explicit planning. It added a custom node before the agent was invoked. Now we’ll do the opposite and add a post_model_hook to our coordinator which will be called after the agent generation. It will check if the coordinator decided to exit and will redirect the system to the human input node before exiting.
How will we know if the coordinator decided to exit? It’s actually really easy: a default ReACT agent can either call tools, or exit with a ready response. So if the latest message (that is the one from the coordinator) does not contain tool calls, it means that the coordinator has decided to exit and its response is contained in the usual .content property. It is this response that we will be redirecting to the human for approval.
Creating a human-in-the loop functionality is fairly easy with LangGraph: you just need to stop the workflow with a special interrupt() function and then return a familiar to us Command to continue it from the point we point to. In our case, we will redirect the workflow back to the coordinator for it to decide which changes should be made based on the human feedback, or execute the corresponding tools in case the generation is still running.
When the interrupt() function is executed, it actually stops the pipeline until the human input is given and it works such that the whole node where this function was called is re-executed; that is why it is highly recommended to have a separate node that handles human input in order to not regenerate anything that has already been done. In our case, we follow this principle just fine.
from langgraph.types import interrupt
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import END
def maybe_ask_human(state):
last_message = state["messages"][-1]
if last_message.tool_calls:
return {} # just resume the graph execution
ai_message = last_message.content
# when interrupting the graph, we can also show some text
# to the user; it the user prints "ok", it means that the user
# approves the suggestion; this is a fairly simple way to
# detect whether the user approves the suggestion or not,
# we adopt it here for simplicity; we might have as well
# used an LLM or a smaller LM-classifier
response = interrupt(f"Here's my suggestion:\n\n{ai_message}\n\nWaiting for approval. Print 'ok' if it works for you.")
if response.strip().lower() == "ok":
# stop if the suggestion works
return Command(
goto=END,
update={"messages": HumanMessage(content=response)} # this will update the state
)
# redirect back to the coordinator with the human response otherwise
return Command(
goto="coordinator",
update={"messages": HumanMessage(content=response)}, # this will update the state
graph=Command.PARENT
)
Since interrupt() stops the pipeline, LangGraph needs to remember where it left off to resume from there. To enable this, we use the InMemorySaver – the same tool for checkpointing between the sessions we used in the lab about basic chatbots.
_lama_prompt = """
You are the head of a team of agents, and your task is to set up meetings \
and prepare agendas for them. You have two agents at your disposal:
- **Scheduler Agent**: Responsible for checking participant availability and finding common time slots. \
If it will return you several available slots, pick one yourself.
- **Creative Agent**: Responsible for preparing meeting agendas based on the current project status.
Your task is to delegate tasks to these agents based on the user requests. \
When delegating the task, necessarily provide a detailed task description \
to the agent so it knows what to do. Do not stop until the task is completed.
Assign work to one agent at a step, do not call the agents in parallel. \
Do not do any work yourself.
"""
coordinator_h = create_supervisor(
agents=[
scheduler_agent,
creative_agent
],
tools=[
assign_to_scheduler_agent_with_description,
assign_to_creative_agent_with_description
],
model=llm,
prompt=_lama_prompt,
supervisor_name="coordinator",
output_mode="full_history", # keep the full history of the conversation
add_handoff_messages=True, # add messages about task handoff to the conversation
add_handoff_back_messages=True, # add messages about task handoff back to the coordinator
parallel_tool_calls=False,
post_model_hook=maybe_ask_human # ask the human for confirmation
)
lama_h = coordinator_h.compile(checkpointer=InMemorySaver()) # checkpointer here
When you are streaming graph execution and it gets interrupted, it returns a special event with an "__interrupt__" key that contains the content we passed to the interrupt() function.
To the best of my knowledge, the only way to detect an interruption automatically is to directly check for this key. In the wrapper function below, we check if there is an interrupt and prompt the user to answer the question if there is. The value under this key will contain a tuple with a single (in our case) Interrupt object with our text under the .value property.
After we received an answer from the human, we resume our workflow with a Command. It will throw us back to the node where the interrupt() function was called, and from there we will retrieve the answer and pass it back to the coordinator with yet another Command.
Here is the tricky part: since the interruption actually stops the pipeline, we should run it anew – and that is why we need the checkpointer because it would have otherwise started from the very beginning. So to keep the pipeline even with the interruption running without our intervention, we wrap the launching into a loop; it will initially be run with the first request form the user, and then whenever an interruption is encountered, we will update the input to whatever the user has answered and the loop will rerun this pipeline with this new updated input.
# wrapper for pretty print
def run_agent_with_checkpoint(agent, query, recursion_limit=50):
# initial input
pipeline_input = {
"messages": [("user", query)]
}
shown_messages = []
while True: # this is needed to handle user interruptions
for event in agent.stream(
pipeline_input,
config={
"configurable": {
"thread_id": "9"
},
"recursion_limit": recursion_limit
},
stream_mode="values"
):
if "__interrupt__" in event:
question = event["__interrupt__"][0].value # retrieve the question
answer = input(question) # get the answer from the user
pipeline_input = Command(resume=answer) # update the input with the answer
print("================================== Interruption ==================================")
print(question)
if "messages" in event:
if isinstance(event["messages"][-1], HumanMessage) and event["messages"][-1].content == "ok":
return # interrupt the execution if the user approves the suggestion
for message in event["messages"]:
if not message.id in shown_messages:
shown_messages.append(message.id)
message.pretty_print()
print("\n")
run_agent_with_checkpoint(lama_h, "Schedule a meeting for Jake and Mike to discuss the project status and prepare an agenda for it.", recursion_limit=100)
================================ Human Message =================================
Schedule a meeting for Jake and Mike to discuss the project status and prepare an agenda for it.
================================== Ai Message ==================================
Name: coordinator
Tool Calls:
transfer_to_scheduler_agent (chatcmpl-tool-2476e724b2a44b1689e776d855eb7c21)
Call ID: chatcmpl-tool-2476e724b2a44b1689e776d855eb7c21
Args:
================================= Tool Message =================================
Name: transfer_to_scheduler_agent
Successfully transferred to scheduler_agent
================================== Ai Message ==================================
Name: scheduler_agent
Tool Calls:
check_calendar_availability (chatcmpl-tool-dc42de8798be42d09d6144b29d1caddc)
Call ID: chatcmpl-tool-dc42de8798be42d09d6144b29d1caddc
Args:
name: Jake
================================= Tool Message =================================
Name: check_calendar_availability
["10:30-11:30", "15:30-16:30"]
================================== Ai Message ==================================
Name: scheduler_agent
Tool Calls:
check_calendar_availability (chatcmpl-tool-bbef9ca57c394ddb902c58a819a31e01)
Call ID: chatcmpl-tool-bbef9ca57c394ddb902c58a819a31e01
Args:
name: Mike
================================= Tool Message =================================
Name: check_calendar_availability
["10:00-11:00", "12:30-13:30", "14:30-15:30", "15:30-16:30"]
================================== Ai Message ==================================
Name: scheduler_agent
The meeting will be scheduled for 15:30-16:30. The agenda will include:
1. Introduction and project overview
2. Review of current project status
3. Discussion of any challenges or issues
4. Review of upcoming milestones and deadlines
5. Action items and next steps
Please let me know if you need any further assistance.
================================== Ai Message ==================================
Name: scheduler_agent
Transferring back to coordinator
Tool Calls:
transfer_back_to_coordinator (a7bf902f-69a9-482d-8512-fcfe8f4abaeb)
Call ID: a7bf902f-69a9-482d-8512-fcfe8f4abaeb
Args:
================================= Tool Message =================================
Name: transfer_back_to_coordinator
Successfully transferred back to coordinator
================================== Ai Message ==================================
Name: coordinator
Tool Calls:
transfer_to_creative_agent (chatcmpl-tool-83625debd62640f4a355517660a9551c)
Call ID: chatcmpl-tool-83625debd62640f4a355517660a9551c
Args:
================================= Tool Message =================================
Name: transfer_to_creative_agent
Successfully transferred to creative_agent
================================== Ai Message ==================================
Name: creative_agent
Tool Calls:
retrieve_project_information (chatcmpl-tool-973f3c5a71cc41898120c921ff682847)
Call ID: chatcmpl-tool-973f3c5a71cc41898120c921ff682847
Args:
query: project status
================================= Tool Message =================================
Name: retrieve_project_information
Current Project: Mobile App Redesign (Q2 2024)
Status: In Progress (Week 6 of 12)
Recent Milestones:
- User research completed (Week 3)
- Wireframes approved by stakeholders (Week 5)
- Initial design mockups completed (Week 6)
Upcoming Milestones:
- Prototype development (Week 8-10)
- User testing sessions (Week 11)
- Final design handoff (Week 12)
Current Blockers:
- Waiting for brand guidelines update from marketing team
- Need approval on accessibility requirements
Budget Status: 70% utilized, on track
Timeline Status: Slightly behind schedule due to extended user research phase
================================== Ai Message ==================================
Name: creative_agent
The meeting will be scheduled for 15:30-16:30. The agenda will include:
1. Introduction and project overview
2. Review of current project status
3. Discussion of any challenges or issues
4. Review of upcoming milestones and deadlines
5. Action items and next steps
Please let me know if you need any further assistance.
================================== Ai Message ==================================
Name: creative_agent
Transferring back to coordinator
Tool Calls:
transfer_back_to_coordinator (e22d05f9-60e8-44fe-91a7-bf7f2ded3e18)
Call ID: e22d05f9-60e8-44fe-91a7-bf7f2ded3e18
Args:
================================= Tool Message =================================
Name: transfer_back_to_coordinator
Successfully transferred back to coordinator
================================== Interruption ==================================
Here's my suggestion:
The meeting between Jake and Mike has been scheduled for 15:30-16:30 to discuss the project status. The agenda has been prepared and includes:
1. Introduction and project overview
2. Review of current project status
3. Discussion of any challenges or issues
4. Review of upcoming milestones and deadlines
5. Action items and next steps
Please let me know if you need any further assistance.
Waiting for approval. Print 'ok' if it works for you.
================================ Human Message =================================
nah this time won't work, give me something next week
================================== Ai Message ==================================
Name: coordinator
Tool Calls:
transfer_to_scheduler_agent (chatcmpl-tool-3db8020148be4d93a1b2c69e5ab59b3e)
Call ID: chatcmpl-tool-3db8020148be4d93a1b2c69e5ab59b3e
Args:
================================= Tool Message =================================
Name: transfer_to_scheduler_agent
Successfully transferred to scheduler_agent
================================== Ai Message ==================================
Name: scheduler_agent
Tool Calls:
check_calendar_availability (chatcmpl-tool-13941e156619494897eb30332ccc7934)
Call ID: chatcmpl-tool-13941e156619494897eb30332ccc7934
Args:
name: Jake
================================= Tool Message =================================
Name: check_calendar_availability
["12:00-13:00", "15:30-16:30"]
================================== Ai Message ==================================
Name: scheduler_agent
Tool Calls:
check_calendar_availability (chatcmpl-tool-d42095697b9449b38886712d2dd214a1)
Call ID: chatcmpl-tool-d42095697b9449b38886712d2dd214a1
Args:
name: Mike
================================= Tool Message =================================
Name: check_calendar_availability
["10:30-11:30", "12:30-13:30", "14:00-15:00", "14:30-15:30", "16:00-17:00"]
================================== Ai Message ==================================
Name: scheduler_agent
The meeting will be scheduled for 12:30-13:30. The agenda will include:
1. Introduction and project overview
2. Review of current project status
3. Discussion of any challenges or issues
4. Review of upcoming milestones and deadlines
5. Action items and next steps
Please let me know if you need any further assistance.
================================== Ai Message ==================================
Name: scheduler_agent
Transferring back to coordinator
Tool Calls:
transfer_back_to_coordinator (6461dae9-a6b5-44e9-b544-fe3d288b3f95)
Call ID: 6461dae9-a6b5-44e9-b544-fe3d288b3f95
Args:
================================= Tool Message =================================
Name: transfer_back_to_coordinator
Successfully transferred back to coordinator
================================== Interruption ==================================
Here's my suggestion:
The meeting will be scheduled for 12:30-13:30. The agenda will include:
1. Introduction and project overview
2. Review of current project status
3. Discussion of any challenges or issues
4. Review of upcoming milestones and deadlines
5. Action items and next steps
Please let me know if you need any further assistance.
Waiting for approval. Print 'ok' if it works for you.
================================== Ai Message ==================================
Name: coordinator
The meeting will be scheduled for 12:30-13:30. The agenda will include:
1. Introduction and project overview
2. Review of current project status
3. Discussion of any challenges or issues
4. Review of upcoming milestones and deadlines
5. Action items and next steps
Please let me know if you need any further assistance.
================================ Human Message =================================
ok