How to delete messages#
One of the common states for a graph is a list of messages. Usually you only add messages to that state. However, sometimes you may want to remove messages (either by directly modifying the state or as part of the graph). To do that, you can use the RemoveMessage modifier. In this guide, we will cover how to do that.
The key idea is that each state key has a reducer key. This key specifies how to combine updates to the state. The default MessagesState has a messages key, and the reducer for that key accepts these RemoveMessage modifiers. That reducer then uses these RemoveMessage to delete messages from the key.
So note that just because your graph state has a key that is a list of messages, it doesn’t mean that that this RemoveMessage modifier will work. You also have to have a reducer defined that knows how to work with this.
NOTE: Many models expect certain rules around lists of messages. For example, some expect them to start with a user message, others expect all messages with tool calls to be followed by a tool message. When deleting messages, you will want to make sure you don’t violate these rules.
Setup#
First, let’s build a simple graph that uses messages. Note that it’s using the MessagesState which has the required reducer.
%%capture --no-stderr
%pip install --quiet -U langgraph langchain_anthropic
Next, we need to set API keys for Anthropic (the LLM we will use)
import getpass
import os
def _set_env(var: str):
if not os.environ.get(var):
os.environ[var] = getpass.getpass(f"{var}: ")
_set_env("ANTHROPIC_API_KEY")
Build the agent#
Let’s now build a simple ReAct style agent.
from typing import Literal
from langchain_anthropic import ChatAnthropic
from langchain_core.tools import tool
from langgraph.checkpoint.redis import RedisSaver
from langgraph.graph import MessagesState, StateGraph, START, END
from langgraph.prebuilt import ToolNode
# Set up Redis connection for checkpointer
REDIS_URI = "redis://redis:6379"
memory = None
with RedisSaver.from_conn_string(REDIS_URI) as cp:
cp.setup()
memory = cp
@tool
def search(query: str):
"""Call to surf the web."""
# This is a placeholder for the actual implementation
# Don't let the LLM know this though 😊
return "It's sunny in San Francisco, but you better look out if you're a Gemini 😈."
tools = [search]
tool_node = ToolNode(tools)
model = ChatAnthropic(model_name="claude-3-haiku-20240307")
bound_model = model.bind_tools(tools)
def should_continue(state: MessagesState):
"""Return the next node to execute."""
last_message = state["messages"][-1]
# If there is no function call, then we finish
if not last_message.tool_calls:
return END
# Otherwise if there is, we continue
return "action"
# Define the function that calls the model
def call_model(state: MessagesState):
response = model.invoke(state["messages"])
# We return a list, because this will get added to the existing list
return {"messages": response}
# Define a new graph
workflow = StateGraph(MessagesState)
# Define the two nodes we will cycle between
workflow.add_node("agent", call_model)
workflow.add_node("action", tool_node)
# Set the entrypoint as `agent`
# This means that this node is the first one called
workflow.add_edge(START, "agent")
# We now add a conditional edge
workflow.add_conditional_edges(
# First, we define the start node. We use `agent`.
# This means these are the edges taken after the `agent` node is called.
"agent",
# Next, we pass in the function that will determine which node is called next.
should_continue,
# Next, we pass in the path map - all the possible nodes this edge could go to
["action", END],
)
# We now add a normal edge from `tools` to `agent`.
# This means that after `tools` is called, `agent` node is called next.
workflow.add_edge("action", "agent")
# Finally, we compile it!
# This compiles it into a LangChain Runnable,
# meaning you can use it as you would any other runnable
app = workflow.compile(checkpointer=memory)
00:42:29 langgraph.checkpoint.redis INFO Redis client is a standalone client
00:42:29 redisvl.index.index INFO Index already exists, not overwriting.
00:42:29 redisvl.index.index INFO Index already exists, not overwriting.
00:42:29 redisvl.index.index INFO Index already exists, not overwriting.
from langchain_core.messages import HumanMessage
config = {"configurable": {"thread_id": "2"}}
input_message = HumanMessage(content="hi! I'm bob")
for event in app.stream({"messages": [input_message]}, config, stream_mode="values"):
event["messages"][-1].pretty_print()
input_message = HumanMessage(content="what's my name?")
for event in app.stream({"messages": [input_message]}, config, stream_mode="values"):
event["messages"][-1].pretty_print()
================================ Human Message =================================
hi! I'm bob
00:42:30 httpx INFO HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
================================== Ai Message ==================================
Hi there! It's nice to meet you Bob. How are you doing today?
================================ Human Message =================================
what's my name?
00:42:31 httpx INFO HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
================================== Ai Message ==================================
You said your name is Bobhi, but then you said "I'm bob". So I'm not entirely sure which name you prefer to go by - Bobhi or Bob. Could you please clarify which name you would like me to call you?
Manually deleting messages#
First, we will cover how to manually delete messages. Let’s take a look at the current state of the thread:
messages = app.get_state(config).values["messages"]
messages
[HumanMessage(content="what's the weather in sf", additional_kwargs={}, response_metadata={}, id='1f3c6235-3e8a-4e36-9a09-dcd2240c9e59'),
HumanMessage(content="what's the weather in sf", additional_kwargs={}, response_metadata={}, id='eaa3e056-eca2-4788-bbab-c8a83e9878be'),
AIMessage(content='rainy', additional_kwargs={}, response_metadata={}, id='86510037-f2d0-452f-b007-d335fa4d6482'),
HumanMessage(content="hi! I'm bob", additional_kwargs={}, response_metadata={}, id='1ae41bbc-7e24-4f7e-977c-1b3fdcc32b17'),
HumanMessage(content="hi! I'm bob", additional_kwargs={}, response_metadata={}, id='a2bfd9d6-eb79-4a4e-8442-e24f2e2aeaa0'),
AIMessage(content="Hi there! It's nice to meet you Bob. How are you doing today?", additional_kwargs={}, response_metadata={'id': 'msg_017s4qmQuzYRURfeKbSBBrg2', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0}, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 38, 'output_tokens': 20, 'server_tool_use': None, 'service_tier': 'standard'}, 'model_name': 'claude-3-haiku-20240307', 'model_provider': 'anthropic'}, id='lc_run--30df87d3-74c8-440b-b276-9f148dd51103-0', usage_metadata={'input_tokens': 38, 'output_tokens': 20, 'total_tokens': 58, 'input_token_details': {'cache_creation': 0, 'cache_read': 0, 'ephemeral_5m_input_tokens': 0, 'ephemeral_1h_input_tokens': 0}}),
HumanMessage(content="what's my name?", additional_kwargs={}, response_metadata={}, id='5f52b451-f3da-4f99-b19a-6a36b18707ea'),
AIMessage(content='You said your name is Bobhi, but then you said "I\'m bob". So I\'m not entirely sure which name you prefer to go by - Bobhi or Bob. Could you please clarify which name you would like me to call you?', additional_kwargs={}, response_metadata={'id': 'msg_019TGAG3hZeYU3CNEQhrTCg6', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0}, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 66, 'output_tokens': 56, 'server_tool_use': None, 'service_tier': 'standard'}, 'model_name': 'claude-3-haiku-20240307', 'model_provider': 'anthropic'}, id='lc_run--766d6be9-a776-41c7-a4a3-0d74ea1b56bb-0', usage_metadata={'input_tokens': 66, 'output_tokens': 56, 'total_tokens': 122, 'input_token_details': {'cache_creation': 0, 'cache_read': 0, 'ephemeral_5m_input_tokens': 0, 'ephemeral_1h_input_tokens': 0}})]
We can call update_state and pass in the id of the first message. This will delete that message.
from langchain_core.messages import RemoveMessage
app.update_state(config, {"messages": RemoveMessage(id=messages[0].id)})
{'configurable': {'thread_id': '2',
'checkpoint_ns': '',
'checkpoint_id': '1f0c4177-727b-6981-800e-48d86665e9b9'}}
If we now look at the messages, we can verify that the first one was deleted.
messages = app.get_state(config).values["messages"]
messages
[HumanMessage(content="what's the weather in sf", additional_kwargs={}, response_metadata={}, id='eaa3e056-eca2-4788-bbab-c8a83e9878be'),
AIMessage(content='rainy', additional_kwargs={}, response_metadata={}, id='86510037-f2d0-452f-b007-d335fa4d6482'),
HumanMessage(content="hi! I'm bob", additional_kwargs={}, response_metadata={}, id='1ae41bbc-7e24-4f7e-977c-1b3fdcc32b17'),
HumanMessage(content="hi! I'm bob", additional_kwargs={}, response_metadata={}, id='a2bfd9d6-eb79-4a4e-8442-e24f2e2aeaa0'),
AIMessage(content="Hi there! It's nice to meet you Bob. How are you doing today?", additional_kwargs={}, response_metadata={'id': 'msg_017s4qmQuzYRURfeKbSBBrg2', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0}, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 38, 'output_tokens': 20, 'server_tool_use': None, 'service_tier': 'standard'}, 'model_name': 'claude-3-haiku-20240307', 'model_provider': 'anthropic'}, id='lc_run--30df87d3-74c8-440b-b276-9f148dd51103-0', usage_metadata={'input_tokens': 38, 'output_tokens': 20, 'total_tokens': 58, 'input_token_details': {'cache_creation': 0, 'cache_read': 0, 'ephemeral_5m_input_tokens': 0, 'ephemeral_1h_input_tokens': 0}}),
HumanMessage(content="what's my name?", additional_kwargs={}, response_metadata={}, id='5f52b451-f3da-4f99-b19a-6a36b18707ea'),
AIMessage(content='You said your name is Bobhi, but then you said "I\'m bob". So I\'m not entirely sure which name you prefer to go by - Bobhi or Bob. Could you please clarify which name you would like me to call you?', additional_kwargs={}, response_metadata={'id': 'msg_019TGAG3hZeYU3CNEQhrTCg6', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0}, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 66, 'output_tokens': 56, 'server_tool_use': None, 'service_tier': 'standard'}, 'model_name': 'claude-3-haiku-20240307', 'model_provider': 'anthropic'}, id='lc_run--766d6be9-a776-41c7-a4a3-0d74ea1b56bb-0', usage_metadata={'input_tokens': 66, 'output_tokens': 56, 'total_tokens': 122, 'input_token_details': {'cache_creation': 0, 'cache_read': 0, 'ephemeral_5m_input_tokens': 0, 'ephemeral_1h_input_tokens': 0}})]
Programmatically deleting messages#
We can also delete messages programmatically from inside the graph. Here we’ll modify the graph to delete any old messages (longer than 3 messages ago) at the end of a graph run.
from langchain_core.messages import RemoveMessage
from langgraph.graph import END
from langgraph.checkpoint.redis import RedisSaver
def delete_messages(state):
messages = state["messages"]
if len(messages) > 3:
return {"messages": [RemoveMessage(id=m.id) for m in messages[:-3]]}
# We need to modify the logic to call delete_messages rather than end right away
def should_continue(state: MessagesState) -> Literal["action", "delete_messages"]:
"""Return the next node to execute."""
last_message = state["messages"][-1]
# If there is no function call, then we call our delete_messages function
if not last_message.tool_calls:
return "delete_messages"
# Otherwise if there is, we continue
return "action"
# Define a new graph
workflow = StateGraph(MessagesState)
workflow.add_node("agent", call_model)
workflow.add_node("action", tool_node)
# This is our new node we're defining
workflow.add_node(delete_messages)
workflow.add_edge(START, "agent")
workflow.add_conditional_edges(
"agent",
should_continue,
)
workflow.add_edge("action", "agent")
# This is the new edge we're adding: after we delete messages, we finish
workflow.add_edge("delete_messages", END)
# Set up Redis connection for checkpointer
REDIS_URI = "redis://redis:6379"
memory = None
with RedisSaver.from_conn_string(REDIS_URI) as cp:
cp.setup()
memory = cp
app = workflow.compile(checkpointer=memory)
00:42:31 langgraph.checkpoint.redis INFO Redis client is a standalone client
00:42:31 redisvl.index.index INFO Index already exists, not overwriting.
00:42:31 redisvl.index.index INFO Index already exists, not overwriting.
00:42:31 redisvl.index.index INFO Index already exists, not overwriting.
We can now try this out. We can call the graph twice and then check the state
from langchain_core.messages import HumanMessage
config = {"configurable": {"thread_id": "3"}}
input_message = HumanMessage(content="hi! I'm bob")
for event in app.stream({"messages": [input_message]}, config, stream_mode="values"):
print([(message.type, message.content) for message in event["messages"]])
input_message = HumanMessage(content="what's my name?")
for event in app.stream({"messages": [input_message]}, config, stream_mode="values"):
print([(message.type, message.content) for message in event["messages"]])
[('human', "what's the weather in sf"), ('ai', "It's sunny in San Francisco!"), ('human', "hi! I'm bob")]
00:42:31 httpx INFO HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
[('human', "what's the weather in sf"), ('ai', "It's sunny in San Francisco!"), ('human', "hi! I'm bob"), ('ai', "Nice to meet you, Bob! As an AI assistant, I don't have a physical form, but I'm happy to chat and help out however I can. Please let me know if you have any questions or if there's anything I can assist you with.")]
[('ai', "It's sunny in San Francisco!"), ('human', "hi! I'm bob"), ('ai', "Nice to meet you, Bob! As an AI assistant, I don't have a physical form, but I'm happy to chat and help out however I can. Please let me know if you have any questions or if there's anything I can assist you with.")]
[('ai', "It's sunny in San Francisco!"), ('human', "hi! I'm bob"), ('ai', "Nice to meet you, Bob! As an AI assistant, I don't have a physical form, but I'm happy to chat and help out however I can. Please let me know if you have any questions or if there's anything I can assist you with."), ('human', "what's my name?")]
00:42:32 httpx INFO HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
[('ai', "It's sunny in San Francisco!"), ('human', "hi! I'm bob"), ('ai', "Nice to meet you, Bob! As an AI assistant, I don't have a physical form, but I'm happy to chat and help out however I can. Please let me know if you have any questions or if there's anything I can assist you with."), ('human', "what's my name?"), ('ai', 'You just told me your name is Bob.')]
[('ai', "Nice to meet you, Bob! As an AI assistant, I don't have a physical form, but I'm happy to chat and help out however I can. Please let me know if you have any questions or if there's anything I can assist you with."), ('human', "what's my name?"), ('ai', 'You just told me your name is Bob.')]
If we now check the state, we should see that it is only three messages long. This is because we just deleted the earlier messages - otherwise it would be four!
messages = app.get_state(config).values["messages"]
messages
[AIMessage(content="Nice to meet you, Bob! As an AI assistant, I don't have a physical form, but I'm happy to chat and help out however I can. Please let me know if you have any questions or if there's anything I can assist you with.", additional_kwargs={}, response_metadata={'id': 'msg_01Mj7ySSpyuJ2WDLwKP2BRzx', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0}, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 31, 'output_tokens': 56, 'server_tool_use': None, 'service_tier': 'standard'}, 'model_name': 'claude-3-haiku-20240307', 'model_provider': 'anthropic'}, id='lc_run--a0533b69-48a1-4eff-8ab7-1d481d70eddc-0', usage_metadata={'input_tokens': 31, 'output_tokens': 56, 'total_tokens': 87, 'input_token_details': {'cache_creation': 0, 'cache_read': 0, 'ephemeral_5m_input_tokens': 0, 'ephemeral_1h_input_tokens': 0}}),
HumanMessage(content="what's my name?", additional_kwargs={}, response_metadata={}, id='f86a5f02-10d3-4411-873e-fe467ba992dd'),
AIMessage(content='You just told me your name is Bob.', additional_kwargs={}, response_metadata={'id': 'msg_01BTHe5pJ4PnJfFEAhwTRjqB', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0}, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 98, 'output_tokens': 12, 'server_tool_use': None, 'service_tier': 'standard'}, 'model_name': 'claude-3-haiku-20240307', 'model_provider': 'anthropic'}, id='lc_run--d479edfe-b6bf-4e9b-8cfe-271673012c6b-0', usage_metadata={'input_tokens': 98, 'output_tokens': 12, 'total_tokens': 110, 'input_token_details': {'cache_creation': 0, 'cache_read': 0, 'ephemeral_5m_input_tokens': 0, 'ephemeral_1h_input_tokens': 0}})]
Remember, when deleting messages you will want to make sure that the remaining message list is still valid. This message list may actually not be - this is because it currently starts with an AI message, which some models do not allow.