Developer ‘Tiqs & Tricks
January 2024
Managing AI Conversation Memory with Service Procedures
In last month’s ‘Tiqs & Tricks, we looked at an easy and practically automatic way to maintain AI conversation within an App. This month, we’ll look at the programmatic way to keep conversation memory.
Anatomy of a Conversation: Much like a court transcript or a screen play, AI conversations are blocks of dialogue, each containing at minimum two vital pieces of information: 1) Who said something, and 2) What was said
In Vantiq that conversation block is called a ChatMessage, and it has the structure:
{
type : <one of “human,” “ai”, “system”, “chat”, or “function”>,
content : <the string value of the content>,
ars_properties: <optional set of characteristics for later search>
}
There are more properties, but we’ll leave those aside for another blog post on AI functions. (Coming soon, I promise!). The structure of the ChatMessage schema type comes to us courtesy of the io.vantiq.ai.ChatMessage Service, which, not coincidently, also provides a series of “builder” procedures for creating ChatMessages, including:
• ChatMessage.buildSystemMessage(content String Required) : ChatMessage
• ChatMessage.buildHumanMessage(content String Required) : ChatMessage
• ChatMessage.buildAIMessage(content String Required) : ChatMessage
…and so on, for each of the types of ChatMessage in the list above.
Physiology of a Conversation: It’s all well and good to have a pile of these message blocks lying around, but how are they ordered and chained into actual conversations? Another Vantiq Service accomplishes this feat: io.vantiq.ai.ConversationMemory! Search the list of built-in procedures for this Service for clues:
• ConversationMemory.startConversation(initialState io.vantiq.ai.ChatMessage Array, conversationId String): String
• ConversationMemory.addChatMessages(conversationId String Required, newMessages io.vantiq.ai.ChatMessage Array Required): io.vantiq.ai.ChatMessage Array
• ConversationMemory.endConversation(conversationId String Required)
• ConversationMemory.getConversation(conversationId String Required): io.vantiq.ai.ChatMessage Array
Use Case Example : Grease Monkey Advice Application
What it does: Users flip open this mobile app, choose the make and model of their vehicle, then ask the “Grease Monkey” (ahem, AI) for advice:
Upon submitting the initial query, the App perceives the beginning of a new conversation, filtered down the “NewConvo” channel due to the absence of a “convoID”:
We’re managing the conversation programmatically in VAIL blocks within the App. Here’s the code for CreateConvo:
var convoId = Math.random(0, 100000)
var sysStart = ChatMessage.buildSystemMessage(“You are a very humorous and helpful car mechanic.”)
var humStart = ChatMessage.buildHumanMessage(event.message)
var cArray = [ sysStart, humStart ]
var convoId = ConversationMemory.startConversation(cArray, toString(convoId))
event.convoID = convoId
event.messages = cArray
As you can see, by retrieving a random number, from a large pool, for the conversationID, we are taking pains to make sure no one else accessing the applicaton at the same time will become part of the same conversation. Also, since the parameter expected for the conversationID needs to be a string, you’ll see that we’re consistently converting our convoID variable to one in the procedure arguments.
If you’re curious, maintaining the string content in an array as part of the event is not necessary – it was just my lazy way of having the strings available for display in the Client.
The AskTheMonkey task is the actual AI involvement, the response from which gets added to the ongoing conversation in AddToConvo:
var AIMsg = ChatMessage.buildAIMessage(event.llmResponse)
var cArray = ConversationMemory.addChatMessages(toString(event.convoID), [AIMsg])
var lArray = ConversationMemory.getConversation(toString(event.convoID))
event.messages = lArray
…before being sent back to the Client:
If the user has a follow-up question, the application adds that to the ongoing conversation, queries the AI, adds the AI response to the conversation, and around and around we go.
Throughout, we’re using a “builder” procedure from the ChatMessage service to convert the last bit of “dialog” into a ChatMessage; then the ConversationMemory service to place that dialog into the conversation order.
At any time, a call to ConversationMemory.getConversation will retrieve the whole exchange, for use by the AI to keep the context of the thread, or by humans to recall what was said before.
In this use case, a great way to test whether or not the AI “remembers” the whole of the conversation is to ask something like, “What kind of car do I have?” and see if the response correctly identifies the make and model that was provided at the beginning.
Summary: The judicious use of two Vantiq Services: io.vantiq.ai.ChatMessage and io.vantiq.ai.ConversationMemory, makes it easy to manage and maintain AI contextual conversations!
Attachments:
You must be
logged in to view attached files.
Developer ‘Tiqs & Tricks
January 2024
Managing AI Conversation Memory with Service Procedures
In last month’s ‘Tiqs & Tricks, we looked at an easy and practically automatic way to maintain AI conversation within an App. This month, we’ll look at the programmatic way to keep conversation memory.
Anatomy of a Conversation: Much like a court transcript or a screen play, AI conversations are blocks of dialogue, each containing at minimum two vital pieces of information: 1) Who said something, and 2) What was said
In Vantiq that conversation block is called a ChatMessage, and it has the structure:
{
type : <one of “human,” “ai”, “system”, “chat”, or “function”>,
content : <the string value of the content>,
ars_properties: <optional set of characteristics for later search>
}
There are more properties, but we’ll leave those aside for another blog post on AI functions. (Coming soon, I promise!). The structure of the ChatMessage schema type comes to us courtesy of the io.vantiq.ai.ChatMessage Service, which, not coincidently, also provides a series of “builder” procedures for creating ChatMessages, including:
• ChatMessage.buildSystemMessage(content String Required) : ChatMessage
• ChatMessage.buildHumanMessage(content String Required) : ChatMessage
• ChatMessage.buildAIMessage(content String Required) : ChatMessage
…and so on, for each of the types of ChatMessage in the list above.
Physiology of a Conversation: It’s all well and good to have a pile of these message blocks lying around, but how are they ordered and chained into actual conversations? Another Vantiq Service accomplishes this feat: io.vantiq.ai.ConversationMemory! Search the list of built-in procedures for this Service for clues:
• ConversationMemory.startConversation(initialState io.vantiq.ai.ChatMessage Array, conversationId String): String
• ConversationMemory.addChatMessages(conversationId String Required, newMessages io.vantiq.ai.ChatMessage Array Required): io.vantiq.ai.ChatMessage Array
• ConversationMemory.endConversation(conversationId String Required)
• ConversationMemory.getConversation(conversationId String Required): io.vantiq.ai.ChatMessage Array
Use Case Example : Grease Monkey Advice Application
What it does: Users flip open this mobile app, choose the make and model of their vehicle, then ask the “Grease Monkey” (ahem, AI) for advice:
Upon submitting the initial query, the App perceives the beginning of a new conversation, filtered down the “NewConvo” channel due to the absence of a “convoID”:
We’re managing the conversation programmatically in VAIL blocks within the App. Here’s the code for CreateConvo:
var convoId = Math.random(0, 100000)
var sysStart = ChatMessage.buildSystemMessage(“You are a very humorous and helpful car mechanic.”)
var humStart = ChatMessage.buildHumanMessage(event.message)
var cArray = [ sysStart, humStart ]
var convoId = ConversationMemory.startConversation(cArray, toString(convoId))
event.convoID = convoId
event.messages = cArray
As you can see, by retrieving a random number, from a large pool, for the conversationID, we are taking pains to make sure no one else accessing the applicaton at the same time will become part of the same conversation. Also, since the parameter expected for the conversationID needs to be a string, you’ll see that we’re consistently converting our convoID variable to one in the procedure arguments.
If you’re curious, maintaining the string content in an array as part of the event is not necessary – it was just my lazy way of having the strings available for display in the Client.
The AskTheMonkey task is the actual AI involvement, the response from which gets added to the ongoing conversation in AddToConvo:
var AIMsg = ChatMessage.buildAIMessage(event.llmResponse)
var cArray = ConversationMemory.addChatMessages(toString(event.convoID), [AIMsg])
var lArray = ConversationMemory.getConversation(toString(event.convoID))
event.messages = lArray
…before being sent back to the Client:
If the user has a follow-up question, the application adds that to the ongoing conversation, queries the AI, adds the AI response to the conversation, and around and around we go.
Throughout, we’re using a “builder” procedure from the ChatMessage service to convert the last bit of “dialog” into a ChatMessage; then the ConversationMemory service to place that dialog into the conversation order.
At any time, a call to ConversationMemory.getConversation will retrieve the whole exchange, for use by the AI to keep the context of the thread, or by humans to recall what was said before.
In this use case, a great way to test whether or not the AI “remembers” the whole of the conversation is to ask something like, “What kind of car do I have?” and see if the response correctly identifies the make and model that was provided at the beginning.
Summary: The judicious use of two Vantiq Services: io.vantiq.ai.ChatMessage and io.vantiq.ai.ConversationMemory, makes it easy to manage and maintain AI contextual conversations!
Attachments:
You must be logged in to view attached files.