Function Calling with GPT Models

GenAI is a powerful tool for generating content and interacting with applications using natural language. However, this tool also has significant limitations when you plan to use it in your own software. GenAI's knowledge is limited to information that is already several months old at the time of the model's release. Moreover, the accuracy of text produced by LLMs cannot always be fully trusted. These shortcomings can be addressed by enhancing LLMs with your own program code. This would allow the software system to mix the natural language capabilities with more traditional functionality of software. For example, if the user asks a shopping advisor for the price of a particular product, as well as a link where it can be bought, most LLMs will indicate that such time-sensitive information is unavailable because the dataset is outdated. The LLM alone is limited to producing natural language and does not have factual and current data. In an ideal case, we would augment the system by feeding this factual information (the link and price) into the context of the model, so that it can use it to generate an answer for the use. To make it possible, the following part could be added to the prompt:

"If the customer asks for detailed information that you cannot answer, answer the request with: 'Details required for product: X', replacing X with the name of the product."

Before the LLM's response is given to the customer, you can check if it contains the pre-formulated part. If this is the case, you can query your database and integrate these details into the LLM's context (e.g., through another 'system message'). The LLM is then presented with the current chat history again, but this time it has the necessary information and can answer the query. Although this method will work in many cases, OpenAI offers so-called function calls with the GPT models and its API, for which the model has been specifically trained to return structured answers. Other LLM operators offer similar functionality. The general idea is, to provide the LLM with the information about functions the software is able to provide. This means that we as the developers tell the LLM what we can do for it, if it needs help. In our example above, we would provide the LLM with a search product function, so that when the user requests information, the LLM knows that the software is able to help. It then can call the function for assistance. Function objects called “tools” can be added to the LLM's context, informing the model that the program can provide the described functionality, enabling the LLM to call it when needed. In code, using the Open AI SDK, a function that provides more detailed information for products might look like this:

1const tools: Array<ChatCompletionTool> = [
2       {
3           type: 'function',
4           function: {
5               name: 'search_on_amazon',
6               description: 'Used to find information about the product on amazon',
7               parameters: {
8                   type: 'object',
9                   properties: {
10                       search_term: {
11                           type: 'string',
12                           description: 'The search term to find the product'
13                       }
14                   },
15                   required: ['search_term']
16               }
17           }
18       }
19   ];
20
21   const completion = await openai.chat.completions.create({
22       messages: systemWithContext.concat(postPrompt),
23       model: 'gpt-3.5-turbo',
24       tools
25   });

The following happens in the code: a tools variable is created, which provides a function. Which is given the name "search_on_amazon" and the description "Used to find information about the product on amazon". Furthermore, parameters are added which the GPT model should fill so that the function can be called in the software. In this example function, the search_term parameter is created, which is of type string and contains the following description "The search term to find the product". The tools variable is then, additionally to the chat history, attached to the create request.

The goal of the information, name, description, and parameters, are not to increase the readability of the code, but are directed at the GPT model. The LLM "reads" these components and "understands" what feature the software provides in which the model is integrated. In this case, the LLM is given the capability to search for products on Amazon with the help of our software. If the GPT model is now confronted with a situation where information about the product is missing, and it "believes" it can find them on Amazon, it will return a response where the parameters are filled that the software needs to be able to search on Amazon. The model is calling our function The response then looks as follows:

1"choices": [
2   {
3     "index": 0,
4     "message": {
5       "role": "assistant",
6       "content": null,
7       "tool_calls": [
8         {
9           "id": "call_HqnWSogcLlQZ7QoUh4uRrW3E",
10           "type": "function",
11           "function": {
12             "name": "search_on_amazon",
13             "arguments": "{\"search_term\":\"Razer Black Widow\"}"
14           }
15         }
16       ]
17     },
18     "logprobs": null,
19     "finish_reason": "tool_calls"
20   }
21 ]

The finish_reason in this response is tool_calls, and the GPT model returns the name of the function (helpful if several Functions exist), as well as the filled parameters. In the program code, it can then be recognised by querying the finish_reason if the answer provided by the LLM can be directly send to the user (finish reason stop) or if the LLM needs help with completing the request and a tool needs to be called. In the example above, the function was used to add extra information to the model, these can come from databases or also live from the Internet. But since the intention behind this functionality is to run your own program code, Function calling can be used to send emails, calculate the answers to mathematical questions, call external API’s or for many other use cases. You can find another small introductory example in our Blog. Software development and product design thus receive practically endless new possibilities with the new tool enhanced LLMs.