Using Tools

One of the most powerful features in Ailoy is the tool calling system. It allows you to extend the capabilities of your LLM by connecting it to external tools or APIs. This way, the agent can access real-time or domain-specific information, even if it wasn’t part of the model’s training data.

For example, you can attach weather or location services to your agent. You could also build a financial decision-making app using real-time exchange rates or stock prices. Moreover, the agent can execute financial decisions itself with LLM, enabling powerful automation.

How Tool Calling Works

Let's take a quick look at how tool calling works in general. In most agent system, tool calling can be achieved by the following process.

1. [Tool Description] Assistant (or LLM) can recognize a tool based on its description.
2. [User Query] User provides an input prompt to the assistant.
3. [Tool Call] If that query is related to a tool, assistant can invoke the tool through a specified format.
4. [Tool Result] Tool performs its task as invoked and returns the result.
5. [Assistant Response] Assistant can incorporate the tool's output to produce a more accurate answer.

Building an Agent with Tool Support

Now, let's see how to make a tool-aware agent in Ailoy.

How to add tools from presets

In this example, we’ll use the Frankfurter API to add functionality to lookup real-time currency exchange rates into your Agent.

The Frankfurter API tool is already defined as preset and included in the library along with the other useful tools. So you just can call add_tools_from_preset() with the tool name to attach the tool into the Agent.

Some presets can contain multiple tools which work together.

Python

agent.add_tools_from_preset("frankfurter")

JavaScript(Node)

agent.addToolsFromPreset("frankfurter");

How to notice tool calls

You can find out what tool the agent invoked and what result it produced through the response generated by agent.query().
The type field included in the response shows what type of response it is, and the content field has the content regardless of the type.

For more information on response format, refer to the Agent Response Format document.

Python

for resp in agent.query(question):
    # skip printing when the response is related to tool calls
    if resp.type in ("tool_call", "tool_call_result"):
        continue
    agent.print(resp)

JavaScript(Node)

for await (const resp of agent.query(question)) {
  // skip printing when the response is related to tool calls
  if (resp.type in ["tool_call", "tool_call_result"])
    continue
  agent.print(resp);
}

Example code

Let's see the entire example code using the Frankfurter API tool.

Python

from ailoy import Runtime, Agent

rt = Runtime()

with Agent(rt, model_name="Qwen/Qwen3-8B") as agent:
    # Attach frankfurter's API
    agent.add_tools_from_preset("frankfurter")

    question = "I want to buy 250 U.S. Dollar and 350 Chinese Yuan with my Korean Won. How much do I need to take?"
    print(f"Question: {question}\n")
    for resp in agent.query(question):
        if resp.type == "output_text":
            print(resp.content, end='')
        elif resp.type == "tool_call":
            tool_name = resp.content.function.name
            tool_arguments = resp.content.function.arguments
            print(f"Tool call: {tool_name}({', '.join(f'{k}="{v}"' for k, v in tool_arguments.items())})\n")
        elif resp.type == "tool_call_result":
            tool_result = resp.content
            print(f"Tool result: {tool_result.content}\n")
        else:
            raise TypeError("Unexpected response type.")
    print()

rt.stop()

JavaScript(Node)

import { startRuntime, defineAgent } from "ailoy-node";

(async () => {
  const rt = await startRuntime();

  const agent = await defineAgent(rt, "Qwen/Qwen3-8B");

  // Attach frankfurter's API
  agent.addToolsFromPreset("frankfurter");

  const question = "I want to buy 250 U.S. Dollar and 350 Chinese Yuan with my Korean Won. How much do I need to take?";
  console.log(`Question: ${question}\n`);
  for await (const resp of agent.query(question)) {
    if (resp.type === "output_text") {
      process.stdout.write(resp.content);
    } else if (resp.type === "tool_call") {
      const toolName = resp.content.function.name;
      const toolArguments = resp.content.function.arguments;
      const argString = Object.entries(toolArguments).map(([k, v]) => `${k}="${v}"`).join(", ");
      console.log(`Tool call: ${toolName}(${argString})\n`);
    } else if (resp.type === "tool_call_result") {
      const toolResult = resp.content;
      console.log(`Tool result: ${toolResult.content}\n`);
    } else {
      throw new TypeError("Unexpected response type.");
    }
  }

  console.log();

  await agent.delete();

  await rt.stop();
})();

You can see that the agent uses the Frankfurter API to incorporate real-time exchange rate information into its response.

Here's what the output will look like:

Question: I want to buy 250 U.S. Dollar and 350 Chinese Yuan with my Korean Won. How much do I need to take?

Tool call: frankfurter(symbols="USD,CNY", base="KRW")

Tool result: {"CNY": 0.00518, "USD": 0.00072}

To buy 250 U.S. Dollars (USD) and 350 Chinese Yuan (CNY) using Korean Won (KRW), you need to calculate the total amount of KRW required based on the exchange rates:

- **1 USD = 0.00072 KRW**
- **1 CNY = 0.00518 KRW**

### Calculation:
- **For USD**:  
$ 250 \, \text{USD} \times \frac{1}{0.00072} = 250 \times 1388.89 = 347,222.22 \, \text{KRW} $

- **For CNY**:  
$ 350 \, \text{CNY} \times \frac{1}{0.00518} = 350 \times 193.18 = 67,613.39 \, \text{KRW} $

### Total:
- **KRW needed**: $ 347,222.22 + 67,613.39 = 414,835.61 \, \text{KRW} $

You will need approximately **414,835.61 KRW** to buy 250 USD and 350 CNY.

warning

Keep in mind: Tools aren't free — every token counts.

Using many tools can generate a large amount of information for the AI to process, potentially leading to longer context lengths and decreased performance.

Whether you're running AI through external APIs or using on-device models, it consumes resources. API usage can lead to unexpectedly high bills, while on-device AI may slow down your machine or even cause it to crash. Whether you're using external APIs or running models on-device, resources are consumed. API usage might result in unexpectedly high costs, while on-device models could slow down your machine or even cause it to crash.

Avoid using unnecessary tools. Keep your chat context focused and concise.

Defining Your Own Tool

If the tool you are looking for doesn't exist, or you want more fine-grained control, you can define your own tool through the simple APIs of Ailoy.

If you are using Python, you can define your own tool as a python function. Just write a function with google-style docstring, and add it as a tool.

Python

def get_current_temperature(location: str, unit: str):
    """
    Get the current temperature at a location.

    Args:
        location: The location to get the temperature for, in the format "City, Country"
        unit: The unit to return the temperature in. (choices: ["celsius", "fahrenheit"])
    Returns:
        The current temperature at the specified location in the specified units, as a float.
    """
    if unit == "celsius":
        return 25
    elif unit == "fahrenheit":
        return 77
    return

with Agent(...) as agent:
    agent.add_py_function_tool(get_current_temperature)  # Just pass the function!

note

You can add tools with various behaviors such as REST API, as well as Python functions.
To see more details of using tools, refer to the Tools section.