Skip to main content

Using Tools

One of the most powerful features in Ailoy is the tool-calling system. It allows you to extend your LLM’s capabilities by connecting it to external tools or APIs. This way, the agent can access real-time or domain-specific information, even if it wasn’t part of the model’s training data.

To use a tool, two components must be defined: Tool Description and Tool Behavior.

note

Please refer to the Architecture section for more about the tool architecture.

Let's explore how to make a tool-aware agent in Ailoy.

Using Tools​

In this example, we’ll use the Frankfurter API to enable your agent to look up real-time currency exchange rates.

note

Ailoy also supports MCP as a tool. Please refer to the MCP Integration section for using MCP tools.

Defining the Tool Description​

A tool description defines how the tool is exposed to the model. It includes the tool’s name, purpose, parameters, and optionally its return types. This schema helps the model understand how to invoke the tool correctly.

Parameters are typically expressed in JSON Schema format.

info

Refer to OpenAI's documentation or Hunggingface's documentation for more details on defining tools in JSON Schema format.

import ailoy as ai

tool_desc = ai.ToolDesc(
    name="frankfurter",
    description="Get the latest currency exchange rates of target currencies based on the 'base' currency",
    parameters={
        "type": "object",
        "properties": {
            "base": {
                "type": "string",
                "description": "The ISO 4217 currency code to be the divider of the currency rate to be got."
            },
            "symbols": {
                "type": "string",
                "description": "The target ISO 4217 currency codes separated by comma."
            }
        },
    }
)

Defining the Tool Behavior​

A tool behavior defines what the tool actually does when invoked. It represents the executable logic that runs when the model calls the tool.

The function receives the parameters specified in the tool description and returns a result that matches the expected schema.

When the agent runs, the model decides when and how to invoke this function based on the conversation context and the provided tool description.

def tool_behavior(base, symbols):
    if not base:
        raise ValueError("Missing 'base'")
    if not symbols:
        raise ValueError("Missing 'symbols'")

    query = parse.urlencode({"from": base, "to": symbols})
    url = f"https://api.frankfurter.app/latest?{query}"

    try:
        with request.urlopen(url, timeout=10) as resp:
            if resp.status != 200:
                raise RuntimeError(f"Frankfurter API returned HTTP {resp.status}")
            payload = json.loads(resp.read().decode("utf-8"))
    except error.URLError as e:
        raise RuntimeError(f"Failed to reach Frankfurter API: {e}") from e
    except json.JSONDecodeError as e:
        raise RuntimeError("Failed to parse Frankfurter API response as JSON.") from e

    return payload

Registering the Tool with the Agent​

Once a tool is defined, the Agent automatically detects when the model wants to invoke it and executes the corresponding function during the next iteration.

You can register a tool by passing it to the Agent constructor.

lm = await ai.LangModel.new_local("Qwen/Qwen3-4B")
tools = [ai.Tool.new_py_function(tool_behavior, tool_desc)]
agent = ai.Agent(lm, tools)

Alternatively, you can register the tools after creating the agent by using the add_tool or add_tools methods.

agent = ...

# Add a single tool
tool = ai.Tool.new_py_function(...)
agent.add_tool(tool)

# Add multiple tools
tool1 = ai.Tool.new_py_function(...)
tool2 = ai.Tool.new_py_function(...)
agent.add_tools([tool1, tool2])

You can check which tool the agent invoked and what result it produced from the response returned by agent.run() or agent.run_delta(). The agent.run() function includes a field called tool_calls. If this field contains any values, it indicates that the LLM’s output has triggered one or more tool calls.

Auto-generate Tool Description (only in Python)​

You can access a function’s metadata in Python through its function object.
This allows Ailoy to automatically generate tool descriptions from the function’s parameter type hints and its docstring.

Similarly to libraries like transformers, Ailoy can automatically create a tool description when you define a Tool from a Python function that includes both Google-style docstrings and type hints for its parameters.
In this case, you don’t need to manually create a ToolDesc.

See example Python code
def frankfurter(base: str, symbols: str):
    """
        Get the latest currency exchange rates of target currencies based on the 'base' currency

        Args:
            base: The ISO 4217 currency code to be the divider of the currency rate to be got.
            unit: The target ISO 4217 currency codes separated by comma.
    """

    if not base:
        raise ValueError("Missing 'base'")
    if not symbols:
        raise ValueError("Missing 'symbols'")

    query = parse.urlencode({"from": base, "to": symbols})
    url = f"https://api.frankfurter.app/latest?{query}"

    try:
        with request.urlopen(url, timeout=10) as resp:
            if resp.status != 200:
                raise RuntimeError(f"Frankfurter API returned HTTP {resp.status}")
            payload = json.loads(resp.read().decode("utf-8"))
    except error.URLError as e:
        raise RuntimeError(f"Failed to reach Frankfurter API: {e}") from e
    except json.JSONDecodeError as e:
        raise RuntimeError("Failed to parse Frankfurter API response as JSON.") from e

    return payload

tool = ai.Tool.new_py_function(tool_behavior)  # don't need to provide `tool_desc` here
# tool = ai.Tool.new_py_function(tool_behavior, tool_desc)  # if you provide, it will override
agent.add_tool(tool)

Complete Example​

import asyncio
import json
from urllib import error, parse, request

import ailoy as ai

tool_desc = ai.ToolDesc(
    name="frankfurter",
    description="Get the latest currency exchange rates of target currencies based on the 'base' currency",
    parameters={
        "type": "object",
        "properties": {
            "base": {
                "type": "string",
                "description": "The ISO 4217 currency code to be the divider of the currency rate to be got.",
            },
            "symbols": {
                "type": "string",
                "description": "The target ISO 4217 currency codes separated by comma.",
            },
        },
    },
)


def tool_behavior(base, symbols):
    if not base:
        raise ValueError("Missing 'base'")
    if not symbols:
        raise ValueError("Missing 'symbols'")

    query = parse.urlencode({"from": base, "to": symbols})
    url = f"https://api.frankfurter.app/latest?{query}"

    try:
        with request.urlopen(url, timeout=10) as resp:
            if resp.status != 200:
                raise RuntimeError(f"Frankfurter API returned HTTP {resp.status}")
            payload = json.loads(resp.read().decode("utf-8"))
    except error.URLError as e:
        raise RuntimeError(f"Failed to reach Frankfurter API: {e}") from e
    except json.JSONDecodeError as e:
        raise RuntimeError("Failed to parse Frankfurter API response as JSON.") from e

    return payload


async def main():
    lm = await ai.LangModel.new_local("Qwen/Qwen3-4B", progress_callback=print)
    tools = [ai.Tool.new_py_function(tool_behavior, tool_desc)]
    agent = ai.Agent(lm, tools)
    query = "I want to buy 250 U.S. Dollar and 350 Chinese Yuan with my Korean Won. How much do I need to take? Get the currency data if you need."
    print("Query:", query, "\n")
    async for resp in agent.run(query):
        if resp.message.role == "assistant":
            if len(resp.message.tool_calls) > 0:
                print("Tool call:", resp.message.tool_calls[0].function, "\n")
            else:
                print(resp.message.contents[0].text, "\n")
        elif resp.message.role == "tool":
            print("Tool response:", resp.message.contents[0].value, "\n")


if __name__ == "__main__":
    asyncio.run(main())

Output​

You can see that the agent uses the Frankfurter API to include real-time exchange rate information in its response.

Here's what the output will look like:

Query: I want to buy 250 U.S. Dollar and 350 Chinese Yuan with my Korean Won. How much do I need to take?

Tool call: frankfurter(symbols="USD,CNY", base="KRW")

Tool response: {"CNY": 0.00518, "USD": 0.00072}

To buy 250 U.S. Dollars (USD) and 350 Chinese Yuan (CNY) using Korean Won (KRW), you need to calculate the total amount of KRW required based on the exchange rates:

- **1 USD = 0.00072 KRW**
- **1 CNY = 0.00518 KRW**

### Calculation:
- **For USD**:  
$ 250 \, \text{USD} \times \frac{1}{0.00072} = 250 \times 1388.89 = 347,222.22 \, \text{KRW} $

- **For CNY**:  
$ 350 \, \text{CNY} \times \frac{1}{0.00518} = 350 \times 193.18 = 67,613.39 \, \text{KRW} $

### Total:
- **KRW needed**: $ 347,222.22 + 67,613.39 = 414,835.61 \, \text{KRW} $

You will need approximately **414,835.61 KRW** to buy 250 USD and 350 CNY.
warning

Keep in mind: Tools are not free — every token counts.

Excessive tool usage can cause large amounts of information for the AI to process, potentially leading to longer context lengths and decreased performance.

Each tool call consumes tokens. API calls may lead to unexpected costs, while on-device models can slow down your machine or even cause crashes.

To avoid inefficiency, use only the tools you need and keep the chat context focused and concise.

Using Builtin Tools​

Ailoy provides several builtin tools out-of-the-box. You don't need to define tool description and behavior to use these tools.

See Resources > Builtin Tools for the list of available builtin tools.

Creating the Tool​

In this example, we use terminal tool to execute shell commands.

info

Note that terminal tool is not available on web environment.

You can create the builtin tool as follow:

import ailoy as ai

tool = ai.Tool.new_builtin("terminal")

Registering the Tool​

agent = ...

agent.add_tool(tool)

Complete Example​

import asyncio

import ailoy as ai


async def main():
    lm = await ai.LangModel.new_local("Qwen/Qwen3-4B", progress_callback=print)
    agent = ai.Agent(lm)

    tool = ai.Tool.new_builtin("terminal")
    agent.add_tool(tool)

    query = "List the files and directories in the home directory."
    print(f"Query: {query}\n")
    async for resp in agent.run(query):
        if resp.message.role == "assistant":
            if len(resp.message.tool_calls) > 0:
                print("Tool call:", resp.message.tool_calls[0].function, "\n")
            else:
                print(resp.message.contents[0].text, "\n")
        elif resp.message.role == "tool":
            print("Tool response:", resp.message.contents[0].value, "\n")


if __name__ == "__main__":
    asyncio.run(main())

Output​


List the files and directories in the current directory, including hidden files.
Tool call: { name: 'terminal', arguments: { command: 'ls -a' } } 

Tool response: {
stdout: '.\n' +
  '..\n' +
  '.env.template\n' +
  '.gitignore\n' +
  '.prettierrc.json\n' +
  'npm\n' +
  'package-lock.json\n' +
  'package.json\n' +
  'README.md\n' +
  'scripts\n' +
  'src\n' +
  'tests\n' +
  'tsconfig.json\n' +
  'typedoc.json\n' +
  'vite.config.ts\n' +
  'vitest.config.ts\n',
stderr: '',
exit_code: 0
} 

The files and directories in the current directory, including hidden ones, are as follows:

- `.` (current directory)
- `..` (parent directory)
- `.env.template`
- `.gitignore`
- `.prettierrc.json`
- `npm`
- `package-lock.json`
- `package.json`
- `README.md`
- `scripts`
- `src`
- `tests`
- `tsconfig.json`
- `typedoc.json`
- `vite.config.ts`
- `vitest.config.ts`