It looks like everyone is creating agents nowadays. Most of the time with elaborate prompts to tell a language model what it should do.
Great, but we all know a language model isn’t good at doing everything. Also, I don’t want it to do everything either as it would need to be granted access to every possible resource in my environment.

To extend the capabilities of an agent (and the underlying language model), you can provide tools. With good tool documentation, agents are empowered to do more. Take, for example, counting how many times the letter r appears in Strawberry. Browsing around the internet, it looks like this is one of those important life questions everyone wants an answer to.

The problem is that people are asking this question to a model that’s good at language and guessing what the best possible output should be, not a model that’s good at counting.

What I did was create an agent that’s empowered with a few tools capable of doing simple math, just to see how this is supposed to work. I’m using the Microsoft Agent Framework. The team isn’t shying away from making breaking changes with every release, so newer (and older) versions may need a different implementation. Currently, I’m working in Python with the agent-framework-azure-ai package, version 1.0.0b260130.

Normally, I’d refer people to the documentation and samples provided on Microsoft Learn and GitHub. That isn’t very useful advice for this purpose though, because both were out of date, and it took me some figuring out how to do this with the current release(s) of the packages.

The groundwork

First, set up the required objects.

# The relevant imports
from agent_framework import tool
from agent_framework.azure import AzureAIAgentsProvider
from azure.ai.agents.aio import AgentsClient
from azure.identity.aio import AzureCliCredential

# Set up the groundwork
async with (
    AzureCliCredential() as credential,
    AgentsClient(
        endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"], credential=credential
    ) as agents_client,
    AzureAIAgentsProvider(credential=credential) as provider,
):
    agent_instructions = """You are a calculator agent with access to the following tools:
    1. count_letters(character, phrase) - Counts how many times a specific character appears in a word or phrase
    2. calculate_square_root(number) - Calculates the square root of a number

    IMPORTANT: You MUST use these tools to solve problems. Follow these rules:
    - When asked to count characters/letters in a word or phrase, ALWAYS call the count_letters tool
    - When asked to calculate square roots, ALWAYS call the calculate_square_root tool
    - If a question requires multiple steps (e.g., "find the square root of the count"), call the tools in sequence:
        * First, call count_letters to get the count
        * Then, call calculate_square_root with the result from count_letters
    - NEVER guess or manually calculate - always use the provided tools
    - In your final response, explain which tools you used and show the chain of calculations
    """

The important part above is creating the AgentsClient and the AzureAIAgentsProvider. There are different ways (according to the samples and docs) to implement agents with tools, but I found these objects worked for the package version I tried.

Create an agent with tools

With the above, you should be able to create an agent.

calculator_agent = await agents_client.create_agent(
    model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
    name="CalculatorAgent",
    instructions=agent_instructions,
    # Adding this yields an error: `TypeError: ClientSession._request() got an unexpected keyword argument 'default_options'`.
    # default_options={"response_format": calculator_response},
    # Adding tools over here yields an error: `Object of type FunctionTool is not JSON serializable`.
    # tools= [count_letters]
)

According to the (current) samples, create_agent is also capable of retrieving an already existing agent. Based on my experience, this isn’t true. It creates a new agent every time it runs. To retrieve an existing agent, you need its identifier, which is another round trip to the Foundry project.
This will probably improve in later versions of the package.

Notice the default_options and tools in comments?
According to some samples, this is required to add tools to the agent. I found this not to be true either. Adding them broke the application with different types of errors.

What did work is to invoke the get_agent method and provide the tools there.

agent = await provider.get_agent(
    calculator_agent.id, tools=[count_letters, calculate_square_root]
)

The tools aren’t very fancy in itself. Just a method that counts characters and one that calculates the square root of a number. What’s important is to provide descriptions of every parameter and what the method does. That way, the language model can determine if—and when—to use the tools.

@tool(approval_mode="never_require")
def count_letters(
    character: Annotated[
        str, Field(description="The character that needs to be counted in the string.")
    ],
    phrase: Annotated[
        str, Field(description="The word or phrase that needs its characters counted.")
    ],
) -> int:
    """Count the number of specified characters in a specific word or phrase"""
    counted_characters = phrase.count(character)
    return counted_characters


@tool(approval_mode="never_require")
def calculate_square_root(
    number: Annotated[
        float, Field(description="The number you want the square root to be calculated for.")
    ],
) -> float:
    """Calculate the square root of the provided number and return it."""
    square_root = sqrt(number)
    return square_root

Ask the all-important question

You can now ask the most important question of all by invoking the run method.

answer = await agent.run(question, options={"response_format": calculator_response})
return answer.value

I’m using a Pydantic model, just to get some more information about the process. This can be omitted if you prefer.

class calculator_response(BaseModel):
    """Structured calculator response"""

    final_number: float
    reasoning: str
    chain_of_thought: str
    answer: str
    model_config = ConfigDict(extra="forbid")

I’ve asked a slightly more complicated question to the agent:

“I want to know the square root of the number of times the letter r is found in Strawberry?”

And do you want to know the answer?

{
  "finalNumber": 1.7320508075688772,
  "reasoning": "I counted the letter 'r' in the word 'Strawberry' and found it 3 times. Then, I calculated the square root of that count, which is approximately 1.732.",
  "chainOfThought": "First, I used the count_letters tool to count how many times 'r' appears in 'Strawberry', which resulted in a count of 3. Next, I used the calculate_square_root tool to find the square root of 3, resulting in approximately 1.732.",
  "answer": "The square root of the number of times the letter 'r' is found in 'Strawberry' is approximately 1.732."
}

The chainOfThought has a nice write-up of what the agent did to come to this answer.

To conclude

As you can see from the above example, working with tools isn’t hard and empowers your agents to do quite a lot of useful stuff.
In my current project, we’re using it to do some tough calculations on big datasets. We retrieve these datasets via tools and also run the algorithms via tools. That way, we know for sure the language model doesn’t make up an algorithm with a slightly off implementation. If you use this wisely, it can also save you quite a few tokens and improve the overall performance of your agents.

Do use this with care, as you don’t want to limit your agent to only using tools. If you did that, you could just as well create a ’normal’ application.