My current project requires us to create multiple agents running complex algorithms with large volumes of data.
The algorithms work on large datasets and compute outcomes to be used in the next steps of our workflow. Our data scientists are creating these algorithms, and they’re most comfortable in Python, so that’s what our application is running as well, in a setup similar to the one I created in my Trial and Error repository.
If you head to this repository, you’ll notice a file called large_data_analysis.py. This file is responsible for creating agents that are capable of working with large volumes of data.

As you probably know, context windows for agents are limited. They are growing at a rapid pace, but ideally you keep the context small.
When you need to work with millions of records from a repository, you can put them in the agent’s context, but you’ll notice the limitations of this practice quite quickly. Either the agent will start hallucinating or forget details, or you’ll be prompted with an error stating that the context is too large. Both scenarios are disastrous when relying on the agent and its outcome.

Have tools query your repository

In my previous post, you could see how to create tools that can be used in agents.
Oftentimes, these tools require data to work with. If the data already exists inside the prompt, that’s great. The language model will take care of passing the correct parameters based on the descriptions.

I’m a big fan of the coding agents we have at our disposal. I use OpenCode a lot myself, switch to GitHub Copilot regularly, and several of my colleagues use Claude Code.

All of these coding agents have similar capabilities, but they work slightly different. When you are doing the same thing over and over again, it makes sense to create generic commands for it. Some examples can also be found in the Awesome Copilot repository.
Another very useful feature is Agent Skills. Commands and skills overlap in some areas, but they differ in an important way.

The description for commands is:

Custom commands let you specify a prompt you want to run when that command is executed in the TUI.

The description for skills is:

Agent skills let OpenCode discover reusable instructions from your repo or home directory. Skills are loaded on-demand via the native skill tool—agents see available skills and can load the full content when needed.

Can you spot the difference? It’s this: “loaded on-demand”.
Skills are only used when the agent determines they are relevant, so the frontmatter fields are important.
You can also grant permissions to skills, which saves you from having to press the “Allow” button every time.

Skills with scripts

One lesser-known feature is that skills can invoke scripts.
You might be asking yourself, “Why would I do that?” The main reason is consistency.
All of the agents we use are capable of creating and invoking scripts themselves. However, those scripts may differ from one session to another, which can lead to inconsistent results.
Another benefit is that it reduces token usage and compute. If a script already exists, the model does not need to create one first. I’m sure you can think of many repetitive tasks you do during the day where an agent could help.

You know what’s cool? Having agents talk to each other and letting them figure out how to get to the answer you’re looking for.
One way to do this is by using the Agent-to-Agent protocol in your application. Version 0.3.0 is the latest released version, and there’s an RC v1.0 available already.
The Microsoft Agent Framework (MAF) also has an implementation of this protocol available. The current version of the MAF packages, 1.0.0b260130 at the time of writing, isn’t compatible with the proposed changes of 1.0, but I’m pretty sure this will be supported in upcoming releases. The team is adding and changing features quite quickly. There are also newer versions of MAF available now, but I have yet to validate those.

In my current project, we’re creating a dozen agents, each doing its own little thing. What we could do is create some workflow or state machine, invoking each agent in turn, much like the good old days. However, we don’t always need to run every agent or run them in a specific order. While it is possible to add this dynamic nature to an application, we can also leverage the power of a language model for this. Based on the knowledge of what an agent can do, the language model can figure out which agents to invoke and in what order. The A2A protocol can help here.

GitHub Copilot is great, but somehow I get better results and a nicer UX/DX with OpenCode. It integrates with all the models I have available via GitHub Copilot and more.
There’s also a great ecosystem around this software and the documentation is quite good as well. The Awesome OpenCode repository lists quite a few useful tools, plugins and agents. Currently, I use the Smart Title Plugin and Open Agents Control, but Oh My OpenCode also has my interest even though it has a bit of overlap.

All of this is awesome and greatly enhances my joy in creating solutions during my dayjob and side projects. However, it does cost quite a bit of (premium) requests. Especially when using the rather expensive (and good) models.

What is great about OpenCode is you can connect it to your models deployed in Azure too!
This way, when your premium requests are all used up, you can use a GPT-Codex or Kimi model deployed in your own environment. Or even when you do still have premium requests available, you can offload your questions to a model of your choice.

When I was searching for the correct configuration I couldn’t find the correct documentation for this, so decided to post it over here.
If you have OpenCode installed, there should be a folder with an opencode.json file on your system. On a Mac it’s at ~/.config/opencode/opencode.json. In this file you can add a property called providers, if it doesn’t exist already, and configure your models.

It looks like everyone is creating agents nowadays. Most of the time with elaborate prompts to tell a language model what it should do.
Great, but we all know a language model isn’t good at doing everything. Also, I don’t want it to do everything either as it would need to be granted access to every possible resource in my environment.

To extend the capabilities of an agent (and the underlying language model), you can provide tools. With good tool documentation, agents are empowered to do more. Take, for example, counting how many times the letter r appears in Strawberry. Browsing around the internet, it looks like this is one of those important life questions everyone wants an answer to.

The problem is that people are asking this question to a model that’s good at language and guessing what the best possible output should be, not a model that’s good at counting.

What I did was create an agent that’s empowered with a few tools capable of doing simple math, just to see how this is supposed to work. I’m using the Microsoft Agent Framework. The team isn’t shying away from making breaking changes with every release, so newer (and older) versions may need a different implementation. Currently, I’m working in Python with the agent-framework-azure-ai package, version 1.0.0b260130.

In an earlier post I wrote about using Semantic Kernel to create an Agentic AI solution, all using C#. Of course, similar flows can be created with Python. To try this, I’ve created a sample solution to update a resume so it’s more likely to pass the ATS requirements used by various companies nowadays.

My sample is heavilly inspired by Gian Paolo Santopaolo his CV-Pilot repository, which I was not able to use due to the CrewAI tooling phoning home and my DNS (PiHole) blocking those requests. Even after disabling the tracking features, the library/ies still tracked ‘something’, which caused the logic to break so I decided to create something myself using Semantic Kernel.

What did I create?

A tool/flow to update a resume so it can pass the ATS (Applicant Tracking System), based on the job description. These systems ofen check for specific wording and skills. For a human, it takes quite a bit of time to (re)write a resume to pass the ATS requirements. It’s the perfect job for a LLM to perform, as it’s built to create text based on other text input.

You can try doing this in a single prompt, but there’s a high probability this won’t perform the way you like. In my sample, I’ve created a ‘Project Manager’-, ‘Job Market Analyst’- and ‘Strategist’-agent. Each with their own specific job and goals.

We are finally at a state in the GenAI-space where we can create agentic AI solutions with ease.
I’m most familiar with Semantic Kernel, when working with LLMs, and this library works great for creating these solutions.

In a nutshell, what you need to do is create a group chat, add your agents to it, and then let them work together to solve a problem.
Do keep in mind, at the time of this writing, version 1.58.0 of the Semantic Kernel library is used. Development is going fast in this space, so behavior might change in future versions.
For my proof of concept, I’ve created a simple solution capable of creating a Fibonacci sequence, validate if a number is part of that sequence or answer random questions you would also ask to a regular LLM-powered chatbot. If you’re interested, the full sourcecode can be found on GitHub in my console-agent repository.

Create the agents

Agents are created with the ChatCompletionAgent class. You should provide the instructions & the kernel to use.
The instructions is just a regular prompt we all know and love when working with an LLM and specifies what the agent should do. I won’t repeat it over here as it’s not relevant for this post.

I’m very happy GitHub Copilot exists and lately with the Agent-mode it’s even better. It’s making sure I can focus on the relevant pieces of my solutions and not have to worry too much about the plumbing part.

The models it’s using are quite powerful and contains a lot of (old) data. When using new libraries or versions of already existing libraries, the LLMs used under the hood often don’t provide useful suggestions or edits. Currently, telling GitHub Copilot to check the repository of the library to as reference material doesn’t work the way I’d like it to.
I’ve tried this, but it only fetched the contents of the readme file of the repository and didn’t traverse all files and searching for relevant bits.

What I’d like is for GitHub Copilot to index the entire repository/repositories of my libraries of choice and provide me suggestions or edits based on the codebase or documentation.
I figured this can be achieved by adding this capability as an MCP server. Turns out, I’m not the only one who wants this. Leonid Bugaev has created an MCP server solution, docs-mcp, that can be used to search through repositories. It’s powered by Probe, also created by him.

Create an MCP server to search in repositories

The readme of the docs-mcp repository contains all the information to set up your MCP server.

The current large language models, like GPT-4, GPT-4 Turbo and GPT-4o are great when you need some output generated based on data you feed in the prompt. Even the small language models, like Phi-3, are doing a great job at this. However, these models often don’t know a lot about the data within your company. Because of this, they can’t do a good job at answering questions that required data from your organization.

There is of course the M365 Copilot available, which is able to index all of the organization its data and provide answers based on it. On a high level, what this is doing, is using Retrieval-Augmented Generation (RAG). There’s a decent post about this on the IBM Research site and there’s also a good post on the AWS site on it.

By using RAG in combination with your LLM, you are able to index your own data and let the model interpret it.
A great way to get started with this, is by using the Azure Open AI Assistants feature. The MS Learn page on this topic is quite good. If you’re interested in the topic, I’d suggest to check it out: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/assistant.

Get your data

The first thing you need to do is make sure all data is available to the assistant. At this moment, there’s a large list of supported file types, like docx, pptx, pdf, png, txt, etc. The most important file types for us engineers are CSV, JSON, and XML, because these are able to hold (semi-)structured data so the LLM can infer relationships and create appropriate answers.