Bringing Some Source of Truth to Your NetAI Agentic Playground

[ad_1] Hey everybody, I’m again to exploring how agentic AI may match right into a community engineer’s workflow and change into a useful device in our device chest.

In my weblog submit, Making a NetAI Playground for Agentic AI Experimentation, I started this journey by exploring how we are able to make the most of Mannequin Context Protocol (MCP) servers and the idea of “instruments” to allow our AI brokers to work together with community units by sending present instructions. When you haven’t learn that submit but, undoubtedly test it out as a result of it's some actually fabulous prose. Oh, and there's some actually cool NetAI stuff in there, too. 😉

Whereas it was fascinating to see how nicely AI might perceive a community engineering process introduced in pure language, create a plan, after which execute that plan in the identical means I'd, there was a limitation in that first instance. The one “device” the agent had was the flexibility to ship present instructions to the community gadget. I needed to explicitly present the main points in regards to the community gadget—particulars which can be available in my “supply of reality.”

To understand the facility of agentic AI, NetAI must have entry to the identical info as human community engineers. For right now’s submit, I needed to discover how I might present source-of-truth knowledge to my NetAI agent. So, let’s dig in!

NetBox gives an MCP server NetBox has lengthy been a favourite device of mine. It's an open-source community supply of reality, written in Python, and out there in varied deployment choices. NetBox has been with me via a lot of my community automation exploration; it appeared becoming to see the way it might match into this new world of AI.

Initially, I anticipated to place a easy MCP server collectively to entry NetBox knowledge. I shortly discovered that the group at NetBox Labs had already launched an open-source fundamental MCP server on GitHub. It solely supplies “learn entry” to knowledge, however as we noticed in my first NetAI submit, I’m beginning out slowly with read-only work anyway. Having a place to begin for introducing some supply of reality into my playground was going to considerably pace up my exploration. Completely superior.

Including NetBox to the NetAI playground Have you ever ever been engaged on a venture and gotten distracted by one other “cool thought?” No? I suppose it’s simply me then… 🙂

Like most of my community labs and explorations, I’m utilizing Cisco Modeling Labs (CML) to run the community playground for AI. This wasn’t the primary time I needed to have NetBox as a part of a CML topology. And as I used to be prepping to play with the NetBox MCP server, I had the thought…

Hank, wouldn’t or not it's nice if there have been a CML NetBox node that could possibly be simply added to a topology, and that might mechanically populate NetBox with the topology info from CML?

In fact I answered myself…

Heck yeah, Hank, that’s an amazing thought!

My thoughts instantly began understanding the main points of the right way to put it collectively. I knew it could be tremendous simple and quick to knock out. And I figured different folks would discover it useful as nicely. So I took a “quick detour.”

<img loading="lazy" decoding="async" class="wp-image-476938 size-full" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/cml-netbox-01.png" alt="network topology shows NetBox node in Cisco Modeling Labs (CML) workbench" width="800" height="444" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/cml-netbox-01-300x167.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/cml-netbox-01-768x426.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/cml-netbox-01.png 800w" sizes="auto, (max-width: 800px) 100vw, 800px"/> Including NetBox to my Cisco Modeling Labs (CML) topology I’m positive lots of you raised your eyebrows once I stated “tremendous simple” and “quick.” You have been proper to be skeptical, after all. It wasn’t fairly as simple or easy as I anticipated. Nonetheless, I used to be capable of get it working, and it's actually cool and useful for anybody who desires so as to add not solely a NetBox server to a CML community but in addition have it pre-populated with the units, hyperlinks, and IP particulars from the CML topology.

I nonetheless must compile the documentation for the brand new node definition earlier than I can submit it to the CML-Neighborhood on GitHub for others to make use of. Nonetheless, contemplate this weblog submit my public accountability submit, indicating that it's forthcoming. You'll be able to maintain me to it.

However sufficient of the aspect monitor on this weblog submit, let’s get again to the AI stuff!

Including NetBox MCP server to LM Studio As I discussed within the final weblog submit, I’m utilizing LM Studio to run the Giant Language Mannequin (LLM) for my AI agent domestically on my laptop computer. The principle purpose is to keep away from sending any community info to a cloud AI service. Though I’m utilizing a “lab community” for my exploration, there are particulars within the lab setup that I do NOT wish to be public or threat ending up in future coaching knowledge for an LLM.

If this exploration is profitable, utilizing the method with manufacturing knowledge could be the subsequent step; nonetheless, that's undoubtedly not one thing that aligns with a accountable AI method.

Cloning down the netbox-mcp-server code from GitHub was simple sufficient. The README included an instance MCP server configuration that supplied all the pieces I wanted to replace my mcp.json file in LM Studio so as to add it to my already configured pyATS MCP server.

{ "mcpServers": { "pyats": { "url": "http://localhost:8002/mcp" }, "netbox": { "command": "uv", "args": [ "--directory", "/Users/hapresto/code/netbox-mcp-server", "run", "server.py" ], "env": { "NETBOX_URL": "http://{{MY NETBOX IP ADDRESS}/", "NETBOX_TOKEN": "{{MY NETBOX API TOKEN}}" } } } }

As quickly as I saved the file, LM Studio found the instruments out there.

<img loading="lazy" decoding="async" class="size-full wp-image-476949" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-01.png" alt="NetBox MCP server added to LMStudio" width="800" height="460" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-01-300x173.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-01-768x442.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-01.png 800w" sizes="auto, (max-width: 800px) 100vw, 800px"/> NetBox MCP server added to LM Studio There are three instruments supplied by the NetBox MCP server.

netbox_get_objects: Generic device that bulk retrieves objects from NetBox. It helps “filters” to restrict the returned objects. netbox_get_object_by_id: Device to retrieve a single object of any sort from NetBox given an ID. netbox_get_changelogs: Device to search for audit and alter occasions

I used to be, and proceed to be, within the method utilized by the NetBox Labs of us on this MCP server. Slightly than offering instruments to “get_devices” and “get_ips,” they've a single device. NetBox’s APIs and object mannequin are nicely thought out, and make a generic method like this doable. And it actually means much less code and growth time. Nonetheless, it basically provides API entry to the LLM and shifts the load for “thought” and “processing the info” again to the LLM. As Agentic AI and MCP are nonetheless very new requirements and approaches, there aren’t actual greatest practices and particulars on what works greatest in design patterns right here but. I’ll come again to this method and what I see as some doable downsides afterward within the submit.

I then loaded the newly launched open mannequin by OpenAI, gpt-oss, and despatched the primary question.

<img loading="lazy" decoding="async" class="wp-image-476954 size-full" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-02a.png" alt="Asking AI how many devices are in NetBox" width="800" height="516" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-02a-300x194.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-02a-768x495.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-02a.png 800w" sizes="auto, (max-width: 800px) 100vw, 800px"/> Asking AI what number of units are in NetBox My first thought… Success. After which I scratched my head for a second. 10 units? Scroll again as much as the CML topology picture and depend what number of units are within the topology. Go forward, I’ll wait…

Yeah. I counted seven units, too. And if I verify NetBox itself, it additionally reveals seven units.

<img loading="lazy" decoding="async" class="size-full wp-image-476953" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/netbox-01.png" alt="NetBox shows 7 devices" width="800" height="587" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/netbox-01-300x220.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/netbox-01-768x564.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/netbox-01.png 800w" sizes="auto, (max-width: 800px) 100vw, 800px"/> NetBox reveals seven units So what occurred? LM Studio reveals the precise response from the device name, so I went and checked. Positive sufficient, solely seven units’ value of knowledge was returned. I then remembered that one of many notoriously meme-worthy failings of many AI instruments is the flexibility to depend. Blueberries anybody?

So this changed into a pleasant teachable second about AI… AI is incredible, however it may be incorrect. And it will likely be dangerous with a number of the strangest issues. Keep vigilant, my buddies. 😉

After resolving the difficulty with the ten units, I spent a substantial period of time asking extra questions and observing the AI make the most of the instruments to retrieve knowledge from NetBox. On the whole, I used to be fairly impressed, and gaining access to source-of-truth knowledge might be key to any Agentic NetAI work we undertake. Whenever you do that out by yourself, undoubtedly mess around and see what you are able to do with the LLM and your NetBox knowledge. Nonetheless, I needed to discover what was doable in bringing instruments collectively.

Combining source-of-truth instruments with community operations instruments I needed to begin out with one thing that felt each helpful and fairly easy. So I despatched this immediate.

I might prefer to confirm that router01 is bodily linked to the appropriate units per the NetBox cable connections. > Observe: The credentials for router01 are: `netadmin / 1234QWer` Are you able to: 1. Examine NetBox for what community units router01 is meant to be linked to, and on what interfaces 2. Lookup the Out of Band IP handle and SSH port from NetBox, use these to hook up with router01. 3. Use CDP on router01 to verify what neighbors are seen 4. Examine the NetBox to CDP info.

I nonetheless needed to inform the LLM what the credentials are for the units. That’s as a result of whereas NetBox is a incredible supply of reality, it does NOT retailer secrets and techniques/credentials. I’m planning on exploring what device choices exist for pulling knowledge from secret storage afterward.

If you're questioning why I supplied a listing of steps to deal with this downside slightly than let the LLM “determine it out,” the reply is that whereas GenAI LLMs can appear “sensible,” they're NOT community engineers. Or, extra particularly, they haven’t been skilled and tuned to BE community engineers. Seemingly, the long run will supply tuned LLMs for particular job roles slightly than the general-purpose LLMs of right now. Till then, the very best observe for “immediate engineering” is to supply the LLM with detailed directions on what you need it to do. That dramatically will increase the probabilities of success and the pace at which the LLM can deal with the issue.

Let’s have a look at how the LLM dealt with step one within the request, wanting up the gadget connections.

<img loading="lazy" decoding="async" class="size-full wp-image-476958" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-03.png" alt="The AI plans and executes the cable lookup." width="800" height="381" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-03-300x143.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-03-768x366.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-03.png 800w" sizes="auto, (max-width: 800px) 100vw, 800px"/> The AI plans and executes the cable lookup At first look, this seems to be fairly good. It “knew” that it wanted to verify the Cables from NetBox. Nonetheless, there are some issues right here. The LLM crafted what seems to be a sound filter for the lookup: “device_a_name”: “router01.” Nonetheless, that's truly NOT a sound filter. It's a hallucination.

A whole weblog submit could possibly be written on the rationale this hallucination occurred, however the TL;DR is that the NetBox MCP server does NOT present express particulars on the right way to craft filters. It depends on the LLM to have the ability to construct a filter based mostly on the coaching knowledge. And whereas each LLM has benefited from the copious quantities of NetBox documentation out there on the web, in all of my testing, I've but to have any LLM efficiently craft the proper filter for something however essentially the most fundamental searches for NetBox.

This has led me to begin constructing my very own “opinion” on how MCP servers needs to be constructed, and it includes requiring much less “guessing” from the LLMs to make use of them. I’ll most actually be again extra on this matter in later posts and shows. However sufficient on that for now.

The LLM doesn’t know that the filter was incorrect; it assumes that the cables returned are all linked to router01. This results in different errors within the reporting, because the “Thought” course of reveals. It sees each Cable 1 and Cable 4 as linked to Ethernet 0/0. The reality is that Cable 4 is linked to switch01 Ethernet0/0. We’ll see how this components in later within the abstract of knowledge.

As soon as it has the cable info, the LLM proceeds and completes the remainder of the device’s use to assemble knowledge.

<img loading="lazy" decoding="async" class="size-full wp-image-476965" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-04.png" alt="The AI succeeds in getting CDP information" width="800" height="432" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-04-300x162.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-04-768x415.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-04.png 800w" sizes="auto, (max-width: 800px) 100vw, 800px"/> The AI succeeds in getting CDP info Discovering the Out of Band IP and SSH port was easy. However the first try to run “present cdp neighbors” failed as a result of the LLM initially didn’t use the SSH port as a part of the device name. However this is a wonderful instance of how Agentic AI can perceive errors from MCP servers and “repair them.” It realized the necessity for SSH and tried once more.

I’ve seen a number of instances the place AI brokers will resolve errors with device calls via trial and error and iteration. In reality, some MCP servers appear to be designed particularly with this because the anticipated habits. Good error messages may give the LLM the context required to repair the issue. Much like how we as people may react and regulate after we get an error from a command or API name we ship. This is a wonderful energy of LLMs; nonetheless, I believe that MCP servers can and needs to be designed to restrict the quantity of trial and error required. I’ve additionally seen LLMs “hand over” after too many errors.

Let’s check out the ultimate response from the AI agent after it accomplished gathering and processing the outcomes.

<img loading="lazy" decoding="async" class="size-full wp-image-476969" src="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-05.png" alt="The NetAI agent reports it's findings." width="800" height="629" srcset="https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-05-300x236.png 300w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-05-768x604.png 768w, https://storage.googleapis.com/blogs-images-new/ciscoblogs/1/2025/08/lmstudio-netbox-mcp-05.png 800w" sizes="auto, (max-width: 800px) 100vw, 800px"/> The NetAI agent studies its findings So how did it do?

First, the great issues. It appropriately acknowledged that the hyperlink to switch01 from NetBox matched a CDP entry. Glorious. It additionally known as out the lacking CDP neighbor for the “mgmt” swap. It’s lacking as a result of “mgmt” is an unmanaged swap and doesn’t run CDP.

It will have been actually “cool” if the LLM had seen that the gadget sort of “mgmt” was “Unmanaged Change” and commented on that being the rationale CDP info was lacking. As already talked about, the LLM is NOT tuned for community engineering use instances, so I’ll give it a go on this.

And now the errors… The issue with the filter for the cable resulted in two errors within the findings. There aren’t two cables on Ethernet0/0, and the “Different unused cables” aren’t linked to router01.

Hank’s takeaways from the check I used to be undoubtedly slightly disillusioned that my preliminary exams weren’t 100% profitable; that might have made for an amazing story on this weblog submit. But when I’m trustworthy, working into a couple of issues was even higher for the submit.

AI will be downright superb and jaw-dropping with what it might do. However it isn’t excellent. We're within the very early days of Agentic AI and AIOps, and there's a lot of labor left to do, from growing and providing tuned LLMs with domain-specific information to discovering the very best practices for constructing the very best functioning instruments for AI use instances.

What I did see on this experiment, and all my experiments and studying, is the true potential for NetAI to supply community engineers a strong device for designing and working their networks. I’ll be persevering with my exploration and look ahead to seeing that potential come to fruition.

There’s a lot extra I discovered from this venture, however the weblog submit is getting fairly lengthy, so it’ll have to attend for one more installment. Whereas I’m engaged on that, let me know what you consider AI and the potential for making your every day work as a community engineer higher.