Salesforce Agent Fabric Targets Cross-Platform Evals for January 2027. Rivals Already Ship Them.

Salesforce’s agent control plane adds Agent Evals, API Evals and cost controls. The cost and kill-switch layer is generally available today.

For two years I’ve been complaining about evals to anyone at Salesforce who would listen, and to plenty who wouldn’t. The question never changed. How do you know your agent is any good, and how will you know when it stops being good? The answers usually involved the word “trust” and very few test results. Meanwhile, the rest of the AI industry built its whole craft on evals. Every benchmark you’ve ever seen is just an eval with a press release attached.

To be fair to Salesforce, it hasn’t shipped nothing. It announced Agentforce Testing Center in November 2024 and made it generally available in the Spring ‘25 release on Feb. 17, 2025. Testing Center generates synthetic test cases with AI and runs them against an agent in a sandbox. Salesforce has added testing metrics and observability to Agentforce since then. But those tools test Agentforce agents, inside Salesforce’s own builder. In my experience, the generated tests were thin. They didn’t help with the agents a company runs on Copilot Studio, Bedrock or anything else, which is where most enterprise agents now live.

Salesforce Updates Agent Fabric with Evals

So, when the Agent Fabric press release landed in my inbox under embargo, I did something I rarely do with a Salesforce announcement. I got my hopes up. There it was in the bullet list: Agent Evals for agents, LLMs and MCP servers, on any platform. Automatically generated tests. Production scoring against quality thresholds. A kill switch wired to the results. I read it twice and poured a second cup of coffee to make sure I wasn’t hallucinating, which in this business is a real occupational hazard. For about ninety seconds I believed the mothership had finally picked up the signal.

Then I scrolled to the availability section. Salesforce targets beta in November and general availability in January 2027. The more I dug into what the rest of the market already ships, the faster those ninety seconds of hope curdled into something more familiar.

I also went into my conversation with Andrew Comstock carrying a second grudge, this one about the watchtower itself. Comstock is SVP and general manager of MuleSoft and Agent Fabric at Salesforce. My premise was that most enterprises don’t need an agent watchtower yet. They haven’t figured out how to wire a single agentic application into their workflows, so why would they buy a control tower to coordinate a fleet?

Comstock didn’t fight the premise. He dated it.

He told me that if I’d asked him 18 months ago, he would have agreed, and that Salesforce built the product ahead of where customers actually were. What changed, he argues, is not that enterprises learned to orchestrate agents. It’s that agents started showing up whether anyone orchestrated them or not. That distinction is the key to reading today’s announcement. It’s also why the most important feature in the release is not the kill switch or the cost dashboard. It’s cross-platform evals, and that is the piece furthest from shipping.

Sprawl, Not Orchestration, Is the Buying Trigger

Comstock’s version of the story starts with discovery. Agent Fabric’s registry reaches into platforms like Microsoft Copilot Studio, Amazon Bedrock and Agentforce to catalog what’s running. He described a pattern his account teams keep reporting. A customer believes it has a few dozen agents. The scan finds roughly ten times that, because new groups got access to Copilot Studio, Bedrock or Agentforce and started building their own automations.

That is an anecdote, not a dataset, and no customer was named. But Salesforce’s own research points the same way. Its 2026 Connectivity Benchmark Report surveyed 1,050 IT leaders at companies with at least 1,000 employees. The survey was fielded with Vanson Bourne in October and November 2025. Respondents reported an average of 12 agents per organization, and half of those operate in isolated silos rather than as part of a multi-agent system. Respondents said 36% of their agents are prebuilt SaaS agents and 34% are embedded in enterprise platforms, so together 70% arrive through software the company already buys. Treat it as vendor-sponsored research, but the shape is telling.

Seven in ten respondents’ agents are prebuilt SaaS or embedded platform agents.

Comstock added a nuance I found persuasive. He argued that sprawl comes from the variety of platforms as much as from the raw number of agents. A small restaurant chain with one agent on its website, he conceded, has no sprawl problem. A company building on many platforms across many regions does, even if it has never connected two agents together.

Orchestration came up too, and it’s worth being precise about it. Agent Fabric already supports multi-agent orchestration, an Agent Broker routing service and the A2A protocol, according to Salesforce’s product page. Comstock described a regional bank that wants service, support, sales and new-account agents bought from different vendors to work together. That’s a real requirement. But in his telling it was a plan, not a deployment, and he acknowledged that Salesforce still talks to enterprises who say, “I haven’t built my first agent yet.”

So, my premise survives, but in a narrower form. In my view, enterprises are buying the watchtower first as a fire extinguisher and a budget meter, not as air traffic control. Inventory, spending caps and a kill switch sell now, because CFOs and CISOs can already see the sprawl. The orchestration features are there for customers who are ready, but broad adoption depends on companies having agentic workflows worth coordinating. That timing is my read, not a Salesforce forecast.

Evals Are the Real News

Agents rarely fail loudly. Traditional software throws an error, but an agent drifts. A model gets swapped, an API changes underneath it, and an agent that passed its first tests starts giving confidently wrong answers. Few teams catch that drift in production. In LangChain’s State of Agent Engineering survey, 44.8% of respondents whose organizations have agents in production said they evaluate them on live traffic. The survey drew 1,340 responses in late 2025, and about half came from companies with fewer than 100 employees, so the enterprise figure may differ. In the same survey, 94% of those respondents said they run some form of observability. Teams can watch their agents. Far fewer grade them.

Salesforce Agent Evals targets that gap. According to the release, it generates tests before deployment, then evaluates agents in production against defined quality thresholds. The release’s example is a retailer that swaps in a cheaper model, and Agent Evals automatically runs scheduled tests that catch inaccurate answers. Scheduled tests in production are useful. They are not the same thing as scoring every live conversation, and Salesforce hasn’t said whether Agent Evals will score live traffic directly. The release says the product will cover LLMs and MCP servers as well as agents, but the Agent Fabric product page still labels MCP server and LLM evaluations “coming soon.”

When an agent falls short, Agent Fabric can trigger a predefined response, up to quarantining the agent, with human approval where policy requires it. A companion product, API Evals, generates functional and performance tests from API specifications, so a breaking change gets caught before an agent trips over it. Don’t confuse API Evals with the Agentforce Testing API. They are different products.

Comstock walked me through the mechanics. Teams start by defining what an agent is supposed to do and not do, often from existing requirements documents. The system turns that into a test set and grades results with an LLM as judge, plus internal checks for data quality.

The part that matters most is portability. Comstock says customers are now rebuilding the first agents they shipped, and some are moving them to a different provider. He argued that today, when a customer switches platforms, it loses the history of what it was testing for, because the tests are tied to the platform. Agent Fabric’s pitch is a neutral test bench. Comstock described running the current agent next to a cheaper model or a different provider and comparing results before committing. Treat that side-by-side comparison as an expansion plan. The release describes only a sequential version: switch the model, rerun the evals, confirm performance.

That’s the discipline that has driven progress across the AI industry for years. In my view, evals have been a quiet topic inside the Salesforce ecosystem for far too long, and that silence has held customers back. A shared, editable library of evals is how a company makes AI smarter for its own business. Agent Fabric is the first time Salesforce has promised one that isn’t tied to its own agents.

Comstock sees it the same way. He said some of these tests won’t need a smarter model. “They’re going to need a smarter service,” he said, meaning better context, instructions and guardrails.

That points to the next step. Comstock said that starting in February, Salesforce plans to connect evals to the context packs in its Enterprise AI Harness, so teams can test the content fed to an agent end to end, not just the model’s answer. He also described a compliance-oriented evals framework as a follow-on next year. Both are roadmap statements from the interview, not commitments in the release. Comstock was careful on compliance: “It’s not like this by itself will make you compliant.”

If it works, this is how Salesforce answers the hardest question in enterprise AI right now: why not just let a frontier model do it alone? Run the same tasks with and without your data and context layer, and let the scores answer.

The Rest of the Industry Got Here First

Outside the Salesforce ecosystem, none of this is new. Arize launched Phoenix in 2023, and I use it in my own work. Phoenix is source-available under the Elastic License 2.0, which lets teams self-host it but bars offering it as a managed service. It traces agent runs through OpenTelemetry, runs evals against those traces, and versions datasets and experiments so teams can compare prompts and models side by side. How deep the traces go depends on how the agent is instrumented. Arize said in February 2025 that Phoenix draws more than two million downloads a month. Its integrations cover LangGraph, CrewAI, OpenAI’s and Anthropic’s agent SDKs, and Amazon Bedrock.

Phoenix isn’t alone. LangChain made LangSmith generally available in February 2024 with tracing, test datasets and LLM-graded evaluations, and said then that LangSmith worked whether or not teams used LangChain. Online evaluators for production traffic came later. Langfuse, which is MIT-licensed open source, has offered evals since April 2024, and ClickHouse acquired it in January 2026. Braintrust has offered online scoring of production logs since September 2024, and it announced an $80 million raise on Feb. 17, 2026, at a valuation Axios reported as $800 million.

The big platforms have caught up too. AWS made AgentCore Evaluations generally available on March 31, 2026, with 13 built-in evaluators and online scoring of live traces. Google Cloud made agent and model evaluation generally available on July 31, 2026, including online monitors that grade live production traffic and trace-level evals. IBM added a Trace Inspector and custom LLM-as-judge evaluators to watsonx Orchestrate in August 2026. ServiceNow says more than 150 customers had run its Evaluation Suite by May 2026, and its August–September release added runtime agent evaluations scored in near real time.

Having the tools doesn’t mean everyone used them. The LangChain numbers above show plenty of teams still don’t score live traffic. But the tools were there. Salesforce did bring a bottle to this party in 2025. It just poured only for Agentforce guests. Agent Fabric promises to pour for everyone, starting in January, and by then the other guests will have been drinking for a while.

Traces are the bigger gap. A developer in Phoenix or LangSmith can open a failed run and see each step, tool call and retrieved document, as far as the instrumentation reaches. Agentforce has its own session tracing and can export telemetry over OpenTelemetry, and Agent Fabric documents execution logs. Comstock positioned Phoenix as a tracing tool that instruments the agent itself, a different job from Agent Fabric’s. What Salesforce hasn’t said is whether Agent Evals will show step-level traces for the third-party agents it grades. Without them, a red score tells you an agent failed but not why.

To be fair, Agent Fabric is aimed at a different buyer. Phoenix and LangSmith serve the developer who built the agent. Agent Fabric serves the IT team that has to govern agents it didn’t build, on platforms it doesn’t control, and it can pause an agent, not just grade it. That distinction is real. It doesn’t excuse how late the cross-platform quality layer is arriving.

Agent evaluation and observability: who ships what

Tool	Evaluation availability	Step-level tracing	Live-traffic scoring	Can pause the agent	Model
Arize Phoenix	Launched 2023	Yes, OpenTelemetry; depth depends on instrumentation	Evals run on collected traces	Not established in reviewed sources	Source-available (ELv2), self-hosted or Arize cloud
LangSmith (LangChain)	GA Feb. 2024	Yes	Yes, online evaluators, added after GA	Not established in reviewed sources	Commercial SaaS
Langfuse (ClickHouse)	Evals since Apr. 2024	Yes	Yes	Not established in reviewed sources	Open source (MIT), self-hosted or cloud
Braintrust	Online scoring since Sept. 16, 2024	Yes	Yes, online scoring	Not established in reviewed sources	Commercial SaaS
AWS AgentCore Evaluations	GA Mar. 31, 2026	Yes, via AgentCore Observability	Yes, samples and scores live traces	StopRuntimeSession API ends a session	AWS service
Google Cloud agent evaluation	GA July 31, 2026	Yes, trace evals	Yes, online monitors	Not established in reviewed sources	Google Cloud service
IBM watsonx Orchestrate	Trace Inspector Aug. 17, 2026; custom LLM judge Aug. 31, 2026	Yes, Trace Inspector	Live-traffic evaluators reported Sept. 30, 2026	Not established in reviewed sources	IBM SaaS
ServiceNow AI Control Tower	Evaluation Suite, 150+ customers by May 2026; runtime evals Aug.–Sept. 2026 release	Yes, Trace Collectors	Yes, near-real-time runtime evaluations	Yes, kill switch revokes credentials	ServiceNow platform
Microsoft Evals for Agent Interop	Starter kit, Jan. 26, 2026	Not established in reviewed sources	No, scenario harness	Not applicable	Open-source kit on GitHub
Salesforce Agentforce Testing Center	Announced Nov. 2024; GA Feb. 17, 2025	Agentforce session tracing (separate feature)	No, batch tests; Agentforce agents only	Not applicable	Part of Agentforce
Salesforce Agent Fabric Agent Evals	Beta target Nov. 2026; GA target Jan. 2027	Not established in reviewed sources	Scheduled tests in production; live-traffic scoring not stated	Manual kill switch GA; triggered kill switch GA target Jan. 2027	Part of Agent Fabric

Sources: vendor announcements and documentation, February 2024 to October 2026. Availability dates are vendor-stated.

What Ships When

Read the availability list carefully. Salesforce targets beta in November for Agent Evals and API Evals, and general availability in January 2027. The Triggered Agent Kill Switch, the piece that lets a failing eval pause an agent automatically, also targets GA in January, with no beta date given. Until then, the closed loop in Salesforce’s headline is a plan. The headline sells you a smoke detector wired to the sprinklers. The fine print says the wiring arrives after the holidays.

The customer quotes tell the same story. The release carries four. GE Appliances and Newell Brands talk about visibility and governance they have now, and ASICS Digital also weighs in. The one quote about Agent Evals, from Capita’s Tiina Stephens, is written in the future tense: Agent Evals “will help” Capita validate its agents.

What is generally available today is the defensive layer. That includes a manual Agent Kill Switch, a FinOps Dashboard and Model Wallet with hard budget limits, and Cost Optimization Policies. Salesforce says those policies cut token usage by up to 75% in test scenarios through semantic caching, payload compression and response trimming. The savings apply only to traffic that runs through the configured policies. Treat 75% as a best case until customers report production numbers. The Kill Switch and Model Wallet were first described on MuleSoft’s blog in June. Open Policies for Federated API Gateways is also GA, letting teams push one policy, such as rate limiting, across Azure, Apigee and Kong gateways at once. The Custom Agent Scanner, for discovering agents behind firewalls, is in beta now and targets GA in January.

That sequence supports the narrowed premise. Salesforce is shipping cost and risk control first. Cross-platform quality control comes next. Orchestration is already in the product, waiting for customers to catch up to it.

A Three-Way Race, and a New Name

Comstock named the competition plainly. He said ServiceNow and Microsoft are the two rivals Salesforce meets most often in deals for agent governance. ServiceNow got there early with AI Control Tower, extending its CMDB model to treat agents as enterprise assets. Comstock framed Salesforce’s approach as treating agents as components in a broader architecture. That reflects MuleSoft’s integration heritage and its agnosticism about where the agents run.

The multi-platform argument is Salesforce’s strongest card here. Comstock pointed to embedded agents like Atlassian Rovo and Workday’s Sana arriving in companies that never chose those platforms for AI. His argument was that hyperscalers, frontier model providers, business applications and data lakes are all becoming AI platforms. Someone has to make those cohesive.

That neutrality also explains the branding. The product is now officially Salesforce Agent Fabric, with no MuleSoft or Agentforce label, because, Comstock said, “it can work for anything.” Some customers use it to govern agents without being Agentforce customers at all. Salesforce will need to hammer that message. To most buyers, an agent governance product from the company selling the agents sounds like a referee who also owns a team.

Comstock framed the long game as history repeating. He compared agents to the move from SOA to microservices, arguing that the industry keeps returning to composable architectures because they work, and that agents need to be built the same way.

The agent control plane field

Vendor	Approach	Discovers agents on other platforms	Kill switch	Evals	Hard spending caps	Status
Salesforce Agent Fabric	Integration layer built on MuleSoft; Agent Broker and A2A support	Copilot Studio, Bedrock, Agentforce and others; Custom Agent Scanner in beta	Manual GA; triggered GA target Jan. 2027	Agent Evals GA target Jan. 2027; LLM and MCP evals “coming soon”	Yes, Model Wallet	Mixed GA, beta and targets
ServiceNow AI Control Tower	Agents as CMDB assets	AWS, Google Cloud, Azure, SAP, Oracle, Workday and more; traces from CloudWatch, Foundry, Vertex AI	Yes, revokes credentials across ServiceNow, Okta, GCP and Bedrock agents	Yes, Evaluation Suite and runtime evaluations	Not established in reviewed sources; cost and ROI dashboards	AI Gateway GA Sept. 10, 2026; runtime evals in Aug.–Sept. 2026 release
Microsoft Agent 365	Identity-first, via Entra and Purview	Syncs Bedrock, Google Cloud, Databricks and Claude agents	Blocks risky tools and local agents	Separate Evals for Agent Interop starter kit, Jan. 2026	Not established in reviewed sources	GA May 2026, $15 per user per month
Google Gemini Enterprise	Governance bundled into Google’s agent suite	Registry plus partner agent marketplace	Not established in reviewed sources	Agent evaluation GA July 31, 2026, in Google Cloud	Not established in reviewed sources	Rolling out since April 2026
IBM watsonx Orchestrate	“Agentic control plane”	Agent Catalog; Bedrock discovery GA Aug. 31, 2026; Foundry and Vertex discovery planned	Runtime policy enforcement	Trace Inspector and custom LLM judge, Aug. 2026	Not established in reviewed sources	Available since June 2026
Boomi Agentstudio	Integration platform with an “Agent Control Tower”	Native and third-party agents, including Snowflake Cortex; monitoring excludes Copilot and Salesforce agents	One-click disable	Not established in reviewed sources	Not established in reviewed sources	Available

Sources: vendor announcements and documentation, April to October 2026.

A Note to Salesforce: Make Evals the Centerpiece

Frontier models are becoming shared infrastructure. Every company can rent the same ones. What competitors cannot rent is your definition of “good”: the answers your best rep would give, the things your agent must never say, the edge cases only your business sees. Evals are where that judgment gets written down. Every failure investigated in production can become a new test case. Every successful intervention can sharpen the standard. The library grows, and the accumulated knowledge compounds. Over time, it becomes something competitors cannot simply buy: a company-owned record of what good AI performance truly means.

Salesforce customers are sitting on the data, context, and expertise to build that advantage. Agent Fabric and Enterprise AI Harness could be how they preserve and compound it. Right now, evals are a feature in the bullet list. Here is what it would take to make them the centerpiece.

Ship every context pack with its evals. When Salesforce connects evals to the Enterprise AI Harness, every context pack should come with a versioned test suite: expected outcomes, prohibited behaviors, edge cases, and grading criteria. Keep the tests separate from the context delivered to the model but make them travel together as a governed package. A context pack without evals is the AI equivalent of untested code, and Salesforce knows better than to ship untested code.

Make the eval library the customer’s property. Test sets, grading criteria, and results should export in open, machine-readable formats, and execution traces should use standards like OpenTelemetry. Comstock’s portability pitch only works if the evaluation history travels with the customer, across models, platforms, and providers, including when they leave Salesforce. A moat a customer can’t take with them is a cage.

Grade live traffic and show the trace. Scheduled tests catch regressions. Live scoring catches drift and the failures nobody anticipated. Every failed score should link to the most detailed available execution trace, including for third-party agents Agent Fabric governs. Where an external platform doesn’t expose its internal steps, show that limitation rather than hiding it behind a quality score. An alarm without an address is only half a safety system.

Tell one eval story. Testing Center for Agentforce and Agent Evals for everything else is two products for one discipline. Customers should be able to carry their Testing Center investments straight into Agent Evals, preserving their test cases, historical results, and grading criteria. The evaluation system should outlive the agent, the model, and the platform that originally produced it.

Prove the context layer with scores. Salesforce’s best answer to “why not just use a frontier model?” is a number. But the experiment needs to prove more than the obvious fact that models perform better when given relevant information. Run the same business tasks against the same model, controlling for access to the same underlying enterprise data, with and without Harness-managed context. Measure answer quality, reliability, cost, and latency. Show whether Salesforce’s architecture improves the outcome, not merely whether proprietary data is useful.

That last one is my call to action, and it’s measurable. By the January GA, Salesforce should publish the evaluation methodology. When context-pack integration arrives, publish a reproducible benchmark comparing a frontier model alone, a model with equivalent enterprise data access, and that same model using Enterprise AI Harness. Use Agent Evals to score the results. Publish the test cases, grading criteria, and performance differences. Let independent developers reproduce the experiment. Agent Fabric should be the referee, not a competitor in the experiment.

If the harness wins, Salesforce has its story. If it doesn’t, it has its roadmap. Either way, Salesforce demonstrates an eval culture the rest of us can inspect.

I’ll be refilling my coffee and waiting for the numbers.

Vernon Keenan is a senior industry analyst and the publisher of SalesforceDevops.net, based in Oakland, California. Over more than four decades, he has covered enterprise application development, DevOps and AI, and he coined the term “VE Economics” to assess AI agents that function as virtual employees. He collaborates with UC Berkeley Haas on research into the total addressable market for AI agents. Disclosure: Vernon Keenan is a paid advisor to Salesforce and to other vendors in the Salesforce ecosystem.