You’ve launched your Agentforce expertise. Congrats! But when there’s one factor we’ve realized at Salesforce, it’s this: success doesn’t begin at go-live. It begins with what comes subsequent.
As somebody serving to run Agentforce on our personal Assist website, I’ve seen firsthand what it takes to maintain an AI agent good, secure, and useful lengthy after go-live. Right here’s what works for us with regards to measuring reply high quality, sustaining enhancements, and constructing suggestions loops that matter by ongoing agent analysis.
“An excellent agent expertise doesn’t come from day-one polish. It comes from studying from actual conversations.”
Deal with Agentforce like a teammate, not a ticket
The largest entice? Considering your work is finished post-launch. Agentforce isn’t a “set it and neglect it” software. It’s a dynamic teammate that requires construction, oversight, and suggestions.
We maintain:
- Weekly evaluations targeted on efficiency: utilization patterns, decision charges, content material gaps.
- Actual-time monitoring, each quarter-hour, to catch errors early.
- Month-to-month checkpoints to zoom out and recalibrate.
These common processes type the spine of our agent analysis framework. It’s a rhythm that mirrors how we handle dynamic merchandise, as a result of that’s what Agentforce is. We give it a roadmap. We ship updates. We evolve with our customers.
In the event you don’t preserve it, high quality slips. Quick.
Undertake the suitable AI mindset
See how pondering of AI as a teammate — not a software — can rework your outcomes. This weblog shares methods to start out working successfully with AI immediately.
Outline what “good” appears like, then construct the system to assist it
Earlier than we may measure reply high quality, we needed to agree on what “high quality” meant. For our Assist agent, we landed on three core standards:
- Completeness: Does the reply totally resolve the consumer’s query?
- Relevance: Is it truly addressing the query that was requested?
- Appropriate: Does the reply comprise all the knowledge required to unravel the issue, and is it factually correct?
These are easy, however they maintain up below stress. Your standards may range, product suggestions, next-best actions, escalation guidelines, however the secret’s alignment. Outline the usual. Stick with it.
Use AI to check AI – sure, actually
A core a part of sustaining a sensible and useful AI agent is steady agent analysis, utilizing a mixture of automated testing, human evaluate, and real-time monitoring to make sure high quality stays excessive.
Right here’s the place it will get enjoyable. We use AI to judge Agentforce’s personal solutions. There are just a few methods we do that:
- Artificial utterance testing: We pull hundreds of historic buyer questions from chat logs and case titles, then group them by intent utilizing AI. From there, we generate check questions that symbolize real-world asks.
- Brokers testing brokers: One AI mannequin generates solutions. One other scores these solutions utilizing structured “choose prompts” that describe what a high-quality response appears like. It’s like peer evaluate – however quicker.
- Golden pairs: These are question-and-answer units we’ve vetted as “superb” responses. They function benchmarks to measure high quality in opposition to. Helpful for calibration and consistency although they want repairs as your information base adjustments. We’ve even constructed go/fail guidelines into our choose prompts so we will automate high quality checks at scale.
And shortly, we’ll take this additional with our new Testing Middle and Interplay Explorer – Instruments designed to assist groups consider Agentforce efficiency with much less guide effort.
Get the Agentforce Testing Information
Launch your first check set in minutes utilizing the Agentforce Testing Middle Fast Begin Information. Step-by-step directions assist you to create, add, and automate evaluations.
Prospects know what’s damaged, ask them
Buyer suggestions performs an important position in our agent analysis, offering real-world alerts that complement inside testing. All the inner testing on this planet gained’t exchange listening to instantly out of your prospects. That’s why we run two suggestions mechanisms on Assist immediately:
- Buyer Confirmed Decision: After a consumer interacts with Agentforce, we ask, “Did this remedy your drawback?” A “sure” counts as a confirmed decision.
- Expertise Ranking: For resolved circumstances, customers can price the interplay 1–5 stars. It’s a light-weight sign, however a helpful one.
Quickly we’re rolling out a easy like/dislike function for particular person solutions, and sentiment evaluation to assist us perceive tone even with out direct suggestions.
My rule of thumb: In the event you’re not listening to from customers, you’re in all probability not asking the suitable approach. Suggestions must be designed into the expertise, not left to probability.
See How Salesforce Measures Success
Study which metrics we monitor throughout our Assist website to enhance efficiency, accuracy, and belief with Agentforce.
Don’t intention for excellent, intention for higher
Agentforce isn’t one thing you set and neglect. It’s an evolving a part of your buyer expertise, one which improves by iteration and with the suitable construction.
We’ve seen regular positive factors by staying near the info, setting clear expectations, and treating high quality as a steady course of. That’s how decision charges enhance. That’s how belief builds.
Need to skip straight to hands-on studying? Be part of an Agentforce NOW digital workshop, led by product specialists, and begin constructing, testing, and tuning your agent dwell — no expertise wanted.
