- Full Copy Sandboxes are important for life like AI agent testing. Utilizing a whole duplicate of manufacturing information permits builders to precisely take a look at advanced situations, efficiency below load, and edge circumstances with none threat to reside operations.
- Testing is essential for constructing belief in AI. The Agentforce Testing Heart enabled systematic stress checks, detrimental checks (utilizing dangerous information), and regression testing, offering deep visibility into the AI’s decision-making to eradicate “hallucinations” and guarantee reliability.
- This Buyer Zero method proves the platform’s worth. Through the use of their very own instruments to construct Agentforce, Salesforce’s inner IT workforce created a profitable AI that resolves 75% of buyer queries routinely, validating their merchandise’ effectiveness for creating enterprise-grade AI.
In the case of constructing, testing, and deploying synthetic intelligence (AI) brokers, Salesforce Sandboxes are an important software for the Agentforce Lifecycle Administration journey. This can be a key focus for Harini Woopalanchi, our director of IT for product administration, and her workforce. They work to make sure that Agentforce, the agentic layer of the Salesforce Platform, is developed and examined in the precise atmosphere, guaranteeing a clean and dependable launch.
Agentforce powers assist.salesforce.com, which has dealt with over 1.5 million assist requests since launching final 12 months. Agentforce additionally helps us nurture greater than 10,000 leads per week.
In a latest dialog with Woopalanchi, we unpacked this Salesforce on Salesforce story, studying how her workforce used Salesforce Sandboxes as Buyer Zero to develop, take a look at, and deploy Agentforce.
Desk of contents
Boldly and securely innovate
Create safe, remoted copies of your manufacturing atmosphere for all of your constructing, testing, and deploying wants with Salesforce Sandboxes.
Half 1: The facility of sandboxes for growth and testing
So thrilling seeing these brokers come to life! Salesforce selected to construct these brokers utilizing our personal merchandise. Are you able to inform us why counting on Salesforce Sandboxes was a important determination for this mission?
Woopalanchi: Thanks! It’s been an unimaginable journey. The choice to “dogfood” our personal merchandise, particularly Salesforce Sandboxes, was completely important.
We had been capable of develop and take a look at in an atmosphere that was a real reflection of our manufacturing occasion, however utterly remoted. This meant we may iterate quickly, experiment with new options, and push the boundaries of the agent’s capabilities with none threat to reside buyer operations.
Moreover, this remoted atmosphere prevented the agent’s growth and testing from disrupting our different utility launch cycles, making certain a clean and uninterrupted move for all of our growth initiatives. This gave us the boldness that what we had been constructing would carry out precisely as anticipated when deployed. It’s the final word proof level for our personal expertise – if it’s ok for us, it’s actually ok for our clients.
How did utilizing a full copy sandbox allow a extra life like growth and testing atmosphere in comparison with different sandbox sorts?
Woopalanchi: A full copy sandbox was a game-changer for testing these brokers. In contrast to partial or developer sandboxes, a full copy offers a whole duplicate of our manufacturing information, together with all our buyer data, historic transactions, and, crucially, our whole information base. For an agent designed to reply advanced questions like transaction lookups or information customers by detailed troubleshooting, having real-world information was indispensable.
Initially, we performed Agent experimentation in our current QA sandboxes. Nonetheless, as our use circumstances grew in complexity and concerned a number of brokers, a devoted full copy sandbox turned important. This enabled us to check advanced use circumstances with integrations to exterior programs in a extra strong manner.
Think about attempting to check if an agent can precisely discover a particular transaction for a consumer with out having thousands and thousands of various transaction data to look by. Or validating if it may retrieve the right information article when confronted with a nuanced question, with out the total breadth and depth of our precise information articles. A full copy sandbox allowed us to simulate these real-world situations with unparalleled accuracy, making certain our agent wasn’t simply theoretically appropriate, however virtually strong and dependable. It helped us catch edge circumstances and efficiency bottlenecks that merely wouldn’t floor in a smaller, much less consultant atmosphere.
How did Person Acceptance Testing (UAT) within the full copy Sandbox assist to make sure the agent carried out reliably with consultant information volumes earlier than going reside?
Woopalanchi: UAT within the full copy sandbox was the ultimate, essential validation step. With a whole duplicate of our manufacturing information, we may conduct UAT with inner groups and even a choose group of pilot customers, mimicking precise buyer interactions. This wasn’t nearly checking if the agent gave the precise reply; it was about evaluating its efficiency below life like load, and its skill to navigate by advanced consumer journeys with the precise quantity and number of information it will encounter in manufacturing.
For instance, we may take a look at situations the place a consumer may need a really lengthy historical past of transactions, or the place a information article about Product documentation may need many variations. This complete UAT allowed us to fine-tune the agent’s logic, optimize information retrieval, and guarantee scalability. It offered the final word assurance that when brokers went reside, they might not solely be correct but additionally carry out reliably and effectively for all our customers, no matter their particular information profile or question complexity.
(Again to prime)
Half 2: Agentforce Testing Heart for agent intelligence and reliability
Considered one of my favourite options at Salesforce is the Agentforce Testing Heart, a no-code testing software out there proper in your sandbox! How did the Testing Heart particularly allow you to completely put together and take a look at your AI brokers?
Woopalanchi: The Agentforce Testing Heart was indispensable for completely validating our AI brokers. For transaction lookups, the complexity multiplies as a result of sheer quantity and number of information. The Testing Heart, built-in with our full copy sandbox, allow us to run checks in opposition to consultant information volumes. We may confirm that the agent not solely understood the request but additionally precisely queried Knowledge Cloud, retrieved the right transaction particulars, and offered them clearly to the consumer. This systematic, repeatable testing was key to making sure each the precision and the safety of those delicate operations.
What sorts of take a look at circumstances may you run within the Agentforce Testing Heart that wouldn’t be attainable or as environment friendly in a reside atmosphere?
Woopalanchi: There are a number of sorts of take a look at circumstances which are both not possible or extremely inefficient/dangerous to run in a reside manufacturing atmosphere, which the Agentforce Testing Heart made not solely attainable however environment friendly.
- Excessive-volume stress testing: We may simulate as much as 1,000 concurrent consumer requests to grasp how the agent performs below excessive load. That is essential for assist.salesforce.com, given the worldwide consumer base. You merely can’t try this in manufacturing with out risking service disruption.
- For detrimental testing & edge circumstances: We may deliberately feed the agent incorrect, ambiguous, or malicious inputs to see the way it handles them. This helps establish vulnerabilities or surprising fallback behaviors. For instance, what if a consumer asks for a transaction lookup however offers a totally invalid ID? The Testing Heart allowed us to check these “failure” situations safely.
- Regression testing: Each time we made an replace or added a brand new function, we may run our whole suite of current checks to make sure that the adjustments hadn’t inadvertently damaged any beforehand working performance. This automated regression testing saved immense time and prevented new bugs from reaching manufacturing.
- Testing with artificial information: For situations the place particular real-world information is likely to be delicate or unavailable for broad testing, the Testing Heart’s skill to generate artificial information allowed us to create customized situations to completely take a look at new agent logic or integrations with out privateness issues.
How did the Agentforce Testing Heart contribute to the general reliability and belief in our AI brokers at Salesforce?
Woopalanchi: The Agentforce Testing Heart was foundational to constructing reliability and belief. On this planet of AI, “hallucinations” or incorrect responses are a serious concern. The structured testing framework, mixed with detailed logging and monitoring throughout the Testing Heart, allowed us to systematically establish and eradicate these points. With Agent builder and Testing Heart, We may see precisely why an agent responded the way in which it did, hint its decision-making path, response accuracy and pinpoint the place changes had been wanted.
This iterative technique of testing, analyzing, and refining in a managed atmosphere meant that by the point the brokers went reside, we had excessive confidence of their accuracy and consistency. It reworked agent growth from a speculative course of right into a data-driven, quality-assured one. For our clients, this interprets instantly right into a extra dependable and reliable self-service expertise, which is paramount for a assist portal.
Growing AI brokers comes with distinctive challenges. How did you guarantee appropriate responses and actions for queries?
Woopalanchi: That is the place the Agentforce Testing Heart actually shines in addressing the “black field” problem typically related to AI. It offers deep observability into the agent’s inner workings. For each take a look at run, we get detailed logs and audit trails that present:
- Motion execution: If the agent was alleged to carry out an motion (like initiating a password reset move). Testing Heart makes certain appropriate or anticipated subjects and actions had been referred to as and executed.
- Fallback conduct: How the agent responded when it couldn’t discover a direct reply or encountered an surprising enter. Did it gracefully escalate to a human agent, or present a useful various?
This stage of visibility allowed our workforce to debug successfully, perceive the place the agent’s reasoning may deviate, and constantly fine-tune its logic and information base connections. It’s like having an X-ray imaginative and prescient into the AI’s “mind,” making certain transparency and enabling us to ensure that the agent constantly arrives on the appropriate solutions and takes applicable actions.
(Again to prime)
Half 3: Deployment and what’s forward
What’s an enhancement you wish to see in Salesforce Sandboxes to make agent growth and deployment simpler?
Woopalanchi: I’m excited in regards to the Command Heart, a sophisticated observability platform from Salesforce. This software is about to be a game-changer for us, providing a deep, data-driven look into how our AI brokers are performing – proper out of the field.
Powered by a unified Agent information mannequin, it offers detailed session tracing for particular person interactions and collects data at runtime. This may enable us to investigate all the pieces from dialog high quality to technical metrics, giving IT and enterprise leaders an outline of the place brokers are deployed and figuring out alternatives for enchancment.
It’s precisely what we have to elevate our growth and deployment course of and perceive how our brokers are driving worth.
Construct on the Salesforce Platform
Since its deployment, Agentforce now handles all the pieces from responding to routine inquiries and performing widespread duties to opening service tickets that seamlessly escalate to human reps with the total case historical past and context.
Thus far, Agentforce has dealt with greater than 1 million conversations, resolving an over 75% of buyer queries with out human intervention.
Attending Dreamforce? Study extra in our session, How Salesforce Builds Brokers Securely in Sandboxes, which will even be out there on-demand on Salesforce+.
Take a better have a look at Dreamforce
Wish to see how Salesforce constructed an agentic enterprise — and the way your organization can develop into one too? Try Salesforce+ throughout Dreamforce (Oct. 14-16) and get a digital entrance row seat.
(Again to prime)
