NCP-AAI Practice Test

NCP-AAI Practice Test practice questions, page 5 of 5, with answer review and explanations.

Question 21

A customer service agentic AI is designed to resolve billing inquiries. It consistently resolves inquiries accurately and efficiently. However, a significant number of customers are reporting frustration due to the agent’s tendency to repeatedly ask for the same information (account number, address) during each interaction, even after it’s already been provided. Which evaluation method would be most effective for addressing this issue?

  1. Adjusting the agent’s reward function to prioritize speed of resolution over customer satisfaction.
  2. Analyzing the agent’s dialogue transcripts to identify patterns in its questioning techniques.
  3. Implementing a “conversational flow” analysis to optimize the order of questions asked during each interaction.
  4. Increasing the agent’s processing speed to reduce the time it takes to handle each inquiry and increase customer satisfaction.

Question 22

A financial services agentic AI is being used to automate initial customer onboarding. The agent is completing the process efficiently and accurately, but reviews of its conversations reveal it often uses overly formal and complex language that confuses customers. Which type of evaluation is best suited to address this issue?

  1. Controlled user testing sessions to collect user feedback on the clarity and tone of responses
  2. Compliance review of the agent’s access to regulatory guidelines and policy documentation
  3. Continuous user feedback collection, specifically gathering subjective assessments of the agent’s communication style
  4. Statistical analysis of the agent’s decision-making patterns to detect overly formal and complex response choices

Question 23

You’re evaluating the performance of a tool-using agent (e.g., one that issues API calls or executes functions). From the list below, what are two important features to evaluate? (Choose two.)

  1. Tool use accuracy
  2. Tokens per second
  3. Tool use rate
  4. Task completion rate

Question 24

When analyzing user feedback patterns to improve a technical documentation agent, which evaluation methods effectively translate feedback into actionable optimization strategies? (Choose two.)

  1. Collect broad user feedback as-is, enabling rapid accumulation of suggestions and diverse perspectives for potential future analysis.
  2. Design iterative feedback loops with version tracking, A/B testing of improvements, and regression monitoring to ensure changes enhance rather than degrade performance
  3. Incorporate user suggestions rapidly to maximize responsiveness and demonstrate continuous adaptation to evolving user needs.
  4. Implement feedback categorization systems grouping issues by type (accuracy, clarity, completeness) with quantitative impact scoring and improvement prioritization matrices