NCP-AAI Practice Test

NCP-AAI Practice Test practice questions, page 4 of 5, with answer review and explanations.

Question 16

An engineer has created a working AI agent solution providing helpful services to users. However, during live testing, the AI agent does not perform tasks consistently. Which two potential solutions might help with this issue? (Choose two.)

  1. Remove schema validations and assertions on tool outputs to avoid inconsistency.
  2. Increase randomness (e.g., temperature) and remove fixed seeds to avoid determinism.
  3. Identify where dividing the tasks into subtasks and handling them by multiple agents can help.
  4. Refine the prompt given to the AI Agent; be clear on objectives

Question 17

A development team is building a customer support agent that interacts with users via chat. The agent must reliably fetch information from external databases, handle occasional API failures without crashing, and improve its responses by learning from user feedback over time. Which of the following tasks is most critical when enhancing an AI agent to handle real-world interactions and improve over time?

  1. Applying a well-structured training process with foundational generative models and prompt engineering
  2. Utilizing internal knowledge bases to support agent responses alongside external APIs
  3. Implementing retry logic for error handling and integrating user feedback loops for iterative improvement
  4. Designing conversation flows that provide consistent responses based on predefined scripts

Question 18

What NVIDIA framework can be used to train a better agent?

  1. NeMo-RL
  2. NeMo Guardrails
  3. TensorRT-LLM

Question 19

You are evaluating your RAG pipeline. You notice that the LLM-as-a-Judge consistently assigns high similarity scores to responses that contain irrelevant information. What should you investigate as the most likely potential cause with the least development effort?

  1. The temperature setting used by the LLM during response generation.
  2. The size of the knowledge base used to power the RAG pipeline.
  3. The quality of the synthetic questions used for evaluation.
  4. The prompt used to instruct the LLM-as-a-Judge to assess the response.

Question 20

You’re managing an agentic AI responsible for customer support ticket triage. The agent has been consistently accurate in routing tickets to the appropriate departments. However, a team leader has noticed a significant increase in the number of tickets requiring “escalation” – cases where the agent initially misclassified a complex issue as a simple, routine one, leading to delays and frustrated customers. What would be an appropriate first step in resolving this issue?

  1. Analyzing the agent’s decision-making process, focusing on the specific criteria it uses to classify tickets, and identifying potential biases or blind spots.
  2. Adjusting the agent’s reward function to prioritize speed of resolution over accuracy, as a first step in analysis of the problem.
  3. Increasing the agent’s autonomy, granting it more decision-making power during triage to improve its efficiency.
  4. Conducting a “red-teaming” exercise, having human agents deliberately create complex and ambiguous scenarios to analyze the agent’s robustness.