CCAR-P Practice Test

CCAR-P Practice Test practice questions, page 3 of 5, with answer review and explanations.

Question 11

An operations engineer reports that a Claude-based pipeline began returning malformed JSON responses after a scheduled maintenance window, causing downstream processing failures. Which two investigative steps most directly isolate the root cause? (Choose two.)

  1. Compare the current system prompt and output schema configuration against the last known good version from before the maintenance window.
  2. Switch to a different Claude model tier to rule out provider-side changes as a contributing factor.
  3. Increase the max tokens limit to determine whether output truncation is causing incomplete JSON structures.
  4. Replay a set of premaintenance requests against the current configuration and inspect the raw model output before downstream parsing.
  5. Clear the prompt cache and resubmit all pending requests to eliminate stale cached prefixes.

Question 12

You are running a controlled experiment to compare two prompts and must complete the design steps before executing the experiment. Which two steps must be completed BEFORE running the experiment with random assignment? (Choose two.) Each correct answer presents part of the solution.

  1. Analyze the results against the predefined success metric and significance threshold.
  2. Determine the minimum detectable effect size and the sample size needed for power.
  3. Document the recommendation, the trade-offs accepted, and the alternatives considered.
  4. Decide whether to promote, reject, or iterate the candidate based on the analysis.
  5. Define the hypothesis and the primary success metric for the comparison.

Question 13

An architect is reviewing a Claude-based candidate-screening tool prior to deployment. A stakeholder asserts that because the model was not trained on company data, no bias evaluation is necessary. Which two responses most accurately challenge this assertion? (Choose two.)

  1. The model may carry demographic biases from pretraining that manifest in screening outcomes regardless of fine-tuning.
  2. Transparency obligations are satisfied by disclosing that an AI system is in use, without further evaluation.
  3. The system should be evaluated on outcome-disparity metrics across protected groups before deployment.
  4. Fairness testing is required only when the training dataset is known to contain protected-class labels.
  5. Bias evaluation is unnecessary when the model provider has published a responsible-use policy.

Question 14

A pilot AI assistant for procurement specialists shows 89 percent first-response acceptance, but follow-up surveys reveal that specialists frequently override the assistant’s vendor recommendations after consulting unrelated criteria the assistant did not consider. The pilot owner wants to ship the assistant as is, given the strong acceptance rate. Which two discernment-competency observations should you raise BEFORE approving the launch? (Choose two.) Each correct answer presents part of the solution.

  1. Acceptance does not measure whether recommendations are right after specialist review.
  2. The survey response rate may not be statistically representative of all the specialists.
  3. The unconsidered criteria represent a scope gap in the assistant’s input space directly.
  4. The acceptance rate alone is sufficient evidence of readiness for general production use.
  5. The pilot duration was likely too short to establish reliability across the full year.

Question 15

The platform team at Trenova Systems, Inc. needs to reduce per-query cost and p95 latency for a high-volume Claude pipeline without degrading output quality on the core use case. Which two optimizations directly target both cost and latency simultaneously? (Choose two.)

  1. Add a retrieval step that fetches the full source corpus into the context window before generation.
  2. Expand the system prompt to include additional few-shot examples for every request.
  3. Increase max_tokens to reduce the frequency of truncated responses requiring follow-up calls.
  4. Enable prompt caching on the static system-prompt prefix to reduce billable input tokens on repeated calls.
  5. Route straightforward query types to a smaller, faster Claude model and reserve the full model for complex cases.