We develop a simulation model to aid in identifying and evaluating promising alternatives to achieve improvements in weapon system-level availability when outsourcing logistics services for system components. Two outcomes are valued: improvements in average operational availability for the weapon system, and reductions in the probability that operational availability of the weapon system falls below a given planning threshold (readiness risk). In practice, these outcomes must be obtained through performance-based agreements with logistic providers. The size of the state space, and the non-linear and stochastic nature of the variables involved precludes the use of optimization approaches. Instead, we use designed experiments to evaluate simulation scenarios in an intelligent way. This is an efficient approach that enables us to assess average readiness and readiness risk outcomes of the alternatives, as well as to identify the components and logistics factors with the greatest impact on operational availability.