In the “serial dependence” effect, responses to visual stimuli appear biased toward the last trial’s stimulus. However, several kinds of serial dependence exist, with some reflecting prior stimuli and others reflecting prior responses. One-factor analyses consider the prior stimulus alone or the prior response alone and can consider both variables only via separate analyses. We demonstrate that one-factor analyses are potentially misleading and can reach conclusions that are opposite from the truth if both dependencies exist. To address this limitation, we developed two-factor analyses (model comparison with hierarchical Bayesian modeling and an empirical “quadrant analysis”), which consider trial-by-trial combinations of prior response and prior stimulus. Two-factor analyses can tease apart the two dependencies if applied to a sufficiently large dataset. We applied these analyses to a new study and to four previously published studies. When applying a model that included the possibility of both dependencies, there was no evidence of attraction to the prior stimulus in any dataset, but there was evidence of attraction to the prior response in all datasets. Two of the datasets contained sufficient constraint to determine that both dependencies were needed to explain the results. For these datasets, the dependency on the prior stimulus was repulsive rather than attractive. Our results are consistent with the claim that both dependencies exist in most serial dependence studies (the two-dependence model was not ruled out for any dataset) and, furthermore, that the two dependencies work against each other.