Recent work in neurolinguistics (Kemmerer et al., 2009) reports brain-damaged patients that are selectively impaired in their knowledge of the semantic constraints that govern prenominal adjective order. They performed poorly on a test that required them to discriminate between preferred and dispreferred adjective orderings (e.g. a big brown dog vs. a brown big dog) (Task 1). At the same time, their knowledge of the semantic classes with which these constraints interact (VALUE, SIZE, COLOR, etc.) seems to be spared, since they performed well on a task in which they had to choose which of two adjectives was most similar to a pivot adjective (e.g. whether the adjective good was more similar to bad or to tiny) (Task 3). They were also still able to discriminate between correct and incorrect orderings of adjectives in relation to other parts of speech (e.g. big field vs. field big) (Task 2), which suggests that they still have unimpaired knowledge of the purely syntactic constraints on adjective order.
In this study, we provide a more explicit cognitive characterization of these patients as ‘overeager abstractors’. More specifically, we provide evidence for the claim that the performance of these patients can be explained as the result of oversmoothing in a similarity-smoothed bigram word prediction model, using a similarity metric that relies on the distribution of conditioned words over conditioning words.
To simulate the performance of the patients, we varied the amount of smoothing, i.e. the number of nearest neighbors that is used for extrapolation, and looked at the effect of this manipulation on the model’s accuracy for Tasks 1 and 2. Performance on Task 1 already started to break down when taking a relatively small number of neighbors into account (i.e. more than five). On the other hand, performance on Task 2 was more robust to oversmoothing and only started to decrease when extrapolating from a much larger number of neighbors (i.e. more than 10,000). A Wilcoxon rank sum test showed that the mean rank of the per-item breakdown level was significantly higher for Task 2 (Mdn = 30,000) than for Task 1 (Mdn = 400), W = 281, p (one-tailed) = .002. At the same time, the model's accuracy on Task 3 (19 out of 25 items correct) showed that the similarity metric captured relevant semantic similarities between the adjectives.
Using mixed effects logistic regression (Bates & Maechler, 2010), we directly compared the model predictions to the behavioral data. According to the 'overeager abstraction' hypothesis, the items for which the model starts to predict the wrong order at low levels of abstraction (i.e. when still taking only relatively similar neighbors into account) should also be the most difficult for the patients to get right. Supporting our hypothesis, we found a positive effect of breakdown level on the probability that the impaired patients will choose the correct adjective order, β = 0.13 (SE = 0.05), p = .009, and a negative interaction of this effect with the effect of group (impaired vs. unimpaired), β = -0.12 (SE = 0.06), p = .04.
Taken together, these findings provide strong support for our claim that the patients reported by Kemmerer et al. (2009) can be characterized as ‘overeager abstractors’. Thus we were able to explain adjective order constraints and impairments to those constraints in terms of computationally explicit cognitive principles.
References:
Bates, D., & Maechler, M. (2010). lme4: Linear mixed-effects models using S4 classes (R package version 0.999375-34) [Software]. Available from CRAN: http://CRAN.R-project.org/package=lme4
Kemmerer, D., Tranel, D., & Zdanczyk, C. (2009). Knowledge of the semantic constraints on adjective order can be selectively impaired. Journal of Neurolinguistics, 22, 91–108.
| 
