To further the understanding of the mechanisms of strategy choice, in three experiments, we investigate the role of explicit awareness and working memory in strategy adaptivity. Experiment 1 provides correlational evidence that individual differences in strategy adaptivity to changing base-rates are related to individual differences in awareness of those changes but appear not to be related to individual differences in working memory capacity. Experiment 2 replicates the role of awareness and suggests that it is awareness at the time of the base-rate change rather than afterwards that is related to increased strategy adaptivity. Experiment 3 measures working memory capacity using a different procedure and manipulates working memory load with a dual task procedure, and finds, once again, no apparent role of working memory capacity in strategy adaptivity. This juxtaposition of findings presents a challenge for existing models of strategy choice.
This paper explores an alternative approach to the study of individual differences of cognitive function-that people may have the same strategies but differential ability to adaptively select among them in response to success and failure feedback from the environment. Three studies involving the complex and dynamic Kanfer-Ackerman Air Traffic Control Task© (Ackerman & Kanfer, 1994) demonstrate 1) that individuals do differ systematically along this strategy adaptivity dimension, 2) that those differences have important consequences for overall task performance, and 3) the differences are primarily associated with reasoning ability and working memory capacity.
Goodie and Fantino (2000) make two main criticisms of the predictions of Lovett and Schunn's (1999) RCCL model. In both cases, we believe the criticisms reflect a failure to appreciate the difference between broad frameworks and specific mathematical/computational models. In this paper, we show the value of a broad framework, such as RCCL, in directing new empirical analyses and guiding theoretical development. In particular, RCCL expands on existing work to reveal how variability and change in mental representations influence base-rate sensitivity. We also address several other issues raised by Goodie and Fantino (2000) and show that qualitative shifts in individuals' choice behavior are present in their original data-a key prediction of RCCL that does not appear in previous accounts.
A theoretical account of the mirror effect for word frequency (low frequency words elicit more hits and fewer false alarms than high frequency words) and of dissociations in the pattern of responding Remember vs. Know (R vs. K) for low and high frequency words is tested in three experiments, both empirically and computationally by comparing predicted to observed data for each experiment. The SAC (Source of Activation Confusion) theory makes the novel prediction of more Know responses for high frequency words than for low frequency words, for both old and new items. This prediction was confirmed in all three experiments. Experiments 1 and 2 used a continuous presentation and judgment paradigm, varying experimental frequency from 1 to 10, along with normative frequency. The computer implementation of SAC's theory closely modeled the pattern of results for high and low frequency words, fitting new, know and remember patterns of responding at each level of experimental presentation. Experiment 3 required list discrimination as well as new/old and R/K responding. One group was required to indicate on which of four lists a word had appeared after making Remember responses; a second group was required to make the discrimination regardless of type of old response. Results suggest that "R" responses are indeed associated with better memory for episodic details than are "K" responses. All three experiments were modeled with the same parameter values, most of which had been imported from earlier SAC modeling enterprises in other domains.
Previous research on scientific reasoning has shown that it involves a diverse set of skills. Yet, little is known about generality or domain-specificity of those skills, an important issue to theories of expertise and to attempts to automate scientific reasoning skills. We present a study designed to test what kinds of skills psychologists actually use in designing and interpreting experiments, and contrast expertise within a particular research area with general expertise at designing and interpreting experiments. The results suggest that psychologists use many domain-general skills in their experimentation and that bright and motivated Carnegie Mellon undergraduates are missing many of these skills. We believe that these results have important implications for psychological and AI models of expertise, as well as educational implications in the domain of science instruction. Full paper
How are base-rate and case-specific cue information combined to produce choice? Previous results have found that base-rate information tends to be underweighted relative to case-specific cues (e.g., Tversky & Kahneman, 1982), although some paradigms find base-rate sensitivity fairly consistently but not universally. We explain the different ways in which people combine base-rate and case-specific cues using a new model, called RCCL. The model makes use of the construct of task representationsÑthe subset of features used to encode the task environment--- and makes additional predictions regarding variability in choice over time. Experiment 1 shows that task representations can be influenced by feedback from the environment, producing changes in base-rate and cue sensitivity. Experiment 2 shows that variations in the format of a task can lead to different representations, which in turn produce very different base-rate and cue sensitivities. Moreover, both experiments find systematic variability in choice over time in ways predicted by the model.
To explain the origins of new scientific ideas, historians and philosophers of science point to examples where scientists appear to have drawn analogies between their scientific domain and some very different domain. By contrast, research from the psychology lab suggests that those kinds of analogies are very difficult to obtain in even the simplest situations. To resolve this potential conflict, we examine the analogies that occur in psychology lab group and formal colloquium settings. This approach can be viewed as a cross-sectional approximation of an historical analysis. We find that as the setting moves further away from the original discovery, the way different types of analogies appear to be used changes. In particular, analogies between very different domains are never used in reasoning in the lab group, whereas they are frequently used in reasoning in formal colloquium presentations. Yet, we find that analogy between very similar domains remains an important source of new ideas and a method for solving problems in scientific settings.
How much information about meaning is contained in the statistical structure of the environment? LSA is a theoretical and practical tool that is challenging previous notions about what is contained in the statistical structure of the environment. This paper examines what kind of category knowledge can be obtained from the environment using LSA. In particular, two experiments are conducted with LSA to test what kind of category structure it embodies. LSA ratings about the relatedness of categories to their properties are compared with human judgments regarding the centrality of properties to the categories. LSA is found to capture aspects of property centrality for some object and event categories. However, it is found to only capture those aspects related to typicality: how often do members of the category have that property? LSA fails to capture other aspects of centrality that can be found in human category judgments. Thus, it appears that humans do bring other constraints to bear in shaping their categories.
In a case study of the growth of cognitive science, we analyzed the activities of the Cognitive Science Society with a particular emphasis on the multidisciplinary nature of the field. Analyses of departmental affiliations, training backgrounds, research methodology, and paper citations suggest that the journal Cognitive Science and the Annual Meeting of the Cognitive Science Society are dominated by cognitive psychology and computer science, rather than being an equal division among the constituent disciplines of cognitive science. However, at many levels, a growing percentage of work was found to involve a conjunction of multiple disciplines, such that approximately 30-50% of recent work in the Cognitive Science Society is multidisciplinary. In a questionnaire study of cognitive scientists involved in collaborative research, multidisciplinarity was found to shape the research process and affect the factors associated with successful research. Full paper
How do people decide whether to try to retrieve an answer to a problem or compute the answer by some other means? We report two experiments showing that this decision is based on problem familiarity rather than retrievability of some answer (correct or incorrect) even when problem familiarization occurred 24 hours earlier. These effects at the level of the individual problem solver and results reported by Reder and Ritter (1992) are well fit with the same parameter values in a spreading-activation computational model of feeling-of-knowing in which decisions to retrieve or compute an answer are based upon the familiarity or activation levels of the problem representation. We therefore argue that strategy selection is governed by a familiarity-based feeling-of-knowing process rather than a process that uses the availability of the answer or some form of race between retrieving and computing the answer. Full paper
The mechanisms by which a concept used in solving one complex task can influence performance on another complex task were investigated. We tested the hypothesis that even when subjects do not spontaneously make an analogy between two domains, knowledge of one domain can still spontaneously influence reasoning about the other domain via the mechanism of priming. Four groups of subjects (two experimental and two control) were given a simulated biochemistry problem on Day 1 and a simulated molecular genetics problem on Day 2. For the two experimental groups, the solution to the biochemistry problem involved inhibition. For the two control groups, the solution did not involve inhibition. On Day 2, all subjects received the same version of the molecular genetics problem in which the solution involved the concept of inhibition. Subjects in the experimental conditions were more likely to attain the correct answer, to propose inhibition, and to propose inhibition early in the problem solving session, than subjects in the control conditions. However, subjects in the experimental conditions made no reference to the biochemistry problem either in their verbal protocols, or a post-task questionnaire. The results are interpreted as demonstrating that an implicit process -priming- can make old knowledge available for current problem solving. Full paper
Also appears as:
Schunn, C.D., & Dunbar, K. (May, 1995) Priming, analogy, and
awareness in complex reasoning. McGill Papers in Cognitive Science.
Technical Report No.1195.
There are two traditional approaches to the study of individual differences in cognitive skill. One assumes that people differ in the strategies that they use. The other assumes that all people use the same strategies/processes but differ in one or more performance parameters affecting how these processes are executed (e.g., speed, memory capacity). This chapter explores another possibility, that people differ in how well they adaptively shift strategies in response to changing features of the task environment. To test this, we examined the performance of 57 participants in the Kanfer-Ackerman Air Traffic Control Task (c), a dynamic task in which features of the environment frequently change. We found that, overall, participants adapted their strategy selections in response to our manipulations of the task environment. However, not all participants were equally adaptive. Furthermore, using the CAM 4 (Kyllonen, 1993), a cognitive assessment battery, we were able to determine what cognitive subskills were associated with adaptiveness. In this context, we found that inductive reasoning skill (and not working memory, declarative learning, procedural learning, or processing speed) was associated with adaptiveness to our specific manipulations, and to the general dynamic character of the ATC task.
Two series of experiments investigating the nature of category structure are presented. The studies focused on object categories and event categories. It was found that, for both objects and events, property centrality (a rating of how important the property is to the category) is not entirely predicted by property typicality (how frequently the object or event possesses that property). By contrasting conditions in which adult subjects rank-ordered properties according to various criteria, it was found that causal theories about the role of the properties in the categories strongly predicted property centrality, although recognitional and definitional factors did play some role. This relationship held for both familiar and newly acquired categories. Recent empirical work on the attribution of causal relationships to sequences of events is reviewed. The evidence from both the past work and the current studies indicates humans do indeed have causal theories which they use to categorize objects and events. Full paper
Previous research on scientific reasoning has shown that it involves a diverse set of skills. Yet, little is known about generality of those skills, an important issue to theories of expertise and to attempts to automate scientific reasoning skills. We present a study examining what kinds of skills psychologists actually use in designing and interpreting experiments. The results suggest: 1) that psychologists use many domain-general skills in their experimentation; 2) that bright and motivated undergraduates are missing many of these skills; 3) some domain-general skills are not specific to only scientists; and 4) some domain-specific skills can be acquired with minimal domain-experience.
The process of scientific discovery has been characterized as a search in two problem spaces: the space of possible experiments and the space of possible hypotheses (Klahr & Dunbar, 1988). More recently, there have been several proposals to include additional problem spaces. In particular, we have proposed the addition of a space of experimental paradigms and a space of data representations (Schunn & Klahr, 1995). These proposals raise meta-theoretical issues: (a) Why these spaces? (b) Why only these spaces? Moreover, these issues are not specific to models of scientific discovery---they are general to all problem solving activities. We propose a general set of logical, empirical and implementational criteria for resolving these issues. We illustrate this distinction in the context of distinguishing between the data representation and hypothesis spaces in scientific discovery, focusing on results of psychological labs studies of discovery activities.
The field of cognitive science is inherently multidisciplinary. However, it is unclear to what extent truly interdisciplinary work occurs in cognitive science. That is, is cognitive science merely a collection of researchers from different disciplines working separately on common problems? Data gathered from a recent cognitive science conference are presented. Interestingly, a significant proportion of interdisciplinary collaborations were found. Analyses were also conducted on the impact of same vs. different backgrounds on the structure of collaborations, and it was found that interdisciplinary collaborations involved more equally distributed contributions among the authors than did intradisciplinary collaborations.
An extension of Klahr and Dunbar's (1988) Dual space model of scientific discovery is presented. We propose that, in addition to search in an experiment space and a hypothesis space, scientific discovery involves search in two additional spaces: the space of data representations and the space of experimental paradigms. That is, discoveries often involve developing new terms and adding new features to descriptions of the data, and the also often involve developing new kinds of experimental procedures. The 4-space model was motivated by the analysis of human performance in a discovery microworld. A brief description of the data is presented. In addition to the general 4-space framework, a description of the component processes involved in each of the four search spaces is also presented.
Other-generated hypotheses are often considered easier to test than self-generated hypotheses. To determine the precise effects of other-generated hypotheses, we propose three kinds of effects and describe a study designed to test for these effects of hypothesis source. The three kinds of effects considered are: (i) hypothesis plausibility changes, (ii) skepticism changes, and (iii) process changes. Forty-two undergraduate subjects were given a microworld discovery task called Milktruck. Subjects either had to generate their own initial hypothesis or were given the most frequently generated hypothesis. It was found that the other-generated hypothesis lead to more thorough investigation of hypotheses, resulting in a decrease in false terminations with incorrect solutions. The results suggested these effects were caused by an increase in skepticism rather than changes in hypothesis plausibility or process changes.
Previous psychological research about scientific discovery has often focused on subjects' heuristics for discovering simple concepts with one relevant dimension or a few relevant dimensions with simple two-way interactions. This paper presents results from an experiment in which subjects had to discover a concept involving complex three-way interactions on a multi-valued output by running experiments in a computerized microworld. Twenty-two CMU undergraduates attempted the task, of which sixteen succeeded, in an average of 85 minutes. The analyses focus on three strategies used to regulate task complexity. First, subjects preferred depth-first to breadth-first search, with successful subjects regulating the number of features varied from experiment to experiment most effectively. Second, subjects systematically regulated the length of their experiments. Third, a new explicit search heuristic (Put Upon Stack Heuristic) used by successful subjects is described.
Back to the home page
Number of visits to this site since 8/2/01: