I wrote reviews of creative thinking tests during my study at the Lettuce University of Salads in 2006-2007. The tests attracted me as they present various games to practice thinking skills as well as different ways to conceptualize and operationalize creativity. While I personally oppose any scientific and managerial thinking about tests and creativity that organizes people into hierarchies, the entries featuring creativity tests in this museum do not aim to criticize the tests. Information about the scientific and managerial aspects of the tests will be retained.
This entry offers a very brief introduction to divergent thinking tests, with a focus on the Torrance Tests of Creative Thinking. More information about the Torrance Tests of Creative Thinking could be accessed via the following links:
- Torrance Tests of Creative Thinking – Figural and Verbal Demonstrator
- Torrance Tests of Creative Thinking- presentation made by Evelien Rogie
- Can We Trust Creativity Tests? A Review of the Torrance Tests of Creative Thinking (TTCT)
- The Creativity Crisis: The Decrease in Creative Thinking Scores on the Torrance Tests of Creative Thinking
1. Guilford’s Structure of the Intellect
The term divergent thinking was introduced by Guilford in 1950 as a part of his Structure of the Intellect (SOI) model of intelligence. The model has been revised several times, and in its most recent form, the SOI model includes 180 components formed through combinations of types of content, operation or product (Starko, 2005). The SOI model is represented in Figure 2.1.
Figure 2.1. The Revised Structure of the Intellect Model
In the SOI model, divergent thinking, defined as thinking of many possible responses to a given question, is considered as one of the basic processes of intelligence. Four components of divergent production are identified: fluency (generating many ideas), flexibility (generating different types of ideas from different perspectives), originality (generating unusual ideas), and elaboration (adding to ideas to improve them). Guilford suggested divergent thinking abilities as inherent to creative people. He also recognized the importance of sensitivity to problems and redefinition abilities in creative thought (Torrance, 1988). Accordingly, creative thinking cannot be equated with divergent thinking. However, assessing creativity in terms of divergent thinking has been so widely used that tests of divergent thinking have tended to become synonymous with tests of creativity in the research literature (Christensen, 1997).
Guilford assumed that creativity and intelligence are largely unrelated and Torrance, the most prolific constructor of creative thinking tests, has designed his tests in keeping with Guilford’s assumption. However, the relationship between creativity and intelligence is theoretically left unanswered.
2. Torrance Tests of Creative Thinking
Developed by Torrance and his associates in 1966, the Torrance Tests of Creative Thinking (TTCT) have had more than 40 years of development. They have been renormed four times in 1974, 1984, 1990, and 1998. There are two forms (A and B) of the TTCT-Verbal and two forms (A and B) of the TTCT-Figural. As these tests are the most frequently used in both education and research (Cramond, 1999), the discussion of the TTCT is more detailed in comparison with that of other tests.
2.1. Description of TTCT Tasks
The tests prescribe open-ended activities. While the TTCT-Verbal requires verbal responses, the TTCT-Figural involves responses that are drawing or pictorial in nature. The only required verbal response is labelling some of the pictures that have been drawn. The two sets of tasks corresponding to the two tests are presented in Table 2.1.
Table 2.1. Open-ended Activities of the TTCT
|Seven Word-based Activities of the TTCT – Verbal|
|Asking||list all the questions the participant can think of about a given picture (e.g. an elf-like form observing his reflection in the water)|
|Guessing Causes||state as many causes as possible causes of the occurrence in the picture given in the Asking task|
|Guessing Consequences||mention possible consequences of the situation pictured in the Asking task|
|Product Improvement||list possible improvements for a product (e.g. a stuffed toy elephant)|
|Unusual Uses||list unusual uses for common objects (e.g. cardboard boxes)|
|Unusual Questions||suggest unusual questions about the objects mentioned in the Unusual Uses task|
|Just Suppose||describe all the things that might happen if an improbable situation (clouds having strings attached that hang down to the earth) should occur|
|Three Pictured-based Activities of the TTCT – Figural|
|Picture Construction||draw something clever and unusual using an egg shaped figure on a piece of paper as the basis for the picture|
|Incomplete Figures||stretch presented variety of abstract lines or designs into unusual pictures or objects|
|Parallel Lines||essentially the same except that all the line forms are pairs of straight, parallel lines|
2.2. Score Dimensions of the TTCT
Basically, the four categories of fluency, flexibility, originality, and elaboration remain the backbone skills of divergent thinking. However, as Torrance became dissatisfied with these scoring criteria, some changes occurred.
On the TTCT-Figural, Torrance was concerned about the high correlation between flexibility and fluency scores and the failure to measure additional creative attributes that individuals demonstrated (Cramond et al., 2005). As a result, in 1984, a “streamlined” scoring of the test became available. The system includes five norm-referenced scores and 13 criterion-referenced measures of creative strengths (see Table 2.3). Torrance deleted flexibility, and added two new norm-referenced scores: abstractness of titles and resistance to premature closure.
On the TTCT-Verbal, elaboration was removed from scoring because it is difficult to get interrater reliability for untrained scorers on that dimension (Cramond et al., 2005). Torrance also planned to add a list of criterion-referenced scores to the verbal test, but did not get to do so before he died (Cramond et al., 2005).
The norm-referenced score dimensions of the TTCT are summarized in Table 2.2., mostly based on the discussion by Cramond (1999). Comments by other authors are indicated.
The TTCT also yields a creativity index, a measure that serves as an overall indicator of creative potential. This composite score is achieved by pooling the creative strength ratings and the average standard score from the profile.
Table 2.2. Six Norm-referenced Score Dimensions of Creativity Measured on the TTCT
|Dimension||How to score||What to score||Comments|
(Figural & Verbal)
|a count of relevant responses given to a novel stimulus||ability to respond to both a single stimulus in several different ways and to respond to many different stimulus effectively||assessment of divergent thinking may be provided by scoring only fluency (Christensen, 1997).
ideational fluency was a precondition for original ideas (Christensen, 1997).
(Figural & Verbal)
|based on the statistical infrequency of a pertinent response||a measure of the unusualness of a response||an important dimension of creativity, but taken alone it may be more indicative of style of response than degree of creativity|
|a count of different categories reflected in relevant responses||ability to shift mental set and produce responses from several different categories//
ability to break set, reconceptualize the problem, and respond in a different way
|removed from scoring on the figural test in 1984 due to a high correlation with Fluency (Cramond et al., 2005).|
|credit is given for each pertinent detail (idea, piece of information, etc.) added to the original stimulus figure, its boundaries, and/or its surrounding space.||ability to add details to create a more complete response.||found to correlate most with school achievement
highly developmental (tending to increase in higher grades)
|Abstractness of Titles
|ability to capture the essence of an idea and go beyond the concrete response
ability to synthesize the response into a meaningful caption (a verbal measure)
|a high score on this , esp. if it is much higher than the other scores, would be a good indicator that the child should be given a verbal measure of creativity.
indicative of higher level thinking
|Resistance to Premature Closure
|tendency to close the incomplete figure immediately with straight or curved lines||ability to resist the tension toward closing the figures in the simplest way
ability to handle open-ended situations and ambiguity
|affects the originality score|
Table 2.3. 13 Criterion-referenced Measures of Creative Strengths
|What to measure||How to measure|
||e.g., in drawings, title||Any genuine appearance of a strength is indicated by a plus sign (+). If the strength appears three or more times, this is indicated by two plus signs (++).|
||e.g., inside, cross section|
||e.g. contest, environment|
||e.g., running, dancing, flying, falling|
||in titles, captions, drawings|
||combination of two or more|
||variety, vividness, strength|
||e.g., exactingness, earthiness|
||e.g., above, below, at angle, etc.|
||e.g., figures in myths, fables, fairy tales, science fiction|
2.3. Technical Issues of the TTCT
Research has shown that the TTCT is a reliable assessment battery. The test manual reports a median inter-rater reliability derived from a number of studies of the TTCT-Verbal as high as .97 and other research indicates that the inter-rater reliability of both the TTCT-Verbal and TTCT-Figural is commonly greater than .90 (Cropley, 2000). Starko (2005) noted that despite high correlations across raters, significant mean differences across self-trained raters were found. That means raters may rank the test performances in approximately the same order, but some judges may give generally higher scores than others. Thus, if the tests are used for comparison, all the students should be evaluated by the same judge(s), or adjustments should be made to compensate for differences in raters. Similarly, program evaluation using the TTCT should employ the same scorer(s) for pre- and posttest.
TTCT test-retest reliability ranges from .50 to .93 with most retest figures in the .60 and .70 range. (Wilcoxson, 2005).
The predictive validity of the Torrance tests varies in different research studies (Starko, 2005). Torrance (1988) reported two longitudinal studies in which subjects’ results on early versions of the TTCT were correlated with their creative accomplishments as adults after 12 and 22 years. The correlations ranged from .51 to .63, far from perfect but as high as most predictive validity of scores of achievement or intelligence tests. Howieson (1981, 1984, as cited in Starko, 2005) found that the TTCT-Verbal did not predict adult creative achievement measured 23 years later at levels that were statistically significant. Both Torrance and Howeison found that the tests were more accurate in predicting creative accomplishments for males than for females (Starko, 2005).
Given the complex nature of creativity, the TTCT has been criticized for measuring a limited number of abilities. Hence, “naming a specific test performance as an indicator of creativity would not be possible” (Wilcoxson, 2005, p. 89). However, studies of the TTCT have supported its discriminant validity. Torrance (1979, as cited in Kirchenbaum & Armstrong, 1999) found that if the students scoring among the top 20% on an IQ test were selected for special programs, 70% of the students scoring among the top 20% on a creativity test would be excluded due to the low correlation between intelligence and creativity tests. This finding is affirmed by research showing that students selected for a gifted program had lower creative thinking test scores than the norm group (Federhar, 1983, as cited in Kirchenbaum & Armstrong, 1999).
Sources of Variability
According to Johnson and Fishkin (1999, p. 277), “motivational and environmental variations of test administration and of the individual’s present state potentially contribute to error of measurement, thereby lowering the test’s reliability and its validity”.
Experimentation regarding the ways of administering the TTCT suggests that the conditions under which the test is taken can have a significant effect on the scores. Torrance (1988) listed 36 studies that experimentally varied the environmental conditions and task unfamiliarity. Of these, 27 studies found significant differences in scores on at least some test forms after changes in test conditions. Torrance (1988, p. 62) commented: “From them we learn much that is useful about maintaining classroom conditions favorable to learning and creativity”.
Generally, warm-up exercises that set a climate for reflection and incubation of ideas result in higher scores. Permissive, untimed, gamelike testing conditions produced higher scores than standard test administration conditions for some of the studies, but not for others (Johnson & Fishkin, 1999). These findings imply that if students are compared using scores from the TTCT, testing conditions should be made consistent. Furthermore, the individual’s level of motivation, persistence, and self-confidence, and the perceived relevance of testing tasks to real-life activity may influence test scores (Johnson & Fishkin, 1999).
3. Other Tests of Divergent Thinking
Torrance and his associates developed two other important divergent thinking tests. In “Thinking Creatively in Action and Movement” (TCAM) (Torrance, 1981, as cited in Jonhson & Fishkin, 1999 and Starko, 2005), fluency and originality are examined as expressed in movement. That shows another alternative for assessing divergent production by varying the kind of required responses so that the test can be administered to children of preschool age. “Thinking Creatively with Sounds and Words” (TCSW) (Torrance, Khatena, & Cunningham, 1973, cited by Jonhson & Fishkin, 1998 and Starko, 2005) presents a different use of stimuli. Abstract sounds and onomatopoetic words are employed to stimulate mental images which are then evaluated on originality. Guilford also introduced the Creativity Tests for Children in 1973. However, “at this point, it appears that the second generations of divergent thinking tests (notably the TTCT) have outdistanced Guildford’s original measures in estimates of reliability and validity” (Starko, 2005, p. 432).
Another influential divergent thinking battery is the Wallach and Kogan Tests (1965), whose major contribution was perhaps their emphasis on an untimed, gamelike atmosphere in the testing procedure (Cropley, 2000; Starko, 2005). One unusual aspect to the scoring of the Wallach and Kogan battery is the score of uniqueness, a count of the number of ideas given that are not given by any other member of the group tested. That may help to account for cultural differences, but scores are heavily affected by the number of individuals taking the test (Starko, 2005).
4. Suggestions for Further Development
Divergent thinking tests have received suggestions for improvement. Score interactions are the most common concern of test reviewers. Fluency is considered as a “contaminating factor” because higher fluency scores accompany higher scores on originality and flexibility (Starko, 2005). It is also suggested that more valid scores could be obtained by summing divergent thinking scores or using a weighted fluency score (Starko, 2005) .
Rather than restricting divergent thinking arbitrarily, researchers have applied the concept to problem finding and problem solving (Christensen, 1997). It is hypothesized that divergent thinking tests might be more accurate in predicting creative behavior if they contain problems more similar to real-world problems.
Christensen, J. R. (1997). A study of divergent thinking skills of gifted elementary students. Unpublished doctoral dissertation. The University of Texas at Austin.
Cramond, B. (1999). Going beyond the scores of the Torrance Tests of Creative Thinking. In A. S. Fishkin, B. Cramond, & P. Olszewski-Lubilius (Eds.), Investigating creativity in youth: research and methods (pp. 307-327). Cresskill, NJ: Hampton Press, Inc.
Cramond, B., Matthews-Morgan, J., Bandalos, D., & Zuo, Li (2005, Fall). A report on the 40-year follow-up of the Torrance Tests of Creative Thinking: alive and well in the new millennium. The Gifted Child Quarterly, 49, 283-293.
Cropley, A. J. (2000). Defining and measuring creativity: are creativity tests worth using? Roeper Review, 23, 72-80.
Guilford, J. P. (1987). Creativity research: past, present and future. In S. G. Isaksen (Ed.), Frontiers of creativity research (pp. 33-65). NY: Bearly Limited.
Johnson, A. S. & Fishkin, A. S. (1999). Assessment of cognitive and affective behaviors related to creativity. In A. S. Fishkin, B. Cramond, & P. Olszewski-Lubilius (Eds.), Investigating creativity in youth: research and methods (pp. 265-327). Cresskill, NJ: Hampton Press.
Kirschenbaum, R. J., & Armstrong, D. C. (1999). Diagnostic assessment of creativity in students. In A. S. Fishkin, B. Cramond, & P. Olszewski-Lubilius (Eds.), Investigating creativity in youth: research and methods (pp. 329-349). Cresskill, NJ: Hampton Press.
Starko, A. J. (2005). Creativity in the classroom: schools of curious delight (3rd ed.). NJ: Lawrence Erlbaum Associates.
Torrance, E. P. (1988). The nature of creativity as manifest in its testing. In R. J. Sternberg (Ed.). (1988), The nature of creativity (pp. 43-75). Cambridge University Press.
Wilcoxson, J. I. T. (2005). Attention deficit hyperactivity disorder and creative potential of children: a multiple case study. Unpublished doctoral dissertation. Saybrook Graduate School and Research Center, San Francisco, California.
 One involves high school students tested in 1959 and followed up 7 and 12 years later. The other involves elementary school pupils tested in 1958 and for 5 subsequent years and followed up 22 years later (Torrance, 1988)
 The criteria of creative adult behavior consisted of the following indices (Torrance, 1988)
- Quantity of public recognized and acknowledged achievements
- Quality of creative achievements
- Quality of creative achievements implied by future career image
- Quality of high school creative achievements (used only in elementary school study)
- Quantity of creative style of life achievements, not publicly recognized(such as organizing an action-oriented group, designing a house, designing a garden, etc.) (used only in elementary school study)
 Test takers’ responses should be compared with responses of others in the same group to take into account the fact that a response that might have been unique in a norming population might be very common in the tested group (Starko, 2005).