Mental ability tests measure differences in tasks that will soon be performed for all of us by computational agents. Such abilities probably have nothing to do with genius.
1. This book's title (Jensen 1998, 1999) refers to the science of mental ability, 'not' the science of intelligence. Why is that? Because, as Jensen himself explains in "Chapter 3: The Trouble With 'Intelligence'," 'intelligence' is irremediably equivocal, as two historic conferences have made painfully clear. The first was held in 1921 (Thorndike et al. 1921), with the purpose of producing a consensus definition of 'intelligence.' Despite the attendance of some rather clever chaps (e.g., Terman, Thorndike, and Thurstone), when the end of the conference rolled around, 'intelligence' was, if anything, murkier than ever. Jensen reports that fourteen different definitions were proposed, including such wonderfully helpful phrases as Terman's "the power to think abstractly" and Thurstone's "that which can be judged by the incompleteness of the alternatives in the trial and error life of the individual." Sixty-five years later, Sternberg & Detterman (1986) valiantly organized a second attempt, in the form of a symposium entitled "What is Intelligence?" The result was the same: no agreement even on an intuitive account, let alone on a formal definition of the sort that is surely a 'sine qua non' for a true science of intelligence. At this point, if there is consensus about anything, it should be that a third attempt would be utterly quixotic.
2. Jensen, of course, believes he has a solution, one reflected in the subtitle of his book: it is to supplant 'intelligence' with 'mental ability,' and to offer a suitably precise definition of the latter. We read:
"To put the study of mental ability on a firm scientific footing, we must begin by using theoretically neutral, objective, operational definitions. From this position "intelligence" (or any synonym or conceptual substitute for it) never needs to enter the discussion. Just blot out whatever this word with all its ambiguities and emotional baggage may mean to you (or your dictionary). (Nor shall I try to provide another word or concept as a verbal substitute.)" (p. 49)
What does this imply about the connection between the science of intelligence and Jensen's science of mental ability? The answer is obvious. On Jensen's scheme, there simply is no connection -- because the former "science" has been "blotted out." Given this, if it should turn out that the science of mental ability is pointless when considered in and of itself, Jensen will not be able to give it significance by some such move as claiming that this science illuminates the concept of intelligence.
3. Jensen stipulatively defines mental ability through a series of definitions that build upon each other. His first step is to define 'item performance', or just an IP, for short:
[Df-1] An IP is any distinct voluntary behavioral act that is observable and recordable in some way. Examples include: saying or writing "four" in response to the question "what is two plus two?", doing a triple axel, writing one's name, hitting middle C on the piano, ad indefinitum.
(Had I sufficient space, I could develop a refutation of Jensen's entire project from the fact that Df-1 cannot possibly be a precise definition of 'anything', for within it occurs a word -- voluntary -- that, despite centuries of analysis, is hardly less dark and impenetrable than it was in the days of the pre-Socratics. Could physics, the mathematical quarter of which lays an unrivaled claim to being a true science, allow itself to rest upon a definition containing a primitive that has even the slightest whiff of the sort of ambiguity plaguing 'voluntary'?)
4. Next, Jensen defines an 'ability' as follows:
[Df-2] An ability is an IP that (i) has some specified degree of temporal stability (consistency or repeatability); (ii) can be reliably classified, measured, ranked, rated, graded, or scored; (iii) has some specified degree of generality.
(The third clause here precludes holding that Jones has the ability to repeat a string of seven digits when he does so once but is unable to repeat the feat in any of, say, 10 subsequent trials.)
5. Now finally we come to Jensen's definition of mental ability:
[Df-3] An ability is a mental ability if (i) an individual's performance on it is not essentially dependent on any particular sensory or motor system, and (ii) within a group of people who have no major sensory or motor handicap (as independently determined), individual differences in the ability are insignificantly correlated with measures of sensory acuity, physical strength, endurance, agility, or dexterity.
As will come as no surprise to even readers with but slight background in the study of intelligence, the specific mental abilities that by Jensen's lights are the best to focus on when doing science in this domain are those performed when established "intelligence" tests are taken. And among such tests, the one that Jensen considers best is the Raven Progressive Matrices Test (RPM). (Jensen holds RPM in such high regard (relative to other "intelligence" tests such as the Wechsler) because among all the tests that are used in his science of mental ability, RPM typically has the highest g loadings, usually about .80. Moreover, RPM has very low loadings on factors other than g.) In this test, subjects are confronted with 3x3 arrays, or matrices. Each location in the array contains a geometric shape, except for location 1,3, which is empty. The subject must choose from a number of possible geometric figures the one which, when located in 1,3, will continue a certain pattern implied by the figures located in the rest of the array.
6. Clearly, the science of mental ability, according to Jensen, is one in which measuring, grading, ranking, scoring, etc. are crucial [recall clause (ii) of Df-2]. Furthermore, measurement and such is conducted exclusively via administering and measuring performance on certain tests (and, sometimes, even narrower "elementary cognitive tasks," such as pushing a button as fast as possible in response to a signal), and the best of such tests, according to Jensen', is RPM. To extract a key fact:
[P1] The science of mental ability implies that if agent x invariably scores significantly higher on RPM than agent y, x has higher mental ability than y.
Of course, there are many propositions like P1 that can be extracted from Jensen's book, and many of these are perhaps a bit more circumspect than P1. For example, here is one such proposition:
[P1'] The science of mental ability implies that if agent x time and time again invariably scores significantly higher on RPM and other tests as well (e.g., the Wechsler Intelligence Test) than agent y, then x has higher mental ability than y.
For ease of exposition in this short review I use P1 and focus on RPM. The argument I give below based on AI agents could be refined by appeal to P1' and the like, and by adjustments that that appeal would require.
7. The field of Artificial Intelligence (AI) is devoted to building artificial agents. Such agents take in percepts from the environment, convert these percepts into information, manipulate this information, and then perform certain actions on the environment. (For full coverage of these matters, see Russell and Norvig 1994.) As an example, imagine an artificial agent that can play Tic-Tac-Toe. Percepts would be images showing the familiar configurations (e.g., a 3x3 matrix, blank save for an O in location 1,3). Actions, of course, would be things like writing down an X or an O (e.g., writing an X in location 2,2). Even undergraduate students can build artificial agents that are unbeatable in the game of Tic-Tac-Toe. This can be done on the strength of basic search techniques.
8. Now, suppose that in my Lab we engineer an artificial agent whose performance on RPM is flawless; call this agent 'R.' This supposition, given proposition P1, and given that Einstein and/or other mental "giants" would not perform flawlessly on RPM and other "intelligence" tests, implies that
[P2] The science of mental ability implies that R enjoys higher mental ability than those who are universally regarded to be geniuses (e.g., Einstein).
But it follows from P2 and
[P3] If the science of mental ability implies that R enjoys higher mental ability than those who are universally regarded to be geniuses, the science of mental ability is silly.
that Jensen's science of mental ability 'is' silly.
9. The logic of this argument is unexceptionable; inferentially, it has the status of a bona fide proof, since it's based on little more than modus ponens and elementary quantifier logic. Hence, 'if' the premises are true, then Jensen is in deep trouble. So, 'are' the premises true? Well, the supposition that R exists must be granted. After all, there are two computer systems that come close to R, FAIRAVEN and BETTERAVEN (Carpenter, Just & Shell 1990). These programs were written in an attempt to simulate the kind of reasoning that humans engage in when confronted with RPM. For well-known reasons, tasks like solving RPM problems can be better automated if the constraint of having to model human reasoning is dropped. Deep Blue, for example, uses alpha-beta minimax search with pruning, which allows it to prevail against even the likes of Kasparov -- but such search reflects a departure from attempts to render in computation how great human chess-players play chess. There is no question whatsoever that standard AI techniques and tools can produce R. (The grandfather of all R-like systems is Evans' (1968) ANALOGY program, a landmark in AI.)
10. What about the remaining premises in my argument, that is, what about P1 and P3? Premise P1 would seem to be unassailable, for it appears to be entailed by Jensen's scheme. Premise P1 merely expresses in stark, declarative form the fact that Jensen has stipulated mental ability to be a "rigorously measurable" thing. If Einstein (and others in the same category, e.g., Shakespeare) took RPM and other tests, and got a number of problems wrong (and this is something Jensen tells us would almost certainly happen, since great geniuses according to him sometimes have IQs at least as low as 120), the brute fact would be that artificial agent R's perfect performance serves to classify it as having higher mental ability than these geniuses.
11. So what about P3? This premise is based on the apparent silliness that attaches to the follow scenario: Imagine that Smith, a professor of art, visits a Professor in his lab. Suppose that Smith knows essentially nothing about the ins and outs of psychometrics, "intelligence" tests, factor analysis, g, and so on. The Professor says that he would like to show Smith something interesting, and proceeds to show her problems from RPM, etc. He also shows her the artificial agent R, and explains how it handles RPM problems. He then announces to Smith: "Do you know what? According to my formal definitions and calculations, R enjoys higher mental ability than Einstein, Michelangelo, Renoir, Shakespeare, and so on." At this point Smith will doubtless conclude that he is just being silly. (Notice that he confronts Smith with a position that is rather different from saying of Deep Blue that it is a better chessplayer than Gary Kasparov, or saying of a bipedal robot that it is a faster runner than Carl Lewis. In these examples, there is nothing to balk at. If Jensen claims only that R is better than Einstein at solving RPM problems, nothing silly is being said.)
12. One rebuttal that isn't available to Jensen, as I took pains to make clear in section I, is to say something like: "Well, but mental ability in my formal sense is really just a precise correlate for the kinds of behaviors that cause us to classify people like Einstein as geniuses." No such rebuttal is available to Jensen because, as I explained, he explictly affirms the opposite position.
13. So, again, what might Jensen say? His book is of no help in furnishing him a cogent reply here. In "Chapter 5: Challenges to g," he does consider and rebut many objections to his position -- but these objections seem to me astonishingly bad, and at any rate they fail to include the one I'm articulating herein. I suspect that Jensen's rejoinder might build upon the fact that in my little parable above, Smith connects 'mental ability' in Jensen's sense of the phrase with its normal, everyday, common-sensical sense. After all, in common parlance, 'mental ability' is a phrase that we use as a rough synonym for 'intelligence.' Hence people say such things as that "She has tremendous mental ability." To put the point in terms of my core argument, the connection between these two senses of 'mental ability' is precisely what undergirds premise P3. So, Jensen's defense would probably be to reject P3 on the basis of a reminder, viz., that he has explicitly said that he is defining 'mental ability' in such a way as to create his own precise definition that is wholly divorced from common construals of this phrase.
14. In keeping with this response, let us call Jensen's own concept 'mental shmability.' Now, what is the point of spending a lifetime studying mental shmability? One might say that some of the relevant math is intrinsically interesting, and though this is doubtful (since factor analysis is rather simple), even if it's true it helps us not one iota in seeing why the study of this particular thing, mental shmability, has a point. Read as an essay on mental shmability, Jensen's book looks to me to be a tedious show of mildly difficult mental gymnastics. The somersaults are a diversion, but what's the point?
15. Not that long ago one could earn a professorship in arithmetical calculation. With the advent of computers, the "science" of such calculation began to seem pointless, and indeed it is, for today's handheld calculators do more, and do it more quickly, than these "brilliant" professors. When agents like R are routine, I predict that a parallel evaporation will wipe away the science of mental shmability, and its focus on this exceedingly narrow phenomenon.
Carpenter, P., Just, M., & Shell, P. (1990) What One Intelligence Test Measures: A Theoretical Account of the Processing in the Raven Progressive Matrices Test. Psychological Review 97(3): 404-431.
Evans, T. (1968) A Program for the Solution of a Class of Geometric Analogy Intelligence Test Questions. In Minsky, M., ed., Semantic Information Processing (Cambridge, MA: MIT Press), pp. 271-353.
Russell, S. & Norvig, P. (1994) Artificial Intelligence: A Modern Approach. (Saddle River, NJ: Prentice-Hall).
Sternberg, R.J. & Detterman, D.K. (1986) (Eds.) What is intelligence? Contemporary viewpoints on its nature and definition. Norwood, N.J.: Ablex
Thorndike, R.L., and others (1921) Intelligence and Its Measurement: A Symposium. Journal of Educational Psychology 12: 123-147; 195-216; 271-275.