**************************************************************************************

The Journal of Social Psychology, 1983, 121, 81-96.

REVIVING THE PROBLEM OF ACQUIESCENT RESPONSE BIAS*



University of New South Wales, Australia

JOHN J. RAY

SUMMARY

Rorer's claim that meaningless acquiescence to attitude and personality scale items does not generalize and hence cannot be a source of artifactual relationships is critically examined. Rorer excepts ambiguous scales from his own generalizations without providing or suggesting any test to tell which scales are ambiguous to a problematical degree; thus his generalizations are of unknown applicability. Three studies are reported in which balanced scales are scored for acquiescence only -- i.e., scored without any reverse-scoring. Many high correlations between such scores were found -- in direct contradiction of the findings used by Rorer. Correlations between acquiescence scores derived from balanced attitude scales and balanced personality scales were, however, generally low. It is concluded that meaningless acquiescence may be evinced by respondents to any scale if they find its items ambiguous and that such acquiescence will generalize from scale to scale. Meaningless acquiescence could then create many spurious relationships between unbalanced scales.

A. INTRODUCTION

Although many psychologists, had been aware of it for some time, the problem of acquiescent response set first rose to prominence in the psychological literature in connection with the California F scale (6, 7, 39). There was a school of thought that people who agreed with F scale items often did so not out of any genuine agreement with the sentiments embodied in the items but simply because they were "acquiescers" -- people who tended to agree with almost any proposition put to them. One might be an acquiescer either out of a desire to please or out of simple indifference to the questionnaire-answering task. A person with a high F scale score might be simply a careless responder rather than a genuine Fascist.

The solution to this difficulty seemed easy enough: Change the wording of half of the items so that they became anti-Fascist. An "acquiescer" on such a "balanced" revision of the scale would then get only an intermediate score and would be rightly categorized as a person who takes no particular position either way on Fascist issues. High scores, in other words, could only be obtained by agreeing with some items (pro-Fascist ones) and disagreeing with others (anti-Fascist ones). Attempts to produce such a revision of the F scale, however, foundered on the fact that the revised items tended to be orthogonal to the original items rather than negatively correlated with them (6, 17, 21, 39). The F scale seemed "irreversible."

If meaningless acquiescence could not be controlled for, then, it had to be measured. Some assessment had to be made of how big a part it played in F scale scores. To many it must seem that the influential paper by Rorer (39) satisfactorily resolved this question. Rorer claimed that acquiescent response "style" (as he called it) is generally an unimportant influence on attitude and personality scale scores. A vital element in his reasoning was the finding that when one tries to measure acquiescence as such, the different scales of acquiescence that we use tend not to correlate with one another. From this fact, Rorer concludes that there is essentially no such thing as a general bias towards acquiescent responding. He reasons that if there were substantial numbers of people who tended to agree with almost anything, then they should get equally high scores on all acquiescence measures and those measures would then tend to intercorrelate. The fact that this does not happen would imply, then, that almost all acquiescence to attitude and personality scale items is genuine and meaningful.

Unfortunately for this optimistic conclusion, Rorer does enter the fairly obvious caveat that his generalizations might not apply to particularly ambiguous scales. He allows the possibility that although acquiescence may not be a trait, it may be a response to a particularly ambiguous questionnaire-answering task. Peabody (16) seized on this point and published an extensive account to show that the F scale was just such an ambiguous instrument. Whether or not what Rorer concludes is generally true, it does not therefore help us with the F scale. In fact, it does not help us with any scale unless we can demonstrate that that particular scale is one of the unambiguous ones. Nobody ever seems to have attempted such a demonstration.

Not only, therefore, are Rorer's conclusions unhelpful in enabling us to exclude meaningless acquiescence as an influence on any particular scale, but they are also unhelpful in setting at rest the doubts about validity that arise where a scale is known to be "irreversible." Even if we accept that meaningless acquiescence is an unimportant influence on F scale scores, how do we explain the fact that it is impossible to write opposites to its items? Surely, if a statement has any meaning at all, it should be possible to deny it. Falsifiability is, after all, one of the most important criteria of empirical meaning among empiricist philosophers. The fact that it is so difficult to write opposites to F scale items would suggest, then, that they are very low in meaningfulness. Alternatively, they may be meaningful but what they mean may be very different from what writers of reversed items suppose. From either point of view, the conclusion must be that the content and construct validity of the items is severely lacking. They mean either virtually nothing or something quite different from what they are supposed to mean or apparently mean.

As it happens, later work has shown that the F scale is not quite as devoid of meaning as the earlier work suggested. By dint of sifting through a large number of alternative reversed items, enough reversed items have in fact been found to form a successful balanced F scale -- i.e., a scale in which the two types of item do in fact correlate highly negatively (13, 17, 24, 25). Thus, although the meaningfulness of the F scale has been shown to be low, it is not nonexistent.

Although the discussion so far has, for the sake of convenience, centered on the F scale, it is important to note that the problem is not confined to that scale alone. Similar validity defects have been observed in at least the Rotter (8) Locus of Control scale, the Christie and Geis (5, 36) Machiavellianism scale, the Blood (3, 32) Protestant Ethic scale, the Budner (4, 31) Intolerance of Ambiguity scale, and the Rokeach (22) Dogmatism scale. All these scales have shown on at least some occasions negligible correlations between their negatively and positively scored items instead of the high negative correlations that should in theory be expected. If, contrary to Rorer, acquiescence is responsible for such effects, it certainly is a problem that is very widespread.

Some evidence has in fact now accumulated that Rorer's conclusions were overgeneralized. As has previously been noted, it is not necessary for acquiescence to be a trait for it to constitute a problem. Even if we conceptualize it as a response to ambiguity it could still give rise to spurious correlations between two unbalanced scales-provided that both scales were ambiguous. What is ambiguous to one person or population may not, however, be ambiguous to another person or population. Thus, the finding that the Wilson (40, 41) Conservatism scale shows strong correlations between its two opposed halves on some occasions but not on others (28) could be evidence that the scale is ambiguous to some populations but not to others. Where it is ambiguous, meaningless acquiescence is elicited: i.e., items tend to be responded to similarly (i.e., with all "Agrees") where their nominal meaning is working to cause the positive and negative items to be responded to oppositely. The two effects (the effect of acquiescence and the effect of the item meaning) are thus opposed and tend to cancel each other out, resulting in the observed orthogonality between the two halves of the scale.

A great difficulty with such effects is that they are substantially unpredictable. For instance, the Ray (18) Environmentalism scale showed a correlation between its positive and negative items (rPN) of -.53 on its norming sample (a door-to-door survey in a big city) but reanalysis of data obtained with the same scale in a small-town door-to-door sample (27) reveals that the rPN there dropped to a nonsignificant -.04. It might be easy with hindsight to explain why two random Australian community samples gave such different results but how could we ever be confident that such differences would not arise between two samples? It appears that validity collapse could strike any scale at any time and checks against such an eventuality should therefore be as routine as tests of statistical significance. To carry out such checks, however, we must do precisely what Rorer thought unnecessary -- use only balanced scales.

Some scales do seem to be more resistant to validity collapse (and thus, inferentially, the effects of acquiescent response bias) than others. Thus, while the Ray (24) Balanced F scale has had its failures (29), the "Directiveness" scale of authoritarian personality has so far on all of its many applications (19, 20, 25, 29, 30) showed significant and negative rPNs. This is also true of the Ray-Lynn Achievement Orientation scale (23, 25, 26, 29, 32). It may be significant that both of these are personality rather than attitude scales. It could be that direct questions about one's own actual behavior are inherently more meaningful and less ambiguous than questions about social attitudes and are hence less susceptible to acquiescence effects.

So far, the limitations of Rorer's position have been outlined but no fundamental attack on his conclusions has been made. There are, however, good grounds for such an attack. The studies upon which Rorer relied were largely based on scales deliberately designed to measure acquiescence (a tactic influenced by the unavailability at that time of a workable balanced F scale). This may have been an unnecessarily inferential approach. Would it not be more instructive to use measures of acquiescence derived from the particular substantive scales with which we wish to work? Any balanced scale can be scored as a scale of acquiescence. One simply adds up the scores on all the items without doing any reverse-scoring (14, 21). The scale thus becomes a measure of the tendency to agree with statements regardless of their meaning. With some scales in some circumstances (21, 38, 35), such acquiescence scores can show high reliabilities in the internal consistency sense ("alpha"). A central inquiry must surely be whether acquiescence scores derived in such a way intercorrelate. In particular, does the acquiescence score from a balanced F scale correlate with the acquiescence scores derived from other balanced scales commonly used in authoritarianism research? If, for instance, meaningless acquiescence to balanced F scale items can be shown to be related to meaningless acquiescence to balanced Dogmatism scale items, then we can with some confidence conclude that a common acquiescence factor alone could explain any correlation between these two scales when administered in their original, unbalanced, forms. Yet another, perhaps even more serious possibility is that some acquiescence scores may have relationships with substantively scored scales. If, for instance, F scale acquiescence is found to correlate with racism as measured by a balanced Ethnocentrism scale, then we may conclude that the original correlation between the unbalanced F scale and this variable could have been an acquiescence artifact alone.

The above argument, then, accepts Rorer's basic datum that acquiescence is not completely generalizable but rejects the inference that this means that acquiescence is completely specific. There could well be an intermediate state where acquiescence has just some generalizability. The number of subtypes of acquiescence could in face be quite small. There is, for instance, some suggestion from Martin's (14) work that acquiescence to personality scales generalizes to other personality scales but not to attitude scales and that acquiescence to attitude scales generalizes to other attitude scales but not to personality scales. Martin's work, however, is somewhat marred by the fact that the scales from which he derived his acquiescence scores were not completely balanced. Some of what he was measuring could therefore have been the substantive meaning of the scales concerned. In any event, it seems quite clear that even a limited degree of generalizability of acquiescent response bias across scales could provide an artifact problem.

Because Rorer's paper virtually eliminated acquiescence as a field of study, finding any evidence of limited generalizability is difficult. Nonetheless two papers of relevance exist. Ray (21) reports significant correlations between balanced F scale acquiescence and acquiescence scores from balanced scales of social desirability and achievement motivation. Given that the latter two scales are personality scales while the BF scale is an attitude measure, the degree of generalizability shown in this study is even greater than that suggested by Martin's work. Possibly of even greater importance are Heaven's (9) results showing BF scale acquiescence score correlations of .380 with the Pettigrew conformity scale, -.324 with an ethnocentrism scale., -.534 with an economic conservatism scale and -.339 with education. He finds, in other words, that not only BF scale content but also BF scale acquiescence has significant relationships of its own. BF scale acquiescence generalizes, then, not only to other acquiescence measures but also to a wide range of other variables. This is precisely the state of affairs feared by the original critics of the F scale and one that runs contrary to the generalizations suggested by Rorer's article.

Heaven's scales, however, were not all balanced: the conformity scale was all-positive, the economic conservatism scale had two positive and eight negative items, while the ethnocentrism scale was exactly balanced. Only with the ethnocentrism scale, therefore, can we be sure that the relationship observed was not partly due to a common acquiescence component. Since the correlation between the F scale and ethnocentrism was the very core of the original Adorno et al. (1) study, Heaven's demonstration that such a relationship was produced solely by acquiescence rather than by F scale content can hardly be overestimated in importance. Heaven shows that the relationship between the original F scale and racism could have been produced by the effects of acquiescence alone. We have evidence from other sources, however, that even balanced F scales predict racism (25, 31), though with correlations much lower than those reported by Adorno et al.

The correlation between BF scale acquiescence and economic conservatism reported by Heaven is so high as to require closer scrutiny. As the correlation is negative, he is reporting that people who acquiesce to F scale type items regardless of their ideological polarity are also strong supporters of socialist economic policies. This contrasts most markedly with the Adorno et al. findings that people who agree with F scale items are strong supporters of conservative economic policies. Part of the reason behind the apparent divergence of findings may lie in the fact that the data were gathered in South Africa. Economically radical policies in South Africa apparently consist of statements such as: "The rights of white workers should be better protected" and "I am in favor of greater government control of prices." At least some of these policies, then, might be seen as conservative elsewhere.

Nonetheless, the scoring of Heaven's economic conservatism scale also seems to be of considerable potential significance. Eight out of 10 items were "radical" and hence were reverse-scored. This did not seem to have any discernible effect on the distribution of the final scale score as the mean observed (20.72) was very close to the theoretical scale midpoint of 20 (where the midpoint is conceived as the number of items multiplied by the indifference score for each item). It did, however, mean that acquiescers must automatically get low scores on the economic conservatism scale. All "Yes" responses were reverse-scored to earn the minimum item score of "1." If we find, therefore, that F scale acquiescers get especially low scores on the economic conservatism scale, we must be rather strongly led to conclude that on the given occasion acquiescence was an overwhelming component of what the economic conservatism scale was measuring. There are no considerations of content that would lead us to postulate any great relationship between BF scale acquiescence and preference for particular government economic policies; yet a very strong negative relationship was found. This suggests that content was not the main effect on economic conservatism score. We have a finding of precisely the character that Rorer believed did not exist: acquiescence from one scale correlates strongly with acquiescence from another scale. BF scale acquiescence and economic conservatism scale score correlate highly because both measure acquiescence of apparently the same type. That the sign of the correlation is negative is simply an artifact of the fact that the economic conservatism scale had almost all items worded in a direction requiring reversal. This may seem a somewhat complex argument but it will be seen that if one assumes the generality of acquiescence, the findings fall into place. Heaven's findings, therefore, also support the view that although acquiescence may not be completely general, it is also not completely scale-specific. There may be more than one type of acquiescence, but the number of such types is apparently finite and could in fact be quite small.

Because of their failure to use uniformly balanced scales, however, both the Heaven (9) and Martin (14) studies pose at least some uncertainties of interpretation. One could, for instance, argue that Heaven's results simply show that BF scale acquiescers are the type of people who want the government to do a lot of things for them. The Ray (21) results suffer from a different limitation: the BF scale acquiescence was correlated only with acquiescence from personality scales. Could it be that the fairly low (though significant) correlations there reported would have been much higher if only attitude scales had been used? The aim of the research reported below, then, is to overcome both these difficulties by examining the correlations between balanced scales of the same general type when scored for acquiescence rather than for content. It is hypothesized that at least some generalizability, some significant correlations, will be found.

B. STUDY I

The hypothesis is examined by reanalyzing data from three studies (20, 22, 34) that employed balanced attitude scales. This reanalysis has the side-benefit of adding to the depth in which the relationships within a particular data set can be studied. One can concentrate on the acquiescence effects in the data without slighting the substantive relationships also present. Briefly, a quota sample of 87 residents of the Australian city of Sydney received a questionnaire containing short forms of scales to measure dogmatism, alienation, internal-external control, Machiavellianism, authoritarianism, achievement-motivation, and social desirability. The authoritarianism scale was the Ray (19) "Directiveness" scale rather than the California F scale in any of its forms; hence the attitude scales in the study were limited to the Dogmatism scale and the alienation scale. All scales, however, were scored for acquiescence and intercorrelated. It was decided to examine particularly closely the scores derived from the balanced Dogmatism scale as the scale closest to the authoritarianism research tradition. The correlations found for Dogmatism scale acquiescence were as follows: achievement motivation acquiescence .356, alienation acquiescence .361, authoritarianism acquiescence .254, social desirability acquiescence .170, internal-external control acquiescence .407, Machiavellianism acquiescence .261. All but one of these correlations (.170) are significant (<.05) and some are very significant, a picture, then, of a high degree of generality in acquiescent responding. If the scales concerned had been used in unbalanced forms, many relationships of apparently great significance would have been found as a result of the presence of a common acquiescence artifact alone. This is precisely what the critics of the original F scale feared and is contrary to what Rorer's paper would have us believe.

A matter that may require some mention in connection with this study concerns the question of scale reliability. As all the scales were balanced scales deliberately designed to eliminate the effects of acquiescence, we cannot expect the internal consistency of the acquiescence score to be anywhere near the internal consistency of the substantive score, but surely some internal consistency has to be demonstrated before we can accept a score as measuring anything. The coefficients "alpha" for the scales when scored for acquiescence were as follows: achievement motivation .205, alienation .318, authoritarianism .183, social desirability -.381, internal-external control .374, Machiavellianism .385, and dogmatism -.001. Five out of seven scales, thus, showed a preponderance of positive relationships between their acquiescence-scored items. With the use of Hoyt's (11) approach to testing the significance of the coefficient, the critical level for significance at the .05 level is .24. Only three of the given reliabilities reach this level.

How, then, do we explain the consistent pattern of correlations? Can a scale (such as the acquiescence-scored balanced D scale) be totally unreliable and still have a range of meaningful correlations? This question has been thrashed out at some length in connection with the use of unreliable projective tests for the measurement of achievement motivation. Although the traditional answer is "No," some vigorous dissent has been registered (2, 12). In the present case, however, where it is acknowledged that there may be several major subtypes of acquiescence with little or no relationship between them, the apparent conflict of findings is easily explained. It would seem that acquiescence of several not-necessarily-related types may be elicited by the Dogmatism scale. This explains the considerable number of other acquiescence-scored scales with which the acquiescence-scored Dogmatism scale correlates and at the same time explains the fact that the acquiescence-scored Dogmatism scale shows no internal consistency. Important for the present purposes is the demonstration that some generalization or acquiescence across scales does occur. How many types of acquiescence there might be and which scales provide the best measure of them must be a later inquiry. Obviously, although the Dogmatism scale must be affected by several types of acquiescence, it is not a good measure of any one of them. This is not inherently surprising, since the Dogmatism scale is something of a mixed bag of attitude scale type items and personality scale type items. Since all the scales used in the study were quite short, having around 10 items each, reliabilities of any kind would be expected to be on the low side.

C. STUDY II

As the study above was primarily concerned with personality scales, it was desired to see whether the same conclusions would emerge with a study involving principally attitude scales. It was decided to use the data from a random postal (mail-out) sample of the Australian states of Queensland and New South Wales which have populations of approximately two million and five million, respectively. Some of the findings from the study have already been reported (30, 33, 37). Four balanced scales and one unbalanced scale were administered to a total sample which finally numbered 219 Queenslanders and 158 New South Wales residents. As 700 questionnaires had been sent out in Queensland and 500 in New South Wales, the response rate in both states was virtually identical at 31%. The scales were a 33-item scale of general social conservatism, a scale of moral conservatism consisting of 16 items covering permissive stances vs traditional stances on issues generally concerned with sexual morality, and an 18-item economic conservatism scale dealing with issues of government involvement in welfare and economic management. The two latter scales had an exact balance of radical vs conservative items, while the first had 16 radical items and 17 conservative items. The questionnaire also contained a balanced dominance scale, having many items in common with the Ray (19) "Directiveness" scale, and the Munro-Adams (15) Attitude to Love scale. As the latter was not balanced, it will not be further considered in this paper. For the purpose of maximizing scale reliability, some of the items of the first three scales have been deleted from consideration in previous papers, but the full set of items will be used here with the exception that one item will be deleted from the general social conservatism scale in order to make it exactly balanced. The item chosen for deletion was the one showing the poorest correlation with the score on the full substantively-scored scale.

The reliabilities (alpha) of the four acquiescence scores were as follows: general social conservatism .55, moral conservatism .30, economic conservatism .53, dominance .37. The three correlations between the acquiescence scores of the three attitude scales were .397, .404, and .461. The correlations between the three attitude scale acquiescence scores and the personality scale acquiescence score were .131, .177, and .190. Given the large N (377), all correlations were significant.

Once again, correlations between acquiescence scores were found at a magnitude which is in no way inferior to that commonly found in other psychological research. The entire basis of Rorer's article has once again been undermined. Acquiescence does generalize from scale to scale.

The findings so far have also done much to validate the generalization that emerged from Martin's work: those who acquiesce meaninglessly to attitude scale items (i.e., items requiring a response of "agree" or "disagree") are very largely different from those who acquiesce meaninglessly to personality scale items (i.e., items requiring a "yes" or "no" response). The multidimensionality of acquiescence may turn out to consist of nothing more than two-dimensionality. Although only one personality scale was used in the present study, it seems clear that its acquiescence score correlated with the attitude acquiescence scores at a level that would have been found as nonsignificant with the sort of (smaller) N more characteristic of published psychological research. Significant statistically though it may be, a correlation of .131 represents a relationship that is hardly likely to be of great explanatory usefulness or importance. It is clear, however, once again that correlations across scale types are much less than correlations between scale types.

At this juncture, it seems pertinent to ask how it was that the present findings were so divergent from those relied on by Rorer. Do all questionnaire studies show such a strong effect of acquiescence as the above two studies have shown? If we accept the view that acquiescence is a response to ambiguity, the answer is clearly, No. There must surely be some occasions when all the scales are well-understood and meaningfully answered; however, there seems no easy way to detect when this is so. What is ambiguous to one group may clearly not be ambiguous to others. The big-city group mentioned earlier (18) gave responses to the Ray Environmentalism scale that were highly meaningful relative to the intention of the scale; the small-town group (27), on the other hand, answered in a way that betrayed a total divergence from the scale's intended meaning.

The only possible guidelines seemed to be highly subjective ones: It is possible that studies using well-tested and highly reliable scales might betray less influence of acquiescent set than others. One further study from the present author's archives was, therefore, selected for reanalysis in the hope that it might meet these guidelines.

D. STUDY III

The study selected for reanalysis (25) was a random doorstep sample of 101 Los Angelenos answering scales of authoritarian personality (the Directiveness scale), authoritarian attitudes (the BF scale), achievement motivation [the Ray "AO" scale (23, 26)], and ethnocentrism [the Ray "Attitude to Aborigines scale as modified by Heaven and Moerdyk (10) to make it an Attitude to Blacks scale]. All scales had undergone extensive pretesting, were explicitly designed as balanced scales, and had shown reliabilities which, given the short 14-item length of each scale, seemed relatively high. The reliabilities of the substantively scored scales as assessed by coefficient alpha were as follows: directiveness .73, achievement motivation .72, attitude to blacks .88, and BF scale .79. The reliabilities for the substantively-scored scales in Study II above had been similar (general social conservatism .75, moral conservatism .82, economic conservatism .68, and dominance .87) but were in two cases derived from much longer scales and the scales themselves were not standard ones but ones especially devised for that study. Given the constantly changing nature of what constitutes political and social issues, it is an occupational hazard of conservatism measurement that one's scales must constantly be revised if they are to remain topical and relevant. Wilson and Patterson's (41) Conservatism scale, for instance, list as issues "Computer music" and "Pajama parties." It is doubtful if such items nowadays would elicit anything but acquiescent responding.

On reanalysis, the reliability of the acquiescence score for each scale was found to be as follows: directiveness .45, achievement motivation -.56, ethnocentrism .20, and balanced F .33. The correlations between the negative and positive halves of the same four scales were -.290, -.647, -.554, and -.445, respectively. The BF acquiescence score correlated -.038 with the directiveness acquiescence score, .110 with achievement motivation acquiescence score, and .122 with the ethnocentrism acquiescence score. None of these three latter correlations was significant at the .05 level. There were, in fact, no significant correlations between any pair of acquiescence scores.

E. DISCUSSION

The present work has not only shown that the basic generalization upon which Rorer relied was wrong, but it has also shown why it was wrong. In very carefully prepared studies which in fact work in the way intended, meaningless acquiescence (or "acquiescent response style") does not emerge and scores of pure acquiescence will hence tend not to show any relationship. In Study III above, for instance, the meaning of the achievement motivation and attitude to blacks scale was evidently so well taken by the Californians answering them that there was evidently virtually no meaningless agreement with the items of those scales. This may be inferred both from the low alpha coefficients of the acquiescence scores of these scales and the high correlation between the positive and negative halves. Since so little acquiescent response style was evident in these scales, there could not be a common acquiescence influence giving rise to a correlation with other scales (the Directiveness and BF scales) where such acquiescence clearly was present. Then why did not the acquiescence scores of the BF and Directiveness scales correlate? An answer to this has also been provided in the present work, in the form of the repeated finding that acquiescence to personality scales and acquiescence to attitude scales generally have been found to be quite different things.

In all the above, it has been assumed that we can accept Martin's (14) practice of measuring the acquiescence present in a balanced scale by adding item scores without any reversals of the negative items. Rorer might wish to challenge this. In his 1965 paper, he stresses the claim that this cannot correctly be done with balanced versions of the F scale. He points out that agreement with both original and reversed F scale items may be quite meaningful and in no way contradictory. An "acquiescence" score from such a scale may therefore really measure some sort of content. As the present work has concentrated on scales that have been shown generally to have highly significant rPNs (in the expected direction), however, this criticism is much less applicable to the studies reported herein. To agree with all the items in any one of the scales used above does demonstrably involve the respondent in contradictory assertions. Acquiescing to demonstrably opposed sentiments is simply the operational definition of meaningless aquiescence that is adopted in the present paper. That the present report has shown such responding to generalize from scale to scale does therefore reinforce the idea that the operational definition is sound. It shows that correlations can be obtained between items almost regardless of their ostensible theme or themes. Acquiescent bias is simply the most parsimonious explanation of the correlations.

While there may be occasions on which acquiescence scores do not correlate with one another, the present paper has shown that we cannot rely on such being the case. What Rorer and many others subsequently seem to have overlooked is that we can never know in advance that our measurement intentions will be successfully realized. In many cases not readily predictable, the intentions behind the scale items will not be matched by similar perceptions in the subject population and the plague of acquiescence as a response to ambiguity will descend upon our work. People will acquiesce meaninglessly when the item they are answering does not make much sense to them, and that acquiescence will carry over from one scale to another. It will be a potential source of artifactual relationships.

While Rorer acknowledged that acquiescent response style could be a problem with particularly ambiguous scales, he did not seem to realize that any scale can be ambiguous in some applications or on some occasions.

In both the present and a previous paper (28) many instances have been reported of a scale that initially seemed to work very well but later was found to work very much less well when applied to other samples. The only way we can test for attacks of meaningless acquiescence in data is to use balanced scales. Meaningless acquiescence places very obvious stigmata on balanced scales: the reliabilities and positive-negative correlations decline while the reliability of the acquiescence-scored scale rises. With unbalanced scales, however, none of these tell-tales will be present; acquiescence will in fact inflate the reliability and will tend to make the scale look best at precisely the time when it is functioning most poorly.

On the present evidence we must draw the exceedingly melancholic conclusion that any investigation with one-way-worded scales is not only of unknown meaning but is in fact even of unknowable meaning. Data from balanced scales can at least subsequently be reanalyzed to examine whether there was much evidence of meaningless acquiescence present. Unbalanced scales cannot possibly be checked, and all of their relationships could be due to the artifactual influence of acquiescent response style.

REFERENCES

1. ADORNO, T. W., FRENKEL-BRUNSWiK, E., LEVINSON, D. J., & SANFORD, R. N. The Authoritarian Personality. New York: Harper, 1950.

2. ATKINSON, J. W., BONGORT, K., & PRICE, L. H. Explorations using computer simulations to comprehend Thematic Apperceptive measurement of motivation. Motiv. & Emot., 1977, 1, 1-27.

3. BLOOD, M. R. Work values and job satisfaction. J. Appl. Psychol., 1969, 53, 456-459.

4. BUDNER, S. Intolerance of ambiguity as a personality variable. J. Personal., 1962, 30, 29-50.

5. CHRISTIE, R., & GEIS, F. Studies in Machiavellianism. New York: Academic, 1970.

6. CHRISTIE, R., HAVEL, J., $C SEIDENBERG, B. IS the F scale irreversible? J. Abn. Soc. Psychol., 1956, 56, 141-158.

7. CHRISTIE, R., & JAHODA, M. Studies in the Scope and Method of "The Authoritarian Personality." Glencoe, Ill.: Free Press, 1954.

8. GATZ, M., & GOOD, P. R. An analysis of the effect of the forced-choice format of Rotter's internal-external scale. .7. Clin. Psychol., 1978, 34, 381-385.

9. HEAVEN, P. C. L. Authoritarianism or acquiescence? South African findings. J. Soc. Psychol., 1983, 119, 11-15.

10. HEAVEN, P. C. L., & MOERDYK, A. Prejudice revisited: A pilot study using Ray's scale. J. Behav. Sci., 1977, 2, 217-220.

11. HOYT, C. J. Note on a simplified method of computing test reliability. Educ. ~'r Psychol., Meas., 1941, 1, 93-95.

12. KARON, B. P. Reliability: Paradigm or paradox, with especial reference to personality tests. J. Project. Techn. & Personal. Assess., 1969, 30, 223-227.

13. LEE, R. E., III, & WARR, P. B. The development and standardization of a balanced F scale. J. Gen. Psychol., 1969, 81, 109-129.

14. MARTIN, J. Acquiescence: Measurement and theory. Brit. J. Soc. & Clin. Psychol., 1964, 3, 216-225.

15. MUNRO, B., & ADAMS, G. R. Love American style: A test of role structure theory on changes in attitude to love. Hum. Relat., 1978, 31, 215-228.

16. PEABODY, D. Authoritarianism scales and response bias. Psychol. Bull., 1966, 65, 11-23.

17. RAY, J.J. (1972) A new balanced F scale -- And its relation to social class. Australian Psychologist 7, 155-166.

18. RAY, J.J. (1975) Measuring environmentalist attitudes. Australian & New Zealand J. Sociology 11(2), 70-71.

19. RAY, J.J. (1976) Do authoritarians hold authoritarian attitudes? Human Relations, 29, 307-325.

20. RAY, J.J. (1979) The authoritarian as measured by a personality scale Solid citizen or misfit? J. Clinical Psychology 35, 744-746.

21. RAY, J.J. (1979) Is the acquiescent response style not so mythical after all? Some results from a successful balanced F scale. Journal of Personality Assessment 43, 638-643.

22. RAY, J.J. (1979) Is the Dogmatism scale irreversible? South African Journal of Psychology 9, 104-107.

23. Ray, J.J. (1979) A quick measure of achievement motivation -- validated in Australia and reliable in Britain and South Africa. Australian Psychologist 14, 337-344.

24. RAY, J.J. (1979) A short balanced F scale. Journal of Social Psychology, 109, 309-310.

25. RAY, J.J. (1980) Authoritarianism in California 30 years later -- with some cross-cultural comparisons. Journal of Social Psychology, 111, 9-17.

26. RAY, J.J. (1980) The comparative validity of Likert, projective and forced-choice indices of achievement motivation. J. Social Psychology, 111, 63-72.

27. RAY, J.J. (1980) Does living near a coal mine change your attitude to the environment: A case study of the Hunter valley. Australian & New Zealand Journal of Sociology 16(3), 110-111.

28. RAY, J.J. (1980) Acquiescence and the Wilson Conservatism scale. Personality & Individual Differences, 1, 303-305.

29. RAY, J.J. (1981) Achievement motivation and authoritarianism in Manila and some Anglo-Saxon cities. J. Social Psychology 115, 3-8.

30. RAY, J.J. (1981) Authoritarianism, dominance and assertiveness. Journal of Personality Assessment 45, 390-397.

31. RAY, J.J. (1981) Explaining Australian attitudes towards Aborigines Ethnic & Racial Studies 4, 348-352.

32. RAY, J.J. (1982) The Protestant ethic in Australia. Journal of Social Psychology, 116, 127-138.

33. RAY, J.J. (1982) Climate and conservatism in Australia. Journal of Social Psychology, 117, 297-298.

34. RAY, J.J. (1982) A cluster analytic exploration of what underlies popular social science constructs. Journal of Social Psychology, 116, 263-267.

35. RAY, J.J. (1982) Authoritarianism/libertarianism as the second dimension of social attitudes. Journal of Social Psychology, 117, 33-44.

36. RAY, J.J. (1983) Defective validity of the Machiavellianism scale. Journal of Social Psychology 119, 291-292.

37. RAY, J.J. (1983) The workers are not authoritarian: Attitude and personality data from six countries. Sociology & Social Research, 67 (2), 166-189.

38. RAY, J.J. & PRATT, G.J. (1979) Is the influence of acquiescence on "catchphrase" type attitude scale items not so mythical after all? Australian Journal of Psychology 31, 73-78.

39. RORER, L. G. The great response style myth. Psychol. Bull., 1965, 63, 129-156.

40. WILSON, G. D. The Psychology of Conservatism. London: Academic, 1973.

41. WILSON, G. D., & PATTERSON, J. R. A new measure of conservatism. Brit. J. Soc. & Clin. Psychol., 1968, 7, 264-269.




Go to Index page for this site

Go to John Ray's "Tongue Tied" blog (Backup here or here)
Go to John Ray's "Dissecting Leftism" blog (Backup here or here)
Go to John Ray's "Australian Politics" blog (Backup here or here)
Go to John Ray's "Gun Watch" blog (Backup here or here)
Go to John Ray's "Education Watch" blog (Backup here or here)
Go to John Ray's "Socialized Medicine" blog (Backup here or here)
Go to John Ray's "Political Correctness Watch" blog (Backup here or here)
Go to John Ray's "Greenie Watch" blog (Backup here or here)
Go to John Ray's "Food & Health Skeptic" blog (Backup here)
Go to John Ray's "Leftists as Elitists" blog (Not now regularly updated -- Backup here)
Go to John Ray's "Marx & Engels in their own words" blog (Not now regularly updated -- Backup here)
Go to John Ray's "A scripture blog" (Not now regularly updated -- Backup here)
Go to John Ray's recipe blog (Not now regularly updated -- Backup here or here)

Go to John Ray's Main academic menu
Go to Menu of recent writings
Go to John Ray's basic home page
Go to John Ray's pictorial Home Page (Backup here)
Go to Selected pictures from John Ray's blogs (Backup here)
Go to Another picture page (Best with broadband)