*****************************************************************************************

(Article written for the academic journals in 1990 but not accepted for publication)

ACQUIESCENT RESPONSE TENDENCY: AN UPDATE AND SOME DATA ON THE INVALIDITY OF THE DOGMATISM SCALE
J.J. Ray

University of N.S.W., Australia

Abstract

There appears to be a strong consensus among those who research acquiescent response tendency to the effect that one-way-worded scales are undesirable. Many psychologists, however, continue to use such scales. This suggests that the occasional dissenters who have defended such scales are surprisingly influential and should be taken seriously. Some faults in the reasoning of such commentators are therefore outlined. Some further data showing how the presence of acquiescent bias can be detected are presented using a balanced form of the Rokeach Dogmatism scale. It is shown that only the presence of an acquiescent response tendency serves to explain the findings presented. The balanced Dogmatism scale is shown to work best with highly educated respondents. This suggests that the one-way-worded version of the Dogmatism scale also has serious problems.

INTRODUCTION

I am indebted to a massive bibliography supplied to me some time ago by Dr Lewis R. Goldberg of the Oregon Research Institute for the information that the earliest academic paper on acquiescent response tendency goes all the way back to the time of the first world war (Cogan, Conklin & Hollingworth, 1915). I have been unable to check the reference personally as the library facilities available to me are less than encyclopedic but I have no reason to doubt the accuracy of such a distinguished scholar as Dr Goldberg.

As the references that follow will show, acquiescence has continued to be a phenomenon of interest to researchers right up to the present day. All of the references given show concern with acquiescent tendency as a problem in attitude and personality measurement (Altemeyer, 1981; Bachman & O'Malley, 1984; Bass, 1955; Bentler, Jackson & Messick, 1971 & 1972; Berkowitz & Wolkon, 1964; Blau & Katerberg, 1982; Block, 1965; Byrne & Bounds, 1964; Campbell, Siegman & Rees, 1967; Cherry & Byrne, 1977; Cloud & Vaughan, 1970; Cogan, Conklin & Hollingworth, 1915; Couch & Keniston, 1960; Cronbach, 1946; Davison & Srichantra, 1988; Duhan & Keown, 1988; Eisenman & Townsend, 1970; Gage & Chatterjee, 1960; Gibbins, 1968; Goldsmith, 1986 & 1987; Goldsmith, White & Stith, 1987; Heaven, 1983; Hui & Triandis, 1985; Jackson, 1967; Krenz & Sax, 1987; Lambley & Gilbert, 1970; Lee & Warr, 1969; Lentz, 1938; Martin, 1964; Milbrath, 1962; Neel, Tzeng & Baysal, 1983; Oskamp, 1970; Peabody, 1966; Ray, 1970, 72, 74, 79a, b & c, 80a & b, 81, 82a & b, 83, 84a, b & c, 85a & b; Ray & Pratt, 1979; Roberts, Forthofer & Fabrega, 1976; Schmitt & Stults, 1985; Trott & Jackson, 1967; Vagt & Wendt, 1978; Van Heerden & Hoogstraten, 1979; Wilson & Patterson, 1968; Winkler, Kanouse & Ware, 1982). The references listed are only a small sub-set of those that might have been listed with the major omissions being less recent studies.

With such a strong consensus that acquiescence is a problem requiring corrective measures (e.g. use of "balanced" scales containing equal proportions of "True" and "False" items) in attitude and personality measurement, one would think that there was little left to be said on the topic and that all social scientists would now use balanced scales. Surprisingly, however, this is not so. There have been isolated apologists for one-way-worded scales (e.g. Rokeach, 1967; Rorer, 1965; Samelson, 1972) and it appears that such writings have been seized on by many researchers as saving them from the need to use balanced scales. The reasoning seems to be something like: "Some say balanced scales are needed and some say they are not so both options are equally legitimate". While such thinking may be understandable at some level it is remarkably poor science. A variety of authors have shown that acquiescent response tendency can have important correlates of its own (e.g. Gage & Chatterjee, 1960; Milbrath, 1962; Eisenman & Townsend, 1970; Goldsmith, 1987; Goldsmith, White & Stith, 1987; Heaven, 1983; Blau & Katerberg, 1982) so the correlates of any one-way-worded scale will always be susceptible of at least two interpretations. Science, however, is precaution-oriented and failing to take measures that will preclude alternative interpretations of one's findings is quite simply careless and asks faith of the reader. Faith and science are hardly of a piece.

The major point that critics of balanced scales (e.g. Rorer, 1965) seem to seize on is that different measures of acquiescence often show little correlation between themselves (e.g. McGee, 1962). They then seem to reason: "Well if it does not generalize it cannot be a problem". There is some truth in that reasoning, of course. The trouble is that sometimes acquiescence scores do intercorrelate (e.g. Vagt & Wendt, 1978). So the same reasoning consistently applied must say that on such occasions acquiescent tendency could be a problem. But how can we know in advance which circumstance will prevail? How can we know whether we will have a problem or not? I will not be so foolhardy as to say that we will never be able to know but, certainly the seemingly obvious predictors that I have tried did not work (Ray, 1983). That being so, it would seem the path of prudence always to use balanced scales so that if acquiescence problems do arise they can be both detected and controlled for.

Another approach used by critics of balanced scales is to point out that double agreement with oppositely-worded items is not necessarily a sign of acquiescent bias (e.g. Rokeach, 1967). This is, of course, perfectly true but it is, for all that, to fail to see the wood for the trees. Surely such double agreement is a problem whatever its source. It shows that the scale author has got it wrong in one way or another and that items he intends to be of opposite import are not so seen by those he surveys. It is a clear indication that the scale lacks construct validity. Only by using balanced scales, however, can we detect such validity deficiencies.

Other commentaries on the arguments used by the critics of balanced scales have been widespread but perhaps Peabody (1966), Campbell, Siegman & Rees (1967), Jackson (1967), Bentler, Jackson & Messick (1971 & 1972), Bentler (1973) and Ray (1983 & 1985b) might be specifically mentioned.

At any event it seems clear that many psychologists remain unpersuaded of the importance of using only balanced scales so yet more efforts to demonstrate the usefulness and informativeness of such scales seem needed. The tenacity with which some researchers cling to their one-way-worded scales can, in fact, be remarkable. One recent author (Van Ijzendoorn, 1989) used a one-way-worded scale even though he knew of the arguments against such scales and even though he knew of an alternative measure of the same construct in balanced form!

As we have seen, one of the persistent defenders of one-way- worded scales was Rokeach (1967). This may be connected with the fact that his widely-used Dogmatism ('D') scale is one-way-worded. It seems appropriate, therefore, to see how badly (if at all) his Dogmatism scale is acquiescence-affected. This has now been possible for some time since the production of two different balanced revisions of the 'D' scale (Ray, 1970 & 1974). Such an examination will be attempted below.


METHOD

The results to be reported below were in fact obtained in 1972. Some results of the study concerned were reported fairly promptly (Ray, 1974; Ray & Martin, 1974) but a full write up of the findings was not carried to completion and became overlooked under the pressure of other work. As the results do not appear to be in any way time dependant, however, it still seems appropriate to report them here.


The Statistics
For a start, it is accepted that double-agreements with original and reversed items cannot be seen as proof that acquiescent bias is present (Rokeach, 1967). Other methods will be needed if such a demonstration is to be accomplished. For similar reasons, nor is it sufficient to show that agreement is by far the commonest response to the items. The presence of such a phenomenon may simply show that both sides of the argument are persuasively put. The degree of agreement could, in other words, be quite meaningful and not at all vacant.

I have long used two statistics to demonstrate the presence of meaningless acquiescence: coefficient alpha and r(P-N). Acquiescent tendency should inflate alpha and deflate r(P-N). As Davison & Srichantra (1988) have recently reported findings that generally support my approach I will confine myself here to pointing out that the reasoning behind both indices is fairly simple. Anything that causes items to be responded to similarly will cause the items to correlate positively. Those who score highly on one item will also tend to score highly on other items. Acquiescent bias will therefore tend to increase the correlation between one-way-worded items. Coefficient alpha, however, can be represented as average inter-item correlation weighted by test length (Cronbach, 1951; Lord & Novick, 1968) so it should rise as acquiescent bias affects the one-way-worded scale (test length or number of items being constant).

The second index is useful as representing the outcome of two opposing pressures. The statistic r(P-N) represents the correlation between the two subscales made up respectively of the positively-worded and negatively-worded items. The opposition in meaning between these two groups of items should cause the items to be responded to oppositely and thus bring about an r(P-N) that is high and negative.

Insofar as meaning-independent acquiescence is present, however, it will cause all items to be responded to similarly and this could lead to an r(P-N) that is high and positive. If both things are true (i.e. the items are of opposed meaning and meaningless acquiescence is present) the two tendencies should cancel one-another out and leave an r(P-N) that approximates zero. The latter circumstance quite commonly prevailed with early attempts to balance the F scale (Christie, Havel & Seidenberg, 1956).


Subjects and materials
The Ss for the study were students in the School of Behavioural Sciences at Macquarie University in Sydney, Australia who completed a questionnaire containing the Ray (1970) Balanced Dogmatism (BD) scale in class time. There were 74 First-year students, 52 second-year students and 51 third-year students, to a total of 177.


RESULTS

The results of interest here are given in Table 1.


TABLE 1

Statistics from the Ray (1970) BD scale when applied to three groups of students.

Statistic....... Year 1 ...... Year 2 ..... Year 3

Alpha............ 00.83 ........ 00.80 ....... 00.72
r(P-N)............ -0.34 ......... -0.37 ....... -0.51
BD mean ...... 86.95 ........ 86.08 ....... 79.55
BD S.D.......... 14.42 ........ 13.02 ....... 10.71
n................... 74.00 ........ 52.00 ....... 51.00



It will be noted that there was a trend (non-significant) for the coefficient alpha to decline as exposure to the University increased. At the same time r(P-N) tended to rise. Thus the more sophisticated subjects (third-year students) showed less organization of measured attitudes (as indexed by alpha) even though the intended opposition between the positively and negatively worded items was most evident in their responses.


DISCUSSION

If the students with greater exposure to the University had in fact (as we might have expected) had attitudes which were more organized, thought-out and consistent one would surely expect that both indices (alpha and r(P-N)) would have risen (both being measures of internal consistency). Ray (1970) certainly found students to have more organized attitudes than the general public and Sniderman, Brody & Kuklinski (1984) found that education generally increased attitude organization. That did not seem to happen on the present occasion, however. Why? Acquiescent bias provides the answer.

It must be reiterated that alpha is expressible as the weighting of mean inter-item correlation against the number of items. Since the number of items (36) is constant for all groups in the present study, it follows that variations in alpha are wholly traceable to variations in mean inter-item correlations. The implication of reduced overall correlations combined with increased (or even stable) pos-neg correlations can only be therefore that the intercorrelations between the positive items only and the negative items only (those being the only other correlations) must have dropped. And this is precisely what reduced acquiescent set would have led us to expect! Why? Because the positive items alone or the negative items alone form one-way-worded scales. As mentioned above, the effect of acquiescent bias on such scales is to inflate the intercorrelation between their items. If such bias is reduced, however, the intercorrelations between such items will drop (i.e. the correlation due to common direction of wording will be removed) and the contribution of such correlations to the average intercorrelation also therefore will drop -- leading in turn to the effect observed: An alpha that is slow to rise and which may even fall.

Thus reduced acquiescent set due to the mentally organizing effect of increased education does provide a complete explanation for the effects observed on the present occasion where the effects of education alone would not do so.

Clearly, then, the Dogmatism items are acquiescence affected and need to be used in conjunction with reversed items in order to control for any effects this may have. The fact that increased higher education causes the Dogmatism items to be responded to more and more as they should be, does, of course, have a corollary: The Dogmatism items are less and less valid the less educated are the respondents to whom they are applied. If increased education reduces acquiescent bias, lesser education should lead to more of it. It would appear likely, then, that the Dogmatism scale in a balanced form is suitable for use only with students. This is also what emerged from Ray's (1979c) study of balanced Dogmatism scales applied to a general population sample so it is revealed that even the balanced Dogmatism scale is a very limited measure. How much less valid must be the one-way-worded form of the scale.


REFERENCES


{Articles below by J.J. Ray can generally be accessed simply by clicking on the name of the article. I am however also gradually putting online a lot of abstracts, extracts and summaries from older articles by other authors so if an article not highlighted below seems of particular interest, clicking here or here might just save you a trip to the library}

Altemeyer, R. (1981). Right-wing authoritarianism Winnipeg: University of Manitoba Press.

Bachman, J.G. & O'Malley, P.M. (1984) Yea-saying, Nay-saying and going to extremes: Black-white differences in response-styles Public Opinion Quarterly 48, 491-509

Bass, B.M. (1955) Authoritarianism or acquiescence? J.Abnorm. Soc. Psychol. 51, 616-623.

Bentler, P.M. (1973) An analysis of responses to adjectives: A reply to Samelson Psychological Bulletin 80, 133-134.

Bentler, P.M., Jackson, D.N. & Messick, S. (1971) Identification of content and style: A two-dimensional interpretation of acquiescence. Psychological Bulletin 76, 186-204.

Bentler, P.M., Jackson, D.N. & Messick, S. (1972) A rose by any other name. Psychological Bulletin 77, 109-113.

Berkowitz, N.H. & Wolkon, G.H. (1964) A forced-choice form of the 'F' scale -- free of acquiescent response set.Sociometry 27, 54-65.

Blau, G. & Katerberg, R. (1982) Agreeing response set: Statistical nuisance or meaningful personality concept? Perceptual & Motor Skills 54, 851-857.

Block, J. (1965) The challenge of response sets N.Y.: Appleton Century.
Byrne, D. & Bounds, C. (1964) The reversal of F scale items. Psychological Reports 14, 216.

Campbell, D.T., Siegman, C.R. & Rees, M.B. (1967) Direction of wording effects in the relationship between scales. Psychological Bulletin 68, 293-303.

Cherry, F. & Byrne, D. (1977) Authoritarianism. In T. Blass (Ed.) Personality variables in social behavior Hillsdale, N.J.: Erlbaum

Christie, R., Havel, J. & Seidenberg, B.(1956) Is the 'F' scale irreversible? J. Abnorm. Soc. Psychol. 56, 141-158.

Cloud, J. & Vaughan, G.M. (1970) Using balanced scales to control acquiescence. Sociometry 33, 193-202.

Cogan, L.C., Conklin, A.M. & Hollingworth, H.L. (1915) An experimental study of self-analysis, estimates of associates, and the results of tests School & Society 2, 171-179.

Couch, A. & Keniston, K.(1960) Yeasayers and naysayers: Agreeing response set as a personality variable. J. Abnorm. Soc. Psychol. 60, 151-174.

Cronbach, L.J. (1946) Studies of acquiescence as a factor in the true-false test J. Educational Psychology 33, 401-415.

Cronbach, L.J. (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16, 297-334.

Davison, M.L. & Srichantra, M. (1988) Acquiescence in components analysis and multidimensional scaling of self-rating items. Applied Psychological Measurement 12, 339-351.

Duhan, D.F. & Keown, C.F. (1988) Effect of biasing an attitude scale: Acquiescence, reactance or balancing? Psychological Reports 62, 567-574.

Eisenman, R. & Townsend, T.D. (1970) Studies in acquiescence: I. Social desirability; II. Self-esteem; III. Creativity; and IV. Prejudice J. Projective Techniques & Personality Assessment 34, 45-54

Gage, N.L. & Chatterjee, B.B. (1960) The psychological meaning of acquiescence set: Further evidence. J. Abnormal & Social Psychology 60, 280-283

Gibbins, K. (1968) Response sets and the semantic differential British J. Social & Clinical Psychology 7, 253-263

Goldsmith, R.E. (1986) Personality and uninformed response error J. Social Psychology 126, 37-45

Goldsmith, R.E. (1987) Two studies of yeasaying Psychological Reports 60, 239-244

Goldsmith, R.E. & Nugent, N. (1984) Innovativeness and cognitive complexity: A second look. Psychological Reports 55, 431-438.

Heaven, P.C.L. (1983) Authoritarianism or acquiescence? South African findings. J. Social Psychol. 119, 11-15.

Hui, C.H. & Triandis, H.C. (1985) The instability of response sets Public Opinion Quarterly 49, 253-260

Jackson, D.N. (1967) Acquiescence response styles: Problems of identification and control. In I.A. Berg (Ed.) Response set in personality measurement Chicago: Aldine.

Krenz, C. & Sax, G. (1987) Acquiescence as a function of test type and subject uncertainty Educational & Psychological Measurement 47, 575ff.

Lambley, P. & Gilbert, L.H. (1970) Forced choice and counterbalanced versions of the 'F' scale: Prediction of prejudiced attitudes. Psychological Reports 27, 547-550.

Lee, R.E. & Warr, P.B. (1969) The development and standardization of a balanced 'F' scale. J. General Psychol. 81, 109-129.

Lentz, T.F. (1938) Acquiescence as a factor in the measurement of personality Psychological Bulletin 35, 659.

Lord, F.M. & Novick, M.R.(1968) Statistical theories of mental test scores Reading, Mass.: Addison Wesley.

Martin, J. (1964) Acquiescence -- measurement and theory. British J. Social & Clin. Psychol. 3, 216-225.

McGee, R.K. (1962) The relationship between response style and personality variables: I. The measurement of response acquiescence J. Abnormal & Social Psychology 64, 229-233.

Milbrath, L. (1962) Latent origins of Liberalism-Conservatism and party identification: A research note. J. Politics 24, 679-688.

Neel, R.G., Tzeng, O.C.S. & Baysal, C. (1983) Comparative studies of authoritarian-personality characteristics across culture, language and methods Internat. J. Intercult. Rel. 7, 393-400.

Oskamp, S.S. (1970) Internal inconsistency in the stereopathy- acquiescence scales. J.Social Psychol. 81, 73-77.

Peabody, D. (1966) Authoritarianism scales and response bias. Psychological Bulletin 65, 11-23.

Ray, J.J. (1970) The development and validation of a balanced Dogmatism scale. Australian Journal of Psychology, 22, 253-260.

Ray, J.J. (1972) A new balanced F scale -- And its relation to social class. Australian Psychologist 7, 155-166.

Ray, J.J. (1974) Balanced Dogmatism scales. Australian Journal of Psychology 26, 9-14.

Ray, J.J. (1979a) A short balanced F scale. Journal of Social Psychology, 109, 309-310.

Ray, J.J. (1979b) Is the acquiescent response style not so mythical after all? Some results from a successful balanced F scale. Journal of Personality Assessment 43, 638-643.

Ray, J.J. (1979c) Is the Dogmatism scale irreversible? South African Journal of Psychology 9, 104-107.

Ray, J.J. (1980a) Acquiescence and the Wilson Conservatism scale. Personality & Individual Differences, 1, 303-305.

Ray, J.J. (1980b) Acquiescence and coefficient Alpha: A reply to Porritt. Australian Journal of Psychology 32, 144-150.

Ray, J.J. (1981) Sample homogeneity, response skewness and acquiescence: A reply to Feather. Australian Journal of Psychology 33, 41-46.

Ray, J.J. (1982a) Machiavellianism, forced-choice scales and the validity of the F scale: A rejoinder to Bloom. J. Clinical Psychology 38, 779-782.

Ray, J.J. (1982b) The construct validity of balanced Likert scales. Journal of Social Psychology 118, 141-142.

Ray, J.J. (1983) Reviving the problem of acquiescent response bias. Journal of Social Psychology 121, 81-96.

Ray, J.J. (1984a) Alienation, dogmatism and acquiescence. J. Clinical Psychology 40, 1007-1008.

Ray, J.J. (1984b) Reinventing the wheel: Winkler, Kanouse Ware on acquiescent response set. J. Applied Psychology 69, 353-355.

Ray, J.J. (1984c) A further comment on the Winkler, Kanouse Ware method of controlling for acquiescent response bias. J. Applied Psychology 69, 359.

Ray, J.J. (1985a) Acquiescence and response skewness in scale construction: A paradox. Personality & Individual Differences 6, 655-656.

Ray, J.J. (1985b) Acquiescent response bias as a recurrent psychometric disease: Conservatism in Japan, the U.S.A. and New Zealand. Psychologische Beitraege 27, 113-119.

Ray, J.J. & Martin, J. (1974) How desirable is dogmatism? Australian & New Zealand Journal of Sociology 10(2), 143-145.

Ray, J.J. & Pratt, G.J. (1979) Is the influence of acquiescence on "catchphrase" type attitude scale items not so mythical after all? Australian Journal of Psychology 31, 73-78.

Roberts, R.E., Forthofer, R.N. & Fabrega, H. (1976) The Langner items and acquiescence. Social Science & Medicine 10(2), 69-75.

Rokeach, M. (1960) The open and closed mind N.Y.: Basic Books.

Rokeach, M. (1967) Authoritarian scales and response bias: Comment on Peabody's paper. Psychological Bulletin 67, 349-355.

Rorer, L.G. (1965) The great response-style myth. Psychological Bulletin 63, 129-156.

Samelson, F. (1972) Response style: A psychologists fallacy Psychological Bulletin 78, 13-16

Sniderman, P.M., Brody, R.A. & Kuklinski, J.H. (1984) Policy reasoning and political values: The problem of racial equality. Amer. J. Polit. Science 28, 75-94.

Trott, D.M. & Jackson, D.N. (1967) An experimental analysis of acquiescence. J. Experimental Research in Personality 2, 278-288

Vagt, G. & Wendt, W. (1978) Akquieszenz und die Validtaet von Fragebogenskalen. Psychologische Beitraege 20, 428-439.

Van Heerden, J. & Hoogstraten, J. (1979) Response tendency in a questionnaire without questions Applied Psychological Measurement 3, 117-121

Van Ijzendoorn, M.H. (1989) Moral judgment, authoritarianism and ethnocentrism. J. Social Psychology 129, 37-45.

Wilson, G.D. & Patterson, J.R. (1968) a new measure of conservatism. British J. Social & Clin. Psychology 7, 264-269.

Winkler, J.D., Kanouse, D.E. & Ware, J.E. (1982) Controlling for acquiescent response set in scale development. J. Applied Psychology 67, 555-561.




Go to Index page for this site

Go to John Ray's "Tongue Tied" blog (Backup here or here)
Go to John Ray's "Dissecting Leftism" blog (Backup here or here)
Go to John Ray's "Australian Politics" blog (Backup here or here)
Go to John Ray's "Gun Watch" blog (Backup here or here)
Go to John Ray's "Education Watch" blog (Backup here or here)
Go to John Ray's "Socialized Medicine" blog (Backup here or here)
Go to John Ray's "Political Correctness Watch" blog (Backup here or here)
Go to John Ray's "Greenie Watch" blog (Backup here or here)
Go to John Ray's "Food & Health Skeptic" blog (Backup here or here)
Go to John Ray's "Eye on Britain" blog (Backup here or here)
Go to John Ray's "Immigration Watch" blog. (Backup here or here).
Go to John Ray's "Leftists as Elitists" blog (Not now regularly updated -- Backup here or here)
Go to John Ray's "Marx & Engels in their own words" blog (Not now regularly updated -- Backup here or here)
Go to John Ray's "A scripture blog" (Not now regularly updated -- Backup here or here)
Go to John Ray's recipe blog (Not now regularly updated -- Backup here or here)
Go to John Ray's "Some memoirs" (Occasionally updated -- Backup here)

Go to John Ray's Main academic menu
Go to Menu of recent writings
Go to John Ray's basic home page
Go to John Ray's pictorial Home Page (Backup here or here).
Go to Selected pictures from John Ray's blogs (Backup here or here)
Go to Another picture page (Best with broadband)



Note: If the link to one of my articles is not working, the article concerned can generally be viewed by prefixing to the filename the following:
http://pandora.nla.gov.au/pan/42197/20060526/jonjayray.tripod.com/