Subject:
|
Re: Personality test vs. Religion
|
Newsgroups:
|
lugnet.off-topic.debate
|
Date:
|
Tue, 2 Nov 2004 16:27:43 GMT
|
Viewed:
|
2494 times
|
| |
|
|
In lugnet.off-topic.debate, David Eaton wrote:
> Regardless that the method isn't engineered to be scientific, is there merit in
> its predictions? Is there perhaps some scientific componant that's not
> understood underlying the leaves? Or perhaps insofar as the people
> subconsciously responding to signals and PICKING who they were meant for? But
> despite those possibilities, is there actual MERIT it the prediction, given the
> odds?
I'm sorry, but this hypothetical example doesn't interest me. Suppose I posit a
car that delivers infinite gas mileage--wouldn't you buy it? Heck, yes! But
what's the point? It's not difficult to create examples that have no relation
to reality, and in this context they don't strike me as helpful.
But I sense that your underlying question is this: If a non-scientific system
can be shown to have a great deal of predictive accuracy, would it not be valid
to make use of that system? I guess it would, but so what? Show me such a
system in the real world, and we'll discuss it.
> > My argument is that the test is inadequate as the tool it is marketed and
> > sold to be; namely, it is marketed and sold as a tool of professional and/or
> > career planning, and it is not suited to that role.
>
> Well, there's no argument from me on that one-- nor has there been. My opinion
> is that its suited to tracking general personality trends, and I'm curious as to
> whether there's any correlation to religious beliefs or not.
>
> Your argument there is (apparently) that it's NOT good for tracking trends, and
> NOT good for correllating perhaps how people think to what people believe
> religiously.
To be fair to both of us, the intent of my argument has been to demonstrate that
the test is of no value in making claims re: professional potential. If we both
accept that point, then I'll stop belaboring it.
However, I haven't really focused on whether or not it's good at spotting
trends. I would say that the test, when used as it is designed to be used, has
no greater ability to identify trends than normal interpersonal interaction.
That the test applies jargon-eque acronyms makes no difference; it's the same as
making the casual observation that "Bob's a jerk" or "Bob's a great guy."
Actually, casual observation is in fact more useful than the test. The test
will only return results like "Bob's a great guy," "Bob's a swell guy," "Bob's a
nice guy," "Bob's a fine guy," etc. Casual-but-honest observation, though
hardly scientific, is probably more usefully applicable to daily interaction
than a test that only returns generally positive answers.
> > Well, I hadn't quite thought of it like that, but that's a point worth
> > mentioning. I don't have a problem with a test that *could* return 100%
> > positive results. I have a problem with a test that is designed so that it
> > is not possible to give a negative result.
>
> Again, I keep bringing this up, it IS possible (I think) to get a negative
> result. If it scored me as an ENFJ instead of an ISTP, that (I would hold) would
> be a negative result for me.
That's true only if you change the nature of the test, and then we're not
arguing about MBTI anymore, are we? Built into the test is the ability to
massage the results to get them to match reality, and this is the fatal flaw I
mentioned previously. If you take away this post hoc tweaking, then you haven't
completed the test as it is designed.
Suppose that scientific precedent demonstrates cleary that an antibiotic will
cure Infection X if you take it for a week, but you decide to take it for only
five days, and Infection X persists at the end of that time. Can you reasonably
claim that the antibiotic didn't work? No, because you didn't employ it as it
is designed to be employed. You *can* claim that the antibiotic doesn't work if
only taken for five days, but so what? You've changed the conditions of the
test (Kobayashi Maru, anyone?) so you didn't actually take the test and can't
make claims about it.
If you strip away one of the central elements of the MBTI, then you're no longer
making claims about the MBTI. Instead, you're talking about "Dave Eaton's
Modified MBTI," and that's not the test I've been discussing.
> > I think that's how it was meant--there should be a bell-curve distribution of
> > each trait, rather than of each four-character personality descriptor.
>
> If that's how he's meaning it, he'll have a hard time doing it without
> incremental levels per dimension. I mean, if the bell curve is smack dab in the
> middle for one trait, you'll get a 50/50 distribution in your two categories,
I think that we're thinking of this differently. The way I'm breaking it down,
the critic is expecting a bell-curve representation for each of the 16 (or
hypothetical 625 or 1,089 or whatever) traits. Is that how you're reading it?
I have the sense that you're basing your (pixellated!) curve on a strict binary
reading; EITHER Intravert OR Extravert. I don't think that you're wrong to do
that, but I don't think it's what the critic's describing, either.
> I'm interested in its
> use as containing at least something resembling empirical evidence (arguably
> some of the closest you can get in the world of psychology), albeit not 100%
> accurate evidence, but more like "pretty accurate" evidence.
Now we're getting somewhere, but I think that you should describe it as
"testimonial evidence" instead of "empirical evidence." At best, the test
provides *inferential* evidence of correlation. The leap from a subject's
response of "open" or "close" to deciding that the subject is therefore
"Intravert" or "Extravert" cannot be corroborated except by testimony and/or
inference, and this is dangerously close to wishful thinking.
> > > No? I knew about introversion/extroversion before the test and rated myself
> > > an introvert.
> >
> > Not the terms Extravert and Intravert. I'm talking about the arbitrary
> > combinations of "ISTJ," "ESFP," etc., which were certainly created by the
> > designers.
>
> Huh? How come the combinations are somehow different than the sum of their
> constituent parts? Are you implying that the Extrovert/Introvert subcomponent of
> M/B is accurate, but not when put into context with other classifications?
Oops--forgive the sloppy wording. The terms "ISTJ" and "ESFP" are presented as
labels unto themselves, and on that basis they must be assessed whole, or else
you're not evaluating the MBTI. People in this forum have lightheartedly
identified themselves by one MBTI signifier or another, so the labels do indeed
have a life of their own.
Beyond that, I am not convinced that the traits as described by MBTI have any
meaningful, independent correlation to the personality trends you describe. You
say that you, your mother, and the MBTI all identify you as an
introvert/Intravert. On what basis? Did your mother at some point sit you down
and quiz you about a list of artificial binary opposites? I suspect
not--instead, she assessed your personality based on countless factors, many of
them non-verbal. The fact that the test, in the space of a few simplistic
questions, seemingly correlates you to its definition of "Intravert" is hardly
conclusive.
> > You're succumbing to the Forer effect, as
> > previously described. And if it didn't match your assessment, you have
> > explicit instructions from the designers to massage the results until they
> > *do* match your assessment.
>
> Well, yeah, sorta. In the end, it acknowledges its own shortcomings as a test,
> which we've both acknowledged.
To be honest, though, I've mentioned a shortcoming that you seem not to accept;
the test is non-falsifiable because MBTI *as designed* can't provide a wrong
answer. You and the designers of the test seem to discard this objection, but
it's central to my rejection of the test overall.
> And as such, you're right, it only invalidates
> the test portion moreso. But it does nothing to the credibility of the
> dimensions themselves.
But, again, the dimensions can only be assessed by inference rather than
correlated empirically to the test or the test subject, and to that end the MBTI
is of no real value even in simply identifying those dimensions. You can easily
apply those dimensions (or approximations of them) to people by yourself without
need of a test to guide you.
I know people who read their horoscopes daily and claim that they're accurate
because the describe "Mary the Libra as forceful" or "Sam the Pisces as
preoccupied." Maybe "forcefulness" or "preoccupation" is a useful dimension,
but their invocation by the horoscope is meaningless.
> > > Let's take another example. Stubbornness again. Is this a personality trait
> > > that exists? Are there such things as stubborn people? Suppose I invented a
> > > test that yielded 80% "accurate" results as compared to a team of 100
> > > psychologists, friends, family, and co-workers who rated each subject
> > > independantly? Are the psychologists, friends, family, and co-workers unable
> > > to make the distinction of stubbornness? Is the test?
> >
> > Sure it is. But is it a good one? I don't think so. You need to spell out
> > explicitly the criteria by which your friends, family, coworkers, and
> > psychiatrists evaluate the subject as "stubborn." This may seem like an
> > unreasonable burden to place upon the reviewers, but if their assessments are
> > being offered as confirmation of the test, then specificity is required.
>
> Wow, breakthrough! What criteria would you accept? "Remember the time that XXXXX
> and XXXXX?" "Led subject to believe one thing, then presented evidence to the
> contrary, tested length of time and willingness for subject to change his mind",
> etc. What level of specificity would you allow as "empirical"? How can you, as
> an evaluator, give any sort of non-subjective input as to someone else's
> stubbornness? Isn't "stubbornness" an evaluative word humans created in an
> attempt to describe a personality trait? (actually, I think it simply got
> applied to personality after its invention, but regardless) At what precise
> point does someone go from being "leniant" to "stubborn"? Where's the midpoint?
> Where are the endpoints?
I'm not comfortable with this example, because you're asking me to shoehorn it
into an empirical model when I didn't design the example in the first place.
Certainly there's no distinct spectrum inherent in linguistic matters, because
language is fundamentally interpretive. Lenny made the good point that
psychology isn't a pure science, and that's worth repeating here.
A committee of psychologists could get together and say "A person who
demonstrates traits A, B, C, D, E, F, and G can be described as "stubborn."
Thereafter they can assess a subject and determine whether or not the subject
is, according to the scale, "stubborn."
Where does that leave us? Well, the MBTI folks decided that people who give a
particular range of answers can be described as an Intravert. This label may or
may not intersect with the mainstream definition "introvert." If it does
intersect, then proponents of the test will claim that it's accurate. But if it
does not intersect, then proponents of the test will claim that results can
vary. This is, of course, a demonstration of non-falsifiability, which
demonstrates that the test doesn't provide useful answers except by inference.
> What would have to be done to convince you that I'm stubborn when it comes to,
> say, my beliefs in relative morality?
Why go there? You're already demonstrated to me that you're stubborn re: the
MBTI! 8^)
I don't know that I can give a clear articulation of the way I determine that
someone is stubborn, or generous, or gloomy, or charismatic, etc. These are
subjective terms that do not lend themselves to concise definition. In fact,
that's part of the reason I reject the MBTI--it seeks to give soundbyte labels
to complex systems of personality.
> Is any of that empirical?
No.
> Or is it "just good enough for you to believe it, if only slightly more than
> 50%"?
Well, let me ask it this way. If a friend of yours took the MBTI and his
answers revealed him to be a total jerk who really doesn't like you, would you
stop hanging out with him? I'd hope not, because then you'd be subordinating
your own interpersonal assessment to an arbitrary and (frankly) poorly-founded
instrument. The point here is that the test is seriously impaired by its
shortcomings, and I see these shortcomings as fatal, while you do not.
> > > Yep. Like I've said, the test, especially because it's yes/no instead of
> > > gradular, will probably make mistakes, and is hence unreliable in the
> > > individual case. But in the general case, I'd still guess it's mostly
> > > accurate.
> >
> > Your conclusion really baffles me. I'm saying that the test cannot possibly
> > yield an inaccurate result, and you're saying "therefore it's mostly
> > accurate." Maybe it is, but the accuracy has no value.
>
> I disagree that the test cannot possibly give an inaccurate result.
The MBTI *as designed and described by its owners* cannot yield an inaccurate
result. Your casual use of the test may or may not yield inaccurate results,
but that's separate from what I've discussed all along. We're back to "Dave
Eaton's Modified MBTI."
> > > If you had a lonely, pessimistic friend (alright, I admit those are
> > > unempirical attributes) who got evaluated independantly by 4 psychiatrists,
> > > who all diagnosed him with depression, would you believe it to be true?
> >
> > It depends on the method they use, of course. If they lick his elbow and say
> > "he's depressed," then I probably won't be too impressed with their
> > diagnosis. If they consult a star chart and say "he's depressed," then I
> > likewise won't have much confidence. But if they undertake a serious system
> > of inquiry based upon proven and reproducible criteria, then I'll be much
> > more likely to put stock in their assessment.
>
> Ahh. Define "proven and reproducable criteria" as they relate to depression.
I can't give you an exhaustive list of criteria, of course, for the reasons I
mentioned above. However, here are a few guidelines:
One must establish a definition for "depression."
One must enumerate criteria that qualify a subject as "depressed"
One must articulate methods by which these criteria can be evaluated
One must demonstrate that the criteria qualifying a subject as "depressed"
correlate overwhelmingly to the condition of "depression" in preference to
other conditions
Ideally, others who share your definition of "depression" should likewise
conclude that a given subject qualifies as "depressed," and ideally the
methods for making that determination should differ from one's own
Where does this leave us? At best, we might get a bunch of people who agree
that Subject X meets the criteria for their shared definition of "depression."
But you're risking a mistake to ask this list of me. You're getting very close,
intentionally or not, to claiming that, if I can't propose a better system, then
I have no basis for objecting to this existing system, and that's a fallacy. I
believe that this isn't your intent, but it's only a little further down the
road in the direction you're currently heading.
> > The M/B test, in contrast, uses neither proven nor reproducible criteria.
>
> Proven, I'll agree. It's not "proven". Reproducible? I'm not sure I'd agree. I
> am admittedly curious to try and locate that report online that gave a 50%
> repeatability bit, since my suspicion is that most of the remaining 50% were off
> by a letter or two.
Being "off by a letter or two" is the same as being wrong, especially when
you're talking about a field of only four letters! To suggest that 25% or 50%
inaccuracy is acceptable is to succumb to the practice of "shotgunning:"
http://skepdic.com/shotgun.html
Your choice to accept this one- or two-letter margin of error is equivalent to
changing a letter post hoc--you thereby render the test non-falsifiable. At
that point, it becomes useless as a predictive instrument, even for casual use.
Dave!
|
|
Message has 3 Replies:
 | | Re: Personality test vs. Religion
|
| (...) Well, I think it's results are more useful that a lame "Bob's a great guy." In some ways, I wonder if the greatest value isn't in the actual results, but in understanding what the different categories are supposed to be and that people really (...) (20 years ago, 2-Nov-04, to lugnet.off-topic.debate)
| | | Re: Personality test vs. Religion
|
| (...) Awesome. The only question that remains is "how much of a 'great deal' of accuracy is needed?", and to show whether or not M/B does or doesn't follow. (...) I think we've agreed on that from the start-- I would argue that while SOME job (...) (20 years ago, 2-Nov-04, to lugnet.off-topic.debate)
|
Message is in Reply To:
| | Re: Personality test vs. Religion
|
| (...) Hm. Here's a question. Let's say that some old kook of a witch doctor uses tea leaves to predict the names of who his clients will marry (or perhaps clients ask who "friend X" will marry). The leaves predict 49,928/50,000 people's marriage (...) (20 years ago, 1-Nov-04, to lugnet.off-topic.debate)
|
53 Messages in This Thread:
 
 
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
This Message and its Replies on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
