Subject:
|
Re: Personality test vs. Religion
|
Newsgroups:
|
lugnet.off-topic.debate
|
Date:
|
Mon, 1 Nov 2004 20:38:15 GMT
|
Viewed:
|
2373 times
|
| |
|
|
In lugnet.off-topic.debate, David Eaton wrote:
> In lugnet.off-topic.debate, Dave Schuler wrote:
> > But your wife is correct--the tool has no predictive power because its
> > predictions are so deliberately general and open-ended as to be useless.
>
> "Deliberately" general? Got any empirical proof? :)
Well, it was designed, was it not? And presumably the designers made deliberate
choices to include some results and not others, right? QED.
> I wouldn't say it's useless at all, except insofar as it IS error prone. If its
> category divisions are indeed correct (I'd say they seem to be), they may indeed
> help us understand how people think; and further might be able to allow us to
> cater to individual tastes more.
Sure, they *may* do that, just as tea leaves *may* tell you who you're going to
marry.
Let me underscore at this point that I'm not debating people's choice to use
this test for self-evaluation or for harmless assessments of one's friends and
companions. I have no interest in that debate, other than to say that I think
it's a waste of time, but that's your business.
My argument is that the test is inadequate as the tool it is marketed and sold
to be; namely, it is marketed and sold as a tool of professional and/or career
planning, and it is not suited to that role.
> And of course, as I've been saying, and (as it sounded to me) Lenny's wife was
> saying, the ability for the test to give a subject a 100% perfect type analysis
> 100% of the time is of course, not expected. But I'm not sure you're arguing
> against this point, since you're objecting to the ability to HAVE a 100% perfect
> analysis of ANYONE based on the 4 dimensions provided.
Well, I hadn't quite thought of it like that, but that's a point worth
mentioning. I don't have a problem with a test that *could* return 100%
positive results. I have a problem with a test that is designed so that it is
not possible to give a negative result.
> > Quoting the boundlessly entertaining Dave!
> "any reasonable psychiatrist (which, parenthetically, neither Myers nor Briggs
> was) would recognize that the testing parameters of Myers-Briggs are inherently
> non-scientific and non-falsifiable, and the psychiatrist would reject the test
> as folly."
>
> So... the majority of psychiatrists/psychologists are unreasonable? :)
Smiley aside, I would say yes, as pertains to this issue. The inability or
unwillingness to recognize the flaws in a pseudoscientific "test" strikes me as
unreasonable.
> Not sure if I'd intuitively expect [a bell curve] or not... People of ONE
> particular stat may also follow a 2nd stat (or, I would intuitively expect so),
> and people in the same family and culture may show tendancies to some particular
> types (or I would intuitively expect), but I'm not sure I'd expect a normalized
> distribution over one of the 16 types in the general population. Maybe over some
> of the dimensions, perhaps.
I think that's how it was meant--there should be a bell-curve distribution of
each trait, rather than of each four-character personality descriptor.
> "In other words, once an INTJ, always an INTJ."
>
> Disagree. As a child I was HIGHLY introverted. I've become much less so over the
> years. The brain is a developmental thing. A psychotic killer will always be a
> psychotic killer? A manic depressive will always be a manic depressive?
>
> "Several studies, however, show that even when the test-retest interval is short
> (e.g., 5 weeks), as many as 50 percent of the people will be classified into a
> different type."
Then that's a major failing of the test, isn't it? Remember, I'm not objecting
to your casual use of the test; I'm objecting to its use as an indicator of
professional potential. With this in mind, a reasonable critic must ask how the
test can claim to be useful for longitudinal predictions if its consistency
can't be relied upon even over the span of a few weeks.
> Now, this was the only interesting bit to me in disproving the theory; however
> as it's worded, it worries me. "Different type" might mean a 1 letter
> difference, which they were close on before anyway. Again, with the 625 result
> test, you could judge more easily, but I'd be curious to see HOW different those
> types were from their initial testing. In fact, if 50% were the 100% the same,
> that's probably actually a good indicator (assuming I'm correct that only a
> letter, MAYBE two might have changed for most other people) that the test IS
> pretty repeatable.
Consistently producing the same result is only a demonstration of consistency,
rather than of accuracy. Additionally, if you're equating this form of
"repeatability" with the principle of "reproducibility," then I'd caution that
merely performing the same test on the same person and getting the same result
is not really reproducing the result. You need independent confirmation before
you can claim that the M/B's results have been meaningfully reproduced.
> > The makers of that test created the personality "types" while creating the
> > instrument that purports to measure them. Because the personality types are
> > essentially proprietary to the M/B test, no other test can reasonably be
> > expected to return corroborating results.
>
> No? I knew about introversion/extroversion before the test and rated myself an
> introvert.
Not the terms Extravert and Intravert. I'm talking about the arbitrary
combinations of "ISTJ," "ESFP," etc., which were certainly created by the
designers.
> Test matched my assessment.
Did it? Did it match you specifically, in a way that others could not claim to
"match" with equal certainty? You're succumbing to the Forer effect, as
previously described. And if it didn't match your assessment, you have explicit
instructions from the designers to massage the results until they *do* match
your assessment.
> Granted I didn't create a test for others to take, but suppose I did? Do you
> expect it would produce highly different results?
I would expect it to produce results that can be duplicated by an independent
assessor who has no knowledge of the original results. Otherwise, it's not a
reproducible evaluation.
> Let's take another example. Stubbornness again. Is this a personality trait that
> exists? Are there such things as stubborn people? Suppose I invented a test that
> yielded 80% "accurate" results as compared to a team of 100 psychologists,
> friends, family, and co-workers who rated each subject independantly? Are the
> psychologists, friends, family, and co-workers unable to make the distinction of
> stubbornness? Is the test?
Sure it is. But is it a good one? I don't think so. You need to spell out
explicitly the criteria by which your friends, family, coworkers, and
psychiatrists evaluate the subject as "stubborn." This may seem like an
unreasonable burden to place upon the reviewers, but if their assessments are
being offered as confirmation of the test, then specificity is required.
But here's an answer based on a few assumptions:
Assuming that there is a specifically identifiable trait called "stubbornness;"
Assuming that the trait can be authoritatively assessed by your friends et al.;
and
Assuming that the proposed test can be reliably correlated to your friends'
assessment;
THEN
I would say that the test can be considered a reliable indicator.
However, you can't really demonstrate more than two of the above assumptions, so
this hypothetical construct is not relevant to the Myers-Briggs test.
> > In essence, they're saying "the test gives correct results, but if it doesn't
> > give correct results, then tweak it until it gives correct results."
>
> Yep. Like I've said, the test, especially because it's yes/no instead of
> gradular, will probably make mistakes, and is hence unreliable in the individual
> case. But in the general case, I'd still guess it's mostly accurate.
Your conclusion really baffles me. I'm saying that the test cannot possibly
yield an inaccurate result, and you're saying "therefore it's mostly accurate."
Maybe it is, but the accuracy has no value. Must I roll out my World Series
metaphor again? Or how about the one about the pinball machine?
> I might modify that to say that you'll accept testimony of experts if
> (1) the source appears to be a respected source and (2) the theory presented
> appears to be something empirically verifiable. Hence, if some new star were
> discovered by the Hubble, you'd probably believe it, because the source is
> trusted, and the method is based on empirical evidence. However, if NASA
> announced after "a study" that in general, women were more in touch with their
> emotions than men, you would utterly dismiss that theory as 100% useless because
> it's totally unempirical and unprovable.
Well, qualifier (1) doesn't matter as long as qualifier (2) is in place. If
Bozo the Clown posits a new cosmological theory, then his status as a clown
neither helps nor hurts the validity of the theory. If it's empirically
verifiable, then let's verify it.
I grant that it is sometimes easier to gain access to the public stage if you
*are* a respected figure. Stephen Hawking could probably request lecture time
at Cambridge more readily than I can, but that's a political, rather than
scientific, issue.
For these purposes it doesn't matter whether or not I can personally perform
reproduce the results; that would be a hopelessly unwieldy requirement.
However, I (or any interested party) must *in principle* be able to reproduce
the results before they can be considered reliable.
If a truly bizarre claim were made by someone I respect, I would still require
that person to present his evidence. Okay, I'll admit that, in practice, I
don't demand a case study when my friend tells the story about how he
fist-fought ten guys to a standstill, but that's the difference between casual
discourse and scientific proposal. M/B is proposed as a scientific tool,
whereas my friend's claims of pugilism are not, and his claims are therefore
subject to far less stringent demands of authenticity.
> If you had a lonely, pessimistic friend (alright, I admit those are unempirical
> attributes) who got evaluated independantly by 4 psychiatrists, who all
> diagnosed him with depression, would you believe it to be true?
It depends on the method they use, of course. If they lick his elbow and say
"he's depressed," then I probably won't be too impressed with their diagnosis.
If they consult a star chart and say "he's depressed," then I likewise won't
have much confidence. But if they undertake a serious system of inquiry based
upon proven and reproducible criteria, then I'll be much more likely to put
stock in their assessment.
The M/B test, in contrast, uses neither proven nor reproducible criteria.
Dave!
|
|
Message has 3 Replies:
 | | Re: Personality test vs. Religion
|
| (...) Hm. Here's a question. Let's say that some old kook of a witch doctor uses tea leaves to predict the names of who his clients will marry (or perhaps clients ask who "friend X" will marry). The leaves predict 49,928/50,000 people's marriage (...) (20 years ago, 1-Nov-04, to lugnet.off-topic.debate)
| | | Re: Personality test vs. Religion
|
| (...) Patient readers may recall the extended exchange that DaveE and I had back in November or so regarding the Myers-Briggs Testing Instrument. Suffice it to say that we were, in the end, of surprisingly like mind; I objected to the test's use in (...) (19 years ago, 28-Mar-05, to lugnet.off-topic.debate)
|
Message is in Reply To:
| | Re: Personality test vs. Religion
|
| (...) "Deliberately" general? Got any empirical proof? :) I wouldn't say it's useless at all, except insofar as it IS error prone. If its category divisions are indeed correct (I'd say they seem to be), they may indeed help us understand how people (...) (20 years ago, 1-Nov-04, to lugnet.off-topic.debate)
|
53 Messages in This Thread:
 
 
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
This Message and its Replies on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
