Issues with changing the "passing score" in the NY Bar exam
Of relevance to recent discussion about changing the scaled score necessary for passing the New York state bar exam:
--> discussion of the work of Klein appears in the Record of the City Bar of New York.
Klein's methodology for the New York study was denounced by the New York City Bar Association. Bar groups in several other states, notably Minnesota and Florida, have likewise attacked his work.
On the last point, the three deans said Bosse was closely questioned by senators as to documented evidence -- rather than anecdotal -- of incompetent New York lawyers.
The deans said Bosse offered no documentation. [reminds one of discussion of patent reform. There is more discussion of the peanut-butter-and jelly patent of Smucker than there is of a systematic problem with patent quality.]
In an e-mailed response for comment, Bosse wrote, "The purpose of the bar exam is to protect the public by providing reasonable assurance that those licensed to practice law in New York have met minimal competency standards. The criteria for determining who is qualified ... are set by the courts and include the requirement that applicants pass the bar exam. ... The passing score is intended to separate those applicants who are minimally competent ... from those who have not yet met that standard."
She added, "The bar exam samples broadly the legal knowledge and lawyering skills of the population of candidates for admission to the bar. Malpractice claims and disciplinary complaints are snapshots of individual performances in practice. They are not a systematic or comprehensive evaluation of the competence of lawyers, either individually or collectively. While we looked at some available evidence ... these measures don't give us enough information either to validate an existing passing score or to support some alternative standard."
The NY bar exam people have a formula to convert the multistate results to give them a score of 660. Then, they scale the essay results (6 essays, each scored 1-10), to the same 660 median. Then they fuse all results.
The NY City Bar points out that people could be answering all essays very intelligently or very stupidly, but the results are scaled the same. Changing the cut point only changes the % who pass.
The first point of the City Bar was that changing the cutpoint would have a dramatically greater impact on black candidates than white (apparently, NY has data on median scores for blacks and whites, and the black median is lower, and many more blacks (proportionately) would not be admitted if the cut point were raised).
Any given individual can benefit by more study and get a higher raw score which will translate into a higher scaled score. The NY City Bar was merely pointing out that changing the cutpoint of the scaled score only changes the number who will be admitted. Klein and others argued that raising the scaled cutpoint would eliminate more incompetent lawyers. The City Bar's counter is that the scaled score is not a direct measure of "goodness" on the exam (the raw score would be).
As a different point, the City Bar questioned Klein's correlation only to the essay portion. On the bar exam (when I took it and apparently still true), test takers do very poorly on the NY state law multiple choice. Any given question is only 55% likely to be answered correctly, far lower than for the multistate. Because this portion accounts for only 20% of the result, the bar reviews tell test takers not to waste much time on this part of the exam, and most people don't. Anyone who actually practiced regular law in NY state would do well, but a student (like from Univ of Chicago) with no NY practice would have to invest a lot of time to score well.
--> recall the case HOOVER ET AL. v. RONWIN ET AL., 466 U.S. 558; 104 S. Ct. 1989; 80 L. Ed. 2d 590 (1984):
One who failed the Arizona bar examination brought suit in the
United States District Court for the District of Arizona against members of the
Arizona Supreme Court's Committee on Examinations and Admissions, alleging that
the committee members had conspired to restrain trade in violation of 1 of
the Sherman Act by artificially reducing the numbers of competing attorneys
in the state of Arizona through setting the bar examination grading scale with
reference to the number of new attorneys sought desirable rather than
with reference to some suitable level of competence.
Respondent Ronwin was an unsuccessful candidate for admission to the Bar of Arizona in 1974.
n5 According to Ronwin's complaint, the Committee announced before
the February examination that the passing grade on the test would be 70,
but it assigned grades using a scaled scoring system. Under this system, the
examinations were graded first without reference to any grading scale.
Thus, each examination was assigned a "raw score" based on the number of
correct answers. The Committee then converted the raw score into a score on a
scale of zero to 100 by establishing the raw score that would be deemed the
equivalent of "seventy." See n. 19, infra.
**ALSO
Note Impact of the Increase in the Passing Score on the New York Bar ...
at www.nybarexam.org/summary.pdf
--> discussion of the work of Klein appears in the Record of the City Bar of New York.
Klein's methodology for the New York study was denounced by the New York City Bar Association. Bar groups in several other states, notably Minnesota and Florida, have likewise attacked his work.
On the last point, the three deans said Bosse was closely questioned by senators as to documented evidence -- rather than anecdotal -- of incompetent New York lawyers.
The deans said Bosse offered no documentation. [reminds one of discussion of patent reform. There is more discussion of the peanut-butter-and jelly patent of Smucker than there is of a systematic problem with patent quality.]
In an e-mailed response for comment, Bosse wrote, "The purpose of the bar exam is to protect the public by providing reasonable assurance that those licensed to practice law in New York have met minimal competency standards. The criteria for determining who is qualified ... are set by the courts and include the requirement that applicants pass the bar exam. ... The passing score is intended to separate those applicants who are minimally competent ... from those who have not yet met that standard."
She added, "The bar exam samples broadly the legal knowledge and lawyering skills of the population of candidates for admission to the bar. Malpractice claims and disciplinary complaints are snapshots of individual performances in practice. They are not a systematic or comprehensive evaluation of the competence of lawyers, either individually or collectively. While we looked at some available evidence ... these measures don't give us enough information either to validate an existing passing score or to support some alternative standard."
The NY bar exam people have a formula to convert the multistate results to give them a score of 660. Then, they scale the essay results (6 essays, each scored 1-10), to the same 660 median. Then they fuse all results.
The NY City Bar points out that people could be answering all essays very intelligently or very stupidly, but the results are scaled the same. Changing the cut point only changes the % who pass.
The first point of the City Bar was that changing the cutpoint would have a dramatically greater impact on black candidates than white (apparently, NY has data on median scores for blacks and whites, and the black median is lower, and many more blacks (proportionately) would not be admitted if the cut point were raised).
Any given individual can benefit by more study and get a higher raw score which will translate into a higher scaled score. The NY City Bar was merely pointing out that changing the cutpoint of the scaled score only changes the number who will be admitted. Klein and others argued that raising the scaled cutpoint would eliminate more incompetent lawyers. The City Bar's counter is that the scaled score is not a direct measure of "goodness" on the exam (the raw score would be).
As a different point, the City Bar questioned Klein's correlation only to the essay portion. On the bar exam (when I took it and apparently still true), test takers do very poorly on the NY state law multiple choice. Any given question is only 55% likely to be answered correctly, far lower than for the multistate. Because this portion accounts for only 20% of the result, the bar reviews tell test takers not to waste much time on this part of the exam, and most people don't. Anyone who actually practiced regular law in NY state would do well, but a student (like from Univ of Chicago) with no NY practice would have to invest a lot of time to score well.
--> recall the case HOOVER ET AL. v. RONWIN ET AL., 466 U.S. 558; 104 S. Ct. 1989; 80 L. Ed. 2d 590 (1984):
One who failed the Arizona bar examination brought suit in the
United States District Court for the District of Arizona against members of the
Arizona Supreme Court's Committee on Examinations and Admissions, alleging that
the committee members had conspired to restrain trade in violation of 1 of
the Sherman Act by artificially reducing the numbers of competing attorneys
in the state of Arizona through setting the bar examination grading scale with
reference to the number of new attorneys sought desirable rather than
with reference to some suitable level of competence.
Respondent Ronwin was an unsuccessful candidate for admission to the Bar of Arizona in 1974.
n5 According to Ronwin's complaint, the Committee announced before
the February examination that the passing grade on the test would be 70,
but it assigned grades using a scaled scoring system. Under this system, the
examinations were graded first without reference to any grading scale.
Thus, each examination was assigned a "raw score" based on the number of
correct answers. The Committee then converted the raw score into a score on a
scale of zero to 100 by establishing the raw score that would be deemed the
equivalent of "seventy." See n. 19, infra.
**ALSO
Note Impact of the Increase in the Passing Score on the New York Bar ...
at www.nybarexam.org/summary.pdf
posted by Lawrence B. Ebert at 11:15 AM
0 Comments:
Post a Comment
<< Home