Brown placed 47th out of 50 schools in a study on civic literacy conducted by the Intercollegiate Studies Institute.
The results were released two weeks ago in a report titled, "The Coming Crisis in Citizenship."
In 2004, the ISI administered a multiple-choice exam that included 60 questions on American history, institutions and economic theory to seniors and freshmen at 50 schools. The ISI then compared the differences in scores between seniors and freshmen in an attempt to evaluate how effectively each school presents this knowledge as part of its curriculum.
Rankings were then based on the difference in percentage points between freshmen and seniors at each school.
"Our focus was not on the student, but on the curriculum. So what we wanted to know was whether the educational experience at Johns Hopkins (University) or Rhodes (College) is better based," said Mike Ratliff, executive director of the American Civic Literacy Program, the branch of the ISI that produced the study.
The ISI "intended (the exam) to be an accurate gauge of student learning at the freshman and sophomore introductory course level," Ratliff said. "We pre-tested it in 2004 and removed all the questions where nobody got them. As a result we believe that the test probably doesn't need to be on a curve."
Faculty criticismThe methodology, administration and content of the exam received harsh criticism from faculty at several of the surveyed schools interviewed by The Herald.
"I think that this is a completely worthless study," said Richard Polenberg, a professor of history at Cornell University, which placed 48th out of 50 schools. Polenberg pointed out that no data was released on the number of students surveyed at each college. The results from many schools might be based on an unrepresentative sample, he said.
The report stated that there was a quota for each school, but it did not specify the size of the quota.
Polenberg also criticized the administration of the exam, mentioning that one of his senior students was enticed to take the exam after being promised free entry into an iPod raffle. Since the student was pressed for time, he took the exam in a matter of minutes by randomly selecting answers.
Rhodes College, located in Memphis, Tenn., placed first in the study. However, Gail Murray, chair of that school's history department, expressed skepticism regarding the study's validity.
Murray criticized what she described as an apparently haphazard selection process that did not account for gender distributions or SAT scores between freshmen and seniors at a given school or among different institutions.
The content of the exam has also raised doubts among faculty at several schools.
The report did not include actual survey questions and instead displayed a list of "question themes."
The ISI is an avowedly conservative organization that runs a number of programs to "nurture an appreciation of the principles of freedom," according to its Web site. These principles, as posted on the site, include "limited government," "free market economy" and "moral norms" - which are defined as "(t)he values, customs, conventions, and norms of the Judeo-Christian tradition."
Seth Rockman, assistant professor of history at Brown, expressed concern that questions about the "gains from trade" and other topics may have underscored the ISI's ideological slant and may not have been entirely based on factual information.
Rockman also questioned the legitimacy of certain questions about issues that are still politically contested, such as the "separation of powers."
Polenberg echoed Rockman's skepticism.
Given the ISI's political bent, displaying actual questions asked about themes like "Keynesian economic thought and policy," "alternate forms of government" and "George Washington's founding role" would be necessary for any evaluation of the report's credibility, he said.
The report was "not trying to be evenhanded," said Joel Carpenter, a member of the history department at Calvin College, a Christian liberal arts college that ranked third in the study. "I think the ISI has its own political axe to grind, but I wouldn't dismiss the findings altogether," he added.
In addition to concerns about the content of questions asked, faculty at several schools expressed concerns about its format.
"The whole study seems incredibly flawed because it was a multiple-choice exam," Murray said. "The notion that that is how you assess how much people know about their country, or about their history, is flawed."
Professor of History Howard Chudacoff also objected to the multiple-choice format. The memorization of facts does not necessarily demonstrate an understanding of those facts, he said.
For Chudacoff, part of the larger "function of a Brown history course is to get students to think about the human condition, to ask questions, to be able to undertake research and to present material and arguments in a cogent, effective way."
Rating students' relative performanceThe ISI's ranking method does not incorporate the relative performance of students at different schools. However, the report does provide information separate from the rankings that describes how students at different schools performed.
Though Brown finished 47th in the rankings of score improvements between freshmen and seniors, Brown seniors' mean scores were 16th out of 50. Florida Memorial University, located in Miami Gardens, Fla., placed ninth in the study, a full 38 places ahead of Brown. But Florida Memorial finished dead last in freshmen and senior mean scores, while Brown first-years more than doubled the mean score of Florida Memorial seniors.
The report claims that an "Ivy League education contributes nothing to a student's overall knowledge of America's history and institutions." Other elite schools that fared poorly in the rankings of score improvements between freshmen and seniors included Georgetown University (No. 43), Yale University (No. 44), Duke University (No. 46) and Cornell University (No. 48).
According to the ISI's Ratliff, there are several problems plaguing undergraduate education, including the disappearance of core curriculums.
"Certainly at Brown that doesn't exist, and students are given very wide latitude in what they would pick," Ratliff said.
Carpenter said low scores may be the result of an emphasis on specialized seminars instead of broad survey courses.
"Rather than biting the bullet and saying, 'What do students need in an education generally?' (professors) go for the intellectual candy, rather than the vegetables," Carpenter said.
Associate Professor of History Michael Vorenberg had a different take on the importance of broad-based requirements.
"I think it's naive to think that that (strenuous requirements) alone would create a change ... in the level of civic engagement," Vorenberg said.
Murray stressed the fact that Rhodes College does not require any specific history courses, adding that many of students' humanities requirements can be filled in non-historical departments such as philosophy or literature.
"I don't think we go about the process of teaching history different than most small, selective liberal arts colleges do," Murray said.
