Coming Full Circle – Rethinking the Use of Standardized Assessments for School Admissions

Coming Full Circle – Rethinking the Use of Standardized Assessments for School Admissions

Coming Full Circle – Rethinking the Use of Standardized Assessments for School Admissions

Major universities are beginning to see the value of standardized assessments to help disadvantaged students gain admission into their programs.

 

Exam Master launched the PA-CAT in 2020 in response to PA educators telling us that some of the metrics they were using to help decide which applicants should be admitted into their programs, including prior GPA, were not particularly reliable or predictive.  So-called grade inflation and undergraduate program variability were perceived as a big factor in making prior grade history less reliable as an academic predictor.  Consequently, some incoming PA students who appeared to have the academic chops for PA school were really struggling during parts of their didactic training, particularly in core subjects like anatomy and physiology.

Most PA educators we spoke with agreed that having an admissions test that objectively measures the academic readiness of an applicant would be useful.  This value included helping programs choose among applicants who may appear similarly qualified based on grades or other criteria. Nevertheless, some PA educators we spoke with had at least some concerns that use of a PA school admissions exam might act as a barrier to entry for under-represented students seeking to apply to PA school.  Many PA programs have a stated goal to improve the diversity of their student body to better represent the communities that they serve.  These attitudes existed at a time when educational stakeholders generally were questioning the value of standardized admissions exams.   Research showing the effects standardized entrance exams might have on the ability of minority or lower-income students to be competitive in their college application process was a concern.

 

COVID-era policies related to optional or no-use of standardized admissions exams found to adversely affect disadvantaged and minority applicants.

 

Then came COVID… During COVID, some colleges and universities were dropping the requirement that applicants must take the SAT or ACT exams as a condition for admission consideration.  Some schools left these test requirements as optional for applicants.  (Many schools still make these exams optional.) To us, this seemed a bit counter intuitive since we experienced firsthand how disruptive COVID was to the many health professions programs we serve.  And we know how difficult COVID was for many colleges and universities trying to fulfill their educational commitment to their students.  The question people were wondering (but not always saying) was, what impact did these disruptions have on these students’ readiness to go into college or graduate school?  Are they as academically prepared as pre-COVID cohorts?

 

Recent studies show that standardized exam results are better predictors of student potential than prior grades and other common application requirements.

 

Recently, a group of New York Times researchers and reporters took a hard look at the issue of standardized exams and their impact on admissions to college.  School administrators were reporting to the Times researchers that, for getting into college at least, standardized tests like the ACT and SAT were better predictors of student performance than high school grades alone.  They provided data to back this up.  Moreover, the researchers pointed out that a standardized exam might be less biased of a marker of student readiness than such things as letters of recommendation, essays, and extra-curricular activities, metrics which inevitably are subject to greater bias and subjectivity than a standardized exam.  MIT reported that bringing back the SAT test requirement helped it to admit the most diverse incoming student cohort on the college’s history.  You can read the full article here.

 

Dartmouth, Yale and MIT adjusted their admissions requirements to include standardized assessments as a tool to improve diversity of their incoming cohorts.

 

More recently, Dartmouth College announced it was going back to requiring a standardized assessment for admissions consideration.  This policy change came on the heels of a recently concluded Dartmouth study showing that test scores were a better predictor than high school grades (and other admissions requisites) of how well students would perform at Dartmouth.  Moreover, researchers at Dartmouth (which had been using a test optional policy since COVID) found that test scores from lower-income students, had they been submitted as part of the applications process, would actually have helped many of these students present a stronger admissions profile than grades and recommendations alone. In other words, those lower-income students who withheld their optional test scores from admissions were actually harmed by this misstep.  More information on Dartmouth’s experience can be found here.  Yale University also just announced it will be going back to use of standardized assessments for admissions in part because of its belief that such assessments will actually improve disadvantaged students’ ability to be admitted into their university.

Throughout this time, Exam Master has remained strongly committed to the belief that an academic readiness exam (e.g. the PA-CAT) will help PA educators better identify those students who are prepared to handle the academic rigor required in PA school.  A standardized assessment, assuming it covers the key subjects incoming PA students are supposed to already know, should also help students from less advantaged backgrounds.  Students coming from weaker under-graduate programs, or whose educational journey might have been less privileged or less traditional, could still demonstrate with a good PA-CAT score that they were academically ready to tackle the demanding PA curriculum.

 

[April 2024 Update:  Harvard University revives standardized testing requirements.  Details here.]

Recent research shows that the PA-CAT has been predictive of PA student performance during the didactic semesters.

The research and developmental phase of the PA-CAT showed a statistically significant correlation between PA-CAT scores from incoming PA students and how they performed in PA school.  More recently, research at several PA programs has shown a statistically significant correlation between applicant PA-CAT scores and their subsequent didactic performance.  We believe these studies and others to follow will continue to support that the PA-CAT can both better predict student performance and also offer applicants from disadvantaged backgrounds an opportunity to demonstrate that they are academically ready for PA school.

See here for the latest on the PA-CAT research studies.

Related Topics:

PA-CAT Receives High Marks from Candidates for Exam Relevance and Study Support

PA-CAT Receives High Marks from Candidates for Exam Relevance and Study Support

PA-CAT Receives High Marks from Candidates for Exam Relevance and Study Support

Emerging research suggests that the PA-CAT, now in its fourth year of deployment, has statistically predictive value as it relates to PA student performance during the didactic phase of their studies and training. While this outcome confirms the findings of our original PA-CAT developmental research, we now have over 3500 PA school candidates who have completed the PA-CAT.  So, in addition to the ongoing program-level research, we felt it would be useful and important to ask the candidates who took the PA-CAT what they thought of it.

 

Candidate Feedback Consistent with Objective Research Findings 

Building off of feedback received from an early candidate survey we conducted in 2021, we were looking for current feedback on such questions as whether the candidates felt the exam covered material they would expect to know or learned in undergraduate school? Did the exam cover material that was relevant to what these students are now studying in PA school?  Was the exam at an appropriate level of difficulty?  Were the Exam Master-provided study materials relevant and helpful to preparing for the actual exam?

In early January, a survey invitation was emailed to over 2800 PA-CAT exam takers.  Of this group, we had an impressive 60% open rate, and of these opens 63 candidates submitted completed surveys (3.8% response rate).  38% of the respondents (n=24) reported taking the PA-CAT within 6 months of the survey. 24% of respondents (n=15) reported taking the PA-CAT within the last 6-12 months.  6 survey respondents reported they had not yet taken the exam but did make some comments about the study materials.

The responses from the 2021 survey surprised us in that most respondents expressed rather favorable attitudes toward the PA-CAT.  Frankly, we expected more grumbling and pushback at that time.  We wondered, two years and thousands of completed exams later, would these candidate attitudes be different?  And, with a more robust set of study materials now available to candidates than before, were these materials deemed relevant and useful to these candidates preparing for this comprehensive assessment of academic preparedness?

 

Most Candidates Express Positive Attitudes Toward PA-CAT and Study Materials 

The results again were overwhelmingly positive.  This candidate’s response sums up much of the feedback we received:

I decided to take the PA-CAT because it is one of the requirements for the PA program I was applying to. The study resources included with the PA-CAT exam were the only resources I used and they prepared me well for the exam, especially the videos and the practice exams. While I was taking the exam I felt prepared and believe these resources are tailored for the exam and for future success in PA school. Other benefits to taking the PA-CAT exam is getting more knowledge to prepare you for PA school, I actually found it extremely helpful as I had a surprise mini quiz on my interview day, and was able to answer due to what I had just learned from my Exam Master resources.

 

Feedback on the available study materials was generally very favorable as well as exemplified in this response:

I took the PA-CAT because it best correlates to a high PANCE passing score more so than other entrance exams, such as the GRE, does not focus on any applicable skills or information. The included study materials were my primary source of content review and test questions. It is the best review material I have ever used. It is perfectly concise yet comprehensive and the explanations given for the practice questions made all the difference. The only change I would make is I felt the content review could have been more thorough for the general chemistry portion as it lacked in info needed compared to other subjects.

 

Some Contrarian Opinions About the PA-CAT

While the majority of responses were favorable toward both the PA-CAT and the Exam Master study materials, of course not all candidates thought highly of the study materials or the PA-CAT itself.  One candidate described the PA-CAT as just another hoop that candidates have to jump through to get into PA school.  Some individuals felt the PA-CAT was weighted too heavily in some subjects.  Some individuals felt that the study videos were too dry or not an efficient way to prepare for the exam.

While this survey was not intended to be a scientific examination of overall PA-CAT candidate attitudes about the PA-CAT, the survey responses indicate that many candidates felt the PA-CAT was fair, relevant, and more targeted than a more general admissions exam such as the GRE.  It also appears that our study materials generally hit the mark in terms of helping candidates prepare for the exam.

For, a complete set of de-indentified responses please click here.  If you have thoughts or questions about the PA-CAT please feel free to reach out.

Related Topics: