Exploring Remote Proctoring of High Stakes Assessments

Exploring Remote Proctoring of High Stakes Assessments

Exploring Remote Proctoring of High Stakes Assessments

We recently participated in Prometric’s demonstration of their remote proctoring system called ProProctor™.  Remote proctoring has been gaining traction as an acceptable, secure delivery method for a variety of high stakes exams. More recently, remote proctoring has become acceptable for some credentialing examinations that were previously administered at testing centers, which are now closed due to the COVID-19 crisis. Remote proctoring allows examinees to conveniently schedule and take an examination safely from their homes or other off-site locations and avoid the logistical hassles and inconveniences of having to appear at a testing center.

The concern for educators and administrators responsible for high stakes assessments are exam security and exam integrity. Ensuring that the results are fair, accurate and reliable over time is most important. Frankly, cheating is always a concern. Typically, these examinations require test-takers to be at a testing location, such as a computer lab, classroom or other location, where they can be monitored as a group. These involve using controlled access computers to ensure there is no cheating or other activities that may compromise the examination. The technologies and protocols now available with remote proctoring, including ProProctor™, offer a suitable alternative to the traditional computer lab, test center or classroom-delivered assessments.

In our ProProctor™ demo, we watched how much effort and work goes into ensuring the integrity of an examination from a remote setting. The better remote proctoring systems, like ProProctor™, do involve having human proctors who monitor examinations remotely. Human proctors are typically responsible for monitoring six to eight examinees at one time. These systems use a variety of technologies, processes, and protocols to ensure that exam security and integrity is protected. Our impression was that remote proctoring protects the examination process almost as well as it would be at a testing center, and probably better than a classroom or computer lab setting where the ratio of proctors to students is often not favorable.

As we learned from watching a demo of Prometric’s system, a big part of the remote proctoring process involves the initial technology set up, authentication of the exam candidate’s identity, and verification that the candidate’s examination environment strictly adheres to a stringent set of protocols. We are talking about the assessment of a candidate’s computer, capturing a candidate’s facial image for later monitoring, a thorough review of the candidate’s testing location, authentication via a government-issued ID, and including having candidates roll up their sleeves and show their ears to make sure they do not have hidden devices! During the examination, there is constant monitoring, both from human proctors as well as AI-type technologies that examine candidate movement and behavior during the assessment. Our impression was that very few candidates could or would attempt to game the system.  It is that good.

ProProctor™ will be used for early administrations of the Physician Assistant College Admissions Test (PA-CAT) as a safe alternative to testing centers. We believe Prometric’s remote proctoring system offers a robust and safe alternative for PA-CAT candidates. Testing for the PA-CAT begins May 1st.

Thank You Drexel University!

Thank You Drexel University!

Thank You Drexel University!

While developing Exam Master’s PA-CAT website, we thought it would be great to use images of actual PA students on the site. Drexel University is local to the Exam Master headquarters and was a participating school in the research phase of the PA-CAT. We were delighted to be invited to take pictures of their students and facilities.  

We walked through the halls of the College of Nursing and Health Professions building, stopping in classrooms and simulation rooms to photograph the learning environment. There were mannequins with bright eyes and tattoo sleeves laying in hospital beds in every room. Our photographer snapped pictures of simulation activity prompts, small examination rooms, and hospital-like hallways.

During our visit to Drexel, we met students who had just entered their didactic year of Physician Assistant school. When we mentioned the PA-CAT to the students, many of them acknowledged that they had taken the test during their first week of the program. The students had positive feedback about the development of an admissions test specifically for physician assistant programs and were excited to be a part of the research phase.

We were able to watch an educator demonstrate a level of patience and care while showing her students how to perform a patient evaluation of fine motor skills that was exceptional. Four students allowed us to capture the process of them evaluating each other under the close watch of their instructor. It was evident from our time at Drexel that the educators take great pride in the way that they prepare the future PA’s in their program. 

Exam Master would like to thank Drexel University for allowing us to visit their PA program to take pictures for our PA-CAT website. We understand that instruction time is valuable to both educators and students, and they were gracious enough to allow us to capture photos of that time. 

Take a look at some of the great pictures we took while at Drexel University!

PA-CAT: The First Step to a Career as a Physician Assistant

PA-CAT: The First Step to a Career as a Physician Assistant

PA-CAT: The First Step to a Career as a Physician Assistant

Over the past five years, there has been a significant increase in the total number of applicants to PA schools. While this is partly a result of the tremendous increase in the number of accredited PA programs in the U.S., it means that a typical PA program must manage hundreds of applications. Many applicants are highly qualified, making the choice of filling the small incoming class for a typical PA program very difficult. Acceptance rates at PA programs hover around 33%, even lower for the most competitive programs.

Undergraduate GPA, undergraduate science GPA, prerequisite coursework, and prior work history are among the criteria that PA programs use to make their admissions decisions. Since there has never been an admissions assessment specifically designed for the PA curriculum, many PA educators have had to also rely on the GRE to help with admissions decisions. With difficulty in comparing GPAs from different undergraduate schools, variability in grading practices (e.g. grade inflation), and with a standardized exam that does not focus on the science subjects needed for success in PA school, PA educators have struggled to make the best admissions decisions from a large group of highly qualified applicants.

After conducting extensive research, including conversations with dozens of PA educators, it became clear to us at Exam Master that an admissions exam truly targeted to the PA profession was needed and wanted. Given the demands of the PA curriculum, PA educators felt it was paramount that applicants be assessed on their academic preparedness to ensure they could handle the rigor that awaited them —applicant familiarity with prerequisite science subjects upon admission would be fundamental to their success during the critical didactic year. 

Led by Johnna Yealy, PhD, PA-C and Scott Massey, PhD, PA-C, the research and development phase of the PA-CAT featured field testing with over 35 participating PA programs, involving over 1600 students and interview candidates. The 180-item PA-CAT research exam was developed to assess student comprehension of common basic science subjects required as prerequisites by most PA programs. Results from initial testing were tracked and correlated with subsequent student performance.

PA-CAT field testing benefitted from significant community involvement and feedback. The exam was challenging, but anecdotal feedback from students suggested that the material covered was relevant to what they learned in college. PA educator Sandra Keavey, PA-C reported  a “strong correlation between scores on the PA-CAT and academic performance.” The strongest students in her program’s current cohort performed well on the PA-CAT. Keavey sees this PA-CAT as a tool helpful for screening potential applicants and identifying incoming students who may be at risk for falling behind during their didactic training.

The commercial version of the PA-CAT is available for registration in January 2020, with the exam window opening in May of 2020. This exam features 240 items and covers the following subjects: 

  • Anatomy
  • Physiology
  • General Biology
  • Biochemistry
  • General and Organic Chemistry
  • Microbiology
  • Behavioral Sciences
  • Genetics
  • Statistics

Used as part of a holistic admissions process, the PA-CAT has been developed to help PA programs better predict which applicants are more likely to be successful in their demanding programs. There will be rich opportunities over the coming years to refine the exam, perfect it, and ensure that it meets the needs of the PA community and all of its stakeholders.

If you would like more information about the PA-CAT, visit us or contact us directly.