The Problems of Panoptic Invigilation Programs as Evidence of Cheating

Abstract: During the pandemic, many faculty have turned to online programs to invigilate online exams. The consequences of finding a student in violation of honor code strictures can have devastating impacts on a student’s career. Therefore, it is urgent that honor councils in universities that have adopted these panoptic surveillance programs understand not only the limitations of these programs but the dangerous biases that are built into them and their usage. In this essay, I briefly review the german literature on this subject in hopes of advancing this debate. In this essay, I argue that gaze aversion cannot be the sole basis upon which to decide an honor council violation has been made.

Gaze Aversion: What Is It Good For?

The demand for online exam supervision technologies was already expanding concomitant with the burgeoning demand for online courses in higher education prior to the pandemic. However, this surge in global demand further ballooned with the sudden emergence of the COVID-19 pandemic and the swift movement of universities to online education. It seems that for the foreseeable future, online education and online invigilation programs are here to stay (Flaherty, 2020; Ginder et al. 2019; Goghlan et al. 2021).

Given the perduring nature of this demand, it is imperative that those who are availing of these programs understand the foundational problems with the algorithms they use and biases that inhere in these algorithms as well as the biases that undergird faculty assumptions about what constitutes cheating. One of the key elements flagged by these programs as “suspicious” is gaze aversion. In other words, if the program detects that a student’s gaze is not persistently aimed at the screen, the program assumes that the averted gaze is searching for material that may permit the student to cheat. Gaze aversion is then produced as evidence of malfeasance to academic integrity programs.

This fundamental assumption about the correlation of gaze aversion and academic impropriety is fundamentally flawed and cannot be taken as dispositive evidence of cheating because there is a large literature on the use of “gaze aversion” during problem-solving and information recall. In fact, gaze aversion is a natural brain function during problem-solving, especially when one doesn’t know the answers immediately. Depending upon the kind of problem being solved and the kind of information being retrieved, one may avert their gaze in one direction or the other depending upon the hemispheric allocation of that information in the brain. Salvi and Boden (2015) note that “More frequent movements of the eyes are found when people are engaged in tasks that require a search of long-term memory than when they are engaged in tasks that do not require long-term memory search, even when the tasks do not seem to have any visual component” (Salvi and Boden, 2015: 2). In other words, gaze aversion is more frequent when we are struggling to recall the answer. Similarly, Bergstrom and Hiscock (1988), report that gaze perseverance is correlated with the memory demands of different kinds of questions while Glenberg et al. (1998) report that individuals avert their gaze when they are trying to respond to different questions. Ehrlichman and Weinberger (1978) found that participants were more likely to have gaze persistence (or stare) when they were answering visuospatial questions rather than verbal questions.

Kocel et al. (1972) found that the direction of lateral eye movements was strongly correlated with the type of question, with verbal and arithmetical questions eliciting more rightward eye movements than did spatial and musical questions.” Other scholars (inter alia Ellis et al.) actually use eye movements exhibited while a person is solving a problem (anagrams) to understand how they solve problems. Susac et al. (2014) monitor eye movements during mathematical problem solving to derive insights into the cognitive processes and contend that such eye movement “may be used for exploring problem difficulty, student expertise, and metacognitive processes.” Benedek et al (2017) observe of eye movement and cognition that “Gaze aversion refers to the aversion of one’s eyes (or even brief eye closure) during demanding processes requiring internal attention. There is strong evidence that gaze aversion serves the function of reducing cognitive load during demanding cognitive activities (e.g., mental arithmetic) by avoiding the processing of potentially distracting external stimuli in order to shield internal processes” (Benedek et al, 2017: p. 2. See also Doherty-Sheddon and Phelps, 2005; Markson and Paterson, 2009). Benedek et al. also note that gaze aversion enhances visual imagination (Vredeveldt et al., 2011; Buchanan et al., 2014) as well as information retrieval (Glenberg et al., 1998), particularly during face-to-face interactions (Benedek et al, 2017). They also cite an eye-tracking study according to which “insight solutions are preceded by longer blink durations and gazing away from the stimulus, which was interpreted as a shutting out or interruption of visual input in moments of insight” (Benedek et al, 2017: 2. See also Salvi et al., 2015; Salvi and Bowden, 2016). Benedek et al, 2017 conclude from these varied studies that indicate that ocular motion supports internally directed recall by diminishing distracting sensory stimulation. These observations are all germane to cases in which gaze aversion detected by online invigilation programs is presented as evidence of academic dishonesty because they demonstrate that gaze aversion is a fundamental element of information recall.

Not only is gaze aversion a natural activity associated with cognitive recall, but there are also biological bases for the observed variation in gaze aversion. Notably, Alexander and Son (2007) attribute gender differences in eye movements during problem-solving to differential levels of androgens. This is extremely important because it may imply that using such a measure as “gaze aversion” or “gaze persistence” absent other information will disproportionately and adversely affect women who are more likely to avert their gaze during recall. Alexander and Son (2007) also observe that there is as much variation among women as there is between men and women. They report that women with “higher circulating testosterone levels” were more likely to engage in gaze persistence during problem-solving.

Biased Technology Hurts Students

In addition to the gendered nature of gaze aversion (Alexander and Son 2007), there is a large body of literature that makes it clear that commercial panoptical surveillance programs and facial recognition programs have significant race and gender bias (Leslie 2020; Castelvecchi 2020). Buolamwini, who is black, studied facial analysis software that is used in a variety of applications. When she submitted photos of herself to several commercial facial-recognition programs, the programs often failed to recognize her photos as depicting a human face and, when they did, the programs consistently incorrectly assessed her gender (Hardesty 2018).

Gebru and Buolamwini (2018) in their study of commercial facial recognition systems observed:

substantial disparities in the accuracy of classifying darker females, lighter females, darker males, and lighter males in gender classification systems” and argue that these algorithms and packages “require urgent attention if commercial companies are to build genuinely fair, transparent and accountable facial analysis algorithms” (Gebru and Buolamwini, 2018: 1). These studies of facial recognition programs matter because they are at the basis of the panoptic surveillance used in commercial proctoring programs.

Swauger (2020) dilates upon these issues at length:

While racist technology calibrated for white skin isn’t new (everything from photography to soap dispensers do this), we see it deployed through face detection and facial recognition used by algorithmic proctoring systems. Students with black or brown skin have been asked to shine more light on themselves when verifying their identities for a test, a combination of both embedded computer video cameras and facial recognition being designed by and for white people. A Black student at my university reported being unable to use Proctorio because the system had trouble detecting their face, but could detect the faces of their white peers. While some test proctoring companies develop their own facial recognition software, most purchase software developed by other companies, but these technologies generally function similarly and have shown a consistent inability to identify people with darker skin or even tell the difference between Chinese people. Facial recognition literally encodes the invisibility of Black people and the racist stereotype that all Asian people look the same.

Swauger’s entire essay should be required reading of faculty reporting or investigating suspected academic misconduct and arguably, it should be included in any anti-racism training or curriculum required by faculty.

Other Forms of Biases Inherent in Online Proctoring

There are other biases that should be evident in the reliance upon such programs. First, they have significant technological demands. They presume access to a quality laptop with a quality camera and uninterruptable and reliable internet connections. Second, they also expect students to have a private space, free of distractions, with good lighting. Clearly, not all students have such technology or facilities and the variation in that access is to be deeply dependent upon economic class, race, gender, and age (Goldberg 2021). Poorer students may not have access to a private space or quality technology. Women who have dependents or live-in partners frequently report that their personal space is violated when husbands or children barge into the room to ask questions or make other demands (Hall 2021). Some people live in congested urban environments where ambient noise is loud no matter where you are in your residence. Third, these technologies are deeply ableist (Goldberg 2021). In short, students may be falsely accused of academic dishonesty include those students: who are, more often than not, women with family duties whose attention is drawn away from the screen; who have physical and/or neurological disabilities who may find it physically difficult or impossible to do what these programs require; poorer students who lack the ability to purchase a computer with a high-quality camera or other recording requirements; and women and racial minorities whose faces are less likely to be recognized as human.

Conclusions and Recommendations

As this cursory literature review demonstrates, gaze aversion is an inaccessible metric for academic dishonesty as it is literally a physical activity associated with cognitive recall. There are gender differences in gaze aversion, with women being more likely to do so. And there is variation among individuals based upon biological factors such as individual levels of various hormones. Moreover, the technologies used to detect such cheating are riven with race, gender, socio-economic status, and other kinds of biases that are simply orthogonal to any university’s commitment to anti-racism and creating a university that is more accessible and equitable to all. Given that these allegations of cheating can have career-wrecking implications, reporting faculty should be able to marshal actual evidence of cheating rather than vague concerns about “gaze aversion.” Universities should revisit any and all cases wherein students have been found guilty of academic misconduct and ex post facto absolve students of such accusations where gaze aversion is the sole evidence provided for misconduct. Prospectively, academic integrity councils should demure from using such evidence as dispositive evidence of malfeasance and require other, supporting evidence for such a finding. Students’ lives are at stake. If we don’t take our obligations seriously, who will?

References:

Alexander, Gerianne M., and Troy Son. “Androgens and eye movements in women and men during a test of mental rotation ability.” Hormones and Behavior 52.2 (2007): 197–204.

Benedek, Mathias, et al. “Eye behavior associated with internally versus externally directed cognition.” Frontiers in psychology 8 (2017): 1092.

Bergstrom,K.J., andHiscock,M.(1988).Factorsinfluencingocularmotilityduring the performance of cognitive tasks. Can. J. Psychol. 42:1. doi: 10.1037/h0084174

Buolamwini, Joy, and Timnit Gebru. “Gender shades: Intersectional accuracy disparities in commercial gender classification.” Conference on fairness, accountability and transparency. 2018.

Buchanan, H., Markson, L., Bertrand, E., Greaves, S., Parmar, R., and Paterson, K. B. (2014). Effects of social gaze on visual-spatial imagination. Front. Psychol. 5:671. doi: 10.3389/fpsyg.2014.00671.

Castelvecchi, Davide (2020). “Is facial recognition too biased to be let loose?,” Nature, 18 November 2020. https://www.nature.com/articles/d41586-020-03186-4.

Coghlan, Simon, Tim Miller, and Jeannie Paterson. “Good proctor or “big brother”? Ethics of online exam supervision technologies.” Philosophy & Technology 34.4 (2021): 1581–1606.

Doherty-Sheddon, G., and Phelps, F. G. (2005). Gaze aversion: a response to cognitive or social difficulty? Mem. Cogn. 33, 727–733. doi: 10.3758/BF03195338

Ehrlichman, Howard, and Arthur Weinberger. “Lateral eye movements and hemispheric asymmetry: a critical review.” Psychological Bulletin 85.5 (1978): 1080.

Ehrlichman, Howard, and Dragana Micic. “Why do people move their eyes when they think?.” Current Directions in Psychological Science 21.2 (2012): 96–100.

Ellis, Jessica J., Mackenzie G. Glaholt, and Eyal M. Reingold. “Eye movements reveal solution knowledge prior to insight.” Consciousness and cognition 20.3 (2011): 768–776.

Flaherty, C. (2020). Online proctoring is surging during COVID-19. https://www.insidehighered.com/ news/2020/05/11/online-proctoring-surging-during-covid-19.

Ginder, Scott A., Janice E. Kelly-Reid, and Farrah B. Mann. “Enrollment and Employees in Postsecondary Institutions, Fall 2017; and Financial Statistics and Academic Libraries, Fiscal Year 2017: First Look (Provisional Data). NCES 2019–021Rev.” National Center for Education Statistics (2019).

Glenberg, A. M., Schroeder, J. L., and Robertson, D. A. (1998). Averting the gaze disengages the environment and facilitates remembering. Mem. Cogn. 26, 651–658. doi: 10.3758/BF03211385.

Goldberg, Suzanne B. “Education in a pandemic: the disparate impacts of COVID-19 on America’s students.” USA: Department of Education (2021).

Hall, Claire. 2021. “Women Are Facing Greater Interruption Challenges with Remote Work Than Their Male Colleagues,” UConn Today, December 13. https://today.uconn.edu/2021/12/women-are-facing-greater-interruption-challenges-with-remote-work-than-their-male-colleagues/

Hardesty, Larry. 2018. “Study finds gender and skin-type bias in commercial artificialintelligence systems,” MIT News, https://news.mit.edu/2018/study-finds-gender-skin-type-biasartificial-intelligence-systems-0212.

Kocel, Katherine, et al. “Lateral eye movement and cognitive mode.” Psychonomic Science 27.4 (1972): 223–224. Leslie, D. (2020). Understanding bias in facial recognition technologies: an explainer. The Alan Turing Institute. https://www.turing.ac.uk/sites/default/files/2020- 10/understanding_bias_in_facial_recognition_technology.pdf.

Markson, L., and Paterson, K. B. (2009). Effects of gaze-aversion on visual-spatial imagination. Br. J. Psychol. 100, 553–563. doi: 10.1348/000712608X371762

Salvi, Carola, and Edward M. Bowden. “Looking for creativity: Where do we look when we look for new ideas?.” Frontiers in psychology 7 (2016): 161.

Salvi, C., Bricolo, E., Fronconeri, S. L., Kounios, J., and Beeman, M. (2015). Sudden insight is associated with shutting out visual inputs. Psychon. Bull. Rev. 22, 1814–1819. doi: 10.3758/s13423–015–0845–0

Smallwood, Jonathan, et al. “Going AWOL in the brain: Mind wandering reduces cortical analysis of external events.” Journal of cognitive neuroscience 20.3 (2008): 458–469.

Susac, Ana NA, et al. “EYE MOVEMENTS REVEAL STUDENTS’STRATEGIES IN SIMPLE EQUATION SOLVING.” International Journal of Science and Mathematics Education 12.3 (2014): 555–577.

Swauger, Shea. “Our bodies encoded: Algorithmic test proctoring in higher education.” Critical Digital Pedagogy (2020).

Vredeveldt, A., Hitch, G. J., and Baddeley, A. D. (2011). Eye closure helps memory by reducing cognitive load and enhancing visualization. Mem. Cogn. 39, 1253–1263. doi: 10.3758/s13421- 011–0098–8

Walcher, Sonja, Christof Körner, and Mathias Benedek. “Data on eye behavior during idea generation and letter-by-letter reading.” Data in brief 15 (2017): 18–24.