No, that title is not a typo. I know some of you grew up reading stories of the supernatural that featured some mysterious thing called a Black Mass. Details differed, but it was supposed to be some occult ritual at which unspeakable things were done for unspeakable purposes.

Well, this post is not about that. It is about what I will call Black Math, as a convenient abbreviation for a whole whack of really awful ideas about education, largely but not entirely regarding black kids. My starting point will be a person, one Danny Martin, who would very much like you to know that he is a Professor of Education and Mathematics in the Department of Curriculum and Instruction in the College of Education at the University of Illinois at Chicago.

Note that is not The University of Chicago, nor is it The University of Illinois. The University of Illinois at Chicago is a separate institution from those other two, and I would not wish you to think either of them provides employment to Professor Martin. Note also, please, that despite his very misleading title, he is certainly not a Professor of Mathematics. He’s in the College of Education for chrissakes, and I am sure no self-respecting Department of Mathematics would hire him. Those cats are weird, to be sure, but still…..

How I came to be familiar with Martin will be revealed below, but I will start this tale with a full rendition of the Abstract from a paper he published in a journal called Race, Ethnicity and Education in 2019. The paper is titled ‘Equity, inclusion, and antiblackness in mathematics education’ and its Abstract, in its entirety, reads as follows:

Despite decades of equity- and inclusion-oriented discourse and reform in mathematics education, Black learners in the U.S. continue to experience dehumanizing and violent forms of mathematics education. I suggest that equity for Black learners in mathematics education is a delusion rooted in the fictions of white imaginaries, contingent on appeasing white logics and sensitivities, and characterized at best by incremental changes that do little to threaten the maintenance of racial hierarchies inside or outside of mathematics education. Moreover, the forms of inclusion offered up in equity-oriented discourses and reforms represent contexts of containment and enclosure that keep Black people in their same relative position. Refusal is suggested as a strategy for Black learners to resist the anti-Black character of mathematics education, and as a first step in actualizing forms of mathematics education that are worthy of Black learners.

Let me start by noting that while I can imagine a student finding math difficult and even humiliating as a subject, I doubt seriously that by ‘violent forms of math education’ is meant smacking students who get things wrong in math class. Nor do I have any idea what is a ‘white imaginary’ a ‘white logic’ or a ‘context of containment’. I do suspect, from other things by this person that I have read, that by ‘refusal’ he means that black students should just avoid math class. I could, however, be wrong. As you will see, clarity is not Martin’s strength.

Now, just because I don’t know what a phrase or term means doesn’t mean it means nothing. Lord knows people in my own former discipline throw around terms like ‘Pareto efficiency’ and ‘consumer surplus’ that no one outside Econ understands.

So, I dug into the paper itself to see if I could suss out what Martin was on about.

I tried, I really did. Bear with me as I subject you to some of the prose in this paper.

Antiblackness is readily evidenced in mathematics education via several forms of systemic violence–epistemological, intellectual, and symbolic–that inflict injury and trauma on Black people (Martin, Price, and Moore in press). Teo (2010), for example, has characterized epistemological violence in the production of knowledge as occurring ‘when theoretical interpretations regarding empirical results implicitly or explicitly construct the Other as inferior or problematic, despite the fact that alternative interpretations, equally viable based on the data, are available’.

So far as I can discern, he is saying that when the performance of black students in math is studied, the fact that they do worse (although only on average), and so the discussion is about how to help them do better, is itself violence that inflicts trauma and injury. There is at least nothing really racial about this, in my view. It is all of a piece with the general view among ‘educational theorists’ these days that one should not focus ever on student shortcomings. Doing so is bad for them, or at least bad for their self-esteem. To correct is to traumatize.

I will take exception to that bit about interpretations that implicitly or explicitly construct the so-called ‘Other’ as inferior. That’s just horseshit, saying that no matter what an instructor or analyst of data on achievement does or says, what is really going on – implicitly – is racism. Boolsheet, as my former Russian colleague liked to say.

Another sentence from the paper:

Mathematical illiteracy is not a naturally occurring trait of Black children but has become a widely accepted signifier for Blackness and a decidedly Black geography (McKittrick and Woods 2007) subject to, and deserving of, violent description and intervention.

OK, agreed that ‘mathematical illiteracy’ (I thought the standard term was ‘innumeracy’, btw) is not a naturally occurring trait. But who says it is a ‘widely accepted signifier for Blackness’? Only Martin, so far as I know. And – ‘violent description’? Really? A description can be violent?

This is another shibboleth of this kind of academic. Anything they do not agree with, even words, is said to be ‘violent’, which then means that it is terrible, awful, horrible, because everyone is against violence, right?

Bloody hell.

Anyway, I hope it is clear, faithful readers, that I am trying here to use the context in which Martin uses these words and phrases in his article to figure out what he is really trying to say. I got nothing about white imaginaries this way, nor about white logic, he uses those only in the Abstract, so on to ‘refusal’.

Well, here’s a mouthful for you from the paper:

Envisioning and actualizing refusal requires one to invoke Black radical imagination (Kelley 2002) and engage in Black liberatory fantasy (Dumas and ross 2016) about the form and structure of such refusal and how it facilitates Black people flourishing in their humanity.

Yea, didn’t help me understand him any better, either. Here’s more….

Specifically, I refer to refusal in and refusal of mathematics education institutions, practices, and policies that instantiate anti-Black racism and white supremacist orientations. I am not suggesting that Black learners avoid pursuing mathematics within the existing system of mathematics education. For most learners, it is the only system available. Refusal in is a recognition of this fact. I am suggesting that the pursuit of mathematics knowledge within this system should not be for the sole purpose of being accepted into anti-Black and white supremacist spaces.

So, if you imagine yourself as a ‘Black learner’, what do you think this paragraph is telling you to do? If it is okay for one to pursue math, just not for that ‘sole purpose’, then….what should you do?

How about this?

Refusal in would also entail working against the commodification of and trafficking in Black suffering, especially as a way to counter appeals to white benevolence and liberal notions of care.

More helpful? Not to me, but my readers may be more perceptive than I.

This is what I have come to realize characterizes most writing by this sort of person. Absolutely nothing concrete is ever said. What in the name of anything holy is this guy saying anyone should do? I have read many many words, that say nothing anyone – including a ‘Black learner’ – can use as a guide to any kind of behaviour or plan or…..bloody anything. The mark of a well-trained academic used to be clarity. Now it is obtuseness, vagueness.

Ok, so how did I get onto this whole thing, and Prof Martin? I cannot trace to the absolute beginning, but something sent me to this article on a site called RealClear Investigations called ‘Billionaires Backing Woke Math Doesn’t Add Up Amid DEI Rollback’  which was about that: some wealthy people funding initiatives like one called Racial Justice in Early Math which aim to eliminate separate classes for students who are mathematically inclined/gifted and also one called ‘TODOS Mathematics for All, an Arizona-based organization that calls for elevating DEI practices and anti-racist activism into all math instruction’ according to the article.

Another such org, the Erikson Institute, has a video on its website titled Using Math to Learn about Racial Representation in K-2 by Priscila Ulegis Pereira.

I watched it, you can too, here.

For a taste, at one point in the video what is on the screen is headed with the words

2^nd Grade: Is Our Library Equal?

 Then below that one sees:

 After building background knowledge, students embarked on the complex task of identifying the race of characters in books in our classroom library.

This is for a 2^nd grade math class, remember. So I guess they had to count the number of white people, black people, etc.

Over the years, as I watched the standards to which students are held at all levels of the education system deteriorate, I became convinced that this was being done almost entirely by those within the system. That is, teachers (not all, thankfully) and administrators (nearly all) all the way up from kindergarten to post-graduate, were themselves the driving force behind lowering standards. And the primary reason for this denigration, I firmly believe, is that teaching to a high standard is very hard. It is hard work, because the teachers have to work just as hard as they expect their students to work. Instructors have to read a lot of what students write and provide copious and clear feedback, they have to go through their math and correct and explain why things are wrong, and do so clearly. Thus, lowering standards makes things easier for everyone, but in particular for the instructors.

Thus, I think Martin’s ‘math is epistemological violence’ bullshit is just one dimension of a much bigger phenomenon. He is, after all, a faculty member in a College of Education; ya know, the people who are supposed to be training teachers.

Final note (I promise). Hypocrisy is not a 21^st century invention, but I would assert none the less that we are living in The Hypocritical Century.

If you read the RealClear Investigations article I linked to above, you will find that it ends on this note –

Last year, during a Racial Justice in Early Mathematics Project webinar titled “Who Is Labeled Smart?” Martin addressed the backlash against San Francisco’s push for educational equity. He toned down his scathing critique of merit-based advanced education programs that he believes harm black and Latino students and made a surprising statement about his own son’s schooling.

“I’m guilty, I’m guilty,” Martin said, almost sheepishly. “My son is, quote unquote, in one of those tracks.”

Almost sheepishly?

I know what my Dad, who worked two jobs so his sons could avoid the local public school, would have said about that quote…..`Yea; Refusal, my ass, buddy.’

I have been writing this post in my head since I started this blog, as it’s prompted by an event that occurred just after I retired from my faculty position. I’m getting around to it now because a friend sent me a recent analysis of university course outlines that dovetails nicely into that story. So – a post in two parts.

I. Hiding Information at UWO

Shortly after I retired in a huff, I was in my office (my Chair did not kick me out of it upon retirement) and for a reason I don’t remember, went to the Departmental website with the intention of looking at a particular course outline for one of our courses. It had been standard practice in our Department to post on our site all of the course outlines for all the undergraduate courses that were being taught in each semester.

To my surprise, those course outlines were no longer posted. I asked a colleague about it, and was told that a directive had come down that Depts were to no longer post course outlines on their websites. Students could have access to them on the actual online sites for the courses in which they were registered, but those would require them to input their student credentials. In other words, the general public was to no longer have access to the syllabi of the courses taught at Western.

My colleague suggested they thought it was because not long ago some nutcase had gone into a classroom at U of Waterloo in which a gender studies course was being held, and stabbed two students and the professor. A terrible incident, to be sure, so the reasoning is that course outlines available to the public will allow crazy people to find out where and when courses of which they disapprove are being taught and then go and attack those in the classroom. Of course, this has not ever happened before or since, but you can’t be too careful, eh?

Thus, bowing to pressure from the Usual Faculty Suspects, the UWO Admin had ordered that course outlines with this information could not be made public. I was distressed – no, pissed off – by this, and went to see the Dept Chair to get confirmation that was what was going on. He confirmed it, and also mentioned in passing that some had suggested that we post course outlines that have the classrooms and meeting times deleted, but that was decided to be too much work.

I said ‘If I were still teaching I could prepare a version of my course outline with that information removed in six seconds’. He just looked at me.

Having recently looked again at the Dept website, I note that a sort of workaround has been found. One can view and download the syllabi from offerings of the course in previous years. Of course, if the instructor of the course changes, the previous year’s syllabi may not be very accurate, and a course that was not offered previously will have no info posted about it. I went to the websites of some other Depts in the Faculty and found they do the same: post previous year’s syllabi. I found some of those posted syllabi quite interesting, but will save any thoughts on them for another post.

[It is an irony lost on The Usual Faculty Suspects that said faculty, holding the safest and cushiest jobs on planet earth, are utterly consumed by worries about ‘safety’. Note, however, that students screaming into the faces of other students who are in a graduation ceremony – that is not a safety issue.]

I very much doubt any of you have much interest in reading university course outlines in any case. The thing is, if you wonder where the shit we all deal with daily comes from, you could learn a lot from reading the course outlines in gender studies and social work and sociology and anthropogy and even history and business these days.

I would suggest maybe you should, even if they are a year out of date. Which leads directly to part II below.

II. What is on university course outlines?

A friend sent me a copy of a paper titled ‘Closed Classrooms? An Analysis of College Syllabi on Contentious Issues’ .

In this analysis, the three authors make use of an interesting data base called OSP. This is, to quote them –

“…a unique database of college syllabi collected by the “Open Syllabus Project” (OSP). The OSP has amassed more than 27 million syllabi from around the world primarily by scraping them from university websites. They date as far back as 2008, though a majority are from the last ten years. Most of the data comes from universities in the United States, U.K., Canada, and Australia.”

[Note that ‘syllabus’ = ‘course outline’. The terms are interchangeable.]

  Using this data base, the analysis they undertake can best be explained by an example.

Take a contentious topic – for example, Racism and the American Criminal Justice System.

A highly influential book on this topic is Michelle Alexander’s The New Jim Crow: Mass Incarceration in the Age of Color Blindness, published in 2010.

It lays the responsibility for the undeniable increase in incarceration rates in the US at the feet of white racism. The three authors of the paper summarize the thesis of Alexander’s book thusly:

“Alexander’s book claimed that American history should be understood as a cyclical struggle between white supremacists and advocates for racial justice. While advocates for racial justice eventually succeed in overthrowing systems of racial oppression (as they did in the case of slavery and Jim Crow), white supremacists are resilient. Thus, after the successful passage of the Civil Rights Act and Voting Rights Act, Alexander says, they waged a new drug war that stirred up racialized fears of urban crime and social breakdown. In this way, Jim Crow—a system of racial oppression and social control—was reborn, just in a more sinister form.”

They then point out that the book elicited immediate and ongoing pushback from a number of other writers and scholars. This generated an entire literature of agreement and disagreement on the causes of mass incarceration. The authors then ask this question:

In what proportion of course syllabi in which the Alexander book appears do any of the books arguing against Alexander also appear? In other words, when faculty tell their students to read Alexander, with what frequency do they also tell the students to read her critics?

Here are the results from the OSP data base:

Table 1: Frequency with which Prominent Critics are Assigned with Alexander’s The New Jim Crow

__________________________________________________________________

Forman (2011) 149 of 4,309 (since 2012) 3.46%

Pfaff (2014) 3 of 3,758 (since 2015) 0.08%

Fortner (2015) 53 of 3,405 (since 2016) 1.56%

Pfaff (2017) 49 of 2,438 (since 2018) 2.01%

Forman (2017) 84 of 2,438 (since 2018) 3.45%

Sharkey (2018) 9 of 1,951 (since 2019) 0.46%

The names are the authors who have written books contesting Alexander’s analysis. The (since 201x) indicate that the study only looked at syllabi that were dated one year after the book indicated was published. They understand that a professor cannot be faulted for not putting a book on a reading list before it is actually published.

So, what we have here is that faculty who assign Alexander to be read almost never assign any of her critics.

But wait, there’s more.

One can also ask the opposite question. Among those reading lists that assign Alexander’s critics to be read, what percentage also assign Alexander?

Here’s the result of that:

Table 2: Frequency with which Alexander is Taught in Courses that Assign her Critics

____________________________________________________________

Forman (2011) #1* 82%

Pfaff (2014) #1 75%

Fortner (2015) #1 72%

Pfaff (2017) #1 72%

Forman (2017) #1 56%

Sharkey (2018) #1 26%

The two tables are kinda asymmetric, ain’t they?

The authors don’t stop there. They ask also: what other readings are assigned to students by faculty who have those students read Alexander? Answer: other leftist critics. The authors write:

“The top three titles assigned with Alexander are Angela Davis’s Are Prisons Obsolete?, Ta-Nehisi Coates’s Between the World and Me, and Michel Foucault’s Discipline and Punish (see Table 3). Others in the top ten include Paulo Freire’ Pedagogy of the Oppressed and Ruth Wilson Gilmore’s Golden Gulag. Broadly speaking, then, it seems that most of the authors assigned with Alexander are those that reinforce rather than complicate her critical perspective.”

Teaching critical thinking, yewbetcha.

The paper goes on like this, and if you would like a copy of all 67 pages of it, drop me a line, as I would be happy to send along the pdf. They look at other contentious topics like abortion policy and the Israel/Palestine conflict. The story with those is essentially the same. To echo something I wrote in part I above, if you asked the profs who assign Alexander but not her critics why that is so, most would answer that they are not willing to make their students feel unsafe by exposing them to these other clearly racist and evil writings.

One other element is of interest here, I think, and it is captured in this table:

Table 4: Assignments of Alexander’s The New Jim Crow by Discipline __________________________________________________________________

Discipline Count, Percent

__________________________________________________________________

History 1,052, 22.4%

Sociology 1,032, 21.9%

English Literature 636, 13.5%

Criminal Justice 606 12.9%

Political Science 523, 11.1%

Social Work 296, 6.3%

Law 205, 4.4%

Philosophy 134, 2.8%

Education, 126, 2.7%

Theology 96, 2.0%

Here, the number after the discipline is the number of syllabi in that discipline that include Alexander on the reading list, and the percentage indicates the percentage of course outlines which include Alexander’s book that are from courses in that discipline. That a book on the US criminal justice system would be assigned in some History and Sociology courses is no surprise. Same with Criminal Justice. But that it is assigned in more Eng Lit courses than Criminal Justice courses? Huh? And way more often in Eng Lit than in Law?

21^st century English profs have little interest in teaching literature or, god help us, writing. They are busy being front-line warriors for social justice.

As I noted in Part I, I looked through some (past) course outlines posted by other Depts in the UWO Faculty of Social Science, partly because I was curious about how they looked relative to what is in the OSP data. It was, as I said, quite interesting, and I will be writing more on that soon.

This is going to be a post without a strong message, prompted by a graph that I happened on while looking into something completely unrelated. The graph is below, and it shows changes in spending per pupil in the US state of Oregon, along with changes in the NAEP scores Oregon students earned in Reading and Math, all between 2013 and 2024. NAEP stands for National Assessment of Educational Progress, which is a set of tests given typically to grade 4 and 8 students in the US. NAEP is administered by a section of the US federal Department of Education. I am showing the Oregon results only because they are what I stumbled across. One can go to the website and see results for all other states too, but if you are so interested, I would suggest you go here instead.

Edunomicslab.org is a website of the McCourt School of Public Policy at Georgetown University, devoted to education finance, and it is the org that produced the graph below, using the stats provided by the Department of Education. It is simply more reader-friendly and organized than the US gov website.

Again, not to pick on Oregon, you can find a similar graph for each of the US states, my quick perusal said that Oregon is not atypical. Spending on education has increased at a rate much greater than inflation in Oregon, at the same time that NAEP scores have decreased.

Below is the same graph for the US as a whole, which certainly indicates that most US states look like Oregon, since the national graph does.

Finally, here is another graph from the same source, which compares staffing on a national basis to student numbers.

So, to note the obvious, staffing in US elementary schools has increased by close to 12% over this 10-year period, while the number of students has decreased by 2%. The graph does not indicate, and I could not find, any stats telling us how many of those added staff actually work in classrooms, rather than in the bureaucracy. I know what I would bet on in that regard, but cannot find corroboration, so far.

Anyway, the aggregate message here is clear. Over the covered period, US primary education has seen spending increase at a rate much greater than inflation, while also seeing staffing increase while student numbers fell (a primary reason for the spending increase, one must think) all resulting in worse student NAEP outcomes. It says nothing good about the current state of (largely publicly funded and administered) primary education in the US. I hasten to add, as does Edunomicslab, that there are US states in which the picture is better, in that student NAEP outcomes have gotten at least a bit better over this period. Their takeaway from that is that what states do with regard to education can make a difference.

I am not going to dive into state-by-state comparisons, I don’t live in the US. I am interested in how things look along these same dimensions in Canada, and here I run into the usual problem. No org in Canada does anything like the data gathering and organization of important public stats that you find in the US. StatsCan in the early years of my professional life had a reputation as one of the best such orgs on the planet. No more. Getting things from their website is difficult, they no longer produce user-friendly graphs and tables of such things, and on top of that, Canada does not have the same density of non-profit think tanks and such which take up the slack. The one exception to that latter  statement that I know about is The Fraser Institute, whose efforts are none the less dwarfed by what is done by edunomicslab, among others.

My complaining aside, I did what I could to find something comparable for Canada, and succeeded partially, with the results below.

A Fraser Institute Report from August of 2024 provides the following stats, among others:

– Per-student spending adjusted for inflation (price changes) increased by 5.1% nationally from 2012/13 to 2021/22.

–   Student enrolment across Canada increased by an average of 5.1% from 2012/13 to 2021/22. Three provinces saw a decrease in enrolment: Newfoundland & Labrador (5.8%), New Brunswick (1.8%), and Ontario (0.1%).

–  Compensation remains the largest and costliest aspect of education spending and has contributed the largest portion to the growth of total education spending in Canada.

The time period here is close to the one in the US graphs above, but it aggregates primary and secondary school stats together. I could not find any stats on staffing level changes in Canada.

Canada has a variety of provincial testing regimes for its students, but what I found readily available were results on the PISA scores in Canada. PISA is the Program for International Student Assessment (PISA), a triennial survey run by the Organization for Economic Cooperation and Development (OECD) that assesses 15-year-olds’ abilities in reading, mathematics, and science. Thus, it is given to secondary students every three years in most countries. I don’t know what percentage of students in every country take it, but get the impression that it is high. It has the advantage of making Canada US comparisons easy, and the last round of PISA test was done in 2022.

I found the following:

Average mathematics scores have dropped consistently in Canada, from an average of 532 in 2003 to an average of 497 in 2022.

PISA reading scores in Canada went from 520 in 2018 to 507 in 2022, just four years.

PISA science scores in Canada went from 528 in 2015 to 515 in 2022

The Canadian average scores in 2022 in Math and Science were notably better than for the US, the reading scores in the two countries were almost the same. It is worth mentioning that Canada’s average score over the three areas was the eighth highest of all countries in 2022, while the US’s average ranked it eighteenth.

So my take, based on this admittedly non-optimal data, is that the situation in the two countries is similar, but rather more stark in the US than in Canada. Neither country’s educational systems are performing well in producing student outcomes from the resources they use in the sense that outcomes have been getting worse while spending and staffing have increased. No doubt some of the declines in achievement can be attributed to school lockdowns during the Covid era, but they started before that, so that is not the entire story.

None of this points clearly at what to do about any of this, the data is too aggregate for that. However, if more is being spent per student to hire more personnel per student, and outcomes are getting worse, I don’t see any way to avoid the implication that the increased spending and personnel are being badly deployed.

….it’s not thinking.

I write these posts and most everything else in Word, like most of the human race, at least in North America. Every time I start a document, and then every time I start a new line in a document, a symbol pops up in Word on the left hand margin letting me know that Copilot is ready and waiting to make my writing better and easier.

I cannot get rid of this annoying symbol, because the Word program I am using is not mine, I am just renting it from Microsoft, and I have very few choices about how it works. I expect before I die that Word will no longer give me the option to ignore Copilot, just as my Microsoft Windows OS no longer gives me many real choices about how my laptop operates. We who use computers -as in, almost everyone – are all just dancing to Microsoft’s tune, nowadays.

Well, I want to report here on a blog post I just came across on a website called Mind Matters, written by Gary Smith, an economist at Pomona College. It has nothing to do with Econ, trust me, and if you want to skip the middleman (me) and read his long-ish post yourself you can do that here.

Assuming you are still with me, I’ll give you the basic story and then some related commentary based on other stuff I’ve come across on the interweb thingie regarding LLMs (Large Language Models).

Smith is reporting on what happened when he put a question to four AI LLMs – Deepseek, Copilot, Gemini Flash and OpenAIo1.

Here’s the initial question, which Smith asked in two parts –

Gary:  I’d like to discuss a new version of tic-tac-toe called 90-degree tic-tac-toe.

The four chatbots all respond enthusiastically with some variation on – ‘sounds interesting, tell me more’

Gary: It is still a 3×3 grid but the grid is rotated 90-degrees to the right.

Smith says he edited the LLMs responses down a bit, as they do go on and on, and here are the four responses.

OpenAI:  Interesting concept! By rotating the grid 90 degrees to the right, you’re visually changing the perspective of the classic 3×3 Tic-Tac-Toe board. The gameplay itself might remain the same, but the rotated perspective could create a novel experience for players.

DeepSeek: Ah, I see! So in 90-degree tic-tac-toe, the 3×3 grid is rotated 90 degrees to the right….This could be a fun twist because the rotated grid might disorient players initially, making it harder to spot winning opportunities or block opponents.

Copilot:  That sounds like a fun twist on the classic game! By rotating the grid 90 degrees to the right, it gives players a fresh perspective and challenges them to think differently about their moves.

Gemini:  This rotation doesn’t change the core concept of getting three of your symbols in a row, but it does significantly alter the strategy and how players approach the game.

Philosophers have an accepted definition of ‘bullshitting’. One is honest when one says what they believe to be true, one is a liar when one says things they believe are false, and one is bullshitting when one is unconcerned about the truth or falsity of what one says.

LLMs bullshit, because it is all they can do. They have no idea what is true. One writer on AI that I read said that what comes out of an LLM is simply the result of a billion incredibly fast coin tosses. Not exactly right, I suspect, but close enough.

As for LLMs learning as they go, Smith goes on from here and drags more BS out of the LLMs. He asks them if they think it would be better to rotate the grid right or left, and they have definite opinions about that, as well as on the question of which direction of rotation will make the game more difficult for expert players.

Smith notes some key features of how LLMs generate answers –

They write hundreds of words when a dozen would do, they are always positive and enthusiastic (lots of !s), reluctant to say ‘I don’t know’, and they lack common sense.

His bottom line: ‘If you know the answer, you don’t need to ask an LLM; if you don’t know the answer, you can’t trust an LLM.’

All this made me realize something. LLMs are algorithms that rather faithfully replicate the behaviour of all the bad students I taught over the years.

I never asked painfully obvious questions about tic-tac-toe during my years in the trenches, but I almost always asked a question or two that could be answered with a few simple lines by any student who had studied. (I never asked multiple choice questions in my last 20 or so years of teaching.) And, there were always some number of students who gave long and ludicrous answers to those questions, because their lack of effort in the course meant they had no idea there was a simple clear answer. They were left trying to guess at words and phrases that would match up well with the words in the question. They were, in a word, bullshitting, in hopes that something they wrote would get them some points, in spite of their utter incomprehension.

I don’t suppose any company is likely to come out with a general use LLM called ‘BadStudent’, though.

A further (last, I promise) thought. I bet if you put Smith’s rotated tic-tac-toe question to a hundred second-year undergrads, some positive number of them would answer with bullshit not too dissimilar to what the LLMs spit out. However, those students would be doing that because they were not confident that they weren’t missing something about the question, perhaps reinforced by a lack of confidence in their own thinking. The LLMs have no ‘reasons’ for bullshitting, it’s just what they are designed to do.

I sometimes write posts for this blog and then file them without posting, either thinking I will get back to them later and improve/shorten them, or just deciding that the post isn’t up to snuff. I have a looong one sitting in limbo right now on grade inflation, specifically in US universities. I think it’s an issue in Canadian universities (and secondary schools) also, but there is better data on it for the US.

Anyway, I may post that still-simmering article at some point, as it is a topic about which I care, but the Harvard Crimson, that august institution’s student paper, has published an article about grade inflation at Harvard that provides a nice introduction to the topic – and a shorter post.

Kurt Vonnegut is reported to have said ‘Most kids can’t afford to go to Harvard to be misinformed.’, and he wrote that long before Harvard’s most recent troubles – in 1987, in fact. It turns out that if a kid can afford it, he is at least all but assured of getting an A while being misinformed.

   or……….  

The Crimson article is from 2023, and titled ‘Harvard Report Shows 79% A-Range Grades Awarded in 2020-21, Sparking Faculty Discussion’

The Crimson has no paywall, so you can find the article and read it yourself, I expect, but here’s the graph that it opens with.

So, nearly 80% of the course grades awarded in 2020/1 were As or A minuses. Wow, is all I can say. I thought we gave marks that were too high at UWO back in the day, but I don’t think we ever approached 70+% As.

As is always the case, the percentage of As awarded varies by discipline. Here’s another graph from the Crimson report, which also goes further back into Harvard history.

SEAS is the School of Engineering and Applied Science, and I have to admit that I am surprised that Social Science profs were just as tough graders as were those in Science. (Economists, almost always housed in Social Science, are typically tough graders, but this is normally more than made up for by the Sociologists, Psychologists, Political Scientists, etc. Not at Harvard, apparently.) That the A & H faculty were the easiest graders should surprise no one. They would almost certainly look tough compared to the faculty in the Faculty of Education – if Harvard has one. And, of course, I am using ‘tough’ here ironically. The 60% A marks handed out in SEAS is anything but ‘tough’.

Yet the truly remarkable thing here is that the percentage of A grades awarded has more than doubled in 20 years in all faculties.

Now why do you think that happened?

I can tell you that there were administrators at Western in my day who attributed the ever-rising admission average to get into the Faculty of Social Science to the fact that ‘Western is just attracting better and better students.’ Secondary School grade inflation? – oh, no, certainly not.

The Crimson has some quotes from various administrators at Harvard about this, one of them from Dean of Undergraduate Education Amanda Claybaugh:

In the meeting, Claybaugh said that the “report establishes we have a problem — or rather, we have two: the intertwined problems of grade inflation and compression.”

By ‘grade compression’ academics mean the fact that all grades awarded are concentrated on a few possibilities, making it hard to distinguish really outstanding students from others. I find it encouraging – remarkable, even, given my own interactions with administrators – that the Dean thinks this is a problem. She is later quoted again:

“There is a sense that giving a wider range of grades would give students better information about their performance, and it would give us better information about where they are ranked against other students,” Claybaugh said in an interview after the meeting.

Claybaugh said that the evidence for the existence of grade inflation was less clear, as many student grades are well-deserved and faculty have increasingly focused on learning objectives.

Yea, that sounds more like an administrator – these grades are well-deserved. At least, ‘many’ of them are.

The next quote actually makes me grumpy. Well, grumpier…..

Nonetheless, she said it seems, as one faculty member put it, external “market forces” are influencing grading, particularly as faculty rely on positive course evaluations from students for professional advancement, she said in the interview.

Ok, I’m an economist, I am used to the fact that everyone wants to blame those awful ‘market forces’ when things go bad, but c’mon, unnamed faculty member. If faculty are giving high marks to get higher student evaluations there is nothing ‘market’ about that, it is an internal Harvard decision to use those (almost entirely uninformative, imho) student evaluations for ‘professional advancement’. You could stop doing that, you know. I’m talking to you, Harvard.

Here’ my favourite quote:

Claybaugh said she would defer to the full faculty to decide whether or not to implement concrete reforms to Harvard’s grading policies, but said she would be “interested in exploring” changes “that put more information on the transcript that put the grade in context.”

The Crimson then mentions some of the things that ‘the full faculty’ considered in their meeting. Someone from A&H (natch) suggested grading simply be abolished, but in the end nothing was decided. Shocking. A faculty meeting in which nothing was decided.

Here’s a fact, from my time in the trenches, that is never mentioned in the article. Giving grades that use the entire grading range, from F to A at Harvard, from 0 to 100 at Western, is hard. It is hard for faculty to give students low marks, partly because they will come to your office and bitch and moan and plead, and their parents may call you (yes, they do) and plead on their behalf, but also because it is not fun. It is not pleasant. Few faculty are sadists, few want to be known as The Grim Reaper of Harvard – or Western.

But the thing is, it is part of the job. That is, it is part of the job to give accurate information to students about their learning, and giving almost everyone an A is not doing that. It is also part of the job to inform the world outside academia how good a student is. Giving almost everyone an A is not doing that, either.

This is, if you ask me – and I know you didn’t, but it’s my damn blog – the fundamental reason why over the last twenty years marks in secondary and post-secondary institutions all over North America have gone up and up and up. Instructors (most of them) don’t want to do that part of their job. And, since they don’t have to, they don’t. Giving lots of As is waaaaay easier on everyone, so that is what happens.

It is the ultimate slogan of the 21st century. ‘Easy is best.’

Standards in general, and in academia in particular, are a keen concern of mine, and I will be writing about them frequently here. This post is about an open letter written by a faculty member in the Department of Computer Science at the University of Regina to the Head of the Department, a letter which he shared in a publication to which I subscribe.

The letter concerns an email sent to all CS students and faculty at UR on August 30, 2023, in which the following sentence appears – “In an effort to provide timely feedback on student work, some of our courses will be moving to a sample-based marking approach.”

The email goes on to explain what that means – that when a student turns in an assignment or test, not all of it will be marked. The parts to be marked will not be revealed until after the marked work has been returned to the students, and their grade will be determined only by that which is marked. So if a test consists of 6 problems, perhaps only three will be marked and feedback provided; the same three for all students.

The reason given for this is the increase in the size of CS classes, driven in turn by an increase in the number of CS majors, said to be nearly 1000 in the email.

The faculty member wrote his open letter to the Head of his Department (Computer Science) decrying this new grading approach, and explaining why he thought it would lead to a decline in academic standards.

I will first just record here that ‘sample-based marking’ as described, is in itself a reduction in academic standards. When I taught at University my assignments and tests were conceived as a whole, different parts of it designed to test different parts of the material, but also different abilities. Some questions could not be answered well without having the ability to write clearly and concisely about something complex, while other questions were designed to test one’s ability to deal with more formal logical or technical issues. To mark or provide feedback on only some aspects of the work is to ignore some part of what the course is about.

I understand well that the idea behind this is that, because the students don’t know up-front which parts of their work will be marked, they still have an incentive to work hard learning all of it. This does not change the fact they will get no feedback on some of their work, a primary point of marking. But in addition, anyone who knows students knows this will lead to a cottage industry in figuring out which parts of work any given instructor is likely to mark, which is not in any way part of what higher education is supposed to teach students.

This policy is, in the end, a further piece of evidence as to what University administrators’ goal is. Get as many students through to a degree as possible, at as low a cost as possible. So far as I can tell, their political masters in Canada are perfectly in agreement with this goal.

This is why sample-based marking is being implemented, rather than the solution suggested by the CS faculty member who objected to it; hiring more faculty to accommodate the growing number of students. Faculty are expensive. And, note that the letter did not indicate that CS students would be seeing a discount on their tuition bill to accompany this sample-based marking initiative.

Imagine a McDonald’s franchise-holder, or local restauranteur, who found themselves with a (delightful) increase in patronage, and responded by filling only part of all food orders, rather than hiring more workers, while charging for everything ordered.

One final note. The original email laying out how this scheme is envisioned working at UR also says that the parts of any student work that are not marked will none the less have solutions posted for the non-marked parts, or they will be gone over in class so as to provide students with the correct solutions. So, there’s your all-around feedback, eh?

Right. In a university atmosphere in which students feel free – indeed, are encouraged – to argue for higher marks for most any reason they can think up, this will open up a whole new area of student appeals. To wit: “ I got the parts of the exam you did not mark nearly perfect, according to your own solution key, so I deserve much more than the 63 I received, which is based solely on the parts you did mark.”…followed by the ever-popular ‘This isn’t fair.’