Skip to content
An image of Ludd, the fictitious leader of the Luddite protests.

AI policies and informed consent

Published on May 6, 2026

There sure is a lot of noise out there about so-called “artificial intelligence.” (I won’t bother with the scare quotes through the whole post, but it’s worth noting: “AI” is a term that’s been in use, mostly for marketing, since the 1950s (here is the term’s original usage), and as such, “resists definition because it is continually reappropriated by people to mean different things.”) I’m a little sorry to add to it, but there are many people out there being pressured to use AI products, who I think should be informed of the risks and allowed to decide whether to use them or not. My primary focus, here, will be “generative AI,” the chatbots that use huge corpora of text and images to generate outputs that appear novel, and my approach will be largely practical; I’ll save ethical and environmental concerns for the last section, and mostly link to better sources than this post. I also limit myself to work applications — so I removed a section about chatbots’ effects on mental health, despite my belief that their sycophancy could potentially cause problems at work.

Contents of this post

To save you some scrolling in case you’re here for something specific.

Why there’s no incentive for management to require AI usage:

Why people deserve informed consent before using these products:

An additional AI policy suggestion:

And last, but in a just world, not least:

The benefits are usually negligible, but often nonexistent and sometimes even negative

Let’s cover this first, since it’s the part management is most likely to care about—and because it’s counter to everything the marketers are telling us, I have citations—AI does not improve individual performance in any meaningful way. Occasionally, for some tasks, there are small improvements, but when weighed against the near certainty that these products will increase in price, the potential for catastrophic mistakes, and the likelihood of damage to the individuals using these products (which is the bulk of this post), I argue that management has no incentive to force AI use on workers.

Now

According to MIT’s The State of AI in Business in 2025 (emphasis mine), “Despite $30-40 billion in enterprise investment into GenAI, this report uncovers a surprising result in that 95% of organizations are getting zero return.”

An uncomfortable number of publications about worker productivity focus on qualitative assessments:

  • From the Wall Street Journal, “Two-thirds of nonmanagement staffers said they saved less than two hours a week or no time at all with AI. More than 40% of executives, in contrast, said the technology saved them more than eight hours of work a week.”
  • Also WSJ, summarizing a different study: “A new report from the business-software company Workday goes so far as to call frustrations with the technology an ‘AI tax’ on productivity. Though 85% of the roughly 1,600 employees it surveyed reported saving one to seven hours a week by using AI, much of the time was offset by having to correct errors and rework AI-generated content.”
  • From upwork, “Nearly half (47%) of employees using AI say they have no idea how to achieve the productivity gains their employers expect, and 77% say these tools have actually decreased their productivity and added to their workload.” (The same executive summary says 71% of workers are burnt out, which… yeah, that sounds right.)

On the quantitative front:

The future

I can almost hear the AI fans shouting, “But that’s 2025! We’re looking at the future!” And, you know what? That’s fair. Let’s talk about the future.

A great deal of the hype is based on a simple bet: that these products, which are currently pretty mid, will improve over time. But it’s a bad bet. Last September researchers at OpenAI proved that there is no way to create a large language model that doesn’t hallucinate. (I linked you to a summary, there, but if you like mathematical proofs, here is the original OpenAI article.) These are inherently probabilistic technologies, which means their output cannot be constrained to what we commonly refer to as “the truth.” No amount of engineering, scaling, or guardrails can prevent these machines from lying to us. We’re also running out of scaling opportunities; the internet, though large, is finite. A second bet, that the LLMs can feed each other, leading to a “superintelligence,” has also been shot down by mathematical proof. There’s a convergence point beyond which they don’t improve and actually get significantly worse. (I’ll be honest: mathematical proofs aren’t my thing. This post is not at all restrained in its rhetoric, but it does a good job explaining what’s going on in the paper.)

If a singularity is coming, it won’t be from this branch of technology.

So, OK, genAI won’t really get better over time, unless we pour a whole lot more human-generated data into it; and even then, there are limits. The companies can make little refinements, but we’re stuck with hallucinations and they need to stop slurping up everything, lest their products degrade further in quality.

Conclusion: there is no benefit to the enterprise in pushing individual contributors to use AI.

(Question: do I need a follow-up post about how silly the idea of replacing workers with AI is? Klarna may be one of the 55% of businesses that admit they shouldn’t have cut workforce in place of AI — and that’s just the ones willing to admit to being wrong. Even for replacing freelancers, AI’s a no-go.)

Harms to individual users

I’d like to believe management would care equally about this, and to be fair, a lot of managers will: good managers want their employees to be successful, even after they’ve moved on. Above a certain level, though, it seems like individuals stop mattering. So at this point, I’m talking mostly to line managers and to individuals who need to decide whether to implement genAI products in their own work.

Not all of these harms are equally applicable to everybody, but I believe they should all be shared with anyone who is thinking of using these products for their work.

AI usage damages critical thinking and encourages cognitive surrender

AI products encourage dependence and decrease self-confidence

  • This one’s interesting because this person is pro-AI. But I found this quote telling: “If you think about it, that 30-second wait for AI responses can be seen as a variable ratio schedule — Random rewards delivered at unpredictable intervals — the same psychological pattern that makes slot machines, social media, and mobile games addictive.” – Vibe Coding Is Creating Braindead Coders, by Namanyay Goel, Blog post, September 2025
  • This one will hurt to read, a bit, if you’re already using these products, and I’m sorry. It’s a good warning to the folks who aren’t, yet, and it comes with citations. Generative AI runs on gambling addiction — just one more prompt, bro!, by David Gerard, June 2025.
  • “The study included 1,923 online adult participants from the United States and Canada who were told to use commercially available AI programs to complete 10 simulated work tasks, such as developing plans with incomplete or evolving information, interpreting ambiguous data, and articulating reasoning for strategic decisions.

    “After the tasks, 58% of the participants agreed that AI ‘did most of the thinking’ to complete the work, especially in activities related to planning or sequencing. Those participants also reported reduced confidence in their own independent reasoning, lesser perceived ownership of ideas, and making trade-offs between task speed and depth of thought.” (quote from this summary) Furthermore, “Greater prompt dependence and lower override frequency were associated with reduced self-reported confidence in independent reasoning” – Generative Artificial Intelligence Reliance and Executive Function Attenuation: Behavioral Evidence of Cognitive Offload in High-Use Adults, by Sarah Baldeo in Technology, Mind, and Behavior, 2026.

AI usage hinders skill formation and learning on novel tasks


AI usage erodes expertise and damages performance even on familiar tasks

  • Using AI assistance hurts performance on both arithmetic and reading tasks as soon as the AI is removed. – AI Assistance Reduces Persistence and Hurts Independent Performance (website, arXiv preprint), by Liu et. al., arXiv preprint, April 2026.
  • “[O]ver the course of six months, clinicians became over-reliant on AI recommendations and became themselves ‘less motivated, less focused, and less responsible when making cognitive decisions without AI assistance.'” – New Study Suggests Using AI Made Doctors Less Skilled at Spotting Cancer, by Miranda Jeyaretnam, Time, August 2025 (and here is the study it references)


AI usage amplifies individuals’ biases and flaws in judgment, in ways that are invisible to them

  • “AI systems can exhibit biased judgements in domains ranging from perception to emotion. Here, in a series of experiments (n = 1,401 participants), we reveal a feedback loop where human–AI interactions alter processes underlying human perceptual, emotional and social judgements, subsequently amplifying biases in humans. This amplification is significantly greater than that observed in interactions between humans, due to both the tendency of AI systems to amplify biases and the way humans perceive AI systems. Participants are often unaware of the extent of the AI’s influence, rendering them more susceptible to it.” – How human–AI feedback loops alter human perceptual, emotional and social judgements, by Moshe Glickman and Tali Sharot, Nature Human Behaviour, volume 9, pages 345–359 (2025).
  • Autocomplete affects the answers people give, and most concerningly, “the people in the study didn’t tend to think the AI autocomplete suggestions were biased or to notice that they had changed their own thinking on an issue in the course of the study.” – AI autocomplete doesn’t just change how you write. It changes how you think, by Claire Cameron, Scientific American, March 2026, extending a 2023 study by Jakesh et. al.
  • The bullets above are a problem because these models are demonstrably biased.
    • “Here, we advance studies of generative language model bias by considering a broader set of natural use cases via open-ended prompting… In this setting, we find that across 500,000 observations, generated outputs from the base models of five publicly available language models (ChatGPT 3.5, ChatGPT 4, Claude 2.0, Llama 2, and PaLM 2) are more likely to omit characters with minoritized race, gender, and/or sexual orientation identities compared to reported levels in the U.S. Census, or relegate them to subordinated roles as opposed to dominant ones. We also document patterns of stereotyping across language model–generated outputs with the potential to disproportionately affect minoritized individuals.” – Intersectional biases in narratives produced by open-ended prompting of generative language models, by Shieh et. al., Nature Communication, 2026.
    • “Essays attributed to Black students received more praise and encouragement, sometimes emphasizing leadership or power. … Essays labeled as written by Hispanic students or English learners were more likely to trigger corrections about grammar and ‘proper’ English. When the student was identified as white, the feedback more often focused on argument structure, evidence and clarity — the kinds of comments that can push writers to strengthen their ideas. The AI models addressed female students more affectionately and used more first-person pronouns.” AI gives more praise, less criticism to Black students: Identical essays get different feedback in Stanford study, by Jill Barshay, April 2026; original article preprint.
    • “Both large language models significantly underestimated disability in a population of people, and linguistic analysis showed that descriptions of people, patients, and athletes with a disability were generated as having significantly fewer favorable qualities and significantly more limitations than people without a disability in both ChatGPT and Gemini.” – Disability Ethics and Education in the Age of Artificial Intelligence: Identifying Ability Bias in ChatGPT and Gemini, by Urbina et. al., Archives of Physical Medicine and Rehabilitation, January 2025.

Sending uncorrected “workslop” to someone else is always inappropriate

Executive function theft (digression: this is such a great post; it really clarified something I’d observed but hadn’t previously had a cohesive description for) has been a problem in workplaces since time immemorial, but we’re already seeing genAI products making it worse, with huge productivity costs to workers receiving “workslop” products. (“Workslop” definition: “AI generated work content that masquerades as good work, but lacks the substance to meaningfully advance a given task.”) As Harvard Business Review summarizes the issue: “When coworkers receive workslop, they are often required to
take on the burden of decoding the content, inferring missed or false context. A cascade of effortful and complex decision-making processes may follow, including rework and uncomfortable exchanges with colleagues.”

I have to wonder if this is some of the time savings reported by executives: they do a little prompting, send off the output, and one of their employees is stuck doing the difficult part. Anyway…

Besides the obvious, “let people opt out,” a second concrete proposal I made for my workplace’s AI Strategy was “AI products should not be used to generate outputs that can’t be tested and corrected by the team using the product.” (Even that is too broad; they should be tested and corrected by the person using the product.)

  • An example of an acceptable use: someone needs a ton of data parsed, and Excel is choking on the volume; they use a genAI product, which creates a pandas script to parse the data for them. The code itself isn’t the point and likely won’t be used beyond that session. They spot check the output of the script with smaller portions of their Excel file, to make sure it isn’t totally off base, before using it to make decisions.
  • An example of an unacceptable use: someone uses a genAI product to create an application and asks the technology group to deploy it. The technology group, if they’re doing due diligence, now has to read through and probably correct that code. (So this person has just generated a ton of work for another team, when instead they should have worked with the experts on that team, who likely would have been able to code something simpler, or at least something they understand well enough to deploy safely.)
  • Acceptable: someone uses a genAI product to generate a bulleted list of some sort, within their own knowledge domain. They go through, add necessary context, remove duplicates and fix weird phrasing. When it’s clean, they send it off to their task force, to add to a final report.
  • Unacceptable: someone uses a genAI product to generate a bulleted list about something they don’t fully understand. They send it off to their task force uncorrected, and someone else will now have to do the work to fix it.

AI outputs can be hard to parse

I genuinely don’t know how universal this is. I can find a lot of neurodivergent folks talking about the difficulties they have with AI output, so I know at least a subset of people have a really difficult time with this. (Of course, others are capitalizing on it; but even there, at least some of what’s happening is full rewrites.)

I will say, for myself, that I do not have a diagnosis of autism or ADHD, and I slide right off AI-generated text. Have you ever been reading a book, or you thought you were, and you realize you have no idea what the last page said? Reading AI writing is always like that for me; if it’s something I’m going to be in any way responsible for, I consistently have to close the file and write my own, rather than try to edit it.

I’ll link you to some other folks’ reports of their experiences with it, as well:

AI outputs must be understood to be corrected (and they always need to be corrected)

  • “Before we can safely change code, we first need to understand it – understand what it does, and also oftentimes why it does it the way it does. In that sense, this is nothing new.

    “What is new is the scale of the problem being created as lightning-speed code generators spew reams of unread code into millions of projects.

    “Teams that care about quality will take the time to review and understand (and more often than not, rework) LLM-generated code before it makes it into the repo. This slows things down, to the extent that any time saved using the LLM coding assistant is often canceled out by the downstream effort.” – Comprehension Debt: The Ticking Time Bomb of LLM-Generated Code, by Jason Gorman, September 2025.
  • Companies are drowning in code, without enough people to read, understand, and review it. The big bang: a.i. has created a code overload, by Mike Isaac and Erin Griffith, April 2026.

Ethics and the environment

I pushed this part to the bottom, despite it being so much a part of my original decision to become an AI vegetarian, because every organization creating an AI policy (unless that policy can be summarized as “don’t”) will already have determined that these concerns are secondary to their other goals. Organizations might give lip service to “using AI ethically,” but the people whose opinions I most respect on this topic generally argue that, for people empowered to make organizational decisions, there is no ethical use of generative AI products. (Well, OK, a small subset of the people I most respect are arguing that, perhaps, if you use small models on standard consumer hardware, maybe it’s not so bad — again, they’re arguing from an ethical and environmental standpoint, and many of the harms outlined above still apply. I actually have a use for Whisper, though you’ll note I spent all this time writing this post, rather than deploying it on my Mac.)

At a minimum, I think that if you have the money, you should spend the $20 (unless you can find a free screening near you) and just under 2 hours to watch Ghost in the Machine. It does a pretty good job describing the eugenicist aspects of AI, and, crucially, it spends time with data workers in the Global South whose work makes these products possible. It’s less gut-wrenching than reading some of their first-hand accounts, but it’s enough that you can no longer pretend to think these products aren’t hurting anyone, just by existing. If you’re more interested in the growing US data underclass, Karen Hao just released a 16 minute documentary video, which is free on YouTube.

Some other folks who have done a better job covering the social and environmental costs than I ever could:

I want to shout out to people, books, and articles that helped organize and clarify my thinking, even if I didn’t cite them explicitly here

And why not share my TBR (to-be-read) list, while we’re here

A final note:

I’m ready to hit “publish,” but at the same time, this does not quite feel finished. I reserve the right to come back and add citations (and you should feel comfortable sending me any big ones I missed, of course!) and, if needed, entire sections. I apologize if that messes up your RSS reader or is otherwise frustrating for you.

Published insocial justicetechnology

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *