Authorship and Quality in the Age of AI
This essay looks at how society, our institutions, and individual attitudes should evolve to address the increasing diffusion of AI. As a cautious optimist, I want to engage in a critical discussion that will allow us to capture the best of its promise while avoiding the worst.
Trends
What prompts this question? The diffusion of AI throughout the world may deeply change our world β our relationships and the dynamics therein. As a professional software developer, I wonder whether AI will obviate my whole profession. Second is the related, but broader, concern: of all the activities that were performed (exclusively) by humans, which will still be performed by humans? This question is both about the loss of economic value experienced by supplanted professionals, and about the almost-spiritual loss of meaning and purpose.
How we approach these questions is grounded in how we project AI will diffuse throughout society and the economy; and the relative benefits and costs it carries. Profoundly transformative technology always incite a broad range of reactions. On the edges are utopian and dystopian narratives that hold AI to be a near-supernatural force. But such prophetic notions do not generally translate into critical discussions. Critical discussions are what actually allow us to capture the good and avoid the bad. (This relates to Venkatesh Rao's notion of epic times as inescapable flows in time; the dystopian and utopian narratives become a kind of escape, which may be useful in its allegorical value, but must become more critical if it is to helpfully serve the trajectory of AI.)
Concerns
Concerns are where that critical discussion begins. Each objection points to a place where our technology, institutions, and humanity are buckling; a dilemma; a problem, in the Popperian sense, to be resolved.
Nothing interesting is produced
The first criticism is that there is nothing interesting produced by AI. Not in the sense that nothing interesting is produced β which is trivially false β but rather that what is produced is simply a reflection of what was in the AI's training regime, so nothing truly novel is produced.
I think at some level this is how humans work, and arguably how most knowledge grows (it's mostly incremental, except the rare quantum leap). Of course the volume of data AI trains on is vast; but that is a matter of degree and not of kind. Synthesis is genuinely useful, and arguably a lot of what humans do.
And even if we hold that nothing new is being generated, the act of having explored a part of the extremely large manifold β through the human's understanding, intuition and beliefs β is still valuable. This is what expositors and science popularisers do; the value is in the selection and the route taken through the material, not in the novelty of the material itself.
This criticism is often really about something else: the injustice whereby human art and output is used to train these models, and these humans have not been compensated for their work. That should be addressed directly, via the existing protections that owners of content have. No one should support AI developed via illegal means - i.e. by stealing books, or by not attributing them. But some of this is fair use, and arguably rightfully so. Where existing property rights do not cover these cases, they must evolve; and laws must evolve with them.
The human contributed nothing
The second concern is that all the interestingness is in the AI, and that the human contributed nothing.
This criticism arises from a confusion about the production process. Thinking and writing are not monolithic processes, but rather a collection of co-operating ones. Writing breaks down into: grand ideation, finding supporting arguments, finding evidence for those arguments, constructing a holistic flow, drafting, transitions, word choice. These are separate skills and competencies. And finally, the amount of time you can spend on each will translate into better output.
The grand idea is the core of the work. So if one uses AI for that, it is truly worth reflecting on in what sense the human authored the content. However, if AI was used for an auxiliary task β to develop an argument, or to find examples β it is not exactly the case that the human did not author the text; but nor did they fully author it either. It is one thing to express the blub version of an idea, and quite another to fully develop it. And we already have ways of generalising this: primary and secondary researchers on a paper. Where we land on the distinction will depend on the domain, but in general we will need a richer vocabulary to deal with it.
There is also a case on the other side. If AI helps us better express ourselves β to overcome some of the gaps in our skills β this is overall a good thing. Someone who has something worth saying and cannot yet say it well is not served by a norm that treats assistance as contamination.
AI Fatigue and Quality
There is a notion of AI fatigue- and I think this is related to another failure mode of AI usage - slop. Slop is fundamentally a quality problem. I fundamentally view both of these as quality problems. And ultimately, the human who produces the content -minimally from the prompt; but typically from a more engaged discussion - is responsible for the quality.
Quality has always been a fuzzy notion - hard to enumerate as an attribute list; but in a local context, itβs almost common sense. A local context could be a whatsapp group of friends, where theyβll generally agree on whatβs good quality food. The same extends to, for example, communication in a corporate setting. There is a notion of what acceptable or even high quality communication looks like. And our notion of quality, as with all norms, evolves over time. This is how we need to think about AI assisted content. We need to update our holistic notion of quality - and what is acceptable to share. Perhaps, we extend our notion of quality to include something like the right framing. Because a failure mode of some AI output is correct but inaccessible. I intended to ask for ELI5 and it gave me a PhD thesis. The output is correct and true but misses the mark on the framing. The human is the final adjudicator on quality. If youβre simply sharing low quality output, you, the human, are using the tool, AI, poorly.
I think an important but adjacent, psychological force here is how we think about human authorship in near-metaphysical terms. That there is something unique about humans and our product, and that this unique humanness must be preserved. I have sympathy for that - as I am subject to the same meaning crisis it imposes.
There is also the feeling that by using AI something is lost β the small edit, the word choice, the transition you didn't make. Every time you iterate manually, you engage it in a deeper way. And there is no substitute for this. And indeed, perhaps the way through this is to (atleast) occasionally do an AI fast- lest your brain atrophy too much.
Domains and norms
How much any of this matters comes down to how much we value AI output in different endeavours. In some areas β creative endeavours, music, art β humans may prefer human output; i.e. for things grounded in preference at the outset. But these positions are untenable in mathematics, engineering and pharmacology. We will not reject theorems, proofs, systems or compounds because AI produced them. We may reject visual art and music. But even for subjective and aesthetic domains there is a kind of objectivity to music β whether it sounds good. There is a measure of goodness that is inherent in the music, and it is independent of its authorship. So I suspect that, unless users explicitly choose to filter for a label such as human-produced, the quality of AI output will likely be equally preferable. Some users will choose to filter and some won't; and like a classical liberal, I would not take this as a moral choice, but as different operators making different choices.
There is also the question of how social norms should evolve. In some contexts, the exchange of ideas and prose is all that matters, independent of authorship. In professional mathematics, engineering, and pharmaceuticals, whether I wrote something or AI wrote it will not matter, because the route to the solution does not matter. The product is evaluated in an "objective" environment β it can be grounded in some part of reality.
So we will likely have a rich ecosystem of norms. And depending on the broader context, the same activity may be acceptably performed with varying degrees of assistance from AI. Professional mathematics may be solely concerned with results β but even there the notion of attribution may change. And in the context of mathematics tests and examinations, access to calculators, let alone AI, may be restricted. The activity is the same; what the institution is measuring is not.
But what of writing, which lacks this objective measure? There too we will find different communities with varying degrees of AI acceptance. In many, getting your point and perspective across will be paramount β though these will still have to deal with the quality problem. In others, such as journalism, usage may be acceptable but require proper attribution.
Mechanisms
Norms only bind if something makes them checkable. So it is worth asking what the mechanisms actually are, and what each of them can and cannot do.
Attribution is the first, and it needs to become finer-grained. Self-attribution already carries weight: if someone says they authored a book and it turns out she had it ghost written, this is a problem today. We will need some generalisation of property rights, attribution and provenance. Attribution must cover what parts were driven by the human and what parts by the machine β the idea, the argument, the evidence, the prose. Not as a binary label on the finished work, but as a description of the division of labour. We do this already for research papers; we can generalise it.
The second is identity. Here we have one mechanism, which is the Facebook model - you attach the human identity to the machine identity, or to the generated content. And such a medium will have inherent rate limits; generally, behaviour that is spammy in nature will have to be checked. A human identity is a scarce thing: you get one, and it accrues a history. So binding output to it imposes a cost on volume that a machine identity does not carry on its own. This tells you nothing about how something was made, only who stands behind it. But in many cases that is the more useful question.
The third is provenance. Again, we have ways of tracking this. Maybe users self-attribute, or you have a deep bibliography, or you show the record of how you engaged with the work. But I think at a fundamental level we will have to move away from this notion of human-authored content, because it will be very hard to distinguish the output of a hardworking adversary who wants to fool you. Provenance can show that a tool was used; it can never show that one wasn't.
Which leaves verification, and the quality side. Ultimately we will need to become more peer-to-peer. Again, this is not a new mechanism β peer review, code review, reputation among people who actually know each other's work. But these were built in a world where producing something plausible was expensive. They will have to carry far more weight now that it isn't.
Deception, Enshitification, and Breakout
There are a few failure modes, which are genuine - but in my opinion, they donβt bear on how we use AI in (somewhat) trustworthy environments.
The first is all the leverage that a deceptive human being gets to propagate deceit β spamming, phishing, misinformation campaigns. This is something we have thought a lot about, and we have reasonably simple ways of addressing it. We already have many legal protections against fraud and spam. These will need to be updated; but the basic institutional/social knowledge is there.
The second is the AI breakout scenario; which is unique in that we have no prior institutional machinery for it at all; it is also the one about which I have the least useful to say here.
A third criticism is that AI is speeding up the enshittification of the internet. Again, this is a problem of quality β and the honest question is how we deal with low quality now. The answer is largely provenance and trust: we route around bad sources, and we rely on reputations built up over time. Those are the same mechanisms as above, which is why they matter. The difficulty is that they were calibrated to a world where producing plausible-looking content was itself a filter.
The loss of human ability
Lastly is the concern about the loss of human ability β to think, to review, to judge β and the degradation of the human institutions built around human production.
There is a feeling amongst some that AI is inherently destructive. My take is that, like all tools, it will enhance some of our natural abilities, and in doing so will let the underlying capability soften. This is the standard trade-off of tool use: you lose some acuity in the specific skill, and you gain in capability or in time. What is different here is the breadth of what is being offloaded.
What are these natural capabilities? At the broadest level, all kinds of "active" thinking β when you consume some medium and must digest it. Books and written text are the typical medium. But this is a spectrum and not a binary. At the other end of the spectrum is short-form video; and yet even there, a short, incisive video can make you mentally spin. So it is less about individual messages than about the broader practice of how we engage with a medium β and that observation applies to AI as much as to video.
Perhaps, then, it is worth asking what we will relinquish control over. At an implicit level, anything we do not understand is already beyond our control. Hence mission- and safety-critical systems must be understood: even if the machines make all the actual code and config changes, the human must sign off. This requirement will hold in safety-critical systems, though perhaps for only a small fraction of people.
But what about when our livelihood or our lives are not at stake? Should we give up thinking? No! Examination is itself a good. Read Socrates (Plato) if you have doubts. But everything I have argued only holds if minimally we have the meta-thinking to know when we are actively thinking and when we are offloading (rather than compulsively offloading); and when we need to interject. The norms, the attribution, the peer-to-peer verification all assume a population that can still tell good work from bad. If that erodes, the mechanisms erode with it. One answer is deliberate practice: periodically working without the tool, sampling the scenarios, in the way one does chaos testing on a system.
Closing
To simply say that we will stop using AI at large is not tenable. But in limited capacities, where trust can be strong, we may create spaces which are free of AI β so that we can focus on a limited activity where, for a multitude of reasons, that is useful. Examinations are one such space. There will be others.