7 min read
Interpreting Opinions on AI

Introduction

The release (and subsequent un-release) of Anthropic’s Fable 5 model has led to a burst of opinion-sharing on the usefulness of AI. Online AI opinions are largely decided by two factors:

  • Business interest
  • Subjective personal experiences

As a result, the discussion is extremely muddy and if we want to do our due diligence and impartially understand the utility of AI for our own personal goals, we need a robust lens through which to understand it:

Software Engineering Is About Describing Things

The first axiom we must establish to understand how useful LLMs are for software engineering, is what software engineering is. I put it to you that software engineering, like all forms of engineering, is just the process of describing something (in this case software).

Code describes behaviour. Tests describe desired outcomes from that behaviour. Architecture and infrastructure describe how the end user accesses that behaviour.

LLMs Can Describe Things For Us (When We Don’t Want To)

LLMs can, however, describe things for us when we don’t want to. Rather than considering prompting in and of itself to be some sort of “new form” of programming language, we can instead just say that LLMs can take the reins on describing things at computer granularity for us based on a brief high-level description.

We’ll be using this framework to break down what I see as the main groups that are vocal on this discussion at the moment:

  • Those limited by knowledge & understanding
  • Those limited by typing speed
  • Those that don’t know what camp they’re in
  • Those who just aren’t limited

Let’s walk through these and look at the way they interact in the conversation.

Those Limited By Knowledge & Understanding

This is the prevalent point of view. There are masses of people who have no practical experience or understanding of software engineering, what it involves, or what’s possible, who are now able to participate in software engineering discussion.

The reality is, at the risk of sounding harsh, this is very similar to spending a lifetime not learning law, making friends with a lawyer, and then claiming to have relevant legal opinions of your own.

This group is responsible for the majority of “rage bait” / “engagement bait” nonsense posts on LinkedIn because they cannot understand their limitations and the drawbacks of relying on third-party technology for their abilities. It’s incredibly easy to conflate a visually pleasant website with a source of genuinely novel value if you’ve never made anything of genuinely novel value before.

This group is the most vocal about improvements in LLM quality because every single thing they do is limited by the model. There is no recourse for them if the model can’t solve their particular problem, they must just wait for a model that can. Fable 5 is a very capable model, so this group has been particularly vocal about it.

Those Limited By Typing Speed

Software engineers who have been active in the field for several years approach this from a different angle. They don’t need the LLMs to describe solutions for them on the basis that they don’t understand the solution, they’re in it for the velocity improvements.

In general, “it’s really hard to do X as fast as the X machine” holds true, and this is no different: it’s really hard to type as fast as the typing machine.

Software engineers limited by typing speed prompt differently from ordinary people and see LLMs as a tool of great utility, but that’s it. If you can describe the solution to the LLM well enough for it to catch your drift and not f it up, then you get a pretty hefty speed improvement over your normal workflow. This requires you to be able to conceptualise a solution and then articulate it accurately to the LLM.

Rather than:

Write a website for me that does X

Change Y it’s not working

I don’t like Z fix it

A more experienced engineer may prompt more closely to:

We’re going to implement a website that does X using a React + Vite front-end served by a Python + FastAPI backend. The database will be managed by SQLAlchemy models with Alembic responsible for migrations on a Postgres database. Local development cluster will use Docker Compose, deployment will use Vercel.

We’ll organise the code base like so: …

We’ll test / verify progress with: …

Before starting, clarify any ambiguous design decisions with me

The logs indicate that the backend route POST ... failed with inputs ... due to a division by 0, implement some preventative validation

You get the picture. The descriptiveness of the prompts is orders of magnitude greater, and if the model ever fails to produce the desired results the developer is able to increase the granularity until it’s no longer worth it, at which point they may decide to jump in and write the code themselves if they believe that would be faster.

The nature of the approach is fundamentally different. The software engineer is trying to save themselves time while still implementing a solution that they know to be trustworthy and robust, rather than using the models as the sole source of their abilities and being unable to implement their grand designs until Anthropic release their next version.

This group is receptive to, but not overly vocal about, model improvements because the effect is relatively small. A better model might require a sentence less per prompt to understand the vision, and that’s cool, but it’s not exactly a massive speedup.

Those Who Don’t Know What Camp They’re In

There are many software engineers, primarily junior engineers, that simply don’t know what camp they’re in. Realistically, a little of both. They know they like the speedup of LLMs, but they also know that LLMs sometimes do things better than them and that’s a scary situation to be in when you’ve just spent the last four years in full time education trying to learn these things for yourself. They don’t have the wealth of experience that more senior developers do, so they can’t draw on that experience and deep domain understanding to guide the LLM as a pure typing speedup.

If you’re in this group it may feel like you’re unable to get above the clouds and see that LLMs aren’t destroying your potential completely, you just need to keep learning until you can start using them as an extension not a requirement.

Those Who Just Aren’t Limited

There’s a small number of developers that are, frankly, just not limited. They sit down for the day and they just start programming. They program so quickly, and make such instantaneously good decisions, that they can tear through an unlimited number of features without any real pain points. It would take them just as long to describe the solution to an LLM as it would to just write it themselves.

This group is naturally quiet because there just aren’t many of them but it’s worth knowing that they’re out there and that it’s definitely possible to be this good at development, and striving to be this good is exactly how you stay in the group for whom LLMs are a tangible benefit, rather than a necessary requirement or limitation.

TL;DR

LLMs are used by different people with different skill levels in different ways (obviously) and you shouldn’t base your opinions on how someone from a different experience level thinks. Next time you read some rage bait nonsense ask yourself: how skilled was this person prior to using LLMs? Is what they’re showing me their own vision or their LLMs vision?

Use the models, understand for yourself, keep learning, and keep building!