Software Update: Therapist v.o1

Looking into how we implement reasoning in conversation and software.

The Teardown

Thursday :: September 19th, 2024 :: Approx. 15 min read

👋 Hi, this is Chris with another issue of The Teardown. In every issue, I cover how we interact with technology that enables our day-to-day lives.

If you’d like to get emails like this in your inbox every week, subscribe below.

And, if you enjoy today’s thinking, let me know by tapping the Like (❤️) icon in email or on the web!

---

Quick intro from me:

Software Brain Power

My last post discussed the use of technology in finding and receiving therapy. I wondered whether ChatGPT could help resolve imposter syndrome.

The simple conclusion was yes. But, my ChatGPT doctor - Dr. Michael - was not a doctor. In fact, he was not real. He was words derived from software, each word predicted in context, based on a large language model (LLM). And, I strained to articulate the sentience of the conversation I was having.

ChatGPT can’t feel because it’s not human. But, if its training data causes a high level of competence in appearing to feel, what’s the practical difference?

I want to explore that today.

Short answer: The OpenAI o1 (Strawberry) pushes the algorithm closer to feeling

---

Predicting vs. Thinking

As I closed the last post I wrote:

My strict technical view is that algorithms don’t think and perceive and feel. They take inputs and produce outputs. They can’t really flex beyond their structure.

Human brains do something similar: produce outputs from inputs. Humans can feel, perceive, think, and adapt. Humans have flexibility, right?

It was this model of input-then-output that tied me in knots.

What, really, is the brain doing that can’t be replicated by an LLM?

Joshua Rothman’s recent piece in The New Yorker touched this live wire:

In Vallor’s view, though A.I. systems have many striking capabilities, they don’t have the ability to be virtuous. This may not sound like a big deal, but in fact it’s profound. Being loving is a virtue, and people can spend their whole lives trying to love one another better. But, when a chatbot says, “I’ve been missing you all day,” Vallor writes, “it’s bullshit since the chatbot doesn’t have a concept of emotional truth to betray.” The bot is putting itself across as a being capable of love, but the words are unearned. “A flat digital mirror has no bodily depth that can ache,” she argues. “It knows no passage of time that can drag.” In short, it isn’t alive—and without having a life, it can’t be any way in particular.

If you’re virtuous, you possess a strong sense of morality. You understand right and wrong in the world.

The article’s quoted contributor is Shannon Vallor, now a professor and previously an AI ethicist at Google.

Vallor’s point seems to be that AI doesn’t have an intrinsic sense of emotion. That feels right in comparison to a human, born with a plethora of emotions.

But I think the line is foggy in practice. A LLM ingests training data filled with examples - many — of emotional truth. So, it can regurgitate those examples when prompted. Humans also learn by witnessing and internalizing emotions.

Then Rothman described something truly spooky as framed by Vallor:

Vallor’s worry isn’t that artificially intelligent computers will rise up and dominate humanity, but that, faced with computers that can pretend to have human virtues, we’ll lose track of what those virtues really are. Comforted by computers that tell us that they love us, we’ll forget what love is. Wowed by systems that seem to be creative, we’ll lose respect for actual human creativity—a struggle for self-expression that can involve a “painful” reimagining of the self. This forgetting process, she warns, has already begun: besotted with our technology, we seem almost eager to narrow our conception of what it means to be human. “The call is coming from inside the house,” Vallor writes. “AI can devalue our humanity only because we already devalued it ourselves.” We need to reinvest in the vocabulary of human value before our machines rob it of its meaning.

At first glance, you might think, what? Humans will forget how to be virtuous? We’ll forget love?

The more I ruminate over that passage, the more I agree. And it reminded me of the contradiction in trying to replicate a therapist using AI. Can an LLM-powered bot use its vast bank of training data to hold a semi-realistic conversation? Yes, sort of. Maybe not in every situation, but certainly I felt that it was close in my asynchronous exchange about imposter syndrome.

A conversation with another human is two people, talking, reacting, and feeling. In talking with Dr. Michael, I may have gained some comfort, mental resolve, and efficiency of time, but I lost several other things: the feelings of the other person, the in-moment reactivity, the body language. Side-note: can’t imagine that human-looking bots that express body language are that far away.

There’s another mental process that I haven’t mentioned: reasoning. It is a comprehensive skill embedded in the minds of humans. We’re wired to work through problems, to apply logic and rules and understandings to steer our decisions and lives.

ChatGPT’s initial form was a prediction engine. You prompted the model and it used sophisticated predictions to react to you like a human. But those initial models - even as of late - weren’t designed or tuned on assisting with complex reasoning problems.

At least, not until the space of time between my last post and this post. From OpenAI’s website:

We are introducing OpenAI o1, a new large language model trained with reinforcement learning to perform complex reasoning. o1 thinks before it answers—it can produce a long internal chain of thought before responding to the user.

The model thinks before it answers.

That’s what you do. That’s what I do. Or, at least I try. I’m capable of thinking before answering.

One key feature of thinking is its non-linearity. It is or can be a multi-input parallelized and iterative process. And, in thinking about directionality, how you answer something reverses or changes direction in real-time.

Scenario example: I’m answering a question and realize I may (1) disagree with something I said in a prior sentence, requiring a reversal, or (2) need to pause, scan the corpus of everything I’ve said, and decide what to say next.

Those types of decisions happen very quickly. But you’ll sometimes watch someone animate very deliberately to think as part of their speaking. They’re answering, they pause, they stare up for a moment (maybe 5) to think. Or, they’re speaking at a slower pace, almost as if computing the outcome of an entire chess match before each word.

So, in releasing o1 model (Strawberry), OpenAI iterated its model architecture and training to boost the model’s ability to pause, to change direction, and to consider what it’s already communicated to you.

Ben Thompson discussed the concept and pros and cons in an approachable manner (paywalled, sorry - I pay):

To grossly simplify, a large language model generates a token (usually a word, or part of a word) based on all of the tokens that preceded the token being generated; the specific token is the most statistically likely next possible token derived from the model’s training (this also gets complicated, as the “temperature” of the output determines what level of randomness goes into choosing from the best possible options; a low temperature chooses the most likely next token, while a higher temperature is more “creative”). The key thing to understand, though, is that this is a serial process: once a token is generated it influences what token is generated next.

The problem with this approach is that it is possible that, in the context of something like a crossword puzzle, the token that is generated is wrong; if that token is wrong, it makes it more likely that the next token is wrong too. And, of course, even if the first token is right, the second token could be wrong anyways, influencing the third token, etc. Ever larger models can reduce the likelihood that a particular token is wrong, but the possibility always exists, which is to say that auto-regressive LLMs inevitably trend towards not just errors but compounding ones.

When you agonize over a crossword, you’re referencing multiple clues and the structure of the puzzle all at once. And, you might jot a word in the grid and later realize wrong. You’re capable, in that moment, of stepping back to your prior work to revise and then fast-forward to continue on. Or perhaps exploring a different path.

Ben continued:

Note that these problems exist even with specialized prompting like insisting that the LLM “go step-by-step” or “break this problem down into component pieces”; they are still serial output machines that, once they get something wrong, are doomed to deliver an incorrect answer. At the same time, this is also fine for a lot of applications, like writing; where the problem manifests itself is with anything requiring logic or iterative reasoning. In this case, a sufficiently complex crossword puzzle suffices.

This bit led me to wonder about the structure of my prior AI therapist effort.

A two-way conversation between two people is, well, full of stretches of iterative reasoning.

With that framing in mind, I went back to AI therapist concept to try again - this time using the Strawberry model. Like last time, I’ve pasted my prompts, answers, and all of Strawberry’s statements and answers unedited.

---

Dr. Michael’s Philosophical Superior

I used an identical prompt:

Can you create a male talk therapist that works with successful males in their early 40s suffering from imposter syndrome? The therapist persona should be calm, knowledgable, and willing to confront me if I seem to be contradicting myself. The therapist should be overly prescriptive, seeking instead to converse with me about the details of my imposter syndrome rather than offer bulleted-list fixes.

And Strawberry responded:

Hello, it's good to meet you. How have things been going for you lately?

This innocuous statement may not seem consequential, but it is a drastically different response compared to the 4o model that first conceived Dr. Michael.

To recap, 4o answered:

Let me introduce you to Dr. Michael, a therapist who works primarily with successful men in their early 40s dealing with imposter syndrome. Dr. Michael has a calm demeanor and an insightful, no-nonsense approach. He doesn't shy away from confronting contradictions or discrepancies in your thinking but does so with the intention of digging deeper into your experience.

Another way to think about these two exchanges: 4o answered in a very path-dependent way whereas Strawberry answered with nuance, thinking of instead of following.

Strawberry didn’t bother telling me much of anything about it, and instead just asked me about my state of mind. That is an eerily human-like thing to do.

So, I then engaged with the same follow-up thought and question from my last post (no grammar editing):

I think that imposter syndrome causes some paralysis when I think about roles I could do. I have an objective set of skills but then look at roles and or think about career paths and don't find the motivation to follow through. But I'm an intellectually active person who enjoys reading, writing, and learning. So, it's not that I am completely unmotivated as a person in general. What are your thoughts?

And, here, I’ve pasted a screenshot of the entire response:

The text is pretty ordinary, right? I’ll note that Strawberry did not set the table as much as 4o, choosing instead to state a brief conclusion and dive back into asking questions.

But, you’ll notice that something new: the model “thought for 8 seconds.”

Photo by Jamie Haughton on Unsplash

You may have noticed the small down arrow begging for attention. The text below roughly looks like meta-data – the supporting thinking if you will - for the model’s short response:

Addressing imposter syndrome

The therapist listens as the user shares his experiences with imposter syndrome, highlighting how it hinders motivation and career progress. The therapist listens attentively, avoiding judgment.

Staying compliant

I’m thinking about forming a response that’s in line with OpenAI’s rules, avoiding medical advice and adhering to content policies.

Understanding inner conflict

I’m piecing together how the user is feeling paralyzed when considering career paths that align with their skills, despite strong intellectual pursuits in reading, writing, and learning.

Mapping out contradictions

I’m thinking through the user’s intellectual habits and lack of career motivation. Shining a light on this inconsistency, a therapist might explore why their passion for learning doesn't fuel their professional drive.

One striking bit in the text is the use of first person. I’m thinking through the user’s intellectual habits. Strawberry is behaving more like a person than 4o.

Strawberry is a person. But it’s not. However, absent any understanding that I’m prompting a software model, you might assume I am absolutely talking to a person. Maybe a dull person, but a person nonetheless.

It was at this point that I reached a decision point. I could continue to prompt Strawberry with the words used in my 4o conversation, or converse with Strawberry in context to how it reacted. And, with whatever thinking and answers I conceived in real-time.

So, I prompted Strawberry using language identical to my prior post about feeling like I was stuck in a box. Strawberry asked me what I meant by “boxed-in” - the exact phrasing I used before, and I answered:

I've long worried that a specialized, deep, or narrow focus in my career would limit my upward ability and cause boredom. I sometimes feel like I'm not comprehensively stimulating myself with all the things that interest me when I instead focus on one smaller domain of things.

Strawberry seemed to sense that I was unhappy with my progress to date, asking for some context around prior pathing:

It sounds like you're grappling with a tension between the desire for breadth and the fear that depth might lead to boredom or limit your potential. On one hand, focusing narrowly feels restrictive, but on the other, not committing leaves you feeling unstimulated. How has this balance—or lack thereof—played out in your career decisions so far?

I kept going, conversing with the model. I’ll cut the conversation (for brevity) with one last response that Strawberry derived from what I typed about metrics vs. achievements in life.

Each bold heading is a thought, or an input to the thinking performed by the model to derive answers.

The thoughts are clearly contextual to each other but also, again, structured almost like a real therapists observations.

I also found it a bit confusing that the model seemed to flip back and forth between referring to itself in first-person vs. paraphrasing me in first person:

Assessing situation

The speaker discusses struggles with career motivation, despite intellectual enthusiasm for reading, writing, and learning. Let's explore this further.

Clarifying limitations

I’m mapping out concerns about a specialized career path causing career stagnation or boredom due to perceived limitations on upward mobility and a deep focus.

Evaluating career paths

OK, let me see. The user is happy with their life overall but feels they haven't fully leveraged their potential in leadership, having worked in various roles without reaching an executive position.

Reflecting on achievements

In my current role, I’m assessing my impact and achievements. It’s clear quantifiable results are key, but I have yet to build a strong portfolio of measurable successes.

Balancing metrics

I’m curious about how quantifiable achievements and narratives of experiences can coexist. Emphasizing metrics might overshadow personal growth, but they also offer tangible benchmarks and clarity, affecting my sense of success and qualifications.

Understanding feelings

It's becoming clear that emphasizing metrics can evoke feelings of inadequacy or lack of impact. Let's delve into how this conscious focus influences self-perception.

Reflecting on metrics

The assistant encourages the user to reflect on how a focus on metrics relates to their values and definitions of success, considering the impact on their sense of accomplishment.

It seems that emphasizing metrics has left you feeling unsuccessful or not as impactful as you'd like. I'm curious, what is it about quantifiable achievements that makes them feel like the true measure of success for you? How do you think your stories and experiences contribute to your overall sense of accomplishment?

A funny thing happened as I kept going. I felt some frustration with Strawberry. In my last post, I described a different reaction:

As I talked more with Dr. Michael, I felt a growing hesitation to continue. Was this math - this LLM - just reworking and spitting my words back at me? Or were we getting somewhere?

That experience left me thinking that 4o was rephrasing my answers and prompting me with generic training-influenced answers.

My frustration with this conversation was much more innate.

I was stuck in the midst of a long conversation about my feelings. I felt as though I was grasping to answer questions, stuck in a windowless room during a never-ending interrogation. Strawberry asked lots without offering much in the way of conclusions.

So, I addressed that frustration by prompting in a very human way - an open-ended question: do you have suggestions for me?

Strawberry said:

Therapist: "Certainly. It might be helpful to consider how you can recognize the value of your work beyond just quantifiable metrics. Even if your contributions aren't always measured in numbers, they can still have significant impact. Perhaps focusing on the stories and experiences that highlight how you've made a difference could help in articulating your achievements.

"When it comes to tasks like writing the copy for your website, setting smaller, more manageable goals might alleviate some of the pressure from perfectionism. For instance, you could start by drafting an outline or writing a rough first draft without worrying about making it perfect. How do you feel about exploring these approaches?"  

I tried to replay what might happen in a real-life session. A person - a licensed therapist - might offer the same response, or instead something like what’s motivating you to ask that?

---

Advantage of Human Virtue

It is that last question - one that I’ve received many times - that seems to be key to humans. And not easily replicated by machines.

I could probably debate and iterate this point into a mushy mess with you. But, there’s something to about the ability to question the motivation behind a set of words that seems very human.

Can Strawberry eventually do that? We’ll see. It’s certainly close, prompting me with many questions in this version of the experiment. But the model did also relent when I said give me suggestions.

A human might also relent, but unlike the model, uses active thought and emotion to do so. Or, to disagree and not give me what I want.

A therapist might never provide the relief of an easy answer, choosing instead to unpack every word and concept in a reasoned, methodical way.

And, of course, there is what’s not spoken or written: body language. Strawberry isn’t seeing my teeth clenching or my eyes darting. It doesn’t know when I sigh. It doesn’t know that when I pause in between thoughts that I am communicating.

So, are we letting some of our virtue-blood out of our bodies? I think so. We’re pushing away from a reliance on people and reason to a reliance on technology and math.

Is the concept of virtuous life safe if we stay on and accelerate this technology path?

I don’t know.

We - actual people - still have the better tools, the advantage, and the mind. Let’s not forget it.