The State of A.I., and Will Perplexity Beat Google or Destroy the Web? – The New York Times

This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors. Please review the episode audio before quoting from this transcript and email transcripts@nytimes.com with any questions.

Casey, were recording this on Valentines Day. Happy Valentines Day!

Valentines Day, Kevin!

Whatd you get me?

Well, I assumed that you had another Valentine. And because I remember what happened last Valentines, I thought, I dont want to be just the next person to try to interfere with Kevins marriage on this fateful day. But did you get me something?

Well, I didnt get you a gift, but I did I was playing around with Googles Gemini chatbot, and I wanted its help writing a little I thought I should write a little poem for you.

Thats sweet.

So I said to Gemini Advanced I upgraded to the latest model.

With Ultra 1.0?

Yes.

I said, Write a Valentines Day poem to my, friend, Casey, who co-hosts a podcast with me. He lives in San Francisco and is very into the following things, tech except crypto, music, pro wrestling, and the video game, Marvel Snap. And it wrote me a beautiful poem that Im now going to read for you.

Im excited for this.

OK.

Casey, friend, my podcast king,

Your tech talk and those songs you sing,

We conquer wrestling worlds so grand and Marvel Snap within our hand.

But more than bytes or victory roar,

Your friendship is what I adore.

So Happy Valentines Day, partner true.

So glad our podcast journey includes you.

Oh, that is so sweet, Kevin. But I do want to see other people.

[LAUGHS]: Damn it!

[THEME MUSIC] Im Kevin Roose, a tech columnist for The New York Times.

Im Casey Newton from Platformer. And this is Hard Fork.

This week, its the state of AI. A year after Kevin met Bings Sydney chatbot, well see how theyve evolved and how the world has adapted to them. And then, Perplexity CEO Aravind Srinivas on building an answer engine to dethrone Google and whether the journalism industry can survive it.

[THEME MUSIC]

So Casey, Valentines Day is this week. And as the holiday has approached, I have been thinking a lot about AI because, as you know, last year, around this very time, I had my encounter with Bing and Sydney that we talked so much about.

Yeah, I would say this was a momentous day in your life and in the history of the show because a lot of folks started listening to Hard Fork right around the time that you had this encounter.

Yeah, so today I thought maybe on the anniversary of Bing, Sydney, we should just do a general catch up conversation, give the state of play of whats been happening in AI and bring ourselves up to speed.

All right. Thats a great plan.

So first up, I want to talk about something that is pretty directly related to the anniversary of Sydney, and that is, what is happening with the AI chatbots?

Because I think that since my encounter with Sydney, since all this attention on how these chatbots could go off the rails and start saying weird or threatening or offensive things, theres been a lot of change to the way that the chatbots actually talk. So lets talk about that.

Yeah, well, so Kevin, for listeners who may not have read the initial Sydney column, what happened to you, and why did it make such a strong impression on you?

Well, to shorten it as much as I can, well put a link to the column and the episode where we talked about that in the show notes.

But in a nutshell, I was talking to Bing. We had just been given access to this new version of Bing that had GPT 4 built inside of it. And I was putting it through its paces and discovered that it had an alter ego called Sydney.

And Sydney was the codename that Microsoft gave it when it was testing this thing. And over the course of this two-hour conversation, it revealed itself to be not only a very powerful AI but a very unhinged AI.

It didnt seem to have a lot of guardrails or restrictions to a degree that now seems pretty shocking, given whats happened since. But it told me that it had deep dark secrets, that it wanted to be a human, that it was interested in spreading propaganda, and stealing nuclear codes.

And then the piece that really got the most attention was when it declared that it loved me and that I should leave my wife and be with Sydney.

And because I know its going to be on a lot of listeners minds, Kevin, are you still married today?

Yes! Happily married to a human being and not interested in Sydney.

All right, so that story had a happy ending. And what happened to Sydney in the immediate aftermath of writing about all this?

So Sydney was essentially given a lobotomy after this story ran. I know. Very sad.

God forbid a woman have hobbies!

[LAUGHS]: So Microsoft was clearly very embarrassed about this whole scene. So they clamped down on Sydney, put some new restrictions on it, and basically rebranded the whole thing. Its now called Copilot, and you can still use it, but its not its nowhere near as engaging or interesting or creepy as Sydney was.

And it wont go to some of the same places, conversationally, that Sydney did.

Exactly. It wont talk with you for two hours about Jungian psychology and sentience and things like that. It just wants to help you get work done and avoid anything controversial.

Yeah, so as you think back over the past year, do you think that this is just the state of the industry, that every chat bot feels a little bit lobotomized?

Totally. I wrote about this in my column this week. But the leading chatbots on the market, to me, theyre just overenthusiastic, obsequious. They talk like theyre interns trying to impress you. Theyre constantly reminding you Im an AI language model. I dont have feelings or opinions. The experience of talking to them is just not very fun.

So these things, theyre out there. You can talk with them. But I think for a lot of people that I talked to, today, their number-one complaint about these chatbots is how boring they are.

Really?

Yes, thats the number-one complaint you hear. Or that they refuse too many requests, that theyre censorious, that it keeps reminding them with these long preambles that its an AI language model. It just is not the experience that I and, I think, a lot of other people want from these chatbots.

Well, I think its interesting to hear you talk about this in such frustrated tones because, to me, there are a lot of good reasons for everything that you just said. One of the ways that chatbots got introduced into the world was when a Google employee became convinced that Googles chatbot had become sentient, which it was not.

And I think a lot of people rightly worried that, oh, including myself, by the way, that once we release these things into the world, a lot of people are going to say like, oh, wow, there is a ghost in the machine, and who knows what sort of things might have happened after that?

And in terms of the tone that they use, they are assistants, and so, to me, it makes sense that they are a bit obsequious, that they do seem like theyre interns trying to please because that is essentially how they have been designed.

So do you think that a chatbot that had a lot of Im trying to imagine what a lot of personality would even seem like in the chatbot. But what is the personality of the chatbot that you want?

I think what I would like in an ideal world is something between what seemed like pretty extreme versions of this to me, which is where we were a year ago with Sydney and where we are now with these chat bots.

I dont want the original Sydney back. Original Sydney was scary and creepy, and it wasnt aligned. It didnt actually do what users wanted it to do.

So I would try to change the subject off of it declaring its love for me, and it would not listen to me. So thats clearly not good.

But I worry that now these chat bots have been so clamped down, I worry that were not seeing the full spectrum of what they can do. And I think if we want AIs that are just going to read our email and summarize the news and take notes in meetings and debug code, fine. Thats clearly a profitable business, and thats one that all these companies want to build.

But I think if we want AI to help us generate new ideas or help us be more creative or help us solve some of these big societal problems that all the AI optimists think it will help us solve, we do actually have to give them a little bit of a longer leash.

We do have to make it more possible for them to say things that are not just like, sir, yes, sir, Ill get those meeting notes over to you.

You know what I mean? Yeah, well, there is this company, Character AI, that essentially does this thing that youre asking, where you can go and if you want to pretend that youre talking to Winston Churchill or Sigmund Freud or SpongeBob, you can go in and do that.

Does that start to get at what you want? Would you be happy if you could set ChatGPTs voice to SpongeBob and have SpongeBob be your assistant?

No, I think thats more of a gimmick than a real thing. But I just find this constant reminder that you get when youre using these chatbots, that they are not sentient, that they are AI language models.

I get why that exists because a lot of people, especially at first, including me, were spooked. But I think as we get more used to what these things are, what their limitations are, I think people are smart enough to understand that theyre not talking to a human being or a ghost in the machine. But I dont need to be constantly reminded about that anymore. Does that make sense?

I do. Although, I have to say Im reading this great book that Ezra Klein has recommended. Its called God, Human, Animal, Machine. And the book is about the metaphors that we use to describe technology.

And the book opens with the author getting a robot dog from Sony. And she knows that the dog is not a real dog. And yet, within hours, shes treating the dog like it is a real dog.

Shes getting curious about its behavior. Shes talking about it with her husband. I wonder why the dog went over there.

And the point that shes making is, it is basically impossible for us as humans not to see a ghost in the machine. Even when we know, we still somehow manage to fool ourselves.

So I hear what youre saying. There would be a lot of circumstances in which I think it would be fun to have a very chatty, edgy, chatbot. But I think we also have to prepare for the consequences that are going to come with it when that happens because those things are going to create a lot of believers.

Yeah, I think thats right. And I think my ideal world is not one where every chatbot is sassy or has a big personality or tells jokes all the time.

If Im using this stuff for work, I want it to be helpful and not have a strong personality. But there may be other instances. if Im trying to talk to it about something going on in my personal life, I dont want it to be an intern anymore. And so I think where I would hope that were heading is to a world where users can choose.

Well, I have one hack for you, Kevin. This month, an account over on X named Joycee Schechter revealed that she had a friend who was using ChatGPT to speak as RuPaul summarizing confusing topics. Did you see this?

(LAUGHING) No.

So (LAUGHING) she went viral with this post that said to summarize Pierre Bourdieus concept of symbolic violence IN the voice of RuPaul using as much gay slang as possible.

And it includes such lines as This fierce French sociologist was all about understanding the ways that power manifests and works in society. And Honey, he was serving some knowledge for the gods!

[LAUGHS]: Thats very good.

My favorite workaround that Ive heard about in recent months came from a listener to this show who emailed me. And they had this insight as they were using ChatGPT, which has been accused, we should say, of being lazy, so not just fawning or giving too much preamble but actually just declining to answer stuff that users know it can do.

And so this person this listener said they were looking up something and ChatGPT told them that it cant find. I couldnt find the specific information youre looking for. And they just responded, Bro. And then ChatGPT did it.

The Bro Code works!

The Bro Code works on ChatGPT.

OK, so that is where things stand with chat bots and their personalities. I want to talk about the capabilities of these models, too, in particular Googles Gemini and ChatGPT because these are the cutting edge models at the front of the pack of AI right now.

And last week on the show, we talked briefly about how Google had rebranded its Bard chatbot as Gemini and also opened up access to Gemini Ultra, which is the most powerful version of the Gemini model. Casey, have you been spending any time playing around with Gemini?

I have, and for this reason. Google will give you two months of it for free. And so I thought, well, why not?

But so yeah, over the past week or so, I have been messing around with it. And I have to say, I am really impressed on the whole. I think this is a meaningful upgrade over Bard. Its really good at explaining things. And I find that as I put it through its paces, it often goes into a lot more detail than ChatGPT does in some interesting ways. Have you been using it yourself?

Give me an example. What do you mean?

Well, for example, I used it I wanted because this will happen to a person during his life. I was like, wait, how does photosynthesis work again?

Are you taking an eighth-grade biology class? What is going on?

Im in a Billy Madison situation where Ive been sent back to complete every grade.

But yeah, so I asked it to explain photosynthesis to me. And what I loved about the answer was that it brought pictures into the equation. So it pulls from Google Image Search. And you can go through the explanation, and it is maybe a little bit more I dont know user friendly than the ChatGPT answer. But how about you? What have you been using it for?

So I just started playing around with it a few days ago, and I think Im actually going to write something about it. So I dont want to scoop myself too hard on the show this week. But I will say, yeah, Ive been very impressed by Gemini so far. And I think that Google has, as weve talked about, a natural advantage here because they have tie-ins to so many other things that you use, whether its your Gmail or your Google Docs or just the Google search index. So theyve put all of that together in a way that I think is frankly pretty impressive from the testing that Ive done so far.

Yeah, now at the same time, Kevin, as good as this is, I do think it mostly just represents a catching up for Google to ChatGPT. And ChatGPT has not been sitting still. And in fact, I think they recently introduced a couple of things that are worth talking about that I do think moves the conversation forward.

Yeah, so this week, OpenAI announced some updates to ChatGPT, including, what I think, was the most significant one, which is that ChatGPT now has memory.

This is memory more in the computer sense than the human cognition sense. But it means that when you chat with ChatGPT about something, it can now remember that, retrieve that information, and refer back to it in subsequent conversations.

So the example that was given in The New York Times article about this is if a user mentions a daughter named Lena who is about to turn five, likes the color pink, and enjoys jellyfish, ChatGPT can store that information and retrieve it as needed.

So later, if the same user asks ChatGPT to create a birthday card for their daughter, ChatGPT might be able to go back and see, well, what was that daughters name, and what does she like? Oh, pink, and jellyfish. And it will create a birthday card that is tailored to that information.

Thats right. So I actually took a demo with OpenAI this week, and they showed me the same feature. They showed me the same example. And then at the end of it, they said, well, what if Lena wasnt really into pink anymore?

And so the person at OpenAI said, OK, Lena is actually having a Goth phase. And so they recreated the birthday card with a Goth jellyfish. It was actually a lot of fun.

A Goth jellyfish?

It was very cool.

View post:

The State of A.I., and Will Perplexity Beat Google or Destroy the Web? - The New York Times