<markdown>

@Stephen Wolfram : 人工智能的概念起源于20世纪50年代，最初人们认为只要制造更大的计算机就能实现自动化思考。人工智能领域有两种主要方法：符号方法和统计方法。符号方法旨在通过计算方式表示世界，并让AI以计算方式理解世界；统计方法则忽略规则，仅通过统计数据来猜测世界如何运作。神经网络是一种古老的统计方法，其历史可以追溯到19世纪中叶，与大脑的工作方式有关。1980年代，专家系统成为主流，但后来神经网络复兴，并在2011年通过图像识别取得了突破。2022年底，ChatGPT的出现标志着自然语言处理领域取得了重要进展。未来，人工智能将继续发展，但其发展方向可能与人类的期望有所不同，我们应该关注如何利用人工智能为人类服务，而不是试图复制人类智能。 Stephen Wolfram: 我认为人工智能的发展方向是超越人类智能的，它将以一种非人类的方式进行计算，就像自然界中的许多现象一样。我们应该关注如何利用这种非人类的计算能力来解决人类的问题，而不是试图将人工智能限制在人类的框架内。例如，我们可以利用人工智能来分析大量的科学文献，从而发现新的科学规律。我们也可以利用人工智能来诊断疾病，从而提高医疗水平。总之，人工智能的未来是光明的，但我们需要以一种开放的心态来迎接它。

我最近与计算机科学家、物理学家和企业家Stephen Wolfram进行了一次深入的对话，探讨了人工智能的方方面面。Wolfram以创立Wolfram Research以及开发Mathematica和计算知识引擎Wolfram|Alpha而闻名。他是一位天才，15岁时就发表了物理学方面的论文，20岁时获得了加州理工学院的博士学位。他后来又撰写了《一种新科学》，提出简单的计算规则可以解释自然界中的复杂现象。Wolfram在符号计算、计算思维和人工智能领域一直是先驱。他的工作持续影响着科学、教育和技术。

我们的谈话涵盖了人工智能的诸多方面，从其起源到未来发展趋势，以及对社会的影响。以下是一些关键观点：

人工智能的概念起源于20世纪50年代。当时，人们普遍认为，只要制造更大的计算机，就能实现自动化思考，这是一种过于乐观的想法。实际上，人工智能的发展道路远比想象中复杂。

Wolfram 指出，人工智能领域存在两种主要方法：符号方法和统计方法。符号方法试图通过构建世界的计算表示来让AI进行推理；而统计方法则依赖于从数据中学习统计规律来预测结果。神经网络就是一种古老的统计方法，其历史可以追溯到19世纪中叶，它试图模拟大脑的工作机制。

在20世纪80年代，专家系统一度成为人工智能领域的主流。然而，神经网络在2011年通过图像识别任务取得了突破性进展，标志着其复兴的开始。2022年底，ChatGPT的出现则标志着自然语言处理领域取得了重大进展。

Wolfram 认为，人工智能的发展方向并非仅仅是复制或超越人类智能，而是创造一种全新的、非人类的智能。这种智能将以一种我们目前难以理解的方式进行计算，就像自然界中许多复杂现象背后的计算过程一样。他将这种非人类的计算能力比作自然界的复杂性，例如湍流的计算过程。

他强调，我们应该关注如何利用这种非人类的计算能力来解决人类的问题，而不是试图将人工智能限制在人类的思维框架内。例如，我们可以利用人工智能来分析大量的科学文献，从而发现新的科学规律；也可以利用人工智能来诊断疾病，从而提高医疗水平。

Wolfram 并没有对人工智能的未来发展表示担忧。他认为，人工智能的发展将带来许多机遇，但也存在一些挑战。其中一个重要的挑战是如何将人工智能与人类社会有效地整合。他认为，人类的独特之处在于我们能够进行选择、进行创造性的思考，以及与他人进行互动。这些都是人工智能目前难以复制的能力。

关于人工智能对就业市场的影响，Wolfram 认为，自动化将导致一些工作岗位消失，但也将创造新的工作岗位。他认为，人类应该适应这种变化，并学习如何利用人工智能来提高自己的生产力。

关于财富观的变化，Wolfram 认为，虽然人工智能可能会导致一些商品和服务的成本下降，但稀缺性仍然存在。人们仍然会追求稀缺的资源和体验，这将塑造未来的财富观。

Wolfram 指出，软件开发的自动化是一个持续进行的过程，他本人就致力于此领域的研究长达数十年。他认为，人工智能可以极大地提高软件开发的效率，但它并不能完全取代人类的创造力和判断力。软件开发的未来仍然需要人类的参与，特别是对于那些需要创造性思维和复杂决策的任务。 关键在于定义软件的目标和功能，而不是仅仅关注代码的编写过程。

总而言之，Wolfram 对人工智能的未来持乐观态度。他认为，人工智能将成为人类社会发展的重要驱动力，但我们需要以一种开放的心态和长远的眼光来迎接它，并关注如何利用人工智能来造福人类。 关键在于拥抱变化，适应新的技术，并专注于发展那些人工智能难以取代的人类能力。


◉

00:21 我想了解人工智能领域的最新进展和未来发展方向。

◉

00:50 我可以根据我的了解来讲述人工智能的历史。

◉

01:08 1956年，John McCarthy在达特茅斯会议上创造了“人工智能”这个词，并在之后发明了Lisp语言。

◉

01:26 我和John McCarthy的观点不一致，因为他认为Lisp是实现人工智能的唯一语言。

◉

01:46 1979年，我开始构建自己的符号计算系统，由于当时Lisp不实用，我使用了C语言。

◉

02:15 John McCarthy对此耿耿于怀，这也让我意识到Lisp虽然有趣，但不太实用。

◉

02:40 1956年John McCarthy创造了“人工智能”这个词，源于早期人们将计算机描述为“巨型电子大脑”，认为计算机可以自动化思考。

◉

02:56 在1950年代到1960年代初，人们认为只要制造更大的计算机，就能像自动化推土机一样自动化思考。

◉

03:14 1960年代，人们投入大量资金和精力研究人工智能，其中一个原因是冷战。

◉

03:40 当时人们希望用机器翻译取代外交场合中的人工翻译，以避免误导和引发战争。

◉

03:48 用机器翻译更好，可以避免人为错误。

◉

04:14 1960年代初，人们认为人工智能将创造出下一个物种，与今天人们的看法几乎一样。

◉

04:31 唯一不同的是，当时的语言带有性别歧视色彩，现在已经进行了调整。

◉

04:59 人们对“巨型电子大脑”的看法长期以来保持一致，人工智能领域有两种主要方法：符号方法和统计方法。

◉

05:20 符号方法旨在通过计算方式表示世界，并让AI以计算方式理解世界。

◉

05:40 统计方法则忽略规则，仅通过统计数据来猜测世界如何运作。

◉

06:07 神经网络是一种古老的统计方法，其历史可以追溯到19世纪中叶，与大脑的工作方式有关。

◉

06:31 1870年代，Golgi发明了一种染色神经细胞的方法，使人们可以在显微镜下观察大脑组织。

◉

06:58 Golgi认为神经细胞连接成一个大网络，而Romani Kajal认为它们是独立的细胞，两者对此存在争议。

◉

07:22 这场争议甚至导致两人共同获得了早期的诺贝尔奖。

◉

07:45 人们很早就知道神经是带电的，并且大脑中存在神经网络。

◉

08:04 1870年代，人们开始讨论大脑中的神经网络如何以某种电的方式实现逻辑。

◉

08:30 1943年，Warren McCullough和Walter Pitts发表了一篇关于神经元网络的逻辑理论的论文，奠定了后续研究的基础。

◉

08:56 1950年代中期，人们开始在计算机上实现神经元网络，特别是感知器。

◉

09:26 感知器是一种简化的人工神经网络，可以通过统计图像来识别图像中的内容。

◉

09:54 但感知器存在一些问题，例如在坦克识别中，感知器只是根据照片的拍摄时间（白天或晚上）来判断是否有坦克。

◉

10:16 理解AI的关键在于区分结果是源于数据的简单特征还是AI的深度理解。

◉

10:44 1960年代初，Marvin Minsky认为感知器和神经网络是微不足道的。

◉

11:01 Minsky和Seymour Papert的书认为感知器和神经网络无法做任何有趣的事情，导致神经网络在1960年代末被认为毫无希望。

◉

11:31 当时的人工智能如果要有任何发展，都将是在符号方面，即找出世界运作的规则，并尝试将心理学中的想法加以提炼，使之能够在计算机上实现。

◉

12:20 1980年代，人工智能领域的主要主题是专家系统，即从专家那里学习规则，然后计算机就能像专家一样工作。

◉

12:40 我一直对如何自动进行数学计算感兴趣，这被认为是人工智能的一个重要测试案例。

◉

12:55 自动进行符号数学计算被认为是人工智能的一个重要测试案例。

◉

13:15 1979年，我开始构建SMP系统，它可以进行符号数学计算和许多其他符号操作，但我从未声称它与大脑的工作方式相似。

◉

13:29 我一直对如何自动访问世界知识感兴趣。

◉

13:52 在构建符号计算系统后，我开始思考如何处理更模糊的知识，并假设需要创建一个类似大脑的系统来解决通用人工智能问题。

◉

14:17 我对模式匹配感兴趣，但未能找到实现方法。

◉

14:34 1982年左右，神经网络复兴，一些实验表明它们能够做一些事情。

◉

14:58 我做了一些神经网络实验，但没有得到任何有趣的结果，因此失去了兴趣。

◉

15:26 我的第一家公司最初旨在进行数学计算，但后来转向了专家系统。

◉

15:53 这家公司后来更名为Inference Corporation，是一家人工智能公司，为许多公司构建了各种系统。

◉

16:22 我没有过多参与公司的专家系统业务，该公司后来上市，但我当时已经很久没有参与其中了。

◉

16:47 了解1980年代人工智能的发展对于理解今天的人工智能至关重要。

◉

17:06 1980年代，日本政府启动了“第五代计算机计划”，旨在解决人工智能问题。

◉

17:17 日本的第五代计算机项目旨在解决人工智能问题。

◉

17:40 该项目使用Prolog语言，但其问题解决思路最终效果不佳。

◉

18:09 1980年代，人们对神经网络的兴趣复燃，Jeff Hinton和Terry Sanofsky等人继续研究神经网络。

◉

18:34 他们将神经网络视为并行分布式处理，并结合计算和大脑研究。

◉

18:52 1980年代末至1990年代初，人工智能发展陷入低谷。

◉

19:10 2011年，Jeff Hinton在深度神经网络方面的研究重新启动了人工智能。

◉

19:28 Hinton尝试使用深度神经网络进行图像识别，并在ImageNet图像集上进行年度竞赛。

◉

19:50 Hinton的学生无意中让神经网络训练了一个月，通过数百万张图像进行调整。

◉

20:12 结果表明，该神经网络表现良好，赢得了当年的ImageNet竞赛。

◉

20:41 这次胜利是一个重要的警示，表明神经网络已经回归，并且能够做有趣的事情。

◉

21:03 我原本计划使用图像处理技术来计数图像中的人数，但神经网络的出现使这个项目变得过时。

◉

21:18 随后，人们对神经网络和深度学习产生了极大的兴趣，特别是在图像识别方面。

◉

21:37 图像识别问题在一定程度上得到了解决，但机器学习的典型情况是，正确率达到80%到95%左右，很难达到100%。

◉

22:04 即使在2012年，图像识别系统的水平与今天相比也没有太大差异。

◉

22:11 在过去的13年中，图像识别领域没有取得太大进展。

◉

22:36 一旦达到某个阈值并开始能够做某事，该能力就会被构建到系统中并变得有用，但它本身不会再有更多飞跃。

◉

23:00 大多数人认为ChatGPT或Gemini等消费者版本的人工智能是最近才出现的，但实际上图像识别方面早有突破。

◉

23:24 那么，是什么让人们在过去几年里感受到了新的人工智能革命？

◉

23:51 2022年底，ChatGPT的出现以及使用神经网络根据提示编写文本的能力是重要转折点。

◉

24:09 这种能力有着悠久的历史，可以追溯到1950年代甚至1960年代。

◉

25:00 我自1980年左右就对回答关于世界的问题感兴趣，并认为智能和计算之间没有明显界限。

◉

25:21 因此，我认为可以构建一个系统，根据人类文明积累的知识来回答问题。

◉

25:42 这促成了2009年发布的Wolfram Alpha系统，该系统的目标是将自然语言转化为精确的计算语言，然后计算答案并告知人们。

◉

26:03 当时人工智能已经没落，人们认为类似Wolfram Alpha的系统不会奏效，因为过去几十年里，人们尝试过各种人工智能技术来构建问答系统，但都失败了。

◉

26:27 在发布Wolfram Alpha前几周，我给Marvin Minsky展示了这个系统，但他一开始并不感兴趣，因为他见过太多失败的问答系统。

◉

26:49 但当我坚持让他仔细看时，他意识到这个系统真的有效，并开始四处宣传。

◉

27:08 这反映了当时人们普遍认为人工智能已经没落。

◉

27:37 通过Wolfram Alpha，我们首次解决了自然语言理解问题，将纯文本转化为计算机可以理解的精确计算语言。

◉

28:03 我们的自然语言理解不是抽象地让计算机理解，而是将人类的语言翻译成我们可以进行计算的精确计算语言。

◉

28:25 ChatGPT的传统与此不同，它起源于二战期间的密码分析。

◉

28:47 密码分析的目标是从加密信息中统计地找出原始的英文信息。

◉

29:05 关键在于，英文不是随机的字母序列，而是具有一定的统计规律，例如Q后面通常

00:00

A lot has changed in the world of AI since we first spoke, and I wanted to really get your thoughts around everything that's happening, where it's going. But I think it's good to just get a basic idea for people. What is the first version of AI? What was that that we can kind of coin and define? So just… 语法解析

00:21

as where it's going. And everybody has different interpretations, but I just want to know yours. I can tell you the history as I know it. I mean, back, you know, the term AI was coined by John McCarthy, who I knew. We didn't get on that well, but that's a different story. What's that? Why was that? Why is that? Just intellectual clashes between you two? To begin the AI story. So in 1956, there was this conference at Dartmouth where John McCarthy coined the term AI and 语法解析

00:50

And one of the things John McCarthy did in the years right after that was invented this language called Lisp, which was kind of a very early computer language, very early but sophisticated in conception computer language. It was hard to implement, and it wasn't really implemented fully for decades. 语法解析

01:08

So the reason John McCarthy and I perhaps didn't see completely eye to eye was that Lisp, he saw as being the only language in which you could implement artificial intelligence and what was then thought to be an important piece of artificial intelligence, which was things like symbolic mathematical computation. 语法解析

01:26

Well, in 1979, I kind of got into building my own kind of symbolic computation system, and it wasn't practical to use Lisp at that time. It was just there weren't good implementations and so on. So I used this then newfangled language called C, which isn't very newfangled anymore. 语法解析

01:46

And John McCarthy kind of never forgave me for that. And I think that was kind of the, it was also the beginning of realizing that, I mean, while Lisp was very interesting language, it wasn't yet very practical. Had history worked out differently, Lisp would have been kind of a much more prominent language today. It probably would have been the thing that I have implemented my system into. But anyway, John McCarthy coined the term 语法解析

02:15

artificial intelligence back in 1956, kind of the backstory of that whole thing was that when computers, when electronic computers were first kind of coming on the scene in the late 1940s and so on, the typical description of them was giant electronic brains. So people had the idea from very early on, what computers are going to do is automate thinking kinds of things. And people kind of assumed that that wouldn't be terribly hard. 语法解析

02:40

And that was kind of the in the 1950s into the early 1960s. It was kind of like, yeah, you know, we just build a slightly bigger computer. We'll be able to automate thinking just like we've been able to automate kind of making a bulldozer or something like that. 语法解析

02:56

It was there were a couple of different approaches that were taken. Well, actually, as one indication of kind of the way people were thinking in the 1960s, where a lot of kind of money and effort was put into AI. One of the problems of the time was the Cold War. 语法解析

03:14

And people were thinking, well, there are occasionally these sort of high level diplomatic exchanges between, you know, the U.S. and Soviet Union and so on. And they had this idea that, well, you know, in those exchanges, there would be some interpreter who'd be, you know, translating Russian to English, et cetera, et cetera, et cetera. And they were like, we're really worried the interpreter is going to mislead everybody and it's going to lead to World War III or whatever. So. 语法解析

03:40

Let's put a machine in place instead. Let's have an automatic translator. Yeah, way better, right? No possibility of human error. 语法解析

03:48

Well, people thought that was going to work in the 1960s. And it's actually very interesting to read what people wrote in the early 60s about kind of AI and what was going to happen with AI and the idea that, you know, kind of we were a species that was mostly going to go down in the history of the earth as being the thing that created the next species that was AI and so on. And it's really, really funny to read these things now because honestly, 语法解析

04:14

Pretty much word for word, they're the same as what people say today, with the one exception that a bunch of the kind of language of, you know, men will do this, so to speak, has been, you know, adjusted in modern times. It will be people will do this. 语法解析

04:31

Everything else is pretty consistent. So people have been saying and thinking the same things about the future of, well, essentially giant electronic brains for a long time. But there were really two approaches that, well, first thing, in the 1960s, there were two different approaches that were taken to AI, and there continue to be two different approaches today. There was kind of the symbolic approach and the statistical approach. 语法解析

04:59

The the the idea, the symbolic approach was kind of the thought that you can have sort of a computational representation of the world and you can have your AI kind of figure out things in the world in a kind of computational fashion. That's actually been an approach that I've been deeply involved in for a very long time. 语法解析

05:20

But that's one of the approaches. The other approach was the statistical approach, just saying, forget about having rules for how the world works. Just say, we notice this and that and the other thing. Let's extrapolate from what we notice. Let's just do the statistics of the world to guess how things work. 语法解析

05:40

The main approach that got used there was neural networks. Neural networks are famous today, but neural networks are an incredibly old idea. I mean, I've actually been researching a bit the ancient history of neural networks, and it goes back even much further than I imagined. It goes back to the mid-1800s. How far does it go back? So what happened was neural nets were all entangled by the question of how our brains actually work. 语法解析

06:07

And so the people had looked at brains under microscopes. There was a chap called Golgi who figured out in, I think, the 1870s how to stain nerve cells. Because when you look at a slice of brain tissue under a microscope, it just looks really complicated. You can't see anything. Golgi figured out this way to stain it so that some tiny fraction of the cells would turn purple. 语法解析

06:31

And then you could see those cells that had turned purple, and you could kind of see the pattern of those things. For a while, there was a big sort of dispute between Golgi and a chap called Romani Kajal, because Golgi thought that sort of every nerve, that you would see these sort of nerve cells and that they were really all connected in one big net. Romani Kajal thought they were all separate cells that had sort of synapses, gaps between them. 语法解析

06:58

But that was that was a dispute of the late 1800s the two of them even disagreeing violently about that one one of the early Nobel Prizes in together For studying those kinds of things I think there were more things to give interesting Nobel Prizes to in those days and there are perhaps today but that's a different story on the in any case the 语法解析

07:22

People knew that nerves were electrical because actually the way electricity was discovered was by Volta and people kind of noticing that frog legs kicked when you gave them electric shocks. So people kind of knew it's electrical and then they kind of knew there's a network of nerves in the brain. 语法解析

07:45

And what did that mean? Well, by the 1870s, people were talking about how there might be it might be implementing logic in the network of nerves in the brain in some kind of electrical way. This is before there were really electrical machines that did anything like logic. So. 语法解析

08:04

Then, well, the next probably big step was 1943, a chap called Warren McCullough and a chap called Walter Pitts. Warren McCullough was a neurophysiologist and psychiatrist. Walter Pitts was a young math, very young math person. They worked together, wrote a paper about kind of the logical theory of neural nets. 语法解析

08:30

That paper is kind of the foundation of everything that's been done since. It kind of laid out this way of idealizing the thing that might actually be in brains, but idealizing it in sort of a mathematical way. Well, computers didn't yet exist. By 1946, there were starting to be electronic computers. By the mid-1950s, people were implementing kind of this idea of neural nets on computers. 语法解析

08:56

early computers and on special purpose computers that they'd made particularly for doing neural nets and particularly there were this idea of the so-called perceptron which was a kind of a way of having sort of this this very simplified artificial neural net that would do kind of statistical would figure out given an image it would sort of figure out from the statistics of that image what was in the image it all was going reasonably well 语法解析

09:26

But there were some glitches. So a famous glitch was there was some trial for the military of these things where there were a bunch of pictures of tanks which had tanks in them and other pictures that didn't have tanks in them. And the perceptron did really well figuring out which pictures had tanks in them. But then somebody realized that all the pictures with tanks were taken during the day and the pictures without tanks were taken at night. 语法解析

09:54

So really all the perceptron was doing was something very trivial. And this is a repeated issue with trying to understand what's happening in AI. Is what you see just a consequence of some feature of the data that you didn't happen to notice, but it's sort of trivial? Or is it that you're seeing something that's kind of a deep figuring out that's being done by the AI? 语法解析

10:16

So then actually in, so what happened after that was in the early 1960s, a person I actually knew pretty well named Marvin Minsky, who was a kind of AI pioneer who'd originally been interested in neural networks. He wrote his PhD thesis about neural networks, even built a neural net machine. But he kind of decided perceptrons and all things neural nets are trivial. 语法解析

10:44

And he and a chap called Seymour Papert wrote this book about perceptrons, which argued that perceptrons and neural nets can't do anything interesting. Game over. So by the late 1960s, everybody said neural nets are doomed. They'll never do anything interesting. They're all trivial, et cetera, et cetera, et cetera. 语法解析

11:01

uh still uh and and so sort of ai at the time was if there was going to be anything that happened with ai it was going to be sort of on the symbolic side of figuring out rules for how the world works and uh sort of trying to take things like ideas from psychology and sort of take the ideas which had been expressed kind of vaguely in psychology and try to sort of tighten them up and make them things that you could implement on a computer that was the idea so then 语法解析

11:31

We get to the beginning of the 1980s, and I was by this point sort of actively interested in these kinds of things. And, well, let's see. This history is a bit complicated. So, well… 语法解析

11:51

The thing that was sort of the dominant theme of AI in the 1980s was these things called expert systems. And the idea was you would have sort of a rules-based way of describing the world that you would learn from an expert. Somebody would essentially write the rules based on some expert in geology or some expert in medicine or something like this, would write the rules, and then the computer would be able to do whatever the expert could do. That was kind of the idea. 语法解析

12:20

Now, I have to say, and that idea was kind of a significant idea in the 1980s about how AI would work. Meanwhile, the, well, I myself have been interested in kind of how you would do things like math 语法解析

12:40

sort of automatically, how you would do symbolic math, algebraic math automatically. That had been viewed as a kind of a thing that if we could do this, we'd know we had AI. It had been viewed as sort of a big test case for AI. 语法解析

12:55

But I, in 1979, I already mentioned that I started building a system called SMP, Symbolic Manipulation Program, that did kind of symbolic math and many other kinds of symbolic things. And it did it in a way that I would never have claimed was anything like how brains do it. 语法解析

13:15

Meanwhile, I myself have been interested in how would you take the knowledge of the world and make it somehow automatically accessible. It was a thing I've been interested in since I was an early teenager. 语法解析

13:29

I kind of having had success in sort of building up the symbolic computation system, I kind of got to thinking, could I do something with sort of vaguer knowledge, knowledge that wasn't as precise as the kind of knowledge in math? And so I got to thinking, it was around 1980 or so, I got to thinking, you know, how would I make something that deals with that? And I kind of assumed that to make a thing that could deal with vaguer kinds of knowledge, 语法解析

13:52

I would have to kind of make a brain-like thing. I would have to sort of solve the general problem of artificial intelligence. And so I started thinking about how would I do that? And I was interested in kind of pattern matching and how would you sort of fuzzily match, you know, is that roughly a picture of this or not? And I had a bunch of ideas, never figured out how to do it at the time. Meanwhile, neural nets, 语法解析

14:17

had kind of a comeback. This is around 1982 or so. There were a couple of sort of experiments that were done with neural nets where it was like, wow, they're able to actually do things. And the fact that they were sort of, you know, 语法解析

14:34

squashed flat by the perceptrons analysis wasn't really right. If you had deeper neural nets that had sort of more layers of computation in them, then they might be able to do something interesting. So, you know, I did some experiments on neural nets. I could never get them to do anything terribly interesting. I kind of lost interest in those things. Meanwhile, 语法解析

14:58

In my own personal trajectory there, I had started my first company that was sort of aimed at doing sort of mathematical computation kinds of things through a series of probably not great business decisions of deciding that, you know, I should bring in other people to run the company and things like this. We ended up getting venture capital and the venture capital was like these expert systems things. They're amazing. You should go chase that particular, you know, shiny direction, so to speak. 语法解析

15:26

So the company kind of pivoted to having, well, a division of the company doing expert systems kinds of things. And that's actually kind of strange to realize that the company changed its name eventually to Inference Corporation. That was probably 1983. And I feel I hadn't even thought about that in so many years. It seems so very modern today. It was an AI company. It was doing expert systems AI and the company built all kinds of things for companies. 语法解析

15:53

for, I don't know, it built early credit reporting systems or credit assessment systems. It built a bunch of testing systems for NASA and things like this. So it was really doing, in those days, kind of symbolic AI. I wasn't much involved in that side of the company. The company eventually, well, eventually went public in a very undistinguished IPO sometime in the 1990s, but I hadn't been involved with it for a long time by that time. 语法解析

16:22

The other thing that happened in AI in the 1980s, so I told you this is a long, shaggy story, but it's kind of interesting to understand if you want to know where AI is today and kind of what the roots of what's going on today have been. So I guess two other things of note happened in the 1980s with AI. 语法解析

16:47

There was a time when Japan was viewed as a country where kind of, oh, it just copies American technology. Everybody was kind of very down on that. Japanese government had this great idea. They said, let's do a research project that's going to jump ahead of everybody else. 语法解析

17:06

It was called the Japanese Fifth Generation Computer Project. And it was a project in, I forget when it started, early 80s sometime. It was a project where Japan was going to solve AI. 语法解析

17:17

And their methods, while they were using particularly a language called Prolog, which is sort of a lisp-ish kind of thing, but it had a particular idea about problem solving, which doesn't work out so well in the end. But that was another kind of injection of, you know, AI is coming type thing. You know, Japan is going to solve AI type thing. 语法解析

17:40

There's another piece. Another piece in the 1980s was the kind of the, as I mentioned, sort of the rebirth of interest in neural networks and a bunch of people, including the people who sort of continued working on that till the present day. People like Jeff Hinton and my friend Terry Sanofsky, who were kind of the early people who got interested in could neural nets really be made to do something interesting? 语法解析

18:09

They talked about it as parallel distributed processing. And it was a sort of big mixture of thinking about things computationally and thinking about actual brains and dissecting brains and trying to figure out how they worked and so on. So again, that was sort of a thing that was happening in the 80s. By, I would say, end of the 80s, early 90s, this stuff basically hadn't worked out. 语法解析

18:34

And people said, ah, AI is doomed. And everybody was kind of, you know, everybody who might have been saying they were doing AI didn't say they were doing AI anymore. And AI was really at a very low ebb for quite a long time. The thing that 语法解析

18:52

The relaunched AI was something that happened in 2011, actually, that aforementioned person, Jeff Hinton, had been continuing to study deep neural nets and something people, including myself, were just like, I don't know, I don't think anything interesting is going to happen here. 语法解析

19:10

But he had been studying, trying to do image recognition, trying to tell, this is a picture of a cat or a dog or whatever. There was a big collection of images, ImageNet, that existed. And there were these annual competitions for, you know, who can recognize images best. 语法解析

19:28

Well, a student of Jeff Hinton's left a neural net training kind of by mistake for a month, trying to, you know, going through millions of images saying with the neural net being told there's a cat, there's a dog, there's a cat, there's a dog. And, you know, trying to tweak the neural net. That's how neural net training works. Tweak the neural net to get the answers right more and more often. 语法解析

19:50

So it was like, okay, it was kind of a mistake. The computer was just sitting doing this. It was an early GPU computer. And it was, you know, before throwing the neural net away, I guess, I don't actually know every detail of what happened in those days, so to speak. But it was like, okay, let's just try this neural net, see how it does. 语法解析

20:12

It did pretty well. It won the ImageNet competition that year. That was a big wake-up call that neural nets are back and they're able to do interesting things. The buzzword at the time was deep learning. And so that time, people got very excited about that. I have to say, I myself had been just about to use image processing to just sort of make little functions that would count images 语法解析

20:41

you know i don't know people in an image and things like this and do that by kind of a bunch of image processing hacks and i i my my friend sarah sanovsky said no no we're going to be able to do it with neural nets one day uh it actually arrived within the year and i'm happy i didn't do that project because that project would have been kind of dead meat um 语法解析

21:03

So in any case, what then happened was sort of a big sort of wave of excitement about neural nets and deep learning and so on, particularly applied to image identification. And that problem was kind of solved. 语法解析

21:18

At some level. And by solved, I mean, you know, you get it right 90 something percent of the time or whatever. And that's kind of typically the story of machine learning. You get it right, you know, 90, 80, 95 percent of the time, something like that. It's not 100 percent, but it's not even clear what you mean by getting it 100 percent right. I mean, it's like, you know, is that… 语法解析

21:37

Dog that has been given a cat suit or something. Should that be a dog or a cat? It's hard to define what the answer should be So it's hard to know whether you were like got it exactly, right? in any case the the thing that's also interesting to notice in terms of kind of the the evolution of AI is that the things that one had by 2012 or so we were making image identification systems and so on using these neural net ideas and 语法解析

22:04

they're not that much worse than what one has today in other words in the last 13 years or so 语法解析

22:11

Things haven't, in that particular domain, haven't improved that much. It's like you reach a threshold, you start to be able to do something, then that works, and then you build that capability into a bunch of systems and it becomes useful. But it's not as if that capability itself, just because it made that one jump doesn't mean it's going to make lots of other jumps. 语法解析

22:36

It's interesting you say that because most people that are playing around with what the consumer version of what they think AI is from Chachapiti or Gemini, people would think that there's been like people think this is pretty much AI. Like this is the last two or three years. People just think AI just kind of popped out of nowhere, right? Like from 2021 onwards. 语法解析

23:00

Before that, AI didn't even exist. It just started to come because now they can play around with it. You mentioned that there was a tipping point from the image recognition perspective. So what was the difference between what was it that made people see and feel the new revolution of AI in the last couple of years? What was the segment particularly that improved? 语法解析

23:24

This is a very shaggy story. It's a long story. You asked, what's the history of this? It's quite a long, complicated story. I mean, it… For sure, yeah. So another… I mean, the big thing that happened at the end of 2022 was ChatGBT and the idea that you could get a neural net to write text based on a prompt. 语法解析

23:51

That had a long history. That has a history going back into the 1960s, even into the 1950s. People had, well, okay, there's a couple of branches here. Let me explain the, well… 语法解析

24:09

Yeah, yeah, right. I think it's kind of interesting, perhaps. People will find it… I'm not sure this history is well told, actually. Yeah, you can go through the whole thing. Let's talk about a couple of branches. Well… 语法解析

24:30

There's a branch that I was involved in, there's a branch I wasn't involved in. The branch that I was involved in was this thing that I'd been interested in doing since before 1980 or so of can one answer questions about the world from the knowledge that we've accumulated in the world? And I had made some sort of science advances that made me convinced that there wasn't sort of a bright line difference between intelligence and mere computation. 语法解析

25:00

And so it's like, well, if I'm right about that sort of almost philosophical claim, it should be the case that we can build a system that does this kind of answering of questions based on the knowledge that our civilization has accumulated. So in the mid-aughts, I decided, okay, it's time to actually try and do this. 语法解析

25:21

And so that led to this system that came out in 2009 called Wolfram Alpha. And Wolfram Alpha is kind of the idea is to take pure natural language, turn it into a precise computational language, then be able to compute answers from that and tell them to people. 语法解析

25:42

And when we were introducing Wolfram Alpha, it was sort of interesting because AI was absolutely dead at the time. Everybody thought nothing like this is going to work. People had tried to make question answering systems with various kinds of AI techniques, statistical, symbolic, whatever. They tried to do that for decades. It had never worked. 语法解析

26:03

It was kind of particularly notable, I mean, to tell a sort of story that indicates what was happening. It was a couple of weeks before we released Wolfram Alpha. I happened to see Marvin Minsky, who I mentioned earlier, who was sort of a big AI pioneer. And so I say to Marvin, you've got this cool new thing that's coming out. Let me show it to you. So I kind of show him a couple of things, and he's like, change the subject, not interested. 语法解析

26:27

It's like, because for him, he'd seen a zillion examples of people saying, I built a question answering system. And so I said, look, Marvin, you know, you should look more carefully. This time it actually works. And so he types a few more things and then he's like, oh my God, it actually works. And he's running around this event that we were at telling people, you've got to see this. You've got to see this. It actually works. 语法解析

26:49

It was interesting because it was a time when, as I say, that was kind of a kind of a snapshot of the fact that at the time people just thought I was dead in 2009. And it was that was, you know, and so. 语法解析

27:08

With Wolfram Alpha, we for the first time solved the natural language understanding problem of taking plain text. People had thought, well, just make a computer understand that. But it wasn't clear what it meant to make a computer understand it. I only realized this after we'd built what we built. We had this huge advantage because we already had this underlying computational language that we built, came out in 1988. 语法解析

27:37

Mathematica and what's now Wolfram Language, which is this kind of way of representing things in the world computationally. We already had that kind of precise computational representation of the world. And so our natural language understanding wasn't just abstractly get the computer to understand this. It was translate what those pesky humans say into this precise computational language that we can then do computations with. 语法解析

28:03

So that was kind of one piece of that story. Now, the next question, the next thing is sort of what was the tradition that ChatGPT came out of? It's a very different tradition, very different kind of line of work. It kind of started, I guess, in the 1940s when people were doing cryptanalysis in World War II. 语法解析

28:25

And the question was, when you do cryptanalysis, you know, when you encrypt a message, you're turning it from meaningful English letters or the English sequence of English letters to something where the letters look completely random. What you have to do to fish out the English is to statistically figure out what was done. 语法解析

28:47

And so the sort of critical idea was, well, English isn't just a random sequence of letters. English has certain statistical regularities. Like if you see a Q, there's going to be a U next, probably. Yeah, right. You see more E's than you see X's and so on. 语法解析

29:05

Okay, so a chap called Claude Shannon worked out this thing he called information theory. It actually worked out, I think, during the war, probably Alan Turing was somewhat involved in this also, wrote this paper in 1948, I think, introducing information theory and this idea of sort of the statistics of things like language. 语法解析

29:26

So, given that you had the statistics of language, you knew which letters were more common, which pairs of letters were more common, and so on, you could start imagining generating language statistically. You say, okay, if you happen to randomly pick a Q, the next letter is going to be a U, etc., etc., etc., 语法解析

29:44

So that was kind of a way of thinking about language. And so the thought immediately came, well, maybe we can get language to we just generate language statistically and it'll be meaningful. 语法解析

29:57

Well, it didn't work out that way. There was a whole sequence of efforts. It was particularly of interest people doing speech recognition where you're trying to figure out, you know, we hear these speech sounds and we can sort of statistically work out, oh, that speech sound is roughly a vowel. But then the question was, well, how would you assemble those sort of somewhat imperfect sounds? 语法解析

30:18

things about well that might have been an l that might have been an r whatever else how would you assemble those rather imperfect kind of guesses about what those phonemes what those fragments of speech were like into something which could actually be a meaningful piece of text and so people are very interested in kind of the statistical structure of text to be able to do those kinds of things so that became sort of a whole area well what happened in the um 语法解析

30:46

what was it, by the around maybe the early 2010s, was people had done, there were speech recognition systems that worked by taking speech, breaking it down into phonemes, different speech sounds, recognizing particular patterns of phonemes and so on, trying to do the statistical reconstruction of language. It was a big sort of painful process. 语法解析

31:14

People started just trying to do neural nets to just go straight from the audio you hear to the text that's being generated. And turns out it worked. 语法解析

31:25

And so just as image recognition got solved, so kind of speech to text kind of got solved too. And that was by 2010 when Siri came into the world and we were kind of the computational back end for Siri and it had a kind of voice recognition system at the front end. 语法解析

31:46

And we were constantly frustrated because it would send, you know, whenever people were asking about math and, you know, pi and math, the voice-to-text system was sending P-I-E as the word that they were saying. But that got solved. But it was, so that was sort of the next success, next big success of neural nets was 语法解析

32:09

Then there was the question of could you do what was called sequence prediction? Could you do a better job of knowing, given that you have a piece of text that starts this way, what will come next? And what existed, even in like 2021, times like that, of sequence prediction was really cruddy. 语法解析

32:29

It really didn't work well at all. We tried to use it a bunch for doing things like predicting pieces of code for autocompletes, things like this. Really did not work well. 语法解析

32:41

And then, well, the guys at OpenAI basically collected this huge amount of training data. There was one kind of technical idea, which I'm not sure how significant it will be in the long view of history, this idea of transformer nets. The question is, 语法解析

33:04

If you look at a slice of brain, so to speak, I'm too squeamish to actually do that in real life, but you'll see that the neurons are all sort of very randomly connected to each other. It's as if every neuron is more or less connected to every other. But there are areas like on the retina, for example, where kind of the, well, in the retina, it's photoreceptors, but then in the visual cortex, it's actual neurons where things are connected more locally. If you're dealing with an image, 语法解析

33:32

the image has all its pixels kind of laid out in two dimensions, and you want to kind of have your neurons kind of laid out in two dimensions as well. That led to so-called convolutional neural nets, which are the things that were used for this image identification problem. 语法解析

33:48

When it comes to things like language, language has the feature that it's sequential. It's just this stream of letters or words or sounds or whatever coming out. And there were these things called transformer nets, which are networks that only have connections sequentially, but their connections can be quite long range, just as a word somewhere in a sentence can refer back to a word that's much earlier in the sentence. 语法解析

34:16

So anyway, big sort of training effort was done using a lot of material from the web and elsewhere. And… But if they take it from Google as well, couldn't Google have some open source information out there that OpenAI used for their initial algorithm? 语法解析

34:36

I don't know. Google had been, when the original kind of image identification thing was done, Google had swooped in and sort of bought the company that was created overnight, so to speak, by the academic researchers who'd been working on that. So Google had sort of developed that. And I think we're using it quite a bit for search because when you're ranking, it's always this problem of how do you rank 语法解析

35:02

the the pages that come out of a search you know what's the most relevant and so on and there are many different signals you can use and figuring out what sig how to combine those signals is something that's a pretty good case for using neural nets and i think they were using neural nets for a while for doing that um and that was uh um but but yes the the um uh there had been i mean 语法解析

35:27

Really, there was quite a lot of excitement about neural nets going to be able to do things, but it wasn't clear what they were going to be able to do. And when ChatGBT sort of arrived, I remember chatting with the folks who worked on it just after it came out, and sort of my obvious question was, did you know it was going to work? They were like, no. And in fact, probably had they known it was going to work as well as it did, 语法解析

35:54

they would have tried to constrain it in a lot more ways than they did. And a lot of things they did were kind of sort of what you do if you're just trying to do the experiment, not what you do if you're trying to build a production system. Just to see if it works. So then the big surprise was that it worked. I mean, I kind of liken it to sort of a history of technology thing, which was the invention of the telephone. 语法解析

36:18

People had known that in principle you could transmit sounds over electrical wires, but when people had done that, you would try and listen at one end of the wire and you wouldn't understand anything that was being said. Eventually, Alexander Graham Bell found a bunch of hacks that, again, I don't think he knew were going to work. 语法解析

36:42

But suddenly, people could actually understand what was being said. I don't think it sounded very good, but it was good enough that people could understand it. And I think the same kind of thing happened with neural nets. It got good enough that it was like you could read the text and it read like meaningful texts that a human might write. 语法解析

37:00

And that was sort of the chat GPT moment where kind of the being able to generate meaningful text, being able to kind of start from a prompt 语法解析

37:12

You know, if on the web it was like this question was asked, this was the answer given, you could expect it to sort of statistically follow that. Now, there was an additional trick, which was this idea of reinforcement learning and particularly reinforcement learning with human feedback, which went beyond just how would the sentence continue. You know, the cat sat on the, what's the next word? Statistically from the web, it's probably mat. 语法解析

37:39

et cetera, et cetera, et cetera. But the idea of get the thing to actually do what it's told and do things like answer questions. Um, that was a thing that was sort of a, a, a, another layer of, of training that, uh, uh, that was done and, and it worked out. And, you know, I was when, right when chat GBT came out, Oh, so many people were asking me, how does this work? You know, why is this working? Et cetera, et cetera, et cetera. So I eventually wrote what turned into a little book about, um, 语法解析

38:09

What is it called? What, what, 语法解析

38:11

What is ChatGPT doing and why does it work? Which became very popular. It was kind of, in a sense, disappointing for me because it took me a week to write. And there are lots of other things I've written that took me years to write. And the one that took me a week to write, people seem to like very much. It's now translated into lots of languages and things. I don't know whether done by humans or AIs. I'm not sure. But anyway, it's always fun to see the book covers from different countries and so on. 语法解析

38:40

But the two things that sort of came out of that, one was why does it actually work? And the other was how does it sort of fit into the ecosystem of the world? 语法解析

38:56

the why does it work question, the thing I realized is that in a sense, the reason it works and the thing that surprises us so much is a little bit like the thing with the tanks in the daylight or not. There's something about language that we hadn't noticed that actually is more regular than we had imagined. 语法解析

39:15

And, you know, we're all used to the idea that we do grammar and we know that, you know, in English sentences are formed with, you know, noun, verb, noun, the different parts of speech and different combinations. But there's more to it. Most sentences that is noun, verb, noun are completely meaningless. But there's a kind of semantic grammar, a grammar that's based on meaning that says what noun, what verb, what noun can go together. 语法解析

39:40

And there's, I think, a lot of regularity in that structure. There's a kind of a semantic grammar of language that is effectively what ChatGPT discovered statistically by looking at the web. And people were very surprised that, for example, it could discover logic, that it could make arguments that were made logical sense. I think the way it did that 语法解析

40:02

is the same way that Aristotle discovered logic back a couple of thousand years ago, which was you just look at a bunch of sentences that people say and you ask what is the kind of structural pattern that they have led to syllogisms in Aristotle's time. Somewhere, the statistical knowledge that Chachi BT has 语法解析

40:22

is of things like syllogisms and that is okay so it's it's able to produce something that is logical just like it can produce that the next word after the cat sat on the is mapped but it's it's it's kind of learnt sort of the semantic grammar of language which includes things like logic so the thing that was was very clear was that you know chat gbt good at generating text 语法解析

40:48

It was also good at generating, well, one of the things that we had from Wolfram Alpha was a system that could take text and compute from it. So that was a pretty nice combination because you can take the things that ChatGPT is producing as text, which includes ChatGPT making up a question that it wants to get answered. And the thing we realized, well, very immediately actually, was that 语法解析

41:17

that you could use English effectively as the transport layer between the AI and our computational system, and you could have ChatGPT kind of call Wolfram Alpha as a tool. One of the surprises early on was that not only could it do that using English as a transport layer, also our Wolfram language 语法解析

41:41

which, you know, I've been building for so many years and lots and lots of people use as a language for doing computation as a way to represent kind of one's thinking computationally. Well, ChatGPT could do the same thing. It can write Wolfram language code. And so we built, you know, at first it was kind of a plug-in to ChatGPT and then lots and lots of other things where kind of the picture is what… 语法解析

42:08

what the LLM, what the large language model is doing is, by the way, maybe a historical point that perhaps is of interest is why they're called large language models. 语法解析

42:20

Well, it's because there were also language models, which were the things that people used to try and figure out, you know, what would the next word be statistically to be able to disentangle speech to text and so on. So there'd been a long tradition of language models, often so-called Markov models. 语法解析

42:39

where you essentially have these different states and you're going from several different sort of states of a system deciding what the next letter will be. So large language models were like, let's do this with neural nets and put billions of neural net weights in there. That's a large language model. So LLM's large language models of which ChatGPT is an example. There are now many, many others. 语法解析

43:00

we have a nice little website where we every week do a kind of a rating of how these various LLMs are doing and it's kind of remarkable it's an active enough field that practically every week there are changes at the top so to speak in terms of what the winning LLM is this particular week but it's 语法解析

43:20

It's the thing that sort of has emerged is that LLMs are really good at doing this kind of linguistic interface, at dealing with things language sort of for languages sake. 语法解析

43:36

They're not good at doing things which require computation. That's not what they're intended for. That's not what their structure is. It's actually, we have a big clue that they're not going to be good at that because humans aren't good at that either. You know, we don't get to be able to do, you know, run programs in our minds and things like this. 语法解析

43:53

And so it's been nice for us because I've just spent four decades building this whole system for being able to do broad kinds of computations about the world. I mean, programming languages are intended as a way to take what computers can do and give humans a way to tell the computer in its terms what to do. 语法解析

44:17

What I've been building for a long time is this kind of computational language which tries to describe the world computationally, where the world could be cities or movies or images or geography or whatever else, and be able to represent the world computationally so that one can compute about it. 语法解析

44:38

And that's something where it's sort of a higher level thing than has been attempted with programming languages. This has been, I mean, I use the system we built all the time, so do a few million other people, you know, mostly for research and development and those kinds of things, although there are an increasing number of kind of practical systems in the world that use our tech underneath. It's always so strange when you've built a bunch of tech and 语法解析

45:06

And then you are a consumer user of some big system in the world. And you know that your tech is underneath it, but it doesn't do you any good. Still, the consumer system does what it does. But in any case, the thing that… 语法解析

45:21

you know, it's been interesting for us because there's been sort of this, you know, we used to have human users primarily. Now we also have AI users where, you know, LLMs are interfacing with humans through natural language, but underneath they're doing these computational things using our tools. It's been an interesting thing to watch that happen. And it's something where, you know, what LLMs kind of, 语法解析

45:49

are readily able to do is quite different from what you can readily do with sort of raw computation. The combination is really powerful. And that combination, you know, one of the things that will eventually happen, although I don't yet know how to do it, is to make a more sort of fine-grained integration of those things. Because right now, the LLM is going along, it's generating a bunch of text, then it generates kind of 语法解析

46:16

the things it needs to call our system as a tool, gets the results back, then keeps going and takes those results and sort of weaves them into the text that it's writing. What's the gap then from what we have right now with what people are now coining, you know, AGR, artificial general intelligence? What is the technological gap between what we have to 语法解析

46:44

And first, actually maybe define what AGI means for you. 语法解析

46:47

before we do that? And then what is the gap that we have from where we are today to what we need to be, where we need to be to get the AGI? And what's that going to look like for the consumer's perspective? Because most people don't really know the technical details, I guess. Nobody knows what AGI really means. It's a buzzword that it's kind of like, you know, first AI was a buzzword and people didn't quite know what that meant, except they thought it meant things that do stuff that's kind of like what people do. 语法解析

47:14

And it's like, well, now we've got things that do lots of stuff that people do, whether it's being able to generate language, drive cars, whatever else. But still, there's got to be something else that people do. And that must be AGI, some general intelligence thing. It's kind of ironic that I just realized this as I'm talking to you, that the term general intelligence was, I think, coined in the 1930s. 语法解析

47:42

And it came in when the same time as the concept of IQ came in, a very, in my opinion, a very troublesome concept. 语法解析

47:52

But it was at a time when people were trying to figure out, I don't know, for recruits for the army and things like this. It's like, was there a way, just like you could say this person is, you know, five foot nine inches tall. It's, you know, let's give a number. Let's give a metric to, you know, quotes how smart are they? In my opinion, a rather doomed concept. 语法解析

48:14

but that led to this thing called G, the coefficient of general intelligence. And then people started making tests. So I think that's where this comes from, which is a very doomed kind of place. But anyway, that, that, but what does it mean? And what, you know, there are, the thing is that there are, you have to define what do humans do? How do we make something that does more of what humans do? And, and, 语法解析

48:42

One of the things that's difficult there is if you say, well, do I have an AI that can pick something up with its hand? Okay, that's a human-like thing. Do I have an AI that can have an emotional response? That's a human-like thing. Do I have an AI that feels its own mortality? 语法解析

49:07

you know all these things in the end the only thing that is going to check all the boxes for being like a human is a human so you know you'll be able to check more boxes the the thing that's the the box of electronics sitting on your desk it's going to have certain things that can be human-like but other things that won't be and in the end if you say well you know what we mean by agi is something that's human-like in all respects 语法解析

49:36

That's kind of a doomed idea because the only thing that's going to be human-like in all respects is a human. So now the question is, well, what happens? What does it look like as we go in the trajectory that we're going on so far of we've got these neural nets, they've got billions of weights in them, they've got millions of neurons, they've got… What happens when we make that bigger, for example? 语法解析

50:01

And we have kind of a model of that, which is a little bit kind of strange to think about, which is, you know, you start off with, you know, I don't know, a fruit fly, for example. It has 130,000 neurons in its brain. We have about 100 billion neurons in our brains. You know, cats and dogs have, I don't know how many, maybe a billion or so neurons in their brains. Well, you know, we get to do some stuff. We get to do a whole bunch of stuff that fruit flies don't get to do. 语法解析

50:28

And in particular, one of the ones we're proud of is human language. That's one of the things that sort of made our civilization possible and so on. And, you know, cats and dogs don't quite have that. They maybe have, you know, sit, fetch and so on, but they don't have the sort of compositional language that we have being able to put words together in arbitrary combinations to make sentences and so on. And so, again, 语法解析

50:50

A question, a reasonable question to ask is, well, you know, if we're going in the trajectory we're going and we've got these neural nets that have, you know, human brain-ish numbers of neurons and so on, what would happen if there were even more than that? What is the next step from cats and dogs to humans to the next level of minds? What does that look like? 语法解析

51:13

And that's kind of a, you know, that's kind of a thing that you might wonder, sort of, you know, we've got something that is human-like at some level right now. We can go on checking the boxes of giving it, you know, letting it be, you know, my guess is within, you know, the next big thing that will get solved in AI is robotics. And, you know, it's been super difficult to get, you know, 语法解析

51:39

robot hands to pick things up and be able to pack boxes with whatever stuff you want from a warehouse, things like this. It's something we humans manage to do fairly easily. It's something that it's been a little difficult to get training data. For the stuff that humans write well, 语法解析

51:59

well, there's a trillion words on the web and things like this. It's a little hard to get information. You can watch videos and things like that to see how one manipulates things in the world. I don't know exactly how that's going to work out, but my guess is that that will be another thing that people will be like, oh my gosh, this can now be done with AI, and that will have a whole bunch of consequences for practical things that are possible in the world. 语法解析

52:27

But that's sort of a different story. - But is that AGI or is that just AI in a different form, in a physical form through hardware? - I mean the only thing that's, what is the limit of AI? What I'm saying is the limit of AI, if you define intelligence to be the thing that humans have, is pretty much humans. 语法解析

52:48

Now the question is, if you're going that trajectory sort of technologically, can you zoom right past the humans? Can you get to something that is sort of the greater, I don't know whether we call it intelligence, but the greater capability that humans have? And I think the answer is, well, yes. 语法解析

53:10

But then the question is, well, okay, what does that, what is that like when, you know, if we have a thing that is computing much more than humans compute, you know, we have a hundred billion neurons. We have a thing with a hundred trillion neurons. We have something that's doing not the number of computations that we do in our brains every second, but zillions of times more than that. What is that like? Well, the fundamental thing is it's not very human like. 语法解析

53:36

And we have a really good model for what it's like, which is the natural world. The natural world is absolutely full of things that compute much faster than brains compute. You look at any kind of babbling brook or something like this, all that fluid turbulence and so on that's happening in the water, that you can think of as computation, just like the electrical signals in our brains we can think of as computations. There's lots of computation going on in the babbling brook. 语法解析

54:05

It's computation that isn't very human-like. It's not the same kind of thing as happens in brains, but it's a lot of computation. And so I think one of the things that one sees is that getting more computation is something that one can really expect. 语法解析

54:23

One can, it's in fact, one of the things I've done in science for a long time, one of sort of my big discoveries and directions in science is to understand what computation in the wild looks like. You know, when we do computation, usually we write programs, we set up programs to do particular things that we want to do, so to speak. 语法解析

54:46

But a question that I got interested in the beginning of the 1980s was, what does a program that you just pick at random, what does it typically do? You might assume if it's a tiny little program that it would just do tiny little simple things. 语法解析

55:00

The huge surprise that took me a while to kind of really get used to is that's just not true. In the computational universe of possible programs, even very simple programs can do incredibly complicated things. And I sort of realized that from a science point of view, that's kind of the secret that nature uses to make all this complexity we see in nature. But it's also something that is, in a sense, it's sort of computation, 语法解析

55:27

It's even what you might think of as intelligence, but happening in a very non-human way. It's just there is a lot going on. You can look at these things. You can say, wow, that's really complex and intricate and interesting, but I don't really understand it. 语法解析

55:44

I mean, we can think of science. In fact, the mission of science, in a sense, is to take what exists in the natural world and kind of make a bridge between that and what we can understand with our finite minds. I mean, it's like saying, oh, you know, we're not going to in our minds, we're not going to understand what every molecule in the in the. 语法解析

56:03

you know, in the river does, for example, but we can say, we roughly can talk about certain laws of fluid mechanics that govern roughly what the, what the river does. We're making a bridge between what's actually happening in nature and what fits in our finite minds. 语法解析

56:20

So now we have this picture of what's happening. I'm realizing your podcast is called Growth Minds, and I think this is a highly appropriate topic for the question of what higher minds are like. It seems like a highly appropriate topic for your podcast. Yes. 语法解析

56:43

Yeah, hence the conversation with you. Particularly this notion of what does the greater mind kind of look like? And, you know, there's this sort of question of what, if we put in more neurons, if we make a bigger mind, so to speak, what kind of thing will it be like? And I think the thing we have to realize is it will be very non-human. And the question of whether it will 语法解析

57:13

you know, it may be that it does all these things that we could try and make a bridge to from what it does to what we understand. And we could say, wow, we're very impressed. That's scientifically interesting, but it's like what it's doing is it's not like a human just, you know, running, you know, there'll be some things where it will be a human running faster, but mostly I think it will be things that are extremely non-human and, and, 语法解析

57:41

where the real question is, how do we take that non-human computation and kind of lasso it into something we care about? It's very analogous. 语法解析

57:54

Yes. So the definition of AGI of people saying AI reaching human intelligence or human level intelligence, do you find that's like the wrong way of even looking at the progression of AI? Because AI is heading towards a place where it'll actually be the very non-human intelligence from what you're saying. It'll just transcend it. When I first started paying attention to AI, late 1970s, there was this kind of checklist of things. When one has this, we will have AI. 语法解析

58:23

Like one of the things was being able to do symbolic mathematics. Another was being able to do question answering. Okay. You go down the checklist. We've got those things, but still people say, but there's something different about humans. Well, yes, there's something different about humans. Humans are humans and they, you know, they eat and drink and, and, you know, and, and die and those kinds of things. It's different from what this electronic technology, 语法解析

58:50

device, software system, whatever does. And so, you know, people, I mean, throughout history, people have kind of wanted to search for what is so incredibly special about us humans that is sort of fundamentally special, not special in the particulars of we are humans that happen to be the way we are, but something where we're sort of on some chart that you could make. We're this data point that's just way out abstractly. 语法解析

59:17

And I don't think that, I mean, you know, the lesson of the history of science has been we keep on getting humbled in the fact that, no, we're not special in that way or this way or whatever. And I think the way in which we're special is that we are precisely the way we are, so to speak. We have all the particulars that we have of, you know, two eyes and ears and all this kind of thing. We have, we are the particular thing that we are. 语法解析

59:42

And yes, you could make sort of an AI that more and more closely approximates the particular thing we are. You know, the humanoid robot that has experience of the world is similar to ours because it's walking around the same way we are. The thing with two eyes that has a similar experience. If we had a thing with a million eyes that was seeing all kinds of things from everywhere around the world, 语法解析

01:00:06

That wouldn't be a very human-like experience. The kinds of things that the million-eyed AI would kind of think were useful to think about are probably very different from the ones we would think were useful to think about. Yeah, it's interesting that there's so much fear brewing around society, around how AI is going to be replacing humans, yet we keep 语法解析

01:00:27

putting this goalpost up of trying to see how much we can replicate what AI is with human characteristics as if like we are actually trying to replace ourselves yet there's fear brewing at the same time. It's kind of ironic. If one is saying, what's our technological objective? Oh, it's to make this thing that's human-like. That's a goalpost you can see type thing. 语法解析

01:00:53

But that's the wrong goalpost you're saying. If the whole idea of us fear brewing that AI is going to replace us, shouldn't we try to make sure that humans exist as our individual selves and make AI this completely different thing that can help us rather than trying to create human level intelligence where we will actually just be negligible? We are negligible relative to nature, but we seem to have just a fine time in that position. 语法解析

01:01:19

You know, relative to the relative to the universe, we are negligible relative to even the computation that happens, you know, around us on the earth. We're negligible. Yet we feel pretty pleased with ourselves regardless. And I think that's the that's the way to think about it is that, you know, it's like you could, you know, you could we we exist in a certain niche. That is the one that is the human niche. 语法解析

01:01:48

And we can say, well, gosh, we'd like to be on other planets. We'd like to be this, that, and the other. It doesn't, you know, it's still the case that the one, the niche we care the most about is this human niche. Now, you could say abstractly for the universe, it's more significant to the universe if our AIs are running around, you know, 语法解析

01:02:10

going to other stars or whatever else. I don't know what it means to say it's more significant for the universe because that's not like there's an abstract definition of, you know, you're kind of asking for the ethics of the universe, which is something that doesn't really have a definition. These kinds of questions of what's it significant for? What's it meaningful for? What's it kind of right for? Those are questions which in the end, they're anchored in us. 语法解析

01:02:36

You can't ask those questions abstractly in a meaningful way. I mean, I think that one of the things to realize is, what is technology? Technology is an attempt to take what exists in the world and kind of take pieces of it that we can use for human purposes. And different kinds of things, when people discover magnetic rocks, 语法解析

01:03:01

I don't think they knew what they were good for at the beginning. When people discovered liquid crystals, they certainly didn't know what they were good for at the beginning. And then it was realized, well, you can make a display out of liquid crystals. And that was a use case for that. And so similarly, there's sort of a vast, there's all this stuff in the computational universe, all of this computation in the wild. Most of it, we don't know how to use it for anything that we humans care about. 语法解析

01:03:27

What happens in the progress of science and technology and society is that there starts to be, you know, we gradually get to have more and more things that we think we care about. And we probably forget about other ones that we don't care about anymore. I mean, you know, there are lots of things where people would say, 语法解析

01:03:46

I can't imagine that you could make a living playing video games, streaming video games or something. It's a thing where… I can't imagine people will find it interesting to do this thing that's a little video game or something. They're these things that we… 语法解析

01:04:06

that happen that cause us to decide there is a purpose to that. I mean, if we look at what we do today and we imagine it from a thousand year ago lens, a lot of what we do today would seem absolutely pointless to somebody from a thousand years ago. I mean, I think one of my favorite examples of that is, you know, walking on a treadmill, you know, explain to somebody from a thousand years ago why you walk on a treadmill. 语法解析

01:04:32

well, it's to improve my health, so I'll live longer, so this, so that. It's like, why does that make any sense? We do what we do for the greater glory of God or whatever, and this is our sort of brief time on earth type thing, and why are we, why are you? There are many, you could look at different things that people might have thought were significant a thousand years ago, 语法解析

01:04:58

none of which would explain why you would walk on a treadmill, why you, you know, walking and not getting anywhere type thing. And I think that's, that's something you see in the progress of, of society and civilization is you see sort of the things that seem purposeful and meaningful gradually change. And, you know, the, the thing, and what, what does, you know, AI or automation in general do, it takes the things that humans want to do and it somehow makes them easier to do. And, and, 语法解析

01:05:27

The thing that you might say was that maybe there'll be no need for the humans because everything's going to be easy. But then you still have the question, well, what do you actually want to do? And that's something where there's no AI that's going to answer that question because there is no answer. There's no abstract answer to the question of what should you do. 语法解析

01:05:45

what should the universe do? The universe does what the universe does. It doesn't, you know, there's no, there's this choice of what to do is something that is sort of the ultimately quintessentially human thing, because there are many things you could do and it's, it's up to, well, you could pick, you know, it could be cats and dogs that were deciding too, or it could be aliens that were deciding, but something, some, 语法解析

01:06:11

arbitrary thing has to decide what it is that you want to do. So, I mean, I think the way I see it, the sort of the future of, well, the sort of arc of technology has always been to take things humans want to do and make them more automatic, easier to do. 语法解析

01:06:31

And so what does that do for the humans? Well, you know, I looked a while ago at what's happened to jobs that humans do. Like in the US, for example, there's data back to like 1850 or so of what jobs do people do. Back in 1850, most people were doing agriculture. Most people were actually, you know, plowing fields and things like this. That almost all got automated. 语法解析

01:06:55

So then what happened to the economy? Well, what happened is that what was a big chunk of the pie fragmented into a zillion different areas. And you see that typically as economies get more advanced, there are more different categories of jobs that develop. And that's sort of typically the pattern of, you know, some category of thing was hard to do. A lot of people were just on the ground doing it. Then we automated it. 语法解析

01:07:22

Then that very process of automation opened up lots of other things that were possible. And, you know, it was… 语法解析

01:07:32

And that then, you know, was that then people found things to do that in the end, the front line usually ends up being things people have to make choices about. That's places where you can't feed in. There's no abstract way to feed that in. Now, you can be in a situation where people say, well, let's just abdicate those choices to the AIs. That's kind of a bad situation because then, in a sense, you're saying, let's just take 语法解析

01:08:01

society and civilization as it has been, as captured by the AIs, and let's just run the same thing over and over and over again. It's kind of the humans never get to sort of do anything that is kind of new and different. Now, then you ask the question, well, why can't the AIs be creative? It's very trivial for an AI to be creative. It just has to pick a random number, and that's doing something that is creative. 语法解析

01:08:30

Now, the question is, is the random thing that it picks, is that something a human will care about? An interesting question that's kind of a frontier question right now is, for example, something I've looked at quite a bit, is if you look at mathematical theorems, it's pretty easy to get a computational system to just go spewing out zillions and zillions of theorems, billions of theorems. They're all true theorems. You might say, wow, that's exciting. That's making progress in math. 语法解析

01:08:58

but it's not making progress in math that people care about. Because most of those theorems, people look at them and say, okay, I guess it's true, but so what? 语法解析

01:09:07

And what counts as kind of math that humans care about tends to be this kind of prong that gets built, this tower that gets built. Humans care about this. Okay, given that they care about that, now they care about this. And you're kind of building it up that way. It's the same kind of thing. Well, you see it all over the place. I mean, I did a thing a year or so ago. I was curious about… 语法解析

01:09:33

sort of alien minds and they're the mental imagery of alien minds. So what does that mean? So the, you know, you can make, you can get an image generation AI system. You tell it a description, you know, a cat in a party hat, it will generate a picture, it will generate zillions of pictures that are all would match the description, a cat in a party hat. 语法解析

01:09:58

Okay, so that's what the AI would do, you know, having learned from humans who've labeled their pictures and so on. So now you ask the question, well, what if you take sort of the, if you modify the innards of the AI, or alternatively, if you take kind of the internal description that it has of cat in a party hat and you start tweaking that? Well, you move away from the concept of cat in a party hat into what I was calling inter-concept space. 语法解析

01:10:26

you move away from these human defined concepts to concepts which exist in the mind of the AI, but are not familiar to us humans. 语法解析

01:10:36

And what you see, I was referring to it as cat island. You see this kind of island in that interconcept space that is around the cat concept. We humans recognize that as those pictures of cats. Then you move off into this interconcept space, which is a space that is kind of, in a sense, just as meaningful to the AI mind as the cat point in reality. 语法解析

01:11:03

into concept space, but it's not very meaningful to us. You look at these pictures and it's like, I don't know, it's a picture with a bunch of stripes and dots and squares and circles and this and that. I don't know what it's of. I don't know why I care about it. 语法解析

01:11:18

But you also got to realize that some of those pictures, actually, I was amused to see that somebody took some of the pictures that I made in that post, and they're now in some art exhibit somewhere in Paris. So that style of making pictures is now, at least in one minor case, I think actually there have been some others of this as well, but considered as art. So what was something in inter-concept space 语法解析

01:11:48

might one day develop into the such and such style of art that then we would know about. And then it becomes a concept that's part of sort of the concept space for us humans. So the thing to realize there is that the AIs are frolicking around into concept space, so to speak. That what happens inside an AI is full of these things from inter-concept space. 语法解析

01:12:13

They're just extremely non-human, just like nature is doing lots of non-human stuff. And the question of whether, you know, it's a, uh, the, you know, if we open up an AI as it exists today and we say, what's going on inside here is something I've done a bunch of work on recently, actually. It's, uh, it is interesting because it is essentially, uh, 语法解析

01:12:38

They're these kind of lumps of irreducible computation that you find there. And what's happened in the training of the AI is it's kind of fitted together these lumps of irreducible computation to do the things we want it to do. Kind of the analogy I've been using is it's kind of like 语法解析

01:12:53

It's building a wall out of rocks. It's building a stone wall, so to speak. It's taking these lumps of computation that sort of happen to fit in and more or less correspond to the thing you need to tell cats from dogs or whatever. And it's putting a bunch of those together so that you get something that is kind of that achieves the objective we want, like distinguish cats from dogs. What's happening inside? 语法解析

01:13:20

is something that in some ways is sort of randomly picked because it's which rock happened to be lying around. In some ways, it's incomprehensible because it's full of these kind of lumps of irreducible computation. And you can say, well, gosh, what will the world be like when it's dominated by AIs that are doing these things that are incomprehensible to humans? 语法解析

01:13:43

I think that feeling is very much like the feeling of living in the natural world. I mean, there's been this brief period in history where the technology we build, we expect to be able to understand. In the past, when people were getting transported around by riding horses, you could know something about how to get the horse to do what you want. 语法解析

01:14:04

but knowing how the horse works inside was not something you really cared about. You were able to use the horse for something that was useful to you, but you didn't know mechanistically how the horse worked inside. Post-industrial revolution, for a brief time, we've been operating machines that are simple enough that we know what's going on inside. We're now back to a situation where to make 语法解析

01:14:28

systems that can really make use of computation as it can best be made use of, we sort of inevitably have to deal with this kind of irreducible computation that we can't readily understand with our minds. We can't have a kind of narrative explanation of what's happening inside. 语法解析

01:14:43

So now the question is, well, how do you deal with that? Well, it's the same thing that we've done with technology forever and ever, which is there are things that are in principle possible that might be possible in the natural world that might be possible in the computational world. Now, how do we use those for things that align with the things we care about? 语法解析

01:15:00

And some part of what will happen will be stuff we don't understand that doesn't align with what we want. I mean, in the natural world, there are tornadoes and things that don't align with what we want. We end up, you know, being able to predict them and, you know, having tornado shelters and things like that to kind of exist alongside them. 语法解析

01:15:20

And no doubt there will be things like that that come out of the computational universe, so to speak, perhaps in some sense they already are. And I think that's the view of how to conceptualize what it's like to coexist with something which you could think of as 语法解析

01:15:42

a greater intelligence, I suppose. It's certainly greater computationally, but that's something extremely familiar from the natural world. As someone that has a very high level of intelligence that probably understands things that most people don't really at a deep level, even with AI and how it's evolved, you have a very good understanding of it. Do you fear a world of AI evolvement that 语法解析

01:16:13

exists where you don't understand? Do you fear that world? I make these little tiny programs where I look at the program and it's totally trivial. And then I run it and it does things that I don't understand. I mean, I've lived that myself for 45 years or so. So at the beginning, that was very bizarre. It was very much like this can't be happening. It can't be that this tiny little program makes all this amazing stuff. 语法解析

01:16:39

But, you know, you kind of that's the way nature is. That's one sort of gets used to it. Now, you know, for me, the you know, my general approach to things is use any tool you can you can. And so, you know, for me, you know, one of the things that's funny, I was just realizing this, actually, people are kind of sort of say, well, if the AIs get to be really smart, they'll you know, what will that be like for humans, so to speak? 语法解析

01:17:07

I myself happen to have been in this situation that I've sort of created for myself where I've been building tools that can enhance my ability to think about things. I've been building such tools for, I don't know, 45 years or so. And so the tools I built allow me to take ideas I have and sort of see their consequences really pretty efficiently. 语法解析

01:17:33

I mean, I can see that if once I have an idea, once I know what direction I want to go in, it's like type, type, type, and I'll have a little Wolfram language program and I'll run it and it'll sort of let me work out that idea. It's a very short process. 语法解析

01:17:51

It's actually got even shorter recently because we built this notebook assistant system that is based on LLMs and other technology that we made that allows one to kind of even more efficiently go from kind of a thought you have to computational language code that can actually run. 语法解析

01:18:08

So it's a funny thing because I guess I've lived this perhaps ahead of where other people would have lived it, although there are plenty of people who use our tech who probably are in a similar position to mine, where it's like you have an idea, you want to see its consequences. That's a very short path. It's something where normally, you know, if you didn't have, you know, I've been building technology for, 语法解析

01:18:32

partly for myself, partly for everybody else, that really shortens that path. It really automates the going from idea to reality, at least for things that you can implement on a computer. It's not building rockets and things like that. It's just doing the kinds of things that I'm interested in from an intellectual point of view. But so that's sort of this thing about you imagine it and then you make it real. 语法解析

01:19:01

That's a thing I've been living for a long time. I think people will increasingly come to expect that. And that's not a that's not sort of a dehumanizing thing. In a sense, it's a human amplifying thing, because what gets more important is, well, what is it you want to do? What's the idea that you have? 语法解析

01:19:21

And I think that the question of, well, how does it work inside? I mean, that ship sailed a long time ago. I mean, you know, people, you know, who understands what in detail is going on inside their computer? 语法解析

01:19:34

Even, you know, as I say, when I do science and set up these simple programs, they're always doing things I don't understand. Always. I mean, you know, every when I'm working on those kinds of things, practically every day, I'll be sort of humbled by the fact that I imagine what the thing is going to do and then does something I didn't imagine. 语法解析

01:19:54

Right. How do you answer that, though, from like maybe like more of a nihilistic or doomsday level questioning? You mentioned nature doesn't always we don't understand nature, but nature can certainly kill us. How do you think about that question when you have you mentioned not understanding how things are going to do what how things are going to react from a software perspective? But as we get into hardware and robotics, things that can physically hurt us. 语法解析

01:20:19

What are your thoughts around that as we become less necessary perhaps for AI? Look, there are plenty of self-driving vehicles right now, planes, trains, some cars, things like that. There's plenty of – we have sort of given over to various levels of AI many kinds of things. We've – it's, again, that – 语法解析

01:20:44

This question of sort of, you know, we perhaps humanize, you know, we say, well, maybe there'll be this kind of AI that rises up and wants to kill us all. That's a very complicated concept because, I mean, look, I think from a practical point of view, there are stupid things that one could set up to abdicate to AI. 语法解析

01:21:13

One could connect AIs to lots of systems in the world and just sort of, and say, oh, everything's going to be fine. And it won't be. But, 语法解析

01:21:25

I don't think that's… One thing to realize is it's not like there's one AI in the world. There could have been. It could have been that there was, in science fiction and in even people's early conception of computers, it could have been we just build this one giant computer and that's all there is. That's a little bit of a different situation. It would be like saying if there was one 语法解析

01:21:47

you know, organism and there was no competition between organisms and so on, it would be a different situation. I don't think that the sort of the so we're not in the one big AI situation. So now the question is what, you know, people imagine things like, I mean, I don't know that to my mind, I'm not a big 语法解析

01:22:09

People say things like, well, think about the IQ of an AI and think about the fact that it can improve itself. Its IQ is going to run off to infinity. 语法解析

01:22:21

Well, you know, I think of many people who would be able to do IQ tests really well, who I know, who I can be quite sure are not, you know, that on its own doesn't, you know, it isn't the ticket to sort of, you know, take over the world type thing. I think that the thing, I mean, there are many features of the world that are, you know, there are many physical constraints in the world. 语法解析

01:22:50

Yes, you can have an AI that's figured out all kinds of things, but it's still subject to the laws of physics. It's still, you know, there's and there's things where people might think, oh, it's going to figure everything out. Well, actually, you have to try a bunch of things. The physical world. 语法解析

01:23:07

is partly because it is doing all this computation. You can't figure out in advance things about what's going to happen in the physical world. You actually have to try experiments and so on to see what's going to happen. That's another piece that sort of slows down the, oh, it's just going to figure everything out and take over. But I think the most significant thing is that people tend to project onto AIs 语法解析

01:23:34

The idea that the AI is going to want to do this or that thing. But just as they project onto other people that other people are going to want to do this or that thing, the only thing we ever know for sure ourselves is how we're feeling internally ourselves. Everything else is kind of an assumption, a projection. 语法解析

01:23:54

And the concept that the AI that has all of this sort of computational ability is going to want to do something is a very weird concept. I mean, the generalization of wanting to, for example, all these programs that I've studied in the computational universe, I might, as a human, say, oh, it seems like it wants to fill this thing with black squares or something. 语法解析

01:24:23

But that's really a very weird description. It's a very humanized description of something that really isn't very human. I mean, I think that this question, I was having a conversation with a person who's kind of one of the leading AI doom folk, a chap called Eliezer Yudkovsky. Who's that? We had a long conversation, and I think I finally understood 语法解析

01:24:51

his view of kind of the scenario of doom. And honestly, as I said, I just don't think it's right. I mean, so his theory is this. His theory is AIs will be able to optimize the doing of almost anything. I agree with that. Up to the constraints of the physical world. You know, in other words, if there is a thing you can define that you want to do, to be able to do it 语法解析

01:25:23

one will be able to optimize the path to be able to do it. I mean, that's, as I say, that's what I've lived for the last four decades in software that I've built is, you know, can I automate that? Can I optimize that? So I take that as a reasonable thing. Second statement is AIs will sort of have objectives that are kind of a wide range of different objectives. 语法解析

01:25:54

If you can define them as having objectives, I think that's kind of true. But I'm not sure what it means to say that they have objectives. So then the next claim would be most of those possible objectives don't leave room for humans. That's a much more bizarre claim, I think, because it's like saying that, you know, it's difficult to define this notion of an objective for something that doesn't have 语法解析

01:26:22

the kind of thinking that isn't human-like. It's like, I can say I've got this little program and it does what it does, but does it have an objective? Well, no, not really. It just does what it does. Now, do humans have objectives? Well, at some level, we just do what we do. The nerve firings in our brains cause us to do what they cause us to do. We, from the outside, will have a description that says that human is doing that because 语法解析

01:26:53

of this and that and the other. But that's a from the outside description. That is an imposed notion of an objective. Now, we may feel in our internal thinking about our own thinking that we describe our internal thinking in terms of objectives. That's possible. I'm not sure that that's a thing. It's an interesting question to what extent the description of what we do in terms of objectives is something that is 语法解析

01:27:21

innate and natural, or whether that's something that we learn just as we learn language. For us to describe what we do as I'm doing that because blah, blah, blah, it might be that when we're all babies or whatever, we just do what we do. And it is a higher layer that is our description of objectives, so to speak, just as it's a higher layer to be able to describe things in terms of language. 语法解析

01:27:49

So, you know, I think this notion that we can just sort of say in the space of possible objectives here, the AI is going to pick this and this and this, then they're going to tighten the string. And when they tighten the string, humans will be sort of locked out of the picture. I just think that's a very… 语法解析

01:28:04

I just don't think that's the right picture of what it means to think about sort of a space of objectives. I don't even know what it means to talk about an objective. It's like, you know, asking what, I mean, one gets very quickly into a lot of kind of complicated ethical questions about kind of what, I mean, I think Eliezer has sort of an idea about sort of that we have a responsibility to keep 语法解析

01:28:32

the universe interesting, so to speak. But I have no idea what that means. In other words, spreading life through the universe 语法解析

01:28:40

is that there is a sort of ethical obligation to the universe, to spread life through the universe. I simply don't get that at all. I mean, ethics is a human thing. There is no abstract ethics. You know, people get confused because people try to take ethical questions and couch them as scientific questions. For example, famous one is, you know, the trolley problem. You are 语法解析

01:29:06

Trying to decide are you going to you know make this well these days it'd be a self-driving car You know kill five llamas or something or one endangered lizard or whatever? You know how do you decide that? 语法解析

01:29:21

Well, the point is that the thing that's that's the thing that the cheat in that in that problem is, well, in science, one of the things that makes science possible is that we can do kind of controlled experiments. We can say we're going to do an experiment on this little tiny piece of the world, ignoring everything else that's happening in the world. 语法解析

01:29:42

We don't have to, but in ethics, I don't think that's possible. In other words, there is no answer to the question of the llamas or the endangered lizard. 语法解析

01:29:51

without knowing the whole story of sort of the connections of the llamas, whether that one of those llamas was somebody's pet llama, whether there was a, you know, a group that worships llamas and whether the endangered lizard was a, you know, all kinds of things. It kind of quickly, you know, entangles everything in the world. You no longer get to do this kind of science-y thing of making the controlled experiment, the abstract kind of thing. 语法解析

01:30:19

I think you were asking earlier about what, you know, what a person like me feels like in terms of kind of, you know, is AI just going to be able to do that job for them and so on. You know, my attitude towards that is I spent my life trying to use tools that exist to leverage the things that I can do and to get me on the sort of fastest path from thinking of something to do to being able to execute it. And, you know, for me personally, 语法解析

01:30:46

Talking to the AI to have it help me write a piece of code or something like this is a pure, you know, it just helps that process. Now, there's another thing that I've just started to do, which I'm not quite sure how well it's going to work out, but it's this. 语法解析

01:31:01

For example, I'm interested, we made a big progress in fundamental physics about five years ago. And there's a big question about whether there are experimental consequences of this theory of physics that I made, that whether there are experimental consequences that one can figure out what they are. Maybe there are experimental consequences where the experiment was already done and people just didn't know how to interpret it. 语法解析

01:31:29

Well, there are millions of physics papers out there in the world. I haven't read all of them. I couldn't read all of them. So a question is, can I use… 语法解析

01:31:38

AI to essentially thematically analyze all of those papers and tell me sort of thematically. I mean, it's one thing to sort of do statistics and say, I've got this whole pile of numbers, you know, and seven out of 10, you know, giraffes have long necks or whatever, whatever it is. You know, with numbers, you can do sort of statistics, but there's a thing that's now become possible with AI, which is to say, let's take a million pieces of text and 语法解析

01:32:07

And let's get kind of the, let's try and extract something thematic from all of that text. Not what's the average of the numbers, but what's the mood of the text or something. And so, you know, it's something I'm just starting to try to do. And it'll be interesting to see how well it works. I suspect it's going to work fairly well. And I suspect that's a kind of, you know, is it a discontinuity from what we've had before? 语法解析

01:32:33

Not really, but it's another big step. I mean, back in the 1970s, I was already using online database services where you could do that. People had uploaded the abstracts of all scientific papers. You could do a keyword search. 语法解析

01:32:49

You know, you could find papers that became a lot easier when the web came along and search engines came along, but it was already possible in the 1970s. Um, you know, the web made it easier to do full text searching and so on search engines and such like, this is another step. 语法解析

01:33:05

I don't know how significant a step it will be. I don't really know. I mean, there are many use cases that are kind of interesting. I mean, one that I've been curious about is medical diagnosis. Actually, diagnosis of almost anything. Diagnosis of problems with your computer. Things where there's a body of knowledge about things that can happen, and you have certain symptoms, and you're trying to match those symptoms to what's known. And I kind of have this suspicion that 语法解析

01:33:34

That you know the current round of AIs is going to do quite well at that possibly in a quite superhuman way Because you know we humans I mean it's it's it's this thing about sort of thematic searching of what's out there in the world and it's it's you know some parts of that thematic searching actually don't even use the kind of most AI ish parts of things like LLMs they use things like you know take a 语法解析

01:34:01

a big piece of text, and instead of grinding it up just into words, grind it up into these arrays of numbers that somehow represent the meanings of sentences. And then say, well, do I have a sentence that I'm now asking about that's close in meaning as revealed by being close in numbers to something which was already there? Yeah, you mentioned something interesting, which is like the fundamental question of 语法解析

01:34:27

is what do humans want to do? You talked about this idea of jobs. There was a quote, I think, on an essay around John Maynard Keynes. He said that in the 1930s, he said that in 100 years, due to the advancement of technology, that the productivity is going to increase so much that humans are only going to need to work more than less than 15 hours per week. And you mentioned with AI and the involvement of AI, humans are pretty much 语法解析

01:34:57

The AI is going to be able to do what everything the humans are going to do. Yet here we are pretty much 100 years later. Humans are still working 40 to 40, 50 hours a week. So nothing's really changed there. What does that say about how we identify as purpose? And what are the things that we need to unlearn as society if AI can really do everything that we can? 语法解析

01:35:24

Well, I think in the time of Keynes, lots of people were doing agriculture. I mean, lots of people were working very hard at things that got automated. And as I was saying before, I mean, what we actually see in the data is what happened is those people fragmented into a zillion other jobs. 语法解析

01:35:45

Now, you know, it's like a lot of these things. It's like the paperless office myth. You know, when you can make documents electronically, there won't be any paper. Now, actually, now there isn't much paper, but that took a while. There was a big burst of more paper. And I think that, you know, what I think we see is that the things… There will be, I think, more emphasis on… 语法解析

01:36:14

What do humans do? Humans make choices. Humans interact with other humans. These are things which are sort of uniquely human and independent. Even the interacting with other humans, I'm not sure how that's going to play out. I mean, we have a big project right now to build an AI tutor. You know, people have been trying to do sort of computerized education for 70 years. And it's basically always failed to do. You know, one can get computers to help, but to be the prime teacher has never worked. 语法解析

01:36:45

It looks promising this time around. I'm not saying for sure it will work. We'll know. We'll probably release it in a few months, and then we'll know whether it works or not. Like Khan Academy style, or what's kind of the… Well, I mean, so this is… 语法解析

01:37:00

Literally, you're simply interacting with a particular thing we're targeting as algebra because that seems to be a thing people have a lot of trouble with. It's frustrating for me because usually we build products and I'm in the target market for those products. This is practically the first product we've ever built where I'm not in the target market and I have a pretty hard time kind of internalizing what the interaction looks like. I'll be the better user because I'm terrible with algebra. Sure. 语法解析

01:37:24

Well, that's, yeah, you know, it's a thing where, you know, the question is, I mean, I think what, you know, the center of something like Khan Academy is, you know, videos that explain things. And I know they've done some experiments with LLMs. The raw LLM does not do very well at this. 语法解析

01:37:45

The the thing that, you know, people might say the raw LLM is going to be able to to be a tutor. It is true that if you like upload the class assignment that you had or the notes from your class and you tell the LLM, ask me questions based on these notes, it'll do a reasonable job at that. 语法解析

01:38:04

But if you say, lead me through this whole course and sort of keep me on track and so on, that's a thing that, at least in our observation, seems to need a whole lot of superstructure, a whole lot of kind of, I mean, for us right now, interesting statistic I just learned a couple of days ago, in our AI tutor, there's four times more sort of AI work going on 语法解析

01:38:29

in behind the scenes than the AI that's actually interacting with the student. 语法解析

01:38:34

So there's four times as much kind of the machinery that's keeping the whole thing on track and that's defining what should happen than there is in the actual interaction with the student. And that gives you sort of a sense of the fact that it's a typical thing, I think, that's happening with sort of AI is there's a component of a task that the current generation of AI is really quite helpful at. And it really enables things which were never possible before. 语法解析

01:39:03

But there's still a big kind of, you know, how do you fit that into a sort of harness of how do you fit that into a bigger machine that can do the full task you want to do? But, you know, but back to sort of, so I don't know, as far as I'm concerned, I'll know more in a few months, whether sort of things that involve sort of 语法解析

01:39:24

Things like teaching that seem to involve humans getting convinced of things by humans, humans getting motivated by interaction with humans. I don't know how much of that will turn out to be sort of AI-able. Not sure. But specifically around jobs and humans defining their identity around jobs, when… 语法解析

01:39:47

AI is replacing everything. Does the nine to five need to change? Like what happens when no one needs to work anymore? Well, it depends what people, you know, if people are, what does it mean to work? You know, if you're, you know, if you're playing video games and that's how you make your living, is that working? You know, what I do for a living, so to speak, I don't consider working particularly, you 语法解析

01:40:14

You know, I do what I do because I like doing it. I find it interesting. And, you know, it happens to be a commercially successful thing. Much of what I do is some of what I do is basic science that might be commercially successful in 200 years. But that's not really, really, really the point, so to speak. I mean, I think the people people have. 语法解析

01:40:38

The set of experiences people can have, the set of things people can do will be deeply leveraged by more technology. And as they have been so far, people who spend their time on social media or make a living interacting on social media or doing podcasts or whatever else, these are things that have been enabled by technology. What we're doing right now couldn't have been done without a bunch of technology. 语法解析

01:41:04

And I think what we'll see is more and more things that are possible for humans to do. And yes, it's conceivable that humans will just all become couch potatoes and just be sort of pure consumers. But certainly what has tended to be the case is there's certainly things for humans to do. It's not the case that there's that… 语法解析

01:41:33

I mean, if you ask the question, do humans need to work? In other words, can the world sort of operate without humans doing anything that anybody thinks of as work? 语法解析

01:41:49

It's possible. I don't think that's how things will run because I think people will… It's like, well, we could say we don't need all those television shows that somebody had to invent. We don't need all those podcasts that people are doing and so on. The world could run without that. If we were back 100 years, we would say, yeah, 语法解析

01:42:13

you know what we really need to do is to is to automate agriculture once we've done that nobody's going to need to work it's all it's all good you know there's you know we can just hang out and and you know have food delivered to our table type thing and never have to worry about anything else but that hasn't been the you know that's that's not what has been the history of our species so to speak i mean we end up with things where we we find things that um 语法解析

01:42:41

You know, suddenly something becomes possible. It's like, well, let's, you know, people will do that. There's always a certain driver and there's certain kinds of scarcity that I think continue to drive things. It's like, you know, you could say, well, you know, it's going to be automatic to discover this thing in science, let's say. Well, but, you know, if you decide to go in that direction, there's always going to be the first person who discovers that. 语法解析

01:43:08

And that's kind of an exciting thing to be the first person who does this or that thing. There's always sort of a built-in scarcity to what's out there. I think the thing to understand, perhaps, is sort of the computational universe of possibilities is infinite. You might have thought that, you know, there'll come a time when we've made every invention that can be made. That time will never come. It's, you know, a century ago, people were like, yeah, we've almost made every invention that could be made. 语法解析

01:43:39

Well, turned out was not true, but we now know a bunch of things I've done in theoretical science make it very clear that 语法解析

01:43:48

In no sense will we ever be able to say every mathematical theorem that could be proved has been proved. Every invention that could be made has been made. That will never happen. There is an infinite frontier of these things. And every time there is an invention that can be made, there's something sort of new and different that you can do in the world. 语法解析

01:44:10

And then the question is, well, humans might just say we don't care. You know, it might be that humans just decide, you know, if we look at human society, people have different beliefs about what we should be doing. I mean, it's like you could say, I don't believe in anything that was invented in the last hundred years. I'm not going to live my life in such a way that I'm using things that were invented in the last hundred years. 语法解析

01:44:34

of last thousand years, whatever, you know, you can make a decision that says, I'm just going to lock out the things that are now possible. I think, you know, just as one could have sort of the society of the couch potatoes, so to speak, that have just decided they're never going to try and do anything. I don't think that is the way the human condition is going to play out. But it's not something that, I mean, you know, it is the case, 语法解析

01:45:03

Well, it depends on where you live in the world and how many people there are in the area. But if you're in a place where there are enough natural resources that you can just mine them out of the ground and… 语法解析

01:45:19

make a living, so to speak, just by the fact that there are these natural resources there. There are some parts of the world where that's the case. And it's an interesting question. What do people do in that situation? What do they do when they don't sort of need to do anything? And the thing to realize is, you know, the human condition seems to be such that we still seem to seek 语法解析

01:45:41

some things that, for example, are scarce or whatever else. And I, you know, I kind of think that it is somewhat on us. I mean, yes, it is already the case, at least for some small part of our species, that you kind of don't need to do anything, so to speak. And that would be, you know, but I think that the question of whether you, you know, it 语法解析

01:46:04

I think it's more a choice than it is, oh, the AIs are going to take over. There's nothing for us to do. The fact that the AIs have enabled more things just puts us on a taller platform. There's more that we can then do. And at least, you know, in at least my view of life, 语法解析

01:46:22

that's a thing that's kind of great and lets one go much further. It's not a thing where I say, oh gosh, I should just give up. The AI is going to do everything I can do. I think, you know, again, that's a somewhat, you know, 语法解析

01:46:38

It might be somewhat person-dependent thing. It's my rosy view of the human condition, perhaps, if you think about it that way, as something where people always seek things to do. Whether they have to do it from the point of view of, if they don't, they will die, 语法解析

01:47:02

or starve or whatever else. It's a little bit of a different thing than is that what they choose to do. I agree with you from like the leisure perspective, once you've met those basic necessities, but every human needs to put food on the table. Every human needs a roof. Most humans need a roof over their tops and be able to meet their basic necessities. And throughout history, we've always exchanged some form of value in the economy and work to be able to put 语法解析

01:47:30

currency, like the currency of money in our case, so that we can actually pay to meet those basic necessities. But what happens when there is not enough jobs and people don't have the ability to provide value to be able to actually exchange and meet those basic necessities? Is that a problem that you can solve through other economic means like UBI or how do you think about that? 语法解析

01:47:55

Maybe I have a, uh, uh, okay. So the first point is that, you know, a lot of what people pay for in the world today isn't basic necessity in, but not in all countries, but, but in, in, you know, in, there are many segments of the world in which what people are mostly paying for isn't basic necessities. So there is a thing that people care about paying for that is something much more ethereal than just getting enough food to eat, so to speak. 语法解析

01:48:25

So the question of what fraction of what has to be paid for, there are a lot of things. Okay, here's a surprise. Something that surprised me is back, I don't know, 40 years ago or something, 45 years ago, I used to have fancy computers. I used to have access to computers that were much fancier than the typical person's computers were. But in fact, consumer electronics became cheap enough 语法解析

01:48:54

that everybody has the same kinds of computers now. It's very flat. And this question of, it could have been the case that there was a huge sort of consumer electronics was this thing that was a big sort of mountain where you only got to the top with a lot of effort. 语法解析

01:49:14

you know, things like basic necessities, they'll get cheaper probably, you know, through automation, they'll get cheaper. They've gotten cheaper. I mean, the, you know, when fertilizer was invented, you know, people thought the world was going to run out of food because there weren't going to be enough crops produced to deal with the population increase. But then things like fertilizer, crop breeding and so on were, were, were invented. And, uh, you know, and that problem went away through something and because food effectively became cheaper, 语法解析

01:49:42

and food became easier to produce. And I think this question of what happens when you've kind of zeroed out certain kinds of basic necessities 语法解析

01:49:56

Well, people want other things. You know, people want to see that, you know, they want to watch that amazing special effects television program or something. They want to see this thing. They want to be kind of they want to have this or that experience. They want to do these things which are kind of, you know, I mean, again, it's been the experience. Now, you know, it could be the case that at some moment 语法解析

01:50:25

sort of our species or some segment of our species just decides we've got enough. We don't have to invent anything new. We just hang out. 语法解析

01:50:34

I think it's happened in the history of our species. I mean, people say, I don't know the detailed anthropology of it, but people say there have been periods of thousands of years where there have been groups where, well, here's the funny thing, where people say their basic necessities were taken care of. They lived in a place where they could just pick berries off bushes to eat and so on. And then you ask, well, what did the people do in that situation? 语法解析

01:51:03

And the common anthropological statement, I haven't really dug deep in this, so I don't know. I don't know how much I believe this, but the statement that's made is, well, people do these go into these very ritualistic kinds of behaviors, which is to say from the outside doesn't look like much is going on. It's just people doing quotes, ritualistic kinds of things. 语法解析

01:51:25

I'm amused to realize the extent to which so many of the things we do today could be seen as ritualistic. I mean, from the point of view of you don't know why you're doing it, you know, I'm sitting in front of a computer getting weird pictures coming up on the screen. This seems like a devotional ritualistic kind of activity. If you don't, you know, if you don't have a thread of understanding what the point is, so to speak, you know, by, I mean, in other words, if you can't connect 语法解析

01:51:51

that kind of activity to something that you intrinsically know what the point of it is. You can't make that kind of thread of connection. It just looks, quotes, ritualistic to you. So I think, you know, the thing that sort of what do the humans whose needs have been met, what do they do? 语法解析

01:52:14

The answer might be from our view today, they look like they're just doing ritualistic kinds of things. But to their internal view, they're doing things that are tremendously significant. I mean, when I, you know, if I see some kid who's worrying about something about some social media interaction they're having, it's like, you know, I don't know. I don't know why you care about this, you know, but to them, it's something very important. 语法解析

01:52:41

And, you know, from the outside or from a different time in history, it might not look important. But in the moment, in the internal sort of experience, it can look significant. And my guess is that, you know, there's a view of sort of the future which says, well, we'll figure out brain uploading and all this kind of thing. And pretty soon what the future of humanity will be, you know, a trillion souls in a box, right? 语法解析

01:53:08

playing video games for the rest of eternity. And you might say, gosh, that's a terrible outcome. From our point of view today, from our experiences today, from the things we care about today, that seems like a terrible outcome. But my guess is that in the internal experience of that disembodied soul, you know, playing video game, playing quotes, video games will be perfectly meaningful. 语法解析

01:53:37

And they'll look back at our time and say, gosh, you know, that must have been, you know, they couldn't do all these amazing things we can now do. And, you know, what a boring, meaningless existence, so to speak. Yeah, just like a thousand years ago, us running around Trendles seems crazy, like you mentioned. 语法解析

01:53:58

So, as the trajectory of technology advancements makes luxury items more accessible, you talked about, I mean, now we have, everybody has a private car through Uber. You can rent an Airbnb on wherever you want in the world and just go into someone's homes. Soon you'll be able to have a robot in your house and tens of robots, assistants can do pretty much anything for you. So, if that's the case and you're saying that basic necessities are going to be met, then 语法解析

01:54:26

What's going to be the perspective of how we think about money and wealth in general? Are people going to want to be wealthy in the future like we do today or is that relationship with money going to change? - There'll always be scarcity. Not because, there'll always be the person who lives at the top of the mountain, there's only one top of the mountain. Maybe you can build another mountain eventually. There'll always be the first person who does X. 语法解析

01:54:57

So there'll always be things that are not genericizable. You may not care about them, but there will be things that somebody might think was worth kind of bidding up, so to speak. So I think that's one thing to realize. I think that, you know, in… 语法解析

01:55:18

I mean, one's attitude about, you know, I've lived a life where I like to do interesting things, things I find fulfilling. I've you know, I'm practical enough that that activity has made me a very decent amount of money. 语法解析

01:55:36

But it is not, you know, for me, there have been a vast number of forks in the road where it's kind of like, do the more interesting thing, do the thing that makes more money. I'll always pick the more interesting thing, you know, to some people's kind of horror in a sense. But I think it's a thing where, I mean, people in… 语法解析

01:56:05

This question of whether 语法解析

01:56:08

I think there's a lot of currency in the world that isn't money that exists right now. I mean, there's currencies, a social currency of, you know, do you have friends you like? There's this kind of fame currency. There's a lot of things which are not money related. There's already lots of different kinds of things. And the question of the sort of the money as the thing that buys you food and so on may or may not be kind of the the currency. 语法解析

01:56:36

I think an interesting question, the very concept that there could be a single store of value and its money is an interesting concept that's worked pretty well in thinking about economics. I'm not sure. I don't know. It's a thing I'm hoping to think about, actually. I haven't really figured it out. But it's like people sometimes say you can buy anything with money, but it's actually not true in the world today. 语法解析

01:57:06

It's, you know, and it's like, you know, the things I don't know I've done in figuring out stuff in science, for example, I've had a really good time doing some of that. 语法解析

01:57:17

you could pay as much money as you want, it's not going to help you have that experience, so to speak. It's something where there are things you can do that sort of on-ramp to that, but it's not a thing you can buy with money kind of thing. And I think that that's one thing to realize in the way that… 语法解析

01:57:39

you know there's there's what you need for the basic necessities and the sort of economic model that exists i mean this whole question of of uh what value really is in the world it's an interesting question i'm actually some science that i've done looks like 语法解析

01:57:59

It begins to tell you a bunch of things about how to think about economics. This is a very different topic, which we are not going to have a chance to really dive into, I think, here. And also, I haven't figured it out. It's a very strange thing that this physics project that… 语法解析

01:58:16

kind of got launched five years ago, has led to kind of a formalism for thinking about things that in fact in the last year has let me understand a bunch more about biology and biological evolution, a bunch more about machine learning. I'm pretty sure that it has a bunch of things to say about economics 语法解析

01:58:32

But I don't yet know exactly what those are. But things that are kind of exercises in a sense for that economics is things like, you know, if you take cryptocurrency, is it really worth anything? Or does it have to be the case? Which, by the way, relates to the question you're asking, because people might say, well, cryptocurrency isn't worth anything, really, because I can't go 语法解析

01:58:54

you know, buy food with it. You know, it's not practical to go buy food with it. So therefore it isn't really worth anything. I think that argument is not correct. And that relates to this. I mean, in a sense, 语法解析

01:59:07

It's worth something because there's this whole network of things that depend on on it, on its existence and so on. And I think that that's, again, an example of something where you could say, well, you know, all the value in the world is the fact that we have, you know, houses and food and things like this. 语法解析

01:59:29

but yet there are these other kinds of value that seem to exist. I guess I feel like this question of the importance of, in times in the past, having enough food 语法解析

01:59:51

Still still a problem in some parts of the world, but in many parts of the world, you know having enough food to eat Isn't the problem. In fact, the problem is usually you eat far too much and you know in 语法解析

02:00:05

Whereas at a different time in history, that would have been the big stretch goal as, you know, get myself enough food to eat. I mean, I think, you know, when there are all these portraits of Henry VIII looking extremely rotund, you know, that was a sign of that was a, you know, a sign of success that you could be extremely rotund because most people, you know, didn't have couldn't get enough food to eat at the time. It's I think, you know, in, you know, I think 语法解析

02:00:33

I think this question of sort of what will people be doing with themselves in the future, it's will they, you know, will work look the same? Work has changed a lot. I mean, before the Industrial Revolution, work didn't look anything like the nine to five work looks today. I mean, mostly people were sort of working for themselves, you know, you know, 语法解析

02:01:00

tending crops and things for their own use, things like this, then things got streamlined and centralized. And, you know, now, there's a little bit more of, you know, of people doing more sort of working for themselves, so to speak. I 语法解析

02:01:15

I don't know whether the picture of… And there's a little bit more… You know, another thing I've noticed, I don't know, I don't really have data about it, but, you know, more people have more different things that they do. I mean, there was a period of time where you say, what do you do? It's like, well… 语法解析

02:01:32

you know, I'm a this. It turns out, well, I'm kind of a this, that's my day job, but then I do podcasting and then I do, you know, competitive, you know, bicycle racing or something, and then I do whatever. And it's, you know, that I think even independent of the economy fragmenting into many different job categories, even people's individual lives, people are ending up having more tracks in what they do. And I don't really know how that 语法解析

02:02:01

that maybe part of the reason that's possible is because the cost of getting into those tracks has gone down. That is, it was, take podcasting, for example. If you wanted to be kind of a person who would broadcast things to the world, 语法解析

02:02:19

You had to build this whole stack of things. You had to go work for a radio station or whatever else it is. But because of technology, the cost of getting into the business of podcasting went way down. So we can do that as a gig, so to speak. We don't have to have made it our whole life. And my guess is that that's probably a trend that… 语法解析

02:02:45

of people sort of doing a bunch of different things. Another interesting question is whether there was a period of time when people would do one job for their whole life. One, sometimes, and the question is, and that's often still a great thing, 语法解析

02:03:03

Sometimes, you can be working in one place, but what you do can change completely. I think this fragmentation thing, which again relates to it's more about the choice than it is about the mechanics of what's done. I think that the mechanics of what's done getting automated just means there are more choices. 语法解析

02:03:28

maybe that's manifest in both more kinds of jobs and more jobs done by individual people and so on. That's my guess. I mean, I'm, you know, the idea, as I say, there could be a choice to just sit around as a couch potato and maybe there'll be, you know, a segment of society that does that. And, you know, an interesting, I don't know what, and maybe from our point of view today, 语法解析

02:03:54

That looks like a pretty bad outcome, just like the trillion uploaded souls look like a bad outcome. But I don't know. For me, I'm not a very good couch potato, so to speak. And I don't, you know, for me, if I'm like, you know, there are plenty of things that lots of people find interesting that I don't. Like, I don't watch television. I don't play games. I don't, you know, do, you know, it's just those are things that I personally don't happen to find interesting. 语法解析

02:04:22

but other people find those things absolutely fulfilling and interesting. And it's not, I wouldn't make any claim that there's anything about my particular interests that is in any way more wonderful than, than other people's. It's just, 语法解析

02:04:37

the particulars of what I'm interested in. And I think, again, that's probably another thing that will be true as more gets automated. Each of us with our particular interests and foibles is capable of sort of pursuing those things in a way that wasn't possible before because the sort of the barrier to entry, the amount of mechanical stuff that had to be done was so great 语法解析

02:05:01

That, you know, well, like podcasting is a good example, really, of where you just couldn't have done that without the levels of automation that exist today. And, you know, I think that the that's sort of a. 语法解析

02:05:15

you know, in a sense, maybe it's a, you know, perhaps this is a, an overly optimistic view of things, but, you know, we're all kind of forged with different interests and, and objectives and, and things we care about and so on. You know, 语法解析

02:05:32

You know, that's a pretty arbitrary thing that will be some kind of mixture of genetics and physiology together with our experiences. But, you know, we're all in this position where there are things we care about doing. And one could argue that what's happening as more gets automated is the world becomes more and more ergonomic for us to be able to do what we want to do. In other words, in the past, it was like, well, we have this and this and this thing we want to do. But gosh, I'm never going to be able to do that because it's just too hard. 语法解析

02:06:00

But as more and more gets easy, it's like, well, there's this and this and this thing that I want to do. Well, okay, great. Now I can actually do it. I mean, again, story of my life has been that the things I've been really keen to do, I have built myself. 语法解析

02:06:15

a big tower of automation that makes those things reachable. And without that, I wouldn't have been able to do these things. And it's, you know, it's been the, you know, even in recent times, actually, I've been doing particularly a lot of science that's quite diverse in the kinds of things I've been doing. And it's, you know, from a point of view without the technology tower I've built, 语法解析

02:06:39

and without a certain amount of scientific knowledge that I've accumulated, it would be completely inconceivable to go across all those different areas. That's just crazy. You can't write something about biology one month and about machine learning another month and about foundations of mathematics another month. That's been made possible because I automated a whole bunch of stuff. 语法解析

02:07:03

And, you know, I think that's sort of the thing that as I'm talking to you about it, I'm realizing this really is, I think, the picture is it's kind of like, you know, we all have lots of things we want to do. And the barrier to entry to most of those things has been too high for us to actually do them. 语法解析

02:07:26

As more gets automated, we'll be able to do them. Now, what that will mean in terms of, you know, what do we get for that? Well, we might get money. We might get some other kind of currency. We might just get internal fulfillment, which is perhaps its own currency, so to speak. It's sort of the personal currency. And, you know, but I think in… 语法解析

02:07:52

you know, some aspects just like the putting food on the table became less of a stretch, you know, over time. So it may very well be the case that certain kinds of things that are, you know, as I say, this whole question about like consumer electronics, you know, I could have 语法解析

02:08:11

You know, if I, you know, OK, as a practical person who, you know, makes choices in their life and so on, and it's like, well, OK, I could do this and I would make a bunch of money doing it. I at a very practical level have thought about what do you get to do at this level of money, that level of money and so on. 语法解析

02:08:31

And, you know, it's interesting that these different levels of, you know, what's possible. Now, you know, I've been fortunate enough that I'm not dealing with the, you know, make the rent payment type level. But, you know, I'm but it's still it's sort of interesting that there are things at different levels where it's like, yeah, you know, one could, you know. 语法解析

02:08:54

I get the money to buy a yacht, but I don't care. That's not something I'm interested in. One could do this or that thing that, you know, and plenty of the things that I do, for example, are things where it's just not a question of money. It's a question of, you know, to be able to do it is a question of other kinds of things that are not, you know, are not directly money. 语法解析

02:09:18

What do you think is the right amount of money for someone to make where people can just stop caring about that and just pursue what they want to do for, let's say, the U.S.? 语法解析

02:09:29

It's an interesting question. I don't know. I mean, I think that, you know, I think there was an image right in the 1950s or something of the little house with the white picket fence and so on. And that people, you know, that was kind of the image of sort of the normal thing people would be, you know, would want, so to speak. And I, you know, I would say, 语法解析

02:09:55

And it's sometimes it's a complicated, be careful what you wish for. You know, I have a I have my main house is a is a great house that we built years ago. That's it's really big. And our kids have all moved out now. And now it's kind of too big and a big pain in the neck. And but, you know, so it's sometimes it's a, you know, it you know, I don't know. 语法解析

02:10:22

a number because it depends on, I mean, I think if, if I'm to look at my own, you know, for me, I mean, you know, it depends on the level of what you want to do for me. You know, I do projects where I might burn a few million dollars and where the project might not work out. And you're like, Oh, there it is. Well, you know, that, that's the, but if somebody says here's a project where it would cost a hundred million dollars, I'm like, I can't do that. 语法解析

02:10:55

So, it's a thing where it's a, now, you know, if I suppose that the things that I'm interested in doing are somewhat titrated by the resources that I have to do them. I mean, I don't happen to be interested in, you know, building rockets to go to Mars or whatever, a very expensive endeavor. 语法解析

02:11:23

I, you know, it's kind of an interesting thing, perhaps, you know, in this physics project of mine, I might suddenly become really interested in doing a physics experiment that costs $100 million. And so what happens then? What's that? What happens then? Are you going to care more about making money then just so you can pursue those paths? I don't know. I think that'll be a kind of here it is world. You know, you can, I mean, in that particular case, a funny situation, because for myself, I'm kind of, 语法解析

02:11:53

convinced enough that this theory of physics that we have is right, that seeing the experiments happen will be cool, 语法解析

02:12:03

But it'll be mostly a, okay, world, now you can believe me type thing, which is not something I care that much about. So I think, I don't know, we'll have to see. It's very hard to predict how one will feel about things until they actually happen. But I think in that case, it's kind of like, it'll be, okay, well, you can do this. It's going to cost $100 million. I think it's worth doing. If you care about finding out whether I'm right or not, go do it. 语法解析

02:12:30

you know, not on my dime, so to speak. But I might change my mind. I mean, I might, you know, I might be, I think in, you know, it's a thing where it's always a complicated thing. What you, you know, there are plenty of things that I've done. For example, the science that I do 语法解析

02:12:52

is mostly not very expensive. I mean, it's kind of, you know, the people who help me with it and so on, and that's, it costs a certain amount, but the theoretical science is not terribly expensive. In fact, sometimes these things defeat themselves by being too big. 语法解析

02:13:09

Now, in other words, even, you know, it happens with lots of things, but it happens, you know, if you have, if you've got this small number of people working intensely on something and there's a lot of flexibility in what's going on, you say, well, actually, I'm going to have a thousand people work on this. 语法解析

02:13:24

you end up with necessary structure. Otherwise, it just becomes a total mess. You have structure. That structure kind of reduces the level of flexibility and innovation that becomes possible. And it's then a complicated sort of management issue to sort of carve off the piece that's going to be the innovative piece together with the piece that's going to be the mechanical piece that actually gets the thing done. 语法解析

02:13:51

But I think I've, I'm, I'm, 语法解析

02:13:55

I'm bad at, you know, I'd have to go look at the cost of living data that we have in Wolfram Alpha and so on to have any kind of meaningful thing to say about what, I mean, I'm sure if you, you'd embarrass me if you started quizzing me about how much the different grocery items cost and so on. I have no idea. I mean, it's, you know, I know that it's, you know, it's just like it's, there's a 语法解析

02:14:23

For everybody, I think there's a level of I don't really care about the pennies type thing. And where that set point is depends on lots of things about how you lead your life. But I think in this question of when will it be the case that people say, most people say, I've got enough money, I don't need any more. That will never happen. 语法解析

02:14:49

Because there'll always be scarce things that are scarce where people say, I want to make more money so I can get some people will say, I want to make more money so I can get that scarce thing. I think that, you know, for myself, for example, you know, I guess that for me, it's like I've you know, I kind of feel like I know, you know, I've been fortunate in that I've generally been in a position where I've kind of can do the things I want to do. I have the resources to do the things that I want to do. 语法解析

02:15:18

Now, maybe I'm kidding myself, because really, if I had more resources, I'd think of a lot more things that I could do. I don't think so. I think that, you know, it's kind of a thing where, you know, well, the other thing that I see happen a lot, I remember, well, you know, you see people saying, I can't do that because I don't have enough money to do it. 语法解析

02:15:45

Sometimes that's true, but a lot of the time it's just not true. A lot of the time it's just deciding you're going to do it and there'll be a way to do it and it doesn't really have anything to do with the money. That's just an excuse. It's just an excuse for, I don't really have the initiative to do that thing. Oh, if I had more money, my life would be cushier and then I'd get the initiative to do that thing. 语法解析

02:16:12

I, you know, that maybe that happens that way for some people. I mean, look, I know that, for example, if I, there have been times when our company has done particularly well and where I, you know, at least one of them is correlated with a time when I sort of put more effort into basic science. And so perhaps, and I can't say, you know, for me, that wasn't really a causal connection. 语法解析

02:16:40

But maybe it was. Maybe there was a psychological connection there of, oh, I don't have to put so much effort into technology development because we did really well recently on that. 语法解析

02:16:52

And so that sort of tips me into more, oh, I can put effort into basic science. I'm not sure. But so, you know, it's hard to tell. Even from the inside, it's hard to tell what exactly is leading you to different kinds of motivations. But I, you know, I have to say I've seen an awful lot of cases where people explain, oh, I can't do that because I need more money. And it's just… 语法解析

02:17:17

you know not true it's a you know it's a matter of initiative and um it's uh uh i had a charming case actually many years ago now i i kind of have a hobby of doing uh kind of ceo counseling and um of advising companies and so on and i also have always found an interesting kind of mentoring kids 语法解析

02:17:43

And I kind of noticed at some point that those two categories, both of which I found interesting, were the category of kids and the category of CEOs. They're types of people who believe that anything is possible. And there are lots of other people who don't believe that anything is possible. They're kind of like, we're stuck in this particular track we're in. But those two felt that, you know, it's kind of a more in the anything's possible. And I happened to have this, I was… 语法解析

02:18:13

doing, you know, I have this try and make my life efficient. And when I'm driving from here to there, I'll make phone calls and things. And this was, you know, I had a phone call with one kid and then one CEO. And the kid was explaining that, you know, couldn't do this or that because they didn't have enough money, et cetera, et cetera, et cetera. And I, you know, the CEO I was about to talk to was about to sell their company and about to make about $59. And I 语法解析

02:18:42

I was telling this kid, you know, this next phone call I'm going to do, this person, because I already knew what some of the issues were, was going to tell me they can't do this and that and the other thing. And they said, but clearly money is not their issue. So the reason they can't do these things is not money. 语法解析

02:18:59

Because you know they don't have enough money which is what the kid was saying but because of something else that some internal kind of You know block of I just can't do this because I don't feel confident in doing it I don't have the initiative to do it etc etc etc This kid told me that was a useful conversation And he wanted did the thing that he was thinking about doing and it worked out pretty well So but it was some you know, it's sort of interesting to me, you know 语法解析

02:19:25

You see those cases where it's a people… I don't know. So I'm not answering the question. If you ask me if we could distribute UBI from AI to everybody, would the world be a better place? I doubt it. I mean, I think that people… 语法解析

02:19:49

The experiments people have done, the situations people have where it's just like, you know, now you reset the base level and everybody has this. Still, people are going to seek the scarcity and so on. Some people are going to seek the scarcity. Some people are going to. I mean, it's a thing that's always, you know, one of the things that is kind of a negative value of money story. I mean, I always notice there's positive value of money and there's negative value of money. 语法解析

02:20:17

You know, sometimes you see, you know, I know plenty of people who've been bitten in so many ways by the negative value of money, so to speak. You know, in… 语法解析

02:20:28

Oh, I mean, just there are there are more than more than we can enumerate of things where. But, you know, a very, you know, there are there are typical ones that everybody knows about, about, you know, oh, I inherited a ton of money. Now, what am I supposed to do with my life type thing or, you know? 语法解析

02:20:47

or there's this pot of money and people who would otherwise be friends are arguing to the death over it, so to speak. Even though if that pot of money wasn't there, they'd just be happily friends with each other. There are all these different scenarios and I think it is complicated. It's something where I don't think… Just like 语法解析

02:21:13

I don't think it's true that you can buy anything with money. I don't think that injecting some amount of money into the system is… I don't think that solves everything, so to speak. But this is getting far out of my usual… Me as a common sense observer of the world, I'm not… 语法解析

02:21:40

I can only say that I do think that the AI makes it easier to do things. Another thing to say, I suppose, is that people have different skills, people have different interests. One can be lucky or unlucky in the period of history in which one lives. So for example, I consider myself lucky 语法解析

02:22:11

to been in a period of history when computers started being sort of usable things because they're a good fit for a lot of stuff that you know i like doing you know if i'd been sort of obsessed with finding you know exploring the surface of the earth or something i've lived in the wrong time in history because we got satellites that took pictures of all of it um you know it uh i think that 语法解析

02:22:39

There are times in history when if you were sort of an intellectual and you had a bunch of, you know, I really want to think about ideas. It's like, sorry, you know, you have to plow the fields to get the food or you have to fight in some army to not have your world collapse type thing. It's you know, I've been. 语法解析

02:22:59

lucky enough to live in a time, you know, be in places where, you know, sort of, it's been sort of peacetime everywhere I've been, at least in the more or less in, you know, as an example. And I think, you know, one can be, I think this question of, well, another thing, you know, there was a time when if you were a techie type of person, and you were doing, 语法解析

02:23:22

things in business, it was like, oh, you're just a techie. You know, you're off in the back room, so to speak. And then at some point, 语法解析

02:23:31

the techies, you know, the nerds took over, so to speak. And then it was like, it's pretty cool to be a techie doing business kind of thing. And, you know, you can be, you could be very frustrated in it, you know, being the techie who really wants to be calling the shots in the business and everybody's telling you, no, no, the professional managers will do that type of thing. And so I think, you know, what's coming is, 语法解析

02:23:54

is I do think that what's coming probably is a time when if you like thinking, this is the time for you, so to speak. If you like having ideas and so on, this is the time for you. If you like kind of doing the mechanics of doing things, maybe it's less the time for you. 语法解析

02:24:16

I mean, and maybe that's a, you know, and it's a thing where I think, you know, there are, because the things which are sort of more mechanical are getting automated and will get more automated. And, you know, so that's a, you know, and I think for people, and it relates to what people learn in education and so on, because a lot of education in the last hundred years or so has been a lot about the mechanics of doing things. 语法解析

02:24:45

And it should be more about, you know, learning the facts that you need to know to be able to think broadly about things and then learning how to think broadly about things. But that's not been the sort of industrialized form of education that we've tended to have. I suspect that will become more and more important. 语法解析

02:25:03

I tend to think that the extremes of philosophy as a way of thinking about things and computational thinking as this formalized way of thinking about things, these are two good forms of thinking worth learning, so to speak. Those are things which I think will, in the coming world where lots of the mechanics have been automated… 语法解析

02:25:26

those become things that are very significant for people to be able to do. And, you know, and if you like doing those things, you know, now's the time for you, so to speak. Based on this idea of what's coming and things getting automated, we're entering this time now, particularly in the world of software, where we can put up a prompt and you can create software. So like there's Replit, there's Windsurf, there's 语法解析

02:25:54

programs that are out there now where you can create an app, create a website, 语法解析

02:26:00

within minutes, which would normally have taken weeks or months or years even. So this idea of creating software, people are saying it's becoming more of a commodity. And you can kind of see this world of just the way we can publish podcasts in such a seamless way that distributes around the world, that publishes on YouTube or Spotify, is similar to the way that you're going to be able to publish apps on the App Store. And it's going to be this… 语法解析

02:26:24

kind of this hybrid of like YouTube meets app store people are saying? I don't think so quite. I mean, you've got to understand. Okay. You know, and actually I need to go soon, but, but yeah, yeah. Just, we can end it here. You know, this whole business about automating software, that's what I just spent the last four decades doing. People who use Wolfram language, uh, 语法解析

02:26:48

They go from ideas to stuff that runs in amazingly short amounts of time. The fact that there is this whole ecosystem of people doing manual labor software development is just bizarre. I mean, it's happened because of a bunch of the economics of labor, a bunch of the ways that technology is developed. But the fact is, you know, you say it's, you know, you can create an app. 语法解析

02:27:15

in, you know, that's what I do many times a day, writing tiny amounts of code. You know, it's, that's what, you know, that's been my objective is to automate those things. And the fact that, you know, I mean, what's happened is that there is a craft of software engineering where it's like you get a spec, you spend two weeks implementing the spec. 语法解析

02:27:39

Often, it's a funny thing to see in companies, the people who run them, the CTOs, the CEOs and so on, actually use our tech and build prototypes of things in a very short time. And then they go on and they figure out the next thing they want to do and build that as well. But the people who are now sort of in the trenches doing software engineering are like, well, we just got a spec. We build to that spec. Okay, now if we could do that much more quickly, which they can with our tech, 语法解析

02:28:08

Then they're like, well, now I've got something very difficult to do. I've got to make a new spec. That's not what my job is. My job is to grind out code. And yes, this is exactly an example of where the mechanical stuff is getting automated. I mean, that particular one, I know very well because I just spent the last four decades doing the automation of that one. And people have, you know, the fact that you can go from more like a natural language description to something 语法解析

02:28:37

to code is something which plays very well into our tech stack. But some part of that is, and then you can write a blob of Python code, which would just be one function in our language anyway. You wouldn't be writing that. But the AI can write that. But I think the thing to understand about things like software is, and it relates very much to what we've been talking about a lot in this conversation, is 语法解析

02:29:04

is you say, you snap your fingers and then there's an app. Well, what is that app supposed to do? You have to describe what it does. And that is making a choice, so to speak. And the description of what it does is the thing that is the thing that's going to be of value. 语法解析

02:29:23

It's not the mechanics of, you know, with our tech stack, for example, and all the people who use it, the kind of mechanics of doing a lot of things, those have been zeroed out for years. You know, those are not, I mean, which is great. It means one's been able to go further in lots of things and, you know, lots of kinds of developments in science and so on. But that, you know, so the thing, it isn't just, okay, make me an app. 语法解析

02:29:52

what the heck's the app supposed to do right it's it's like you have to say i want i have you have to conceptualize i want an app that's going to do this 语法解析

02:30:01

And then you start to sort of dig into, well, okay, how are you going to build that, et cetera, et cetera, et cetera. So it's actually, that is a really good example of the things I've been talking about. Namely, you know, what becomes the human act is what do I want it to do? Then, you know, as I say, in that particular case, 语法解析

02:30:23

I've spent the last four decades trying to do that automation, AIs, add another level of automation to that. But it doesn't get you around the, you know, you kind of, you know, it's like there's still somewhere you have to decide what the app is supposed to do. And I think that's the, and yes, that's the thing where you're, you know, again, that's 语法解析

02:30:52

That's what I've been living for many decades now, is getting to the point where it's mostly about imagining what the app is supposed to do, not about the mechanics of actually writing it. Wouldn't that still, regardless, increase the supply of software significantly, just like podcasts have significantly increased? And if that's the case, what's the moat for a software entrepreneur that you're advising if… 语法解析

02:31:21

software used to be hard and now it's easier and there's more supply than ever. Well, I mean, it's a question of what does it do, right? So in what I've done in my life, probably the thing of highest value, I think, is the design of our computational language. In other words, the implementation, sure, that's valuable. It's cost a huge amount of money to do it. And people use it 语法解析

02:31:46

you know, every day, all the time. But for me, it's actually an interesting thing you should say that because for me, the thing that is really of the highest value is what I've spent lots of effort on, which is kind of the functional design of the language. I mean, you know, I actually we live stream many of our sort of software design meetings. So the there's now I don't know what is 1000 hours of actually what's involved in doing that out there in the world. 语法解析

02:32:14

It's actually a very interesting intellectual activity. But that's the, you know, I suppose, in a sense, it's, you know, you could take that spec and you could reimplement it. Good luck with that. I mean, that's a that's a thing where it's a, you know, I think I think. 语法解析

02:32:37

The I have an idea, you know, there are so many different kinds of moats. I mean, there are moats to do with I've got this, you know, social media platform and all your friends are on it. You should be on it, too. 语法解析

02:32:51

Yeah. 语法解析

02:33:09

you know, we have been using various forms of machine learning and AI to help with software development for a long time. I mean, a lot of what we have to do is make meta algorithms that select between algorithms. And that's a thing for which we've used machine learning for a long time. It's again, the strategy of what's happening inside, which is a lot of human choices, is still a thing that you don't get to zero out. 语法解析

02:33:36

I mean, I think my company is what, about 800 people, which is really kind of tiny relative to what we've been able to produce. It's well, I think relative to the amount of software we've produced, 语法解析

02:33:52

How has that been possible? Well, it's because we've automated the heck out of things. And, you know, we're building on sort of a, you know, we built this tower where we're kind of recursively able to do more because we've automated the last thing we were able to do. So I think, I mean, it's an interesting question. What, you know, I think it's really a, well, what do you, you know, the space of possible software is, 语法解析

02:34:18

It's like these little programs that I study in the computational universe. Every one is a piece of software in a sense. Every one does something. Most of them are not things that anybody would care about. 语法解析

02:34:31

Some of them make really pretty pictures. Some of them make good cryptographic systems. Some of them make good image processing filters. Those are ones where we've been able to kind of mine what's out there and turn it into something that we find useful. But the leading piece to that has to be, well, what do you want it to do? What is the thing you want? 语法解析

02:34:53

So if you can say, well, you know, is there going to be a broader set of things that people want software to do? Maybe as people's activities broaden out, there will be a need. I mean, like, for example, well, take podcasting again. There's a bunch of software around podcasting that didn't need to exist until podcasting existed. 语法解析

02:35:11

And, you know, I think that will be, you know, in terms of the expansion opportunities. I mean, I always see this in terms of technology, the sort of technology that comes over the horizon that enables things like, you know, one that almost came over the horizon a bunch of times, but still is firmly sitting at the end of the rainbow is VR. 语法解析

02:35:33

or XR in general. It's kind of like one day that will really come over the horizon and that will enable a lot of new things, which it's like, how do I manage the Post-it notes, the virtual Post-it notes that I put in my environment or whatever? There'll be an app for that, so to speak. 语法解析

02:35:53

And that, you know, as the general tide of technology rises, so there start to be more and more things where we now have to figure out, well, what are we going to do? What particular direction are we going to take? And I really think it's very much the same story as the story of AI that, you know, more becomes possible, more becomes automated. Now it's a question of what do you, you know, what do we humans choose as the next step that we want to do? 语法解析

02:36:19

And talking of next steps, I need to go and do my next step. I think that's a great closing question for someone listening to ask that question. It's what are we supposed to do? Stephen, thank you so much for coming back on the show. This was such an intellectually interesting and thought-provoking conversation as usual. Where can people find you online if that's something you even care about? I don't know. StephenWolfman.com is a good place to start. And that has… 语法解析

02:36:48

Uh, you can also find me on all the usual social media platforms and, um, you can find, I do a, uh, uh, I guess it's a podcast. I do live streams a couple of times a week answering questions and so on. So that's, uh, that's, that's, those are places people can find me. Beautiful. Anyway. Thank you, Steven. Well, thanks. Thanks for lots of interesting questions. Thank you. And fun. 语法解析

</markdown>

Edit:2025.05.14

<markdown> </markdown>

讨论列表 AKP讨论 查看原帖及回帖