All Episodes
May 17, 2023 - Freedomain Radio - Stefan Molyneux
01:39:12
The Truth About Artificial Intelligence Part 2!
| Copy link to current segment

Time Text
Hi everybody, this is Dipan Molonyar.
I hope you're doing well. This is going to be an unapologetically lengthy but I promise value-packed presentation.
This is part two of our three-part series on artificial intelligence.
So we're going to talk about what's going on in the world now.
We're going to talk about the differences between open AI, delve into the details, guts and innards of chat, GPT, what are large language models.
We're going to talk about the leaked report from Google, What's going on in the world of open source AI? And last and perhaps most important, how are you going to prosper?
How are you going to add value in a world with AI? And this is really, really crucial stuff.
Let's just dig straight in. Worldwide interest in AI now.
So this is Bard, Chat, GPT, AI as a whole, OpenAI and Tesla.
I threw Tesla in here just to show you where it stands.
And this right-hand massive bulge.
Oh, it's incredible just how quickly this adoption, this fascination, this interest is going.
It took arguably 300 years for the agricultural revolution to give way to the industrial revolution.
The industrial revolution was 100, 150 years depending on how you measure it.
We have the information revolution, the computer revolution, we have the internet revolution, and now we have the AI revolution.
This is the biggest one, the biggest one by far.
If you look at the agricultural revolution, it was a consolidation of land.
Some people got kicked off, but most people did the same thing.
When they kicked off people ended up as the urban proletariat, then you had a labor source with which to build the Industrial Revolution.
So you went from working on a farm to working in a factory, still largely manual labor, trainable, and so on.
And then, sort of post-Second World War, once we start to get into the information revolution, then people start working more with language.
They start working more with concepts and ideas and arguments and advertising and propaganda, unfortunately, and all that kind of stuff.
And then what happened was the information revolution gave way to the computer revolution.
The computer revolution largely just really sped up what people were already doing.
So instead of going to visit someone, you could make a phone call originally.
Then you could, of course, have a video call.
Instead of mailing something, you could fax it first, and then you could email it as an attachment.
So what people did, they just did a lot faster.
The AI revolution is the first revolution in human history.
Where our language processing ability is being replaced.
It's never happened before.
And it's pretty wild when you think of the people who rely on language for their income.
I mean, that's almost all of the middle to upper middle class and certainly the elite classes as a whole.
So it is just wild what is happening.
And that's why I'm putting a lot of energy here so that you can navigate this because there's never been a bigger change as fast in human history.
All right. Let's look at world GDP over the last two millennia.
I'm going to do two slides on this.
So I did this presentation like, I don't know, 18 years ago on just how much we have grown from sort of the year zero From basically zero to well north of a hundred trillion dollars.
If you just look at the agricultural revolution happened first, but the agricultural revolution gave birth to more crops.
This sort of happened in the 14th, 15th and 16th centuries technologically and then 17th and 18th centuries in terms of land consolidation.
Winter crops, crop rotation, better manure, better ways of controlling pests, and food production went up 10, 15, and sometimes 20, not percent, but times.
I actually wrote an entire novel about this revolution, which is very underappreciated.
It's called Just Poor. You can get it for free at justpoornovel.com.
And so with this explosion in food production, you just don't need as many people farming.
And as I've said before, they migrate to the cities where they're available as a labor pool for the early industrialists.
So when you start to give people property rights and there's a genuine meritocracy The reason why we had such an explosion in food which allowed for an explosion in population which allowed for industrialization is because when land is not held by aristocrats but subject to the free market then the people who can best produce with that land can bid the most for it.
So if you can produce five times the amount of crops that your friend can, then you can bid a lot more for that land because it would be more productive.
So the most productive land ended up in the hands of the people who could produce the most with it, and you get this explosion.
So when you have property rights and free trade and free markets, beginning in land and then to intellectual property and Allowing for foreign trade and so on so that you get division of labor across countries.
This is the unbelievable explosion in human wealth.
And with AI, I think we're going to see a similar explosion as well.
So that's by the world as a whole.
This is... A little bit more broken down by country.
This is from obviously year zero, not very much going on in the US as far as free market capitalism went.
But as you can see here, Bangladesh, and then India, Indonesia, China, Argentina, Russia, South Korea, UK, France, Austria, US at the top with the greatest amount of free market capitalism around.
And you can just see it's absolutely unprecedented.
And this wreaks havoc in the human mind.
And of course you can see some of the hard left ideologies and some of the hard right ideologies, communism and fascism, was a reaction to this explosion in wealth.
And the only way you get this explosion in wealth is massive disparities in income.
The people who are best at producing stuff Have to be the ones who get the most resources.
The star of the movie gets $10 million or $20 million.
The extra in the background gets, what, $14 an hour or something.
That's just the way it has to be in order for this kind of productivity.
There's no music industry if the most popular musicians don't get the most resources.
It's just the same thing with sports.
People go see Taylor Swift concerts line up for three days for some reason.
People don't line up for three days to go see karaoke because that's more lowbrow entertainment, although I like it quite a bit.
So yeah, this massive explosion, we're going to see this replicated in a much shorter time frame.
We're talking, you could argue, 250 years since the height of the agricultural revolution to where we are now.
This is going to be compressed into probably, I would say, less than a decade.
So we're going to take a quarter of a millennia and try and compress it into less than a decade.
So this is going to be wild.
All right. So let's start looking at some of the technology underlying all of this stuff.
Here we have... Here endeth the flowery rhetoric and beginneth the details.
All right. So let's talk about OpenAI.
So, OpenAI is an artificial intelligence research lab consisting of for-profit and non-profit
entities that was founded on December 11th, 2015, initially only as a non-profit.
It was founded by a group of tech leaders including Elon Musk, Sam Altman, Greg Brockman,
Ilya Sutskever, and as names get more difficult to pronounce, Wajish Zadamba.
Probably not even close.
Maybe the last one is close.
And it's got its headquarters in San Francisco probably for at least the next five to seven
It's a converted factory.
It currently has 375 personnel.
There's researchers, engineers, and policy experts who are credited with developing the revolutionary technology.
Such a tiny, tiny number of people.
For those of you who don't know, a crisis law is really, really important to understand this, that in a meritocracy, The square root of the total workers produces half the value, right?
So in a company with 10,000 people, 100 of them are producing half the value, and of
that 110 are producing half the value of that, which means 10 people out of 10,000 are producing
25% of the value of the company.
You either let those people cook or your economy falls apart.
And yet, when you let those people cook, they become very wealthy, people resent, and then
the sophists come in and say, well, he's only wealthy, she's only wealthy because they stole
from you, and then you provoke all of this class conflict and all of that.
So such is the way of the world until we get a truly free society.
So you probably know some of these people.
Sam Altman is an American entrepreneur, programmer, and investor.
He co-founded Loop, Loop, Loop, L-O-O-P-T, I didn't know how to pronounce that.
He's CEO of OpenAI.
He was previously the president of Y Combinator and briefly held the position of CEO of Reddit until the toxic tendrils of Reddit probably half stole his soul.
And he invests in nuclear energy projects as well as tech companies.
So, OpenAI is funded by a combination of donations, investors, and research contracts.
Its mission is to develop and promote artificial intelligence in a safe and beneficial way for humanity's long-term welfare.
I love those mission statements.
they're so abstract. OpenAI aims to advance digital intelligence in a way that best serves humanity by
conducting cutting-edge AI research, creating advanced AI tools and technologies and providing
educational resources to the general public. Their mission claim, quote,
is to develop and promote artificial intelligence in a safe and beneficial way for humanity's long-term welfare.
According to GPT, which I'm sure is not totally
subjective. All right.
Now, in 2019, OpenAI transitioned from a non-profit to a capped profit company by forming OpenAI LP, a for-profit company owned by OpenAI Inc., the non-profit.
This allowed them to attract investment and grant equity to employees while capping the profits at 100 times any investment.
So, that is important.
If you're a non-profit, you're going to have to rely a lot on government money or some donations, but you're just not going to be able to compete with the amount of money that...
I mean, in 2015, when they opened, there wasn't that much interest relative to now in artificial intelligence.
As open AI succeeds and AI fascination and interest grows, then companies are just going to try and poach their employees, so you have to have some for-profit stuff to keep those magical green-thumb producers.
Microsoft invested a billion dollars into the company and OpenAI also announced plans to commercially license its technologies.
So if you don't go for profit, Microsoft's going to take that billion, put it into some other company or within its own investment and so on.
So despite the transition, OpenAI LP has a formal fiduciary responsibility to the non-profit charter of OpenAI Inc., OpenAI Inc.'s board has limitations where the majority of them cannot have financial stakes in OpenAI LP, and minority members with a stake in OpenAI LP are restricted from certain votes because of the conflict of interest.
The switch to a for-profit company has been criticized by some researchers for being inconsistent with OpenAI's claims to democratize AI. You know, it's a funny thing about profit, just sort of by the by.
I hope to profit from doing this presentation.
If you donate, because of all the hard work and went into all of this, and the hard work, of course, is not just knowing how to do these presentations, but the decades also spent in the high-tech world as a programmer and so on.
A lot of unique experience to bring to all of this, 30-plus years of entrepreneurial understanding, selling companies, all that kind of stuff.
So I hope to profit from it.
I hope that you are watching this because you're going to gain something of value out of doing this as opposed to everything else.
Don't do other things. Don't play World of Warcraft while you're doing this.
Please, I'm begging you. So profit is kind of funny, right?
You hope when you date a woman that you're going to be happier with her in your life than with her not in your life.
So you hope to profit from that.
Profit is just you want to benefit out of your time investment and somehow it's become this Dirty word, usually from people who fail to generate profit and instead generate aggression against profit in the hopes of getting other people's money without earning it.
Anyway, that's another sort of talk.
The 2020 Microsoft investment in OpenAI would provide the company with exclusive access to OpenAI's GPT-3 language model.
The deal also involved the two companies collaborating on the development of new AI technologies.
Microsoft's investment helped OpenAI continue its research and development efforts towards more advanced and sophisticated AI models while providing Microsoft with an edge in the highly competitive AI market.
So how has it been doing in terms of market performance?
With market performance, of course, being not just people's interest in it, but people's capacity to gain more out of it than they put into it.
Sorry, just sort of be econ 101.
People respond to incentives and all human desires are infinite and all resources are finite, blah, blah, blah, blah, blah.
But if you don't have a for-profit, then you don't know whether what you're providing is of mere interest or has significant economic value.
If you give away chocolate bars, you might give people a lot of cavities, but you don't know if you have a profitable business model.
So the reason why it's important to have...
A for-profit entity is so that people don't just show their interest because it's relatively free, but have to pay for it and therefore have to gain some economic value out of it.
So you find out whether it's beneficial to the economy or just a distracting interest for hobbyists in a sense, right?
So, released in November of 2022, within just five days of its launch, OpenAI amassed a user base of over one million users.
It took Netflix three and a half years and Spotify five months to get to a million users.
So we're talking about 30 to 1, 300 to 1 in terms of acceleration.
In two months, ChatGPT had hit 100 million users.
Isn't that wild? By March of 2023, ChatGPT had reached 1 billion users.
Now, of course, this might be duplicate emails.
It could be bots.
It could be any number of things.
But let's see. It took me, to get to a billion views and downloads of my shows, it took me 18 years.
So I would say that they're just a little faster than philosophy.
But it's okay. Philosophy is going to overtake AI. Alright, so as of this presentation is May 2023, ChatGPT has reached 1.6 billion users.
Wow, wow, wow, wow.
So, in 2022, OpenAI made an estimated $300 million.
OpenAI projects a 2023 revenue of $200 million and a 2024 revenue of $1 billion.
Now, just so you understand why this is so wild from a business standpoint.
So, when I was a software entrepreneur, I developed software, but it did have to be customized for most of the individual end clients, which meant the moment that human labor has to touch additional users, your profit margins go down, your growth potential goes down enormously.
What is the cost of, say, ChatGPT reaching an extra user?
Virtually zero. I mean, it's so close to zero, it might as well be zero.
So this level of scalability where you have a core value and it costs almost nothing to add somebody to benefit from that core value, this growth potential is truly wild.
And we'll talk about the economic value of it down the road, which is also going to be driving this growth.
So OpenAI received an extra $2 billion investment from Microsoft between 2019 and the beginning of 2023.
OpenAI has successfully completed a $300 million funding round with a valuation from $27 billion to $29 billion.
Often 20x earnings, sometimes more.
So Microsoft overall has invested $10 billion in OpenAI and as a result of ChatGPT integration Bing, which is their search engine, Experienced a 15% daily traffic increase, which is gold, of course, when you're trying to knock Google down a couple of notches from the perch.
So OpenAI is pretty wild, right?
So it has the capability to generate recipes using a provided list of ingredients, and it's probably not three minutes away from photograph your fridge and make some chicken masala.
The OpenAI platform is available in 156 countries worldwide.
And the largest number of companies using OpenAI have 10,000 or more employees.
And the tech sector has the highest number of companies adopting the platform because you have code that writes code, essentially.
Writes code, validates code, parses code, parses out code, maybe even comments on code.
I don't even know. But the fact that you have computers that can write code is just wild.
According to recent chat GPT-4 data, OpenAI is developing a GPT-4 powered mobile app.
And of course, if it gives permission to cameras and voice and so on, and you can voice dictate or take pictures and have them parsed and analyzed and so on, it's just going to be wild.
OpenAI can transform movie titles into corresponding emojis.
So, if you look at the movie title, The Glorious Story of Stéphane Molyneux, it will translate that into a giant thumb emoji.
Oh, can't do a presentation without a ball joke, can I? Alright, so the time it took for selected social media services, let's just go straight to tech to reach 1 billion monthly active users in years.
Facebook Messenger, almost 5 years.
TikTok, a little over 5 years.
WeChat, 7.1 years.
Instagram, 7.7 years.
YouTube, 8.1 years.
WhatsApp, 8.5 years.
Facebook as a whole, back of the day, 8.7 years.
Years to get to 1 billion users.
And again, we're talking months for AI. And now, of course, some of this is going to be, oh, I'm curious, I'm fascinated, it's kind of neat, I'm going to type around, and then you just forget about it and abandon it and move on.
I get some of this. Not every one of this is an entrepreneur and is going to build a company or whatever, but it's still, it's just wild.
So... We'll get into, I mean, the market opportunities for AI are truly staggering, and we'll sort of get into that a little bit later.
But yeah, it's just incredible how fast it's gone.
Where are people finding out about this?
This is the web analytics.
So OpenAI Desktop, where do people come?
Direct traffic, boom. I mean, that's the number you want.
Obviously, having Elon Musk involved in anything is what Tesla has, what?
No marketing budget because everybody is fascinated by Elon Musk.
And so whatever he does is going to translate into instant marketing and so on.
So direct traffic, these are people who come to your site without a referrer to use and just type it in directly.
The referrals here are virtually zero from referring websites.
Organic search is you do the search and boom, you get there.
That's the second biggest by far.
Paid search, virtually nothing.
Social media, from email, from display advertising.
Again, it's very, very organic.
As of May 2023, OpenAI.com is the 16th most visited website globally.
It went up two places in just one month.
In the last month, OpenAI.com has had 1.8 billion, billion site visits.
It's just wild.
I... I feel like I'm surfing with sharks coming out of the slipstream of water because it's just moving so fast.
And if you follow people on social media, it's like, today's news in the world of AI is explosive!
And I get that some of that's hype, but a lot of it is pretty true.
So, user demographics by age.
So, it's very interesting.
So, 18 to 24, you know, 27% or so.
25 to 34, 35%.
35 to 44, a little under 20%.
It goes down 45 to 54.
55 to 64, 65 plus, less than...
5%, and it's 61.52% male and 38.38% female.
Now, if you're a little surprised that there's almost 40% of the users are women, well, it's language-based, and women, much as we love them, are quite significantly interested in language rather than, say, tech or hardware or coding and so on.
So the fact that it's a language model is going to be more interesting to women.
Again, as a whole, lots of exceptions, but that's the general trend.
Countries that send the most traffic to open AI. The US is the biggest at 15.32%, which given the robust VC infrastructure there would make sense.
India's second biggest, 6.32%.
Ironically, of course, India has a lot of tech support, which is probably going to be replaced significantly by AI. Japan, 3.97%.
Colombia was a bit surprised at that, 3.28%.
Canada, 2.74%.
From January to March of this year, 2023, the U.S. has seen a 73.46% increase in its rate of visitors.
It's pretty wild. Okay, so let's start with ChatGPT.
ChatGPT. Three, yesterday's model in the high-tech world of high-speed, let's go ludicrous speed.
So let's look at the large language model.
So GPT-3 was unveiled in 2020 by this OpenAI, and a language model trained on large internet data sets for natural language answering translation and text generation.
So, a large language model, it's an AI system that understands and generates human-like text using deep learning techniques.
Trained on extensive text data, these models excel in tasks like translation summarization and sentiment analysis.
I've heard them described and I get it to a certain degree.
I'll just pass it out for you. You can make your decision.
AI has been described as a word guesser.
So you analyze a huge amount of language and you guess the next word.
It's a word guesser.
Now, I think it's a little bit more than that, but that is one of the technologies that underlies the structure.
As a whole, be unpredictable, squirrel!
Be unpredictable and you can beat AI. And unpredictability and originality are two sides of the same coin, which is why I really, really urge you to study philosophy, to think for yourself, to be original, because then you are loved, hated, and completely irreplaceable as a whole.
So I would look at that for sure.
So a large language model, notable examples include OpenAI's GPT-4, Google's BERT, which leveraged the transformer architecture for efficient processing and improved language understanding.
The transformer architecture, this can effectively simulate understanding and processing of complex language patterns.
And of course, GPT of ChatGPT is generative pre-trained transformer, which is no part of the robots to cars movie sequence, but obviously it should be.
Now, what is the difference between GPT and chat GPT? So chat GPT and GPT-3 differ primarily in their design focus.
Chat Obviously with the chat part of it, chat GPT is specifically tailored for conversation modeling.
GPT-3 has a more diverse set of capabilities.
GPT is the more general model.
You'll hear GPT and chat GPT throughout this presentation sometimes interchangeably because we're just aiming to be as convoluted as your lower intestines.
Just in general, that's the way that it's discussed in the general world view.
All right. So, Dali.
I mean, I'm a stay-at-home dad, so I thought, oh, Dali, that's named after Wally, the cleaning robot from that movie.
But no, of course, it's...
Then on second glance, I'm like, no, no, no, Salvador Dali.
That's where they get Dali.
So... OpenAI launched early in 2021.
It's a deep learning model that uses a version of GPT-3 and can produce digital images based on natural language descriptions.
Just wow. I mean, gosh, I remember way, way back in the day, decades ago, Microsoft had, I think it was the Louvre, it was, you could go through and it would explain to you art and break it down.
The amount of digital imagery that's out there in the world is just amazing.
It's just amazing. And of course, once you have digital anything, you can train a computer to recognize patterns as a whole.
GPT 3.5.
In December 2022, OpenAI gained significant media attention following the release of a free preview for chat GPT, their latest AI chatbot, which is built on the GPT 3.5 model.
OpenAI reported that the preview attracted more than a million sign-ups within its initial five days.
Anonymous sources mentioned by Reuters in December 2022 also confirmed this.
So GPT-4, this is March 14th, 2023.
Again, just a short number of weeks ago.
OpenAI introduced GPT-4, making it available through an API with a waitlist and as a component of chat GPT-plus service.
Now, GPT-4 was trained to be capable of handling vision, though the public has not yet been granted access to this variant and the vision feature.
So, you might be able to give it an image and ask for informational feedback about it, or even get another image as a response.
Not quite a hack. Not quite the opposite of a hack either.
All right. I'm sorry if you're on your cell phone.
I'm afraid, just to get all this information on the screen, it's two-point SquintoVision text, I'm afraid.
So, how do you do this?
So, there's a couple of steps.
For ChatGPT. So you train a supervised policy on human-generated examples.
So a prompt is drawn from our prompt set.
So explain reinforcement learning to a child.
A labeler provides an example of the desired output.
We give treats and corrections to teach.
And then ChatGPT 3.5 is fine-tuned on the data using supervised learning.
So that's step one.
Step two, gather comparative data to train a reward model.
So the prompt is, explain reinforcement learning to a child.
Output A, we give treats.
Right? That's one part.
Output B, rewards R. Output C, in learning.
Output D, teaching requires.
And then you order the outputs from best to worst.
So in this case, D is more important than C, is more important than A, is more important than B. And the reward model is trained on this data.
So, I'm afraid we're going to have to have...
It's funny, I remember way back in the day, talking to a guy, I had a big database, and talking to a guy who ran an army database, and I said, oh, my database will handle acronyms.
And he's like, the database of our acronyms is bigger than your entire database of environmental improvements.
Kind of true, right? So I'm afraid we're going to have to...
Acronym it up just a smidge.
So there's something called proximal policy optimization.
It's a policy gradient method that can be used to train agents to learn optimal policies in reinforcement learning problems.
A policy is a function that maps from states to actions.
The reward model is a function that maps from states and actions to rewards.
The goal of reinforcement learning is to find a policy that maximizes the expected problem.
Reward. The PPO algorithm does this by iteratively adjusting the policy to make it more likely to take actions that lead to higher rewards.
So, do you see the sort of meta-self referential stuff that we're talking about here?
Explain reinforcement learning to a child and we're explaining how reinforcement learning is in computers, but to you, wonderful adults.
This is Inception-style PowerPoint.
We're quite thrilled by all of this.
So, how do you adjust a policy for the AI to make it more likely to take actions that lead to higher rewards?
Well, you use a technique called policy clipping, also known as not letting your ducks fly.
I know this from personal experience. Policy clipping prevents the policy from changing too much at once.
This helps ensure that the agent learns a stable and robust policy.
A policy is a function that maps from states to actions.
It is a probability distribution over actions given a state.
The goal of PPO is to find a policy that maximizes the expected reward.
So, step three, you're using this PPO reinforcement learning algorithm to improve a policy by adjusting it to maximize the expected reward.
So, you get a new prompt from the dataset.
Prompt, write a story about ducks.
Did my daughter help me on this presentation?
I'll let you decide.
So, the PPO model is started with a supervised policy.
The policy produces an output.
Once upon a time. The reward model assigns a value to the output.
The reward is used to improve the policy using PPO. So you can play this back a couple of times.
But this is, in general, how it works and how it iteratively improves over time.
Of course, now that it's open to the public, the public is available and able and often will provide feedback saying, this worked, this didn't work, this matched, this didn't match.
Some people will do that honestly in an attempt to improve the objective quality of the AI model.
Some people will, I assume, spam it with propaganda in order to alter its outcome.
Sorry, gaining control over the output of AI is going to be a foundational fifth or sixth generation warfare battle, to gain control over the output of the AI, because the AI has a godlike Oracle of Delphi sense from people.
They give it the sense that it's objective, it's absolute, it's as trained as humanly possible, it can't get better, and therefore to control the output is to infuse what could be propaganda with a sense of So this is just really going to be an important thing as we go forward.
So, large language models development.
Data collection and pre-processing.
So what do you do? You collect a diverse text data set from sources like web pages, books and articles.
You clean and tokenize the text for training.
So, the term token.
In the field of AI, a token is a unit of natural language.
It could be multiple words, it could be whole words or parts of words.
Including punctuation and single symbols.
Developers can, of course, take different approaches to how they tokenize their dataset, right?
So, in order to reassemble words into new sentences, you need to take existing words, sentences, and parts that are probably not down to the morpheme level, and you have to reassemble them, right?
That's central to that.
So then you model the architecture selection.
You choose an appropriate model architecture, like the transformer, which handles long-range dependencies and paralyzes computations.
So what does that mean?
So you could also include recurrent neural networks, which process sequential data by maintaining a hidden state.
We talked about that in the last presentation, which you should check out.
I probably should have mentioned that at the beginning.
Watch the other presentation first.
I'll put that in the notes.
Convolutional neural network is mentioned in the Truth About AI Part 1, and of course there's many, many different ways of doing this.
So model initialization.
Initialize model parameters with random values or techniques like pre-training on smaller tasks.
So this is where we get the term parameters.
A parameter is an adjustable value in a model, like an x-factor.
So parameters are just inputs to a particular program.
So one of the first programs I ever wrote when I was 11 years old was...
Enter a number. Enter another number.
I will now multiply the two together.
So 6954 you give the response.
The six and the nine are parameters, and they really are the purpose of the program.
So it's an adjustable value in a model, weights and biases, that is learned during training to optimize performance and make accurate predictions on unseen data.
So in pre-training, you train the large language model unsupervised using masked language modeling or autoregressive objectives, predicting tokens based on context.
I know it's a little bit of a word salad, let's break it out a bit.
Masked language modeling is pre-training techniques for large language models where a portion of input tokens are randomly masked and the model learns to predict the original tokens based on surrounding context.
This unsupervised learning method helps the model capture semantic and syntactic patterns in the text.
So you could say to the computer, Once upon an X, right?
Once upon an X. And if the computer says, well, the most likely answer is time, then you reveal, yes, good, here's a kibble, here's a reward.
So you, if you ever played this game, I played this with kids a lot, which is either the
two or three language, two or three letter game.
So you're all sitting around a table and you say, once upon a time there was, and everybody
adds and it gets kind of funny and sort of hysterical as this story goes along because
everyone's adding their own little bits.
Now at the beginning it's fairly easy to predict what people are going to say.
You know, once upon a time there was a, you know, 50% of the time it'll be there was a
or once there was or something like that.
So you know what the next letter is going to be, and you don't tell the computer what the next word is.
You don't tell the computer what the next word is.
If the computer guesses correctly, it gets a reward, or it doesn't get a reward.
It sounds like you're training. It is...
It is told to prioritize that guess higher.
And this is so the word guesser is a lot of training stuff.
And again, some of it is manual.
I assume not a huge amount.
But some of it is unsupervised.
So that's free training.
All right. I'm afraid there's more.
Hold your horses. We're almost done this part.
I know it's a little bit of a speed bump.
So, fine-tuning. Optionally, fine-tune the pre-trained large-language model on domain-specific datasets or tasks to adapt to specific applications.
So, for example, if you are asking a large-language model to analyze haikus, obviously you'd want to give it the syllable argument or the syllable restrictions for haikus.
If you wanted to do iambic pentameter, you would give it those limitations and so on, right?
If you wanted to give it limericks or knock-knock jokes, you could fine-train it for that kind of stuff.
Or, of course, in a more productive way, you could train it on law, procedures, and rules, and precedent, and so on.
So, fine-tuning is, if you want an AI medical assistant, you feed it medical data.
And you want a dungeon master or a convincing NPC? Well, if you want a convincing NPC, you just feed it the mainstream media as a whole, but that's the general idea.
So then you go to evaluation.
You assess the large language model's performance with evaluation metrics, guiding further improvements.
So the evaluation is quantitative measures used to assess an AI model's performance on specific tasks.
They help identify strengths, weaknesses, and areas for improvement in the model's predictions or classifications.
So if you give it, here's a limerick, here's a knock-knock joke, and so on, then does it actually get those things correct?
If not, you figure out what went wrong, give it further improvements.
So then you go to hyperparameter tuning.
Optimize hyperparameters such as learning rate, batch size, and number of layers to improve performance.
So it's all a bunch of...
It's an ecosystem. It's not just one piece of code in and out.
It's a whole ecosystem.
Hyperparameters are adjustable high-level settings of a machine learning model that control its learning process.
So if you want it to learn fast, obviously, I mean, like everyone, you want to learn fast, but not so fast that you make mistakes.
Like when you're learning how to ride a bike, you want to ride fast, but not so fast that you crash.
Learning rate, batch size.
The fewer the batch sizes, the more efficient the process, but you don't want so few batch sizes that you have it make mistakes, and so on.
So, regularization and optimization.
You apply regularization techniques to prevent overfitting.
And we'll get there. We'll get there.
Overfitting is a scenario where a model learns the training data too well, hindering its ability to generate...
So, and use optimization algorithms to update model parameters.
And so there's a variety of things about that.
If some of this stuff is really messy, I will of course link to all of the definitions and sources and notes In the notes, in the description of this presentation, so you can go and read this in more detail.
So then you go to monitoring and iteration.
So you monitor the training process, adjust the strategy or hyperparameters if needed, and iterate to improve performance.
Iteration just means basically rolling it over, starting it again with the new So, I mean, silly example, right?
So, if you have a little program that I wrote, first program, other than print hello, go to 10.
So, you say, input the first number, input the second number.
Here's what they are when they're multiplied, right?
And if your answer is 12, then clearly you've made a mistake in your process, right?
So instead of multiplying them, you've added them.
So instead of 36, you're getting 12.
So then you would check the code, you would tweak the code, and replace the plus with the asterisk, which is basically multiplication for computers, at least it used to be.
So, you just, you check the output, see if the output matches what you expect, and you just, I assume it's just a fairly voodoo process of tweaking until you get it right, because there's probably not one person in the whole world who understands all of this stuff and every way that it's affecting the output.
Alright, have we had enough text?
I think we've had enough text. Let's get to bubbles!
We all like bubbles, don't we?
Alright, so let's compare GPT-4 and GPT-3.5, right?
So 2022, this is one year's difference.
2022, GPT-3.5.
2023, GPT-4.
And just look at these parameters.
So the... I'm not going to go through all these numbers, but trust me, they're wildly different.
Wildly different. Parameters, dark green is GPT-4, and GPT-3.5 is the smaller brown, and it's just many, many multiples, which is one of the reasons why GPT-4 is so compelling for people.
And of course, the cost of training GPT-4 is $100 million.
The cost of training GPT-3.5 was $4.6 million.
So that's, you know, more than 20 times the amount.
It's just wild.
So OpenAI has not released the number of parameters GPT-4 was trained on, but Semaphore.com claims to have spoken with insiders reporting 1 trillion parameters, right?
So it's a trillion parameters for GPT-4, 175 billion for GPT-3.5.
As for tokens, so remember these are units of natural language, like words and parts of words standardized to make it easier for the AI to work with.
The maximum token content is the amount of information you can give to and get from the AI in a prompt.
Maximum token content for GPT-4, 32,000.
For GPT-3.5, 4,000.
I guess I am going through the numbers after all.
So, OpenAI claims GPT-4 is capable of...
32,000 tokens of context.
So we'll use that here. In practice, though, we found ChatGPT has the same token limit as GPT 3.5.
About 4,000 tokens.
About 2,600 words.
This is likely an artificial limit as managing and predicting the cost of serving much higher control prompts and responses is difficult.
So I can imagine the throughput growing over time, though.
So... Of course, we got into the details of petaflops in our last presentation, and so we can do all of that, and it's just wild.
Now, these are only trained on data up to September of 2021.
It doesn't have awareness of anything after that date, unless, of course, you provide it.
AI models that have direct contemporary current access to the internet.
So you can get current stock prices, current weather, and so on.
So let's talk about this in terms of performance.
So if we look at the academic performance for GPT, so this graph shows different academic tests human beings regularly take, so we can see where GPT variants fall in comparison.
In a sense, how smart is it?
The scale is by percentile, so 80th percentile would mean that GPT performed better than 80% of test takers.
As mentioned earlier, GPT-4 has a variant capable of vision, so we included its test scores in here as well.
So, I'm not going to go through all of these in detail.
Sorry if you're just listening to audio, I apologize for that, but to point it out, it is GPT-4 It's doing very well relative to GPT 3.5.
So I'll just go through these real quickly.
UBE, the Uniform Bar Exam, standardized bar examination in the U.S., designed to test knowledge and skills.
So every lawyer should have, before becoming licensed to practice law.
Three components. Multi-state bar exam, multi-state essay examination, multi-state performance test.
So if we look at this... GPT-4 is almost the 90th percentile, which means it does better than just about 9 out of 10 lawyers taking the test or potential lawyers taking the test.
If you look back at GPT-3.5, It only got, well, it got just under 10%.
So we've had a 900% increase in, like, a very short amount of time in terms of how well it's doing.
So that's wild. So then we do the LSAT, Law School Admissions Test, standardized test used for admissions to law schools in the U.S., Canada, and some other countries.
And here again, we can see almost 90th percentile.
GPT 3.5 did better in that it got almost to the 40th, but we've had a doubling of...
The SAT, Scholastic Assessment Test, standardized test used for college admissions in the U.S. The SAT, Evidence-Based Reading and Writing Test, is designed to assess a student's reading comprehension, critical thinking, and writing skills.
And here, it's been a little bit less, because it's a less complicated test, of course.
It's designed for, you know, 17, 18-year-olds to get into college, rather than, obviously, people in their early to mid-20s trying to get into law school, which is going to have a higher IQ source, which means going to be...
It's going to be easier to improve with greater language modeling.
So SAT, AI, GPT-4 and GPT-3.5, very close over the 90th percentile.
That's reading, writing, SAT, math.
There's been more of an improvement and so on, which I find a bit surprising because computers are really great at math.
GRE, Graduate Record Examination.
The quantitative section, part of the GRE, a standardized test, requires for admission to many graduate schools in the U.S. and some other countries.
The quantitative section assesses problem-solving skills, mathematical skills, and the ability to interpret data, which is a huge strength, of course, of GPT-4.
And if we look at this quantitative one, GPT-4 is scoring very well, right?
It's almost at the 80th percentile, and it was just a little over the 60th percentile for GPT-3.5.
Five. So we get to the graduate record examination.
There's two sections, no, three sections.
There's quantitative, verbal, and writing.
And for a quantitative, this is a standardized test required for admission to many grad schools in the U.S. and some other countries.
The quantitative section assesses problem-solving abilities, mathematical skills, and the ability to interpret data.
And GPT-4 is almost at the 80th percentile.
GPT-3.5 is a little over 20th percentile, which means almost four out of five people taking the test do better than GPT-3.5.
Now, the verbal is really the killer app to some degree, right?
It's a large language model, so we respect the verbal aspect of the GRE to be very strong, and it's almost 100%.
It's better than almost every single human being at taking GRE verbal chat.
GPT-4. GPT-3.5 was 60 and a bit percentage, so that's quite a bit different.
GRE writing?
That is a straight-up writing test and not been much difference, really.
It's a little bit over the 50th percentile for that sort of stuff.
All right, let's look at the USABO. What is that?
The United States of America Biology Olympiad.
It's a prestigious biology competition for high school students in the U.S. Various exams attest to students' knowledge and understanding of biology at a fairly high level.
And, boy, again, even better than the GRE Olympiad.
Verbal, the chat GPT-4 is almost 100%, better than almost every other carbon-based life form on the planet.
And GPT-3.5 was only clocking at 30%.
So again, in a very short amount of time, it's more than tripled its competence.
All right, CodeForce rating.
CodeForce is a competitive programming platform that hosts online programming contests where participants solve algorithmic problems within a specified platform.
Time. Code forces rating is a measure of a participant's skill level based on their performance in the contests, and here it's very low.
This is part of, like, coding.
I mean, I've written poems, novels, plays, I've been an actor, and the level of creativity, and I've been a coder for many years as well.
Coding is foundational, inspirational, sparks and fireworks in the brain creativity.
So... It is finding radical new solutions to existing problems, and so the more creativity that is required, the less AI is going to help.
And so that's really, really, again, we'll get to this in terms of how this presentation helps you navigate the biggest change in economics, if not society, in human history, and I'm not kidding about that.
There's a reason I'm pouring heart, mind, and soul into these presentations.
It's to really help you navigate this tsunami of change that is coming and ride it out on top rather than being, I've blown the bubbles from below.
So yeah, they're really terrible, 4-5%, and there hasn't been any difference between GPT-4 and GPT-3.5, which is kind of what you expect.
Large language models are focused on human spoken language rather than code.
All right.
Art history.
That's AP art history, again, because it's analyzing text.
Both GPT-4 and GPT-3.5 scored above the 90th percentile.
Very, very good.
AP biology, again, 90th percentile GPT-4, but it was only the 70th unchanged percentile for GPT-3.5.
Now, let's look at AP Calculus.
So, this is Advanced Placement Calculus BC. It's a college-level calculus course offered to high school students in the US. The BC stands for both courses.
So, the first and second semesters.
If you look at GPT 3.5, it almost didn't exist in this.
It just, it failed, like, insanely badly, right?
But GPT-4 is at the 50th percentile, which means it's right there in the middle of the bell curve as far as this stuff goes.
So, yes, this is very, very important stuff.
All right, let's look at global private investment in AI by focus area, right?
So this is from 2017 to 2021.
Okay. So where are they really focusing on AI providing the largest value as a whole?
So large language models, chat GPT-4, are what have brought AI to the forefront of sort of human consciousness, of global consciousness, but it's interesting to see.
It's not even in the top three area fields of investment as of 2021.
Again, data is really hard to come by that's more recent to this, but...
The top three, data management, processing, cloud, medical and healthcare.
That's huge, huge in AI. I mean, gosh, can AI scan an x-ray to find lung tumors before they're really visible or easy to spot even by a trained person?
Could you upload a full body scan and have AI scan for medical issues?
What about just taking a photograph of your back, uploading it and have AI look for potential melanomas and so on?
Is it just a wart?
What about scanning of x-rays or other forms of non-invasive scans of breasts, looking for breast cancer and so on?
It's just wild. Virtual nurses that step people through triage to get them to a place.
As I mentioned in the last presentation, the vast majority of people prefer talking to an AI doctor
than a real doctor, because they say AI is much more empathetic and sensitive.
And you know the old statistic that a doctor will listen to you describe your issues
for about 18 seconds before interrupting.
So, yeah, so data management process in cloud, you'd understand medical and healthcare.
The US spending on healthcare, largely as a result of public money
and private profit motives is huge.
They really try and bring those costs down a lot, Particularly, I mean, AI is going to carve out and eviscerate the middle management layer of paper pushers and shufflers.
And so that's going to be a huge thing.
And of course, I'll get to the politics of it later, if I remember.
I'm sure I will. I'm sure I will. Much further down is retail.
And then fourth is natural language and customer support.
So I think that's really...
So in 2022, global AI private investment was $91.9 billion.
That's according to a 2023 Stanford report.
Here are some takeaways from the report.
And these have been summarized by chat GPT-4.
One, the industry has surpassed academia in producing significant machine learning models since 2014, with 32 major industry models versus three from academia in 2022, due to the industry's greater access to resources like data, computing power, and funding sources.
So because it's for profit, it's going to attract more money that produces greater growth.
Two, although AI continues to achieve state-of-the-art results, improvements on many benchmarks are marginal, and benchmark saturation is increasing rapidly.
And this will probably plateau for a while, and then there might be another breakthrough when they hook my brain up and it becomes the Internet.
Three, AI models have begun to rapidly accelerate scientific progress, contributing to enhancements in hydrogen fusion, matrix manipulation efficiency, and antibody generation in 2022.
Boy, wasn't there...
I had a screensaver many years ago which was supposed to help cure cancer.
Anyway, issue due cancer processing.
Four, the number of AI-related incidents and ethical misuses has increased 26 times since 2012, evidenced by events like deepfake videos and call monitoring technology in prisons indicating greater AI usage and awareness of misuse possibilities.
Five, AI-related job postings have grown across nearly all American industrial sectors, increasing from 1.7% in 2021 to 1.9% in 2022, revealing a rising demand for AI-related professional skills.
Again, that's not just coders, that's being a chat prompt engineer or whatever you want to call it.
Six, global AI private investment decreased by 26.7% in 2022, with a decline in the number of AI-related funding events in newly funded AI companies.
But overall investment in AI funding over the last decade has significantly increased, right?
So, I don't know, obviously, but as a guy who's been an entrepreneur and an investor for many years, I would assume that the reason why private investment has decreased is...
When you've just produced the new car, the new model, your R&D goes down for a while because you've got to sell and recoup your investment.
So because AI, particularly 4.0, the ChatGPT 4.0, as we saw in the last slide, has just improved by leaps and bounds over 3.5, they just need to make their money back.
So they're going to invest less in improving, especially if the improvements are more marginal, as we're sort of seeing from these reports.
They're going to take a break from investing and they're going to try and profit from what they've already invested in, which is exactly what should happen and exactly right now.
While the proportion of companies adopting AI has plateaued between 50 and 60% in recent years, those that have implemented AI reports, significant cost reductions and revenue increases.
Right. It's got to propagate.
It's got to spread.
Eight. Policymaker interest in AI has grown, with an increase in AI-related bills passed into law from one in 2016 to 37 in 2022, and mentions of AI in global legislative proceedings rising nearly 6.5 times since 2016.
9. Chinese citizens have the highest positive perception of AI products and services, followed by those from Saudi Arabia and India.
While only 35% of Americans believe AI products offer more benefits than drawbacks.
Right. So part of this warfare, I mean, we can say, oh gosh, we've got to take a pause in our AI research and so on.
I think Elon Musk has promoted that.
Okay, fine, but other people won't.
And what you want to do, if you are, say, China or another country, and you want to get ahead with AI, you will share a whole bunch of fear-mongering stuff about artificial intelligence in the hopes that investment dries up and legislation puts a pause on things so that you can race ahead.
You can race ahead.
That's all it's about.
That's all it's about. All right.
ChatGPT is not the only large language model out there.
So let's take a look at some of the others because they're also going to be pretty wild as well.
And some of them can do things better.
All right. Here we go.
Here we go. I know it's a lot of info.
You can take this in bytes, but don't just do the whole thing all at once.
We can do it. You can hang in there.
I promise you. It's a lot of information.
But, you know, a lot of people are going to see this and we don't want to...
We don't want to limit what you can process.
So let's get in there.
So Google, what have they got?
Lambda is a specialized dialogue model with 137 billion parameters.
Trained on 1.56 trillion words.
Actually, it's approximately the number of syllables I utter in 1.47 shows, I think was the last parameter that we got.
So unlike traditional chatbots, it isn't limited to predefined paths and can adapt to conversational directions.
They've also got BARD based on Lambda technology.
Unlike ChatGPT, BARD can provide more current information.
There's also PALM, a 540 billion parameter language model.
It outperforms either model, other models and humans using a few-shot learning approach to generalize from minimal data.
Also, during the making of this presentation, right before breaking news, Google released Palm 2, which looks like BART will be based on going forward.
Not for certain, but seems to be the case.
DeepMind is a subsidiary of Google.
They have Gopher with 280 billion parameters.
It's more accurate than existing large models on specialized subjects and equal in logical reasoning and mathematics.
Chinchilla, for when your neck is chilly, has 70 billion parameters and uses four times more data on the same computing budget as Gopher.
It outperforms Gopher, GPT-3, and Megatron Turing NLG, it's a great name for a rap group, on various tasks while using less computing for fine-tuning and inference.
We didn't show Sparrow, developed by DeepMind.
That's a chatbot designed to provide correct answers while reducing unsafe and inappropriate responses.
It addresses the problem of harmful model outputs, and it's trained to be more helpful and correct.
So it's your finger-wagging, carrying art, which, you know, has value as well.
All right. What else have we got?
Anthropic. Claude, an AI-based conversational assistant, aims to be helpful, harmless, and honest.
As if these three things can all be the same thing.
Using constitutional AI and AI safety methods, Claude was trained to exhibit these behaviors during its training.
Anthropic is an organization specialized in AI research and safety.
They aim to develop AI systems that are trustworthy, explainable, and can be...
Controlled. Baidu always makes me think of Erica and hair pillars.
Baidu is a Chinese multinational technology company specializing in internet-related services, products, and AI. They have Ernie 3.0, excels at natural language understanding and generation with 260 billion parameters.
Achieving steadily-out results in over 60 NLP neuro-linguistic programming tasks, it also performs well in 30 few-shot and zero-shot benchmarks.
We've got the Ernie Bart, an AI-powered language model similar to ChatGPT, capable of language understanding, language generation, and text-to-image generation.
It is part of the global race to develop generative AI. Huawei, or however you pronounce it, Huawei.
They have Pangu Alpha, the Chinese language GPT-3 equivalent, has over 200 billion parameters, trained on 1.1 terabytes of Chinese language sources.
It completes various language tasks.
Matter, formerly Facebook, why they gave up one of the most recognizable names in corporate history, is still a mystery.
So they have OPTIML, a pre-trained language model with 175 billion Parameters excels in natural language tasks like question answering, summarization, and translation.
Also, there is BlenderBot 3, built on Meta AI's OPT175B. It incorporates conversational skills, can carry out conversations using long-term memory and internet searching.
Very powerful. And then there's Lola Llama.
Llama has multiple sizes from 7 to 65 billion parameters with 13 billion parameter models, surpassing GPT-3's performance On NLP benchmarks.
The largest model rivals state-of-the-art models like Palm and Chinchilla.
Its weights were released under a non-commercial license for researchers.
However, within a week, the weights were leaked publicly on 4chan via BitTorrent.
Lama is really the star of the slide, really.
Having been leaked, it's become a focus of the open source community.
This has great repercussions, which we'll get to in just a second.
I haven't shown but wanted to mention NVIDIA Megatron Turing Natural Language Generation.
It's a 530 billion parameter transformer-based model released in 2022.
It performs better than GPT-3, but not significantly.
Now, just remember, like, more parameters doesn't always mean...
Better as a whole, right?
I mean, if you have more megapixels in storage, but you have a worse lens, it doesn't really do you that much good.
So just remember that. It's not just the one number solves everything.
All right. So...
Where does AI training data come from?
So GPT-1, just 4.6 gigabytes, all of this is in gigabytes, just books.
And GPT-2 sunk its courage into the often sewage lava pit of Reddit.
For 40 gigabytes, GPT-3, Wikipedia, books, journals, Reddit, Common Crawl, Other, and so on.
The Pile, again, I'm not going to go through all of these numbers, but there is all but GPT-1 and Gopher took from Reddit and so on, right?
So, the books are a combination of fictional and non-fictional books, and they aid in generating coherent stories and replies.
So, you cause Project Gutenberg and Smashwords and other places like that.
The journals, pre-print and published journal articles, Strong Foundation, they showcase systematic, logical, and thorough writing.
You get RXIV and the National Institutes of Health and so on.
Reddit links and web text is a vast data set derived from web scrape of outbound Reddit links that received at least three upvotes.
This heuristic measure of popularity potentially indicates higher quality content and text data.
And again, when people find out the inputs, they're going to manipulate these to produce with the Oracle of Delphi authenticity perception of AI. They're going to try and produce the output by controlling what goes in.
So, a common crawl, a dataset containing web crawls from 2008 to the present, featuring raw web pages, metadata, and extracted text.
It comprises text from a variety of languages and domains.
A widely used English-only version called C4 is often employed as a dataset by major research labs.
Common Crawl is up to date as of September 2022 and contains 380 terabytes of information.
Although I guess my first website went up in 2006.
So, ooh, escaped at the bottom, right?
So other datasets that don't fit into the categories above include code datasets like GitHub, conversational forums such as Stack Exchange, and video subtitle datasets.
This is where it's getting all its stuff from.
All right, so let's talk about this leaked report that came out of Google.
And this is centralized versus decentralized, pyramid versus fog, the cathedral versus the bazaar, and so on.
According to a critique written by a senior software engineer at Google, the company is lagging behind the open source community in the field of AI. The open-source community comprises, of course, many independent researchers who are making significant progress in AI technology at an unexpected pace.
So what the engineer said, while we've been squabbling, a third faction has been quietly eating our lunch.
Plainly put, they, open-source, are lapping us.
Things we consider major open problems are solved and in people's hands today.
Large language models on a phone So Foundation models can now be run on a Pixel 6 device at a rate of 5 tokens per second.
Foundation model is a pre-trained general purpose AI model that serves as the basis for developing more specialized AI models for specific tasks.
Scalable personal AI. So it is possible to train a customized AI model on your laptop within a single evening.
Responsible release.
Not solved, but obviated.
So websites with unrestricted access to art-generating models abound.
The same holds true for text.
It's the ability of a model or system to process and understand multiple types of data or input modalities such as text, images, audio, and video.
So the engineer writes, open source models are faster, more customizable, more private, and pound for pound, more capable.
They are doing things with $100 and $13 billion parameters that we struggle with at $10 million and $540 billion.
So... Yeah, local, decentralized is best.
And not just in terms of avoiding censorship, but in terms of acceleration of progress.
So the engineer says they're doing so in weeks, not months.
This has profound implications for us.
We have no secret sauce.
Our best hope is to learn from and collaborate with what others are doing outside Google.
We should prioritize enabling 3P integrations.
Giant models are slowing us down.
In the long run, the best models are those which can be iterated upon quickly.
Directly competing with open source is a losing proposition.
Who would pay for a Google product with usage restrictions if there is a free, high-quality alternative without them?
And we should not expect to be able to catch up.
The modern internet runs on open source for a reason.
Open source has some significant advantages that we cannot replicate.
So, what happened?
In early March, Meta's Lama was leaked to the open source community, lacking instruction tuning and reinforcement learning from human feedback.
But it was immediately recognized as significant.
Rapid innovation followed, leading to variants with numerous improvements within a month.
This development has lowered the barrier to entry, enabling individuals to experiment with foundation models using just a strong laptop.
I mean, this is massive.
The biggest tech players in the world are calling it.
Open source will eat their lunch.
Or to put it another way, the world will be further divided, almost metaphysically but certainly epistemologically, the world will be further divided.
Into those with unrestricted access to information and those whose information goes through a programming filter or propaganda filter.
All right. So, MetasLama, as I've mentioned, was leaked to the open source community.
The open source community has gone on to use Lama as the foundation for many unfiltered models.
The writer says, individuals are not constrained by licenses to the same degree as corporations.
So what does that mean? Much of this innovation is happening on top of the leaked model weights from Meta.
While this will inevitably change as truly open models get better, the point is that they don't have to wait.
The legal cover afforded by personal use and the impracticality of prosecuting individuals means that individuals are getting access to these technologies while they are hot.
I'm not a huge fan of IP, and this is sort of one example why.
We wouldn't have classical music if IP had been strictly enforced.
So, paradoxically, says the engineer, the one clear winner in all of this is meta.
Why? Because the leaked model was theirs, they have effectively garnered an entire planet's worth of free labor.
Since most open source innovation is happening on top of their architecture, there is nothing stopping them from directly incorporating it into their products, right?
I mean, it's often the way it works.
Whoever wins hearts and minds first has a great advantage.
Look at the crypto ecosystem, often open source, like much of AI will be, at least I suspect.
There are arguably better smart contracting languages and approaches than Ethereum's solidity language and the Ethereum virtual machine.
As the first mover to bring something practical, at least enough, to the market, every other approach is virtually an afterthought.
In short, AI is nothing without humans, without market feedback, and garbage in, garbage out.
If you want the highest quality feedback, you have to get the inputs correct, objective.
There is a market incentive for models to eventually be leaked to the public.
All right. Relative response quality assessed by GPT-4.
So, of course, open source, I'm sure you know, it's a type of software where the source code is freely available for anyone to view, modify, and distribute.
You don't have to reverse engineer it because the source code is right there.
The values of the open source community include collaboration, transparency, accessibility, and continuous improvement.
Promoting shared knowledge and innovation for the greater good.
Software and works considered open source can fall under many different licenses.
MIT, GNU, or GNU, Apache, BSD, Creative Commons.
All have varying degrees of requirements, like sharing your changes to the code, reuse of the same license for a derivative code, and so on.
MIT is probably the most permissive of these.
So, LAMA is leaked at the beginning of March.
Within two weeks, Stanford had used GPT 3.5 to create Alpaca from it.
Using GPT-4...
As the judge, Alpaca 13b is over 75% as capable as GPT-3.5.
Within just one week, large model systems organization created and released Vicuna 13b from Alpaca.
Vicuna reaches a performance level of almost 95% GPT-3.5 as per our GPT-4 judge.
So it's just wild how quickly it moves forward.
So yeah, even if they censor larger public models...
The open source community will use those larger proprietary models to help create free open source models.
All right, so let's look at this sort of training cost, right?
So at current rates, the cost to train Lama using a dollar per hour rule of thumb would be roughly $82,000 for the 7 billion parameter variant and $135,000 for the 13 billion parameter variant.
Variant, right? But if you look at the difference between llama, alpaca, and vicuna, it only took a couple of hundred dollars to train alpaca and vicuna, and this is the kind of improvement.
Now, you can say, ah, yes, but you're going to get better outputs from the larger trained models.
Not necessarily in specific sub-areas, and is there a law of diminishing returns, right?
And so, also with the larger models, especially those centrally controlled, you're going to get a lot of censorship and so on, which you won't get as much of, if any, other than what may exist in the source data from the other ones.
So, it's going to be infinitely better to get the truth than censorship, and if open source means less to no censorship, its value goes up almost infinitely higher, and so on.
As I don't mean to pack myself on the back, but as I predicted in the last presentation, they're all already available to you.
You can run them on your laptop.
You can even run them on your laptop without using a graphics card.
It'll be slower, but it will still work if you use a CPU instead of a GPU. Unrestricted, unfiltered, open-source AI models available today.
You've got GPT-4X Alpaca, that's a LAMA-based model, trained on GPT-4 outputs, heavily improving the output.
It is claimed to be up to 90% of GPT-4 quality.
You've got Pygmalion, that's a dialogue model based on Meta's LAMA 7B. Quote,"...the intended use case for this model is fictional conversation for entertainment purposes.
It was not fine-tuned to be safe and harmless." Like life should be.
Wizard LM, empowering large pre-trained language models to follow complex instructions.
DALI versus stable diffusion.
So DALI is, at least as of this presentation, a closed source system, meaning OpenAI has not been so open with the code.
It's an AI generation tool created and released by OpenAI.
In comparison, stable diffusion is an open source image generation model.
You can even run it locally.
Stable diffusion is optionally free, has unfiltered variants, and in comparison to DALI, competitive quality and performance.
So, here's the thing.
You can run your own, unfiltered, private AI at home.
No deplatforming, no violation of privacy, no creepy corporate or government, corporation or government somewhere looking at what you've been doing.
No woke filter.
It can contact current information.
And for about $200, you can get two NVIDIA P40s, get you 24 gigs of DDR5 memory.
That's more than enough! The gods have come home.
The gods are no longer in the cathedral.
The gods are in your house.
It's just wild.
It's just wild.
And beautiful, really. Alright, so let's get to how you can not just survive, but flourish and gain success and wealth and yachts from this coming...
Alright, so let's get to the meat of the presentation here.
Thank you for your... I won't say patience and indulgence.
Hopefully I kept it interesting as we go through this stuff.
But here's the meat. Here's how you can flourish, survive, do well, gain access to record numbers of entirely gold-constructed yachts.
This is how you gain value.
Start talking about the business applications and how you can survive this incredible change.
So, banks are finding that AI for fraud detection is fast, effective and efficient.
In 2021, FinTech News reported that financial institutions are deploying AI-based systems in record numbers.
With more than $217 billion spent on AI applications to help prevent fraud and assess risk.
Even more promising is that 64% of financial institutions believe AI can get ahead of fraud before it happens.
Fraud is massive.
A massive loss to the financial sector.
If you could prevent people from becoming fraudsters, maybe fraudsters could quickly get a real job.
Who knows? But it's incredible. GPT-4 outperforms elite crowd workers, saving researchers $500,000 in 20,000 hours.
A team of researchers from Carnegie Mellon, Yale, and UC Berkeley investigating Machiavellian tendencies and chatbots made a surprising side discovery.
OpenAI's ChatGPT-4 outperformed the most skilled crowd workers they had hired to label their dataset.
And this is the breakthrough that saved the researchers over half a million dollars in 20,000 hours of human labor.
Artificial intelligence can deliver improvements to large parts of the international supply chain for retail businesses worldwide.
So what have we got? For retail businesses worldwide, Statista reports that 49% of respondents expected supply chain AI to reduce costs.
44% believed AI would drive increased productivity.
43% stated AI will increase revenue.
40% saw more improved retail decision-making as a key benefit.
This will result into a tangible impact on the bottom line.
A report by Capgemini estimates that retailers could save as much as $340 billion by taking AI to scale across the value chain.
AI can optimize inventory control as well.
One AI provider reports that AI inventory management implementation resulted in a 32% reduction in costs across the operation.
IBM is currently freezing hiring for about 7,800 roles, which it plans to replace with artificial intelligence soon.
With regards to healthcare, AI-powered chatbots are expected to derive significant cost savings in healthcare, reaching 3.6 billion globally, up from 2.8 million in 2017.
AI and automation solutions are reducing the cost of healthcare delivery and streamlining simple administrative tasks like patient registration, patient data entry, and claims processing.
However, hospitals need to invest in workflow optimization at scale to reduce operating costs while improving the patient experience.
AI-powered virtual nurses are being used by healthcare organizations to ask patients questions
about their health, assess symptoms and give them care suggestions, giving healthcare professionals
more time to focus on critical care.
AI is also being applied to correct and improve inefficiencies in the back office, such as
improving workflows and eliminating time-consuming tasks, such as writing chart notes or ordering
tests and prescriptions.
It's just wild what this is all going to do.
Alright. Who owns AI content?
Very, very interesting.
From a Harvard Business Review article, Harvard, from 2021, quote, Though most people don't realize it, much of the technology we rely on every day runs on free and open-source software, FOSS. Phones, cars, planes, and even many cutting-edge artificial intelligence programs use open-source software such as the Linux kernel operating system, the Apache and NGINX web servers, which run over 60% of the world's websites, and Kubernetes, which powers cloud computing.
In the last few years, we have observed an increase in the active role of corporations in open-source software, either by assigning employees to contribute to existing open-source projects or open-sourcing their own code both to allow the community to utilize it and to help maintain it.
I mean, I've been open source from the beginning.
I don't have any ads and you can use my material and people upload my stuff elsewhere.
I don't care. My books are all free.
I'm open source from the very beginning because it just works with my philosophy and works, I think, with general economic productivity.
So who owns the AI content?
Of course, none of this is legal advice.
You've got to check with your local jurisdictions and so on, but the way some experts and in practice the community is generally treating it, if you enter the prompt, you own the output.
You own the output.
In fact, if you try to credit AI as an investor, as an inventor, your patent application would be rejected.
So Stephen Thaler created DABUS, DABUS, device for the autonomous bootstrapping of unified sentience.
He claims that it has several inventions under its belt and has sought patents in many jurisdictions.
Reuters reports that, and I quote, the U.S. Supreme Court on Monday declined to hear a challenge by computer scientist Stephen Thaler to the U.S. Patent and Trademark Office's refusal to issue patents for inventions his artificial intelligence system created.
Thaler has been rejected by Australia, the European Patent Office, Germany, Israel, South Africa, and the UK. It's a very interesting thing.
So, the decentralized voluntary market.
If you want to work with others around the globe, the open-source software community has already shown us how.
There's a whole world of decentralized coordination and economic activity online.
People with shared interests and goals coordinate through various platforms and with the aid of repositories, most of which run on Git, the free open source software.
So code tracking and iterations and merges and so on is all handled by free open source software to make sure that you don't overwrite somebody else's changes and you keep track of changes and so on.
The growth and boom in Bitcoin and crypto in general is proof of how effective this is.
The entire industry was built on and with open source software.
It's just wild.
So AI, yeah, it's going to lead a boom.
There are decades upon decades of readily available code that anyone and everyone can use.
I mean, I remember back in the day, starting to learn database programming, you had a book.
And that's it. And you would just try and wrestle your way through the book.
Maybe you'd phone someone. Maybe they'd know, but they wouldn't.
Now you can just go online, get code examples and snippets, and just go from there.
And it's just wild from how it was back in the day.
And the very first thing that I learned how to program...
I would go to my high school's computer lab.
I was 11 years old, and I spent all Saturdays there, and just with a bunch of other dudes, we just figured out how to program.
We shared ideas. We shared arguments.
That was our intranet, I suppose you could say.
Now, there's just so much code that's out there.
Much of software development is finding information.
Most of software development these days is stitching together pre-existing code and not just blank coding it from scratch.
So with better and better AI to help guide that process, there's going to be a big boom in code.
Alright, how about financial analysis?
It's wild. Researchers from the University of Florida have released a study suggesting that the AI model, ChatGPT, can reliably predict stock market trends.
Using public markets data and news from October 2021 to December 2022, their testing found that the trading models powered by ChatGPT could generate returns exceeding 500% in this period.
This performance stands in stark contrast to the minus 12% return from buying and holding an S&P 500 ETF during the same timeframe.
From minus 12% to plus 500% or more.
The study also underscored ChatGPT's superior performance over other language models, including GPT-1, GPT-2, and BERT, as well as traditional sentiment analysis methods.
I think that's supposed to be sentiment.
A direct quote could be sentient, but I think it's sentiment.
Wild. Okay, so, what can you build?
How about, have you heard of...
DAOs, data access objects back in the old 16-bit days of Windows, decentralized autonomous organizations.
So these are, let's look at one, a blockchain-based organization that operates through smart contracts which automate decision-making processes and eliminate the need for intermediaries.
A DAO's rules and governance are encoded in the blockchain and enforced by its members who can participate and vote on proposals using digital tokens.
So this is wild.
So, this allows people anywhere in the world to coordinate in creating value.
In practice, there are already several working examples, both with and without legal structures in a particular jurisdiction.
Most notable is Shapeshift, founded in 2014 as a corporation.
The Decentralized Finance Service has since dissolved that corporate structure and operates entirely as a DAO. In 2019, D.Org launched the first limited liability DAO and is the first legally established decentralized autonomous organization under U.S. law.
City DAO is the first DAO in history that bought land as a registered LLC in Wyoming.
So, yeah.
I'm calling it now. I'm calling it now.
DAOs are the way organizations are heading.
Localized bureaucracies have made operating any kind of organization a nightmare and people are getting more and more used to working and coordinating from home and around the world.
So, I mean, if you're interested, I'd be happy to do an entire...
I'd be thrilled, beyond thrilled to do an entire...
A presentation on DAOs, but they are the wave of the future because it automates decision-making, it automates triggers for payments and contracts and so on, right?
In Backwash, when I did the first Bitcoin presentation I did was in 2014, after talking about it since 2011, and I was talking about how one of the original ideas behind Bitcoin was...
Let you have your bitcoins.
When you show up in an obituaries area of the newspaper, then the bitcoins are automatically transferred to your son.
And so you could have an entire automated process for allocating your resources when you die, your will and so on.
And these are the kinds of things that are all possible.
So between all the middlemen that AI, crypto, and open source stands to eliminate, there's a vast, vast opportunity for you, my beautiful, handsome, virile, and robust listeners.
Now, maybe you're not a tech head, propeller head like me.
If you're not interested in tech, no worries.
AI has even more value to offer you.
You don't have to have an interest in or time for tech, accounting or business planning.
It's just part of it, right?
I'm not saying you entirely rely on chat, GPT or any other AI for your business planning or the like.
As one of my listeners, you're wise enough not to take the output that an AI gives you
as necessarily true or valuable, but think for yourself.
It is great for a general idea and giving you somewhere to start though.
I asked AI, write me the chapter headings for a book on peaceful parenting, and boom, it came out.
Does that mean I wanted to write the book?
No, but it's not a bad place to start playing around with the ideas.
So already existing and ever improving open source models ensure you cannot be denied access.
This cancel culture is really going nuts these days.
So one reason some people hesitate to learn to use AI is the fear of being cancelled and not having access to it after becoming dependent on it as part of the workflow.
This is exactly why in this presentation I went over the open source options where you cannot be cancelled.
So that's really important as well.
Now, there are, of course, already companies hiring people with skill and using AI. This does not require any more technical acumen than just using the internet as a whole.
There is a real art to crafting the right prompt, which is the input you give, to get what you want from the AI. You can also give it sequential tasks, tasks buried within tasks or nested tasks.
There's a lot of Wild ways that you can use AI and if you're really good at it and you practice at it and you learn about it and there's tons of resources out there for you to learn, it's going to be pretty powerful.
Employees, businesses and entrepreneurs who excel at leveraging AI will out-compete others in the market.
So here's the thing. Look, as a listener to this show, as a watcher of this show, you're probably not much in danger of being replaced by AI. But if you're even thinking about, oh my gosh, I could be replaced by AI, you're looking at it the wrong way.
I mean, it is an opportunity.
It is a wake-up call.
We've seen this coming for many years, for decades, not just because of AI or self-driving cars, but like forever.
I mean, war games from the Matthew Broderick movie from the 80s.
This is as old as time. Technological advancements end old opportunities and create new ones, and I want you guys to be ahead of the wave.
AI cannot replace a position where it does not also open up even more opportunity in the marketplace.
I mean, silly example, if you get a combine harvester, you get to replace like 50 people who harvest your crops.
Okay, does that mean those people starve to death?
No. Opportunities based upon the cost savings of not hiring those people will allow capital to go elsewhere and to create new businesses and new opportunities.
You're going to have to be nimble and not wait for it to roll over you.
Be ahead of the curve.
Be riding the wave, not swamped by the wave.
So, remember, wants, desires, opportunities, they're all infinite.
So, if something can be automated, it opens up a whole bunch of new needs that can be satisfied.
And some of them will be satisfied that we know about.
Others will be satisfied because new stuff...
Is invented. Economics, of course, as you know, runs on supply and demand.
Human wants and desires are infinite.
I mean, everybody knows this.
So you can either leverage AI to pursue your own dreams better.
You can become an expert at leveraging AI to provide value to other people.
And you can certainly look at the areas that AI is going to replace.
AI replaces...
This is my rant.
Is it time for the rant yet?
Is it time for the rant?
Let me see here. Is it time for the rant?
Okay, I think it is. Alright, let's go full screen here.
So, AI replaces predictable language.
AI is not generative in that way.
It's not going to write Hamlet out of nothing.
AI replaces predictable language.
So be somebody who does not create and provide predictable language.
I hope in this presentation you haven't guessed every next thing that I'm about to say.
So AI has made...
The value that this show brings to you and to the world more relevant than ever.
What do I teach you? What do I try and get across to the world and have for like decades and decades and decades?
I've been in philosophy now for over 40 years, right?
And I've been doing this full-time for like 15, 16 years.
What am I trying to do? I'm trying to get you to think for yourself, to look at information, to understand the world, to get...
Creativity out of the raw material of senses and evidence and numbers and facts to bring the magical spice of creativity and original thought.
You can't get that out of AI. It is a word guesser foundationally.
So don't be in a field where the language can be easily guessed or relatively easily guessed.
Be in something which has spontaneous...
What have I been trying to do?
I've been trying to get you to live more philosophically, help you improve and reclaim your volition, your ethics, your integrity, your own free will.
The free will is that which can't be immediately predicted or even long-term predicted.
So a rock bouncing down the side of a hill, you may not know exactly where it's going to land, but you know for sure it's not going to grow wings and fly away or decide to change its direction and crawl back up The hill.
So be somebody who thinks for himself, thinks for herself.
With philosophy, with creativity, with self-knowledge, you can overcome and learn from your own personal history.
What is between you and your creativity?
A lot of times it's upset, it's trauma, and not just necessarily childhood trauma, but the trauma of a society that is frightened of and punishes genuine curiosity and creativity.
Genuine curiosity and creativity...
It goes against propaganda.
Propaganda is the methodology of Geppetto control of the population.
thinking for yourself is cutting the wires that control yourself controlling others so
you get a lot of hostility towards creativity but you're going to need to embrace it to
make wondrous things in the realm of AI.
Now you as a listener to this show and if you're starting out here fantastic love it
you can go to freedomain.com and listen to earlier shows and podcasts you can go to FDRpodcasts.com
to do searches for specific interests but in creating things through deep self-knowledge
through philosophical principles through morals through ethics through integrity through curiosity
through reasoning for yourself that AI will never ever ever be able to replace.
So the way that you ride the AI wave is identify areas where predictable language is going to be replaced and Invest in the areas where AI will replace people.
And I'm not giving investment advice.
I'm just saying, you know, in a theoretical world, this is what you could do, right?
Find areas where the most predictable language is the most replicated and the most time-consuming and the most labor-consuming.
There are areas, huge areas of the economy, where predictable language is easily replaced.
Forms, bureaucracies, paper pushers and pencil pushers and checkbox jockeys of every conceivable dimension are going to come through like a forest fire.
So maybe it's worth looking into investment opportunities in areas where there's a big bulk of easily replaceable people.
Those are going to draw a huge amount of profit.
Trying to figure out where people are going to go when they're ditched through AI. Think of these 7800 people that IBM is not going to hire because of AI. Where are they going to go?
Try and figure that kind of stuff out.
I think that's really important as well.
Can you be early on in leveraging AI for your own business aspirations?
AI has made it so much easier for you to become an entrepreneur.
Write a business plan. Write a financial plan.
Make sure that cash flow is king, that you retain positive cash flow.
Figure out how you can appeal to investors.
All of that stuff can be significantly automated because the business plan is your creative originality.
The content of the business plan is somewhat predictable language on cause and effect and investor and returns and profits and losses and so on.
so you can do all of that stuff incredibly well.
And so, investing either in the profits that AI is gonna generate in business as a whole,
in your own entrepreneurial aspirations, and do it now before everybody else figures this stuff out
and uses AI to generate their own business plans, but also learn how to leverage AI to help other people
profit from it and sell your services that way.
The better and more creative you are with AI, the less you can be replaced by AI,
because learning how to leverage and manipulate this amazing technology is something that only you can do
through your own creative reasoning.
So, yeah, be philosophical. Pursue self-knowledge.
Remove psychological barriers to procrastination and dead repetition.
Unleash your creativity, and you will ride that wave, I think, into a great new future.
Delay, unfortunately.
He said putting the scare tactics right at the end.
Delay, and you will drown.
Accelerate and you will surf to great heights.
So here's the most fundamental thing to really bring home about AI. We outsource just about everything.
I'm outsourcing the delivery of this message to a camera, to a microphone, to computers, to the internet, to your screen, to your speakers, your headphones, whatever.
So we do a lot of outsourcing.
I don't write things out by hand.
I type and print. We outsource a lot of things.
And The degree to which we outsource that which is boring and repetitive and unnecessary is the degree to which we expand our creativity and our thoughts and our mind and everything.
Ah, but, but, but, my friends, here's what you cannot and must not do.
You cannot and must not outsource thinking.
That you must not do, that you cannot do, because you're going to be exposed by AI. If you stitch together, I'm not saying you in particular, but if people as a whole, if you're stitching together a series of propagandized, nonsense, empty-headed slogans, then you are...
Not an artist you are jigsaw puzzle, just assembled in a particular way by the powers that be.
To individualize yourself, to individuate yourself, to think for yourself relative to reality, to create your own thoughts and ideas and arguments, that is the essence of what it is to be human.
That cannot be replaced by AI. It is a very sophisticated word guesser, a stitcher together of pre-existing patterns according to digital probabilities.
It is programmed.
And if you want to succeed in the world and life of AI to come, you cannot be, you cannot afford anymore to be programmed by the media, by ideology, by corporations, by preferences, by social approval, by your peers, by what people like, what is punished and rewarded in society.
You must think for yourself.
You cannot outsource anymore thinking.
It's like being somebody who writes things out.
Think of being a medieval monk.
Writes things out by hand.
All the books in the Middle Ages written out by hand with all this calligraphy and so on.
How do those jobs do?
How are they now in the age of laser printers and faxes and emails?
They don't exist. If you don't think for yourself, you are a pretense of thought.
AI does not think for itself.
It is a pretense of thought.
All who are programmed will be replaced by that which is programmable.
The only survival is thought, a refusal to outsource thought to propaganda or computers.
That is the essence of how to survive and flourish and thrive in the age of AI. A sophist is somebody who pretends to have wisdom when they only have slogans.
The sophists of the world are going to be unmasked by AI. If you don't think for yourself, but you pretend.
You know, you have a book review, never quite got around to reading the book, but you watched some of the movie and you're pretending to know the book.
AI pretends to have knowledge but is actually programmed.
Don't be someone who pretends to have knowledge but is actually just programmed by propaganda.
If it's you against an infinite machine, if it's you against hundreds of billions of parameters, if it's you against the collective programmed works of all of humanity across all history, You cannot out-sophist, the ultimate sophist, which is AI. You cannot win against that level of programming.
You will lose. You can win against the ignorant if you pretend to have knowledge.
You cannot prevail against the infinite that also pretends to have knowledge.
It just has way more source material than you or I or anyone will ever be able to absorb and regurgitate.
It is not knowledge to store things for a test, regurgitate them and then forget them.
It is not virtue or wisdom to repeat socially acceptable slogans and call yourself good.
To absorb, reassemble and regurgitate is the essence of AI. Do not try to replicate what AI does infinitely well and think that you can survive economically or in fact morally or spiritually.
Hold on to the most essential aspect of your mind, which is generative, original, creative, thoughtful and actually wise.
Then, it's not even a contest.
You can't even consider yourself as having won because you're playing entirely different sports.
Be an artist, not a photocopier.
Be a thinker, not a sloganeer.
And be wise, not a sophist.
And only then can you triumph.
Well, thank you so much for enjoying this latest Free Domain show on philosophy.
And I'm going to be frank and ask you for your help, your support, your encouragement and your resources.
and your resources.
Please like, subscribe and share and all of that good stuff to get philosophy out into the world and also equally importantly go to freedomain.com forward slash donate To help out the show, to give me the resources that I need to bring more and better philosophy to an increasingly desperate world.
So thank you so much for your support, my friends.
Export Selection