All Episodes
July 21, 2022 - Jordan B. Peterson Podcast
02:17:52
Zeroes and Ones: Into the Depths of Computation | Jim Keller | EP 272
| Copy link to current segment

Time Text
I knew this wasn't going to work.
So you're in this space.
The direction I'm going is not going to work.
So you know how mosquitoes work?
Mosquitoes are fun.
So they detect two things.
They detect water vapor and carbon dioxide.
So mosquitoes will fly along in a direction as long as the water vapor and carbon dioxide are staying the same or going up.
But as soon as it starts to go down, they change direction in a random direction.
And within a couple of turns, they're aiming right for a mammal.
They can bite.
It's kind of colorful.
So if you know you're going in the wrong direction, a change in direction maybe is just as bad, but there's some chance it's good, especially if you're somewhat smart and you have some experience.
So even a random move is better than no move if the outcome is a certain failure.
And so that is some justification for taking a risk.
Hello, everyone.
I'm pleased to welcome Jim Keller to my YouTube channel and podcast today.
Jim is a microprocessor engineer known for his work at Digital Equipment, AMD, Apple, Tesla, and Intel.
He was co-architect for what were among the earliest of 64-bit microprocessors, the EV5 and EV6 digital alpha processors designed in the 90s.
In the later 90s, he served as lead architect for the AMD K8 microarchitecture, including the original Athlon 64, and was involved in designing the Athlon K7 and Apple A4 through 7 processors.
He was also the co-author of the specifications for the x86-64 instruction set and hypertransport interconnect.
From 2012 to 2015, he returned to AMD to work on the AMD K12 and Zen microarchitectures.
At Tesla, he worked on automotive autopilot hardware and software, designing the Hardware 3 Autopilot chip.
He then served as Senior VP of Silicon Engineering, heading a team Of 10,000 people at Intel.
He is presently president and CTO at TENS Torrent, building AI computers.
He's also my brother-in-law.
And we've talked...
A lot over the last 20 years.
He was a friend of mine before he married my sister and we've known each other for a very long time since we both lived in Boston.
So I'm very happy to have you to talk to you today, Jim.
I'm really looking forward to it.
So thanks for agreeing to do this.
Sure thing.
So, let's walk through your career first.
It takes some unpacking.
Your resume takes some unpacking to be comprehensible, I would say.
So, let's start with digital equipment.
You were working on very early stage, sophisticated microprocessors.
So, tell me about that.
Well, the long story is, you know, I graduated college as an electrical engineer.
I was a bachelor's and I took a job in Florida because I wanted to live on the beach.
And it turned out to be a really interesting job was at Harris.
And I spent like two years working in labs, fixing up electrical equipment, and doing some networking stuff and some digital design.
And at some point, a friend told me I should work at digital.
So I read about digital, and I literally read the computer architecture manual for the VAC 780 on the plane on the way to the interview.
And then I interviewed them with a whole bunch of questions because I just read this architecture spec, which I didn't know that much about, to be honest, but I was kind of a wise-ass as a kid.
And they hired me because they thought I was funny.
And so I got my, you know, architecture education working on the VAX 8800, working for a guy named Bob Stewart, along with some other really great architects.
I spent about seven years in that group, you know, learning to be a processor architect.
And then I spent a little time at Digital Research in California for six months and then went back and joined the Hudson team where they were building alpha processors and became co-architect with Pete Bannon on EV5 and then with Dirk Meyer on EV6. 15 years and we worked on, I would say, three very successful shipping products and then a couple other products that didn't get to market.
So those chips, from what I remember, were remarkably ahead of their time, but that didn't seem to save Digital Equipment Corporation.
Is that fair to say?
That's definitely fair to say.
The digital literally made the world's fastest computer in the years they were going out of business.
And there was a complicated market dynamic.
Digital was very successful building mini computers in the mini computer age, which replaced the mainframes to a large extent.
But they missed the boat on the PC revolution and, let's say, workstations.
And they had a big, expensive computer mindset right when computer prices were falling a lot.
So we were building really fast processors for server and high-end workstations.
And the market had kind of moved on.
And let's say a lot of crazy things were going on inside the company at the time.
Yeah, well, we're going to return to the topic of crazy things going on inside computers.
But it's interesting to note right there so that that's a situation where a company has a great product but doesn't know how to launch it into the marketplace or is blinded by its own preconceptions.
It can't even see necessarily what it has.
I think it's tougher than that.
Gordon Bell was CTO, and he was a really brilliant computer architect, but he also had really good, let's say, observational skills.
And midway through the Vax8800, he decided the technology we were using was too little too late and redirected the program and it became a very successful product because he knew what was going on and he made decisions like that.
When he left, I think digital became an argument between business unit managers, not about technology.
And Alpha was a great technology, but it went into business units.
That we're aiming at high prices and high margins, not market penetration, and not basically keeping up with the software revolution that was happening at the time.
Ken Olson was a great manager, but he wasn't a technical leader.
And without Gordon Bell, the company kind of lost its way.
When companies lose their way, they fail.
And they failed fast too.
They went from record profits to losing a billion a year, and they kind of Rectified that for a little while and then the shadows dropped off again and over.
Yeah, well, one of the things I've been struck by watching your career and talking to you over the years is exactly that.
The rate at which a company that appears dominant can disintegrate and disappear is really something quite stunning.
I think Fortune 500 companies tend to last no more than 30 years.
That's approximately the span.
And that's not a tremendously long period of time.
So there's always dominant companies and always a handful of dominant companies, Well, we should return to this a couple times as the classic S curve in economics.
You start out low, you solve a problem, you ramp up, you plateau, and then you fail.
And this dominates business and dominates business.
Humanity at some level.
It plays out over and over.
It sounds like you didn't have...
How is it that you managed to do this job?
You intimated when we were talking that you weren't really trained for it.
You were trained as an engineer.
Is it a bachelor's degree in engineering?
How prepared were you as a consequence of your degree for any of the jobs that you undertook?
Training is highly overrated.
So, a good engineering degree is math, physics, basic understanding of science, and some smattering of communication skills.
You can probably do a great engineering degree in two or three years if you're dedicated to it.
You know, the things that stretch my brains the most when I went to college is with math and mechanical engineering.
And Penn State, I went to Penn State, and they used mechanical engineering and math as two of the weed-out courses to find out if you had You know, the chops or the gumption to get through engineering.
So there was a fairly high failure rate there.
But mechanical engineering is a really interesting discipline because you have to think about solving mass problems spatially.
Like, you know, do things like how do you calculate the force on a rotating, accelerating object?
Like, it's somewhat complicated.
And it makes you really think.
So you have to learn to think.
And in engineering school, you never answer a multiple choice question.
You learn stuff, but then you calculate to understand what the result is.
The problem sets are like little design exercises.
How much of it do you think is pure screening, let's say, well, it wouldn't be pure screening, but fundamental screening for conscientiousness and IQ, and how much of it is learning to think How much of the education process is that?
Like if you're going to hire an engineer, are you hiring fundamentally on the basis of IQ and you get smarter people from the top schools?
Or do you think that the engineering training actually does prepare people for a technical career?
Well, it depends on the engineer and depends on the school and depends on their approach.
So my IQ isn't super high compared to really smart people.
I mean, it's high enough.
When I went to college, it took me about a year and a half To learn how to think properly, and I found for me personally, I had to do the work on a regular basis early.
I didn't study for finals.
I wasn't the kind of person that could pick up a book, understand it, get an aid the next day, and then forget about it.
I can't do that.
So I had to learn how to do the work, go through the mechanisms, automatize some of the basics so I didn't have to think about them so hard, but literally let my brain work on this stuff.
So then I could use them to go problem solve.
And especially in engineering, there's lots of different kinds of engineering.
There's like highly technical stuff where you turn the crank, like a skilled lawyer might.
But there's other stuff where you have to be really creative.
You have an unsolved problem that nobody's solved before.
And as an engineer, you have a skill set, right?
But you have to apply it creatively.
And there's lots of high IQ people who aren't creative.
And there's low IQ people that are creative.
And you find in a In a big engineering team, there's a real diversity of personality types.
There's open-minded people, conscientious people, gregarious people, and it takes many different kinds of people working together to do something sophisticated.
I'd say some of my senior classes in engineering was just going a little deeper on stuff I already knew.
I could have left after three years and been just fine.
But I do think the work I did Did help me, you know, be an engineer.
But then the problems I saw, you know, I worked on after I graduated college, like in school, most of the problems you're given, there's a known answer because they're in a book, you know, and you're in a room with 20 people and they're doing the same stuff.
When you're an engineer working in the company, they never have two people the same thing to do because that's a waste of money.
Right.
And when you start engineering, you're given relatively small tasks by your manager or supervisor.
But as you go along at some point, it depends again on who you are, you're working on stuff that you don't even know how to deal with.
There's no answer in the book.
But it's not like physics, right?
Physicists are a funny bunch.
I realized this the other day, that physicists, they're supposed to work on stuff that's unsolved.
Whereas engineers, you know, There's a big repertoire of engineering and then it's reduction of practice.
Then the world is complicated.
So you go build a new bridge that's never been built before.
It's not like bridges are an unsolved problem.
This particular bridge hasn't been solved before.
There may be unique challenges to it, but it's not like physics where you're looking for an unknown particle.
There's a pretty big dividing line between engineering and pure science.
Engineers typically work in domains where There's many, many knowns, and the unknowns are problems of the combination of reality complexity, whereas physics, in principle, they're working on stuff that's fundamentally unknown.
As soon as it's known, they have to move on, because then it's engineering.
Like physicists translate the unknown into engineering, and engineering applies known concepts to unknown problems.
Okay, so now you went from digital to AMD, and you learned how to design microprocessors.
So at AMD, you worked on the K8? Yes.
And at that point, AMD was losing ground to Intel.
Yes.
And so how did you fix that?
So basically, Dirk Meyer and I were co-architects of EV6, the third alpha chip.
He left a digital KMD about a year before I did.
He started the K7 project.
When I joined, I started the K8 project and then worked with him significantly on K7 as well.
And how did we do it?
Well...
Yeah, so those were 64-bit chips that you guys designed to compete with the Intel chips that had dominated the home computer market at that point.
Well, so there's a funny thing, which is...
At some level, building fast computers isn't that hard.
You have to have a goal.
A lot of designers, they have a design, and then the easiest thing for the next one is to go look at that design and make it 10% better or 20% better.
But every one of those designs has limitations built into it.
It's sort of like if you buy a two-bedroom house, you can add one bedroom.
You can't add eight bedrooms.
If you want an eight bedroom house, you have to build a different kind of house.
Every design you build has a range that it can play.
You build the first one and you know you can make some improvements, but at some point the improvements don't really help that much.
Right.
So AMD, they had a design called K5, which, for complicated reasons, didn't work out that well, and they lost ground to Intel.
Before that, they had literally the 386 and 46 AMD copied Intel's designs.
They were a clone manufacturer.
The K5 was their first design and didn't work out that good.
And then they bought a company called NextGen, which had K6, which is an okay design, but it wasn't competitive against Intel.
And then K7, Dirk was the chief architect of And he designed a computer that was competitive and ahead of Intel.
And some of that came from our work at Digital on UV5 and UV6. Turk worked on UV4 as well.
And some of it was just saying, in this day, we have this many transistors because you get more transistors every generation.
So you can basically imagine you're building a house.
Suddenly you have way more bricks and way bigger steel beams.
So your idea about what to build has to scale with that.
And then K7 was a 32-bit chip, and then K8 was a 64-bit chip.
You know, somewhat related to that, as it turned out, but also was built to be bigger.
And what I did is I wrote the performance model, I came up with the basic architecture, and I started to organize a team around building it.
And while we were doing that, we also wrote the thing called hypertransport spec.
Which became the basis of essentially all modern server computers or what's called two socket servers.
We wrote that in 98 and 2002 or 2022.
They're still building them that way.
And when you say you wrote it, what does that mean?
What does the process of writing that entail?
What is it that you're writing and how do you do that?
I'm dyslexic.
So I wrote a complete, you know, protocol spec about how two computer chips talk to each other in 18 pages.
Which is relatively terse.
And there's a couple of pictures and, you know, computer protocols are pretty straightforward.
There's a command, there's the address you're talking to, there's the data you're moving, there's some protocol bits that tell you how to exchange commands, right?
And then Dirk took the spec and said, you mind if I flesh this out a little bit?
And three days later, he sent me a 50-page version of it, which clarified all the little bullets.
And then that specification we literally used to build the interface between K8 chips.
Right.
So there's a couple levels of design.
What sort of impact did that have on the broader world?
What's the significance?
It's very difficult for non-engineers to understand any of this.
It's so underground.
AMD market share and server went from 0% to 35%, which was a huge impact to the business.
And it became essentially the standard because Apparently, Intel had a version of that, but it didn't go to market.
But after Opteron came out to market, Intel built a similar version, similar protocol about how to connect a small number of processors together with that kind of interconnect.
And then that, let's say, design framework became standard in the industry.
So if you go into a Google data center and you pull it out, there'll be two sockets with an interconnect between the two of them.
And each socket will have memory attached to it.
And they call it the 2P server or two processor server.
And it had a really big impact.
We didn't do it because we thought it was going to have a big impact.
We did it because we thought it was a better way to build computers.
And at AMD, we were somewhat resource-constrained.
So we couldn't build a thing that looked like a big IBM server.
So what we built was basically a small server.
With the minimal amount of interconnect between it.
So it was a little bit of creativity by constraint.
Steve Jobs line.
And what function do those servers have, again, in the broader world?
What are they doing now for people?
Well, it's basically the entire cloud.
It's all Google, all Amazon, all Facebook, all Microsoft Azure.
But here's the interesting thing.
When we built them, the big server guys, servers used to be backplanes, like this big, with multiple CPU slots, multiple memory cards, multiple IO slots, and the server manufacturer thought the server was oriented around the back.
So IBM, HP, Dell, they all turned this down.
But all the little starters at the time, like Google, were using PCs as low cost servers.
And we made this.
Basically, you could take a PC board, instead of putting one computer on it, you could put two, which radically saved the money.
So when AMD made those kinds of servers, it was a way lower entry point for server-class technology.
And the little startups at the time used it, and then over 15 years, disrupted all the big server manufacturers.
So it's one of those...
I couldn't say we planned it.
The constraints that we had a target market, we didn't know that it was going to become essentially how servers were built for 20 odd years.
But it happens.
After AMD, you went to Apple.
You worked on the A4 through 7 processors.
Well, I worked at two startups that did processors for networking, Sidebyte and TASemi.
And that was probably about five or six years.
And then I joined Apple in 2008.
So I guess I was...
No, it must have been eight years.
I was AMD in 98, 99, 2000.
And then I worked at startups for about eight years.
And then I went to Apple.
Yeah, and I worked on mobile processors.
And so tell me about those chips and what you did at Apple and what that did.
I had some friends that were working at Apple and they wouldn't tell me what I was going to work on.
So when I interviewed there, they said, oh, you should just come here.
It'll be fun.
And I didn't actually know what I was going to work on.
They had a group called Platform Architecture run by a guy named Mike Colbert, who was like the unofficial CTO of Apple.
He worked for Steve Jobs.
And he had a group of architects that looked at what Apple was doing and figured out what they should do next.
And I worked on a MacBook Air definition, like I wrote the power management spec and did some other architectural work, which ultimately was an entity called MCP89. And then I was one of the chief architects of, you know, four generations of SOCs, what's called A4, A5, A6, A7. And we did a lot of stuff there because, but the division was, you know, And SOCs are what?
Yeah, a system on a chip.
Oh, yes.
To pack a computer into a phone, you have a piece of silicon about that big.
And all the components, the CPU, the GPU, the I.O. are all on the same chip.
And when they first started building phone chips, they were considered to be very slow, low-cost, very integrated chips.
And we thought, if you looked ahead...
Because technology shrinks about every two years.
In about six or eight years, we'd have enough transistors on a phone chip that would be more powerful than a PC at the time.
So we started architecting computers, interconnects, and other functions so that when we had enough transistors, we could literally have a high-end desktop in a phone.
And Apple's DNA Is you create the product that kills your current product.
You create the product that?
So every company has a great product and they worry about competitors coming in and kill it.
And Apple wanted to be the first to kill their own products.
So Steve Jobs thought phones and tablets would replace PCs.
And he wanted to be the first to do it.
He didn't want somebody else to do it to him.
Did you know Jobs?
No.
I've seen him a couple times.
I said hi to him twice.
I felt like I knew him pretty well.
Everybody at Apple did.
Like when Steve wanted something done, everybody knew the next day.
My boss, Mike, talked to him every single day, multiple times sometimes.
He used to joke like we'd walk in Mike's office and he'd be holding the phone out like this.
He goes, Steve, he's a pest.
We're like, yeah.
But Mike would translate what Steve wanted into engineering stuff and he didn't.
Steve trusted Mike a lot, and he could translate the vision into engineering.
And Steve's judgment on stuff like this is spectacular.
So, and did you have any sense, do you have any sense of why that is?
I mean, Jobs was famously, obviously originated Apple and then was famously brought back in to save them when they were in danger of extinction and then, in fact, did seem to save them.
And you never know when you hear about these things from the outside, how much of that is sort of a mythologization of a person and how much of it is, you know, this person was really singular and unique.
Yeah, definitely singular and unique.
In your psychological parlance, as we've talked about, he would be considered high in openness and disagreeableness.
Right?
And I think negative emotionality.
Like, he was a very difficult person.
But his solution to, you know, things could go really bad and being disagreeable was, I'm going to make it as great as possible.
And he was willing to take the risk for that.
His public persona was very well practiced.
Mike used to say, the worse the practice for the Apple Keynotes, the better they would go off.
Like he was throwing iPhones at one of the iPhone pre-launch practices because nothing was right.
But then when he showed up and his persona of technical explainer, let's say, That was very real.
That's what he wanted to portray.
He believed every single bit of it.
You could tell.
When I joined Apple, I watched some of his early keynotes when he came back to Apple and changed the Macs.
It's inspiring.
But it's also super tough because he went into a company that was very dysfunctional, had a whole bunch of engineering groups doing basically random stuff.
Let's say, senior managers who felt like they owned their product lines and knew what they were doing.
And Steve wanted them to do what he wanted them to do, and they didn't want to do it.
And I'm pretty sure he cleaned house pretty thoroughly.
And he famously reduced the product lines and who knows how many products to like four.
There was consumer and professional.
So do you think it was that disagreeableness?
I mean, we hear all the time now in the modern world about the necessity for empathy and so forth.
And that's the agreeableness dimension.
And you're making the claim that Jobs was low in agreeableness and that he was able to kill off malfunctioning projects.
And that's not exactly a nice thing to do.
So imagine you go to a room full of people who have dedicated their life in the last five years of their work, five years of their careers.
Building products that you can't sell.
And you say, we have to do something completely different.
Every day as an engineer you work on something, you embrace it, you love it, you care about it.
Engineers are very emotional people somewhere in their pointy little souls, right?
But if it's not working out for whatever reason, you have to do something different.
And if you listen to everybody, you'll never change anything.
Right?
It's difficult to get people reoriented.
Now, another line you gave me, or where it came from, is you run fastest when you're running towards something and away from something.
Yeah, that was from animal experimental literature.
If you threaten a rat and offer a reward simultaneously, it will run faster towards the reward than if you just reward it.
Because you get all your motivational systems on board that way.
Yeah.
So, Steve was very good at the vision.
We are going to build this beautiful computer.
And you better goddamn build it now or you're going to die.
So that was his...
Okay, so the openness...
Super highly motivated.
That's the creativity dimension.
That gives him the vision.
He's extroverted.
Can he communicate enthusiastically?
He can certainly put on the act.
I have no idea if he's extroverted or not.
I never saw him be extroverted in any natural setting.
I've seen him walk around.
Like I said, we used to see him in the cafeteria.
He was visible on the Apple campus even until his last days.
He didn't like to be bothered.
He didn't go up to Steve and say, Hey Steve, how's it going?
Right.
Well, that would be reflective of basic disagreeableness, too, right?
You know, it's hard.
Sometimes people can communicate very effectively, communicate a vision because they are high in openness.
Extroverted people are enthusiastic and assertive, so they tend to be verbally dominant and can inspire people because they generate a lot of positive emotion, but that can be mimicked by openness.
So you'll have to remember, like Steve was part of Pixar and very much part of Hollywood and, you know, creating movies and creating...
Personas and characters and archetypes.
So he was super well grounded in how that stuff works and what works and doesn't work about it.
Right.
And he had an unerring eye for beauty and elegance.
Remarkable.
And he cared about it.
And he would fight for that.
And that's hard.
It's very hard to fight for beauty and elegance.
And I suspect it's particularly hard.
Perhaps, maybe I'm wrong about this, but I would think that would be a hard sell to at least a subset of engineers.
Yeah.
So another, this is explained to my boss at Tesla was, I worked both for Elon and for a guy named Doug Field.
And Doug said, there's this, you know, there's this productivity graph versus order.
So the origin is zero productivity and chaos, right?
And then as you add order to your design methods, your productivity will go up.
And what happens with engineers is they understand as they get better processes, they get better training, they get better working together.
Every single thing that makes the whole organization more orderly improves productivity.
Unfortunately, that peaks at some point, and then too much order, productivity goes down.
And so then, as I say, any idiot can see you should be at the peak.
Enough order to really be effective, but not so much order you grind to a halt.
But why can't you stay there?
And the reason is, is once order takes over the organization, it's unstoppable.
Right?
It feels good.
You get even better at doing what you're doing.
You get even more organized.
You micromanage your time even better.
You close out all the creativity.
You're not open to change.
A whole bunch of bad things happen.
You shut out the disorderly people who actually know how to make a change and do something creative.
And the organization dies.
You think that Jobs was conscientious as well?
You know, like, was he in early?
Was he working 18-hour days?
I know he was up in the middle of the night because he called Michael and that stuff.
Well, the point is, both Steve and Elon were counterforces to order.
Right?
You have to be really strong to avoid your organization getting captured by order.
Well, order also has its remarkable air of moral virtue, right?
Because it's pure and it's efficient.
Everything about it feels good.
But it's like alcohol.
The first drink feels great.
The second feels okay.
The third one, not that good.
But you keep remembering what the first one did, so you drink it.
There's lots and lots of processes where some is good, too much is bad, but the counterforce, the more, is weak.
And that's the thing that...
So Steve was interesting because he was simultaneously super creative.
And had visions which could inspire people, but he also prevented the company from being over organized and preventing them from doing what he was doing.
And that's hard because people, like I said, they get committed to what they're doing.
It's an open question.
Like, imagine that the creative process has a productive component and then a culling component.
And the productive component looks like it's associated with openness, but What the culling component is open to question, and it does seem to me that, at least upon occasion, it's low agreeableness.
It's the ability to say, no, we're going to dispense with that and to not let anything stand in the face of that decision, which would also include often human compassion.
Yeah, and people have different approaches to it.
Jobs would call things so they weren't beautiful or they weren't great.
You know, Elon Musk is famous for getting the first principles and really understanding that fundamentally and culling from a standpoint of knowledge.
Yeah, and you've asked me, like, what makes an engineer great?
Like, so you have to have the will to creativity.
Like, now there's lots of engineering jobs that aren't creative.
Like, you need a skill set, you can exercise the skill set.
But if you're going to build new things, you need to be creative.
But you also have to have a filter good enough To figure out what's actually good and bad.
I know a lot of really creative engineers, and they find a new thing, they're excited, and they go down the rabbit hole on it.
They can work on it for six months and nothing to show for it.
So you have to have that conscientiousness.
I don't know if it's conscientiousness or disagreeableness, you know, that taste on how...
Well, the conscientiousness would keep you working in the direction that you've chosen and doing that diligently and orderly.
The low agreeableness, well, that's the open question because agreeableness is such a...
In a complicated dimension, there's obvious disadvantages and advantages at every point on the distribution.
I mean, disagreeable people are often harder to work with because they don't care much about your feelings.
But one thing I've noted about working with disagreeable people is you always know what they're thinking.
And if you want someone to tell you what's stupid and wrong, they're perfectly willing to do that.
Yeah, I used to wonder.
So Dirk Meyer was a disagreeable manager.
But he could tell you what was wrong with what you were doing in a way.
He would go, okay.
He was very unemotional about it.
He'd go, Jim, I really like this and this, but this isn't working for shit.
What are we going to do about it?
And it would just be all shucks, matter of fact.
I would say when I was younger, I was a lot less disagreeable.
I'm fairly open.
And I like to create new stuff, kind of stuff, things.
But then I saw enough things fail over the years because we didn't make the, let's say, the hard choices about something.
And then you hate to work on something for two years and have it go away because at some point you realize you're doing a couple of wrong things and you didn't do something about it when you could.
And so as a manager and a senior leader, I'm somewhat famously disagreeable.
Part of it's an act to get people to move and part of it's, you know, my beliefs that I can't have people dedicate themselves to doing bad things for very long because it'll bite us.
Yeah, well, we've talked a little bit about this too, about the moral dilemma between agreeableness and conscientiousness.
They're both Virtues, agreeableness seems to me to govern short-term intimate relationships like that between a mother and a child.
And it involves very careful attention to the emotional reactions of another person and the optimization of those in the short term.
But conscientiousness looks like a longer-term virtue, and they come into conflict at some point.
Yeah, they come into conflict in the midterm.
Yes.
Yeah.
It could just be how our brains see the future, but it's like if you're managing a group and you have to fire somebody, it's hard, right?
But do you want to fire five people now or everybody later?
Once you've internalized that and taken responsibility for that decision, then making management, leadership, position choices is always hard, but it's so much better to make them And then succeed than it is to fail because you couldn't make the hard calls.
Yeah, well, it isn't obvious at all who's got the upper hand, you know, someone who fires early out of necessity but is accurate and looking carefully or someone who, you know, is willing to let people drag on.
I'll give you two counterexamples of that.
So Jack Walsh in his book, Straight from the Gut, a weird thing.
He said, you know, once you have a doubt on somebody, you never act fast enough.
Which, you know, it took me years to really believe that.
And then the other weird one is, people say, hey, I have this organization of 100 people, and there's five people that aren't working out, but I'm not sure who they are.
So I'm going to be really careful because I don't want to accidentally fire a good person.
That makes sense, right?
You got five bad people, you know, maybe you figure out who two or three of them are.
But there's this other group of five or 10, you're not sure which ones are the wrong ones.
Here's the sad truth.
There's a lot of people in the world.
You're better firing too many than too few.
And how did you come to terms with that emotionally?
I mean, look, we have a mutual friend who fires people with quite great regularity, and I've talked to him, and he scores very high in disagreeableness.
And I talked to him about firing, which he's done a lot of, and he was actually quite positive about it.
He said, I don't fire anyone who I don't think is causing more trouble than preventing.
And so by firing people, The person that I'm firing, I'm actually doing a very large number of people, including potentially that person a favor.
It didn't bother him, but he was temperamentally wired that way, I would say.
I would say, you know, digital equipment went bankrupt because they had bad people who didn't fire.
I've seen many groups fail because they couldn't clean house, right?
And the impact on, you know, the greater good equation is super easy.
You want to say 90 people or, you know, lose 100.
So that's true.
The thing that took me a while to realize that the world needs shaking up all over the place, and individuals do, right?
A lot of people who are not doing too good, they need a wake-up call.
You give them a bad review and they kind of shrug.
They're like, what are you going to do about it?
You know, it's like a spoiled kid.
Nothing, right?
But when they actually get fired, they really have to do some soul-searching.
And then the fact that if you're doing something good, there's always a queue outside the door of more people.
Now, here's another way to think about it.
Take a group of 100 people and rank them from top to bottom.
Human beings, by the way, are really good at this.
You have four managers in the group, except for the manager's individual friends.
They'll tend to rank 100 people the same way.
I've done this experiment many times.
So we're really good at ranking.
And there's a little bit of what are you ranking for?
You're ranking for creativity, productivity, conscientiousness.
But if you set the criteria right, people rank pretty well.
If you have 100 people in your group and there's 50 people outside, the distribution of those 50 people is around the average of the team, right?
So there's this idea that you fire the bottom 10% of a team because the random people you hired will be better on average than the bottom 10% of your team.
Right?
It's just math.
Unfriendly statistics.
Right.
But yes, I get the argument.
Right.
The problem is that every company that does that, first it gets gamed because managers hire bottom 10 percenters.
So when they get to fire the 10 percent, they don't have to fire their friends.
Right?
And it also really is hard on morale.
Like, people bond, and there may be people in the bottom 10% of your group that are the social glue of the organization.
They may be inadvertently taking out the stuff that makes the team work.
Right.
Well, that's a measurement error, too, right?
It means that your criteria for competence aren't broad enough.
Yeah, that's a good point.
Maybe you're ranking a little wrong, but impact on morale is high.
Teams, generally speaking, Rory Rigg was CEO of AMD when I joined.
And we had a big layoff, which we had to do because we were running out of money.
We were broke.
And when we all settled, we landed on just the right amount of people for the money we had.
And he basically read us to Riot Act.
He said...
He said, guys, teams have to grow.
When you cut, you always cut further and then you grow.
People aren't happy unless they're growing.
It's like when you prune bushes and stuff.
You don't prune the bush to where you want the bush.
You prune the bush past that point so it grows out and looks nice.
Things have to grow.
It's really an amazing dynamic.
Yeah, well, it's never clear how much death there is involved in growth, and the pruning analogy is exactly that.
This is harsh stuff, obviously, but you're looking at one collapse or another.
That's the thing.
It's not harsh.
Because it's beautiful when you prune your bush and it grows back beautifully.
It's great when you rebuild an organization that's really strong and powerful because you made the right calls, right?
Like, this isn't just negative stuff.
It's hard stuff to do that creates something really great.
Like, when I joined AMD in, what was it, 2013 or something, like, they had two product lines, you know, Bulldozer and Jaguar, and they're both failing.
And I canceled both products.
Okay, and so what was the human cost of that, I would say, both to you and also to the people that were involved?
Well, I did the math on it.
It's like, you know, we needed to be building eight-bedroom houses and we're trying to add six bedrooms to a two-bedroom house in one case.
It was never going to work.
Yeah, so you saw that as doomed to failure.
And the other one was structurally screwed up.
It seemed to be the right ballpark for the performance we should get, but the way it was engineered and built was sort of like, you know, you let the plumber do the architecting and the house looks like shit.
And it was difficult.
For complicated reasons, technical reasons, there was no path out of where they were.
And when I realized I had to cancel them, yeah, it was sleepless nights.
Here we are.
We had revenue on that.
We had people committed to it.
People really liked it.
When I canceled it, especially on the Jaguar team, a significant number of people quit because they were angry about it.
There were some pretty big organizations.
There was some management we had to let go.
The best architect at the time, well, one of the best architects at AMD was, he really was my way or the highway and he was, you know, he could not communicate what he was doing.
So I let him go, which is a strange thing to do.
So if he'd rank the organization, you're right near the top and I let one of those guys go because he wasn't ineffective working with the team.
And how did you justify that to yourself?
And how did you check yourself against stupidity and ignorance and, you know, self-interest?
And how did you know that what you were doing was right?
Well, I'm a little lower in conscientious than I should be for a senior leader, first of all.
So I knew this wasn't going to work.
So you're in a space.
The direction I'm going is not going to work.
So you know how mosquitoes work?
Mosquitoes are fun.
So they detect two things.
They detect water vapor and carbon dioxide.
So mosquitoes will fly along in a direction as long as the water vapor and carbon dioxide are staying the same or going up.
But as soon as it starts to go down, they change direction in a random direction.
Right?
And within a couple of turns, they're aiming right for a mammal.
They can bite.
It's gotten colorful.
So if you know you're going in the wrong direction, a change in direction Maybe it's just as bad, but there's some chance it's good, especially if you're somewhat smart and you have some experience.
So even a random move is better than no move if the outcome is certain failure.
And so that is some justification for taking a risk.
There's an infinite number of fail directions, but you're somewhat informed.
And then the other problem is when you build a house, it has a foundation.
Once the foundation is built, it's very difficult to change the top of the house a lot.
If you have a foundation for a two-story house, it's hard to make it into an eight-story house.
When we cancel those projects, we consciously reset some design methodologies, some team organizations, some leadership.
We said we're going to have the best-in-class leadership, design methodology, and some of the architectural tools.
We're just going to take those as givens.
Now that the land has been cleared, we had the opportunity to go back and do that.
And it was interesting.
And in the design teams, it turns out there was some very good pieces, you know, in the two processors they had, but they weren't working together organically like they should.
And say the framework of the design wasn't big enough.
And then the tools over the years have evolved into lots of little local improvements, but it wasn't really the right tool set.
Now, AMD leapt forward when you did this, and they were the only competitor to Intel in a realistic sense.
And so these actions on your part were part of what made that company thrive and kept competition within the microprocessor world.
So these decisions didn't have trivial outcomes.
Oh no, it had really great outcomes.
The really cool thing was when we did that, we didn't really bring in outsiders.
Zen design was entirely based on people who worked at AMD at the time.
What we needed was to clear the plate a little bit, to re-establish some first principles about how we were doing things to have a better goal.
There was a little bit of head-knocking on getting the methodologies straightened out.
You know, I was, let's say, fairly disagreeable about how we were going to get through that because people kept saying, oh, it's too hard to do this.
Well, is it any good?
No.
Well, if it's no good, it doesn't matter how hard it is.
You have to do it, right?
If you're going to drown, you don't go, well, a mile is too far to swim, so I'm just not going to swim.
You're going to drown, right?
If you're a mile offshore, you're drowning.
Swimming a mile is the requirement, right?
And I've explained that like a million different ways.
When something is pretty good, you know, the world's, you know, divided into three things.
Things are good, so you're happy.
When things are bad, you fix it.
And then there's the middle ground, where it's not great, but it's not killing.
Those are the ones where human beings have a really hard time improving.
Right?
So by canceling the projects and declaring everything bad, everything could be improved.
People would bug me about it.
It was like, Jim, this wasn't that bad.
Well, was it great?
Was it going to win?
No.
All right, it's bad.
I defined everything that's not great, bad.
If I moved on the continuum, everything 5% or more away from great was bad, period.
And how did you come to decide that was a good criteria?
What's that?
Why did you decide that was a good criteria?
Well, at the time we were competing with Intel on a whole bunch of metrics.
Like, literally their CPU had twice the frequency, twice the performance per clock.
A whole bunch of metrics were so good.
So we plotted them all.
Like, you know, I had a whole bunch of data on this stuff.
But also as a mindset, like if it wasn't best in class, like a computer has a whole bunch of things in it.
There's something called a branch predictor, which predicts which way branches are gone.
Theirs was way better than ours.
We can measure it.
So we did.
There's a thing called a memory system.
Their memory system was way better.
It was twice as fast.
So it was like, well, we need to be within 10% of them on everything or we're going to get our asses kicked.
And it would be really nice if we were better on some things.
So we measured all their stuff.
If you're going to compete, like in basketball, you don't just play yourself.
You try to see what the other teams are doing, whether they're good at, whether they're bad at.
Every coach is great at doing analysis of all the competition.
And then you win two A's, you meet the competition with what they're good at, and then you have some secret plays that you're better at than surprising them.
Given recent SCOTUS wins, it feels like the pendulum may be swinging back to a time when the nuclear family was situated at the center of American life, where real conversation, learning, and growth began at home.
President Ronald Reagan said in his farewell address that all great change in America begins around the dinner table.
Well, all great meals in America begin with Good Ranchers.
Good Ranchers cares deeply about providing families with steakhouse-quality beef, chicken, and seafood meat at a reasonable price.
Their mission is to bring people to the table, making those shared moments with your loved ones easy, accessible, and delicious.
Good Ranchers ships 100% American meat, born, raised, and harvested in the U.S. right to your door.
Plus, when you subscribe, your price is locked in for the life of your subscription.
Great food creates great conversation, and great conversation makes great change.
So start bringing people back to the table with high quality American meat.
Go to GoodRanchers.com slash Peterson to get $30 off your first order plus free shipping.
That's GoodRanchers.com slash Peterson.
So you went from AMD to Tesla.
So, and we talked a little bit about Elon Musk.
I want to ask you about him and your experiences at Tesla, but also, and what you did there.
Sure.
So let's talk about Elon Musk to begin with.
Sure.
Well, he's a pretty public guy, so I don't know that I can add that much.
Well, kind of what you said about Steve Jobs, was it true?
Yeah, it's mostly true.
Like, Elon's the real deal.
He's a really good engineer.
He has this belief that he can learn deeply about anything found.
And he's practiced it and does it.
I've seen him do it.
Like he gets into lots of details.
He has done five impossible things.
Yeah, yeah.
It's pretty respectable.
And he likes the details and he has a good eye for it.
Usually, before Tesla, when you do a technical presentation, you usually have some kind of methodology for presenting the problem to an executive.
What's the problem?
What's the background of that?
Elon is a solution person.
If I don't like your solution, I don't care about the stupid problem and your data.
Just forget it.
No.
What's the solution?
Is the solution great?
Great.
Then everything else is backup.
What's the data behind that thing?
Oh, we've got the data.
What was the problem?
How did you figure this out?
Oh, here's the original problem we found.
He has a reverse order for most people.
Most people tell you a story.
It's a problem.
Here's how we figured it out.
Here's the background data.
Here's how we built a solution.
Here's the solution.
Here's the next step.
That's a typical technical presentation.
It's not a bad idea.
Elon hated this.
Stop with the bullshit.
What's the goddamn solution?
Can you give me an example of that?
We had a problem with low-resolution camera images, right?
And we were trying to improve how the computer perceived roads in low-light conditions.
So we started with, well, here's the resolution of the camera.
Here's the light sensitivity.
Here's the images we're getting.
And he was like, what the?
Do you have a solution or not?
Well, yeah, we do.
We have this software that does this.
Well, page 12.
Why is it on page 12?
Page one of it in a really good place.
Here's the old image.
Here's the new image.
Here's how we did it.
Right?
That's what he wanted.
Now, partly it's a high bandwidth.
So what I'm wondering is, what was effective about that?
Was it that there was an ethos then that the most important thing that you had to bring forward wasn't a problem, but it was a solution?
And so that people were striving constantly to generate and communicate solutions?
Which seems like a good strategy.
Well, no, I think He's worried.
So an engineer sees a problem and they start investigating it.
And then as you're investigating, you develop understanding.
And you're understanding the problem is good.
You would think that, right?
You think that.
Lots of people think that.
Elon doesn't think that.
Elon wants a solution, right?
And if you're falling in love with your understanding and you're falling in love with your little details...
Well, that also feels like work.
You know, you mentioned earlier that just because it's hard doesn't mean it's useful.
Right.
And focusing on a solution...
He wants to know that you are focused like crazy on the solution.
The best way to do that...
So then you have two states, right?
If there's a problem, there's two states.
You have a solution and you don't have a solution.
If you have a solution that's on page one, we're all happy.
If you don't have a solution, why the hell are you talking to me?
Why aren't you finding a damn solution?
I've seen in the software projects, other projects that I've been involved in too, that focusing on a solution, I think this is along the same lines as what you're discussing is, well, then you get to product a lot faster.
It's like this thing has to exist and work.
Maybe it won't solve.
And the problem with the problem is that you can indefinitely investigate the problem and expand it, and also that that feels like work, but it's not saleable.
Yeah.
Yeah, it puts in the forefront of your mind that constant, you need to be creative in pursuing the problem, but also make sure you're really on track of the solution and you're just not falling in love with the problem.
People call it admiring the problem.
Some engineers are great at admiring problems.
I've worked with lots of people that come in with a 12-page presentation.
And when I'm done, I'm like, did you guys just give me a whole bunch more data on the problem?
He's like, yeah, we're really getting to the bottom of it.
It's like, no, you're not.
You're not getting anywhere.
Well, the bottom of a problem is a solution.
Because why would you just investigate the problem, right?
I mean, your destination...
No, no, they just keep getting deeper and deeper into problem admiration, and nothing happens.
Okay.
I'd say three-quarters of engineers would be perfectly happy to do that their whole life.
Because, so you explained this to me, complex mastery behavior, right?
So humans are very, you know, we like to learn.
Right?
We don't like to do dumb, repetitive things.
Right?
But we like to do things that are complicated, that we've mastered, that take skill and, you know, insight.
But you can have complex mastery behavior just analyzing problems.
Right.
Definitely.
Right?
And there's careers for that.
Some people find they're really good at it.
They're analysts.
They generate lots of data.
But if you're in charge of solving problems, You know, that period needs to be focused, short, concise, and you need to move on to solutions, right?
I think that's probably why I'm not so temperamentally fond of activists.
Are they problem admirers?
Well, that's what it looks like to me.
It's like, well, this is one of the reasons I admire Elon Musk.
It's like, well, you're concerned about the environment?
Well, why don't you build an electric car then?
Right.
Well, so a large number of people, and this is my favorite thing I learned, and it was working with mechanical engineers at Tesla, because they think the world's made out of silly putty.
Right?
They used to design, when we were building Model 3, they would design the part, and they would joke about how they're going to make it.
Are they going to CNC it, like mill it, or are they going to injection mold it, 3D print it, stamp it, make it with a hammer, cut it out with scissors, carve it out of a block, They had this cool machine that could carve 3D models out of clay.
It was funny.
So they could design things in their heads and on computers and then go build any damn thing they want.
If you ever look at a complicated mechanical assembly, there would be some screwed aluminum thing that would be milled somewhere and then drilled and there are screws going through it.
There'll be some little tab sticking off of it that holds another thing.
They can make stuff.
They think they can make anything.
And there's a whole bunch of people in the world that don't think they can make anything.
They think the world is what it is.
I had a friend, he had a rattle on his dashboard.
And he didn't know what to do about it.
And I was asking him where the rattle was, and I was thinking, and I was talking to him about how the dashboard's made, and he goes, Oh, I get it.
You think the dashboard's made of a whole bunch of parts that are put together some way.
I thought the dashboard was just the dashboards.
He didn't conceptualize it as there's this outer piece and there's inner brackets and there's radio and there's these things.
Mentally, I can't help but see the whole thing in 3D and I'm wondering which piece is loose and where it is.
And then how to fix it.
I'm not mechanical engineering creative, but I'm visual.
And for people who get stuck on activism as problem description, they don't think the world can change.
Which, at some level, makes sense.
You know, for human evolution, like, you know, it was pretty much the same for a million years.
Like, it's weird how good we are at change.
And my best theory on it is from zero to 20.
Like, your brain is going through radical change because you're going from You know, silly putty and not knowing much, being pretty smart.
So you have to change and adapt really fast.
And then humans are adapted to deal with each other.
And humans are fairly crafty.
You know, you have to deal with that.
But, you know, the lifestyle of most people from, you know, say, 30 deaths is fairly static.
And so we have this funny capacity for learning rapidly, exponentially.
And then dealing with slowly changing environments, but we're not naturally adapted to rapid change.
In the modern world, especially in engineering, it's rapid change.
It was just so funny.
You never knew what they were up to.
One day, they were working on the interior for the car and they made this crazy looking model, which Kind of looked like a car, but it turned out it was a thing you could move around and have the attachment points for all the interior parts.
So you could basically look like a weird skeleton, but it had the attachment points that you could adjust and you could build all the interior parts and put a Tesla interior together right in the middle of the or the engineering desk where it was really cool.
And let them go build it and think about it.
Because in the CAD model and the computer, you could see it, but it didn't always work out in real life.
We have a scale problem.
When you look at something that's small, even if you scale it up perfectly when it's big, sometimes that's just what you thought and sometimes it doesn't work.
And so you want to do...
Computers like to change things fast, but real scale models that you sit in and live it and get a human experience for it.
And it was really fun for that to just show up and be like, holy cats.
So I did a similar thing.
We took all the electrical subsystems and motors and laid them out on two tables, two big tables, covered with all the electrical parts of a Model 3.
We stared at it.
And once you see them all together, it's crystal clear that could be a lot better.
Because, you know, there's three motors that look almost the same.
Why isn't that one motor?
There's these two parts that are completely separate assemblies, but if you build it together, you could have one thing, do both things really much more naturally at a lighter weight.
Right.
And by laying that all out in front of you, you didn't have to do the mental work of representing that.
You could do the mental work of seeing how all the parts interconnected and what might be...
Yeah.
Yeah, Doug Clark is an architect I worked with when I was a kid at Digital, called it the interocular traumatic test.
When you look at it, it doesn't bug you.
And a lot of things, when you really lay them out like that, you go, oh, we're not doing this right.
And Elon likes that kind of stuff.
Like, you know, you're almost afraid to show them.
When you laid it all out, you looked at it and go, this is crazy.
So it was like, do we show Elon or not?
Because he'll look at it and think this is crazy.
Like, who built this car?
Who did this?
So I wanted to talk to you too.
I want to talk to you like I'm someone very stupid and in this particular regard I am.
I really don't understand how computation works.
And you're a microprocessor architect and you build computers.
And I listened to a discussion you had earlier this month and there was a lot of it I couldn't follow.
I thought it might be helpful and interesting.
For you, just to walk through for me and for my audience, how a computer actually works, what it does, and how you build it, and then what it would be like to design and to architect a microprocessor.
Yeah.
Well, it's somewhat hard to describe, but there's a couple simple things.
So let's start with the easiest thing.
The computers have three components, really.
Memory.
Programs and input and output.
Those are the three basic things we always built.
So memory is like the DRAM or the disk drive, place where you store data.
And it's just stored.
And it can have different representations.
We currently use ones and zeros.
So you can take any bit of information and describe it as a sequence of ones and zeros.
And it's stored in silicon And either static and dynamic memories or on disk drives, which, you know, there's a couple of technologies.
So does memory make sense to you?
A place to store information?
Well, it does, although I have some difficulty in understanding exactly how the transformation is undertaken to represent things in zeros and ones.
I mean, I'll give you a simple example.
So if you shine a light on a photo of photo detector.
Right.
So the light comes in.
And it's a stream of photons, right?
And the photodetector counts the photons, literally.
So every photon that hits it, or a couple photons hit it, they cause some electric charge to move, and that causes the circuit to wake up and say, I saw some photons.
So say you're trying to evaluate how strong that light is.
So it can be anywhere from nothing to super intense, right?
And then you might say, well, let's put that in a range of numbers from zero to a thousand, right?
And then it's trying to lay on it.
And so you count the photons for, say, a microsecond, and then you translate how many photons you counted into the number.
Right?
So, and so just imagine as the light varies up and down, the count, the number coming out of your photo detector is varying between zero and 1000.
Right?
And we use base 10.
But you can translate that to binary, which is base two.
And now you have ones and zeros.
You've basically now translated a light ray optical information to a count.
And so virtually everything can be represented by a count, apparently.
So just think of your computer.
It has a camera, which is essentially doing just that.
When you get different colors, but first you go through color filters.
So you have a green filter and a red filter and a blue filter.
That's enough to represent the color spectrum.
And then you have a little photon counter underneath that.
And it counts for a little while and then you ship out the number and you reset it and count again.
The light varies, you're getting that.
There's a pretty big grid, like a modern camera, 12 million photo detectors in it.
Actually, 36 million because it's got different colors.
Well, it depends on how they build it.
They might have different pixels be different colors and interpolate the colors.
Like a keyboard is really simple.
So the fundamental issue to begin with is that you reduce everything to a count and then you represent that count in base two.
It's base two, right?
Zero and one.
And you can do the same thing with sound.
You can have a sound detector that basically counts, you know, intensity of sound waves.
And the keyboard, well, you have a little grid of keys and when you push a key, it sends which key was that.
That was key number 27.
And it's either on or off.
On or off.
So D might be count number 26, and F is 27, and G is 28.
So everything gets encoded into a number.
And a computer is like binary numbers for technical reasons, but that's not a big deal.
So input and output is the first thing.
So the computer is built around the memory.
So the input and output system, so the inputs, all your photo detectors, sound detectors, keyboard detectors, and there's amazing numbers of sensors these days.
You can detect gravitational waves.
You can detect, you know, photons.
You can detect electrical waves, sound waves.
You can detect temperature.
You know, it's very varied.
You turn all that stuff into a number and then your input device writes that into memory.
And memory Stores information.
Memory used to be small and expensive, and now it's big and cheap.
But it's still just memory.
And nothing happens to it in the memory.
It just gets stored.
If you look inside a computer, there's a big memory, and it's full of numbers.
Now, the person who's writing the program is telling the input-output, like, here comes the input video stream.
Put the video stream address, 1 million.
So all the memory is addressed.
And the address typically starts at zero, and the modern computer goes up to billions.
Okay, so walk through that again, the addressing.
So what's exactly the function of that?
Well, you want to know where the memory is.
Okay, right.
You need to, okay, fine.
Yeah.
So basically, your phone probably, I don't know, has eight or 16 gigabytes of memory in it.
Maybe, yeah, four or eight.
So billion, you know, eight billion bytes of information in there.
So, and when you're designing your programs, you kind of lay out, well, here's where the operating system is going.
Here's what the input and output buffers are.
Here's memory we're going to use to run some program.
So, and all that's addressed.
So you can think of the addresses.
It's just like a post office, right?
So, you know, every house has a postal address.
You know, it's a street address and then your house address.
And so you can find every person.
And the address corresponds to the physical location, in some sense, to the physical location of the...
And how are the zero and ones represented in the memory?
It's literally a voltage that's either high or low.
And zero is using ground, which is zero volts.
And modern DRAM cells are probably stored at 1.1 volts.
And, you know, in a DRAM cell, it's a capacitor that's holding electrons.
So basically, when you store the cell in, either you drain all the electrons out, so it's zero volts, or you put a bunch of electrons in so that it holds a one volt.
So it's literally a number of electrons in there.
There's a couple ways to make memory cells.
There's another way, which is called a bistable element, where you have what's called cross-couple inverters, but that's too complicated to explain.
And then the memories are usually built in rays.
So there's an XY. You take the number and you say, I'll take the bottom half of the number and figure out which row it's in and top half the number of which column it's in.
And where the column and the row overlap, then I'll write my new data of a one and zero in that spot.
It's literally that simple.
So if you look at a memory chip, you'll see this array of bits with little blocks on two edges.
Usually, you know, one side's the row, one side's the column, and then the bottom is what they call the sentence when you read it back out again.
So a memory process is you activate the row and column to a spot, which gives you the address of that bit, and then you drive the bit in and charge up or discharge that cell, and then it holds them.
It's super simple.
Okay.
You could build a memory with a pegboard.
You could build a memory.
I mean, people literally did.
You know, way back when, there was something called Chlor memory, where they had essentially the XY grid, and at each little place, there was a little magnetic bead, which when you put the current through, you could put the current in the same direction.
You could make it be north to south, and in the opposite direction, south to north.
So you basically remagnetize the little beads.
So there's lots of ways to make memory, but currently the really dense memories are called dynamic memories, where you literally put charge in there.
And then there's fun stuff that happens, like flash cells, the cells got so small that the electrons, you know, from the quantum effects, tunnel out occasionally.
Right, because there's some doubt about where the electron actually is.
Yes, and sometimes it could literally jump out of the cell once it jumps out and doesn't come back.
So they got down to like 25 electrons in a cell and they would wander off over a couple hours and you have to refresh them.
So periodically, you go back and you read the data before too much of it's escaped and you write it back in.
So it's called refreshing the memory.
But DRAMs hold more charge than that and then the flash guys figure out how to stack the cell.
So modern flash chips, there's an XY grid, but there's also a Z dimension.
They're like 256 layers thick now.
So it's like a three-dimensional memory.
But the simple thing still is, it's a linear range of addresses where you put some data.
Okay, so that's memory.
So the next component?
Is programs.
This is the compute part.
A simple program is A equal B plus C. Right?
So the data at address A, so when you write the program, you tend to use what they call variable names A, B, and C. But there's a tool called compiler, which will assign A to say address 100, and B to address 101, and C to address 102.
Right?
And then the computer, when it's running, It says, do what I told you to do.
So you see this program, A equals B plus C. So you get B, you get C, you add them together, you put it in A, and typically what happens is you have what's called a local memory or a register file.
So you get the data from memory into the register file, you do whatever operation you're told to do, like add, and then you put C back in memory.
And what are the range of operations?
Or is that too broad a question?
What are the fundamental operations?
Apparently, you're arithmetic.
I've done this.
Like, the number of operations that a computer does, like, the construction sets can have 100 or 500 or 1000 different instructions.
But the most common ones are load data from memory to the processor or the program ones.
Store memory back.
Those are your first two instructions.
And then add, subtract, multiply, divide, you know, clear, you know, very simple.
It's written.
You know, then there's what's called logical operators and or not.
It's stunning to me conceptually thinking through this that the computers which can produce whole worlds in some sense can do that as a consequence of zeros and ones and arithmetic operators.
Sure.
Well, your brain is doing something interesting like that.
There's no magic to it.
So the key to programs is abstraction layers, right?
So at some low level, you know, like I understand computers from atoms up to operating systems, which is, you know, fairly broad range, but there's lots of people who can do that.
Yeah, and I understand them like the surface of the keyboard.
Yes.
Yes, monkey with military helicopter, basically.
Pete Bannon had an interview question.
So, you know, a computer scientist would say, tell me what happens when I move, when I type a key.
Right?
Because you can talk all day, I can talk all day about it.
You know, because the key is at the position which encoded the number, which got sent into the memory.
There's an interrupt delivered to the processor to say, there's new data in memory, go take a look at it.
But you can describe that at many, many levels.
Right?
So it's a good place to start.
It's not bad.
As an interview question.
As an interview question.
Yeah, right.
Some people, by the way, are stumped.
They go to college and they can't tell you what happens.
You know, a key is clicked, which is weird.
So back to the computer.
So it's the basic operations.
Add, subtract, multiply, divide.
You know, clear, set the one, and or not, XOR. You know, you can take a number and you can shift it around.
You can mask it.
And so different architectures.
How are those operators discovered, Jim?
I mean, I know there's their arithmetic operators.
And then is that just the question of how was their arithmetic discovered?
But I mean, there's a logic.
Way after masks.
So computers, at some level, they're doing arithmetic.
It's not very sophisticated.
And I'll get to a little more complicated version of this.
And by the time we invented computers, people had a pretty good idea of number theory.
They'd figured out that base 10 was just one of the bases.
You could have 2, 3, 4, 5, 6, 7, 8.
The philosophers have worked out what logic is.
If this is true and this is true, then this is true, this is true, or this is true.
Like, the logical operators are real.
Like, there's a whole bunch, there was a whole bunch of them.
Yeah, it's the realism of them that's stunning to me.
Yes.
So, there's the basic operator set, and then there's something called control flow.
So, computers typically, they put a program like add, you know, A equals B plus C, you know, D equals E plus F, F equals E plus A, you know.
And you typically put that in what's called program memory, but it's just part of the memory of the computer.
And you have a program calendar, which I don't know what's called calendar, but the thing that points at the next instruction to execute And its default thing is, do this instruction and then do the one right after it.
That became like the way computers are built.
That's an arbitrary choice, by the way.
You can have every instruction tell you where to get the next instruction.
Here's a bunch of things you can do.
But for simplicity, people said, this piece of memory has programs in it.
Start at the first instruction and then do the next one and the next one.
The program counters But that's not good enough because then you would just start at the first one and then you go to the end of memory and be done.
So there's something called control flow.
Sorry, called control?
Control flow.
So imagine you wanted to add up a list of 10 numbers.
So your first instruction says, I'm on the first instruction.
Then you say that the sum equals the current sum plus the next number.
Increment the counter of how many instructions I had.
Increment, count by one, and then test.
Is the counter equal to 10?
If yes, keep going straight.
If no, go back to get the next number.
So you created a little loop.
And it turns out that computer scientists invented a whole bunch of kind of loop, which they call control flow constraints.
Do this while x is true.
Do this until the counter gets to a number.
Right?
So you can create little, you know, basically sub-programs in the program.
Right?
And then there's a couple, you can test, like, hey, I need to decide if this is a dog or a cat.
So if it's a one, go look at here.
If it's a zero, go look at that.
Right?
So that's a conditional branch with root branches.
And then somebody famously invented subroutines.
You notice how he was writing the program, he'd write this little routine, but it would be used a bunch of different times.
So rather than putting the code in multiple times, it was like, define a word, and then whenever I need to use that word, I don't have to put the whole definition for the word, I just put the word.
The subroutine is like a local definition of something or a local computation that's used multiple times.
So your top-level program might be go to the subroutine that counts up numbers, comes up numbers and come back.
Now go to the subroutine that checks whether it's your bank balance or not, come back.
So the program this becomes sequential operation Control flow, like doing loops, let's say, do this until something's done.
And then conditional branches that says, depending on value, do this or that.
And then subroutines to do something atomic.
And that's essentially all appropriate.
Operations, loops, conditional branches, and subroutines.
That's it.
Now, why can computers construct worlds?
So I still remember when...
So if you look at your screen, you know, the computer in front of you probably has two or four million pixels on it.
Seems like a lot, right?
And when they first started, you know, televisions, when they lit up screens, you know, they were scanning the little electron beam across a phosphorescent surface and modulating the intensity of the electron gun.
To make the little fosters brighter, little.
Right, and it was writing, it writes one line, it wrote one line at a time at an incredibly fast rate by human standards, and we saw that as continual Right, yeah, and then the phosphor was designed to decay at the rate, so by the time you came back to it, it had just gotten a little dimmer, and then you wrote it with the next value, so it didn't flicker.
So your eye has some persistence, so the electron hits it, it makes the phosphor light up with some photons of the right color, and then it slowly decays, and you scan down and it gets back there and writes it again before it's too dim.
And so the screen on a phosphor-based television, is it analogous?
It's analogous in some sense to the binary representation?
That's entirely analog.
So it's digitized in the sense it's discrete, I'd say, in the sense that each little pixel, you can see the little phosphors in the screen.
Right.
Especially notice that when they went to color TVs because they have a red, green, and blue thing in there.
Right, right.
And they would hit them with the red.
And they're essentially either on or off?
Well, they have a range.
So that beam is a variable intensity.
Right.
So now modern computers work differently.
So the screen in front of you has a little, it literally has an XY grid and it can address each one of those things.
Right?
So, you know, you don't shoot a beam at it anymore.
You have an XY, you decode.
You know, it's almost like the screen looks like a big flat memory.
But instead of storing ones and zeros, it's storing color.
But they have the same kind of decay property, and you write a new color, and there's a bunch of stuff.
Now, here's the wild thing.
Computers are now so fast, you can run a 10,000-line program for every single pixel on that screen.
And what does that imply?
Well, it turns out for a whole bunch of reasons, like if you want to make something look really good on the screen, so the world is relatively continuous, right?
So if you look at it, there's all this light reflecting around, there's all these things going on, there's no little pixels in the surface of your table, right?
To make a discrete grid look that way, You have to combine the colors.
You have to do a whole bunch of stuff.
You have to pretend you're shining lights on it.
There's a reflection from one surface to the next one.
And it turns out when you have thousands of instructions per pixel, you can start to make those pixels look realistic.
The operations, if you go look in the pixel program, it looks so beautiful.
You think that's incredible.
But if you look in the pixel program, it's load the data into the register.
Add it to a number, test it against the number, subtract something.
There's something called clipping, like make sure the pixel doesn't get brighter than this and dimmer than that.
It's all simple operations.
There's nothing in the computer that's like, do a pixel operation.
Well, there might be a subroutine named that, but underneath it, it's just the same old stuff.
Computers always do load, store, Add, subtract, multiply, divide, branch.
Okay, so how far have we got?
I'm listening to so many things, I'm having a hard time keeping track of the order.
You mentioned earlier that computers consist of four elements.
I believe that's what you said.
Memory.
Yep.
Input and output.
Yep.
And compute.
Okay, three.
So I was counting input and output separately, but okay.
And have we gone through all three of them?
Yep.
Okay.
So memory is just a place to store bits.
Yep.
Input and output is typically the way to...
It depends on what you're doing.
You might just send bits from one place to another, but it might also be...
You could say input and output in the computer and sensors are slightly different things.
Like sensors turn analog real-world signals into bits.
Into digital.
Right.
And then programs basically transform the data in some way.
And programs is basically...
You know, operations like add, subtract, divide, and then branches that either let you do loops or make decisions, and then the hardware that lets you do subroutines to break the program into pieces.
And that's pretty much it.
So, to some degree, you take the world, you transform it into on-off or yes-no.
Billions of those.
And then you manipulate the yes's and no's or the zeros and ones, and that can produce almost any sort of phenomenon that you can imagine.
Yes and no is not a very good, you know, ones and zeros is better because then it's a mathematical representation, you know, a digital representation of an analog reality.
Something like that.
And is the analog reality analog all the way down or is it digital at the bottom?
Quantum at the bottom.
So there's something called the fine constant, which makes the universe look discreet, but it's a very, very small number.
Right.
And there's a fun fact, which is...
Is that the Planck length?
Is that associated with the Planck length?
Yeah.
That's the smallest possible length, I believe.
Like the mass of the universe is 10 to the 40th, and the Planck length is 10 to the minus 40th.
And there's a...
Physics thread about the mystery of why those things are 10 to the 40s and 10 to the 40s.
All right, so let's move from that to, I'm going to ask you, these are the questions.
I just want to say, so the thing that makes computers do what they do is abstraction layers.
So at the bottom, there's atoms.
So there's engineers who know how to put atoms together in a way that makes switches, which we call transistors.
And those guys are experts at that stuff.
And they can operate at that level.
Then there's another thing where you take multiple transistors together and you basically make what's called logic gates, which literally do the ANDs and ORs and inversions.
And then that's an abstraction layer.
We call it the physical design library or something like that.
And then people take those and they make them up into adders and subtractors and multipliers.
This is a well-understood Boolean math.
So how do you add two binary numbers?
So you make those.
And then there's another abstraction layer that says, all right, I'm going to take multiple operation units and put them together to make, you know, part of the computer, right?
And then you make there's a bunch of those blocks.
And then that thing runs a program very simply.
And there's a small number of people who write programs at the low level, but then there's people who use what's called libraries where they, you know, they're doing some higher level program and so they're going to do a matrix multiplying and do this map, but they don't actually write that low level code.
So there's, you know, there's a stack of abstractions.
And when something gets too complicated, you split the abstraction layer into two things.
There used to be when people wrote a program, there was a program called a compiler that translated your C program or Fortran program into the low-level instructions.
But it turns out there's too many languages up here and there's too many instructions here, so now they translate it from the high-level language into an intermediate representation, which is sort of, let's say, a generic program.
And then there's another thing that translates the intermediate representation into a specific computer you have.
But that just keeps going higher and higher.
A lot of programmers, they use frameworks that can do amazing things.
You can literally lay a program that says, search the internet for a picture of a cat, sword by color, output to my printer.
There's a language where that's a program.
Search the internet.
Holy cow, that runs a trillion lines of code on 100,000 computers.
Find a cat.
That's a really complicated program.
So how much of the radical increase in computation power is a consequence of hardware transformation?
And how much of it is a consequence of the increasing density, let's say, of these abstraction layers?
Well, so this is where, you know, there's a really creative tension or dynamic interplay.
So when computers first started, they were so slow, you ran really simple programs, A equal B plus C times D, right?
And we've been going up the math hierarchy.
So then you could run a program that did what's called, you know, matrix math, or linear algebra, systems of big equations, and then matrices, and then more complicated ones.
So as the computational power went up, you could dedicate more and more stuff to that kind of computation.
And then a similar thing happened on abstraction layers.
It used to be if you bought a million-dollar computer, you hand-wrote every line of code because you didn't want to waste time on the computer with overhead.
But today, you know, that million dollar computer costs 10 cents.
You don't really care how many cycles you use, you know, parsing a cat video or something.
And so the computation capacity let the abstractions at the programming level increase a lot.
So somebody had a graph about how many bytes does it take to store the letter A? Like, it used to be one.
And then Word for Windows, it's like 10 kilobytes per letter.
Because the letter has a font, it has a color, it has a shadow.
You know, there's a whole bunch of, you know, and that's fine.
Like, the computer with a million dollars for, you know, a thousand bytes of memory, you wouldn't store letter A like that.
You'd put it in one byte.
But now you have gigabytes and terabytes of storage.
Who cares?
You probably already know that there are data brokers out there selling your internet data off to companies who want to serve you a targeted ad.
But you might be surprised to learn that they're also selling your information to the Department of Homeland Security and the IRS. Mask your digital footprint and protect yourself with ExpressVPN.
One of the easiest ways for brokers to aggregate data and tie it back to you is through your device's unique IP address.
But when you're connected to ExpressVPN, your IP address is hidden, making it much more difficult for data brokers to identify you.
ExpressVPN also encrypts 100% of network traffic to keep your data safe from hackers on public Wi-Fi.
You can download ExpressVPN on all your devices, your phone, your computer, even your home Wi-Fi router.
Just tap one button and you're protected.
Make sure your online activity and data is protected with the best VPN money can buy.
Visit ExpressVPN.com slash Jordan right now and get three extra months free.
That's ExpressVPN.com slash Jordan.
Okay, so you walked us through the basics of computation.
Now, can you shed some light on, like, I don't understand what you do as a computer architect.
Like, when you go to work, when you're working on a project, what is it that you're actually involved in doing?
I make ads go faster.
So, I'm a fairly low-level engineer.
You know, low-level in terms of the abstraction layers.
Like, I understand the higher But, you know, I talk to the people who make transistors and N-gates and R-gates.
And they talk to the people who know the atoms.
Right.
And I hardly ever talk to the atom people, but I know something about atoms.
So I build, you know, when I'm an architect and stuff, the functional units and then how they operate together at the low level that runs programs.
I don't write programs.
I'm an architect of the computer that runs programs.
Right?
And then it used to be you could look at a computer and you know how a program works.
You run the first line, the second line.
If there's a branch, the branch unit, then you branch.
Right?
And the computer would literally have that in it.
Set some instruction, load the data, do the operation.
If there's a branch, execute the branch.
If necessary, change the program counter.
So, you know, people know there was a period of time where computers had like five stages and each one of them could say that's the branch, that's the fetch unit, that's the load unit, that's the add unit, that's the branch.
But monitoring computers are more complicated than this.
Because computers like that would do one instruction every five cycles.
And modern computers, the fastest one I know about Just doing ten instructions.
Ten instructions to cycle in parallel.
And this is difficult.
Unpack that!
Unpack that!
So if you write a program, since you write, when you write, you write linear narratives.
You write a sentence that makes sense, followed by another sentence.
And so as you're writing along, sometimes the one sentence defines the meaning of the next sentence.
And then group it into paragraphs.
You might call those subroutines, right?
And sometimes the paragraphs have to be ordered and sometimes the paragraphs, the order doesn't matter.
Right?
So programs are written by human beings and they're written in the same linear narrative.
So if you want to go faster than parsing the instructions one at a time in order, you have to do some analysis to say, all right, I got two sentences.
Are they dependent or not?
If they're dependent, I do them in order.
If they're not dependent, I can do them in parallel or any order.
So the modern computers, when they're reading the programs out, they're analyzing the dependencies and deciding what can happen in order, what has to happen in order for correctness, correct understanding, and what can be reordered.
And then it turns out there's many places where you say, If there's an error, go here, but there's hardly ever an error, and you can predict that really well.
So you say, I'm going, you're reading along, and you say, here's a point where I'm not sure which, should I read the next sentence, or should I jump to the next paragraph?
Right?
So a modern computer predicts that.
It doesn't wait for you to fully understand all the sentences up to that point, so you know exactly where to read it to.
So now you're reading this book, and you're reading sentences in dependency order, which means you get to a branch, and you haven't read all the sentences before that and understood them.
So you don't know where to read the next paragraph or the next chapter.
But we predict what's going to happen, and we just keep on going.
And how does that tie into the process of designing the...
So the goal of modern computers is to go fast.
Well, let me say there's three kinds of computers.
There's computers that run very simple programs in order.
Right?
They just do exactly what you told them to do.
And they tend to be small and simple.
But they're so small and simple, you can make a chip with a thousand of those computers on them.
So when you build a GPU that does a little program for every pixel on your screen, each one of those pixels gets its own program.
It's very simple.
But you sort of say the first thousand pixels you run on these thousand computers.
So like a modern GPU has currently like six or eight thousand processors in it.
And they literally You do the first 6,000 pixels, and then the next 6,000 pixels, and the next 6,000 pixels.
And they do that fast enough that you can run a fairly big program on every pixel on the screen for every screen refresh time.
So you have simple computers that do stuff in order.
And then you have, let's say, computers that are designed to run complicated, long programs as fast as possible.
Right?
And that's where you parse the instructions carefully and you figure out what order you can do them in, and when possible, you reorder it.
Right?
And the reason you reorder it is because if this doesn't depend on this, I can do in parallel.
Now I can do two things at a time.
The next thing I can predict that I can do it in parallel, I can do three things.
And, you know, like I said, the computer in your desktop is probably doing three to five things at a time, and the best I know of is ten.
Right?
And there's other sophisticated predictors in there.
So to do that, you have to fetch large groups of instructions at a time to figure out where the sentence boundaries are, figure out if they're dependent or not, figure out if you can predict where the next instructions are coming from when you hit branches.
And it turns out that's fairly complicated.
The difference between a little computer that does, let's say, one instruction at a time and a complicated one that does 10 instructions at a time.
It's 100 times more complicated.
Right?
And from a, what's the best way to do lots of instructions?
Complicated computers are not efficient.
But there's so many applications where people care how fast it is.
So when you're clicking on your web page, you want that to come up as fast as possible.
So the part of it stats, let's say, what's it called?
You know, the logic of the of the web page, it's probably a serial narrative written by a human being.
So you have to run that on a complicated computer that does it out of order and predicts what to do as fast as possible.
But when you render the screen itself, that runs on large numbers of simple computers to make all the pixels.
Right.
And then there's a third kind of computer, which We're starting to invent, which is AI computers.
And that's what you're working on now?
Yes.
For TENS Torrent?
Yeah.
And there's a really good talk by Andre Karpathy called Software 2.0.
So the first two kinds of computers, simple computers and complex out-of-order computers, they're running programs written by humans.
Right?
And if you look at the code, It's literally declarative statements about operations and where to go, and it's serial.
It's a linear narrative.
The different thing about AI computers is you use data to train the weights in neural networks to get you the desired result.
The programs are no longer written by humans.
Now, it turns out there's components of the AI stack that are written by humans, but at a high level, you use data to train them.
So you have a big neural network, and you want to detect cats.
So you put a cat picture into the network when you start training, and the output is gibberish.
And you compare gibberish to what a cat is, and you calculate the difference in what the network said versus the desired result, which is the word cats.
And then they do something called backpropagation, which is mathematically sophisticated, but essentially takes the error and partition it across the layers of the network such that you've sort of bumped each neuron a little closer to saying cat next time by taking the bigger at the end and distributing it across.
That's called backpropagation.
And then you put another cat in.
And if you have the right size network and the right training methods, After you show the network a million cats, when you put a cat in, it reliably says cat, and when you put a picture that's not a cat, it reliably says not a cat.
Right?
And you never wrote any code that said anything to do with cats.
And can you understand what it is that the computer is doing now that it's recognizing cats?
A little bit.
So people for years worked on visual computing, and they were trying to detect things like cats.
And cats have a whole bunch of artifacts.
They have round eyes, they have pointy ears, they have fluffy hair.
So you could detect, it's called feature detection.
You would say, this will be a cat if I see the following colors, the following amount of fluffiness, the following number.
You know, two pointy ears, not three.
One or two round eyes, depending on the view.
Right?
So you could write code and the problem with that is, well, now the cat has an arbitrary orientation.
So you have to, you do your feature detect on the picture and the features have to search the whole image and you have to rotate around, you know, and it's sort of, and every single thing you want to detect, you have to write a unique program for.
You're done with cats, now you go to dogs.
And then what about the dog that has slightly pointy ears?
These dogs have round ears and cats have pointy ears.
So it was sort of an endless thing.
Same thing with speech.
Endless detail by detail construction.
I had a friend who worked on speech recognition years ago.
So you break speech into the phonemes so you can see those and then they have frequency characteristics and you can differentiate vowels from consonants.
So those people working on speech were doing a whole bunch of Analysis of analog waveforms of sound.
And they were making some progress.
But it never really worked.
And then they trained neural networks by you put the word in and you have what's called supervised learning.
So you play a language where you know what all the words are and you keep telling the network how to correct.
And with like a billion samples and a big enough neural network, They can recognize speech just fine.
And if you train it with a broad variety of accents, it can work across accents.
And then it turns out the bigger they made these networks, the more information they can put in.
And then on the CAT one specifically, they found...
So when they first had a neural network crack the CAT problem, I forget, it was like 50 layers deep.
And if you looked in the layers, you could see that it was detecting pointy ears and eyes.
But it was also detecting a lot of other things.
And some things we don't know.
Yeah, well, if we see the back end of the cat walking away, we still know it's a cat.
And it pretty much lacks eyes and pointy ears from that perspective.
Funny thing, if you take an object, like in light, take a phone, you can project the phone onto a flat surface.
Say that's a projection.
And as you move it around, you get different.
It's a shadow.
Shadow.
But think of it as a projection.
So that's a projection of a light source on a flat plane.
It's a fairly simple projection.
But what if you had a light shaped like a cat and you shined that on the phone?
What would the projection look like?
And it turns out mathematically, There's an arbitrary number of projections.
We think of projections in three dimensions because we're three-dimensional creatures.
But there can be lots of projections.
And then you can have the projection project on another plane.
So what the neural networks are doing is they're teasing out all the details of what that is.
And some of the projection planes give you what's called, you know, size invariance or rotation invariance.
Like you could recognize a cat in which it's pointing.
Like your brain is a little specialized, like faces.
Right, it likes them to be vertical.
But with a little bit of work, you can recognize an upside down face.
Unless you have a problem.
Okay, so we could do two things here.
We could either talk about No, let's go into your...
You were an engineer and then you were a manager.
And you've worked in lots of companies, some of which were incredibly creative, some of which were thriving to an incredible degree, and some of which were collapsing and irreparable.
So what have you learned about what makes companies work?
And more importantly, what have you learned about what makes them not work?
And maybe what do you do then?
Sure.
Well, that's a fun question.
Well, first of all, I've noticed and many people have noticed, this is not just me, that people in engineering fields, people kind of bucket towards technical people and management people.
And it's not that there aren't good technical managers or there's not good managers or technical people who can manage, right?
Right, but that's an intersection of two skills, say.
Yeah, but generally speaking, most people are one or the other.
And it's like when you wake up in the morning, do you want to solve a problem or do you want to organize the problem?
Like, are you worried about your schedule and your headcounts and how things are getting done and did you hit the milestones?
Or are you working on a technical problem?
And in engineering fields, it's often there's the fellow track for the technical leadership position or the director of EP track for the management leadership.
So I'm a technical person.
But you took on management roles repeatedly.
Well, I did because I found out that if you're, generally speaking, the top of the organization is the manager, the VP. And as a technical person, no matter how high you go, you're an advisor for that person.
And I decided consciously after I worked at Apple that I was going to be a VP and have everybody work for me because then I can.
So then my skill set is somewhat unusual.
I'm not the only one, obviously, but I decided to, you know, get on the management track so I could build the computers I wanted, because sometimes when I wasn't the leader of the group, So managers at some point would decide they own the next decision and they would make some random decision.
I'd be grumpy about it and there's nothing I could do about it because people work for them, not for me.
So that's, you know, it was a conscious thing.
And I hired a consultant, Venkat Rao, who helped me reframe how I approached this.
Now, I'm still a technical person, but I found that it turns out there's a whole bunch of really good technical managers that I like to work with.
I'd like to organize stuff.
And I would say, you know, I maintain my openness and low conscientiousness and disagreeable behavior.
And I have people who work for me, who work on my team or work with people that manage better.
So even though I've been a manager at AMD, it was 2,400 people total at the end until it was 10,000.
My staff is, you know, 15 or 20 people.
And usually half of them are Real managers, and half of them are technical leaders.
That's how we solve it.
And there are lots of companies running away.
A lot of times, founders tend to be technical people, but people working for them are non-technical.
Or they're stronger on the management side than the technical side.
But for everybody, you need to decide who you are.
Like, I had a great technical manager at A&D, and one day, he was a little mess because I was looking to a couple of the really technical heavyweights to solve a problem.
He said, He said, you know, I'm pretty technical.
I said, yeah, I know.
I said, are you technical compared to Jim and to Barb?
And he goes, I guess not really.
I said, I know.
I really like, you know, what I want you to do is you're running this project.
You have 150 people working for you.
You make all the technical decisions you can, but when it's out of your wheelhouse, we got serious experts.
And you have two choices.
You can call them or I can call them.
And he later told me, he said, I found out it was a lot better when I called him than when you called him.
And successful thing.
And he was technical.
He was really good at making good decisions, but he wasn't the strongest technical person in the group.
So.
So that's the first thing is, you know, figure out who you are.
I've seen a lot of people fail in engineering because at some point they think I'm technical, but I want to get on the management track.
But they're bored by management.
And they don't have a plan to deal with it.
Well, you weren't bored by management.
I joke that I decided to see the organization as a computer architecture problem.
Well, that's exactly what I was going to ask.
What transformation did you have to undertake?
One of them was, what do I have to do to be effective?
I hate to work on failed projects.
And then the next was the organizational problem itself is an architectural problem.
And then I kept, you know, for myself, well, I'm a funny kind of, if something has a solution and it's being confidently driven, I'm not that interested in it.
I like problems.
And so in a big organization, there's a million problems, and I start sorting them by priority and then solving some of them or handing them out to the right people.
So there's a whole bunch of technical work to do on that.
And then I'm fairly good at skill assessing people who are technical, either for management or technical positions.
And then, you know, giving them work.
I like autonomy in management.
So if somebody's competent, they can do it and they understand it.
Ben Gatt gave me a bunch of books to read.
And one of the frameworks It's goals, organization, contract, and teamwork.
Or capabilities, I guess, we usually solve for that.
So is the goal super clear?
Do we have the capability to solve the problem?
Is there a contract between me and the groups doing it?
So they know what to do and what their goals box are in.
And is the organization that Like a lot of times, there's a joke that startups start with a problem and build an organization to support it.
But on the second, third system, the organization defines the problem rather than the problem defining the organization and then it breaks up.
Yeah, then the organization becomes the problem.
Yes.
Yeah, they constrain the problem and become the problem.
Well, we had a number of discussions while you were doing this about ethics.
I mean, you said that you go, you look at the problems.
Well, that's hard, right?
Because you have to know enough to know what the problems are.
Then you have to be willing to look at the problems.
Well, then you prioritize them.
Like, you skipped over that very quickly.
But all of that's extraordinarily difficult, I would say, both cognitively and emotionally.
Sometimes it is, and sometimes it isn't.
Like, when I joined A&P, the CPUs were less than half as fast as the competition.
And they had no plan to catch up.
So that wasn't that hard.
No, but what would be hard there, I would presume, is figuring out how it could be that such an obvious problem had gone undetected and unsolved.
No, actually, one of their architects, when I was working at Apple, told me that they believed that CPU performance had plateaued.
It wasn't going to get any faster.
And they were going to work on adding features to the rest of the chip.
And then Intel came in and said, we think computers are going to get 5% or 10% faster every year.
And they did it.
One had one goal, which is, you know, things slow down.
The other had a different goal.
5% or 10% isn't a lot.
But you do that 10 years in a row.
And, you know, the other guys weren't.
So that wasn't that complicated.
Like Elon famously said, he tells everybody secret plans and nobody believes them and then does them and they still don't believe them.
And then they're like, oh, shot.
So, Intel publicly said they were going 5% or 10% faster every year, and AMD said, no, they're not.
You know?
And the results were, at some point, the gap got bigger and bigger, and, you know, the people at AMD were committed to their plan.
I don't know why, but it's interesting how these things get internalized, and then you start, even when they, you know, at some point, you know how it is, it's cognitive dissonance.
You say you're going to do something different, but you learn how to do this other thing really well, and you keep doing it.
Well, you build a whole machinery around it.
Yes, exactly.
They had a big machine that did all kinds of stuff that was perfectly useful.
And good people doing it.
Like I said, we didn't hire any new people to build then.
But we did refactor, reset the goals, refactor a whole bunch of engineering.
Okay, so at AMD, you were successful twice.
And so, the success was both building a chip that was competitive, so you had to put together the teams to build the chip, but also to transform the internal structure of the company so that that became possible, and then also to communicate that to your customers.
And so, what's the problem set there?
I didn't communicate with the customers.
You know, because, you know, computer were all performance cells.
Okay, so that's the first thing you brought to the table, performance cells.
We're going to break that down.
Here's the measurements.
Here's the measurements.
And there's lots of public benchmarks.
Everybody tries to gain it, but generally speaking, there's a really big community of computer users, and they know what they want, and they know it's faster.
Right, and you know exactly how a computer works, so you can actually say, once you decide that what faster is better, does that work on all of the elements of design?
It's a little complicated too, right?
There's certain things like you can make it faster, nobody would care.
Like, yeah, there's some judgment calls in there, but it's not that complicated.
Like, you know, today on phones, there's a thing called Geekbench, and, you know, you get a number at the end.
You know, is your Geekbench score 100 or 50?
100's better.
Right, right, right.
The people who made the benchmark tried to pick the components of your phone experience such that the Geekbench number represented whether the phone was faster or not.
And then whether you care or not is another question.
Like, for the current applications, if they got twice as fast, you might not notice.
But as the computer gets faster, new applications are possible.
And on the phone, where it's possible, it's great.
Where it's not possible, it feels slow and laggy, right?
So performance wins and, you know, different form factors like a notebook or a desktop or a phone have different amounts of power.
People live within the budget.
Okay, so you had a goal.
You had the measurements in place.
You decomposed that into tasks.
You assigned competent people.
What psychological factors got in the way?
How did you see companies...
All of them.
Yeah, fair enough.
All of them.
But what did you see specifically interfere?
Once you have a good plan in place, that doesn't necessarily mean it's going to be implemented.
And so what are the mistakes that people make That you saw in large companies that doomed the companies or that stopped them from transforming internally.
So there's a couple of very separate problems.
When somebody with a good set of ideas says, I need to transform this place.
Are the goals proper?
And then you want to say, do I have the capability in the team to do it?
I worried when I went to AMD, I wouldn't have enough experts in certain things to do it.
I'd have to go hire 50 people to fix it.
But it turns out, there was plenty of good people.
Actually, some really great people.
So I was like, you know, pretty quickly checked off the capability box.
And then you start wondering, well, why the hell are we doing the right thing?
Well, the problem was the goals were wrong, and then your organization was wrong.
Right?
And then, generally speaking, if those aren't right...
So to begin...
So maybe to begin with, the goals weren't unreasonable, and no one knew.
But then, across time, the fact that one set of goals was better than the other...
The belief that computers weren't going to get much faster was a bad goal in the world where the competitor believed they were going to get a lot faster.
Yes, and could do it.
And that became incrementally worse across time, to the point where it became cataclysmic.
Yeah.
So, you got to get the goals right, and you got to establish where you have capabilities.
You know, those are kind of fundamentals.
But then the organization building is hard, because somebody will tell you, so-and-so is a great manager.
Well, is he?
Or, you know, like a lot of times, there's somebody who looks like a good manager, but you just have three people working across the street.
The problem with that is when things are going well, the empty suit manager with his good people supporting him, they can look like they're making lots of progress.
But when they run into hard problems and the technical guys don't want to do it, they go to him and makes a random decision or does something dumb or doesn't believe him.
That happens a lot.
The technical guy goes to the empty suit manager and says, I think this isn't working.
We need to change.
And he says, Now we're fine.
We'll just go right through it.
So you get these weaknesses in the organization because you don't have the skill level.
Like I said, I've worked with a lot of really good technical managers who know when they can make the decision and they know when they have to punt to somebody who's more of an expert.
That's great.
And it turns out some people are so good at that, they can operate way higher than you think because they're not technically strong.
They're super good at translating and making judgment calls.
So you've got to start building your organization.
And then there's stuff about how you build teams.
Like some groups are what's called functional.
But all the people who do software in one group, all the people who do hardware in one group, all the people who do atoms in another group, right?
And then the managers.
But if the thing you're building needs a little of all three of those things, you know, it's called, you know, Functional organization versus product organization.
You might want a team with a couple of programmers, a couple of hardware people, a couple of Atom people, and the same team.
So they all have one goal, as opposed to the functional group that says, I'm making the best software.
Well, is it the right thing for this product?
They go, I don't know.
I don't work on the product.
I work on software.
So I'm generally speaking product focused.
So if you only have five of some discipline, You tend to make a little functional team like that.
There's a couple things in computer design which are functional.
But generally speaking, I like product-focused organization.
So everybody's like, they're all working together on the same thing.
They may have different disciplines.
Now you've encountered all sorts of frustrating situations when you've gone into companies where you're trying to put together a good product.
And so what do you see as particularly counterproductive?
And what have you learned how to conduct yourself so that you can be successful?
Weak leadership.
People who can't make the technical decisions, they have to.
That's a big problem.
Functional organizations where people are optimizing for the function, not the product.
Bad goals is one of the worst things.
Some organizations have real capability gaps.
They think they have the right people, but they don't.
Some managers play favorites.
They think so-and-so is really good and they're not.
Yeah, so that's a real functional analysis.
The company just can't do what it needs to do.
And we're still analyzing.
Here's a group.
They're actually from someplace.
There's a belief that we're going to build this product.
It has to be a great product.
How do you do the basic blocking tax and how to make that successful?
That's different than the malaise that overtakes big successful companies.
Which you can generically call bureaucratic capture.
That's a different problem.
A company that's bureaucratically captured will manifest all kinds of bad behavior in the organization and product development.
And then some big companies where the bureaucracy has taken over, there might still be groups that are really doing a great job making great products.
I think there are separate spaces, and I understand both of them pretty well.
And again, the way you solve big complicated problems, you have some abstractions about what you're dealing with.
So a framework like goals, organization, capability, and contract is a super clear method for evaluating what the hell is going on and then making changes.
Very specific changes to it.
Goals are clear.
If you're not clear, nothing else matters.
Get the goals clear.
Capabilities, are they good?
If you don't have the right capabilities, nothing will save you.
You have to have the ability to do the job you're doing.
Does your organization serve the goals?
That's a big problem.
That's a painful one because that's when you start changing who works for who and what the boundaries are.
But you have to do it.
Okay, so let's tackle it this way then.
So you're going to pick someone who has optimal attributes to create and operate within a highly functional organization.
What are you looking for in that person?
What's crucial?
Well, people are fairly diverse.
That's the funny thing.
So engineers need to have this will to create if they're technical leaders, let's say.
And then they have to have the discernment to make decisions about whether they're actually making progress towards the goals or just wasting their time on something cute.
That's the thing.
Technical managers, they need to know how to run a program.
They need to know how to hire and fire.
They need to know how to structure work.
They need to know how to evaluate how long it's going to take, how to evaluate whether people are making progress.
There's a whole bunch of things, but then people have very different styles.
Some people are very extroverted.
I worked with this woman.
She was great.
She would just have these team meetings and she would really get out there and energize the team.
And another guy in the same building was very low key and he would wander around and talk to people and have a really good sense of the team, like an introvert versus extrovert style.
But they both worked.
They were both very competent.
They were both, to me, you know, Really good technical competency.
They weren't my technical leads, but they were technically competent enough to make the decisions and know when they had to punt the decision up.
So, who do you not want?
Okay, so, I mean, that kind of goes along with the management literature.
You see that you want people who are intelligent, especially for complex jobs, so they can learn.
You want people who are conscientious because they work hard and they have integrity.
Then, with the other dimensions, it looks like there's a fair bit of variability, although...
Too much negative emotionality can be a problem.
I think that's because it's associated with depression and too much anxiety and so on.
But there's diversity in the other personality dimensions and that might be task specific.
But what sort of person do you not want to work with?
Fakes.
There's lots of fakers out there.
They have sales attributes.
They're extroverted, agreeable.
They want to say everything is good all the time.
They're not sufficiently Concerned about disaster and digging into stuff.
They may have some kind of narcissistic personality problem.
So they're imposters.
They're mimicking competence.
Mimicking competence.
That's a problem.
There are people who literally...
They take credit from other people.
I kind of put that in a separate boat, but there's people who take credit for their team.
I realized early on there's two kinds of managers.
People put it up and people put it down.
Like, as a manager, I often tangle with the people I work for, but I always took care of the people who worked for me.
But some other managers, you know, I had this one guy who worked for me.
I thought he was great.
And then I walked by a meeting he was having.
He was abusing his team.
They hated him.
I fired him because he always said the nice things to me.
And, you know, he hit on those people.
So it's...
Yeah, there's a bunch of weird stuff that happens in management like that.
You have to be excited.
If you're a senior manager in a high tech thing, there's many people in the group that are smarter than you, and you have to promote them and put that forward.
You can't be uncomfortable because somebody's smarter than you.
When I was at A&D, I had six senior fellows.
I think they were all smart.
They weren't as generalists or something, and they didn't have my interest in architecture of organizations.
But man, it was smart.
Super good.
I could talk to them all.
I could keep up with them sometimes.
But I was more than happy to promote them as the smart guys.
Why were you confident enough, do you think, to allow you to be surrounded by people that you...
I'm above average smart, but I met people who are so smart.
I knew Butler Lamson, who's a famous unrated IQ, and his wife was smarter.
It was a joke that he spoke at half-lampson because his wife was smart and spoke really fast.
At a fairly young age, I was competent in getting things done and worked with people that were smarter than me, but they liked my, you know, I'm an engineer and I build stuff.
You know, this rocket scientist, they think it up and then they hope somebody will build it for them because they're off onto the next thing.
So that's, you know, a belief I have.
You know, it wasn't always easy.
I still remember working on EV5 and I went to the digital research lab and there was half of those super smart people and I started describing what I was doing and I would describe something for about two minutes and then they would spend five minutes taking it apart and analyzing how it could be like way better and then they'd ask me the next question and after an hour of that I felt like oh my god I was just beating to death and uh they were like this is great Jim I was like you thought that was great they're like yeah we're glad you're doing it so I've
always had that attitude since.
But yeah, it's hard on some people when they realize how smart some people are.
I make up for it because I'm open-minded and I've worked my ass off for many years and then I've dived in lots of things.
I'm not afraid to ask dumb questions.
A lot of people are trying to project who they are so they don't ask the right questions, they don't learn anything.
And I'm like, I don't understand what the hell is going on.
I've done that in a room of 50 people.
And they're like, well, we thought you should know.
It's like, well, I don't, but I'm not going to leave until I do.
And then they give all the information.
And I'm smarter than I used to be.
And, you know, so that takes a certain mental resilience.
And sometimes it's very hard on me.
But, you know, again, it's sort of like...
You know, do you fire the people you have to fire to save the group and save the product, maybe save the company?
Yeah.
Because the net good is really high.
And that's the right thing to do.
So exposing yourself is the right thing to do, ironically.
Now, it's hard and, you know...
Well, if you admit you're stupid, then sometimes you don't have to stay that way.
Yeah.
It's hard in some sick organization.
Sometimes it's not safe to expose yourself.
I feel for people who are in places where they would really like to be more open and can't because organizations that get political and bureaucratic are hard on people that are actually trying to do the right thing and learn.
I totally understand that.
It takes a while.
The psychological safety thing is it gets overused and gets a bad rap, but having an organization where it's actually safe to open your mouth and talk and ask questions occasionally looks stupid.
And fumble a little bit and have your peers support you with that and be happy for you when you learn stuff.
That's really important.
It's hard to do.
And there's great attention because, as a leader, you have to be disagreeable enough to do the hard things.
We're still creating an environment where people can open up and do that.
I would say I have mixed reviews on that topic.
Because once I find things that are wrong and people are doing the wrong thing, you know, I have to get to the bottom of it.
I'm going to close with...
For some people, it's never happened to them before.
Like, the people haven't really taken what apart.
You know, they got A's in college and they got good reviews and they rose to their Peter Principle and Competence point and all of a sudden they're doing something they're over their head and they don't know what to do about it.
They don't have a lot of practice.
So...
Yeah, it's a funny thing.
So I'm going to close with a question about your current venture.
You're now working with a company that does AI computing.
What do you hope to do that you can talk about?
Well, so I was an investor in this company when I first started, Tobisha Project, who's the founder.
He worked with me at AMD, and I always thought he was an especially smart guy, and I liked his approach to building AI computation.
I'm really intrigued about computers programmed by data.
I think it's more like how our brains work.
Our brains are really weird, right?
Because we think in this linear narrative, we have this little voice in our head, but we know we have 10 billion neurons, and they're collecting away, exchanging small amounts of brain transmitters and electrical pulses.
You know, it's bloody hilarious, the gap between what a neuron looks like and what a thought looks like.
And so there is a really interesting opportunity to make big AI computers that are actually really programmable.
So one of the things we're doing is we're building the software stack that lets you build the neural networks you want and then program and get the results you expect reasonably well, as opposed to having a very large army of people tweaking it.
So there's a bunch of architectural interesting things to do.
And then it's a startup, which, you know, we have chips at work.
We started production.
We're going to start selling them.
You know, there's a whole bunch of work to do on how to engage with customers.
And a lot of customers we're talking to are super smart.
There's all these, you know, AI swiper startups with really smart people that have some problems that, you know, basically if the computers are a million times faster, it'll be easier to solve.
So there's like a huge capacity gap.
On what they want to do.
So participating in that is fun.
I like that kind of thinking.
And your goal?
So you go into an organization, you have a goal for the chips?
What's your goal for this organization?
Oh, we're going to be successful selling AI computers to a large number of people.
You know, significantly better performance, better programmability and lower cost.
And there's a bunch of innovation work to do around that.
To make that really possible, doable.
Like the AI field is relatively new.
The computers that run AI today are relatively clunky.
And to me, you know, need a lot of work and refinement so that, you know, from the idea that you want to express in the program you're writing to the result you want better and cleaner.
Okay, so one final question.
For anyone who's listening who would like to pursue engineering as a career or, let's say, who wants to be successful within the confines of a big company, what advice do you have for people?
What have you learned that you can sum up?
Yeah, Lex Friedman asked me that.
First, you have to know yourself a bunch.
What are you good at?
You can't get really good at something you're not into.
And you're not good at it.
So you have to have some natural talent for it.
And then you have to really spend some time figuring out what you like.
I read this thing.
It was interesting.
People think of college as expanding their possibilities.
And the university itself has so many options.
You think that would expand your possibilities.
But once you pick one of them, and you study it for 4, 8, 10 years, you've narrowed your possibilities, right?
You're kind of stuck with your discipline, and you pick that 20.
Which I think is crazy, by the way.
Like, I think if you want to be an engineer, a good general engineering degree, like mechanical engineering or electrical engineering will give you thinking skill sets.
I'm not a huge fan of people getting PhDs unless they really, really know they love it.
Right?
And then take some jobs where, you know, there's an opportunity to do something for a year or two and then do something else.
Like, I worked my first job out of school.
It was a random job.
I worked on like five different projects in two years while I was there, fixing hardware, building something, debugging something.
I learned a lot in the digital.
I had many different roles, even though I set that company for 15 years.
I wrote programs, I did logic design, I did testing, I did lab work.
And so I got to see a lot of different things and get a feel for what I really liked.
And I worked with smart people that I had a lot to learn from.
Working hard when you're young is really useful.
You know, some people are like, well, you know, it's like the 10,000 hour problem.
And if you want to be an expert, you need to do that a couple different times on different things.
And you can't do it unless you really love it.
A friend of mine's wife said, what do they put in the water?
All you guys do is talk about work.
Yeah.
So you figure out what you're competent at.
Because you need that.
Figure out what you're interested in.
I mean, men and women seem to pick different occupations, not based on their competence, but on their interest.
And so interest is a very powerful motivating factor.
I've been in a lot of places where I have the best engineers for women.
So, you know...
You know, we know that numbers are less, but there's plenty of really great numbers.
Yeah, it certainly doesn't make it impossible.
It's just an indication of the, what would you call it, of the impact of interest as a phenomenon.
It's important as well as competence.
And so, a diverse range of experiences.
Don't over-index on something before you're really sure that that's something that you're really going to like or be great at.
Don't be afraid to ask stupid questions if you don't know what you're doing.
Yeah.
Try to work with good people.
Work in organizations where, like, if everybody hates the company you're working in, move somewhere else.
You want to work someplace where the energy is good, people are excited about what you're doing and why.
Like, sometimes you might be in a company that has a Something going wrong, but your group is going to change it.
That can be really fun.
But you need some camaraderie, some hope, right?
When the goal is clear.
Right.
So that's an adventure.
You have a destination and the camaraderie along the way.
Yeah.
And there's so many places doing so many wild things.
Being stuck in a company you don't like that's going nowhere for 10 years, man.
You don't have that many 10 years lasting your life.
You know, make sure you're actually getting, especially if you're getting a different experience.
Somebody said, you know, you have 10 years experience or one year of experience 10 times.
Right.
Now, sometimes you work on the same thing and you refine it and you become the expert.
But then you should feel like you're making progress on that expertise.
Right?
But if you're just kind of going through the motions over and over and doing the same thing.
Then it's time to fire yourself under those conditions.
If you're bored, you're not moving.
Right?
Engineering is not boring.
It's relatively exciting.
Yeah, I think that's actually a pretty good rule of thumb.
If you're bored, you're doing it wrong.
Yeah, something's wrong.
Yeah, something's wrong.
It's funny.
At AMD, we had this group that did tests, and it was kind of dysfunctional.
There was a couple of managers, and nobody liked it.
And at some level, the test engineering wasn't the hardest thing.
But I decided, that's stupid.
Why isn't our test group the best in the world?
We were organized around it.
We had a really great leader.
We had a good team.
I told them I wanted it to be really great.
And I told the engineers to stop complaining about it.
They had a problem come to me and we'll fix it.
Within two years, people come to me and say, man, the best guys are killing.
Yeah.
Yeah, they went above and beyond.
They made it something of value.
You know, it was great.
It was super fun.
That's a really good place to end.
Cool.
Thanks, Jim.
Hey, good to see you, man.
Much appreciated.
Thank you for taking the time.
All right.
We'll talk soon.
Cheers.
Export Selection