X Spaces with George Hotz (2022)
Transcript
0:00Co-hostLike diagram.
0:02George HotzNo, no, no. I mean, like, what do you mean by. Are you crazy?
0:05Co-hostNo, no, seriously, come on, buddy.
0:07Audience MemberCome on. Who.
0:08Elon MuskWho are you?
0:09George HotzWhat do you mean, who am I? I don't know. You gave me the mic.
0:13Audience MemberI got.
0:13George HotzNo, I didn't give you the mic. I'm doing the mic. And let's. Let's keep it. Let's keep it civil in my space.
0:20Audience MemberI mean.
0:24George HotzYes, like what? No, no, man.
0:25Co-hostYou're in charge of the servers and the programming.
0:27George HotzWhatever. Like what? Keep things technical in my space. Please. Take me from top to bottom. What does the stack look like right now?
0:36Co-hostWhat's so crazy about it?
0:38George HotzWhat's so abnormal about this stack versus
0:41Co-hostevery other large scale system on the planet?
0:45George HotzBuddy, come on, give it to me. All right, so first off, amazing. Wow.
0:49Elon MuskYou're a jackass.
0:51George HotzOkay, okay, okay. All right. No, no, no.
0:53Audience MemberI got.
0:54George HotzI got no credibility. Everybody, I got no idea. First off, my state.
1:00Audience MemberSybil, you know, like, you removed him from speakers.
1:05Elon MuskGood.
1:07George HotzCan I go into it?
1:08Audience MemberYeah.
1:09George HotzOkay. One of the biggest problems is that you can't run Twitter locally. So when I was at Facebook, you could run all of Facebook on a laptop. There is no way to run Facebook to run Twitter outside of a very bespoke configuration in a data center.
1:24George HotzAnd it makes it nearly impossible to build anything new on it.
1:30Elon MuskYeah, I mean, there's 20 million lines of Scala. Think about that. That's insane.
1:42Elon MuskDid you post that diagram?
1:45George HotzNo, I didn't post the diagram. But the joke of the diagram isn't that there's that many pieces, it's that when people think of a microservice, they think of 200 lines of code, not 200,000.
1:57George HotzYes, and it's 200,000 for most.
2:00Elon MuskIt should be trivial to do something like add View Count. It should be like literally another sort of line in a database. It should be trivial, but it's not trivial because of the insane complexity of the Twitter software stack.
2:18George HotzI disagree about view count. I think view count is actually inherently a hard feature. But, for example, right now there's a service that runs a model, and there's a service that calls that like, there's a RPC to run a model.
2:30George HotzBasically, at comma, it's one line. If you want to put the model inside the process at Twitter, it's two weeks of the best engineer's time.
2:38Audience MemberRight.
2:39George HotzThat's something that should actually be one line. I disagree that view counts should be easy, even on the best stack in the world. But there are some things that are the likes feature that I wanted to ship in search.
2:51George HotzShould have been five lines.
2:55Elon MuskYeah, I know. I. Well, so I think if you. We do count view counts, we just batch them. So if you reduce the requirement on view count to say like, look, we don't care about the view count being real time, it's okay that it be every hundred seconds instead of every one second.
3:14Elon MuskYou reduce the load by two orders of magnitude, obviously.
3:21Co-hostWell,
3:24George Hotzsort of. You definitely reduce the load two orders of magnitude for tweets that are viewed all the time. Right. But for tweets that are viewed every 200 seconds, it's the same load.
3:38George HotzIt's. If you're basically saying you're going to read every tweet from the table to get the views every hundred seconds, that's a lot of reads.
3:48Elon MuskLook, the point I'm trying to make here is view count is already counted. Like it's already a thing that is happening. But view count, currently the issue is that it's going to Google Cloud.
4:02Elon MuskInstead of the On Prem stuff, the view count is making a call to Google cut in batches at a rate that is towards the magnitude slower than what is needed to render it with the current pipe.
4:19Elon MuskThat's the actual issue. But if you then say, okay, look, we don't care about it being real time because something like a like is being updated about every second or so.
4:30Elon MuskBut if we said on your tweets.
4:32George HotzYes, on most people's tweets.
4:34Elon MuskNo, no, I'm just saying like the update frequency, it's near real time. But the point I'm trying to make is view counts are already counted. They're just being batched and being sent over a slowpipe to an analytics software on Google Cloud.
4:57Elon MuskWhereas the likes are using the On Prem database. But you could, that's like an arbitrary decision. It could use the On Prem database as well.
5:09George HotzYes.
5:10Elon MuskSo if it had not been an arbitrary decision to go off prem to Google Cloud, then it would be trivial to add likes.
5:22George HotzIt would be considerably easier.
5:25Elon MuskIt wouldn't be onerous to add views. So if you're not like too precise about exactly how many views there were, if you like, let it be a little, let it float a little bit and you occasionally drop a view count that.
5:39Elon MuskSo it's because like, we're not like trying to balance a bank book here, then I think views actually would be relatively easy
5:51George Hotzif views were in the same database as likes. Yes, it would be easy.
5:55Elon MuskYeah, that's what I'm saying. And it should be it's absurd that views are on a totally different database in Google Cloud instead of the on PREM
6:06Audience Membersystem.
6:07Elon MuskSo anyway, that's sort of an unnecessary complexity. I mean the Twitter stack right now is like basically combining the difficulties of three different worlds, which is running a complex real time on PREM database that's in three data centers in Atlanta,
6:30Audience Memberone
6:30Elon Muskyear in Oregon and one in Sacramento. Sacramento is like possibly the worst place to have a data center because it's hot. And then like the summer it got so hot that the data center failed because the H Vac didn't work.
6:46Elon MuskBut then in addition to that, you've
6:47George Hotzgot
6:50Elon Muskalso using aws, so using Amazon and using Google Cloud. So all three are required to operate Twitter fleets.
6:59George HotzFleets uses aws. They built all the fleets in aws. Yeah, I read the Google Docs for why too, for why they put fleets in aws. Basically they couldn't get enough provisioning for the internal data center.
7:17Elon MuskYeah, I mean like Sam Pilar has been helping out and like I think Sam was saying there was like some pretty nutty amount of data storage is actually analytics for app versions that don't exist anymore.
7:35Elon MuskYeah, like I think it was like 15%, maybe 20% is like, was literally logging data from apps that don't exist. This is like, okay, that's. Wow. Anyway, there's a lot of crazy stuff going on.
7:55George Hotz25% of the CPUs are running the ads prediction model. It's the tiniest model those CPUs. Only 30% is actually running the model. 70% is serializing and deserializing the data into four different formats.
8:11Elon MuskYeah, and here's the crazy thing. So which is like really kind of mind blowing to think about it, but it took me a few weeks to actually even wrap my head around it.
8:24Elon MuskThe Google Twitter's advertising is actually for the most part irrelevance optimizing. And so it's like, wait a second, like, what do you mean irrelevance optimizing? Well, when I ask people who have used Twitter for a while, it's like, how many things have you bought on Twitter?
8:45Elon MuskThe answer is usually nothing. But it's like, well, how many things did you buy on Instagram? Oh, I bought a whole bunch of stuff last week. It's like, well, don't you think it's interesting that you wouldn't have accidentally bought something on Twitter after all this time?
9:00Elon MuskAnd the reason you haven't is because the ads are maximizing irrelevance. I feel like I'm stuck in a Douglas Adams novel. It's like the spaceship was powered by maximum improbability.
9:14Elon MuskAnd so you're like, wait a second, why on earth would you maximize irrelevance? Well, because the team was told to maximize the impression, the number of advertising impressions. So what happens if you try to maximize advertising impressions?
9:30Elon MuskWell, you actually are going to maximize irrelevance because if you. If somebody is spending five minutes on Twitter and two of those five minutes are spent clicking on an ad, you've now lost 40% of your impressions.
9:45Elon MuskLike head exploding emoji.
9:48George HotzYeah, Dwell time is probably a much better signal for ads than impressions. I mean, this is what made TikTok. What makes TikTok. What makes TikTok so good? They have dwell time on all the videos.
10:03George HotzThe time you spend on an ad is probably better than whether you saw the ad or not, though I somewhat disagree. In the limit, if you had an ideal ML model trying to maximize impressions, I think you'd end up with someone who just scrolls Twitter and looks at the ads all day.
10:20Elon MuskLook, I'm just saying that, like, when I was going through this with some of the engineers a few weeks ago, it took me a moment. I just. It was so insane that I couldn't comprehend it.
10:34Elon MuskIt took me a while to actually understand that because I was saying that, like, no, we want people to click on the ads. That's the whole point of having an ad is so that you might buy something.
10:44Elon MuskThat's the reason for advertising. And so, you know, an ad that is highly relevant is content. An ad that is irrelevant is spam. So we want ads that are as relevant to people's needs as possible.
11:03Elon MuskAnd yet I see many ads that, in fact, most ads are not relevant. And when I ask people, what are the concerns about Twitter, it's like, well, Twitter keeps showing me the same ad that I don't care about all the time.
11:16Elon MuskI'm like, well, that seems crazy. But then I found the reason, to maximize impressions instead of maximizing actual clicks and buying things, which is what they should be doing.
11:31George HotzWhen you run an ad campaign on Twitter, you can choose, I think. I'm not really sure what the back end does. I heard the backend tries to maximize money, but you can choose whether you want to run an impressions campaign or whether you want to run a clicks campaign.
11:43Elon MuskYes, things have changed recently. So we have, in the, like, you know, basically since the acquisition, I was like, no, guys, we need to maximize relevance. And there actually has been good work in the direction of relevance optimization.
12:00Co-hostSo,
12:03Elon Muskwhich, you know, so that, you know, I don't mean to be sort of all negative here, but there's a lot of, you know, there has been good work recently in the direction of actually having ads that are relevant for questions on this.
12:19Co-hostI am super nerdy about ad tech stuff. I'm super curious how Twitter can compete with like Google and Facebook just in terms of their identity network and how well they have like bucketed groups of interest to target.
12:29Co-hostLike, how does Twitter compete?
12:33Elon MuskActually, I think Twitter can actually do arguably better for someone that has used Twitter for some period of time. So if you have used Twitter for, let's say, I don't know, a few weeks, and if you have like, you're several hours in and it's, it's the, the signal, the amount of signal that you have is high.
12:55Elon MuskYou know, what people, what, what, what topics people followed, what, what, what accounts people followed, what they liked, what they, you can say, you know, certainly view count. There's.
13:09Elon MuskAnd a bunch of the stuff is actually used at Twitter for recommended tweets, what is called sometimes the home timeline. You know, if you like, there's like a little stars icon in the upper right of the Twitter interface and it'll say like, for your or latest.
13:26Elon MuskSo for your is recommended tweets. A lot of people actually don't even realize that the upper right stars icon is a way that you can switch between recommended tweets and accounts that you actually follow.
13:41Elon MuskI bet a lot of people on this call currently do not know that that is what that stars icon is.
13:48George HotzSo.
13:50Elon MuskBut the recommended tweets are actually pretty good.
13:57George HotzHome ML is much better than adsml. Home ML is using much larger models, many more features. The adsml is very rudimentary.
14:05Elon MuskYes. But essentially they should be going through the same pipeline.
14:09George HotzYeah.
14:13Elon MuskIf Twitter can accurately recommend tweets to you that you don't explicitly follow, it can by the same token, also show ads that are relevant to you.
14:28Co-hostSo the way that a lot of platforms do ad sales and a lot of purchasers expect ads is, isn't necessarily, I'm going to give you these ads and I trust you to serve them to the right people.
14:37Co-hostIt's usually almost the opposite where it's, I want to target this group of people in this region who went to a Jiffy Lube in the last 72 days. That level of micro targeting is something a lot of ad buyers now expect.
14:50Co-hostAnd like, it's a huge part of how CPMs can get so high on platforms.
14:54George HotzLike YouTube, regardless of whether they're targeted or not. If your goal is C cpms, you should have a very good idea that someone's going to click on an ad before you show it to them.
15:04George HotzAnd you can do this with GoodML,
15:06Co-hostbut sometimes these advertisers will serve entirely different versions of the same app.
15:10George HotzTurn this guy off, please. He's just talking nonsense.
15:12Elon MuskThanks.
15:14George HotzSo
15:17Elon MuskI've actually talked to the advertisers. I talked to some major advertisers just today, this morning, and they were looking for, as you would expect, return on investment. So that if they spend a certain amount of money, are they going to get a return in excess of the amount of money spent?
15:34Elon MuskQuite rational, not arbitrary or random. So, and I won't say who it is, but there are major Advertisers who spent $60 million on Twitter this year. And they were like, we calculated the roi.
15:46Elon MuskActually, another company helped them calculate the ROI and Twitter was the lowest. And this is why they're having difficulty spending additional money on Twitter, because they're literally looking at the lowest roi.
15:59Elon MuskI'm like, yeah, okay, well, that's quite sensible. And then they were like, well, you know, when this particular product is discussed in replies, wouldn't it make sense to at that point provide an ad that perhaps since you are discussing this product, perhaps you would like to buy it?
16:18Elon MuskI'm like, yes, that would make total sense. We should of course do that. They were like, yeah, that would be awesome. If you do that, then we will be glad to spend more advertising money.
16:27Elon MuskThey were very rational and not arbitrary and random.
16:31George HotzI'll stand up for Theo, worked at Twitch. He's provided some good stuff on this space so far. While in the limit, I agree with you, it would require retraining advertisers.
16:42George HotzAnd I'm not saying you can't do it, but advertisers have come to expect a lot of this micro targeting stuff. Facebook pioneered a lot of it.
16:49Elon MuskYeah, look, I think at the end of the day, if you can say like, if you can make a clear equivalence, like, just say like, for. If you spend amount X to generate profit, Y, provided Y is greater than X, you should continue.
17:04Elon MuskAnd provided if you've got good analytics that demonstrate that whoever the advertiser is, you will succeed. And that is not currently the case for Twitter, but it will be.
17:14George HotzDo you want to fight every battle or do you want to like, Like. I mean, is this the battle Twitter should be fighting or should we just provide the most standard boring ads interface and just make it good.
17:25George HotzLike Instagram's.
17:28Elon MuskInstagram has an excellent advertising interface as, as literally talk to anyone and they probably, if they're using Instagram at all, they probably bought something.
17:37George HotzLast week I bought stuff off Instagram ads. Yeah, Facebook has GPUs in the data center too. They're running decent sized models on people.
17:44Elon MuskYeah, it's, it's. I don't think we're trying to split the atom here. You know, we're just literally trying to show people ads for something they might actually want to buy.
17:54Elon MuskYou know, this is not the Manhattan Project, even though the database weirdly is called Manhattan.
17:59Audience MemberSo,
18:02Elon Muskso it's just, I'm just, this is common sense stuff. So like, yeah, like sound like this is a monumentally difficult to implement. It will get implemented and it will get done fast.
18:13George HotzElon, would it be possible to offer
18:15Elon Musklike a more expensive $100 a month version with zero advertisements?
18:20George HotzI mean 1% of people might be
18:21Elon Muskwilling to pay that. Yeah, absolutely. I think it's a no brainer. This is just a question of what complexity do we add when. Okay, so like it's difficult to, frankly it's difficult to overstate the difficulty of adding stuff to the adding features.
18:44Elon MuskFeatures that should be easy to add are not easy to add because the stack complexity is extremely complex.
18:57George HotzSo can we do refactors? Forgot.
18:59Elon MuskSo we've made some simplifications even in the last few weeks, but this is, it will explode your mind to go through the entire stack. It's like wow.
19:15George HotzWell, you have to sort of make a trade off between features and refactoring.
19:20Co-hostRight.
19:20George HotzLike if you have X amount of engineering team, you can decide how you want to allocate that.
19:26Elon MuskYeah, I mean basically we're going to make you know, a series of I think significant short term product improvements. But then I think we'll need to rewrite in order to have a high velocity of improvement.
19:45Elon MuskUnfortunately I think a rewrite is needed. Like say like look back at old school Apple, you know, when they had their old operating system and then Jobs came along with osx, you know, it was a new stack.
20:03Elon MuskIt wasn't like let's like you know, fool around with the old Apple os. It was like he just took the Next OS and adapted that to Apple and said yeah, your old software just doesn't work anymore.
20:16Elon MuskToo bad.
20:17George HotzBut an operating system doesn't run at scale. Like this is doable, but it'd be, I think it'd be pretty hard to do with continuity to roll out this New stack, promising, no downtime on a clean rewrite.
20:32George HotzSeems very, very hard. Yeah.
20:36Elon MuskNonetheless, I think that's what'll be needed long term.
20:41Audience MemberI actually have a question about the stack.
20:44Elon MuskSure.
20:46Audience MemberSo you brought up the 20 million lines of Scala before, and you mentioned that the microservices have a lot of the complexity of the Twitter system. So I think microservices to like.
20:59Audience MemberI believe microservices for complexity, like a system at that scale won't really microservices land.
21:09George HotzSorry, you're breaking up now.
21:18George HotzYou think microservice yes or microservices no.
21:20Audience MemberI'm fine with the microservices. My concern is the infrastructure or the tech stack that you guys are using, because instead of using commercial or open source or any standardized products, you guys have inbuilt or in house, your own implementations of everything.
21:38Audience MemberSure, you're using Kafka, but your RPC layer is Thrift, which is developed internally. Your database is Manhattan.
21:45George HotzWait, wait, wait, wait, wait, wait. Thrift is a Facebook product?
21:48Audience MemberWhich one?
21:51George HotzThrift is a Facebook product.
21:54Audience MemberMy point is that
21:57Elon Muskwe got the TFE, we got the GraphQL, we got the Product Mixer, we got the Ranker, we got the Manhattan database, we got the Strato, we got the Tweety Pie, we got the Gizmo Duck, we got the Twitter memcache, and then wait till
22:11George Hotzyou find out they're all built on top of Finatra and Finagle, which are entirely custom Twitter open source Scala projects.
22:18Audience MemberYeah, I get. My point is that if you have all of this custom and not like super standardized products, when 80% of your workforce happens to leave now, how much more difficult is it for new people or like even existing people to work with that infrastructure and be able to be able to iterate on it, much less like refactor it?
22:41George HotzI mean, I think I could do it, but I think it would take me six months.
22:47Elon MuskHilarious.
22:49Audience MemberSo, George, you mentioned that at Facebook everything could run locally. I've worked at companies where we had a dedicated team just for developer experience, and their job was literally to Dockerize and containerize everything and just make the app run so people that can code, can code.
23:10Audience MemberAnd it just sounds like Twitter's lacking that.
23:13George HotzYeah, you want to hire those people for Twitter. I think Twitter's definitely lacking that. So, I mean, Elon, how far, how
23:21Audience Memberfar are you guys away from having an engineer, like a new engineer on day one? Just, you know, walk in, go through quick orientation, and then get like a single dev instance running and just being able to submit a PR on the same day, or maybe the same.
23:35George HotzA dev instance. No, Twitter doesn't believe in that. We have produced and sometimes.
23:40Audience MemberOr local. Any kind of local.
23:42George HotzIt is local.
23:45Elon MuskNo, I think to your point, it takes a while. There's so much custom stuff that just understanding where everything is and how it all connects, I think is a matter takes at least a few weeks.
24:02Elon MuskSo I think it would be difficult to be productive, you know, without at least a month of intense effort. I mean, that. Yeah. And that would be like, you're really good at software, I'd say, if you can do something useful in a month.
24:18George HotzI think I understand most of it, but I think a lot of the problems are not a question of understanding. I think they're of just toolings and workflows. The fact that I can't stand up Twitter anywhere except for the data center, for example, some services, when you develop them, you have to.
24:38George HotzThey pack. It's a 1.6 gigabyte binary. It has to upload to the data center. That takes 20 minutes and then you unpack it, and then, okay, 30 minutes later, you've tested your one line change and.
24:49George HotzOh, you made a typo.
24:52Elon MuskYeah, I'm just saying, basically, to George's point, it's a good point, is that.
25:04Elon MuskThere's so much complexity and so much interaction across so many different levels that to make progress is very difficult. Progress is being made, but it is very difficult to make progress.
25:15Elon MuskWhen you touch, like one element of the system, it affects like 17 other elements. So, you know, like one argument in favor of, say, a whole bunch of microservices is if they are truly independent, you can add additional microservices without affecting the rest of the system.
25:32Elon MuskBut that is not the case at Twitter. Like I said, we have this challenge of trying to add view count to tweets, which I think is really going to make the site pop, because you'll see just how many people actually view your tweets.
25:49Elon MuskThis is completely normal for video. So if you look at even a video on Twitter, it will actually show you how many views the video has gotten. And that gives you some indication of the popularity of the video, how interested are people in the video.
26:04Elon MuskSo Twitter has views, poor videos, but does not have views for tweets. So you can tweet something and think, man, did anyone. Was it like a tweet fell in the forest and nobody heard it?
26:18Elon MuskDid anyone see my tweet? Actually, lots of people saw your tweet, but you don't know, because they didn't want to put themselves out there by. By actually liking it or retweeting or replying.
26:29Elon MuskSo the site will pop in a big way. Just showing view count of tweets and people realize, holy smoke. A lot of people actually seeing my tweet. That's cool.
26:41Elon MuskAnd then, like I said earlier, we need to add functionality where you can like a tweet without it becoming. Without advertising the fact that you liked it to the world or being able to say that maybe you didn't like it in a.
26:53Elon MuskI love that that thing happened. Like the earthquake example. You don't love earthquakes, but it's noteworthy, certainly. And it's putting a heart next to an earthquake. That's weird. So you want to have more fidelity in how you.
27:13Elon MuskWhat you think of a tweet, and you want to have the option of making that how you feel about that tweet, public or not public. That's like, pretty obvious stuff.
27:23Elon MuskBut making those changes to the existing Twitter system is very difficult.
27:32George HotzWhy not refactor and make the changes later? Why do they need to be done now?
27:38Elon MuskI think we need to make some improvement in the near term. And I think we can make some improvement in the near term.
27:47George HotzSo I ran Git log this morning and I looked at how many commits are going into the code base and what they actually are. Huge green diffs
27:57Elon Muskfor view count or for something else?
27:59George HotzAll sorts of things. Maybe 10% of them are view count related.
28:06Elon MuskOkay.
28:08George HotzBut all of these other ones are still. They're still green diffs. There are a lot of people writing a lot of code today. Timeline Mixer was deprecated for Home Mixer.
28:19George HotzGreat. It's deprecated for Home Mixer. Was Timeline Mixer deleted? Oh, no. Now we have both Timeline Mixer and Home Mixer.
28:29Elon MuskThey're running simultaneously.
28:30George HotzYes.
28:33Elon MuskWell, it's not very meaningful to deprecate something if it's still running, obviously.
28:37George HotzYeah, exactly what I'm saying. Right. Nothing gets actually deprecated to Twitter. So Twitter has three APIs, and you can find this on public Twitter if you go to Chrome Inspector.
28:46George HotzTwitter has a 1.1 API, a 2.0 API, a GraphQL API. It also has 1.1 developer and 2.0 developer. It has five APIs that I could find. And then there's other ones people have talked about too, that I don't know how to find because I don't have Android Emulator on my computer.
29:01George HotzThis stuff needs to be cleaned up and no one's doing it.
29:04Audience MemberIsn't it, it's pretty common.
29:06George HotzHey guys, do you mind if I ask a.
29:11Audience MemberSure.
29:12George HotzYeah. So I just wanted to know.
29:14Audience MemberElon.
29:14George HotzSo obviously, as you know, I think with the, the Tesla, you know, and
29:18Elon Muskwhat you guys do on the autopilot,
29:20George Hotzyou guys are recreating and redoing a lot of the code. And it sounds like with Twitter it sounds like it's kind of a nightmare.
29:27Elon MuskSo how are you prioritizing what you fix, what you're going to implement, like the views, how are you stack ranking everything, especially if things are taking a lot of extra time to get done.
29:43Elon MuskWell, bear in mind like, you know, the acquisition only closed like whatever, like six weeks ago, 5ish. So this is probably like the biggest amount of change that has happened in an acquisition possibly in history.
29:57Elon MuskYou know, we're at a little over 2,000 people. We were at close to 8,000 people. The site is still running as evidenced by us being on it. So, you know, this is reasonably good progress.
30:10Elon MuskThe most essential thing to be done at Twitter was to cut the burn rate so that Twitter does not just go bankrupt next year. That was like the, you know, task number one.
30:24Elon MuskAnd there's still work to do on that front, but it's mostly done. Then there's also, it's also important to move the revenue base from being essentially entirely advertiser dependent on advertisers, and not just dependent on advertisers, but dependent on fuzzy brand advertising as opposed to advertising that has a clear roi.
30:48Elon MuskAnd as we're heading into what appears to be a recession, the first thing that companies will cut is discretionary brand advertising. So it is important for Twitter to therefore have a subscriber base.
31:02Elon MuskThis is the reason for the Twitter Verified. The reason we're able to make progress with Twitter Verified is because it makes use of a product that sort of was pre existing but not very useful, called Twitter Blue.
31:18Elon MuskSo being able to move to a. Have subscriber revenue be a meaningful part of Twitter revenue is going to be essential for the prosperity survival of Twitter. So that's why that's actually the number one priority is getting the Twitter Verified to work and then also getting Twitter verification for businesses and organizations to work so that you're starting to see that rollout now where an organization can actually identify who belongs to that organization.
31:55Elon MuskSo like who is actually, which accounts actually belong to Disney and can speak on behalf of Disney or on behalf of say Stanford or any organization. Like if someone says they're affiliated with an organization, how do you know that besides them Saying it in their bio.
32:12Elon MuskWell, we're now linking that on the back end so that someone organizations can say, yes, these Twitter handles actually do belong to the company and are authenticated. And this is also, you know, obviously an avenue for, you know, business revenue, business to business revenue.
32:32Elon MuskSo the subscriber revenue and sort of back end business revenue is really important as we see a rapid decline in brand advertising.
32:44George HotzBut these are all features and they're all at the expense and refactors.
32:50Elon MuskYeah, no, they are.
32:54George HotzOkay, thank you for acknowledging.
32:55Elon MuskI think we won't get to the refactor if we don't make these changes.
33:00Audience MemberWhy?
33:02Elon MuskBecause we used to be bankrupt.
33:05George HotzWhat's the Runway?
33:14Audience MemberCould I jump in with a quick question to cut the Runway silence there?
33:21George HotzYeah, sure.
33:22Audience MemberYeah. So I feel like, you know, Elon, you're our philosopher king. Plato would be very proud of you. You know, you're extremely technically competent and, you know, you, I think, really understand how to drive things in a good direction.
33:43Audience MemberBut, you know, I really love how you're also leaning into the voice of the people and letting them have a say as well. But you can't have a million backseat drivers.
33:56Audience MemberSo I guess my question to you is, how do you integrate the voice of the people when there are a lot of people that just don't have the right information to make decisions?
34:07Audience MemberAnd how do you create the right governance systems to really find the signal from all of the noise to get Twitter to become something more like a republic rather than a dictatorship, you know?
34:24Audience MemberCurious what you think.
34:30Elon MuskYeah, well, I mean, you know, in the, in the relatively near term, like, these things are all kind of moot if Twitter is already into the ground financially. So I was thinking about the Runway question, like just to sort of, you know, give people some perspective here.
34:49Elon MuskTwitter was tracking to spend about $5 billion next year.
34:55George HotzWe needed the trip to Disneyland.
34:57Elon MuskYeah, exactly. Then because of the nature of this transaction, where there's $12.5 billion of debt and the Fed rate has been going crazy, there's on the order of a billion and a half ish of debt servicing that's required.
35:14Elon MuskSo we're talking about like net cash outflow if you didn't make any changes of on the order of like 6, 6.5 billion next year with revenue probably tracking to 3.
35:29Elon MuskSo that's like a negative cash flow situation of $3 billion a year. Not good since Twitter has 1 billion in cash. So that's why I spent the last five weeks cutting costs like crazy.
35:49George HotzI think we also got to get the advertisers.
35:52Elon MuskYeah, yeah, we do need to get the advertisers. But like I said, I've spoken to a number of the advertisers. They're. Their requests are not fuzzy or irrational or anything.
36:06Elon MuskThey're like, quite reasonable. They're like, just show us an ROI that makes sense. And I'm like, yes, I agree. If I were in their position, I would also want an ROI that makes sense.
36:17Elon MuskEspecially when headed into difficult economic times where these questions are asked a lot more than in prosperous times. In prosperous times, there is plenty of budget for advertising and you can get away with unclear roi.
36:35Elon MuskBut when times are tough, then the hard questions of return on investment are asked. And when you do not have a clear answer, then advertisers don't want to advertise because they're being sane.
36:50Elon MuskThat's the main issue. So this is important. So now with the changes that we're making here on massively reducing the burn rate and building subscriber revenue, I now think that Twitter will in fact be okay next year.
37:10Elon MuskI think we will be about, hopefully, sort of roughly cash flow break even. That's what I expect for next year. But this will be quite. This will be difficult, but I think that's what will happen is roughly cash flow break even, as opposed to minus 3 billion ish on worth 1 billion of cash, which would be therefore dead.
37:35Elon MuskSo that's the reason for my actions. They may seem sometimes spurious or odd or whatever. It's because this is not. We have an emergency fired roll on our hands.
37:51Elon MuskThat's the reason. Not because I am like naturally capricious or at least aspirationally, I'm not naturally capricious. So. But if you sort of, if you say you're looking at it from my standpoint and you're saying, okay, wow, this company is like, like basically you're in a plane that is headed towards the ground at high speed with the engines on fire and the controls don't work.
38:25George HotzI use the same analogy at comma, can we afford good food?
38:32Elon MuskYeah, well,
38:36George Hotzit's really important to me.
38:39Elon MuskNo, I think we can afford good food. The madness of, for example, the headquarters food budget was that it was $13 million a year of fixed rate, no matter how many people showed up.
38:53Elon MuskSo if the normal meal price would have been, if the building was fully occupied, $20 for actually quite a good lunch, the food was good. But if you drop to 5% occupancy, you've now multiplied the effective meal cost by 20 and that's where you get a $400 lunch, literally.
39:16George HotzAbsolutely. The food was great. I loved the chia seed pudding and the smoothies in the morning. But yeah, you got to tell everyone to come into the office. It's crazy to me that people think they can have a job where they don't show up to an office.
39:28Elon MuskRight. And it seems like a logical thing if, if you have a building where there's literally 5% occupancy that you would seek to renegotiate the food budget and not, you know, have it be 20 times more than it should be.
39:45Elon MuskBut that was literally the case. So I, you know, the food's a
39:54George Hotzlot worse now than it was. A lot worse.
39:58Audience MemberYeah.
39:58Elon MuskWell, we can definitely improve that. That's not like, I think some impossible thing to overcome.
40:08Audience MemberSo
40:11Elon Muskthis is not, you know, it's not going to break the bank. The issue is more you can't pay like $13 million a year for a mostly empty building for lunch and not even including dinner.
40:25George HotzI could save $13 million by switching ads prediction from TF serving to ONNX runtime alone.
40:33Elon MuskYeah, I'm saying we can totally have a good lunch. We're not in like lunch purgatory for the rest of time.
40:46Elon MuskWe may have gone too far is what I'm saying. I admit we may have gone too far in cost cutting with respect to the lunch that, that could be the case.
40:57Elon MuskAnd so we'll have to sort of. We may have overcorrected in that regard and we can fix it. This is not an insurmountable problem. So.
41:08George HotzYeah, so I'm like, refactors food. How about free speech?
41:16Elon MuskWell, I mean, free speech, let me tell you that if somebody's got to pay the servers, okay, so the speech has got to cost at least $8 because otherwise how do we pay the friggin server bill?
41:30Elon MuskYou know, there's like a billion and a half ish, you know, of all in server related costs and you know, we're trying to get that lower, but it's a lot, it's like not trivial and a minimum, you know, like there's like somebody's got to pay the bills.
41:52Elon MuskSo advertising for sure, but there's some degree of subscriptions. And then on the, on the speech front, like my general philosophy on that is that we should hew close to the law in any given country, you know, so
42:13George Hotzyeah, Elon, that makes me so happy. If that's actually what the policy is. I'm upset when I see Paul Graham get banned for mentioning Mastodon.
42:23Elon MuskYeah, that was a mistake.
42:24George HotzOkay.
42:29George HotzI mean, yeah, I think there's a far cry, you know, look, you can make arguments for Kanye west, you can make arguments for Elon Jack, but you can't make an argument for Paul Gray.
42:37Elon MuskNo, I agree, that was a mistake.
42:39George HotzOkay? Can people post Mastodon now?
42:42Elon MuskYeah, Fucking post Mastodon all goddamn day long. I don't care.
42:46George HotzYeah, because we're gonna build something way better than Mastodon, right? Like, come on, how hard could it be to beat these guys?
42:52Elon MuskYou know, from an evolutionary standpoint, how'd that work out for the Mastodons?
42:58George HotzI don't know what a mastodon was.
43:03Audience MemberI still feel like there's something to be said about like Open source projects though. You know, like you have Linux that you know, out competed windows. Like you like big, big networks of like armies of volunteers can maintain, you know, pretty complex things like yeah, like sure, maybe Linux is run a little bit like a monarchy, like George said earlier in the space.
43:27Audience MemberBut you know, like I do think that there's something to be said about Open source software in general, you know, and it's, it's like anti fragility.
43:39George HotzI mean Linux is absolutely run like a monarchy, but Linux is not Twitter. Linux has a large code base because it needs to support many different device drivers and many different devices.
43:50George HotzTwitter needs to support basic core functionality and I think open sourcing it. There's no value in the code, right? The operating system has value in the code. The value of Twitter's in the network,
44:02Audience MemberI suppose, but there's still like, you know, composability arguments, right? Like you guys could reuse and other people could build on top of.
44:11George HotzIf Twitter, if Twitter was open source tomorrow, what would you build?
44:18Elon MuskHonestly, I think we should just post the source somewhere. Frankly.
44:22Audience MemberThat would be sick, Elon.
44:25George HotzNo one would ever get it to run.
44:28Elon MuskWell, I mean listen, that would just, I think, finally explain.
44:35Audience MemberDo it.
44:35Elon MuskThat is explained with words because you can just look at it.
44:40George HotzDump a tarball Twitter source. It's unbelievable.
44:44Audience MemberSo I have two ideas that I would use.
44:47George HotzPost it all.
44:47Elon Musk20 million lines of Scala and you can just watch and read to your heart's content because.
44:54Co-hostWow.
44:55George HotzI think it's actually, I think those are just the used lines.
44:58Elon MuskRound numbers, round numbers, you know.
45:00George HotzYep, yep.
45:00Elon MuskWe'll take a million.
45:02George HotzOh yeah, well that's just Scala, you know, we have a lot of Python as well.
45:07Elon MuskYeah, but actually there's a lot of,
45:10Audience Membera lot of most open source code quality is like really poor and On Linux, most of the contributors are paid by like intel and other big firms. So it's not really a fair comparison at all.
45:24Audience MemberYeah, sure, but you still have like composability though, right? Like a lot of people can build on top of what they're building and then that makes like, like, I don't know.
45:33George HotzYeah, yeah, I mean that's absolutely true,
45:35Audience Memberbut who's gonna sponsor it? And no one's gonna sponsor open source engineers unless.
45:40Co-hostAnd composability is not inherent to open source. Composability is a thing that takes a lot of effort and money and time as well. Like there's a lot of code bases that are open source that aren't composable at all.
45:49Co-hostThe theoretical potential is there, but it's not like a thing almost always.
45:54George HotzNo, no one is going to volunteer to your open source project to come in and do meaningful refactors that reduce the code. They're going to write a feature, be like, please merge it so I don't have to maintain it.
46:03George HotzAnd then.
46:04Elon MuskYeah, that sounds about right. Still though, I have a quick idea.
46:12George HotzElon, is there any way to like
46:13Elon Muskgamify getting more people to use Twitter?
46:16George HotzBecause I think a lot of people hear about Twitter in the news, they
46:18Elon Muskdon't really understand how to use it or get the app and use it. But maybe if there's a way to encourage other people to show people how to use it, it could help get more daily active users eventually.
46:30Audience MemberYeah,
46:32Elon Muskwell, so daily active users have been growing reasonably well, especially if you count real users and not like bots and trolls. So I mean like one of the super crazy things that I discovered, I only discovered this like basically about 10 days ago, was that Twitter was being scammed to the tune of $60 million a year for SMS texts, not counting North America.
47:05Elon MuskSo I'm like, what do you mean? I was like, what do you mean? Like there's $60 million of SMS text.
47:09George HotzYeah.
47:10Elon MuskTwitter is paying telcos basically. There are telcos who are not being super honest out there in other parts of the world who were basically gaming the system and running like two factor authentication SMS techs over and over again and just creating a zillion bot accounts to literally run up the tab so that Twitter would SMS text them and Twitter would just pay them millions of dollars without even asking about it.
47:38Elon MuskAnd now this created some problems because I said, well, we got to stop that. I said, like, you know, cut off any telco that's got fraud above 10% now that did cause some havoc in many parts of the world.
47:48Elon MuskAnd then people were like imputing like bad motives to it, but it was literally I just said to the team, hey, cut off any telco who's where. The fraud's above 10%.
47:56Elon MuskThat turned out to be 390 telcos. I'm like, dan, that's hot. Oh my gosh. Outside of North America, you know, so I'm like, okay, I didn't know there were that many telcos in existence, but turns out there's some.
48:10Elon MuskYou know, you get to like the fourth tier telco in Bilo, Russia and you're like, you got some sketchy things going on.
48:17George HotzNow you have people complaining that they can't log into Twitter because their SMS doesn't work.
48:22Elon MuskYes. And so like now we're going back to the telcos and saying, listen, if you, if you stop scamming us, then we will gladly pay you some amount of money in SMS to texts, but you can't turn a blind eye to relentless bogus SMS texts.
48:37Elon MuskSo then we're having that conversation with the telcos and then we can turn them back on and say, listen, you just be reasonable, we'll take 10% of fraud, it's fine.
48:47Elon MuskBut we're not going to take 90% fraud. So we're not going to spend $60 million a year on literally SMS texts outside of North America. So that. So a bunch of those users were just fake and like literally for the purpose of running up Twitter's tab on SMS texts.
49:08George HotzKid you not.
49:09Elon MuskThis is insane. So now, but to your point of making Twitter easier to use, like the obvious move I think is like, if you download the Twitter app or you go to Twitter, it should, it should put you into the search page and immediately show you like what's trending.
49:32Elon MuskSo it's like, as opposed to saying, hey, you need to download the app, you need to install the app, you need to give us a username, you need to give us a friggin phone number and making it like an escape room to open a Twitter account.
49:46Elon MuskIt's like we're going to give you a bunch of math puzzles basically to open a Twitter account. And then once you're in the Twitter account, you're sitting there with nothing to do.
49:56Elon MuskThis is insane. It's not clear to me you should actually have a Twitter account at all. Unless you need to make a write action. Unless you need to write something.
50:07Elon MuskSo if you want to like or retweet or tweet or something like that, okay, at that point, yes, you need an account.
50:16George HotzThen why did people put the pop up back? They said you said they could do it.
50:20Elon MuskWhich pop up?
50:22George HotzThe pop up that I emailed you the first week that I wanted to remove. The one that when you scroll on Twitter when you're logged out.
50:30Elon MuskI didn't say we should put the. Okay, that's the miscommunication.
50:33George HotzOh great, let's get rid of it again. Because I got rid of it entirely. I got rid of it entirely. They added it back and they messed up adding it back so it wasn't dismissible and then they fixed that five days later.
50:44George HotzBut like they didn't mean to make it non dismissible and it was non dismissible and I'd love to remove it altogether. Yeah.
50:50Elon MuskIt's not clear to me why unless there's some reason that I'm not aware of, we should not be prohibiting just a read only scroll.
51:01George HotzNo, there's no reason except that people are scared to lose mdao.
51:11George HotzYeah.
51:12Elon MuskThis optimization of MDAO is like I think creating a bizarre incentive structure that doesn't make sense.
51:19George HotzIt's endemic in the organization. They love mdao.
51:24Elon MuskYeah. I mean the thing that actually matters is like true user minutes, not like fake user minutes. But we're confident this is a real user. We're confident this user actually spent time on the app as measured by screen time on the phone, which is since the phone is the primary user use case and then, and then to get like another layer deeper is what we said the thing to optimize is unregretted user minutes.
51:55Elon MuskSo I've heard like a lot of people say like, hey, I spent two hours on TikTok, but I regret those two hours. I'm not saying I want to trash TikTok because it's quite compelling to a lot of people, but.
52:06Elon MuskBut I do often hear that they. People regret the amount of time they spent spent on it. So I think the thing we'd want to optimize for is unregretted true human user minutes.
52:16Elon MuskThat's the thing to optimize for, not mdao. And then with respect to advertising, as I said earlier, you want advertising that is as close to content as possible where it's something where it's a product or service that you want when you want it, then that's actually useful.
52:34Elon MuskIt's actually at that point becomes indistinguishable from content. So these are the things to aim for. We're going to get rid of this MDOT bullshit and we're going to get.
52:44George HotzYeah, let's get rid of all the pop ups. Let's get rid of when you click likes and quote tweets. Because fundamentally what these things are doing is they're lowering total minutes on the website.
52:52George HotzPeople see that pop up and they
52:54Elon Muskclick the X. Yeah. I mean the thing to that it is important to sort of have some authentication mechanism so people can't see. You can't just create 100,000 instances of virtual machines.
53:12Elon MuskJust create 100,000 virtual machines and just create and have them all behave like users. Although at this point with ChatGPT you could probably just crush frankly, because ChatGPT would exceed.
53:24Elon MuskI think the.
53:25George HotzOh, it's about to be impossible. Capture is about to be impossible. I don't know what we're going to do.
53:31Elon MuskYeah, but I'm like, at least we can try. At least we can make it like. So it's not trivial to have like 100,000 Bard accounts, which it currently is.
53:38George HotzThe best things, the best things to do are like social proof kind of metrics. Gmail. Remember OG Gmail with the invite?
53:47Elon MuskYeah, Just follow. You follow the tree. So if like, if one. The referral basis and if one put. If there's one referral that goes sideways, you just look at that and you cut off that entire referral tree.
53:57Elon MuskThat's how beginning.
54:02Audience MemberGeorge, can I get two questions in or two thoughts?
54:06George HotzYeah, what do you got?
54:08Audience MemberSo I'm in San Diego on my Starlink right now, so this is pretty awesome. I. So I'm a father of four and you want to have two ideas to help boost revenue.
54:20Audience MemberSo my kids, they're going to be exposed to advertisements no matter what. Could we have like a Twitter nest feature where parents can like let their kids have accounts and the parents can control what advertisers their kids see.
54:33Audience MemberBecause my kids are going to buy Robux or you know, they're going to be buying stuff no matter what and I'd like to choose what they're exposed to.
54:39George HotzRefactors first.
54:43Audience MemberI agree with that as a software developer. Then the next thing is there are tweets that I want to boost that are other people's tweets because I just.
54:53George HotzWhether it's a political party or you
54:55Audience Memberknow, some move, just. Yeah. Could we see like a paper trail of where the money comes from? So going back to what Pablo said. Pablo? Yeah, Pablo said earlier how to see through the noise.
55:08Audience MemberI think it's a lot easier to see through the noise when we can see the paper trail, the money that it's influencing the tweets and I think that's what as an older user I
55:17George Hotzwas frustrated with the bots and I'm fine with people paying to get things
55:20Audience Memberboosted, but I'd like to see where, what the source is, where it's coming from. That's it.
55:28Elon MuskOkay. Yeah, agreed.
55:46Co-hostI did see a post a while back about video monetization. Is that a thing you're down to talk more on? Because I have a lot of thoughts after my time at Twitch.
55:56Elon MuskTwitter certainly needs to enable longer videos. I wasn't clear on what the limitation on the Twitter video length was, but I was recently told that it was a 2 gigabyte file size limitation, at least in the current stack.
56:15Elon MuskSo like it's difficult to have video monetization if the videos are like extremely short. So. But if you have a, you know, better codec than is currently used at Twitter and resolution that is reasonable for a phone, you can have some pretty long videos in 2 gigabytes.
56:36Elon MuskSo like the first approximation, I feel like the first order is like you need to actually be able to post a sufficiently long video. If you cannot do that, then nothing else matters.
56:46George HotzYeah, I don't think most people have access to that. So when I did that live stream before I joined Twitter, I had to get Media Studio and I actually got this.
56:56George HotzMy friend was like, oh yeah, I know a guy in a telegram group and he PayPal them 30 bucks and got access to it. So that's what it takes to get access to long video on Twitter today.
57:08Elon MuskYeah, well it's my again, I have to like sort of a lot of things I should say. I should preface by this is like what I think someone told me as opposed to like this is the God's honest truth.
57:23Elon MuskSo that's an important preface for anything I say it. What I say may not be true. At least it's my recollection of what I think someone told me. So take anything I say with a grain of salt.
57:36Elon MuskBut Anyway, if the 2 gigabyte thing is real, then we can actually have long videos with an improved codec. The current codec I think is quite old and then having a resolution that's manageable on a phone, not super high def, especially if it's something like a podcast where you don't need flexi every detail in high res, then it's just like basic stuff like having a video ad attached that is at least reasonably relevant to the material.
58:18Elon MuskSo if it's a video about technology, that the ad is a technology related ad, it's a video about gardening. It's a gardening related ad. It's sort of stuff that would make sense.
58:32Elon MuskLike I said, not splitting the atom. It's pretty basic and that's what needs to happen on that front.
58:38Co-hostThe main problem that we saw was getting good enough ad inventory for video ad. We found it was a lot harder to build up a large pile of ads to serve a distributed enough group of people.
58:49Co-hostAnd it took a while for Twitch to even have bucketing of ads for users in the first place. Are you confident in the ability for Twitter to acquire video ad buckets and targets?
59:01Elon MuskNo, I think that is a legitimate issue. And actually if I look at YouTube, because I actually watch YouTube without YouTube Premium just to kind of see what kind of ads are being served here and the like, very often we'll see sort of nonsense ads that are like quite scammy and irrelevant to the subject.
59:23Elon MuskSo if YouTube is even having trouble getting ads that are relevant to the subject, then probably Twitter will have some challenges that are significant. But I think part of this is also just like reaching out proactively to companies that do things in a particular arena and say we have inventory that is relevant.
59:46Elon MuskWhy don't you try advertising with something that's in this genre and see how it goes? And then like I said, if you can actually have a logical return on investment for advertisers, I think en masse they will be logical.
1:00:08Elon MuskYou might have some irrational actors, but for the most part they will be logical. So yeah, but I do think that the ad inventory thing is an important problem.
1:00:20Elon MuskIf you only have ads of a particular type, then you're going to have trouble achieving relevancy because you simply lack an ad that is relevant.
1:00:28Co-hostIt also has to be a diverse enough set within that because video ads are much more frustrating as a user to have as repeats. So if you see the same video ad three times, it's significantly more frustrating than seeing the same suggested tweet.
1:00:39George HotzI feel like they're so bad everywhere. I feel like I get that all the time. Like when I'm watching, like I have a cheap Hulu account with ads, it's the same ad three times.
1:00:47George HotzAnd I'm like, this is terrible.
1:00:48Co-hostGoogle's the only platform that's really nailed this. Instagram's doing better here because they're just their ad platforms. Incredible. Nowadays, like meta's been killing it. But the inventory problem for a diverse set of video ads is really tough.
1:01:00George HotzYeah. Doesn't take talk.
1:01:02Audience MemberBuild a model per user and they spend something like 10 kilowatt hour, like
1:01:07George Hotzper year, per user. No, I might have spread some of that misinformation. No, you can read TikTok's paper. You can read it Monolith. It's pretty normal content recommendation engine stuff.
1:01:21George HotzThey just like do it well, the papers.
1:01:25Audience MemberIs it actually crazy to do that though?
1:01:29George HotzNo. And we're gonna start to see that. Like, I think that the. Look, eventually you're going to build the Infinite Jazz videotape. Eventually you're going to build content that is so compelling that you'll find people just stare at this stuff for hours
1:01:46Co-hostand hours and hours and hours.
1:01:48George HotzAnd you can, you can do that as the ML gets better and better and better. But you know, unregretted user minutes, right?
1:01:55Elon MuskYeah, I think that's the right thing to aim for. Like, I think it'd be hard to argue, like, okay, we improved unregretted user minutes. That is definitely a good thing and not a bad thing.
1:02:09Elon MuskSo anyway, I gotta throw in the towel here, guys, but I do want to sort of leave off saying that there are a lot of people at Twitter doing great work and it is super appreciated.
1:02:28Elon MuskSo this is like, you know, definitely not meant to be. Like, let's trash Twitter nonstop. It's like, you know, at least aspirationally meant to be self critical and to say like, okay, what, what can we do to make things better?
1:02:46Elon MuskSo, you know, I just do want to make sure that this does not come across as being unreasonably critical of those who are actually working hard to make Twitter better.
1:03:00Elon MuskSo with that, good night guys.
1:03:03Co-hostThanks a lot.
1:03:05George HotzVery supportive. Thanks, Elon. See where Twitter 2.0 goes.
1:03:09Elon MuskThanks, Elon.
1:03:14Audience MemberHey George, can I ask you a question about engineer numbers?
1:03:17George HotzYeah.
1:03:18Audience MemberOriginally why I joined, I didn't think Elon was going to join.
1:03:21George HotzWell, I mean, Elon joined. So after Elon joins, you know, he's like, he, he, he, he's the only one who's allowed to say anything about Twitter. You know, I'm not like, I can't, I can't.
1:03:31George HotzI'm not going to reveal how like Twitter stuff works. I signed an NDA.
1:03:35Audience MemberExactly. So I'm, I'm a software engineer. I similar situation of.
1:03:42George HotzOn a project where there's been a lot of turnover and the skeletal skeleton crew.
1:03:49Audience MemberI feel like I'm being pulled between, like, I want to be doing software, but I also have to understand the business better too. And do you think that's like a bad position to be in when you, when you as an engineer, you got to Start trying to innovate the business.
1:04:03George HotzAnd let's, let's, let's bring this space back to. I mean, you know, Elon shows up, he'll take any conversation, any direction he wants. But yeah, let's bring this back to, you know, the question of the space, which is scale, complexity and number of engineers.
1:04:22George HotzNo, and look, it's great to have Elon join. You know, it's so nice to think Mark Zuckerberg would show up in the space. And, you know, Mark Zuckerberg is not even the worst.
1:04:31George HotzYou think Sundar Pichai would ever come talk informally about Google? This is something right here. This is something I absolutely love about the new Twitter.
1:04:40Co-hostLike that YouTube actually does this, which is really cool. Huge shout out to her.
1:04:45George HotzWait, YouTube does this? Who?
1:04:47Co-hostYep, the CEO of YouTube. I, Susan Wojcicki, believes her.
1:04:50George HotzSusan Wojcicki has like. I don't know how to pronounce her last name. She has like Twitter spaces where she just casually talks about YouTube, not Twitter
1:04:57Co-hostSpaces, but she hops into the conversation on Twitter in the replies all the time. And she shows up on like, really, like small people podcasts on people who are known for pushing back on the YouTube platform, like regularly.
1:05:07Co-hostShe's been surprisingly transparent. Like, I've been super impressed with the leadership at YouTube.
1:05:11George HotzThen why'd we get rid of dislikes?
1:05:13Co-hostBecause of the. The net use of them was harmful enough that they decided it wasn't worth maintaining. It was a long battle internally. Multiple interviews where she talks in depth about this with both Marques Brownlee and with Ludwig.
1:05:27George HotzI appreciate that. You know, I haven't seen these interviews. I haven't looked. I've just complained about dislikes. So, look, I'm. Personally, I'm a huge fan of transparency building in public.
1:05:39George HotzYou know, it's just, it's cool to see.
1:05:42Audience MemberHey, George, was this space inspired by John Carmack leaving Facebook?
1:05:49George HotzI mean, that's one of the things too, right? Like, this is really not all about Twitter. We can discuss John Carmack leaving Meta. right? Like, you know.
1:06:00Audience MemberYeah, something like 5% GPU usage. Oh.
1:06:04George HotzI feel like I can say absolutely anything I want because I don't work there and I have no NDAs in place. Place with there. Yeah. I mean, you know, look how much they spent on Horizon Worlds.
1:06:13Audience MemberAnd then.
1:06:13George HotzHave you actually used Horizon Worlds? Horizon Worlds is the most expensive video game in history.
1:06:18Audience MemberI've never heard of it.
1:06:20George HotzExactly. It's their metaverse thing.
1:06:23Audience MemberI've never.
1:06:23George HotzI'VE never actually used it. I've only seen, seen videos of it.
1:06:26Co-hostThis point, I'm convinced every social media company needs to, like, spend too much money to build a game at their peak. I watched the same thing happen at Twitch.
1:06:33Co-hostThey built a karaoke game. They tried to con or contract Harmonix to do it.
1:06:37George HotzThey're like, wait, they.
1:06:37Co-hostNope, the scope here is absurd. We're not doing that. And they hired a whole game dev team and I had to teach them react after when it was a disaster.
1:06:43Co-hostGood times.
1:06:49George HotzYeah, I think, you know, the metaverse could be so good, but you need a John Carmack, like, character driving it forward.
1:07:00Elon MuskRight.
1:07:01George HotzBecause otherwise you're going to end up with the same sort of blow. You end up with all these companies and, you know, it'll be terrible.
1:07:11George HotzWe got some new speakers.
1:07:13Audience MemberHey, George. Thanks, man. Hey, I was impressed with the. With the way everyone handled that Runway question that was. That really stuck out. And so I was curious just to like, kind of download those numbers real quick.
1:07:28Audience MemberSo it sounded like he was saying 3 billion a year in cash burn was what he was expecting with like a billion in the bank. I'm just trying to, like, track through all that because, like, I don't know if anybody knows, but I would imagine the payroll number previously was probably like 1.5 billion cut to like 0.5 or something like that.
1:07:49Audience MemberSo I'm just trying to figure out.
1:07:52George HotzI have Dan. I don't think he's supposed to talk.
1:07:54Elon MuskI don't think he wants to talk about Twitter stuff on this without.
1:07:57George HotzYeah, no, I'll talk about that quickly. I will. I will talk about that for a minute. I mean, I did. I did actually really appreciate that Elon answered it.
1:08:07George HotzI. Again, I know nothing about this. Everything I know about Twitter's financials comes from public disclosures beforehand. And I'm not just saying that I actually know nothing about it.
1:08:18George HotzSo, you know, that's. I believe what he said was, Twitter's burn is 3 billion and they have 1 billion in the bank. And that's not a long run.
1:08:26Elon MuskIt's like four months or something.
1:08:27Audience MemberRight?
1:08:29George HotzYeah. So, again, things all are above my pay grade. I don't work at Twitter anymore.
1:08:35Audience MemberJust the one thing I think to be aware of and maybe to factor in is just that I believe that the acquisition put a billion dollars of debt expense onto the company.
1:08:46George HotzAnd Elon did say, yeah, that's correct.
1:08:50Audience MemberAnd so, like, I kind of pressed him in a previous space, and I actually really appreciated him engaging with me on it just like you're saying. And he, you know, he.
1:09:00Audience MemberI think, I think if I were to summarize sort of like his answer, I think he kind of feels like, well, Twitter was going to get run the, the ground if he didn't buy it and stuff.
1:09:09Audience MemberAnd I don't know if everyone else agrees with that, but I think he kind of sort of insinuated that he was worth it for that billion year of expenses.
1:09:17Audience MemberBut that's like the equivalent of like probably like the entire company payroll now or something like that, like for one guy. So it's just like I'm just trying to like wrap my head around it.
1:09:30George HotzLook, yeah, I mean, look, this is. I, I can't really speak to that stuff. I know the Runway of, I know the Runway of my startup and it's considerably longer than, than four months.
1:09:41Audience MemberYeah, well, I'm, I'm glad to hear that and I appreciate you having me on and thanks.
1:09:47George HotzCool. Yeah, maybe we'll keep this going for another like five, ten minutes.
1:09:53Audience MemberOkay. So if you can talk about Twitter's financials, let's talk about the Twitter's technology instead. I feel like I didn't really get to phrase my question correctly when I asked it last time when Elon was up.
1:10:06Audience MemberSo I'll try to phrase it again. I think having all of this custom, non standards, tech, tech stack, a lot of it, a lot of which is like invented in house by Twitter employees, Twitter developers, doesn't that like slow down onboarding or working on new features significantly?
1:10:25Audience MemberWhy not use GRPC that a million people out there know how to deal with and instead using thrift RPC, which is something that maybe 500 or 1,000 people in this world know.
1:10:37George HotzThrift is not at all. This is straight up engineering fact. Thrift was invented at Facebook. I used it while I was at Facebook. Thrift is not the problem. And so also open source Twitter, you can find Finatra and Finagle.
1:10:53George HotzSo Finagle is a RPC server. Finotra is like a routing engine on top of it. Finagle can do both HTTP and Thrift. I know these are open.
1:11:04Co-hostPeople are pinning random shit in the on top. Now that has nothing to do with what we're talking about.
1:11:09George HotzOh, I don't care. Just ignore it. How do I.
1:11:13Audience MemberMy point was not that it's not, it's. Oh, it's not open source. My point is that how many people out there the. The.
1:11:21George HotzYeah, okay, so you can say yeah, there's a developer pipeline problem for a language Like Scala. Again, technical debt is technical debt, right? I've never coded a line of Scala in my life and I can watch my last few streams.
1:11:33Audience MemberI did.
1:11:33George HotzAdvent of code and Scala. Look, am I an expert scholar engineer? Not at all. But can I make decent pull requests? Yeah, I think now I can. I don't think that's the problem.
1:11:44George HotzI think the problem is technical debt. It's just like technical debt can appear in any language, in any environment. And I think the largest, I mean, you know, I have.
1:11:54George HotzI have a neural network framework called Tiny Grad. It's a thousand lines and, well, it's a little more than a thousand lines. But the goal is to keep it at a thousand lines because lines are one of the only reliable proxies I know for technical debt.
1:12:11George HotzNow, of course, you can game this, right? Any, anything. Once it becomes an optimization target, it ceases to be a metric. This is true, but at the same time, you can only game it.
1:12:21George HotzSo far, Elon was saying 20 million lines of Scala. What's 20 million lines? You know what on this website really should take 20 million lines.
1:12:35Elon MuskMind if I ask a question?
1:12:37George HotzSure, yeah.
1:12:39Audience MemberSo I've worked on a lot of big tech companies. Amazon, Google. I worked at Google for like 10 years and a lot of teams and
1:12:46Co-hostI also worked at a few startups.
1:12:48Audience MemberSo, like, one thing I noticed is with the JVM languages like Scala, I've worked with a few teams that have used Scala. Is that. Or just any JVM language is that there's this additive engineering problem.
1:13:00Audience MemberLike they keep adding code. It seems like the JVM languages in particular, whoever, and sometimes I think the JVM languages, they almost encourage, I feel like code bloat and lines of code and it's like, no big deal.
1:13:16Audience MemberThere's so many advanced tooling. There's so many tooling around these languages is so good that they can just pump out. The IDEs can just pump out code. And then it gets to the point where it's not readable.
1:13:25Audience MemberIt's optimized for writing, all the refactoring and stuff. So they're not optimized for reading because the tooling is so good, right? And so you end up with like tons of code.
1:13:33Audience MemberAnd then when people want to add features, they instead of reading the code that's already there, which is enormous, they just, like, they just write it again.
1:13:40Elon MuskIt was like.
1:13:41George HotzYeah. So I was gonna, I was gonna first say, like, why would the VM have anything to do with. Right. I can run Python on the JVM if I wanted go get Jython.
1:13:50George HotzSo I don't think it's the J. It's mostly the libraries.
1:13:53Audience MemberThe libraries. Like, like there's so many libraries and there's so much tooling that you end just with like enormous amount of code.
1:13:58George HotzI, I think it is keep like standing around.
1:14:00Co-hostPoints I really agree with, but then the details are nonsense. Oh, it's like I largely agree with you here. I just, I don't think any one of these things is the problem.
1:14:08Co-hostIt's not libraries, it's not the jvm. But like absolutely. We're at a point now where things like you can write much more code much more easily because the tools are getting better every day.
1:14:17Co-hostI think we as like an engineering like community need to figure out how to deal with that reality where the average engineer isn't as talented because the bar for engineering has gotten so much lower because the tools are so much better.
1:14:28Audience MemberBut I think what I was saying earlier is kind of right though is that like every company just has all these bespoke different tools and all this like complex little like onboarding thing that takes, you know, like it probably takes a month for someone to ramp up into any.
1:14:43George HotzNot Facebook. Not. Not.
1:14:46Co-hostYeah, Facebook held it with their thoughts.
1:14:48George HotzWould you agree? Facebook killed it. Exactly how good I worked. I was full time at Facebook in 2011. How good was Facebook's tech?
1:14:57Co-hostI was genuinely surprised they had to do layoffs because they've been able to hire so lean compared to the amount of like product surface area they cover, it's nuts.
1:15:04Co-hostThe way they architect everything is incredible.
1:15:07George HotzThese are the arguments I kept making inside Twitter. I'm like, guys, it's. And people are like, George. George has no idea what he's talking about. I'm like, I've seen Facebook.
1:15:14George HotzIt's beautiful.
1:15:17Audience MemberCSH alum here from 0508. Long time man. I was wondering if you would agree that with the direction AI is going, I just feel like, I feel like with how much code exists open source and accessible to AI, I feel like code monkeying work of computer scientists and software engineers is going to just continue to get less and less and it's going to start to become more of a.
1:15:43Audience MemberWho can communicate with the engine the best and the most creatively.
1:15:49George HotzIt already is if you think about it, right? So programming, unlike say truck driving, is already tool complete. So I mean to specifically talk about the JVM thing, it may very well be.
1:16:01George HotzAnd you can like do this as a Conversation controlled experiment that like do ides like intellij which have all this autocomplete boilerplate functionality. I think it's actually IntelliJ.
1:16:13Elon MuskYeah, I think it is.
1:16:14Audience MemberI think you're right. It's IntelliJ and it's going to get worse. With Chat GPT and all this code generation, it's kind of, we're like it's not going to be good.
1:16:21George HotzI, I, I mean, yeah, I, I don't even think it depends on your perspective.
1:16:24Audience MemberWorse, right?
1:16:25Elon MuskOr, or better?
1:16:27George HotzNo, the AI, the AI code stuff is ridiculous. AI code generation is ridiculous. Ridiculous. And like it's just the whole thing is laughable. And here's why. Programming is tool complete.
1:16:40George HotzMeaning what happens Accounting is also tool complete. When you build better tools for accounting or programming, you end up with less accountants and less programmers having leverage to do more work.
1:16:53George HotzRight. This is in contrast to something like Walmart cashiers or, or truck drivers. Right.
1:16:59Audience MemberNo matter.
1:17:00George HotzThe truck might get better but you still need the driver to log the hours. All right, fine. You can say Walmart cashiers is a bad example because okay, now you have one person overseeing the four self checkouts.
1:17:10George HotzMaybe that is the same thing with technology. So we'll stick with truck drivers, which is not tool complete. But the AI code generation in of itself is like why do you need the extra step?
1:17:20George HotzRight? What do you want the AI to do?
1:17:23Audience MemberHey George. Hey George. One, one question right on, on the topic I think that you are trying to address is where are you at on, in terms of how many employees or engineers are ultimately needed just to sort of get the job done or kind of have like a good healthy work life balance, like that sort of thing.
1:17:39George HotzLike in terms of work, work, work. I, I think, I don't like work life balance. I think work life balance or whatever it is you want.
1:17:45Audience MemberYeah. And the people there want.
1:17:47George HotzNo, I don't think, I don't, I think work life balance is stupid to concept. I think that you know, work is life.
1:17:53Audience MemberTake that part out of the question. Just you know, kind of like
1:17:57George Hotzhow many, how many engineers do you need to do what?
1:18:00Co-hostRight.
1:18:00George HotzIt all depends on what the task is. Right.
1:18:02Audience MemberYeah, I'm trying to think through like sort of what's, what's kind of the unifying vision that that's going to bring all the engineers at Twitter together. Like I like some of the feature points Elon made about like showing the views and how that would raised a lot of interest.
1:18:16Audience MemberI thought he had some like interesting ideas. But like what's the sort of vision and then like, what. What support do you need from, you know, whatever the powers that be are above you as teams to get the
1:18:28George Hotzjob for the folks at Twitter. I don't work at Twitter. I don't know anything about Twitter. I resigned from Twitter today to talk about AI and for various reasons, I'm certainly down to talk about AI and AI coding.
1:18:44Audience MemberCool.
1:18:45Elon MuskOkay.
1:18:45Audience MemberI misunderstood. Thank you.
1:18:47Elon MuskSure.
1:18:49George HotzIf you're not. If you're a speaker right now and you're not speaking, please step down and we'll let some new people.
1:18:54Elon MuskHey, George, I'm sorry if I wasn't
1:18:57Audience Membersupposed to pin up top, but. But I pinned a little bit about the Amber Alerts. If an Amber Alert goes off in an area and a car happens to drive by a Tesla, what are the odds it doesn't?
1:19:08Audience MemberCan the camera scan its license plate, make a model, and beam that instantly to authorities? And I figure this would be a really great application of Tesla.
1:19:17George HotzWho's writing that? Oh, we got to write code. We got to test it. Ask ChatGPT maybe ChatGPT.
1:19:24Audience MemberThe reason why I even float that idea, though, is because, of course, I think there's something bigger here that Elon might be playing at. Right? His 5D chess, which is he has all these cars mapping the roadways and.
1:19:34Audience MemberAnd noting the cars that are driving by him. So now that he has Twitter with phones, with geolocation and other drivers pockets, Honda Accord drivers, is he gonna be able to pinpoint when the Honda Accord passes by, the exact location of a Tesla?
1:19:52George HotzYou know, what features are really easy to imagine until you have to write them, right? Until you actually.
1:19:57Co-hostIdeas are so cheap.
1:19:58George HotzIdeas cheap. Until you have to, like, make these things sort of work at scale. So you say, like, is this what someone's planning? Like, it doesn't matter what someone's planning.
1:20:07George HotzIt matters, like, what's actually going to get built and. Yeah, you know, look. You know, any. Look, I've been working at Common now for six years. You know, like easy it sounds in theory, to make maps from our data.
1:20:19George HotzOh, we have all the data. We know where all the lanes are. We have such an accurate gps. Yeah. Good luck making maps.
1:20:27Audience MemberProductizing a good idea is the hardest part.
1:20:28Co-hostI agree.
1:20:29Audience MemberBut I do think that Tesla is particularly good at making awesome product. Unlike Twitter was.
1:20:38George HotzWell, yeah, I think that you can do certain things with organization, but it really is a different product, right? It is a different product. And there are some challenges.
1:20:47George HotzI mean, Twitter is not perfectly horizontal scale. Right. Tesla is Pretty much perfect horizontal scale. Not entirely perfect or, you know, I'll say, comma, comma, is pretty much perfect horizontal scale.
1:21:00George HotzWe have some things where all the cars connect to the server, but all the driving is done locally on the car. So that scales perfectly. You know, like, there's no.
1:21:09George HotzThere's no complexity to deal with there. If you make it work on one car, it's easy to make it work on all the cars, but if you make it work on one Twitter app, making it work on all the Twitter apps might be.
1:21:20Elon MuskYeah,
1:21:23Audience MemberI think Elon kind of brought up a really good solution, though, which is like, just incentivize people to delete code. Like. Like refactoring and deleting code and simplifying isn't sexy work.
1:21:34Audience MemberAnd no one really wants to do it, but, I mean, pay them more for deleting code.
1:21:40George HotzThat is all I wanted to do with Twitter. That's all I wanted to do. I was there to refactor and make it beautiful. I don't know. But you see the problem with refactoring and making beautiful is you got to do that instead of features.
1:21:52Audience MemberYeah, exactly. I guess just pay the people more who are refactoring.
1:21:56George HotzNo, it's not a question of paying the people more. It's a question of if you have 400 people dumping shit into a pool and a hundred people shoveling shit out of the pool, the shit still becomes.
1:22:09George HotzYou know, there's still so much shit in the pool, but at what point
1:22:12Co-hostdo you say, fuck it and just build it?
1:22:16George HotzWell, and you know what I mean, this is one of those remind me in five years kind of things. Remind me in five years. And let's see. I'm so curious to see where Twitter will be and what their infrastructure will look like.
1:22:32Audience MemberI mean, that's essentially what ended up happening, right? The timeline mixer was deprecated and the new rewrite of the home mixer was implemented.
1:22:40George HotzDid it work? Well, did it work? What do you mean, did it work? There's now both of them.
1:22:48Audience MemberYeah, it increased the cost. But did it help the technique depth?
1:22:53George HotzNo, you're just saying, you have more shit, there's more shit in the pool. You didn't take any shit out of the pool. I.
1:23:02Co-hostThis is like a slight rap. Back to a topic earlier around, like, efficiency and, like, the AI stuff. I think that the problem we see in a lot of these larger companies, especially the ones that, like, do lots of different features, is how they transition from, like.
1:23:18Co-hostI noticed a pattern. I think there's A term for this, but the similar to the Moore's Law where every two years processors get like twice as good. Every three years, any given piece of software becomes two to three times easier to write.
1:23:30Co-hostAnd if your company's been around for 17 years, a lot of efficiency has been unlocked by us improving the tooling that. George, you and I, as younger guys, we look at these services, we're like, what the fuck?
1:23:41Co-hostThis could be like 100 lines of code instead of 10 services that are thousands of lines of code. A lot of that is a shift in the industry based on all the tools that we use now every day.
1:23:52Co-hostFor me, AI is another one of those where certain languages and certain architectures that right now are inefficient to write because of just raw amount of code. For something like Golang, for example, where you're copy pasting like AI, specifically like Copilot can make you much more efficient as a developer.
1:24:11Co-hostI think what we're seeing is one of those 2-3x moments in real time. But I don't think we're seeing developers die. I think we're seeing yet another advantage to the people who are willing to start from scratch and take advantage of the new way to build a pool.
1:24:23George HotzSo with respect to Golang, I mean that's a very deliberate choice on the language designers. It's like, you know, well, we have all these people and like, let's be honest, they're not smart enough to code C, so we'll give them Golang and like that's the real, that's the real justification for it, you know, and I consider myself one of those people not smart enough to code C.
1:24:42George HotzAnd that's why Golang is nice. But that verbosity, like, you know, the very explicit painful error handling and Go having to actually check each error code, it's there to encourage people to think right.
1:24:57George HotzSo you can go and pave over that with chat GPT, but then you just get people who don't think right and then the whole point is kind of ruined.
1:25:05George HotzLike Go was very deliberately designed that way. I can't speak to how Java was designed because that was all before my time. But I remember when Go came out, I like watched all this happen.
1:25:16Audience MemberSorry if this is a Jason topic. Is Copilot's biggest barrier to adoption just installation, the, the intimidation of installation. And with the price of Copilot, would customers, would they be price elastic enough to like pay an extra 99 bucks to have like an Uber like service nearby?
1:25:33George HotzCopilot I didn't find it great. The same thing with ChatGPT.
1:25:37Co-hostLike I got multiple top 100 in the world placements this year in Advent of Code Using Copilot once you learn how to work with it as a tool, it's kind of like vim, where you have to learn to train it.
1:25:47Co-hostLike know how to write, name a variable, leave a comment and trick it to writing the right code. But it's a tool in that way, like you have a relationship with it.
1:25:54George HotzAnd if you're trying to play Advent of Code where you're trying to do it as fast as possible, sure, Copilot might help you.
1:26:02Co-hostI also worked at Twitch for half a decade and run a company and wrote half the code for that too. It works really well in all these environments. From my experience.
1:26:10George HotzYou really you find Copilot helps you because I don't.
1:26:13Co-hostYes, it's when it conflicts with other tools that I find as if not more helpful. Like when it gets in the way of typescript, type definition, autocomplete and stuff like that, it can get annoying.
1:26:22Co-hostBut I do often find it improves my velocity and experience as a developer. When it hits, it hits. And if you know how it's going in this and you'll learn quick when you use it, I'm going to have it write this algorithm.
1:26:35Co-hostI know it's going to put these two things in the wrong place. I'm going to delete this one and move this one somewhere else. It makes me feel better as a developer the same way I did when I first got really into vim.
1:26:44Co-hostBut it takes a bit to really like feel that.
1:26:47George HotzSo do you like Autocomplete in general?
1:26:51Co-hostYes, when it's guided and you have a good type safe system that is defining the autocomplete. I'm really big on full stack type safety and autocomplete. Intellisense, like the Microsoft way where it guides you to the right line of code, is something I'm really big on.
1:27:06George HotzI find. I hate autocomplete. I never use any autocomplete for anything. The things that I like with IntelliSense.
1:27:13Audience MemberMaybe mobile texting.
1:27:16George HotzWhat?
1:27:16Audience MemberMaybe mobile texting because it's so annoying to try to get. That's the only time I use autocomplete.
1:27:22George HotzOh. Oh yeah. No, this is. No, specifically for programming. I like that I can right click on a function. It takes me to the definition. Even if that's only correct 90% of the time or 80% of the time, it's still a lot better than having to grep for it.
1:27:39George HotzBut I don't know. I find most autocomplete to like.
1:27:43Co-hostSo I run a podcast where I talk about all of this type of stuff. I've had the creator of React JS on. I'm going to have the creator of GraphQL on soon.
1:27:50Co-hostIf you're down George, I'd love to have you on to show you how my stack works and how the T3 stack stuff goes specifically because I think you'll see when you have full stack type safety implemented from your database to your user, autocomplete stops being a thing that annoys you and starts being a thing that guides you to writing the right code the first time.
1:28:09Co-hostAlways. It's so powerful when you have it set up right.
1:28:11George HotzLet's set it up. Let's set up a live stream and I will give this stuff a try. Look, it's the kind of thing also where I'm sure if I.
1:28:19George HotzI don't know, I have like. I don't really like tooling. I like absolutely minimal tooling. I used vim for a long time with nothing. No plugins, no nothing. And now I use VS code pretty much the most bare bones you can make it but.
1:28:37Audience MemberSo I'm a little embarrassed. I did not. I was using the. I said copilot earlier. I meant to say open pilot. We were just talking about copilot. I used the wrong word when I
1:28:46Elon Muskwas trying to ask ox is if
1:28:48Audience Memberit was with open pilot was the like with the cost of the system in general and if installation is intimidating for it is like a barrier to entry for adoption then like would increment of 99 extra dollars for like an uber like installation service.
1:29:08George HotzYeah, yeah, you know what? You were on that service. You run that service. I'm bored thinking about it. No, really. So. Oh by the. By the way, by the way we are dropping now that I got.
1:29:19George HotzI got 14k people here. We are dropping an incredible video of the year production car comma 3 Look, I don't want to run a business. I don't give a about any of that.
1:29:33George HotzI want to solve self driving cars, I want to beat Tesla and you hit a deadline.
1:29:38Audience MemberLook at this.
1:29:39Co-hostWhere are you posting that video? George?
1:29:42George HotzWhat's up?
1:29:43Co-hostWhere are you posting the video?
1:29:44George HotzOh, right here on Twitter. I mean it's the best.
1:29:48Co-hostDo you want any help with the YouTube strategy? I'm a. I'm a big fan of helping with YouTube strategy stuff. I help with bunch of like Y combinator companies.
1:29:54Co-hostIt's Something that's like, I'm nerdy about the YouTube algorithm and how to make it tick. So if you want to chat about that. I just had a video go viral about AI recently.
1:30:02George HotzYeah, it's not, it's not, it's not that. It's not that good. I don't really care. Like, it's more of a. Like a lot of things that comma are. We have a thing where it's like, I don't really care what people think of me.
1:30:18George HotzPerception is so meaningless.
1:30:20Audience MemberRight.
1:30:20George HotzLike, let's actually just solve self driving cars and let's build something that's 10x better. And then like, I don't care what you think of me, it's just 10x better.
1:30:29George HotzLike, you have to use it. I'm sorry. And we think of a lot of, like, the program and stuff like that. Like open pilot. Open pilot. Their entire open pilot, which can drive a car with zero disengagements to Taco Bell, is smaller than most of Twitter's microservices.
1:30:50Audience MemberWell, Twitter might be a bad bar
1:30:52Elon Muskto compare with, but.
1:30:52Audience MemberYeah, I get the point.
1:30:53Elon MuskThat's incredible.
1:30:57George HotzYeah. But it comes down to it. It's just the same number of engineers work on both, Right.
1:31:02Audience MemberWow.
1:31:03George HotzIt's just the thing that will determine how many lines of code your thing will have is strictly a function of your number of engineers and a function of nothing else.
1:31:13George HotzAnd I think that kind of brings the space full circle. Does everyone kind of pretty much agree with that? You know, give or take.
1:31:18Audience MemberWhy did Twitter need that many engineers?
1:31:21Co-hostI. I largely agree. I think that it. The way things were built when Twitter started and the way things are built now are different. And a lot of the chaos that we now see as obvious is like, obvious in hindsight.
1:31:34George HotzVery much agree. Yeah. I think that I know that whenever I'm writing something, the first time I write it, it's terrible. The second time I write it, it's like kind of okay.
1:31:44George HotzYou can sort of see that it's
1:31:46Audience Membergoing to be good.
1:31:46George HotzAnd then the third time I write it, it's actually good.
1:31:49Co-hostI have one challenge for this now that I'm like really deep on the product side. The first version's often enough to figure out if you have product market fit.
1:31:57Co-hostAnd it's often not worth me paying you as an engineer to build the second and third version until we know that the product has some market fit. If you have a system where you can identify that and then Go build version 3 when like a very small subset of users are on that early V0.
1:32:13Co-hostAwesome. Do that. But I think the problem is we're trying to find product market fit as cheap as possible. And when we find it, we keep building on that awful V0.
1:32:21George HotzI think nobody has doubt that Twitter has product market fit.
1:32:25Co-hostI think we can afford if you add new features. We don't know. We don't know if Twitter Blue is going to have product market fit. And I know I can't change my pro profile picture right now because I'm a Twitter Blue user.
1:32:33Co-hostI had an automation set up to change my profile picture whenever I went live and it didn't error out when I had changed when I became verified through Blue.
1:32:41Co-hostAnd now I can't change my picture anymore. My account's in this weird locked state. Just Growing Pains Rush the feature. It's not. No, it's not Growing Pains. It's experimenting in favor of product market fit.
1:32:52Audience MemberYeah, I've got the tweet pinned up here on top. Sorry if that's, if that's not okay. But I just thought I would. But like, I think that it's true though.
1:33:02Audience MemberLike, it would be awesome to have more feedback mechanisms into the core teams at Twitter. So like, if, like there's a feature that just makes a lot of fucking sense.
1:33:12Audience MemberLike, like you could just have that feedback directly from the users or something is broken. And then getting those information systems into the core team so that they can really iterate on that, on that product market fit, you know, I mean, obviously Twitter already has it, but, you know,
1:33:27George Hotzyou can, you can add anybody at comma and you can discuss their features with them. I can't speak to Twitter, but I could say, you know, comma, everyone's pretty much on Twitter and you know, you can, you can, you can at them and ask for features.
1:33:42George HotzI don't know, like, I love building in public. I think that so much secrecy is incredibly stupid. I think it would be so cool if Elon actually dropped a 3 gigabyte tarball.
1:33:54Audience MemberIf he did that, that would actually be insane.
1:33:58George HotzYou want to sit there and try to read it? Welcome to my last four weeks.
1:34:03Audience MemberI mean, it was just, it would actually like, like that would be mind blowing, you know, if he just was like, yeah, here, you solved it.
1:34:10George HotzI mean, no, you're like, okay, I hope he actually doesn't do that, because who knows how many secrets are in that code base. I don't mean secrets like, like, like, like, like IP secrets.
1:34:19George HotzI mean secrets, like a literal like AWS
1:34:23Co-hosttoken.
1:34:27George HotzEveryone's got those in Their private business. All right, Ashok's got his hand up.
1:34:34Elon MuskYes, hi. Is there any ideas and possibilities on having unique profile pictures to reduce fake profiles or reduce abuse?
1:34:45George HotzWait, the last thing that you can possibly ask me about is Twitter features. By the way, I'm in the Twitter should build zero features camp and should just do refactors for the next six months.
1:34:57George HotzSo that's my take on any asks for Twitter features. But look, I'm not even at Twitter anymore. I don't even work there. I resigned.
1:35:04Audience MemberWere you joking? So you actually worked there?
1:35:06George HotzWhat were you doing? I tried to fix all this shit.
1:35:13Co-hostI literally live across the street. How do I go to help fix all the react. Bullshit.
1:35:18George HotzAll the react bullshit. Look, I don't work there. I don't, I don't know about any of this.
1:35:23Audience MemberI mean, this is. George, I got a question about comma, actually.
1:35:27George HotzAll right, all right, so let's say
1:35:29Audience Memberthe best case scenario is comma turns into like a 500 billion dollar company, right? How many?
1:35:36George HotzI don't know if that's the best case scenario. That sounds like people want to come take the from me.
1:35:40Audience MemberWhat do you mean by that?
1:35:42George HotzI don't give a about the money. The money's fake. Guys, do you believe in the singularity? The money does not matter. The singularity.
1:35:49Audience MemberYeah. Okay, conspiracy aside, right, but how, how, how?
1:35:55Elon MuskIt's a social.
1:35:57Audience MemberThe money fake. I, I, I know, I know. All right.
1:36:00George HotzGet the money's back. Yeah. All right, go on.
1:36:02Audience MemberHow, how big would comma grow? Because you can't operate that. Like, you would need at least a team for compliance alone.
1:36:10George HotzCompliance with what?
1:36:12Audience MemberI don't know, like people getting car crashes.
1:36:18George HotzWhat if people get in car crashes? What does that have to do with me?
1:36:23Audience MemberBecause you're the CEO and you'd be responsible, but that, that's, that's not
1:36:30George Hotzcomma system and it's open source.
1:36:35Audience MemberWell, the goal is to be level four, right? Or full.
1:36:38George HotzThe goal is definitely not to be level four. I have no idea. Oh, level two forever. Actually, you know what the long term goal of comma is? The long term goal of comma is to have a shop in the mall where you go in and you buy your new comma body and you bring your comma body home with you and it cleans your house and it cooks you some food and you know, you can even have sex with it if you want.
1:37:01George HotzNow that is the goal of comma, right? Compliance. Go to my store at the mall, drop 20 grand, walk home with your comma body hand in hand with helmet shade.
1:37:13George HotzYou're still so.
1:37:14Audience MemberI'm surprised Elon came into your space.
1:37:15Elon MuskDude, it's so funny.
1:37:18George HotzWhat the. Elon came into my space.
1:37:20Audience MemberYeah.
1:37:20Co-hostI'm surprised
1:37:23George Hotzyou're chill, man.
1:37:24Audience MemberHe's a draw too. I get it.
1:37:28George HotzHe was not happy with the asking
1:37:29Co-hostabout ad tech initially. Thank you for defending me. Yeah, I appreciate. I got mad respect for you for that. I did not expect that at all.
1:37:37George HotzThank you. Thank you. No, look, I think. I think he was pissed at the other guy and I think he thought everyone else was like. That the other guy was literal, like, just.
1:37:43George HotzJust annoying him. But no, I mean, look, what you said about ad tech was like, absolutely right. And anyone who works on ads, like, knows us.
1:37:50Audience MemberYeah.
1:37:50Co-hostWhich is. Yeah, I was surprised.
1:37:52Audience MemberHe.
1:37:53Co-hostIt sucks because, like, I really want Twitter to survive and ads are how you survive in this economy. And he just doesn't get a lot of it.
1:38:00George HotzYeah, look, you know what? Whenever I think about. Whenever I think about a decision that Elon's making, I remind myself that he's a billionaire and I'm not. So whatever his process is for making decisions, it's better than mine.
1:38:13Co-hostSo I have a thing I call reset theory, where it's very easy when you go from one place to another in terms of, like, areas of expertise, to think that your current expertise carries over because you're an expert, you're on top of your shit.
1:38:27Co-hostAnd I made this mistake when I was on top of my game as an engineer thinking, oh, I can move over to running a company. No, I can't. It's not that easy.
1:38:33Co-hostIt's really hard to do. I thought when I was popular on YouTube, I could just move that over to TikTok. No, I absolutely can't. It's really hard to do.
1:38:40Co-hostGoing from one type of company to another, from a space where customers are paying you money to a space where advertisers are paying you money to have customers watch their stuff is such a huge shift that it's easy to see, like, certain patterns and, like, the fundamental truths, the one dimensional solutions that you're familiar with, which in Elon's case are people like your thing, people buy your thing.
1:39:00Co-hostThing will be successful. That's not true in this space. And like, the axes you've defined yourself on aren't even, like, the same here.
1:39:08George HotzI mean, I think it can be true. Like, I think that one of the things is baffling to me. You look at Twitter's old things. How is this company possibly spending $5.5 billion.
1:39:23Co-hostYeah, there's like, like there's two extremes here and they're way past the like, too big extreme. But there are and like, I agree, Twitter spending way too much money.
1:39:35Co-hostThey're way too big. They had a lot of debt from both technically and like the architecture of like the company itself, financially. And financially they're, they were fucked on all of these dimensions.
1:39:47Co-hostBut that doesn't mean Elon's knowledge of how to start a business successfully necessarily carries over to how to make a successful advertising platform on a given social media product.
1:40:01Co-hostAnd a lot of the things he's used for his successful career are specifically like harmful in this space. Like those truths don't exist here.
1:40:09Audience MemberBut you're talking about somebody who sold flamethrowers out and burnt hairspray. I think like out of anything. He does know how to generate money. And this is just, it's almost like thinking outside of the box where you just have to apply it in different ways and obviously limit your, your spending in some ways.
1:40:30Audience MemberBut I think it's just, you know, fine tuning his expertise in other companies here and really learning as he goes. Like, he's not an expert there. He didn't really understand the code base or how chaotic it is.
1:40:49Audience MemberYou know, somebody who, like I worked at GM and you have a lot of legacy where they keep stacking shit and they get some new integration tool and everything's just this big ball of different technologies that don't really even connect well, but they just work.
1:41:06Audience MemberAnd once they work, people just don't want to dive in there and refactor it. So I think it's just more so of, you know, getting the right guys behind it that can connect the world a little bit better than it already is.
1:41:22George HotzThere definitely are some truths to running an advertising based business business, but at the same time, this business is not being built from scratch, right? Like Twitter was bought for $44 billion.
1:41:36George HotzYou only need to make this thing profitable to stay alive, right? There are a huge number of users on the platform that even though you might not be doing ads as efficiently as, as, as meta, it may not matter if all you're trying to do is stay alive.
1:41:53George HotzBecause the first fundamental cost to run something like Twitter, this isn't a car company. At a car company you have margins and your margins are kind of razor thin, right?
1:42:02George HotzLike you got to buy all these parts from somewhere, right? You don't make the wheels. You buy the wheels, you don't make the engine. You buy the engine. Maybe Tesla makes the engine, but GM don't make the engine.
1:42:10George HotzActually, maybe they do make the engine, but they don't make the brakes, they don't make the steering wheel. So there's no real. What's the actual cost to running Twitter.
1:42:24Audience MemberI also want to do that. Elon's mentioned this before, and I think this is like, huge. Is that he's going to start incentivizing content creators like on Twitter. Yeah.
1:42:38Audience MemberBut I mean, I think this is a game changers.
1:42:41George HotzWho's gonna build them?
1:42:43Audience MemberAll the, like, substack people, all of
1:42:45Elon Muskthe, like, YouTubers, all these people that
1:42:48Audience Membercan make money, they can do it on Twitter.
1:42:50George HotzI mean.
1:42:51Audience MemberYeah.
1:42:54Co-hostSo right now, Twitter doesn't have a path to profitability. I. We've said a couple things throughout the space, like, Twitter has product market fit. It has product market fit for a product and a market, but we don't know how much money it can make.
1:43:08Co-hostAnd there's two paths. We can try and raise the potential for money to be made so that it can either raise more money so it can get more Runway and be bigger, or we can pick.
1:43:18Co-hostWe can say, okay, Twitter is good enough. Today we're going to trim everything from here, from the waist down so that we know we can fit it under the amount of money it can make at this moment.
1:43:29Co-hostIt is hard to do both at the same time. And when we choose to refactor, now we are choosing to do the latter.
1:43:36Audience MemberAlso.
1:43:36Elon MuskOne of the things from a feature
1:43:37Audience Memberstandpoint that you got to realize is like with Twitch, for example, right?
1:43:41Elon MuskTwitter Spaces is one of the reasons
1:43:42Audience Memberwhy Twitter Spaces is so special is because you're not. There is no video. You are just doing voice. You're able to focus on that conversation.
1:43:48George HotzAnd when you add Copy Clubhouse, when
1:43:50Audience Memberyou add in video, right, you're creating an arms race. Now it's like with Twitch, right?
1:43:54Elon MuskLike, as soon as you have video
1:43:55Audience Membernow as soon as somebody starts getting like a green screen, it starts doing picture overlays, it starts doing animations and starts doing. Getting a ring light. It levels up the quality bar.
1:44:04Audience MemberAnd now there's like the barrier to entry starts to get. Get larger over time. It creeps up.
1:44:10Elon MuskAnd this is what happened with Twitch.
1:44:11Audience MemberAnd now, like, if you were to start streaming on Twitch and be competitive,
1:44:14Co-hostif you think that's what happened, like, you've said some stupid shit, but saying that's what happened to Twitch, really, like, icing on the cake.
1:44:19Audience MemberGod, yeah, that's exactly what happened. And it's also why?
1:44:21George HotzLike, I just, I disagree. That's happened with Stretch. I, I stream on Twitch with the absolute lowest budget and I get a lot of viewers.
1:44:29Co-hostI'm literally streaming on Twitch right now to a thousand viewers and I'm using a $300.
1:44:32Audience MemberBut you can't make a living off
1:44:36Co-hostI. Yeah, I know you can't. I can. Especially since my YouTube gets 2 million views. I produce on Twitch. I worked at Twitch for half a decade. I know how the shit works, man.
1:44:46George HotzYou, yeah, you can, you can be successful those things. Yeah. This is like, I think this is the same people who think you need a fancy keyboard in order to program.
1:44:55George HotzI think these things, the high end people get them but I don't think they really have much to do with
1:44:59Co-hostyour success at all.
1:45:00Audience MemberThey don't have to do success, but it absolutely limits the pool of people who can find success. Right. There's always going to be the outliers, there's always going to be the exceptional people who find something creative to do.
1:45:09Audience MemberThey do.
1:45:10Co-hostYou can buy a $600 phone from Apple, you can buy a $600 phone from Apple and be a top YouTuber with it right now, no questions asked. Like the quality you can get of like recording and quality both video and audio on a cheap Apple device from the store day one is insane.
1:45:27George HotzI was actually looking into, I was looking into buying like a real high end video camera and then I like was watching some stuff and then I was shooting some stuff with my, my iPhone 13 and I'm like, it's not really that different.
1:45:41Audience MemberNo. But good luck being successful without being
1:45:43Co-hostable to graphic edit your thumbnails without
1:45:45Audience Memberbeing having title cards and these things matter tangibly.
1:45:49George HotzTo watch, watch my stream.
1:45:50Co-hostYou have no idea how. Again, discoverability.
1:45:53Elon MuskThis is so fun for making a living.
1:45:55George HotzIt's all about that. Following off topic, do we want to end it with one key thing? The scale complexity, number of engineers. We all agreed that we only need 20 engineers.
1:46:07George HotzThe trick is finding the right 20 engineers. Everything can be built with 20 engineers except rockets. They might need 50.
1:46:16Co-hostThe framing I want to offer you for this, George. This is the thing I try to talk a lot about. I think about problems in dimensions and it should be our goal as engineers not to make the dimension we're working within as rigid as possible.
1:46:28Co-hostIt should be to have as few dimensions as possible. So things like auto scaling, that's a dimension you have to work to maintain. Things like global distribution, that's a dimension you've added of complexity you can make those rock hard, rock solid and hard, keep them in good places, but you're fucked as soon as you have to make changes.
1:46:47Co-hostMy goal is almost always to reduce those dimensions and make it as simple as possible. And tools like serverless are a great example of how to do that. I think that that's the future we're going to see that's really exciting is a future where we're able to reduce complexity not by picking better languages or writing better code with better engineers, but simply fucking like building things that don't have those problems in the first place.
1:47:11George HotzYeah, and I think that's the real truth of the ChatGPT revolution. It's not going to be whether you ask your ChatGPT to write Python or Scala. Like, I think we can almost abstract away the programming language.
1:47:22George HotzWhen can the AI just directly do the compute?
1:47:26Audience MemberI think the issue with ChatGPT is that, you know, there are certain things that it won't be able to know, like the integration or the creativity that comes from the engineers, or like certain times where they asked it to.
1:47:39George HotzTotal bullshit. Total bullshit. I promise you that creativity is, is absolutely Nothing. And the AIs will blow out of the water any human creativity.
1:47:53Audience MemberI won't say creativity, but more so of like asking it to write a REACT application. It's using deprecated libraries. So staying up to speed or depending on how many things you actually want integrated with it, like how, how in depth will those.
1:48:11Audience MemberWill there be like a questionnaire to be able to build everything or kind of like a, I guess a build. I don't know.
1:48:22George HotzAs far as staying up to date goes, those things can read the Internet so much faster. I can retrain the model so much faster than a human could read a book.
1:48:31Co-hostRight.
1:48:31George HotzWell, actually that's not exactly true, but like you could imagine this kind of being true in a limit. The new, the new systems that I think are going to move to right now, they store all the information in the weights, but you could store all the information like kind of like a memory bank and you can do retrieval systems.
1:48:47George HotzFacebook just came out with a paper on this. But I don't imagine it being a questionnaire, I imagine it being a conversation.
1:48:55Audience MemberBut I think that there's something to this though, that AI can't really. There are different kinds of creativity. AI can take different ideas and synthesize from different domains way better than any human could.
1:49:09Audience MemberRight. Because it just has such an insane breadth. But that breadth is composed of all stuff that humans already know running on top of like there's some types of creativity that AI I think will never really achieve.
1:49:24Audience MemberIt's more a tool. But there are some domains and some things that it will create, obviously.
1:49:30Elon MuskRight.
1:49:31George HotzI promise you guys that this stuff is not true. There is no type of creativity that the humans have that is unique to them. You say the AI is just a synthesis of already existing ideas.
1:49:45George HotzThat is all humans are.
1:49:48Co-hostRight.
1:49:48George HotzThere is absolutely no difference. And you know what? Everyone who. Everyone who doesn't believe this is going to be in for a sad awakening.
1:49:59Co-hostI think I'm almost entirely with you. My one piece of pushback is that AI has these levels of creativity. The thing it doesn't have is understanding. And I think that is where humans excel.
1:50:10Co-hostWhat their ability to under.
1:50:12George HotzWhat is understanding?
1:50:14Co-hostWhen something's working, why is it working? That's something you can use AI to help it give you additional insight on. But the decision, like there will always be a level of decision making that comes down to a human and we will keep moving that higher and higher up and keep getting like to do cooler things as a result.
1:50:31Co-hostBut we're always moving layers. We're not necessarily removing humans.
1:50:36George HotzThis is.
1:50:37Co-hostOkay, so this is.
1:50:37George HotzThis is the. This is a smarter take on it. I still think it's wrong, but I think it's less obvious that it's wrong.
1:50:44Audience MemberI think that it's. That AI I don't think will create new paradigms of thought. You know, like AI will incrementally improve and synthesize a bunch of different things, but we're not going to get a whole new paradigm of green open field.
1:51:00Audience MemberDoes that make sense?
1:51:02George HotzBoth of you are wrong. We've already passed this point.
1:51:06Audience MemberOpenAI wrote that Dota bot a while back and it was inventing new strategies and metas in the game that like pro players, I've never seen. And then they were trying to play against the open AI bot to get better at it.
1:51:21Audience MemberWe're like at the point where we're learning from AI. It's. It's like the humans wrong.
1:51:26Elon MuskOr does it not?
1:51:28George HotzYeah, there's definitely no paradigm type of creativity. Any of this. None of that's true. There is a question of the agency of AI Theo. Maybe that's what you were getting at.
1:51:40George HotzYou're like, well, so the AI is always going to sit below humans. It's just going to be like, does it sit right below executives? Right. Like as you kind of move up in an org chart.
1:51:50Co-hostYes, to an extent. There's a level of like, which product should we Build next. And when different products have different responses from users, which should we double down on?
1:52:02Co-hostWhich engineers or which people don't necessarily make sense to keep. But AI can help guide us to the right decisions for a lot of these things.
1:52:10George HotzBut if you. Okay, go ahead.
1:52:15Co-hostMy concern is the cost of certain, like AI making expensive decisions easier to make is different from AI making expensive decisions. And it will be a long time, if ever, where we should let AI regularly make the expensive decisions for us when we could instead use its insight.
1:52:34Co-hostAnd if we disagree with the AI's decisions and make different ones and up as a result, there is a person who can hold accountable for that.
1:52:41Audience MemberAnd I think that.
1:52:44George HotzDid you see that on Twitter? The IBM memo from 1970. Computers can never make decisions because they can't be held accountable.
1:52:52Co-hostYes, and I do think that's a real problem outside of the dev world.
1:52:56Elon MuskI think one of the things AI
1:52:57Audience Memberwill never have is experience. Learned experience.
1:52:59Elon MuskRight.
1:53:00Audience MemberI can read all I want about childbirth. I'm never going to fully understand that experience. And AI will lack that for basically every human experience.
1:53:08Co-hostAnd I don't after he said that. That way I understand why you were so against my take.
1:53:12Elon MuskGeorge.
1:53:12George HotzOkay, we're, we're the, the, the, the people with takes on AI that really make no sense. I've just remember speakers. I mean like I look, I, I can. The problem with these things is you get at the bottom of what these people are generally saying and they eventually get to something like AI can't experience qualia and you can.
1:53:35George HotzThe AI will never know the touch of human skin and this is just completely meaningless. There is no such thing as consciousness. There's no such thing as qualia. Now to talk about the difference between tools and agents, that is a real thing.
1:53:47George HotzThat is a real thing that has a real science. The loss functions look different, right? So agents usually come when the loop is closed. And to come to your exact example about products, you're like, well, the AI is not going to choose the product, right?
1:54:00George HotzThe AI is going to help the humans. The humans are going to look at how the product's performing. The humans are going to tweak the product, put the new one out, maybe with the help of the AI.
1:54:08George HotzYeah, but why not just close the loop entirely? And if your worry is because you're afraid, well, the computers can't be held accountable when it's terrible. I mean, maybe today where humans and AIs are on kind of similar ish playing fields, humans are actually still better.
1:54:22George HotzSure. But someone's eventually not going to have that taboo and that company is going to crush you.
1:54:32Audience MemberHey, George, I have a question about modern security. What do you think about the new iPhones and the new consoles? And they're much more difficult to hack compared to back in 2017 when there was the iPhone3GS, unlike the Xbox.
1:54:53George HotzThat was a long time before 2017.
1:54:55Elon MuskI mean, 2007.
1:54:57George HotzSo when you say it gotten harder to hack, you can quantify this. You can talk about how many dollars an exploit costs.
1:55:03Audience MemberRight.
1:55:03George HotzAnd the price of exploits has gone up a lot. Yeah, yeah.
1:55:06Audience MemberAnd you have to be a near expert. Like, you can't be a teenager doing this, right?
1:55:11George HotzYeah. The best people are probably still teenagers or close to teenagers. They always are. It's just one of those things that selects for. I promise you, the best exploiter in the world today is probably like 24.
1:55:23George HotzYou know, it's just. I don't know who they are. I haven't kept up with the scene, but I bet you that's true. Because it's just like the way your mind works.
1:55:31George HotzBut, yeah, the cost has gone up a lot. But I don't know, that actually might not be because it's harder. It's harder to say. It's actually more difficult. Like, I'm not sure I could really do it anymore.
1:55:41George HotzBut it's like, you know, then I watched the generation before me, I watched, like, Charlie Miller be like, oh, there's ASLR now. It's hard. And, you know, ASLR was never a thing for me.
1:55:50George HotzAnd now you have these, like, new kids who are like, yeah, of course. It takes nine exploits and we have to break out of three sandboxes. No big deal, bro.
1:55:57George HotzYeah, yeah. No, I was just curious.
1:56:02Audience MemberThe other thing too is I. I was just reading that the, like, Gen Z is less technical, technically advanced than, like, millennials, if that makes sense.
1:56:12George HotzThe youth. The youth, man. The world is falling. The youth are.
1:56:16Audience MemberWhat do you think?
1:56:16George HotzSocrates said that. You think so less technically, the Greeks were complaining about how the youth were corrupt and couldn't do X and they didn't know the virtue. Whatever. Whatever this is.
1:56:29George HotzThis is a meaningless thing.
1:56:34Audience MemberHey, George.
1:56:35George HotzAll right, cool.
1:56:36Elon MuskHey, George, I just have a question that you have any unfinished dream ideas for Twitter or weren't they listened or anything that you still feel that is pending with Twitter?
1:56:49George HotzNo, look, I don't work at Twitter anymore. I don't know. I think one of the main things it comes down to is Elon wants to build features. And I would do refactors But I did like that Elon said that.
1:57:03George HotzEven if it's not me, I do hope the Twitter employees get some good, quality food. The food matters a lot to me. It's important to me. Come on, Jordan, you might need to
1:57:12Audience Memberchange your profile picture to I do not work at Twitter anymore. Don't ask me Twitter questions.
1:57:17George HotzYeah, we're gonna get a few people who ask me Twitter questions. I'll start coming up with good, snippy responses. Maybe I'll start spreading misinformation. I spreading misinformation. All right, cool.
1:57:29George HotzThank you, everyone, for being in my space. Check out Comma's Twitter tomorrow. We got a fire video dropping. It's going to be good.
1:57:36Co-hostDM me so we can schedule that podcast.
1:57:38George HotzAll right, sounds good. I'll follow you.
1:57:40Audience MemberDM me.
1:57:40George HotzYeah, Cool.