WARNING: Summaries are generated by a large language model and may be inaccurate. We suggest that you use the synopsis, short and long summaries only as a loose guide to the topics discussed. The model may attribute to the speaker or other participants views that they do not in fact hold. It may also attribute to the speaker views expressed by other participants, or vice versa. The raw transcript (see the bottom of the page) is likely to be a more accurate representation of the seminar content, except for errors at the level of individual words during transcription.

This video explores the need for new metrics for economic policy making beyond the traditional GDP measure. It suggests looking at production and consumption, well-being, and productivity and output from factories. It also considers a dystopian future where post-scarcity conditions could exist, and suggests creating a scorecard to measure the robustness and complexity of civilisation and alternative measures of success. The conversation also covers Artificial General Intelligence (AGI) and the need to track relevant metrics to ensure survival.

The speaker discusses the need for new metrics for economic policy making beyond the traditional GDP measure. They reference the formal theoretical framework for disruption at Rethink X and suggest that alternative measures, such as measuring production and consumption or well-being directly, should be considered. They raise the issue of deflationary regimes and suggest looking at productivity and output from factories to measure economic activity. They also raise the idea of a dystopian future where post-scarcity conditions could exist, prompting the need for new measures to evaluate prosperity and human well-being. These measures need to combine direct physical processes and flows of material and energy with an updated evaluation of more abstract stuff.

Adam and the speaker are discussing the idea of abandoning GDP as a measure of well-being in favour of something else. Saudi Arabia is attempting to shift its economy away from a resource-based economy to something else and is measuring success by tracking production capacity, raw flows of material and energy, and per capita energy consumption. The speaker suggests creating a scorecard that measures the robustness and complexity of human civilisation and its ability to evolve, as a better predictor of civilisation’s success in the long-term. They also suggest looking at other measures of success such as access to opportunities, autonomy, and the ability to make decisions without coercion, rather than focusing on material progress or energy consumption.

The conversation discusses the potential of Artificial General Intelligence (AGI) and the need to measure and track relevant metrics to ensure survival. It suggests that decision makers such as world leaders need to pay attention to the implications of technological advancements and maximise intelligence. The speaker reflects on the current situation and the inevitability of a catastrophe, predicting that a polarization of responses may emerge, with some taking it seriously and others denying it will ever happen. This is similar to the Cold War, where there was a switch that could be flipped and the outcome was uncertain.

The speaker discusses the possibility of new metrics to inform economic policy making, beyond the current GDP measure. They reference the example of Australia, and how it's value of goods and services produced, income and expenditure are useful indicators of productive activity and prosperity. They also mention the formal theoretical framework for disruption at Rethink X, which states that disruptions alter the cost structure of an industry and necessitate new metrics. They do not draw a direct parallel between GDP and technological disruption.

Disruption can cause key measures of macroeconomic activity, such as GDP, to become less useful. An example of this is when measuring the performance of the automobile industry, metrics such as fuel efficiency, cost of fuel and miles driven per year are useful, but when the cost of a passenger mile is negligible, other metrics such as total passenger miles traveled become more important. As a result, it is worth considering alternative measures to GDP, such as measuring production and consumption, prosperity, or well-being directly.

GDP is used by policymakers and individuals to make decisions, but it is not the only way to measure economic activity. It is possible to switch to other metrics to measure the same thing, but it is not clear that this would be the best solution. It may be better to look for different things to measure, rather than just changing the proxies. This could help to provide a more accurate picture of the economy.

The speaker raises the issue of deflationary regimes and how this could lead to productive activity ceasing to be measurable using GDP or its components. They suggest looking at productivity directly, and consider the output from factories to see if they are producing useful goods that are being consumed. They raise the idea of a dystopian future where factories are producing useless rubbish with no value, and where post-scarcity conditions could exist. The speaker is concerned about how this could affect economic activity and human well-being.

The transcript discusses the need for new measures to evaluate prosperity and human well-being, as existing tools are not able to measure the physical reality of a situation. This is illustrated by two thought experiments: an automated island with all the resources necessary to meet 100% of the demand for products and a 3D printer that produces whatever is needed from dirt and garbage. New measures need to combine direct physical processes and flows of material and energy with an updated evaluation of more abstract stuff. This is something that has already been done to an extent.

Saudi Arabia is attempting to shift its economy away from a resource-based economy to something else, such as technology. However, incentives are misaligned, making it difficult to invest in these fledgling industries. To measure success, Saudi Arabia should track the production capacity and raw flows of material and energy, as well as the per capita energy consumption. In the future, when machines are able to produce limitless energy and material goods, it may not be necessary to measure the output of the sun, as the output of machines will be more important.

GDP has traditionally been used as a measure of success, with higher GDP being associated with increased prosperity and human happiness. However, it may be possible to decouple this relationship in the future, with a focus on other measures of success that are not linked to material progress or energy consumption. This could be achieved by creating a scorecard that measures the robustness and complexity of human civilisation, as well as its ability to evolve. This could be a better predictor of civilisation’s success in the long-term, and ensure that humans remain an integral part of the system.

The transcript discusses the concept of measuring human well-being directly, rather than just measuring material wealth. Access to opportunities and choices, autonomy as an agent, and the ability to make decisions without coercion are all measures of well-being. To achieve a post-scarcity economy, it may be necessary to have a non-uniform distribution of material goods and services, even if it means creating a few trillionaires in the process. Ultimately, what matters is achieving a condition of super abundance, and measuring what is ultimately important.

Adam and the speaker are discussing the idea of abandoning GDP as a measure of well-being in favour of something else. The speaker believes that in a world of abundance, people will no longer care about a single metric and will instead track different numbers that are more personally meaningful. The speaker also suggests that predictions about the future should only be made for a relatively short time horizon, and that this is a key feature of their team's research.

GDP has been a useful metric in the past, but it is becoming less and less useful as we move towards a future characterized by abundance rather than scarcity. Policy makers, investors, and civic leaders should be prepared to look at other measures, such as how much is being produced, how much energy is being used, and how much distributional equity there is in society. We should be prepared to move away from the economics of the past, and look at the details more carefully to ensure a better future.

The conversation discusses the importance of measuring relevant metrics, such as childhood mortality and GDP, to guide action and ensure survival. It is suggested that moving forward, metrics such as the number of people working in AI safety and the number of GPUs produced should be tracked instead. The conversation then pivots to the current situation, with the news reporting on topics such as Common Sense Physics and 3D versions of Dolly, which is causing anxiety and fear. The situation is referred to as the foothills of the singularity.

The conversation discusses the potential of artificial intelligence and the role of intelligence in the allocation of resources. It suggests that decision makers, such as world leaders, should pay attention to certain metrics and be aware of the implications of technological advancements in AI. It also suggests that the advice they receive may differ from what they ought to be thinking about and being advised. The conversation highlights the importance of intelligence in various aspects of life, from driving a car to walking across a room, and encourages the maximisation of intelligence.

The speaker is discussing the implications of Artificial General Intelligence (AGI) arriving by a given date and how this affects life decisions and allocation of attention. He reflects on his own experience with the early stages of the COVID-19 pandemic and how it felt to take actions that were different from the people around him. He questions what will happen when the belief that AGI is coming within a decade becomes widespread, noting that this could transform almost every aspect of our civilization before it arrives. He admits he does not know how to feel about this and is unsure of his intuitions.

The speaker is of two minds about how people will react to the looming inevitability of a catastrophe. On one hand, they could take radical action, but on the other, some may ignore the situation entirely. The speaker predicts a startling polarization of responses, with some people taking it seriously and others denying it will ever happen. They suggest that a parallel situation could be the emergence of AGI, which could be the best or worst thing ever to happen, making it a strange and unprecedented situation.

The Cold War and the Space Race provided a cultural backdrop for the development of technologies that could both lead to nuclear annihilation and to human flourishing. This is similar to the situation we are in now, with the potential of Artificial General Intelligence (AGI) to cause a rapid and abrupt phase change in the world. The comparison to the Cold War is apt, as it was also a situation where there was a switch that could be flipped, but with half of the population not believing it and the other half being aware of the potential consequences. Ultimately, the development of these technologies can lead to either catastrophe or rapture, and the outcome is still uncertain.

this third question uh is you know the do we need a new do we need a new set of of um you know sort of sort of I don't know what the right word is but sort of my Keystone or Cardinal measure or something you know it's basically fundamental metrics that are broadly agreed upon as uh uh um uh primary indicators for the guidance of economic policy making and and if GDP is eroding if the value of GDP is eroding we've sort of discussed that and you know I don't know if we've come to affirm position on whether or not it's it's its value is going to um default zero or or just sort of continue to evolve as it's measuring different things but at any rate even setting that that stuff aside I think it is worth at least at least uh posing the question and maybe spending a few minutes talking about um the possibility of new uh measures and what we would want what what we might be able to inform policy making with if it weren't uh if it works GDP as it's currently formulated right so that so today we saw from the example of of Australia that it's the uh value of goods and services produced its income and it's um um expenditures reported expenditure right so those three things sort of sort of production income reported and expenditure uh reported and and um we spoke before about why those are useful I mean they they sort of are uh uh maybe with a few you know distance of a few layers of abstraction they are uh indicators of productive activity and you want your Society to be productive because production again fairly decent proxy for the um uh prosperity and prosperity then a decent proxy for human well-being and happiness um and Imperfections in every layer every every step of the course so they I think this is a worthwhile question I was actually talking with Owen about this a little bit um and uh one thing other thing that I should mention is that part of our formal theoretical framework for disruption at rethink X part of the SEMA technology disruption framework is the idea that um uh disruptions radically alter the cost structure of an industry and that make old business models a way of doing things um uh and viable they tend to necessitate new metrics and this is so this is this is it this is broadly of a piece with that part of our framework what I hesitate to to sort of draw too many direct parallels with is the idea that this is that that GDP is somehow being disrupted it's not a technology disruption in the same way that other things are but at any rate if we're thinking just just more

uh uh by analogy as opposed to sort of having GDP be a direct example of disruption itself disruption is that as the previous primary indicators and metrics foreign industry's performance or for um uh uh the um a sector of the economy's productivity like I'll give you a good example a good example of that is um uh we we measure in the case of in the case of um Automobiles and transportation um we uh we measure um we have measures like you know miles per gallon as an indication of efficiency we have measures uh like um so that's that's sort of a measure of fuel economy we have measures of um miles driven per year we have uh measures and metrics like the cost of the fuel and so on um and some of these are useful and will continue to be useful even with new technologies uh providing the basis of Transportation but then in others in other instances they're not so useful because the um we care about fuel efficiency when the marginal cost of a passenger mile or or a cargo ton mile is high but when the cost is negligible we don't care that much about the um that that cost we care about other other metrics such as for example total passenger miles traveled might be a more important metric to track than the car than the um the the fuel expended per passenger model for decision making purposes in transportation so that's an example of how um uh what is what metrics are and what indicators are meaningful for decision making can change when the cost structure fundamentally shifts and in particular when the cost structure um when the cost over all four so we've seen this in other Industries Industries for specific product segments for specific markets we've seen it for entire sectors of the economy like Transportation information energy and um so we can Reason by analogy there that that we might expect key measures of macroeconomic activity as a whole and GDP being a major one just maybe you know follow some sort of analogous pattern okay um so so so GDP ceases to be useful what can we what should we do instead should we try to measure production and consumption in some other way should we try to measure prosperity for which those are a proxy as in some other way or should we try to measure human well-being for which Prosperity economic Prosperity material prosperity and abundance is a proxy um should we try to measure well-being directly um this is I think this is a good question um it's worth spending a few moments you know maybe just conjecturing about what alternative measures might look like and

how to frame them yeah I don't know but in order to frame that question maybe let's go back to so gbt GDP matters insofar as it guides action so what actions do we want to take uh and how could they be differently informed so what do you want where does GDP actually influence the things that humans do well it's policy makers try to make economic decisions or decisions about interest rates or of a macroeconomic levers based on GDP estimates individuals make decisions based on GDP figures um you can certainly imagine that even if GDP didn't exist it's not like policymakers would simply not have any clue what to do right uh the it's it's almost like well it's not so clear that even they're paying attention to GDP necessarily right suppose you've forbade any politician or Economist from using the term GDP or talking about it well they could of course pay attention to these individual factors that go into GDP right they could just like suddenly care a lot about where the factories are producing at capacity or not or what the average price looks like for a given good or uh there's lots of other I'm not sure if the analogy is accurate but maybe you've heard this story about IQ right I mean uh there are IQ is one way of measuring this intelligence Factor G which is kind of common across many tests you might design for cognitive labor you come up with a an array of tests that seem to require reasoning and memory and and so on and there will be one overall kind of factor which comes out of that analysis and it is correlated with IQ but it isn't IQ you know like IQ is a is measuring some underlying thing and many other tests would also measure that thing so you maybe GDP is a bit similar where uh it isn't you could switch to some other set of metrics and just get something that essentially is GDP just in a hidden way so maybe we should be clear that we expect any you know if you if you go out there and measure 10 other proxies for economic activity well if we can just switch the proxies we use to some other set of things rather than production consumption and investment uh if we can switch them to some the average of some other set of metrics then well fine actually the problem is Trivial right it's just like okay measure some different things I think we have in mind that that whole Prospect just fails somehow that you will not find well maybe there's maybe we should divide into two cases do we think we should just find different things to measure or do we think that it

is fundamentally unmeasurable like it just disappears from any measurable radar inside sense yeah that's that's the thing that and just as this circles back to my original concern and the reason my reasoning for raising the issue and and why my colleagues Tony and Jamie they're they're when they brought this to my attention that this potential of a sort of a deflationary uh regime similar to you know because we see this in technology you know the costs fall over time um that my concern originally was that was that things would cease to be measurable using either GDP or its component portions but but that those things would still be meaningful for for material prosperity and human well-being and that um uh that that if if the if productive activity we're continuing to occur but it were no longer participating in markets and and and and the sort of explicit formal economy in the way that we are accustomed to then this would be very confusing and so we would have what we need to be sure and I think you mentioned this is we would need to look at productivity directly and not look at it through the abstraction layer the abstract lens of um uh of of of finance of of of currency of money of you know of exchange and how that's represented so I liked your suggestion for example of seeing well what is the actual output from factories is it are they producing at capacity um are they producing things that are then being consumed so I mean one could imagine a dystopian situation I think there have been some throughout history where factories were were cranking away but they were producing useless rubbish that that offered no marginal value no value no no uh they weren't producing things that consumers actually wanted and so they weren't doing you know they weren't they weren't um uh they weren't contributing to prosperity and well-being um at least not not at least not via VIA consumption into the by the end user they might have been contributing in some way through stimulus of economic activity more generally but um not in the way that you know one would imagine they were purposefully intended um or aiming towards right so that's the sort of thing that I'm I'm concerned about and and I mean again we can kind of think to we can sort of leap forward to uh a a condition in the future and it doesn't it need to be that far ahead you know 25 30 years something like that where you have sort of a post-scarcity condition and one could imagine I mean imagine you had an i offshore your country it's a big place

um but fully automated and on that island you could do all the mining necessary to obtain all the raw materials necessary to supply all of the production facilities necessary to manufacture every material products that your Society consumes in in all and meet 100 of the demand for those products and it was all automated and you know you no human activity at all just just 100 automated and the material base right the raw material resources were there on that in that Geographic endowment well you you you how would you how would you how would you measure that if if there were to market for any of those things I mean that's sort of the goal if if if that's a condition we would like to achieve um you know uh how does that that would not show up on any of the metrics of um uh expenditure or consumption or income because at least not as we currently conceive of them because they're all monetized through the mechanism of markets right and and you could imagine a situation where that's where that's not the case that just all the prosperity is there um it just isn't legible it is invisible to the uh to our existing tools which are all measure these things in the abstract but the actual physical reality would obviously still be present and um so that's one sort of thought experiment you could have this imaginary hyper productive Island offshore another thought experiment that's similar is you know everybody could have sort of a 3D printer that says Star Trek style replicator and you just put your dirt and garbage in the input and you get whatever you want as an output magically and you know and and um so there's no market for material Goods because they can all be just produced trivially uh in that case you would need some other way to evaluate prosperity and the human and the human well-being that comes from that Prosperity so my question is you know between here and there between here and that and that endpoint um for question three which we're looking at what would what would new measures need to evaluate and presumably it would it would be some combination of more direct measures of the actual physical processes and flows of material and energy and then paired with a a an updated um evaluation of of you know of more abstract stuff um that's measured at several levels you know one or two one or more levels of of um uh removed from those uh but but yeah um and then yeah and I I do think that some of this has happened already I mean some things like the things many of the

things that your phone can do your smartphone can do I've already been through this process I've already through the Looking Glass here on some of those things hmm um so yeah I mean one one thing that I would say is is what you recommended or or at least what you suggested is uh just look at the production capacity and see if it's being utilized I mean I think that would be one one very logical very obvious thing to track another would be if you could if you could track raw flows of material and energy um that would probably be useful even Just Energy might be a pretty darn good proxy for what's moved for what is happening for what's moving because Energy power literally Powers every other process including intelligence right in humans processes it Powers it with via food and it's going to be uh that's a bit indirect right a few steps in the chain there but energy is really going to vary quite straightforwardly power artificial intelligence and so um uh maybe this energy alone is a is a good or is it a more useful proxy than we're we're um yeah like maybe maybe a per capita energy consumption is is the kind of thing you want to be looking at um well yeah here's a question so Saudi Arabia is trying very hard to shift its economy from a resource economy onto something else maybe Tech or you know it seems like they're trying everything uh how do they measure their success so of course the GDP figure for Saudi Arabia is I don't know what the percentage is but it must be almost entirely resource output [Music] but they how do they incentivize the development of a parallel economy I haven't thought much about it but presumably they're they're key problem is is something like it's very hard to align incentives around complexifying some segment of your economy when the returns to uh investment of your personal capital and your career into just gaining more of the rents from resources so heavily outweighs investing your time and effort in in building up a career in one of these fledgling things that you have no idea will succeed or not um I wonder okay so I mean maybe I'm making as I think it's probably clear analogy to where we might be in in some number of decades right where well suppose that machines do there's virtually Limitless energy and machines produce all our material Goods do we really care about measuring you know like on some kardashev scale like how much of the output of the sun we use uh like we're tracking that number why why do we care like it's like we're just

keeping score uh it's it's no it's no longer like okay it might be a reasonable thing and it might actually be like a good predictor of where our impressive civilization will be in a hundred years or something like that but it's no longer serving really the same purpose that GDP serves now which is kind of a guide to organization right it's a signal it's like a part of the central nervous system of our civilization right it's like a pain signal or something that guides in a cybernetic sense how we change the organization of our system so maybe GDP and Saudi Arabia serves a bit already like that where it's that number you can change the number as you like you just pump more oil or less oil it almost doesn't matter um it's this other number that you know for whatever reason you know maybe because you think that can't last in the case of Saudi Arabia and in the case of our future selves maybe we don't think it's psychologically healthy for civilization to just sort of collapse into this very simple system where the machines do everything and we lay on couches and eat grapes uh there's kind of like a value preference for the human part of the civilization to remain robust and complex and evolving and if that's what we want then we should measure that as a way of keeping score and maybe that's got nothing to do with material progress well I can certainly see a decoupling from Material progress and energy consumption and so forth I can see that uh you know arising like like I can see I can imagine a scenarios where we move towards that and really quite rapidly as well um right for right now or up until now and at any rate um GDP has been a bit of a score card it's been a good indicator for success I mean if if because you want prosperity and because Prosperity is a good thing because it translates into human happiness and well-being broadly speaking you know the more GDP the better and so you want as much of it as possible and you're it's it the result you of GDP that you get at the end of the year is an indicator of how well your policies and decision making of other you know allocation of resources and your rule making and your you know fiscal and monetary and other policies and so forth whether that that actually translated into what you what you know the the productivity which translated into the material Prosperity which translated into the human happiness that you really want but I can see all of that becoming decoupled and and then yes what we would really

want to measure instead is is human well-being directly um and maybe not even human but just well-being like the well-being of conscious systems in general and we I can think of you know a number of fundamental measures there that would be you know pretty important um uh access to Opportunities you know access to real choices um a lot of it would be about access uh you can are you can you make the decisions you want to make do you have the sort of the sort of um you know pun intended autonomy as an agent that you'd like to have are you being coerced by either directly by other agents or or uh coerced indirectly by systemic forces at large um so that you're not making the choices you would otherwise make if you were if you um uh you know if you could wave the proverbial magic wand and so I can I can imagine I think we would we could cons you know construct measures of well-being in a sort of post-material economy as a sort of a post-scarcity economy where there's there's sufficient abundance because then it really all comes down to access um uh I'm saying access rather than than sort of distribution because presumably that in a sufficiently abundant system it doesn't really matter how the how the materials and energy are distributed per se there's enough if if there's a super abundance then they they exceed needs and then in the distribution you could imagine a circumstance of sufficient super abundance where you could have a a non-uniform distribution of material goods and services and it would still be um uh you know Supply would still vastly outweigh any individual person's demand so you can imagine you know certain you don't have to have like perfect uniformity and perfect uh fairness of distribution of goods and or of material you know a material uh uh well yeah materials and energy you wouldn't you don't necessarily need to aim for perfectly uniform distribution what you really want to aim for is just super abundance and it may be that the way to get there is to have very non-uniform distribution for a time especially leading into it it could be that we you know really really do want to create a system where yeah it sucks to create a few trillionaires in the process but that's the way to maximize the pat you know our trajectory to a true condition of super abundance um and and yeah and then you would want to measure you would want to measure um what you really ultimately care about and um yeah but I I think oh sorry yeah I don't maybe I'm just being convinced

that uh it's time to not care about GDP or any substitute you say measuring well-being but I I don't see any world in which that's a reasonable thing okay so you know maybe being the the Christian fundamentalist that you are Adam maybe you want to maybe you want to count how many school children have memorized X number of verses of the Bible and that's your your measure of How likely their souls are to go to heaven and that's well-being you know why would I mean you don't see in Star Trek people running around like caring about some number that's like an overall score for their civilization why is this even just something we'll do I mean okay so the Borg invade and then we want to count how many ships we're making because we care about that well we have lots of numbers we may care about for different purposes but it just may no longer be the case that there's any meaning to a single metric because they're like these things like well-being just become so decoupled from production that I mean it's already kind of the case right for many people who find they get their entertainment off YouTube they they make you know their their income a lot of what they enjoy in life is not actually tied to their expenditure perhaps right in which case do they care if GDP goes up and down that's sort of you know nothing nothing to do with them so aren't isn't this isn't the answer to these questions just GDP will last for a while longer then once we hit super abundance okay we'll we'll track the amount of energy we produce and the number of solar panels we have because some people care about that and people who don't care will track some other numbers perhaps but they just may not any longer be like a single number we all kind of care about vaguely that's okay yeah I I think that I mean again the the and you've heard me say this before um uh part of what my team does part of the discipline that we maintain um and that I'm you know hold this very very firmly too is we don't make any predictions out past it you know a 15 year or so time Horizon then that's that's telescoping shorter and shorter I'm just gonna be you know if they were come there will come a point in the not too distant future where you can't make any predictions more than five years ahead because it just won't be meaningful but still so our focus is sort of on the between here and there and that's I think it's one of the distinguishing features of my team's research and work um compared to you know um other other future oriented you know

futurists and future oriented um the foresight uh kind of operations you know the folks who are trying to generate inside out we'll certainly get to a point where where none of this it doesn't matter it just doesn't make any difference and we we sort of achieved super abundance and then we hold new sets of to be concerned about that will be difficult to imagine today and all the rest of that stuff so between here and there what does it look like probably GDP erodes in its usefulness and that's probably the the the advice to policy makers to investors Civic leaders to other decision makers that that I take away from from this entire exercise and thinking through all of this which is that you know GDP has been a pretty useful metric in the past for all its flaws uh but it is going to become less and less useful uh there are going to be larger larger portions of productive human activity that arcs well indicated within that measure and we probably in the interim between now and you know this a condition that doesn't lie too far off of uh quite radically different um uh physical and economic circumstances characterized by abundance more than scarcity uh between here and there we ought to think carefully about watching other uh maybe measuring these things a bit more directly as well like keeping an eye on how much are we actually producing how much are we actually how much energy we're actually using um uh you know what proportion of our society actually can make choices that will lead to quality of life you know and that's something about that's something more like distributional equity and so forth um uh that sort of thing and so again you guys guess the general picture is that is that we ought to we ought to be prepared to relinquish GDP as a as as sort of the the um uh our primary macroeconomic policy making um indicator and and start looking more carefully at the details and with the knowledge or with with the the foreknowledge that um you know we're we're we we are going to be shifting away from from uh you know the economics that we have had for hundreds of years to something quite different in the not too distant future I guess that's my general takeaway yeah I would agree except I I don't I don't really see why it's important to come up with some way of measuring say whether people feel they have a sense of agency and they can make choices I feel like a situation for the next decade or however long it takes is we're trying to thread a needle at very high

speed and on the other side we are the survive in which case we're in Utopia or something like it and and who cares about all these issues right uh so if you're measuring something it's to guide action and the actions that matter between now and when we hit the eye of the needle uh do we get through the needle and how much survives uh so we should be measuring things that are relevant to that problem and everything else is just kind of secondary so we should keep measuring things like childhood mortality and these other figures that we track as part of Public Health and as part of other things we care about other policy areas we should keep measuring numbers but I think GDP why why do we care about it it's because in sort of the you know for the last century or so the primary driver of human progress has been economic growth and that was something we influenced as humans by building machines and organizing capital and this number helped us do that but that's about to end whatever comes next right it's not going to be the case that we want humans sitting around allocating capital and we're building machines all of that is kind of about to pass away and what we want to measure is maybe like the number of people working in AI safety and how many resources they have or how many gpus are being produced and you know track those numbers as a way of having a kind of metric for how close we are to the needle maybe um because that's all that matters yep I I agree with that um uh well that maybe this is a good way to Pivot out of this conversation and back into the thing which is really Weighing on me and I know it is on you too um which is and I like that metaphor of having to you know basically thread a needle at high speed I mean like like to thread a needle by trying to throw the needle across the room at the eye of the throw the thread across the room at the eye of the needle um uh yeah it's a it's kind of crazy um I mean I'm just seeing I I just I don't know what I'm gonna see in the morning each morning now when I wake up and open you know open the news as it were I just don't like this morning it was you know Common Sense physics um and uh uh there was something yesterday oh um you know a 3D version of Dolly basically um you know I just don't it it's it's it yeah it is it is you called it the foothills of the singularity which I I love I think that's definitely the um it certainly feels like that but so so yeah I mean I I I know we've we keep treading on this ground but it's a

conversation I certainly haven't exhausted my interest in it so if you want to Pivot back into thinking about you know where does intelligence and artificial intelligence where does where does the the the the um the maximization of intelligence factor into into all of this because that seems like that seems to be what we're and we are held it on maximizing on increasing that capability and that's you know there's even through these old economic metric lenses there was a role for knowledge and intelligence in In Those Old way of ways of looking at things there's certainly you know intelligence is behind allocation of resources capital and labor intelligences behind labor itself right I mean that's the uh I think we underestimated that terribly for a very long time but you know to go it takes a lot of brains to drive a car um and it takes a lot of brains just to walk across the damn room we make we make a lot of things that require a lot of intelligence seem very simple um because we're so you know we're quite extraordinary biological machines but man as we start to really get close to um uh notice this potential to to produce arbitrary large quantities of intelligence holy moly so um yeah what are the things that I guess I guess maybe maybe a thing to think about is how should decision makers how should policymakers be reacting to what's going on to these technological advancements in AI like what is the right posture and what are the right things to pay attention to metrics is one of the things you mentioned but there's other things what what should what should what should leaders and decision makers be thinking about be aware of be paying attention to leader I'm thinking that not so much the scientists thinking about the control problem and that sort of thing that that's that's a fair amount of reading and had quite a few discussions about that but if you know I think we asked this question a number of weeks ago like if you could sit down and talk to the president of of the United States or the prime minister of of Australia or that you know the uh some other world leaders what would you tell them if they said okay what do I need to know what what should we be watching what would you what do we say what what should they be being advised and then this is quite different from what they are being advised as well we can also try to imagine what they're being told um and how that might differ from what they what they ought to be thinking about and being advised but anyway if

you had any thoughts I think you have I've been judging by by especially um I know you've been thinking about China in particular yeah um yeah I think it's a good time to Pivot back into that uh I'll add on to that so that's a first question what what should policy makers be doing uh a second question is sort of more personal uh so as our level of certainty about AGI arriving by a given date Rises how do we make our own uh judgments about life decisions and allocation of attention how do we make those consistent with those beliefs I I find myself on a daily basis kind of it's weird to have a belief that you feel fairly sure about but which obliges you to do things that feel strange and you don't know if you're just maybe going a bit crazy right so I you know I've had this experience now with covert as well at the beginning kind of being one of the earliest people in Australia or at least in my circle to really know it was going to happen based on just watching China very closely because lavender was and it's it's such a surreal feeling and very hard to act on even if you know it's for someone who's you know quite in my head which is where I spend most of my time and I have a lot of confidence in my rational judgment I guess and it's the basis of my professional life but to be able to take actions which uh very much different to what everybody around you is doing is um it's not easy so that's one question but I think to maybe tie these two together and ask a kind of descriptive rather than normative question what is going to happen when the belief that AGI is coming in 2030 or before becomes widespread because it's not currently at least not seriously random people might see what's going on and be like hey this is pretty crazy but to approach something like a consensus in society that we have less than 10 years until this on-demand intelligence at the level of a median human say sitting in front of a computer they can do any it can do any task a human can do a median human can do it a computer to approach a widespread belief that this is coming within a decade will transform almost every aspect of our civilization much before it comes right so I think this is the the thing that I think is being underestimated the degree to which this shadow of the uh event will reach back and start to derange things as they are now and this I think could happen any moment almost yeah this is a fascinating question um and I don't know how I feel about I don't know where my intuitions are I'm

on the one hand I would not be surprised at all if the long Shadow backwards cast backwards really did you know start to you know people really did start to lose their minds a little bit and go a little bit nuts as this as it as the reality the inevitability of this looms I can see that happening but at the same time I can you know the other half of me I'm of two minds and the other mind of in another mind I can see you know people just just you know walking right up to the precipice of this and even afterwards and and Blissful or deliberate you know denial or obliviousness about that this is coming and and and so the and and I I kind of think that it could be it could be both there could be radical polarization of the responses right that like you know from on the one hand well you get people doing starting to who really you know showing the courage of their convictions really starting to take kind of what seemed to be quite radical action based on on um uh you know on this the belief that this is this is imminent um and you could have people being quite radically obstinate and and just refusing to to accept that you know that it is it's imminent and you so what I I don't know if I would I don't have any confidence one way or the other in any of this but I what would not so what would I suppose if I had to make a guess I would guess that we will see a a startling polarization of responses with with increasingly um yeah an increasingly wide division a divide between people who who are taking this seriously thinking wow some things can happen and people are like what are you talking about this is nonsense it's you know it's never going to happen kind of thing and um uh we might look to to examples from history for that uh so I think that one the examples to look for are looming catastrophe but having said that having said that this is also a strange situation I'm trying to think of a I think a parallel situation I honestly can't um because you know let's let's just call it um you know a singularity okay I know there are some negative connotations but let's just say we get AGI and some sort of Singularity you know um uh some sort of uh okay never mind but we won't go down that alley let's say we get AGI um uh around about 20 30 something like that and in this case what we don't know in advance is whether it's going to be the the the the worst thing or the best thing ever happened and so this is a strange this is very strange and I may perhaps without precedent because there have been

situations where catastrophe lubed what I don't know that whether or not there have been situations where where something momentous loomed and it was unclear whether it was going to be a catastrophe or a rapture and that this is really strange right because even if it reminds me a bit of well it reminds me a bit of When We Were Young I mean I remember growing up with books that I bought which were about how we would have space colonies and you know be traveling through the solar system by 2020 and and that was all of course driven by the rise of rocketry during the Cold War so the the dark side of that was nuclear Annihilation right that was kind of like this Jewel thing where we wrote the I mean I I wasn't thinking about that as a kid but that I suppose is the the cultural background which produced as its kind of Jewel mirror image this idea in human expansion right so I I would say the Cold War maybe had this similar feeling in some respects of the the same kind of set of Technologies or at least the same period of progress producing both the the terror of annihilation and and the prospect of kind of human flourishing beyond all previous limits yeah I guess that's I guess that's true yeah I guess that is that but was there was there I mean it it yeah it's I mean this just seems so radically just so uh that's the right word um I mean the the conditions here are so Stark they're so uh uh you know the the the the the the the phase change is going to be so abrupt so rapid I mean the day before you have AGI in the day after could be quite literally the days could be quite different and I suppose you know maybe that the after the the successful Trinity test and the end you know with new the development of nuclear weapons maybe it was something you know similar to the world really fundamentally just was different the very next day um but in in the analogy that you're drawing to sort of the Technologies behind the Space Race that were also the Technologies behind the Cold War and a potential sort of you know nuclear Armageddon um those didn't have sort of a uh you know a a a switch that was you know at some point could be flipped and and you you know you had half the you had half or more of the population just not even believing there was a switch and then half the population thinking okay well there's this switch and it's going to flip pretty soon and a holy crap it's gonna flip and then if it flips it's like you know it's either going to be Moon basis or it's going to be you know