WARNING: Summaries are generated by a large language model and may be inaccurate. We suggest that you use the synopsis, short and long summaries only as a loose guide to the topics discussed. The model may attribute to the speaker or other participants views that they do not in fact hold. It may also attribute to the speaker views expressed by other participants, or vice versa. The raw transcript (see the bottom of the page) is likely to be a more accurate representation of the seminar content, except for errors at the level of individual words during transcription.


Technology is transforming education, with AI tutors potentially disrupting the traditional lecture format and university experience. AI tutors have the potential to reduce costs while providing tailored content and motivation. The pandemic has made homeschooling more popular, while universities are now providing recreational and social opportunities. Balmall's cost disease is mentioned, as well as the importance of Google and Apple providing training and credentials. AI tutors could revolutionize the educational system, but questions must be asked about potential risks and pitfalls.

Short Summary

The speaker is discussing how technological disruption has not yet been seen in the field of education due to the phenomenon of Balmores Cost Disease. This suggests that wages in areas that are not increasing in productivity can still rise, and this is seen in the rising cost of college tuition and fees. It is suggested that this is due to competition with other sectors, and is causing disruption to university education. The discussion highlights the need to compensate teachers for the opportunity cost, and other factors such as globalization and credentialism are also seen as playing a part. The labour market has its own peculiarities, with job seekers actively pursuing employment and an increase in 'bureaucracy jobs'.
The speaker discusses the idea of how technology can disrupt the education system, from the product to the sector level. It is proposed that new technology could provide more value at a lower cost, circumventing the “rent seeking and extreme bad behavior” that is present. Examples are given of how the internet has disrupted certain aspects of higher education and the importance of socialization and networking. It is suggested that the new technology should not just provide a one-to-one substitution of what already exists, but also provide a different way of doing things.
The speaker proposes changing the entry age for the workforce from 18 to 16, and suggests implementing mandatory public service programs for 18-20 year olds as a way to gain life experience and decide whether to pursue college. AI has the potential to scale private tutoring and see the two Sigma effect, where average students become top performers. This could be a world-changing moment for the education system, although the specifics of the phenomenon have yet to be explored.
The paper discussed in the transcript suggested that private tutoring has an effect size of two sigmas compared to a Mastery based classroom. It is proposed that an AI tutor could replace the need for lecturers and tutors, and provide tailored content to motivate students. Balmall's cost disease is mentioned, and how AI tutors could help people to see the current system more clearly. The pandemic has disrupted traditional education and made homeschooling more popular as parents can see what is actually being taught. AI can be used to reduce the need for lecturers to read textbooks, but it is not as effective as actually sitting down and reading the textbook.
AI has the potential to disrupt the traditional lecture format and university experience. AI can provide the best possible version of a lecture and the interactivity of a professor. This technology can also help students with organization and structure, and could provide a cheaper alternative to the current higher education system. Universities are beginning to provide recreational and social opportunities in addition to a classroom experience. AI could disrupt different or all portions of the higher education system, and other technologies such as the metaverse and virtual worlds could also play a role.
Universities have become increasingly focused on extracurricular activities, rather than the pursuit of knowledge, reducing spaces for the exploration of different ideas. To create a successful higher education institution, extraordinary students must be attracted by providing a self-reinforcing loop of prestige. Western universities remain popular due to their lack of corruption and well-functioning institutions, while Google and Apple offer some training and credentials. UT Austin is attempting to disrupt the system, and AI tutors could potentially revolutionize the educational system. It is necessary to ask questions and think through potential risks and pitfalls of new technologies to prepare for the changes that are coming.

Long Summary

The speaker is discussing how technological disruption has not yet been seen in the field of education, despite some people predicting it. This is likely due to the phenomenon of Baumol's cost disease, which states that wages in areas that are not increasing in productivity can still rise. This is seen in the field of education, where inflation-adjusted prices of commodities have declined, but the cost of education has risen rapidly. The speaker is citing a book by Matthew Bulls to support their point.
Balmores Cost Disease suggests that in sectors where productivity is not increasing, costs may rise due to competition with other sectors. This is demonstrated by the US Bureau of Labor Statistics which estimates the cost of the average good has increased by 260% since 1980, whilst the cost of college tuition and fees has grown by 1200%. This is a result of the economy increasing in productivity and universities having to compete with wages in other sectors to employ people. This increase in cost is seen as unsustainable and is causing disruption to university education, with other technological factors also playing a part.
The discussion is about the importance of education and the need to compensate teachers for the opportunity cost. It is noted that the occupation of teaching is not seen as desirable as it used to be, and this is likely due to the fact that people can earn more money elsewhere. It is suggested that this could be affecting the quality of teachers and the education system in general, and it is also suggested that the cost of college education has been increasing due to credentialism. It is agreed that there are multiple causes for the changes in the education system.
The transcript discusses the concept of bobel's disease and the effect of productivity, supply and demand, mechanization, industrialization, globalization, and the availability of skilled labor on the economy. It explains how globalization has created an illusion of free trade, as capital can move freely between nations, but labor cannot. This has resulted in a greater abundance of material goods and an increased availability of skilled labor, but has also caused an imbalance in the global labor market.
The labour market is more mobile than it was in the past, but still has some artificial forces acting on it. Goods and services have different levels of productivity and abundance, and the labour market has its own peculiarities. Job seekers actively pursue employment, and this has led to an increase in 'bureaucracy jobs', which are jobs defined by the employee as either useless or actively counterproductive. In the post-modern, post-industrial societies, services make up a large portion of the economy and approximately 30% of jobs are in the 'bureaucracy job' category.
University Administration has grown significantly in the past 75 years, with the number of administrators almost equalling the number of undergraduates. This is an absurd situation as the same educational outcomes were achieved with far fewer administrators in the past. An anecdote is shared about a full-time position at UCLA which was paid $68,000 a year to process applications for a $500 grant for teaching assistants, despite the fact that even if all the applications were fraudulent, the position still would not pay for itself. It is suggested that a no-questions-asked policy would be more efficient and less insulting to the teaching assistants.
Harvard University has fewer undergraduates than the University of Melbourne. High schools and primary schools in the US are more diverse than universities when it comes to certain issues. The discussion moved to technologies that can disrupt the education system, by changing the way knowledge is transmitted from generation to generation. Disruption can happen at the product, market, industry or sector level, and education is present at all of these. The aim of disruption is to fulfil the underlying need in a different way, using different technology.
The education industry is a sector of the economy which is comprised of multiple industries including higher education, public and private universities, community colleges, vocational training and professional schools. Disruption in this sector could follow the typical pattern of adoption of new technologies, with far-reaching implications. An example of this is the internet, which has caused a disruption at the product and service level. In the past decade, certain things which were once done at universities have been obviated by the internet.
Education is an aggregate of many services, which is why it has resisted disruption until now. Socialization, networking and preparation for productive labour are all part of higher education. Networking is an opportunity to enter new social circles, establish relationships and friendships, and join networks which may be lifelong. The Greek system of fraternities and sororities is an example of networks designed to facilitate networking.
The speaker discusses the idea of socialization and networking as a major component of education, as well as the "perverse financial incentives" that are in place. He suggests that new technology could disrupt the existing system by providing more value at a lower cost and circumventing the "rent seeking and extreme bad behavior" that is present. He suggests that this is a characteristic of disruption, and that the new technology should not just provide a one-to-one substitution of what already exists, but also provide a different way of doing things.
Adam explains that disruption of education can result in a larger system with more participation and throughput of people, energy, and material. He acknowledges that new externalities can be opened up as a result of transition, which can have unanticipated downsides. He advocates for wise and judicious intervention to anticipate and head off any unintended side effects of new technologies before they become deeply embedded.
The speaker discusses the possibility that if we had anticipated the consequences of social media, we could have enacted stricter regulations. They also mention the signalling and evaluative aspects of education, which can be used to select the best employees. They also mention Brian Kaplan's book, Against Education, which looks at the potential truth behind the idea that education is not necessary for success. They conclude by noting that many courses from top universities are now available online for free.
The speaker argues that the cost of attending Ivy League universities is not worth the moments of enlightenment one gets from lectures, and suggests that the system would be better if students waited until they were 22 to start their undergraduate degree. The speaker also believes that the difference in mindset, maturity, and preparedness of adult students versus high school students would completely transform the efficacy of the education.
The speaker proposes changing the age of entry into the workforce from 18 to 16, which would have a massive and overwhelmingly positive impact. The speaker notes that this could lead to a flood of 18-22 year olds in lower-level jobs, as well as differences in the demographic makeup of the labor market. They also suggest that mandatory public service programs could be implemented when people turn 18, which could be beneficial for most societies.
The proposal of public service for 18-20 year olds is discussed, with the idea that they could gain life experience and decide whether they want to pursue a college education. The dream of education technologists is to disrupt education to achieve the two Sigma effect, where average students become top performers with private tutoring, which automation could help to scale. AI could make this infeasible for a large population, allowing more students to benefit from private tutoring.
The two sigma effect is a phenomenon where one-on-one or small-group tutoring can lead to large gains in the effectiveness of the education system. There have been attempts to reproduce the study, with varying results. A team at DARPA is now working on productizing the system using recent advances in large language models. The specifics of the phenomenon have been explored, and it is believed to be related to factors such as attention, motivation, diligence, personalization, and delivery channel. If successful, this could be a world-changing moment for the education system.
The transcript discusses the effectiveness of private tutoring and how it could be replicated by AI. It suggests that understanding the specifics of the one-on-one relationship between tutor and student could help an AI tutor be more effective. The paper discussed in the transcript used a Mastery based approach, where students moved on to the next topic only when they had achieved a high level of competence. This approach plus private tutoring saw an improvement of two sigmas compared to a Mastery based classroom alone.
John Hattie conducted a meta-meta-analysis of education literature to determine the effect sizes of various components of learning experiences. This literature is difficult to work with due to the difficulties of finding identical classrooms for experiments. The proliferation of educational material online has not made much of a difference to the structure or cost of the university system. If an AI tutor were to appear, it would be likely that lecturers and tutors would no longer be needed, leaving administrators to take control of the university education experience.
The proposal of replacing human lecturers and tutors with AI tutors is discussed. It is suggested that AI tutors could be more educationally effective than passively sitting in front of a lecturer, as they can provide tailored content and motivate the material. AI could also shine a light on the 'dirty secrets' of any given area of human activity, such as in the case of Go, where most of the work is just grunt work and memorizing moves.
University lecturers often just regurgitate material from textbooks, and AI tutors could do the same. Matt mentioned that lecturers may be able to answer moderate questions, but automation could replace a lot of what lecturers do. Balmall's cost disease is also discussed, and how prestige is often attached to high incomes. Universities are able to get away with providing lower quality education at a high price, and AI tutors could help people to see this differently. AI tutors could also bring a new ingredient into the discussion, and make the current system more clear.
Parents were shocked to see the quality of material being taught to their children on Zoom during the pandemic. Homeschooling has become more popular as a result. Technology has disrupted traditional education, allowing parents to see what is actually being taught and removing the carefully constructed filters used to make high status things appear better than they are. AI can be used to reduce the need for lecturers to read textbooks to students, but it doesn't seem to be as effective as actually sitting down and reading the textbook.
The lecture format is popular, as it is easy to interact with a person speaking and having a schedule with classes at scheduled times can be motivating. However, AI systems need to learn to use the human interface to naturally communicate with people, or use something like pleasing video animations. The main difference between talking to someone and reading a book is the interactivity, which is considered to be more effective in achieving learning outcomes, as students can ask or answer questions.
The lecturer explains how lectures are often made interactive by introducing clickers and multiple choice questions to ensure everyone is keeping up with the concepts. They suggest that interactivity is a core challenge of automating education, and that a new single technology or convergence of technologies is needed to achieve multiple sources of value simultaneously. They use the example of a smartphone, which can do many things that used to require multiple products to do. They suggest that an AGI, if convincing enough, could do multiple things done in the classroom by great professors and lecturers, such as delivering the best lecture.
The use of AI in education can provide the best possible version of a lecture, as well as the interpersonal interactivity of a professor. In the US, universities have been turning into resorts to attract students, providing recreational and social opportunities in addition to the classroom experience. These resorts are luxurious and expensive, but provide a great retirement community-like experience for those who can afford it.
University experience is a formative time in a person's life and AI has the potential to disrupt the outside the classroom experience. Schedules and structure are important elements of university life that help students get the most out of their experience, as they help to impose order and discipline. There is an industry of apps and programs designed to help with organization and structure, and universities provide this structure already. This is the best of all worlds as students can take advantage of both the structure and the fun and social experiences that university life offers.
Higher education is an incredibly well-run and well-designed system with many opportunities for socialization, discipline, and growth. It provides a degree that gives the student a sense of identity and belonging for the rest of their life. AI could disrupt different or all portions of the higher education system. Other technologies, such as the metaverse and virtual worlds, could also play a role. The high cost of higher education and its tendency to increase fees could be seen as unsustainable and wrong, leading to the potential for disruption.
Universities have become increasingly fixated on extracurricular activities, such as sports teams, which has caused them to deviate from their original purpose of the pursuit of truth and knowledge. This has caused a loss of spaces where different rules and ideas are valued and protected. The speaker suggests universities should do much less and open up space for universities that are focused on the pursuit of truth and knowledge. There is a challenge between the institution and its students to make this happen.
Creating a world-class higher education institution enabled by new technology is a major challenge. To succeed, extraordinary students must be attracted to participate and give the institution credibility. Establishing such an institution is not easy, as it requires a self-reinforcing feedback loop of prestige and great students. This is a difficult task, but it has been done before with great success, such as Carnegie Mellon, founded by two steel barons. Despite this, few of the world's wealthiest people have attempted to create such an institution, likely due to the difficulty of the task.
Western universities are known for their high cost and large class sizes, but despite their problems, they remain popular for students from other countries due to their relative lack of corruption and well-functioning institutions. Google and Apple have their own campuses and hire people without university degrees, but they also offer some training and credentials. A new program could be offered for free, but only to those who are top-notch and amazing. This could create a prestige feedback loop and improve the university system, while providing an affordable option.
The world is willing to pay a premium to attend Ivy League universities in the US, but the current system has its issues. The University of Texas at Austin is attempting to disrupt the system, but they need to find a way to attract the best researchers and students. AI tutors could potentially revolutionize the educational system, but it remains to be seen how it will shake out in the end. This is a familiar pattern in the history of disruption, where new technologies become more accessible and are deployed on a larger scale.
The changes posed by new technologies are imminent and it is time to commit resources to asking how they can be used to transform education, and how to anticipate and plan for potential risks and pitfalls. To do this, we need to ask questions and think through possibilities carefully, so that we can be prepared for the changes that are coming. We should also look to maximize the benefits and minimize the risks, and ensure that our decisions are guided by principles.
The speaker is discussing the challenge of preparing for a major disruption and how difficult it is to predict the future. They suggest that the most important thing to do is to have lots of conversations about it, as this will help to avoid being blindsided by the changes. They point out that there will be some winners, who were positioned well, but that success could have been more widespread if more effort was made to share insight. Finally, they suggest that no matter what, the disruption will still destroy everything of value.
The current education system delivers large economic rents to certain sections of society, such as the upper middle class and university administrators. This rent-seeking could be transferred to investors in open AI, but the size and who receives the rents may change. Disruption of the education system could lead to lifelong learning and a larger system, with shorter degree lengths and the same amount of learning. This could be achieved by increasing the amount of learning per year and reducing the length of the program, or by keeping the three-year program with more effective teaching methods.
The speaker is discussing how AI could potentially change the rate at which people learn, and how this could affect pre-university education and the workforce. It could mean that high school graduates are more knowledgeable, but it could also pose a threat to pre-AI education people who are now competing with those who have learned a lot in a short period of time. However, AI could also provide access to upskilling, which could help to alleviate the problem. Adam then brings up the alternative of certifications and qualifications for specific skill sets, such as software engineering.
Higher education is being disrupted, and it is unclear what the result will be. There are already vocational and professional qualifications and certifications available, and it is possible to become certified in a programming language or electrician work in a short period of time. It is unclear if higher education disruption will create an alternative that is distinct from the existing certifications and qualifications, or if it will be decoupled from the full higher education experience. Features of higher education such as networking and socializing may be affected by the disruption.

Raw Transcript

uh so the topic I wanted to talk about today was disrupting education which is somehow conspicuous by its absence in some sense given how long we've been talking on this topic of disruption in general um so technological disruption of Education I suppose more narrowly there's many aspects to this so we can start more or less wherever we like um I have a few things I want to throw into the conversation but I don't have a clear structure to to what we're about to do maybe I can spend a few minutes just outlining broadly why What technological changes are relevant why some people predicted disruption of Education in the past as a result of some of those technological changes and why this does not seem to have happened um so maybe the clearest indication of the well why do people think education is right before disruption well one of the reasons is I don't know how to pronounce bomal but bormal Adam I don't know if Burma's cost disease is the relevant economic obviously I don't know the pronunciation either I've only read it yeah um so this is the observation somewhat counter-intuitively that wages in areas that are not increasing in productivity can can rise so um as you probably know overall we the the main reason why in the you know post okay there are many reasons why wealth of a Nation increases you could say that colonizing other countries and uh taking natural resources is historically a significant contributor but for some time maybe at least 100 years uh the main factor has has been increasing productivity um so increasing productivity means you do more with a given amount of input where inputs include time and material resources and so on if your productivity increases perhaps as a result of technological Improvement or knowledge as your productivity increases then there's often an effect where wages increase so you might think that if you see wages increasing in a sector then that must mean that productivity is increasing but that's a logical fallacy that's thinking that a implies B necessarily means that b implies a so increasing productivity often leads not always often leads to increases in wages but the reverse is not true and education is a prototypical example so over the last I don't know 60 years you can see declines in the sort of inflation-adjusted prices of many commodities uh but a very rapid increase in the cost of Education figures here I'll read some more quotes from this book later but I'm going to read a short quote from Matthew Bulls the metaverse
because it talks briefly about education and about this balmost Coast disease so he writes the relative lack of productivity growth in education is demonstrated by its relative increases in costs the U.S Bureau of Labor Statistics estimates that the cost of the average good in 1980 has increased over 260 through January 2020. whereas the cost of college tuition and fees is ingrown by twelve hundred percent the second closest sector Medical Care and services is up 600 now that's College tuition and fees I don't have the figures in front of me for um high school and primary education college education is kind of its own category of crazy in the US in terms of its cost it's not as extreme in other places so there's a long complicated story here but roughly speaking balmores cost disease points to that increase in the cost of education and says well this is because productivity has increased in many other areas that teachers might otherwise choose to work in so if teachers might become accountants or other professionals then you have to pay a wage that is competitive with the wages that you would pay in those other sectors otherwise you'll never get teachers and hence the wages have to rise now that's not the only source of costs but if you look at a typical University say the University of Melbourne something on the order of 60 to 70 percent of its costs are salaries maybe it's even higher than that and that's typical across um universities that's not just faculty that's probably half Administration okay so bull Moss cost disease says that in sectors where productivity is not increasing you may see increasing costs because somehow the rest of the economy is increasing in productivity and you have to compete with that to employ people so as those graphs go up and up to the right the cost of college tuition is I say relative to other costs in people's lives much much higher than it was 20 or 30 years ago um and that increase in cost it's kind of ridiculous uh in many ways what's happening in the US currently uh and to some lesser extent in other places this is one of the key reasons people point to say college and or university and say this can't continue and then there are other technological factors in play that people point to and say well obviously this will um cause the collapse whether that's moocs or um YouTube or 100 other things we can discuss well that's kind of a broad picture of why one of some of the reasons people think A disruption of University education has been on the
horizon and they've been saying that for at least a couple of decades strongly for at least a decade that's not the only kind of education we should discuss I think I'm at least as interested in early childhood education and high school education um but yeah maybe I'll I'll start there and invite comments on on that part any questions about bommel's disease or comments I mean that in a sense productivity is going down in these places because relatively well I'm not sure exactly yeah relative productivity or something like that I think that's right because we're paying more for the same amount sure I really oh go ahead sorry man I was just gonna ask quick a quick question can I think of this as you have to compensate the teachers for the opportunity cost dude there is a cost there is like an increased cost and what type of cost is that it's like some kind of opportunity cost is that right like it's not just some weird well yeah I'm just saying the same thing I think but using the different word opportunity plus well I think you could look at the of course it's uh want us to be very careful about stating generalizations and I don't want to disparage high school or Primary School teachers who are often highly trained into a very difficult job and do it well um but uh you know I think it's it's not that typical for someone with an advanced degree to go and become a teacher in Australia anymore whereas where when I was a child it was more common right it isn't seen as as desirable an occupation as it used to be and if you ask people why it's because well the first thing they say is well I can go and earn a lot more money somewhere else right so you can see that that pressure there right where of course you can you can probably find someone to teach right but the question is are you getting the people that you want I mean given how much Society values education I should value it uh can you actually hire the people you need or want at the level of quality that you think the system deserves so I mean it's it's not like is there some sort of equilibrium there right where maybe the equilibrium and this is why college education is probably shooting up as much because it's tied to other things like credentialism and uh yeah anyway so Adam you had some comment yeah I think I mean I think I agree that there are multiple things going on and maybe that because maybe because this is a function of relative change there are multiple ways to look at causation there and you can look at uh
you can look at it you can look at the situation from a number a number of different angles I mean you can look at it through the lens of productivity which is to my eye what bobal's disease is doing um but most cost disease uh and and maybe that makes some sense um you can also you could conceivably look at it through the lens of just supply and demand and where there is where the relative scarcity and relative abundance um is at the in the economy writ large and so as you know goods and services um have experienced different forces vis-a-vis scarcity and abundance over the years over the decades since since you know the post-war period I guess and especially I mean over the course of the 20th century in general and over the course of the Industrial Revolution in general but you know rocket you know sort of astronomical changes over the last five decades or six seven decades something like that right and uh so the mechanization of the of of uh of production of goods the industrialization with mechanization with the econ you know all of the efficiency gains of Information Technology and all this stuff and and of course globalization where where the cost of goods is being driven down in manufacturing because of competitive pressures globally uh you have you have what is effectively abundance materially uh in in the production of material Goods um relative to the supplier slash abundance of Labor and but if these things are all moving they're all changing there's more people now than there wasn't that were in the past and people are more skilled than they were in the past and so the the the availability of skilled labor has grown just tremendously over the last 150 years is astronomical growth in the availability of skilled labor to the global labor market another thing that's that's different is that the is that trade affects things right so there's there's tremendous mobility of capital now but there's not a corresponding mobility of Labor so we don't actually have free trade that's an illusion you have free trade inside the national economy more or less like inside a Federation of States like the United States we basically have free trade and it's probably a major part of a major formula for Success economically because labor is free to move between states across state lines Justice capital is freedom move but between National between nations you know yeah sure now globalization means that Capital can move across those borders but labor can't and because labor can't
we don't have free trades a huge amount of friction and so there's art there are artificial forces acting on scarcity on on the labor market but even that you know is is eroded a bit relative to certainly 100 150 years ago because people are are more mobile now than they were although not perfectly mobile so I think there are a lot of different ways to look at this relative uh um the relative productivity in with any Goods versus Services the relative abundance of goods versus Services the relative supply and demand of goods versus services and then of course you have this weird thing in the labor market which is that labor is on the one hand a supply yes but then the same people supplying the labor also demand the labor on the on the supply side in other words people are job Seekers It's Not Just employers who are looking for job who are trying to create jobs or trying to fill positions but people are actively trying to add people individuals actively pursue employment they're they're job Seekers and that leads to all kinds of strange stuff um especially in in the advanced industrial and especially even more so sorry not the advanced industrial but the advanced sort of post-modern post-industrial societies where Services make up a huge portion of the economy and so this is the phenomenon of [ __ ] jobs where you have massive massive job artificial job construction within a bloated and inefficient and opaque bureaucracy and that's absolutely intentional right that's absolutely intentional so um uh and I think very for sure higher education is an example where where especially especially in administration [ __ ] jobs is a massive problem um it's also a problem in government and other large corporate bureaucracies but within Academia and especially on the administrative side just massive bloat and massive [ __ ] jobs and just to be clear [ __ ] jobs is an official term this is David graber's work and it means a job that the employee themselves defines as either useless or actively counterproductive so it's so this is this is an employee-based perspective so the person doing the job evaluates it as either zero value contributing contributing zero value or actively harming the um the Enterprise itself that's the definition of a bolstery job it's not an external person looking in and saying oh that's not very efficient or something like that it's self-defined so and and yeah yeah the economy as at large it looks like something like 30 of jobs are in that category in a
University Administration is probably more than half just so just a very well affect you Harvard has almost as many administrators as undergraduates completely insane situation because 75 years ago the ratio of administration to students and the and ratio Administration to staff was Far Far smaller um and students still achieved this educational outcomes that were you know respectable and in principle we now have admin we have uh information technology to assist with the administration so this is it is pretty absurd and I have a couple of anecdotes that are fun examples if bear with me I'll just share a fun one I've told Dan about this before my go-to example out of personal experience my anecdotal example of a [ __ ] job is there was a full-time position they paid sixty eight thousand dollars a year when I was at UCLA this was in 2014 2015. as a and I was an undergrad I was sorry I was a um uh a teaching assistant so a graduate student instructor of undergraduates so I was g i was a TA basically for undergraduate courses and there was a budget Tas could apply for a 500 dollar uh Grant from the University from the slush fund to do things like pay for lunches on field trips and and bus fare and that kind of thing to take kids on field trips and to do extracurricular you know sort of off-campus activities and you had to apply for this money and uh 500 that's what it was that was the limit per semester per ta and the concern was that this would be abused theoretically so there was a full-time position that processed applications for that money and a very quick calculation showed that even if one hundred percent of the applications for that money were were fraud like every single ta applying for that money every semester was fraud it still wouldn't pay for the position that was evaluating whether or not it was fraud and processing those applications so there was a full-time person who was losing the university money even if the university had simply automatically no questions asked handed over the 500 cash to every ta who applied for it and obviously the the percentage of cases that were fraud was nothing like that I mean it might have been zero in a given semester for all I know so there were there was no questions asked uh or sorry a policy of no questions asked would have been much more effective and efficient and of course much less insulting because of course this is insulting to a T8 to have to spend six to eight hours on an application process that is entirely
there because you're presumed to be a thief so this is awful all the way around and there was a full a person a person on campus employed that was their job and the person had been in that job for many years so this is just a perfect example of a [ __ ] job in my mind and I think that that person would have defined it as completely useless or counterproductive for sure if you'd ask them you see the person I'm thinking of was extremely cynical about the work so yeah good question Matt uh regarding undergraduates yeah Harvard is doesn't have as many undergraduates as Melbourne so I don't think it's on the order of twelve thousand if I remember correctly okay yeah that makes that makes a little more sense when you said administrators I was just imagining the hordes of undergrads that we have at Uni mode yeah um yeah I'd know less about the situation with um I mean high schools and primary schools in the US are much more diverse it seems than than universities who at least with respect to these issues seem more or less in in one bucket uh with you know some more extreme and less extreme cases um yeah I guess I don't necessarily want to spend all the time talking about the the failures of universities because I'll enjoy that too much and we'll just spend all the time doing that um yeah maybe let's uh come back to the technology side and maybe just make a list of relevant technologies that that seem to affect education as a system I want to distinguish education and education as a system right because uh when we say disrupting education it's not necessarily that we mean changing the fundamental nature of how knowledge is transmitted from generation to generation although maybe we do mean that but when we're talking about disruption Adam can correct me if I misread this but when we're talking about disruption usually we're talking about disruption of an industry right or a given way of organizing activity or labor and maybe the underlying need is is fulfilled in a different way with different technology but the disruption itself is uh I mean maybe we focus on the institutions uh when we're talking about disruption and so I want to distinguish education and the education system how does that sound yeah I think that that's fair I mean in principle disruption could apply at the product segment level the market level the industry level or the sector level and so education has has a presence at all of those right there are specific products and services that education
offers there are there is a a effectively an education industry with with providers uh of of those goods and services there is a sector of Education overall that's comprised of multiple Industries so higher education and uh with presumably a few different uh domains you know within within just higher education so that disaggregates a bit in the United States you would have you know sort of private universities um public slash government universities what we call community colleges um some universities offer um uh vocational training for the for the trades as it were uh not all universities offer four-year degrees not all universities offer uh postgraduate you know higher up master and phds there are professional schools for dentistry and so forth that are that can sometimes have their own structures so this is a big it's an entire sector of the economy and and disruption in principle could occur and follow the typical pattern that we see an s-shaped you know some sort of sigmoid function of adoption of new technologies that then displace the old Technologies in a given part of that sector or written large across the entire sector and so there there are multiple levels of analysis at which a disruption could occur and have um and have impacts with far-reaching implications and there are multiple levels at which that could occur and the disruption could in principle unfold as we see others very quickly in the space of between 10 and 20 years right and we could probably look at examples where that's happened in the not too distant past certainly you some some of the um information Technologies internet you know digital Technologies certainly some of those have have caused what we would probably categorize As A disruption down at the product and service level at the segment level um in education there are probably some things I'm reaching here I don't know off the top of my head but I'm imagining there are certain things that were once done at universities you know in the 70s and 80s that the internet just obviated and disrupted very quickly over the course of a decade or something like that we could probably one will probably come to me an example of that um um oh I can think of one actually very quickly um when I was even when I was in an undergraduate at the very beginning of the internet so in 93 I mean the internet was for all practical purposes just starting there were still copy shops I don't know if these were a thing for you for you guys or are a thing for you guys in your
experience but the University surrounding the university in the town were multiple copy shops where you would go and you would get photocopied packets of course material those are gone that is now ancient history and that's a disruption like that now you would you could say that that is an industry that got disrupt disrupted but that industry and its goods and services were part of the education sector and so that would be a piece of Education that was disrupted this is a specific example yeah that makes sense um yeah I'm just making a list of some of the aims of Education uh these are the a few obvious ones that occur to me uh there's more social things here as well right I don't know if that's an aim of Education exactly to socialize people but uh I guess it's something that happens at educational institutions so the interesting thing about talking about disruption of education is that it's easy to focus too much on just the the part that's passing on knowledge and then you say well the knowledge is in the books and it's in YouTube videos so uh what's the issue here it's already disrupted isn't it and then of course once you start thinking about it more closely it's it's clear that our educational institutions whether they're primary schools high schools universities uh any of the number of other institutions that Adam just listed they're really Aggregates of many different Services kind of hooked together and that's part of the reason why it seems to be part of the reason why they've resisted disruption until now is that you can't just do a better job of one of the pieces um yeah so let's add to this list anymore I'm going to ask GPT while you're um I I think one thing that I would add maybe this is inside socialization socialization is could be pretty broad but one thing I think that that is very much part of higher education is networking because that that feeds into your your opportunities I suppose also it's you know sort of a preparation for productive labor but you you you you it it is higher education is an opportunity to to identify and then um enter into new Social Circles and establish new relationships new friendships um and just basically enter new networks and presumably some of those networks will be lifelong um and it depends on the discipline and it depends on the individual but some of these networks are expressly designed for that so there are you know the the Greek system of fraternities and sororities for example in the United States is at least partly intended to
give you a social uh experience and a network that then you you then take on uh for life that you um utilize long after you leave the university and so I would I would definitely think of networking as a cat as a category a large subcategory within socialization if not its own thing um and then the uh if we want to be more cynical about education um you could talk about some of the perverse Financial incentives that are here um I don't know if that's worth thinking about this is sort of the flip side I mean the aims of Education are these sort of the noble and and high principled aims of Education or are we also talking about sort of the system's goal of of uh creating indebted an indebted Workforce that then it has you know Inspirations because you could say you could say that those are the system reproducing itself it has uh you know those are emergent incentives that that again that get enacted structurally uh and embedded structurally within you know the larger Enterprise of higher education and maybe we don't need to talk about those but I would just mention that as a possibility in case that's part of your thinking then yeah I think we should talk about them uh there's no point being they're putting on the rose glasses for this discussion but uh for this list let's focus on the um the more positive the the things we would want education to do rather than what it actually does uh well I mean one reason for just including the the more cynical take is that new technology could obviously do an end run around a lot of those uh dysfunctional incentives and the structures around them right and so the one reason why disruption could happen um and happen very quickly is be it is precisely because it could it could circumvent a lot of that crap and deliver more value at lower cost as a result of that so yeah I think that's a good point I'm going to put that on the first board that we we might expect disruption not just because some new technology does the things that the system already does better but it does them in a way that circumvents some of the rent seeking and extreme bad behavior that is very clearly present in the existing system sure that's that's a that's a fun that's a that's is a characteristic and and we theory predicts that our theory predicts that for disruption that disruptions one reason why they tend to occur is that the convergence of the new technologies um doesn't doesn't just lower costs by providing a perfect one-to-one substitution for what already exists but
that it also eliminates like externalities environmental social in this case if we're talking about some of the crap that's with higher education specifically um but those are definitely social externalities right um and uh that as a result of all those things plus perhaps new fundamentally new capabilities that the new technologies offer you end up with a system that has a a a completely different structure new properties and therefore often new opportunities new behaviors and often many more and different uh and behaviors as well so that and the upshot of that is that the new system that is enabled tends to be larger has more participation more more throughput it's more throughput of people uh and their time and energy more throughput of literal energy and material uh more throughput economically so more throughput of value as measured by whatever you want to measure in um economically um that's what we mean by a larger system so that that is something to predict as well that this that this disruption of Education would create a larger education system hey Adam it um this is getting a little sidetracked maybe just quickly um it accustomed to me that while you're talking about this um kind of uh this kind of benefit or this kind of predictor um of disruption or this kind of consequence of disruption um do you think that it occurs to me that sometimes like new externalities might be opened up [Music] um as a result of the transition and they're external because you know the the society hasn't really quite figured out how to price in the true costs of the new technology so there might be like debt unanticipated perhaps downsides um I wonder if you think that's a part of the picture as well or if you think that's actually doesn't happen a lot oh that's absolutely part of the picture and I can give you a very good example in that social media and the social externalities the massive costs of social media that were unanticipated and one of my goals one of the goals in my organization and me personally is to do a better job anticipating those detriments and those ex those social and other externalities the problems this unintended side effects that could emerge and that could become um deeply embedded because of perverse incentives or nasty incentives at any rate um uh that emerge and and if if it's possible to see them coming far enough advance in advance maybe maybe we could with wise intervention wise and judicious intervention you know head them off at the past before they take
hold before they take root if we were smarter and if we'd anticipate it and if we'd known how truly horrific some of the social and psychological consequences of of uh social media would be we might have been able to enact more stringent regulation 10 or 15 years ago and prevented a lot of the worst of it right um of its toxicity um and retained more you know more of the benefits so yeah for sure that's a possibility and the challenge here and it's one of the things my team intends to work on specifically with this disruption of education is to try to see some of those things coming in advance yeah it's obviously complicated because maybe the um the kind of validity with which people are able to experiment throughout the good and the bad so I'm not sure that it's so easy well yeah I mean you're not saying it's easy but we've got to be yeah it's a tricky problem to try and have the good without the path anyway I wanted to maybe add one thing to the list that's a little bit more of a cynical take on the education system but not so cynical I think as um creating an uh a Workforce that's indebted to um you know uh so so this is just this kind of signaling and evaluative aspects of education so it seems like um there's uh there's sort of like some basic signaling like I can get through a degree but also um you you get a Mark that kind of ranks you in terms of some attributes of your ability or your performance and that um can be kind of used um as a way to to sort of you know select the best employees or something um I obviously it's just one characteristic among many that are used by people that are hiring and it may not be even the primary metric but um yeah this seems like a very obvious um role that um education the education system is playing and it's not entirely um negative it seems like it's important to be able to um evaluate workers for example um as a way of efficiently structuring your Workforce oh I also wanted to add one last thing yeah sorry go ahead I was just going to mention Brian kaplan's book against education which is yeah you're probably familiar with yeah I I we could discuss that briefly uh maybe after you say what you want to say I I've discussed this with quite a number of people uh in my Social Circle and I can see propositions for and against it I'm not convinced that I think there's more truth to it than many people admit it's clear a lot of the well since say a lot of Stanford or MIT or Harvard's courses are online and you can take them for free clearly you're
not paying for the courses I mean maybe to some extent you are and you're paying for particular lecturers and but those moments of Enlightenment you get every other month uh from somebody saying something truly surprising in your lecture are not worth tens of thousands or hundreds of thousands of dollars if you're going to Ivy League schools in the US so it's it's clearly to some large degree signaling how much is open to debate I would say one thing one Nuance that I would add to that and and I and I think kaplan's case is is interesting um but uh I I one thing I think that's often neglected um is timing so when I Dan you've probably seen this and and Matt I don't know how much teaching you've done but the the certainly when I was teaching there was such a stark difference between the motivations and the and the mindset and the preparedness and and everything between uh undergraduates coming into undergraduate degree programs out of high school versus people who were older who who were returning as adult students now I know there's a major selection bias there you know the people who do choose to return um you know that's they're self-selected in this massive bias there and we had extraordinary people I mean I'm you know military veterans who are coming back after multiple tours of Duty and they had seen you know unbelievable things yeah the undergraduate experience was not very intimidating for them I mean and lots of different people um so I think part of it is timing in my opinion um it's my personal opinion is that it's a mistake to do an undergraduate program when most people do it I think people I think personally on on the whole students out of high school should not go straight into college I think the entire system would be much better if people spent three four five years just doing something completely different and then going to college and doing an undergraduate degree I think the difference in mindset and maturity and and you know in preparedness on many different levels would completely transform the efficacy of the education even if you changed nothing else except okay students can't start until they're 22. I think this would completely transform the value that that entire process had for people um and so this and I almost never see that discussed you know everything else is sort of a variable but but when people actually attend not part of the conversation so much so um and that would be in the course is funny because that would be of you know
in principle quite a trivial thing to change and we have the facts it would have massive consequences but you I mean you're not you could change you could change that with the stroke of the pen instantly and with no other intervention in the system and have them and I I believe it would have an absolutely massive and overwhelmingly positive impact in terms of how useful the entire thing is yeah it's interesting I mean that immediately it's a very interesting proposal I I think that what immediately comes to mind for me is that the count of factual is that um these 18 year olds are going to work uh as interns or you know lackies or like in some pre-undergraduate position at Facebook instead of uh spending those years at um at University and um or maybe the university is like a safe place to be when you're 19 years old in terms of like you can't do much damage to the world but you I don't know maybe all the world didn't end up for all those centuries so people didn't mostly go to university it's only realization at the scale of power that you know some people could be just by being a part of a large corporation today so um I suppose that you know if people simply joined the workforce the labor market um and that would be good enough I mean I I think that would be interesting that people would do jobs and and I mean it would well I can imagine well positives and negatives emerging from flooding the labor market with with people um without advanced education who are at that age relative to the you know they're the numbers who are in that market those markets today um I mean a lot of a lot of things would change uh uh presumably there's some you know some there's some demographic major demographic differences so you would have a lot of 18 to 22 year olds in rummy dumb jobs um but that were you know who are capable of much more so a lot of brighter people in these crappier jobs and you know For Better or Worse it'd be interesting to see what the result of that was um but I I'm also quite I'm also a big fan of um of public service and uh and so like we have we have a few programs in the US that people can join obviously they're not mandatory um they're countries where national service is mandatory I'm not a huge fan of National military service but I I am open-minded with respect to obligatory National Social Service I mean imagine if you had to spend 18 months or 24 months doing community service when you turned 18 yeah I think this would be marvelous for most societies I I mean yes it would it
would curtail some people's freedoms and and so forth and and uh there could be downsides but I think on balance again you know I could get behind a program it was like that most 18 19 and 20 year olds that I know are [ __ ] useless in almost every respect and we we don't want to put a rifle in those kids hands we want to get them out into communities that are different than the ones they grew up with and have them seeing what the world is like and making a difference and making a contribution and you know getting a little bit of life experience and then they can go and decide whether or not they want to spend 150 000 on undergraduate education and what they want to get out of it and so forth so anyway sorry do you want to derail us with it thanks for attending My teacher talk I like being I like the proposal about public service and also I just want to add a final note that maybe this is my um kind of hatred of Facebook standing through but I I guess I see um signing up a recruit to Facebook sort of army of um pre-undergraduate developers is kind of like putting a raffle in their hands and just giving them some orders um and that seems like an equally fair enough yeah fair enough anyway to return to the conversation I think uh maybe this goes on the first board if the first board is like is this reasons to disrupt education or like consequences of Education or just relevant facts that are that are something about um the disruption of Education either way I think that one thing that most people in education that kind of work on this kind of technology stuff will point to as a big motivation for disrupting education is that it seems like this is like the dream of um education technologists um is called the the two Sigma effect um from a study by Benjamin Bloom um from I don't remember the date it was maybe around 1970-ish um a study on uh private tutoring versus sort of class-based tutoring that uh saw an effect size of two sigmas in performance um for students that were privately tutored which basically I mean there's there's not always that much variation in student performance to begin with so this basically like means that even average students become top performers with private tutoring um applied and the the idea of you know AI in particular but just automation um in general is that you could try to scale private tutoring which is obviously infeasible the scale that we needed for an education system that is going to educate um a large percentage of the population
um you could hope to scale and receive that effect size um which would be a massive increase in uh the effectiveness of the education system um if you can find some way to scale this um so the two sigmas is kind of like the headline result in that paper I I think there have been attempts to kind of reproduce the study and they don't always find a two Sigma effect but you know at least it's a proof of concept that you can get large gains out of doing education better like there's there's a lot of room to improve if only we could solve this kind of scale problem for example so yeah that's a big motivation when it comes to discussing disruption in education yeah just to add an interesting note there so I I won't say who it is but uh the the team at DARPA that worked on this system and kind of developing it after the original work of Bloom uh is now involved in productizing it using some of these recent advances in large language models so this is very much a in play this um this two Sigma effect and the search for AI implementations of it yeah wow that's fast this would be like what this would be like the alphago moment of um the education system if there if they could if you could show a study where an intelligent tutoring system based on AI was able to replicate this result oh absolutely I mean it could be yeah you could imagine it being quite literally world changing my my first question I would ask about this and I I mean this is one of these things where I've heard of this effect I'm sort of dimly aware of it but I don't know any of the details at all one of the first things that occurs to me is is whether the you know the specifics of this phenomenon have been explored like what is it specifically that changes when you have you know one-on-one tutoring or two on or small group tutoring two three group tutoring um which might even be better than one-on-one I imagine I'm just guessing um uh but what is specifically is that do has it been disaggregated down into individual component part like you know is it about attention do people pay attention better are they is are they are there interests cater to more are they do they have more motivation is there more diligence is there is there more you know better responsibility is it is it a more personal personalized and less anonymity so it feels like there's more accountability um dude students remember things better because they are delivered in a different Channel like so they delivered vocally I mean these people are speaking
and they're speaking right at you while you're making eye contact as opposed to you know absorbing material by reading or by just by listening so I mean it's it's I would I would be very interested to know if the specifics are understood um and that's not to say that you couldn't have the alphago moment that simply replicates the whole experience and says well [ __ ] we don't know what precisely what it is about having a one-on-one uh relationship that makes it work for people better so much better but it does and so we're going to replicate that and who cares what the details are but I am I would also I personally would be fascinated and perhaps there would be some value in knowing what those details are one of the reasons why perhaps is that if you knew specifically what it was that really worked it's not impossible that an AI tutor could be hyper effective it could it could do a better job than any human could do if it knew exactly what was working we could you know it could be hyper stimulating or hyper uh anyway you know you understand what I mean so yeah um there would be value in under in knowing exactly what's going on with this phenomenon rather than having it be about black box that's just really effective yeah with a small amount of familiarity that I have with the education mixture the education science literature that is like people who study you know this kind of thing um and this being a landmark paper in that discipline um I would say that there is a there are sort of some established theories about um you know um breaking this down and saying what was effective one notable um variable in this uh paper in this experiment was that I believe those students who were undergoing the private tutoring were using a Mastery based approach if you're not familiar this is basically an approach where instead of for example at a particular deadline moving on to the next topic regardless of whether or not the student has scored 50 or 70 or 90 on the test and then just moving on uh instead the pace is slowed uh or sped up that is until a student and you know you move on to next student once a student has achieved a very high level of competence with a particular topic and so um that's why it's called Mastery because you move on when you've mastered one topic and then you move on to the next and so this uh they also did a Mastery based classroom and that saw maybe one Sigma or a little bit more than one Sigma and then the two sigmas came from Mastery plus private tutoring
or black one-on-one instruction um so that seems to be a big component um one paper that I know of um is a famous meta-analysis actually I think meta meta-analysis it's an analysis of meta-analyzes um by John Hattie who's actually at the Melbourne Graduate School of Education um and Hattie did a very systematic study of um a very large number of studies in the education literature that we're trying to determine effect sizes for various individual kind of components of um how you vary the experience and has a long list of you know this variation leads to this effect size this variation leads to this effect size averaged over a large number a very very large number of studies so if you're actually interested in seeing that uh that's one place to look although yeah I guess that just speaks to the fact that this literature is fairly Advanced and uh it's it's quite difficult to do this kind of science like because the experiments you know are very messy because it's social science um and in particular it's very difficult to do a control when it comes to education studies because you it's sort of very hard to find in identical classrooms so yeah it's it's a bit messy but this literature has it you know has at least been working hard to to make progress on these questions yeah to just take a contrarian take on that for a moment uh what would actually so we're talking about actually increasing the effectiveness of the teaching process itself right um now that doesn't seem to have had much of an impact on Rising costs at U.S colleges for example so certainly the internet I mean it's hard to believe that the internet the free availability of textbooks online whether legitimately or legitimately YouTube and all of that hasn't changed the effectiveness of passing on knowledge to say undergraduates um but you know so if you're going to University to pay for the education or why has the proliferation of educational material not made much of a difference to the structure or the cost of that system so let's just think through for a moment what suppose this alphago moment appears and we have a very effective AI tutor uh yeah I don't necessarily want to focus on College I guess but let's go with it for a moment so what would change about the university education experience I mean you don't need lecturers anymore uh you don't need Shooters anymore is it just the case where the administrators finally complete their Victory and just uh just get rid of all the faculty they keep a few
in order to be high on the rankings the US News rankings and the Shanghai rankings so they keep all the Nobel Prize winners and the fields medalists uh and then just get rid of everybody else and replace them by AI tutors where is the um yeah maybe that's another question okay uh well there's obviously social components to um and the networking components that would be decimated by that particular proposal um but in terms of the um educational experience yeah I guess that the proposal is something like you have a tutor you have a student sit down in front of a computer that has access to the system and they are just engaged in some kind of probably more interactive than a lecture even more interactive than some tutorials if the tutor sort of lacks to do a lot of the talking uh steps the student through solving example problems introduces them to relevant materials at the right time and kind of conducts this entire process um in a one-on-one or a small group uh format acting as a as a kind of person who finds interesting content like lectures online or or just synthesizes them from from scratch um tailored to the students needs at any a moment um and then yeah I guess you would that would be more educationally effective than passively sitting in front of a lecturer and absorbing material that way and I mean not to say that you know um you can uh you can completely replace some of the things lecturers do so you would want to um you know have this AI tutor also do things like motivate the material and um have have a kind of inspirational relationship to like Inspire the students in the way that some lecturers do um Dan being a notable example according to many people I've spoke to who have been in his lectures um yeah so you you would kind of if you had a system that that could do all of this then you could replace um the lecturers and tutors the human lecturers and tutors with it yeah I suppose it's uh like many of these advances in AI I would agree with what Matt just said and that they they tend to shine on light on the dirty secrets of Any Given area of human activity right so a dirty secret of go would be that yeah yeah we tell the public that it's like highfalutin super clever Behavior but really what we're doing is we're sitting down and just playing lots of games and memorizing all sorts of moves and then passing them into our unconscious and it's just grunt work and there's like nothing there's some moments where there's creative genius but most of it is not creative
uh there's lots of dirty secrets in education I'll speak about University education since it's what I know from the sausage making side you know much of lecturing is just regurgitating stuff from a textbook onto the board that's 90 of what lecturing is uh you know there's some times where you know if you've got a particularly motivated lecturer they'll take some particular piece and do a really good job of it and be inspiring but okay thanks Matt for saying what you said earlier but even in my case you know it's most of what I'm doing is pretty routine uh and it it maybe helps you to hear it spoken out loud for various reasons but I think that even my own lecturing uh you know you could replace most of that by an AI tutor and not lose much magic and to be a bit egotistical about it I think many lecturers who are more or less just saying what's in a textbook writing it up I mean they're doing something right because they do understand it to some degree they may not be able to answer a very deep question about it but they will be able to correct most superficial confusions or even moderate confusions they can integrate the lectures with your exercise material so they are doing something but uh when you when you look at it through the lens of automation uh you probably come to realize that I don't know 70 to 80 percent of uh what you're doing as a lecturer is pretty routine low value and then it's yeah this is why I wanted to introduce balmall's cost disease first is I think we often make a mistake of attaching Prestige to things that command High incomes so for whatever reason in 2022 being a lecturer at a university commands a pretty high income I mean my income is much higher than both of my parents can buy they ran small businesses um it's not that much more intellectually demanding probably a bit but it commands a much higher salary and there's sort of prestige Associated to being a university lecturer but probably not with great reason um so I could see that AI tutors first of all making clear the current system so what is and what is not actually deserving of prestige and what is hard versus what is routine what are we getting away with universities get away with an awful lot in terms of the lower quality of Education that they deliver to students at high cost and so the first thing I think an AI tutor will do even if it's not quite here yet will just make people see this differently um that to bring another ingredient into the discussion that clearly happened
with the pandemic and primary and high school education so for the first time many parents saw via Zoom what their students what their children rather were actually being taught and came away from that with maybe a less of a high opinion of the the quality of the material that was being delivered to their children obviously teaching on Zoom is very hard especially for small children and they have a lot of sympathy for for example the teachers who tried to do that to my son's class so I'm not really picking on them in the same way I don't I'm not really picking on my colleagues or myself it's just okay that's that's that's Human Experience right so uh they did a very good job or tried to do a good job struggled through with the pandemic but nonetheless it is not the case that you would look at the curriculum at least in Victoria for primary school students I'm very familiar with that now and think that you can't do better than this uh you know it's a balance of the amount of Labor you have and all of that I understand that but there is also a lot of kind of skating through and settling for good enough and all of that which will come to light uh in this case in when you know parents actually saw what was being taught and homeschooling is erupted to some degree as a result of that and also just trying to avoid covert but schooling is rapidly growing not only in Australia but in the US so yeah that's a technological disruption of a different kind facilitated by the internet and video conferencing and other Technologies but um yeah I guess I just want to emphasize the role of technological disruption and in bringing people to see what's actually there as opposed to the carefully constructed set of filters and obfuscation that we put in place over decades to make high status things appear to be more deserving of high status than they really are yeah that makes some sense I want to underscore something that I said at the start of my um comment about AI like what would be the actual um what would it look like if we had this reduction and that's the social and networking side um if it was just um that a lecturer was reading a textbook to the students with maybe a little bit of integration with the exercises and stuff um it would seem like maybe reading would be like reading a good textbook would be like equally effective but it doesn't seem like that I mean maybe actually sitting down and spending the effort to read with the textbook is you know is effective but people don't
tend to learn this way as much I think I think lectures are like a more popular mode um and this might be more you know that it's kind of easier to interact with a person listening to a person speaking you've got that intonation and everything you know for one thing um but just more generally um you know little things like having a schedule with classes on it and you have to go to the classes at the scheduled time um can be like a good motivating uh reason uh to you know engage with your class um and I think that while um you know maybe it's fairly easy to regurgitate content uh it doesn't seem easy for an AI although it does seem easy for a human because you know this is like what we do to interact with people in a you know a natural way um that seems to be a big part you know using the human interface you could say is something that humans are very comfortable with and don't always um think about as as difficult or a part of their job but when it comes to AI you that is something that has to be automated if you're going to have a similar experience I think you need the need to you either need to uh learn how to you know the AI system either needs to learn how to use the human interface to kind of naturally communicate with people or it needs to figure out something even better like um pleasing video animations or something like that can you dig into a little bit about the human interface you mean something about the way that you interact with the information differently through a person rather than a book what what is that what's different exactly so in in conversation with someone um while you're using a different part of your brain to kind of process the information because it's audio rather than um rather than Visual and reading but even still like reading an audio book is you know slightly different from reading a book but it's not the same as talking to someone so mainly I think the big thing is the interactivity and not all lecturers are like this because there's you know in in a traditional lecturer uh lecture where you have this kind of what's called in education the sage on the stage model where um you have a source of knowledge standing in front of many students and distributing that knowledge um this is considered in the education system to you know not necessarily be a super effective way of achieving learning outcomes um something that has more interactivity is um is considered to be more effective where students are either able to ask or answer questions or
um you know participate in examples and do example problems and stuff like that in the very lecture itself and this is why oh this is one reason why you see many lecturers kind of go into oh we're going to get clickers and we're going to do questions um we're going to do like multiple choice questions during the lecture uh to make sure everyone's on um on uh keeping up with the concepts that we're introducing and um and I think these are considered to be you know good um good practice in terms of educational theory um yeah so interactivity is uh different um that's quite different from you know just writing up the textbook contents onto the Blackboard and I think that's okay we have we have sort of natural language we've made huge progress in natural language uh processing for AIS so maybe this is not going to remain hard for too long but um but you know this I see as like a core challenge of automating education is getting interactivity right uh automatic interactivity seems difficult um I have a couple of uh things to add um just building on what you guys have already been discussing um one is that is that uh it seems to me like like just just judging from other disruptions one thing that's a definitely part of the pattern we see in other disruptions is a new single technology or convergence of technologies that that enables um that allows a multiple sources of value to be achieved simultaneously so in other words you have capability and quality across multiple Dimensions simultaneously and new technologies can sometimes do this you know sort of all in one package um where where before that was not possible and and things had to be done separately the probably the best example is your cell phone your smartphone rather um your smartphone can do you know the things that it took took 20 different products or more in the past you know 30 years before the iPhone you know you had video cameras were separate from microphones and tape recorders which were separate from a compass which were separate from a map that you folded out which were separate from blah blah blah blah and so the new technology allows you to achieve multiple things simultaneously and therefore presents enormous value okay I can imagine in this case um an an AGI especially if it were reasonably convincing uh had reasonably convincing generality if not actual you know sapiens um it could do multiple things that are done in the classroom by great professors and lecturers namely it could deliver the best lecture
so you know one thing that I've seen um as part of the sort of a supposition about the early stages of disrupting education with the internet for example is well you could why listen to your professor or lecturer or their graduate student uh go through the motions and present the material that's in the textbook and give the and give the lecture um when you could just watch you know the best possible version of that lecture that's ever been done if it were recorded by the person who gave it you know gave the best version of it well in principle an AI could do that right an AI could basically give you the best possible version of the lecture but an AI could also then provide the interpersonal interactivity that experience if if it were if if we're capable of achieving that so in other words the great thing about an AI is it would be the best of all of the great things all of the things rather it would be able to do all of the things that make a a Professor great and would fail presumably at none of the things that can go wrong and make a professor terrible okay um so that's that's one thing to think about is that an AI is likely to be better in every respect not just in one respect or the other separately but in all respects at least at least in Fairly short order um the other thing that occurs to me and this is sort of in a separate line of thinking it maybe goes back a little bit earlier in the conversation is that um so outside there's there's inside the classroom and then there's also outside the classroom and so in higher education you have sort of an inside the classroom experience and it's it's fine to talk about disrupting that but then there's also the outside the classroom experience and and I don't know what it's like in Australia but in in the United States it's kind of become ridiculous in my experience over the last 20 years what universities are trying to do to attract students and and they are they are quite literally and they they're advertising and marketing themselves in this in exactly these ways but they are turning themselves into resorts and you are joining a club or you're you're the club you're joining is a resort that you get to spend four years at and you're paying a fortune for the privilege of living at a resort and all of the recreational and social opportunities that go along with that right and they're amazing I mean there's no question about it if if you know if the greatest retirement communities are like a resort living in a resort the
university experience is often formative and one of the best times in a young person's life because Jesus who wouldn't have a great time at an incredibly at a multi-billion dollar Resort that is expressly designed to give you fun experience social experiences learning experiences social opportunities and all the rest of that stuff and so there's that outside the classroom Dimension and and a question to ask is is AI does AI have the potential to disrupt the outside the classroom portion of the educational experience um yes or no in what ways you know simultaneously interacting with that or is it are these things going to become separate you know it's a it's a I think it's it's quite a it's something that's worthwhile thinking seriously about and um one thing where I think just it's just one suggestion for how that might be possible is one of the things that's that actually ironically that's great about a university that's different from an ordinary Resort is that you have a schedule like you have obligations you have a schedule your time is structured and it forces you to get certain kinds of value out of the experience you're not just on vacation at a university in the way that you are at a resort where you can just lie on the beach and a lot of us will just lie around if we if a structure isn't sort of um there to impose order on our lives and of course those of us who achieve a little bit of wisdom if we're lucky um can come to see the value of that and then seek out you know order and impo and deliberately impose structures on ourselves and we often you know we often um uh uh Outsource that like we make a decision and then to it that's necessary for our lives and then we that instructor gets imposed by some external thing in other words we're not having to expend The Willpower in itself you know the discipline every moment to stay on task you know we Outsource a lot of that by by joining and adopting a program that then we'll do that for us and there are apps there's a whole industry that's designed around you know helping you organize yourself in your life and to get value out of it and people know that they need that discipline and we need crutches and enablers not enablers that's probably the weighted term we need we need assistance and help with with with adopting that structure and sticking to it and being disciplined and the university has all that built into it right it has all of that built into it so it's really the sort of marvelous best of all possible worlds this is
incredibly well uh well at a big university it's an incredibly well run and well maintained and well-designed resort with all these opportunities and there's tons of other people there to socialize with and it's got this that just you know the right amount of discipline to keep you focused and moving forward and growing yourself and developing and then you get something out the end of it which is this degree that you know when you join you've joined a club or literally joined a fraternity you've joined a tribe in any case and you have the tribal association with your identity over the rest of your life and it's on and on and on and so that again I think it's it's worthwhile especially with higher education thinking of all of that value and how AI would could or would need to disrupt different or all portions of that in order to truly disrupt higher education at large so it is not just about the classroom experience right and I think we know that I'm just emphasizing that making that work yeah I guess we should also bring in maybe at this point um other Technologies like um the metaverse or Virtual Worlds they don't have that much relevance necessarily right now to those factors maybe in the future there's there's more social engagement that happens uh in places like that but it's not only AI I guess is um is the point at the moment it's sort of hard to see yeah I like the summary just made apart from one should add in there the uh the percentage of American GDP that is being sunk into these Disneylands for teenagers um uh as you know they they get something out of it but I don't know how I don't know how you even think about sending your kids to to college at all like you have to start saving when you're 14 more or less to send your kids to college um so yeah it works out for the people who have the money to do it I guess maybe that's one of the reasons why you might expect disruption right that's sort of like you can see how the system has an internal logic just like newspapers had an internal logic and it's it's very hard from within the universities to even have the conversation about why the trajectory we're on is unsustainable and wrong maybe even morally wrong in many cases uh let me know just to take the University of Melbourne for example it's costs always go up right student fees always go up there is no there's never a change that reduces the number of Administrators or the complexity of things or reduces the number of things the university does it's always what more can we do and then
how much more can we charge and then what more can we do um and there's just no there's almost no intellectual framework for having a conversation about why it might be a good idea to not just to keep costs down for students but just to do less so that you become less fixated on all the sort of frivolous things that you're up to I mean take universities and the sort of package you were presenting so say sports teams are part of that package right now you can't Maybe you know it's part of the team spirit or the the community Spirit to have a sport team at your University but then you look at the contortions that universities go into around their sport teams and the corruption scandals and how many students they admit purely you know it's like this this fixation on these extracurricular activities sort of feeds back and changes the nature of the original purpose to such a degree that I would think that at least most universities that I've personally been at or know much about are not actually universities in the sense that I thought the word was supposed to be interpreted when I was young there's some other institution and it's fine maybe but they're really not actually focused necessarily on you know I thought universities are about the pursuit of Truth or knowledge and passing it on and like the ideal of Truth is a central thing in our culture but that's not really true anymore and I blame some of these extracurricular distractions for for steadily over decades derailing our universities from what maybe they should have been which you know is fine from the point of view of many students who engage in them I don't begrudge them the chance to to have those experiences and be shaped in a way that maybe is beneficial for them but actually I thought it was useful and maybe even crucial that there were little pockets in the world where there was a different set of rules and ideas were actually valued and protected and the people who held them were allowed to express them and discuss them and we've lost that to a very large degree so yeah I I would be much more in favor of universities than did much less radically less and I do hope that the disruption that's coming maybe doesn't destroy the old system I don't care but I would very much like it to open up a space for universities of the kind that I actually imagined to be yeah I think that that's perfectly Fair um one of the I think one of the challenges there's sort of a chicken and egg challenge between the institution
itself and the students that are part of it and that come out of it I think this is a major challenge right so so one would imagine that or what one could hope that a a truly world-class you know uh superlative education could be provided through a fundamentally new kind of higher education institution and especially one built on and enabled by these new technological tools one could Envision that but the and that's you know that's that's one part of the Chicken and the Egg but the other part of that is that then what in order for that to succeed there would need to be some way in which extraordinary students were attracted to participate and come out of that institution and then lend that institution The credibility that it would need to then become self-sustaining in that respect right so so and and all of the the great universities enjoy that privileged uh uh self-reinforcing loop already right I mean if you go to Harvard or Oxford or the great universities anywhere in the world at any University you know the national universities of the world and so forth you know you you the universities are are they have great prestige and so they attract great students and then so the students who come out of that are are you know uh achieve go on to achieve great things and then that Prestige builds the University's reputation and it's you know it's reinforcing it's mutually reinforcing that feedback loop so the question it's the question as is so often the case is how do you get that flywheel spinning how do you spin that feedback loop up how do you get it up to speed and get that engine running that is that is a major question for you know new Enterprises of all kinds and new social institutions and organizations of all kinds and and then you know and more generally systems of other kinds um is how do you get that that that inertia or that momentum I guess built and so that would be if it it I think I think that is just as big of a challenge as designing and establishing the institution itself like you could throw a lot of money at just creating a new University I'm frankly surprised that more you know very wealthy people Bill Gates and Elon Musk and the gazillionaires of the world haven't done that I mean it's happened in the past Carnegie Mellon for example is a is in a private university that was just founded by two steel Barons there's two of the richest men in America 100 years ago and there it's a great University now um the the question if you were to do that would be okay so you do that and
then it's you know it's a three-year program super intense and it's very focused and none of the you know extraneous crap that's that's there it's very affordable super cheap you know maybe it's even free to the students who go maybe that would be your your your real mechanism for getting it spun up is offer the offer the whole thing program for free like if you get in there if you get admitted it's free um yeah Google University exactly man um uh if you get it it's free but you have to be just absolutely Top Notch you have to be amazing and um then you get the prestige feedback loop going um yeah and I I suppose it's it's you know it it could be the Googles and apples of the world to do this but why would they do that when they basically have that on their campuses already anyway I mean they kind of do that with with especially I imagine young employees right new employees come in and they you know there there's a little bit of the that Resort action going on at the big campuses it's my understanding probably no coincidence that they call them campuses too I mean Google is uh does hire people without University degrees now I mean there are tests you can take and other credentials through Google itself I don't know to what extent that's actually a large source of hiring for them but um you could imagine remote work and the difficulty yeah it's um to say something positive about the university system so let's compare the University Systems in say the US Australia the UK other Rich Western democracies uh and say the Chinese system which is improving but has very deep problems so that's why a lot of young Chinese students or rather their parents choose to study at Western institutions despite their problems and their high cost right nobody's forcing them to pay those High fees you could say they're kind of disgustingly high and you know it's kind of like a um bit of a question of uh how ethical is it to charge them very high fees and then shove them into huge classes where they never basically even see their lecturer because they're taught by Tas and it's basically like they're getting ripped off but in some sense it's a transaction and nobody forces them to meet uh well why do they do that well because it's not easy to build institutions that are even reasonably well functioning and not that corrupt right uh it's much harder than it seems uh so despite the problems that Western universities have they they are pretty well functioning institutions they a lot of them cost a lot and are very
inefficient uh and have plenty of problems but uh okay the world wants to attend uh ivy league universities in the US and is willing to pay for the privilege and that's the demand that drives up the price so they are success stories in terms of institutions so I guess often when I say negative things about the university system it's more from a point of view of seeing not seeing why it couldn't be much better yeah and regarding your comment Adam yeah it's interesting to watch the University of Texas at Austin which is this new University that's sort of come out of some of the concerns around the corrosion of academic freedom in University system in the US and elsewhere but also other reasons some of which discussed and yeah they're going to have the same problem right it's sort of like well why would the best okay the best teachers who you could say but probably more like the best researchers the most famous academics they already have well-paying prestigious positions why would they go to this weirdo new University uh and likewise why would the students go there unless the famous professors were there and so on so it sort of takes some kind of extreme circumstance and the sort of proposition that is the founding issue of University of Texas at Austin is to do with a lot of rather high profile prestigious researchers and academics basically feeling pushed out of the existing system for political reasons and that's opened potentially a gap for a new institution to be founded but they really do need to be sort of gaps like that at least you know without further technological disruption but I do think to maybe wrap up I do think this the the role of AI tutors is a big wild card I think people are very skeptical of technological innovation and education because a lot of breathless predictions about you know moocs and online learning and other forms of technology technology LED educational improvements haven't panned out uh but I think I do think the the coming AI services are really different I don't think it I don't know how it's going to shake out but I think it's uh things are going to change yeah it's it's I think this is this is also a familiar pattern in the history of disruption that we see again and again which is once the new technologies become they once they leave the laboratory and the r d environment and they they reach the this the stage of scaling and deployment or deployment and scaling whichever way around you want to think about it one of the
standard conundrums that we face is we start to we start to look at and I say we but that you know public policymakers investors every everybody will begin to look at that at the situation and recognize changes are coming you know uh opportunities are there potential risks are there and so forth but one way one way or another the game is going to change uh how do you anticipate and plan for the details because that's the million dollar question it's it's you know it's what we think X tries to spends a lot of time trying to understand and trying to anticipate trying to formulate trying to map out possibilities so that other people can also join in the effort of thinking through them and evaluating them um and this is very familiar right it's it's I think it's now quite clear that these new technologies are poised to transform education not just higher education but all of Education and it is now the time to make a very concerted effort and commit real resources to asking that the question how could things be transformed what ways would be more desirable in what ways would be less desirable what new possibilities exist and what benefits go with those what pitfalls and Hazards and downsides might might also emerge and how do we plan for anticipate and plan for those so that we don't get blindsided by them as we have in the past with other disruptions and uh I mean basically just this is this is the task now is to do exactly what we're doing which is ask the question and really try to think through things carefully so that we could make some sort of uh effort to be prepared because the change is coming that's the thing is this is the change is a tidal wave now barreling towards us and we can you know people who are paying attention see it on the horizon a little bit sooner than others you know maybe you need binoculars at first in this analogy but you know it won't be long before the tidal waves visible to anybody who who's who's you know halfway looking at at the Horizon and then you know at some point you'll be able to hear it roaring as it's approaching no matter where you look and and you know you could extend the metaphor but the the point is that it it's it's clear now that huge change is imminent and the question is what what parts of that change are amenable to control through decision making and what can we really anticipate what you know how can we maximize the benefits of minimized arms all of those usual questions and what principles ought to guide us I have a little bit in my new
book that's coming out that's got some guiding principles for that that apply across the board to most disruptions um but this is sort of a perennial challenge as you know the world's about to change in a very big way oh [ __ ] what do we do you know how do we think through that and and we don't have a perfect way of doing that yet um and yeah you know my team's trying to develop some tools and maybe a bit of a theoretical framework and we've put together some examples from you know previous experiences that you know maybe we could learn from some of those but I think the bottom line is that it's it's impossible to know and predict very well this stuff in advance yet it's not a perfect or anywhere close to perfect science um of prognostication and uh almost certainly almost certainly or very high in the list at any end rate of what you can do about it is to have lots and lots of conversations just like this and many people have many conversations like this I mean if if you were to do nothing else in my mind this would be the most important thing to do is at least [ __ ] talk about it um and and you know think through it out loud uh so and this is crucial because if this Pro if this part of the process is not undertaken it will the the tidal wave will just wash right over us it we will get blindsided we will get smashed by it the old system will get washed away and everybody will just be standing in the aftermath you know uh I won't invoke any crude you know yeah idioms but everybody will just be standing in the standing around in the aftermath wondering what the hell happened and how did it all go wrong um and why was nobody prepared why did nobody see any of this coming and do something uh to you know to to you know or whatever and and or alternatively maybe there'll be a handful of winners that emerge because they were positioned well and every you know and and that that uh I mean that success could have been more widespread if if a more concerted effort were made to disseminate the insight about it and that kind of thing it's likely there'll be some winners because the tidal wave in this analogy in reality is not you know a natural disaster but there's going to be some people that are actually doing things doing the disrupting and that's the tidal wave right so at least they will be well positioned to benefit from nonetheless whatever they disrupt ends up destroying everything they value as well in some you know horrible accident yeah let me say one cynical thing and
then I want to go back to a comment you made earlier Matt so a cynical thing you could say is that the current education system delivers very large rents to a certain segment of the population so the upper middle class the administrators and lecturers and others benefiting from say the university system there's a huge amount of money that's sort of cream that's that's taken off the top of the American economy and to a lesser extent in Australia and delivered to that sort of slice of the working population um via the university system now do they deserve all those rents do I deserve those rents well maybe maybe not but the rents are there now in a different education system maybe it's the investors in open AI that that siphon off those rents instead but let's not pretend that there isn't rent seeking an enormous rents today and that that somehow will be a new thing uh maybe the the Constitution and size of the rents and who who gets them will be different but uh it's not um it's not that situation necessarily will be fundamentally but I wanted to go back yeah one aspect is I hadn't thought about maybe we can pick this up next time because I do think we could profitably talk about this for more than one week um yeah Adam you said after disruption the education system may be much larger that's a particular instance of a general pattern and disruption since we've discussed that's interesting and uh Matt made a comment in the chat that we didn't discuss about I think lifelong learning so maybe can we talk about that for a few minutes and maybe set ourselves up for next week so how could how could education be a larger system it just means you learn more stuff or you're learning all the time what what is larger about it yeah there's certainly like a new variable like for example the reason one of the reasons I guess we have three-year undergraduate degrees at the moment as like a you know there's maybe variations but um this tends to be the norm I think so um that's got to be I'm not exactly sure where three came from but maybe it's something to do with the amount of stuff that we want people to learn before they're ready to be employed in a particular area and maybe if you had a even more effective system and you could accomplish the same amount of learning um maybe if it increases the amount of sort of stuff that you are learning per year you could reduce to a two-year program and still have the same amount of uh skills picked up another option would be to keep the three-year thing
but then just learn more stuff um over the three years and come out even more skilled um I guess yeah changing the rate at which people can learn stuff um immediately changes the kind of um the the calculus for making decisions about how long to spend studying before you enter the workforce or studying before you move on to the next step of um I guess I'm thinking about University but this is also going to be true in um pre-university education although there I think it's much more about like maturity physical and mental maturity that we have school ending at around 18. so maybe in that case that will stay but we'll end up having much smarter high school graduates because the time that they spent in um in high school was more effectively spent um so they'll come out knowing you know more stuff um obviously that's not the only thing that people are in high school for they're also socializing and stuff but um but in terms of the amount of things learned the naive view of like measuring the amount of things learned per year um more effective teaching and education means yeah more well it's not necessarily more competition oh I guess for pre-ai people um that represents a threat that's pre-ai education people that represents the threat because your your education is kind of outmoded by by these new kids who have learned everything you learned in your first 30 years of life they learned by the time they're 18 and so and now you're competing with them in the workforce but maybe you'll also have access to upskilling from AI so maybe it's not such a problem thank you I'll come back to that upskilling in a moment when I explain how this new feature of the AI integration with meta uni Works which we'll spend a few minutes on but first yeah Adam do you have any comments on the larger no not yet not right now um yeah I'm I it's I I'm struggling to unpack the the the role and the value of Education versus the in higher rather I should say higher education I'm struggling to unpack education from the larger experience and um uh and without without being pretty cynical about it honestly um because we for specific skill sets we already have alternatives to higher education for example I mean there are and you guys probably know this much better than I do but in software engineering for example there are there are certifications and qualifications specific ones that you can become you know you you get these under your belt right I mean I I don't know what they are Microsoft a plus plus certified or
so I don't know what what the things are and um you can become certified you can take you know tests or get qualifications of all sorts of different kinds some of them are you know professional in the in the traditional meaning of that word the professions like taking the bar exam and becoming a lawyer um uh I think there are some there are similar examinations required for medical programs and other professions um there are vocational as opposed to professional and I'm using these sorts of in air quotes because the lines between profession and vocation are blurring and we'll probably continue to blur a new new forms of expertise are emerging um but we already have sort of a a a parallel track of education and qualification and certification that runs you know alongside the higher education system of formal degrees of undergraduate Baccalaureate degrees and and bachelor's degrees and master's degrees and phds and and some of the other ones JDs and MDS and so forth and and so um the thing that I'm struggling to to imagine and understanding here is whether or not A disruption of higher education namely the four-year you know the the four-year undergraduate degree and the advanced degrees um and the professional degrees uh is something that is going to you know is that is that is that going to create a an alternative that looks more like this kind of certifications you can get today you could spend six months and learn a programming language and get certified in it and then be able to do that work or you can you know you can get a electrician certification or you know I or you could could you pass the bar if you studied just for six months and didn't go through a full law program I mean presumably it's possible um the rumors that it's been done so I that's that's that would be my my confu my concern not because well I'm concerned I guess concerning confusion my question I guess that I would ask is um uh if higher disruption if higher education were disrupted excuse me then would that would the resulting new institutions differ substantially enough from you know the other parallel educational programs and certification opportunities and institutions that already exist with with that would that actually occur or um is it unrealistic to think that they that those that that education actually can be uh decoupled from the total higher education experience and we talked about all the features of the resort and the socializing in the networking and so the thing that so