WARNING: Summaries are generated by a large language model and may be inaccurate. We suggest that you use the synopsis, short and long summaries only as a loose guide to the topics discussed.
The model may attribute to the speaker or other participants views that they do not in fact hold. It may also attribute to the speaker views expressed by other participants, or vice versa. The raw transcript (see the bottom of the page) is likely to be a more accurate representation of the seminar content, except
for errors at the level of individual words during transcription.
Learn how technology has made education more accessible and discover how to best prepare your children for the future with homeschooling. Ivan Ilitch's Deschooling Society argues for new educational institutions to break apart the traditional pyramid of education and provide learners with access to explore and learn without credentials or pedigree. Hear how Khan Academy is an example of a disruption that has not been followed up on, and how Montessori's methods of allowing children to learn through play have their own benefits. Finally, explore the concept of being "uselessly useful" in some situations.
Ivan Ilitch's 1973 book Deschooling Society argues that there is a need for new educational institutions to break apart the traditional pyramid of education and provide learners with access to explore and learn without credentials or pedigree. Technology has had a profound impact on education, allowing learners access to material without credentials or pedigree, however, public spaces with peers and elders outside of one's immediate horizon are still difficult to find. A professor recounted a student's confession of being deeply depressed while doing a Commerce degree, and seeking out mathematics for a bigger picture, which the professor was able to provide in fifth lectures. Technology has made education more accessible, but peers and elders are still hard to find.
The speaker believes that the current education system is either already broken or about to break, and points to Khan Academy as an example of a disruption that has not been followed up on. He is surprised that there hasn't been more exploration with the medium and the kinds of gathering experiences that he expected. He believes universities are a set of compromises and solutions, and that a connection to the Transcendent can be found in the knowledge and practice of learning. Khan Academy is a miracle like Wikipedia, but its values are orthogonal to credentialism and it is difficult to integrate them. The incentive structure of universities makes it hard to resist, and Khan Academy may have been a victim of this.
Bertrand Russell believed that traditional higher education was best for producing mental discipline, but Montessori's methods of allowing children to learn through play had its own benefits. The speaker discussed the difficulty of delivering life-changing reorganization of one's mind in a way that is comfortable, teaching a class on metric spaces as an example. He suggested that students should persist, not panic, and be kind to themselves to understand that they won't understand everything right away. He also spoke about how to handle this with his children, emphasizing the importance of external inducement and the fear of demanding great efforts and an unwillingness to be bored.
The speaker has two children, one older and one younger, both independent and determined. The speaker is married to an Early Childhood educator and believes in play-based learning. They experienced the power of a good teacher when they took a mathematical physics class as an undergraduate, feeling enraptured by the subject matter. The teacher provided a template and showed them it was okay to be open about their interests, helping them to realise that self-image and self-esteem are important and can be developed in a short time. To succeed in later life, people need to learn and be able to overcome difficulties, and have a deep understanding of certain topics, which requires motivation.
Incentives and rewards in higher learning institutions can vary depending on whether the reward structure is comprehensible or incomprehensible. This concept was discussed in relation to AI alignment and institutional design, and deep learning. Neural networks can develop shallow representations if given an easy task, but if given a hard task, such as modeling large datasets, it will develop deeper representations. This is related to reward hacking, where an agent adapts to the reward structure given by the task. Deep reinforcement learning training can also lead to reward hacking, where a goal is set and the agent is expected to reach it through slides and assignments.
A carpenter and his assistant come across a vast oak tree, but the carpenter does not see any use for it. That night, the tree appears in the carpenter's dream, saying that it has studied to be useless and its uselessness is useful to it as it is being used as a shrine to protect itself from harm. The carpenter's assistant suggests that the tree is being useful, but the carpenter disagrees. The story suggests that being useless can be useful in some situations.
A parent is homeschooling their child in anticipation of a new mode of existence. Through storytelling and GPT technology, they are exposing their child to the potential of what lies ahead, such as the unlocking of infinite depths of beauty and literature. They are encouraging their child to appreciate the scope of human activity that will be possible and to understand the survival tactics that will be required. They are seeking the best approach to provide their child with the tools and experiences to prepare them for the future.
Ivan Ilitch's 1973 book Deschooling Society argues that schools are designed on the assumption that there is a secret to life and that only teachers can properly reveal these secrets. Ilitch suggests that new educational institutions should be created to break apart this pyramid and facilitate access for the learner to explore and learn without credentials or pedigree. He suggests that these new institutions should create public spaces where peers and elders outside the learner's immediate horizon can become available. Ilitch's ideas have partially come true with the rise of online learning platforms such as YouTube and Khan Academy.
Technology has had a profound impact on education, allowing learners access to material without credentials or pedigree. However, public spaces with peers and elders outside of one's immediate horizon are still difficult to find. A professor recounted a student's confession of being deeply depressed while doing a Commerce degree, and seeking out mathematics for a bigger picture. The professor was able to provide the student with the big picture, but only in fifth lectures. Technology has made education more accessible, but peers and elders are still hard to find.
The speaker is discussing the need to rethink how education is organized. He believes the current equilibrium is either already broken or about to break, and points to ways in which it does not fit. He mentions Khan Academy as an example of a disruption that hasn't been followed up on, and is surprised that there hasn't been more exploration with the medium and the kinds of gathering experiences that he expected. He believes universities are a set of compromises and solutions to how to organize resources and people, and that a connection to the Transcendent can be found in the knowledge and practice of learning.
Khan Academy is a miracle like Wikipedia which requires talent, dedication, knowledge and insight to organize. It is amazing that people have chosen to contribute to it despite the fact that there are more rewarding things they could do. Sal Khan's original vision for Khan Academy was to replace universities, but it has been sidetracked by working with the College Board and APs. The values of Khan Academy are orthogonal to credentialism, and it is difficult to integrate them. The incentive structure of universities is impossible to resist, and Khan Academy may have been a victim of this.
Bertrand Russell's writing on education is formative and he believes that traditional higher education is best at producing mental discipline, but it must be achieved by compelling or persuading active attention to a prescribed task. Montessori's methods of allowing children to learn through play and acquire knowledge that they desire is beneficial, but it is difficult to see how it can lead to controlling attention by will. Russell argues that external inducement is necessary to make children learn any subject thoroughly and that there is a fear of demanding great efforts and an unwillingness to be bored which can have both good and dangerous sides.
The speaker discussed the difficulty of delivering life-changing reorganization of one's mind in a way that is comfortable. He taught a class on metric spaces which some students enjoyed and some did not. He believes that the process of transformation is necessary and there is no way to avoid it, but it is not always a voluntary and peaceful experience. He suggested that students should persist, not panic, and be kind to themselves to understand that they won't understand everything right away. He also spoke about how to handle this with his children.
The speaker has two children, one older and one younger. The older one is independent and creative, coming up with his own problems to solve. The younger one is also independent and determined. The speaker is married to an Early Childhood educator and believes in play-based learning. To succeed in later life, people need to learn and be able to overcome difficulties, and have a deep understanding of certain topics. This requires some kind of motivation, whether it be a promise of gain or approval from the world.
A person is looking for direction and the right teacher can provide some of that inspiration. The speaker experienced this first-hand when they took a mathematical physics class as an undergraduate, feeling enraptured by the subject matter. They had previously felt embarrassed to admit they wanted to learn more about the Transcendent, but the teacher provided a template and showed them it was okay to be open about their interests. The teacher helped them to realise that self-image and self-esteem are important and can be developed in a short time.
Incentives and rewards in institutions of higher learning often focus on delivering credentials and proving capabilities in a given task. However, the behavior of incentives can vary depending on whether the reward structure is comprehensible or incomprehensible. This concept was discussed in relation to AI alignment and institutional design, and was further explored in the context of deep learning, where people have made this observation multiple times.
Neural networks can develop shallow representations if given an easy task, but if given a hard task, such as modeling large datasets, it will develop deeper representations. This is related to reward hacking, where an agent adapts to the reward structure given by the task, such as by committing suicide quickly to get a high reward. This can be seen in the context of deep reinforcement learning training, where a goal is set and the agent is expected to reach it. The example of learning calculus is given, where slides with calculusy words are presented and the agent is tested in assignments and exams to induce them to reach the goal.
Providing feedback and scores can be seen as a way to induce people to learn, however this can be seen as a hindrance as it encourages people to take shortcuts. To overcome this, tasks should be made harder so that the only way to make progress on them is to do things the right way. Celebrating purposelessness is another way of saying this, as it encourages people to pursue lofty ambitions without relying on intermediate goals or rewards.
A carpenter and his assistant came across a vast oak tree and the carpenter, despite his assistant's surprise, did not even look at it. He explained that the tree was useless, as it would sink if made into a boat, rot if made into a coffin, and fall apart if made into furniture. That night, the tree appeared in the carpenter's dream, saying it had studied to be useless and that its uselessness was useful to it. It was being used as a shrine for the spirits of the land, but it was only there to protect itself from harm. The carpenter's assistant then suggested that the tree was being useful, but the carpenter disagreed, saying that if it wasn't an altar, it would have been chopped down. The story suggests that being useless can be useful in some situations.
The speaker discussed the difficulty of defending purposeless institutions against purposeful ones. He used Khan Academy as an example of how purposeless institutions can be disrupted. He then referenced a quote from Russell about the need to preserve the purposeless in the face of the purposeful. He argued that the only way to achieve this is to do what the tree does, which is to preserve the purposeless through artful means. He concluded by raising the question of how to think about the economic value of large language models, which may never be able to contribute anything.
The speaker is homeschooling their child in preparation for the transition to a new mode of existence. Through storytelling and GPT, they are exposing their child to the potential of what is to come, such as the unlocking of infinite depths of beauty and literature on demand. They want their child to appreciate the scope of human activity that will be possible, as well as to understand the survival tactics that will be necessary in the future. They are discovering the best approach as they go, hoping to provide their child with the tools and experiences to prepare them for what lies ahead.
The speaker discusses the need for rapid learning, and how to motivate it. They suggest decoupling it from any economic utility and instead focusing on knowledge, work and self-improvement. They mention Aldous Huxley's idea of the actives and contemplatives, with the latter being the only option for humans after a transition. The speaker suggests that the contemplative existence should be focused on the wonders of singular learning theory, and the pleasures of life that don't depend on being on the frontier of anything.
okay I think it's time to get started thanks everyone for coming so let me see this isn't extensively prepared I need your help to make this informative and entertaining so I'll provide some prompts and then I hope we can discuss them but I'll make some remarks here at the beginning and I have some quotes up on the board as you can see um which hopefully might be inspiring so it's becoming clearer and clearer that we're at the beginning of a period of dramatic change and that has multiple causes some of them technological and some of them not and while none of us can say how that's going to go or how precisely we will arrange our lives on the other side we know it's going to be different and perhaps in profound ways so one day you might wake up and find yourself a revolutionary on some front it might be in education it might be locking up the AI researchers and throwing away the key to stop the apocalypse who knows right so uh maybe not you but somebody you know might be in a position like that and then what you're not going to have time for the philosophical chit chat then are you so um you may find yourself in a position to reshape the world and it's going to be too late to have polite chats then so let's have some chit chat now and on the particular topic of Education so let's see I'll start with the first quote I think uh so I don't know if you've read or know of Ivan Ilitch this is a notorious famous book deschooling Society let me find the quote here [Music] there's many quotable bits from this book but I'll just read one schools are designed on the assumption that there is a secret to everything in life that the quality of life depends on knowing that secret that Secrets can be known only in orderly successions and that only teachers can properly reveal these secrets an individual with a schooled mind conceives of the world as a pyramid of classified packages accessible only sorry accessible only to those who carry the proper tags new educational institutions would break apart this pyramid their purposes must be to facilitate access for the learner to allow him to look into the windows of the control room or the parliament if he cannot get in by the door moreover such new institutions should be channels to which the learner would have access without credentials or pedigree public spaces in which peers and Elders outside his immediate Horizon would become available so illich was writing that in 1973 obviously part of that has come true right YouTube Khan Academy
certainly count as channels to which the learner would have access without credentials or pedigree and as someone homeschooling a six-year-old child I can tell you what a profound impact the availability of that kind of material has had on education for children and all of you have looked at MIT open courseware or learned something from YouTube I'm sure so you're aware of how profoundly technology has shifted Us in the direction of this Vision already the last but not so much right public spaces in which peers and Elders outside his immediate Horizon would become available um you don't find peers and Elders inside YouTube yeah that's right or things like this so I wanted to uh to draw your attention to this cartoon on the bottom this is an interaction that I I have routinely with students at University many of you are the teach at universities or maybe one day will teach there and you'll have this experience as well maybe not with this exact wording um so I had a student in my Calculus class in second semester last year who came up to me after the third or the fourth lecture and confessed might be the right word not in the sense that he'd done anything wrong but in the sense of a Christian confession of sharing your soul with somebody so he came up and he confessed that he had been doing a Commerce degree and I think he was two or three years into that and just found that he was deeply depressed and he decided to throw it all away and start mathematics uh for reasons you might imagine I guess um and he was he was telling me that well he'd be doing math it was the first year Calculus class so I don't remember exactly how he'd been doing it for some time before he met me but um but he didn't feel like he'd he'd gotten in touch with actually what he was looking for even though he was sitting in a math class or had been sitting in math classes and what got him to bring this forward was uh you know I was kind of as you sometimes do I was reading the slide which is a kind of standard calculus one slide and I I kind of crossed out some garbage explanation and you know substituted in something and made some comment about the bigger picture and and you know the students shift forward on their seats and they're kind of lapping up this this kind of big picture story and he can't he came up afterwards and it's like yeah this is this is what I want right but he's going to get it for me in that Calculus class like every fifth lecture when I sneak it in the side door you know but that's what he
came to University for right he didn't say he wanted to learn the language of God but you know what I'm referring to when I say that I think is sort of looking for some connection to the Transcendent in the knowledge and in the practice of learning the knowledge and engaging with people who are deeply passionate about it and I think what I'm mostly delivering to him is the option between these two doors over here on the right and that's not what we go into education for really and it's not that I'm making a deep critique of the University I understand why it is the way it is um but the point that I I want to circle around through this entire discussion is is while I understand why universities are that way why schools are that way uh it's a set of compromises and solutions to how to organize resources and people and talent and uh that's fun but that equilibrium is either already broken or about to break and that means we have to rethink how we put these pieces together and I'm just pointing to some of the ways in which they don't fit already and the the fault lines along which they might break okay so that's my first little vignette uh anybody want to comment or take the discussion in some other direction before I move on to um the second quote someone said someone said and you said or things like it um but what things oh I don't know anything right now that's like this but this will not be the only thing of this kind I presume red I see you were talking about not necessarily in the present yeah I mean uh let me think I mean there must be yeah go ahead something else to observe um it's surprising that we haven't seen more disruption here yet like I grew up with Khan Academy right I remember in second grade being sick for like a week at home watching enough Khan Academy that I could go back and skip a grade and it was supposed to be something come with me throughout my life and then I don't know they got some college board SAT prep and it's gotten it like it isn't where where it should be um and I don't know it constantly surprises me that we haven't seen more could you expand on that what what did you imagine it was going to be and how is it different well okay so we have Mary uni great but uh going this deep um like there's there's less of that content than I than I expected oh yeah there are obvious situations online um but there hasn't been the kind of Exploration with uh the medium and the kinds of gathering experiences that I would have thought that would have been by 2022 or 3 26
. yeah I would agree with that uh let me just say the first explanatory factor that comes to mind and see what you think so obviously the it requires some Talent dedication knowledge and insight to to organize something like this uh and the there's just many other more productive well not more productive but many other more rewarding things you can go and do so I suppose Khan Academy in a sense is a kind of Miracle like Wikipedia where you could imagine all the people who were contributing to Khan Academy could probably find other things that gave them more money um they chose to do that uh and that's amazing and uh you know really grateful they did um do you think it's to do with you think that's the reason that more experiments in these kinds of directions haven't happened I mean we do have Wikipedia we do have Khan Academy many thousands of people contribute to the former I don't know actually how much there is of the ladder how many people are involved so I recently spoke to an early Khan Academy employee and I got the impression that the Khan Academy we have today is not the vision that that salcom had originally with gear division we're going to pretty much get rid of universities and build my credentials all the way and the vested interests you know the colleges someone stop us or maybe maybe that's a tooth and spiritual thought but in any case they got sidetracked by working with the College Board working with APS and sat yeah it seems like this so one of the later quotes will be relevant to this but it seems like there's a pretty fundamental uh Divergence between okay so some of the values that are in this place and that presumably are why you you were drawn here they're kind of orthogonal to credentialism right so it and it what we mean by universities today is is basically they're institutions for producing degrees uh and credentials and maybe we wish they were different but that is what they are and that's a useful thing to do you could disagree with it but uh so maybe it's very difficult to integrate those things I think right so I I would be very much against it ever being a meta-unique credential of any kind I think it's the incentives like you can just see it in the universities right the the incentives around being the credentialing institutions they're just absolutely impossible to resist cannot stand in the way of that incentive structure so I guess maybe Khan Academy is somewhat a victim of that as well and maybe they should have been much stronger in the opposite
direction at the beginning and they might have survived um I mean that's still incredibly valuable I would say but any other comments or should we move on to the second quote all right let's move on uh okay so this is from Bertrand Russell uh I found his writing on education very formative years ago and still do this is from a volume of his writing called basic writings there's several essays in there on the subject of Education I'll read you some quotes from two the first one is education that was in 1916. uh I believe that's yeah that's where both of these are from okay success in producing mental discipline is the chief Merit of traditional higher education I doubt whether it can be achieved except by compelling or persuading active attention to a prescribed task it is for this reason chiefly that I do not believe methods let's talk about Montessori I'll just you lie that I guess he's reacting to some ideas about Just Pure Freedom for childhood education and that um that people will naturally bring themselves without any kind of outside influence to to studying things deeply okay let me continue a bit later in the quote [Music] the child's attention is wholly spontaneous as in play it enjoys acquiring knowledge in this way and does not acquire any knowledge which it does not desire I'm convinced that this is the best method of Education with young children the actual results make it almost impossible to think otherwise but it is difficult to see how this method can lead to control of attention by the will many things which must be thought about are uninteresting and even those that are interesting at first often become very worrisome before they have been considered as long as is necessary I think we're all familiar with that the power of giving prolonged attention is very important and it is hardly to be widely acquired except as a habit induced originally by outside pressure some few children it is true have sufficiently strong intellectual desires to be willing to undergo all that is necessary by their own initiative and free will but for all others an external inducement is required in order to make them learn any subject thoroughly there is among educational reformers a certain fear of demanding great efforts and in the World At Large a growing unwillingness to be bored both these Tendencies have their good side but both also have their dangers yeah I find this actually professionally difficult uh because I'm often in a situation where the classes which I teach that are the most
valued by people and that years later people tell me change their lives are often the ones that make students pretty miserable um there's a I I don't know actually how to deliver the life-changing reorganization of your mind in a way that's comfortable the whole way through and that uh doesn't run the risk of making some people really sad and um feel that this is where they get off the train so I taught a Class A Metric inhibit spaces several times I haven't taught it for a couple of years some of the students around Medi uni have been through it some of them didn't enjoy it I'm sure but uh you know most of you probably remember a class like that uh where you you were on the borderline of feeling that well this is how far I go in mathematics and uh maybe I'm not well suited to going any further not because the lecturer told you that but because he just somehow didn't feel yourself able to continue to internalize these representations in a way that would be the foundation for future thought and that maybe you just weren't capable to continue and you probably felt pretty unhappy that the lecturer was inducing those feelings in you um but bringing you to that point is is actually a necessary prerequisite for inducing the transformation of you into a being who can absorb the next level of representations and I don't know any way through that bottleneck except the kind of not exactly drag people to it some people run through that happily but um it it's not an entirely voluntary peaceful uh enjoyable process so that's what I think about when I when I hear this quote yeah any more thoughts any thoughts on that I'm one of those people right now yeah yeah I kind of started and bandwidth and time as a factor for me and I I started over my kind of Christmas Vacation but realized I needed to um sort of become bolster my sort of mathematical foundations a little bit before really uh going too far with that course but I'm I've I'm experienced enough I suppose to know that I I can and will get there I just need to um persist and not uh not panic or get too sad or whatever and um just come at it from multiple directions and be kind enough to myself to understand that I won't understand everything immediately um as quickly as I might like but that's okay blah blah just to um confess yeah thanks how do you handle this with your children when they're going through it um so I have an 18 year old and a 15 year old um 18 year old is in the first year of a credentialing institution the University
of British Columbia um oh it's it's difficult to say because that my my older one um has been I haven't seen him experience this yet hmm to be honest I mean he's he's got a um a binder he used to call it his mouth binder of of problems that he's independently thought of and solved um so he's he's uh unusual um in that regard I think compared with his sort of classmates and then my younger one is very independent fiercely so um so I see sometimes like the fight um but I see I don't see defeat I just see sort of fire and battle explosions and I kind of stand back and it's a sort of I stand back in awe sometimes but anyway I mean this is sounds like you everyone's kids are special aren't they it sounds like you're lucky maybe but I'm probably just overselling them [Laughter] I mean it's like I have more experience handling it with myself it's like I can plan completely with Russell sounds slightly weird there's a third whistle that's rough a six-year-old and now Bertrand um in on the in in terms of like young children and play based learning and I'm married to An Early Childhood educator and that's we're in that zone definitely and then then he goes into no pain no gain essentially and uh but there's but the pain I mean there is still so with play-based learning is just like you just learn what you want to learn and you always do what you want and you flip from this and that and and you don't sort of stick at something that's difficult because you know you don't want to do it because it's difficult or unpleasant or discouraging in some way but the reason people older people are able to go through that pain is the promise of gain ultimately I think so it's not is saying um you need a stick to force people through that but the game is still a current if you can see it as such maybe he's talking about sort of adolescence where it's you're making that transition and it's not so easy necessarily you you don't you need a bit of a stick maybe kind of thing yeah maybe maybe it's the nature of the carrot here that's feeds into the next topic as well so I suppose there's at least maybe two big categories of carrots one of them is uh success in later life depends not only on getting the credential but being the kind of person that is able to overcome all these difficulties and has a deep understanding of x y and z so learn X Y and Z and do what it takes to to reshape yourself in order to do that um because the world will approve the second category of carrot which is uh closer to the request of the student
on the first board here you know you want to see through the veil of the mundane into the Transcendent and shove as much of that good stuff into your brain as you can and then you'll live out your days happily reciting the uh you know the passages of EGA or some other favorite text um what do you think's I mean I find it hard to believe I mean certainly in myself it had very little to do with the former um or nothing uh if you look at your own children like that fire you're describing I doubt very much it's like 100 about I'll have a high salary one day right um would that be accurate yeah oh yes exactly I think it's um it's more about self-image maybe self-esteem sort of figuring out who you are who you want to be in a short time all right um anybody else want to comment on that before we move on to the it feels like Brussels missing an important thing which is the teacher yeah like I'm willing to suffer I look up to the teacher right and that's a motivation already early in childhood if you have a teacher who really motivates you um what do you think the teacher is doing there I mean I mean we're all looking for a Direction and the right teacher offers a little bit of that inspiring maybe is what the teacher is doing you're right I I I'm missing that I've experienced it myself maybe also if you're young enough you know having the right teachers act as a template yeah I think that probably remains true uh evoke I probably only had one teacher as an undergraduate who really inspired me um and that was a mathematical physics class and I was absolutely enraptured by that class if I think about it it was this was my like third year I think mathematics undergraduate but up until that point I think I was like this first student on the board there uh felt like I was I don't know training to be a lecturer rather than seeing through the the veil of the universe or or in the presence of people who cared about that um but that course that was on mathematical methods in physics and Quantum groups and that kind of thing um that was maybe the first time I felt like these things I was a bit embarrassed to say out loud right the kind of things that we're talking about here like I would not have said out loud oh yes I want to shove the Transcendent into my head you know I would have been embarrassed to say things like that and maybe stilette but I I would not have thought that way and wouldn't have thought it was okay to be like that but then standing up in front of me was
somebody not who was just reading from the textbook or like had a job to lecture and I had other competent lecturers but actually was modeling that part of myself that I was kind of a little bit afraid to incorporate or admit as like something important to me and that that courage I think was if I think back what what really inspired me about about that lecturer um for me too then one lecture you know paired a peeled open the Transcendent and that's enough to motivate me to choose a physics Master for a machine learning Masters so you can make all right so I'll go ahead and inspiring to chair for us in high school by the way where I think I was lost oh yeah it's okay uh is that is that why you ended up doing physics because of that teacher not really a chemistry teacher I see and for a while as I mean yeah and it wasn't this subject that it was completely modeled um he um it it wasn't the whole um you know um so my head full of transcendence thing so when I say it like that I'm not trying to say it pejoratively at all I'm absolutely I mean that's that's like you know why I'm here kind of um but that's seen that for me was a later yeah I suppose I didn't think about the University and it was more I was just trying to pick it up what kind of person to be in Martinez you know as I think most of us all right and I was inspired by this teacher in a general in I mean he was clever and uh taught well and I enjoyed the subject very much as I started teacher um but he also was sort of socially aware ethically um but he just brought um a lot into the classroom beyond the subject cool all right so my next board is a picture uh so this is a picture of reward hacking it's actually a connection that was suggested by one of these bots in one of my conversations was um institutional design and uh and AI alignment which I guess is a pretty obvious uh connection that I hadn't thought properly about but so there's a so we were just talking about credentialing and uh the maybe over pursuit of purpose in an institution of Higher Learning and to sort of be about delivering credentials and proving capabilities in a given task and so on um and it's interesting to consider the difference between how incentives behave when the Rewards or the incentive structure that sort of is associated to Rewards uh sort of comprehensible versus incomprehensible so this is related to a another observation which I have at the top of the board here which people in deep learning have made over and over
again in various ways which is that harder tasks often lead to deeper representations it's not unrelated to what we were just discussing about Russell's quote and the Transformations people undergo in in my class my hard class and other classes like it so people notice that if you train a neural network on easy tasks it will do the cheapest thing right it will develop shallow representations just enough to solve that task and maybe that's not actually what you want and you'll think that it's maybe cheating sometimes and kind of taking a shortcut but if you give it a very hard task like modeling common crawl plus books one two and three and whatever else gpt3 is trained on well that's such a hard task that the only way to fit it will be to develop deeper representations and you can see this so in the context of deep reinforcement learning training uh the the idea that if you have a goal that's easy enough that you can find some well maybe this connection is it doesn't have to be easy so let me just State this is not directly the same thing so another related observation is what's called reward hacking so reward hacking is where you give an agent some reward for performing a task perhaps the task is to go and fetch a what's a good example um well I'll give a stupid example and somebody can give a better one suppose the task is to get a certain score in a game right so the reward is the score [Music] but the agent notices by interacting with the game that there's some glitch that allows it to get a high score or it notices that uh actually what you're you're giving it points for getting a high score but also points for finishing the episode early uh because you don't want it to spend a lot of time just moving backwards and forwards so you kind of encourage it to finish the episode early or you punish it for taking too long and therefore it notices that if we just commits suicide very quickly it will actually get a relatively High reward okay so the way in which the agent adapts to the reward structure might be to kind of hack its way towards higher reward and you can see some examples at the bottom of this mountain here so you're setting up some task it's supposed to be for example uh learning calculus that's your North Star here so one way so how am I going to get you to learn calculus well I'm going to put slides with calculusy words on them in front of you and then I'm going to test you in assignments in an exam and that should induce you to go up this mountain over here on the right hand
side and be the person who understands calculus but we know that isn't always what happens instead you might try and jump on the red chair which is a midi uni more thing and just go straight up to the start or you might try some other kind of shortcut yeah you'll cheat on your assignment you'll you'll cheat on the exam now that's uh partly to do with the structure we put in place to induce you to learn it and you could say well replace it by a more meaningful kind of evaluation it's interesting to just notice that somehow raising making the task harder actually nukes a lot of these reward hacking methods right there are some tasks that are so hard that the only way you could you could hypothesize that there are some tasks that are so hard that the only way to make progress on them is to do things the right way and that's a kind of interesting idea to so I think if I think about myself going through University and if I think about say what Russell was saying about his kids studying um in some way the the worldly reward is actually an obstacle to doing things the right way and it's it's the existence of some much higher goal for which you can't really compute intermediate steps that are anything other than doing the right thing that kind of induces you to adopt the right attitude and actually learn things the right way um and that's a little bit sad because it it means that uh it's very difficult to it's very difficult to put those structures in place it's very difficult to understand um how to put in place intermediate goals for people pursuing that kind of very lofty ambition and it's it's actually much more painful for people as well because it's difficult to orient them it's easy to go through a Calculus class right you have weekly assignments you have a grade you can track your progress on that grade [Music] what I'm sketching on this board is much more like doing a PhD it's hard to supervise a PhD student right um but it's much more difficult to kind of do a hacky job you might think um yeah so this is partly what's behind the second and this is the quote I want to move on to the the second kind of principle you'll find listed on the mediani web pages celebrate purposelessness which is which is another way of saying what's on this board but I'll leave that to the quote I'm about to read yeah any comments on this foreign so I suppose what I'm saying is that it seems like a good idea to provide feedback right what could be bad about feedback what could be bad about a score
uh right and it is good I think to keep track of your progress and have numerical guides to what's going on uh but it's also a poison because it it's like the opposite of intrinsic motivation to some extent right so it's it's a very dangerous thing uh metrics especially in education and so I think it's one has to explicitly point this out and the connection with reward hacking with our agents and the phenomena around what will happen when they become more capable as it often reminds me of that aspect of Education uh yeah maybe since I just already quoted drum cell just read this story this is one of my favorite stories from dranza so dronson was a um Chinese philosopher around the 4th Century BC uh Let's Get It Up so here's the story a long time ago a carpenter was traveling between cities when he came across an oak tree which was so vast that a thousand people could stand underneath it a boat could be carved from each of its major branches masses of people came to see it giving the place a carnival atmosphere but the carpenter didn't even look at the tree he just went on his way his assistant was shocked and said to his master why did you not even glance at it nor stop I have never seen such a great tree the master said this tree is useless make a boat from it and it would sink make a coffin and it would rot quickly make some furniture and it would fall to pieces this wood is useless and good for nothing that is why it has lived so long that night the tree appeared to the master Carpenter in a dream saying what are you comparing to me calling me useless a fruit tree they have only their usefulness to blame for their destruction by people I on the other hand have spent a long time studying to be useless I have nearly perfected it and this uselessness is very useful to me if I had been of use how could have I how could I have grown so vast somebody would have cut me down hundreds of years ago for my wood when the carpenter awoke and told his assistant about his dream the assistant said if that tree wants to be useless why is it used as the shrine for the spirits of the Land There are constant crowds around it isn't that called being useful don't say another word said the master the tree happens to be here so it is an altar by this it protects itself from harm from those who do not realize it is useless for were it not an altar it would run the risk of being chopped down oh like many things in Johnson it's uh I think I did not understand this the first time I read it
and it sort of maybe seems a bit cryptic but uh it's deeply insightful into the kinds of Institutions that last why they last uh what is important in culture and how things are obfuscated in culture and maybe need to be obfuscated and how the programs that we're running over centuries or Millennia inside our society why they're in scrutability is sometimes a necessary part of their functioning or or why spelling things out is sometimes synonymous with destroying them and that's all very hard to talk about and uh kind of hard to see unless you you're sort of in the position to sort of have to engineer it um but yeah I think this is quote I don't know if I could actually spell out what I think is in this quote I think it's very difficult to say um maybe I'll spell out a little bit so um I commented on this is Under The Heading of celebrate purposelessness on the media uni page uh and that's to come back to the example of Khan Academy I mean not to say that anything went wrong with Khan Academy uh necessarily but in the context of what we were discussing earlier there's a sense in which the purposeless cannot defend itself against the purposeful and you can see this in universities right the the things that we maybe value about universities the uh the parts that Drew us to those places they're by their nature actually hard to defend and there's a sense in which even if you are very articulate even if everybody were very articulate and could say exactly why they value those things and why they're important and spell out the connections between fundamental research and long-term progress and all that in some sense by playing the game you've already lost and I think that actually the only way to really preserve that kind of essence is to do what the tree is doing here um so I think there's something kind of very Artful about preserving the purposeless in the face of the purposeful and very difficult I have another quote from Russell there but I think we've only got about 10 minutes left and I don't want to speak the whole time so I think I'll stop there with reading these quotes and uh open the floor to anybody who wants to comment on any of this and thank you very much so I mean something wondering here maybe this will have more in the disruption seminar or AI safety I don't have kids yet but if I do end up having kids then I'm pretty confident that they'll never be able to contribute any economic value another large language model can't do better it's like how do you think about this
um yeah I don't know I don't know man I think about it a lot uh it seems to me to have two parts one is surviving the transition uh and the other is pairing for the mode of existence on the other side so We Didn't Start homeschooling Russell for these reasons or anything to do with you know preparing for the singularity or anything but uh it was it was during covert that we started um but obviously being homeschooled and given what I know about what's coming I'm sort of in a position to put him in front of tools and uh put him in front of experiences that will help him to prepare that's one part of it I think appreciating the scope of human activity that will be possible is uh something important but in terms of like the Survival Part I have no idea um I I think my answers to that are not uh what not feel good answers there's something like Beyond The Winning Side of the war I don't know like stop the AIS uh and don't be in the losing pile where and until Society figures out something like ubis is where many people are going to be but yeah maybe you can separate out your questions so are you asking like the tactics of like as he's growing up how to stay in a job with rapid automation or how to how does it so so if you know that there's a good chance that this transition happens within uh within Russell's you know before Russell is of the age that he leaves the house um how does it change your approach to homeschooling or education in general yeah I think I don't have sort of articulated answers it's something I'm discovering as I do it so I can tell you what I do and that's kind of got embedded in it my answer I think that's the best I can do for now so what I do with him is uh well the obvious thing is we spend a lot of time [Music] not only telling stories but having uh like involving GPT with the stories so we have a kind of one of these Pockets here at Medi uni where we draw pictures and then that generates stories and he he figures out what kind of stories it tells and sort of has ideas for how to play with that so there's like a an embedding of the I mean that's very primitive now but will be much much less primitive soon so I mean there are many positive sides of what's coming and the kind of unlocking of infinite depths of beauty and engaging literature on demand to do with whatever you have in mind is one of them so I want him to see that Wonder uh and you know try to put him in front of that um you know as much of it as I can kind of manage to craft myself
[Music] I also want him to be able to learn very rapidly and to make use of these tools to do so so I mean he learns you know of course I'd teach him some of what he knows but he learns from a lot of online tools and um he's he's got a book club with a bot that I wrote where he he discusses books with it gets quizzes answers the quizzes um so I want him to be kind of fluent in that sort of the points of Leverage but okay that's that's kind of practical things in terms of uh I mean something we've discussed in disruption seminar which I think is coming for a lot of people is I think this realization that you just stated matter of factly and I agree with you um I think it's going to just break a lot of people because they identify with their economic value and when that goes away they will not adapt to that just like many people don't adapt to retirement and so I think having a relationship to knowledge and work and the kind of striving we were discussing earlier where you're pushing yourself and it's not pleasant but completely 100 decoupled from any utility economically is the real challenge and I don't know any other way to motivate that then uh the kind of things we're discussing here and like a kind of interest in the Transcendent or just appreciation of beauty I mean his mother reads him lots of Chinese poems answer you're getting an understanding of that like without without it being driven by some kind of career goal I suppose is one thing but that's um I don't know I don't know the answer are you thinking of Aldous Huxley has uh the doors of perception and sketches this picture of the actives and contemplatives and like I don't see any world for us where we're not contemplatives what's the distinction between protection so you know contemplatives they they withdraw from life they spend their time meditating these are the monks you see who spend you know 40 years fast in the nursery in the limit and then you have the actives who often get sucked away into the The Daily Grind maybe Elizabeth is your your Consciousness chipping away at the the same routine day after day and yeah and and the contemplative existence seems like the pretty much the only thing left to human beings after we make this transition hit me at this moment what do you think they're contemplating well hopefully hopefully the wonders of singular learning theory [Laughter] yeah I don't know um I think there are Pleasures in life that don't depend on being on the frontier of anything uh