WARNING: Summaries are generated by a large language model and may be inaccurate. We suggest that you use the synopsis, short and long summaries only as a loose guide to the topics discussed.
The model may attribute to the speaker or other participants views that they do not in fact hold. It may also attribute to the speaker views expressed by other participants, or vice versa. The raw transcript (see the bottom of the page) is likely to be a more accurate representation of the seminar content, except
for errors at the level of individual words during transcription.
Solid State Physics is a discipline that explores the relationship between the geometry of a crystal lattice and its properties, such as electrical, magnetic, optical and mechanical. It studies materials such as insulators, semiconductors and conductors, and has led to the discovery of graphene and the Van Hove Singularity. The Fermi energy, Fermi surface and the density of states all play a role in determining the physical properties of materials, which can be engineered with impurities and KL Divergence. Learn more about this fascinating field of study!
Solid State Physics is a science and engineering discipline that studies the relationship between the geometry of a crystal lattice and its electrical, magnetic, optical and mechanical properties. It focuses on the geometry of the level sets of the energy function that maps momentum of electrons to some real number, which is similar to the behavior of a learning machine. Insulators, semiconductors and conductors are the three classes of materials, with exploitation of existing materials and design of exotic materials based on microscopic properties being two periods in the engineering of materials. Kittle is the standard textbook for this discipline.
Materials can be classified as insulators, semiconductors, or conductors based on their band gap structure. Insulators have a large band gap of around 5 electron volts, meaning that electrons cannot jump from one band to another. Semiconductors have a smaller gap of around 1 electron volt, allowing electrons to jump with the help of an applied voltage. Conductors have no band gap, allowing electrons to move freely. The band gap can be altered by introducing impurities to the material, a process called doping, which can have global effects even though only a few atoms are changed.
Graphene is a material composed of carbon atoms arranged in a hexagonal lattice. Twisted Bilayer Graphene (TBG) is two one atom thick lattices of carbon atoms arranged hexagonally with an angle of 1.1 degrees relative to each other. When arranged at this angle, TBG becomes a superconductor at 1.7 Kelvin. Its electron band structure exhibits flat bands near zero Fermi energy and quantum oscillations in the longitudinal resistance. Its energy is determined by the KL Divergence, which has caused much excitement in the world of condensed metaphysics.
Van Hove Singularity (VHS) is an old concept from solid state physics associated with a divergent density of states in 2D systems. It is thought that when VHS is close to the Fermi energy, the increased density of states amplifies electron correlation, potentially explaining superconductivity. In learning machines, a similar phenomenon might exist, where a substantial change in an observable property is caused by fine-tuning a parameter. Density of states is the number of states per unit volume at energy e, which can be derived using Gilfan Kirilov and generalized functions in a continuous system. These concepts are used to explain the physical properties of materials.
Density of states is the number of states available for a particular energy level. For a free electron gas in three dimensions, the density of states is proportional to E^(3/2-1). Interaction with a crystal lattice can produce a more complex density of states. The Fermi energy is the energy of the topmost filled level in the ground state, while the Fermi surface marks the boundary between filled and empty states. The Fermi Dirac distribution gives the probability of an orbital at a certain energy being occupied and is used to calculate the electrical properties of materials.
The Fermi energy is the boundary between filled and empty states in a free electron gas. The Fermi surface is the level set of energy in momentum space, which can be used to gain insight into semiconductors. When parameters such as electric or magnetic fields, or temperature gradients are varied, the energy function and Fermi surface can change. This is important because the linear response of a crystal to a gradient is determined by the topology and geometry of the Fermi surface, and the probabilities of electrons moving between the valence and conduction bands are determined by the shape of the Fermi surface.
Density of States is an important factor in SLT, as it is used to calculate the scaling exponent of the RLCT. This is done by taking the logarithm of the integral of the prior over all states and configurations, divided by a positive number. Semiconductors have two types of carriers, electrons and holes, with an effective mass and a Fermi level halfway between the conduction band and valence band. Assuming Epsilon minus mu is much larger than KBT, the Fermi-Dirac distribution becomes a perfectly dense sphere, which can be used to derive useful facts about electrical properties of materials.
The probability of an electron in the conduction band being occupied is determined by its energy and the Boltzmann constant multiplied by temperature. The density of states for a conduction band electron is determined by the energy at the bottom of the conduction band plus a momentum term. The concentration of electrons in the conduction band is calculated by integrating the density of states with the energy from the bottom most energy to infinity. The number of electrons in the conduction band depends on the effective mass of the electron, the Fermi level, the gap between the Fermi level and the electron band, and the temperature. The number of charge carriers in the valence band is similarly calculated with the Fermi level being one minus the Fermi level from above.
Density of states in a semiconductor is determined by multiplying the number of electrons in the conduction band and holes in the valence band. This product is independent of impurity concentration and can be used to modify the properties of the semiconductor. Fermi surfaces and density of states are important to understand electrical properties of materials in solid state physics, analogous to KL Divergence in Singularity Learning Theory. Impurities can affect the number of charge carriers, either increasing or decreasing it, which affects the formula derived and can be used to make physical deductions.
Doping in learning machines is the engineering of valence or bands, similar to electrons in a conductor or semiconductor, to create something with desired properties. This could be achieved by introducing impurities into a pure feed-forward network to bring the bands closer together, making it easier to train. The product of temperature and a material property, such as the band gap, is known as NP, which is maintained by interactions with photons. Additionally, impurities in a semiconductor can be thought of as a choice of true distribution from which data is drawn, biasing level sets closer to the true distribution. KL Divergence can be used to control the band structure by engineering either the true distribution or the model. A stochastic dispersion relation could also be considered, introducing additional stochastic terms to the energy.
Solid State Physics (SSP) is a science and engineering discipline that studies phases of matter with many strongly interacting components, such as crystals. It focuses on the geometry of the level sets of the energy function that maps momentum of electrons to some real number. This is similar to the behavior of a learning machine, which is determined by the change in the geometry of level sets. SSP is not reliant on sophisticated mathematics, and can be understood with Elementary observations.
Solid state physics is the study of the relationship between the geometry of a crystal lattice and its electrical, magnetic, optical and mechanical properties. Kittle is the standard textbook for this discipline and its eighth edition is available. Exploitation and design are two periods in the engineering of materials, with exploitation being the use of existing materials such as silicon and design being the engineering of exotic materials based on microscopic properties. Insulators, semiconductors and conductors are the three classes of materials. The level sets of the energy of electrons interacting with the lattice are used to predict the electrical properties of these materials.
Electrons in materials exist in quantized energy levels that combine to form a continuum of energy states with gaps between them. This continuum of energy states is known as the band structure, and can be used to classify materials as insulators, inductors, or conductors. Insulators have a large gap between their highest and lowest energy levels, while inductors have a medium gap and conductors have a small gap. This gap can be altered by introducing impurities to the material, a process called doping. This can have global effects even though only a few atoms are changed, demonstrating how small changes can have large impacts.
Materials can be either insulators, semiconductors or conductors, depending on the energy band gap. Insulators have a large band gap of around 5 electron volts, meaning that an individual electron cannot jump from one band to another. Semiconductors have a smaller band gap of around 1 electron volt, allowing electrons to jump across with the help of an applied voltage. Conductors have no band gap, and electrons can easily move through the material. The band gap structure is what determines the type of material.
Graphene is a class of materials composed of carbon atoms arranged in a hexagonal lattice. Its electrical properties are determined by the structure of the energy bands. This structure is related to the density of states and the function that assigns energy to a given momentum for an electron. Twisted bilayer graphene is two one atom thick lattices of carbon atoms arranged hexagonally. It has been the subject of much press and research due to its unique properties.
Graphene is a two-layer material, with each layer rotated by an angle of 1.1 degrees relative to the other. In 2018, an experimentalist group discovered that when graphene is arranged at this angle, it becomes a superconductor at 1.7 Kelvin. This phenomenon is known as Twisted Bilayer Graphene (TBG). The electron band structure of TBG exhibits flat bands near zero Fermi energy, and quantum oscillations in the longitudinal resistance. Furthermore, there are small Fermi surfaces near the correlated insulating state. The energy of the system is determined by the KL Divergence. This new discovery has caused much excitement in the world of condensed metaphysics.
Van Hove Singularity (VHS) is an old concept from solid state physics that is associated with a divergent density of states in 2D systems. When VHS is close to the Fermi energy, the increased density of states amplifies electron correlation, which could be a potential theoretical explanation for superconductivity. This idea is applicable to learning machines, where a substantial change in an observable property (such as electrical) can be caused by fine-tuning a parameter (such as an angle). This can be compared to superconductivity and super intelligence.
Spatial dimension plays an important role in materials, allowing for a continuum of energy states. This is not the same in learning machines, but there may be an analogous concept of long-range correlations between different layers of a neural network. This could be related to context lengths in a Transformer, but the interactions don't change depending on the distance in the sequence of tokens.
Density of states is the number of states per unit volume at energy e. In a discrete system, it is a Delta function, with a value of 1 if e is equal to e i, and 0 otherwise. For a continuous system, it can be derived using Gilfan Kirilov and generalized functions. The speaker then explains how these concepts are used in solid-state physics to say something meaningful about materials.
Density of states is the number of new states accessible per unit volume when the energy increases from E to E+ΔE. For a free electron gas in three dimensions, the total number of states of energy less than or equal to E is proportional to E^(3/2). The density of states is proportional to E^(3/2-1). It is an increasing function of E, with zero being the only singularity. Every other energy level is just an expanding sphere, with no special energy levels.
Density of states is the number of states available for a particular energy level. A more complicated and interesting density of states can be obtained by having electrons interact with a crystal lattice, which changes the energy according to the symmetry of the crystal. The dispersion relation is a function that tells the energy of a momentum. The Fermi energy is the energy of the topmost filled level in the ground state, which depends on temperature. The Fermi surface is a surface that marks the boundary between filled and empty states. The Fermi Dirac distribution gives the probability of an orbital at a certain energy being occupied and is used to calculate the electrical properties of materials.
The Fermi energy is the boundary between states that are occupied and not occupied at zero temperature in a free electron gas. This is calculated by taking the energy of the orbital, subtracting the Fermi energy, and adding one. The Fermi surface is the level set of energy in momentum space, and is a subset of the space of momenta. It is useful to use the free gas approximation to gain insight into semiconductors, as it provides a probability of an orbital at energy Epsilon being occupied at thermal equilibrium.
Fermi surfaces are level sets of the energy function of a free electron gas. When parameters such as electric or magnetic fields, or temperature gradients are varied, the energy function and Fermi surface can change, either by changing the level set or by changing the shape of the level set. This is important because the linear response of a crystal to a gradient is determined by the topology and geometry of the Fermi surface. Currents occur when an electron in the valence band is able to go up into the conduction band and zip around, with the probabilities of these movements determined by the shape of the Fermi surface.
Density of States is an important factor in SLT, as it can be used to calculate the scaling exponent of the RLCT. This is done by taking the logarithm of V/T, where V is the integral of the prior over all states and configurations, and T is some positive number. The prior is the second derivative of the number of states, and can be differentiated with respect to energy to give the density of States.
Density of states in solid state physics is a measure of the set of states. It is related to the measure on the per volume of the parameter space. As energy is increased, the projection of the area down onto the space of parameters increases, which is analogous to the volume given by the Fisher information metric. This volume is not the volume of the physical system, but rather the volume of the foreign state in SRT, which is the instantaneous increase rate of change of V of T.
Semiconductors have two types of carriers: electrons and holes with an effective mass. The Fermi level is halfway between the conduction band (EC) and valence band (EV), and the energy gap is the highest energy electrons. Assuming Epsilon minus mu is much larger than KBT, the Fermi-Dirac distribution becomes a perfectly dense sphere. This line of thinking has been taken further in earlier seminars, and can be used to derive useful facts about electrical properties of materials.
The probability of an electron in the conduction band being occupied is approximately the exponential of the difference between the energy of the electron and the Boltzmann constant multiplied by temperature. The density of states for a conduction band electron is the energy at the bottom of the conduction band plus a momentum term. The concentration of electrons in the conduction band is calculated by integrating the density of states with the energy from the bottom most energy to infinity.
The density of states is calculated by multiplying the number of electrons in a particular state by the energy it has, and the probability of finding an electron in that state. Integrating this equation yields the number of electrons in the conduction band, which depends on the effective mass of the electron, the Fermi level, the gap between the Fermi level and the electron band, and the temperature. A similar calculation can be done for the number of charge carriers in the valence band, with the Fermi level being one minus the Fermi level from above.
Density of states for a semiconductor is determined by taking the product of electrons in the conduction band and holes in the valence band. This product is independent of impurity concentration and can be used to modify the properties of a semiconductor. The formula for this is e g (energy gap) minus EV, where e is the energy at the boundary of the valence band and V is the given energy. This can be used to engineer a semiconductor, as the interactions between electrons and photons is determined by these two numbers.
Density of states and Fermi surfaces are important to understand electrical properties of materials in solid state physics. This is analogous to KL Divergence in Singularity Learning Theory, which is used to understand engineering of materials like twisted bilayer graphene. To apply SLT to learning machines, understanding the Ade classification is necessary.
Impurities can either increase or decrease the available amount of charge carriers depending on the type, either N or P. This affects the formula derived, which has a product that stays fixed regardless of the number of impurities. This product is then used to make a physical deduction, though the exact details of this are not discussed in the presentation.
NP is a product of temperature and a material property such as the band gap. It is maintained by interactions with photons, with a rate proportional to the temperature and a function of NP. In equilibrium, NP is equal to the product of temperature and a constant b, which depends on the material property. This formula for NP can be used to deduce further information, such as the concentration of electrons and holes.
Doping in learning machines refers to engineering the valence or bands, analogous to electrons in a conductor or semiconductor, to create something with specific properties. This could mean introducing impurities into a pure feed-forward network to bring the bands closer together. This makes it easier to train, as it is easier to navigate the energy surface. Transformers are particularly successful due to their ease of training, often resulting in a predictable outcome.
Impurities in a semiconductor can be thought of as a choice of true distribution from which data is drawn, biasing level sets closer to the true distribution. This forms an energy level. KL Divergence has two inputs: the true distribution and the model. To control the band structure, one can engineer either of these inputs. Additionally, a stochastic dispersion relation could be considered, introducing additional stochastic terms to the energy.
so today's talk is is going to be about solid state physics it'll be the first of a series of two talks on this topic [Music] so today's part of this series uh won't talk about singular learning theory very much that's meant to be an introduction to the the physics involved here so if you're familiar with that it will maybe not contain a lot of new content they'll be fairly Elementary from a physics point of view but the reason I want to introduce this background physics and then talk about how it relates to SLT is I think it's an excellent example of the underlying principle that would be involved if you were to apply SLT to engineering and understanding deep learning systems so as I'll explain the the whole science and engineering discipline of solid-state physics and more broadly condensed metaphysics is about understanding geometry of the level sets of the energy function that Maps say the momentum of electrons in the case of solid state physics usually uh to some real number and where the level sets of that function are singular you get interesting behavior for example with conduction bands in a semiconductor and it's the relationship between the geometry of those level sets which determines the behavior of the material and that's very similar to the way in which say Watanabe is talking about the behavior of a learning machine is also determined by the change in the geometry of a level sets as you change the value near the zero value near the set of true parameters but also Elsewhere for other level sets and that's the content of the theory of phases in SLT and that's true also in materials so there's a close analogy between the sort of global phase structure of a learning machine and the conduction of the energy bands and the behavior of electrons in a material in solid-state physics so I want to and it's not that that relationship doesn't require sophisticated mathematics it's you can get a long way with it's quite Elementary observations in solid-state physics and that's what I'm going to explain okay um so some Basics solid state physics which I'll just abbreviate as SSP so SSP uh I'll release some notes for this talk I didn't get time to upload them yet studies phases of matter with many strongly interacting components EG crystals or more precisely crystalline solids it's strongly interacting because the vertices in the crystal say a carbon atom they interact strongly with the neighboring carbon atoms and that's why we call it a crystal so uh the reason that it's a separate
discipline is that this class of materials or rather this class of phases of materials um has a you have a very powerful set of tools from going from the geometry or the symmetry of the lattice to the electrical magnetic Optical and mechanical properties of the material so um so the source textbook for I think the standard textbook for solid state physics is Kittle this is what I learned it from as an undergraduate many years ago uh I think maybe that was like the fourth edition and now there's the eighth edition um so what you'll learn if you open up Kittle is how to go from a description of the crystal lattice via density of states two properties of the material say um whether it's an insulator or conductor or semiconductor and if it's a semiconductor what kind and so on so let's see so periodicity in crystal structure used to predict electrical and that's what we'll focus on today magnetic Optical and mechanical properties we're focusing on electrical properties and that's because the particular function whose level sets the geometry I'll be concerned with is the function that takes the energy it's talking about the energy of states of electrons interacting with the the lattice okay understanding this relationship is the foundation of the disciplines of engineering that are concerned with these properties of the material and I think in the context of learning machines it's interesting to note that maybe there are roughly two historical periods that you could divide this engineering discipline into one is a kind of exploitation period where we just took the uh the materials that we could find around us in nature for example silicon and then we tried to understand their crystal structure and what that meant about the electrical properties and then we built on top of those properties to design for example um circuits uh but we're moving into a new well it's in addition to that a very strong component of Designing materials in modern physics and that's uh so my first example today will be semiconductors and the second example will be Twisted by layer graphene which is a sort of fancy modern material of much current interest and that's more in the direction of if you've heard of topological phases and this kind of whole discipline of engineering exotic materials on the basis of Designing microscopic properties okay so from exploitation to to design okay so let me give my first example so there's a classification of materials into insulators semiconductors and conductors
which means High Resistance medium resistance low resistance that's kind of coarse way of describing it a more fine-grained way of saying what the difference is between these is to draw the structure of the the bands the electron bands which I'll do in a moment but I'll just note in connection with the previous point about engineering that the the reason you've heard of semiconductors in connection with electronics is that the resistance of semiconductors can be designed and the way that works is by introducing impurities which just means some some points in the lattice are occupied by atoms that are different to most of the crystal so by introducing impurities in a process called doping just mentioned in passing that um the the doped atoms that you use to create semiconductors they could be on the order of one in a million or even one in 10 million or 100 million and still substantially change the electrical properties of the material electrical properties here means the resistance so from an atomic scale somehow very few of the atoms are the doped material but nonetheless it changes the global behavior of the material and that's actually tightly related to a point that we're often trying to make in connection with SLT right which is that singularities affect things far away globally even though they're very tiny somehow fragile local things and the reason is the same so you can change the topology of the level set drastically by small changes that are somehow and sometimes far away [Music] and so this is a sort of a relevant example of how small changes that seem innocuous at the very microscopic scale uh can add up and have Global changes and that's a common relationship between geometry and topology okay so here's a diagram of the what the band structure looks like for these three classes of materials so this is energy so here I'll draw insulator inductor conductor okay so uh well the material is made up of many uh many atoms in the Crystal and many electrons and as you know the the energy states the energy of a single electron is quantized but when you take a a system which is a composite system of many billions or trillions of electrons those quantized energy levels kind of add together and create a Continuum of energy states but maybe not a Continuum that has no gaps and so the the set of possible energy levels of that very large composite system uh generically has the structure of bands I'll draw it like this so the colored in part you're meant to think of as being
if you divided this up you would see each line that looks like that is a possible energy level for an electron in this material now two electrons can't occupy the same exact state which is sort of important to the next point so there's they're filled in a material so this is these are meant to represent filled bands that is those are possible energy levels in which uh there is actually an electron in the material at this moment in time that occupies that energy level in an insulator um the oh maybe I should color this in more solidly okay so this solid color is going to mean filled and then up here maybe I'll make it higher so this is some other energy band which from a theoretical point of view is just as good as these other ones but in the particular material as it is right now this is unfilled there are no electrons in this energy band and this is called the band Gap and in an insulator it's maybe on the order of five electron volts uh and these two bands the the top one of the filled bands uh and the bottom one of the unfilled bands they have special names this is the valence band and this is the conduction band and the idea the reason it's an insulator is that this energy gap this band Gap is too large for say an incident Photon to give an electron enough energy to jump across that gap of course there are many photons hitting the material and they will transfer energy to many different electrons but for an individual electron to jump from one band to the other it has to get all of it and it's not going to get five electron volts from one of those interactions so in practice it can never enter that band and that means it can't move through the material hence it's an insulator in a semiconductor you have the same picture down here but that band Gap is smaller on the order of one electron volt okay so it is possible for electrons to to make that jump and in particular if they're given a bit of help to make that jump by applying a voltage for instance uh then they will then they will make that jump and that's why semiconductors precisely PN Junctions why those are the foundation of our computers that ability to control the resistivity of the material by applying a voltage and a conductor is just where there is no Gap okay so it's easy for an electron to to just jump up to this conduction band zip to the other end of the material um the hence it's a conductor okay so the point of this is to explain how the structure of the Bands particular the band Gap is what
determines the electrical properties of this class of materials so now what I want to do is explain how how the structure of those bands is related to well I won't say much about the periodicity and crystal structure but how it's related to the density of states and the the function that assigns to a given momentum possible momentum for an electron it's it's energy but I'll pause here for questions are there any about this picture so far oh God you have an electron zipping from right to left but no x-axis really yeah that's a good point that's a nonsense thing to draw yeah uh the x-axis in this picture this is a typical kind of picture but it's it's actually nonsense I don't even know what that's supposed to mean what is this axis uh yeah there is no x-axis that's right I guess probably when people draw this picture they usually have in mind the spatial axis of the material I suppose but that is kind of a bit doesn't really make sense so yeah I agree it's like a rastering along some face space filling it up as you go with electrons or something yeah exactly anyway yeah I wonder um I guess strictly speaking there is there can be no x-axis that's the whole point of the Exclusion Principle so it definitely does not exist this x-axis and but maybe the picture is yeah I don't know I agree even the access is the distribution I think the density um again um accesses would be number of electrons in that with that energy State and another draw in the compound yeah um I think you could you could argue that you could use the x-axis to represent like the expectation of some observable like the the x coordinate of an electron so you could uh you know you could maybe integrate this plot with some kind of expectation like that but the there definitely is no more I mean there is no such thing as the number of electrons in that state or I mean you can talk about the expected occupation value but uh it's it's really and it's either zero or one so I think yeah if you want to put an x-axis there you'd have to think about it a bit harder so I'll Escape that I think so the XX is not it's not a real thing okay so my next example is um Twisted by layer graphene let me see if I can get up a picture one moment so this is very cool one be with me so I imagine some of you have seen the Press around this before yeah here we go so what you can see on the left here uh two uh one atom thick lattices so crystals made of carbon atoms so that's graphene so like the blue layer there it's a hexagonal lattice of carbon that's what
graphene is and there's two of them that are layered on top of each other and they're rotated relative to each other by some angle Theta which is as it turns out about 1.1 degree that's the magic angle so in 2018 well it's been predicted for a long time that graphene has sort of special electrical properties and there was even some theoretical reason to think that it's superconductivity might arise in by layer graphene but in 2018 it was an experimentalist group if I get this correct that discovered that if you if you layer graphene at exactly this angle you get a superconductor this this two layer material at about 1.7 Kelvin so graphene on its own normally is an insulator but if you arrange it in this particular and if you arrange it at angle 0.5 degrees it's an insulator and 0.9 degrees it's an insulator but at 1.1 degree there's a phase transition of some kind and it becomes a superconductor and low temperature superconductors is kind of the ultimate dream of condensed metaphysics so the amount of excitement around Twisted by layer graphene is absolutely off the charts um so what I want to do is write on this right hand board here just a few of the terms that you'll find if you go reading about Twisted by layer graphene the original paper I'll put a citation in the um in the notes but you'll be able to find it easily enough but there's so some statements to do with tbg Twisted by layer graphene and then I'll sort of try and explain some of the terms that appear in in those statements so the first is from this 2018 paper uh so it says at 1.1 degree the electron band structure of tpg exhibits flat bands the SLT like alarm Bells Are Ringing right flat flat uh flat bands near zero Fermi energy I'm going to underline a bunch of terms in green and those are the ones I'm going to explain in a moment so just kind of absorb the lingo by osmosis for a minute Quantum oscillations in the longitudinal resistance that's the left to right motion that I retreated from hastily on the previous board so that's the resistance along the material so the the ratio of the current in that direction uh and the and the voltage okay presence of small Fermi surfaces near the correlated insulating state okay that's one statement uh now again electron electron band structure you should read structure of the level sets of the energy function which Russ is the KL Divergence right so that's a kind of preview of where this connection is supposed to come from not from this original tbg paper but
from another one that's more recent and maybe moment iCal as the following sequence of statements so the I haven't looked at the literature in the last year or so so maybe the state of things has changed but I think the the exact reason or origin of the details of the superconducting state in tbg is theoretically still maybe not completely explained or maybe they're competitive explanations but one of the papers that's exploring this in more detail is is making the following comments sorry I just noticed the chat uh seems like the questions were answered so one such feature is the I'm going to mispronounce this please correct me van Hove singularity [Music] VHS uh this was an old idea from solid state physics you can you'll find it mentioned briefly a few times in Kittle in Mario bands so this is I won't explain this but this is part of the story with real one zones and so on uh generally speaking VHS with Divergent density of states the OS this we're familiar with this is exactly the same sense in which we mean density of states in singular learning theory 2D systems are associated with saddle points of energy dispersion it's another term I'll explain in k-space uh yeah so when uh van Hove Singularity is close to Fermi energy The increased dos amplifies electron correlation and that's one potential theoretical explanation for the superconductivity all right so I'm happy to take questions now but the the aim of the next uh the rest of the talk really is to explain these green underlined terms and get you to a point where you see past the difference in this jargon and uh um or rather translate this jargon to our jargon not to use the word jargon as a pejorative uh to translate this into terms you're familiar with and recognize that what this is saying the second bullet point is simply in our language that the the singularities of the set of true parameters um the worse they are the that's what controls the increased dos the worse they are the more long-range correlations that you'll see in the system and in this particular case that correlates to that determines the superconductivity for learning machines I mean it's not that the not that I'm saying is something like superconductivity in a learning machine and not claiming that but um the high level of observable property which in this case is electrical has a substantial change due to some fine tuning of a parameter the angle so uh is superconductivity and learning machines super intelligence yeah you're the you're the memes man
here Alexander yeah I don't know I don't know that that probably makes sense what's resistivity uh there's no real analog of the lattice right so uh actually that's relevant to the the argument about the diagram earlier so the left right uh axis in that diagram doesn't exist but it's kind of implicit because the continuity of the band structure is due to the spatial dimension of the material right if you just had a single atom sitting there in your Crystal you'd have a discrete set of energy states uh The Wider I mean of course the material isn't infinite so strictly speaking it isn't a Continuum of energy states The Wider you make your material in this case I mean the two layer the the graphene uh you know just the the more area it covers the more the energy Spectrum looks continuous um but the the role of the spatial dimension in the material is sort of invisible uh unless you think about it in terms of just that discrete versus continuous so I don't know what the analog of the spatial dimension for a learning machine is and therefore it should not resistivity means uh it was guess's joke really but uh do you have an idea what long-range correlations could mean in The Learning machine point of view or also not yeah I mean I guess there's there is a spatial analog if you think about it from a sort of RG flow point of view between potentially different layers right so like the final layer and the early layers of a neural network I mean that would be potentially an uh you know that could be potentially an analog of distance I mean there's a yeah this is getting a bit off track but uh there is a sense in which there's an emergent renormalization direction that you can talk of in say icing models um so it probably there is an emergent sort of spatial direction in which you can talk about length correlations uh when you treat something like a large learning machine at least one that comes from a neural network um but I don't yeah that's just speculation so I don't know would this be related to context lengths or or science that's completely different yeah I mean that would be it'd be kind of natural to use that I mean that is kind of one lattice that appears in a Transformer of course but somehow the interactions are not me they don't change depending on the distance in your sequence of tokens right so the tokens are only a sequence somehow to you human feeding things in the Transformer just sees a sum over all the tokens and whether it comes I mean there's the
there's the position embedding I guess which creates a kind of spatial Dimension so maybe yeah I don't know how to think about the role of the position embedding maybe that's I mean aren't setting randomly right I mean every token has some text around it yeah but it's like it's the nearest neighbor interactions that kind of determine what a long-distance correlation is in a material like graphene right and so do do nearby tokens I mean it's not like the in the effective energy that's attributed to that by a Transformer really strongly correlates nearby tokens I mean it only does so through the positions I mean some techs have long range correlations and some don't right there's like yeah but that's more function of the data distribution than the model so I agree with that but I would agree with it I don't know how to map that onto this situation precisely okay um okay uh so I want to repeat something uh one section from one of the earlier seminars on density of states and just because it fits here so for those of you who've heard that I'm sorry but the next 10 minutes will um be a just a recollection of the density of states and then after I've explained that I'll explain the other terms dispersion relation Fermi energy I'll talk about the Fermi Dirac distribution and then I'll derive something about semiconductors using all of that as a kind of demonstration of how all of this actually gets used in solid-state physics to to say a real thing about a real material all right so for a system with indiscrete configurations so here you could think about the states of electrons in our material if it was finite in his finite here or particle in a box um and volume v of course it's a bit arbitrary how you define the volume but anyway I don't want to talk about that so the density of states is well the number of states per unit volume at energy e and in a discrete system it's just this is just a Delta function [Music] right so this is just one if e is equal to e i zero otherwise and so it's just counting so where the possible energy states sorry um where e i is the energy of the art state so for a continuous system so I'm going to be pretty physicsy about this uh that old lecture I think it was seminar five or seven um there's notes attached to that and in that I'm going into like gilfan kirilov and generalized functions and telling you exactly how to do this properly uh and uh and so on um so I'll defer the the real math to there and just do the Kittle level story
yeah but the real version is what you'll find discussed in the um you actually need that to read through various sections of Watanabe right so that's that's sort of important for us but not important for this story okay so for continuous system the density of States at e is per unit volume the number of states accessible that you gain by increasing the energy by Delta e so n e is the number of states so the density of states is not the number of states it's not the number of states per unit volume either it's the number of new States that open up so that's the number of states of energy less than or equal to E okay so I.E the number of new States accessible per unit volume when the energy increases from E to e plus Delta e uh okay so in a continuous system of course we we actually deal with some kind of measure and we're integrating uh Etc all right so let me give you an example and this is the example I want to use it's enough to actually understand the simplest model of a semiconductor [Music] which is a free electron gas [Music] and for a free electron gas in D dimensions D will be three for us for a semiconductor well what's a free electron gas so the only numbers I need to describe the energy are just it's just the momentum so the energy the states are parametrized by the wave vectors that's just a vector in R to the D which gives you the frequency in um every dimension and the energy of that state is P Squared on 2m or H bar so m is the mass of the electron um H bar is what it is and that's the dot product K squared all right so the total number of states of energy less than or equal to E is just the volume of a d sphere with that radius e to the half so the total number of states of energy less than or equal to E proportional to e to the D on 2. and if you look at the formula for d e well it's the derivative of n with respect to e so just factoring out various pies and so on that's just proportional to e to the D on 2 minus 1. okay so that's the density of states for a free electron guess [Music] okay um what I want to say about that well it's an increasing function of e the the higher the energy the more possible electron States you can have but there are no special energy levels here right there's all energy levels basically are treated the same except for zero so zero is the only Singularity critical point of this um density of States but every other energy level right it's just a there's just some expanding sphere and um the energy bands have nothing special to say about them
all right any questions about this so how would you get a more complicated interesting density of States well you would have to have a more interesting energy function and that might come from uh having the electrons interact with something for example a crystal lattice and then the configuration of the atoms in the crystal would change the energy perhaps in some way reflecting the symmetry of the crystal for instance and then the energy function would have a more interesting density of states and some more interesting band structure hence more interesting electrical properties so that's uh how that works yeah any questions we're going to I'm going to return to the rest of that dictionary now um and then do semiconductors and indeed that's what a van Hove Singularity is so a van hoe of Singularity and maybe just the so this this formula here would be proportional to just K1 squared plus K2 squared plus dot dot right so you'll get a more interesting density of states and this actually happens in real materials if the density of states were for example a quadratic form of a different signature right so maybe it's got some pluses it's got some minuses that will give you saddles or you know more precisely uh singularities of different index and they will have a different density of states a more interesting density or you can have a you can have terms that are fourth powers there's also appear naturally all right so let me come back now that I've recalled that material back to the rest of that dictionary those underlying terms from those two papers so glossary the dispersion relation is just this function e is a function of K it tells you the energy as a function of the momentum the density of States well I've explained what that is uh the Fermi energy so you might want to pop out of orbcam and take a look at that board if you see how these terms fit back into that sentence the Fermi energy is the energy of the topmost filled level in the ground state this depends on the temperature so maybe it's not like the ground state you might mean by that the zero temperature ground state I'll just skate over that you'll see how it comes into the calculation in a moment I guess and then the other term was the Fermi surface I'm going to come back to that in a second [Music] but I want to introduce now the Fermi Dirac distribution um there are citations in my notes maybe I'll just add one here this is into Kettle eighth edition and so it gives the probability of an orbital at some energy being occupied in
an ideal electron gas now of course you know in a real material it's not a free electron gas because it's interacting with phonons which are the perturbations away from the ground state of the sort of the lattice itself so there's interactions between those phonons and in the electrons and of course there's interactions with electromagnetic field and so on so it's not a free gas but as usual in physics it's useful to use the free gas as an approximation to vary away from and as you'll see with this calculation I'll do in a minute you can say something some kind of interesting fact about semiconductors even with this very uh kind of naive I guess assumption that it's a free gas so this is useful so it gives the probability of an orbital at energy Epsilon being occupied in a free electron gas at thermal equilibrium it's into the expression I guess here's the formula so Epsilon the energy minus mu I'll see what that is in a second and then it's plus one I'm not going to justify where that comes from that's not the talk but I will note so that's the temperature t KB is the boltzmann constant um let us figure out what mu should be it's one of the terms that's on the board but we can discover which one by taking the limit so in the limit as T goes to zero this F Epsilon or maybe maybe T is infinite decimal given the previous talk so F Epsilon becomes one if Epsilon is less than mu and zero if Epsilon is greater than mu while at zero temperature uh the Fermi energy is exactly the boundary between those states which are occupied and those which are not so the MU is the Fermi energy at zero temperature right so the states below mu are filled and the states above mu are not filled there's the trademarked cow foreign surface was right so I just said that at a temperature zero some set of states are occupied and some set of states are not uh where does that set of States live well the state is a configuration of a particular electron and the possible configurations in our case in the free electron gas are just given by the momenta so the set of occupied States is a subset of the space of momenta so in momentum space foreign so that's the Memento such that the energy is less than or equal to Mu fill up a ball the boundary or Surface of which is the Fermi surface okay I.E Fermi surface means level set of the energy and the energy is a map from the space of configurations in this case the momenta to ah right so in the SLT analogy this would be k and this would be w right so the Fermi surface for us is a
level set of the loss or the KL Divergence all right and so we have some idea that at a given temperature uh the as I said the Fermi energy depends on the temperature um but at temperature zero uh you have some set of occupied States and that surface is is telling you what it is all right and the the real Point here uh so think about this surface and what happens uh if you change one of the parameters okay so where are the parameters here well the parameters are in the energy so as I was saying earlier right so you've got this free electron gas uh has some simple formula and the Fermi surfaces there are just spheres nothing interesting is happening but if you turn on some interactions for example you turn on an electric field or a magnetic field or you have a temperature gradient that's going to change the energy right so you've got some space of energy functions you're varying the energy so for us you know that might be a very true distribution or some other part of the problem varying continuously as you vary those parameters the energy function the dispersion relation changes um and that will change the well if you just change the energy without changing the form of the energy function you're just changing the level set of the same function or you can change the shape of the all the levels set simultaneously by varying some of those parameters in the energy and both of those are interesting so from a point of view of a material the linear response of a crystal to a gradient as I said [Music] um but it's about the infinitesimal variation in the Fermi surface that's what determines the properties that we're interested in the things that we macroscopically think uh interesting about a material for example the resistivity a quote from Kittle I think so the linear response of a crystal to an electric magnetic or thermal gradient is determined by the shape of the Fermi surface I think to this audience I wouldn't quite say shape I'd say topology and geometry it's both matter and in slightly different ways right [Music] right but you'll see both discussed in in these references because currents so what's a current a current is where an electron in the valence band this topmost occupied band of the filled bands is able to go up into the conduction band and zip around right that's a current and of course it's stochastic so it goes up comes back somewhere else and it's not a single electron going from one end of the material to the other but the probabilities of those particular
transitions is what controls whether there is a current and if there is a current how much resistance to it there is the current currents are due to changes in the occupancy of States near the Fermi energy that's what I just said okay um yeah I mean the there isn't a direct analog of electrons for a learning machine maybe there's something like Goldstone bosons I don't know they're like implicit objects you could maybe talk about but it's not like there are explicit particles in the theory okay um maybe let me just recall briefly how the Dos features in SLT I'll maybe say more about that next time am I going for the time yeah so yeah let's move on to the second set of boards and I'll recall how the rlct arises from the density of States just so you have some idea of how this connection is supposed to work so follow me all right um so sort of just aside density of states in SLT so one of the formulas for the rlct is that it is the scaling exponent of the density of states right so if you take the density of states which Watanabe writes is V so it KL Divergence integrate over all this States the configurations the parameters and integrate the prior that gives you the density of states and then if you take so I'll think about in the example of the free electron gas it was some some power of T right and if I want to isolate that power well what I do is a t to the S uh divided by T to the S will give me a to the S and then I take the logarithm of a to the S that's s log a and then I divide through by log a that'll give me S right so that's all I'm going to do here I'm going to take the logarithm of v a t minus VT divided by log a a I don't care it's just some positive thing that's not one uh that isolates the exponent of the Dos so it's like T to the S kind of and that's the rlct so in the in the regular case it literally is a scaling exponent I mean that that that example with the free electron gas is the regular case for us right so that's the rlt's rlct is the scaling exponent of the density of states more on that some other time but that's uh where this plugs in okay I'm happy to take some questions and after that I'm going to talk about um semiconductors so the prior is the second derivative of the number of states the prior is the second derivative of the number of states I'm not sure what you mean I mean if I differentiate the number of states with respect to energy I get the density of States oh I see uh and then you mean you differentiate that again and that kind of the coefficient
there is yeah I'm just trying I don't quite sort of see the the link I'm trying to interpret the prior in solid state physics terms oh I see uh that would just be some kind of measure on the set of states right um right so it would yeah let's see uh yeah actually that that gets it slightly complicated directions that's um because of the because of the fact that two say electrons can't occupy the same state um you like there are more constraints on the prior effective way that you're going to be able to use in the physics setting of density of states [Music] they cannot enjoy the same two states for the same energy yeah that's right yeah that's right so okay so this this density of States um it so in the SLT pictures is related with the um measure on the per volume of the um like parameter space yeah that's right instead of if you were to draw a level set yeah um so it's sort of as you increase the so I'm going a little bit higher in the energy landscape and that lets me climb a little bit further up the walls of the valley here and the infinitesimal amount of territory like if you project onto the the bottom here right so it's not so if you've got some sort of Valley that's sitting above and You by that was the cutoff that was consistent with the previous energy and then you allow higher energy and you go up but that exposes like that increases the projection of that area down here and it's the change in the area of the projection down onto the space of parameters which is kind of what I'm trying to draw in this left hand thing that so so is the volume like the spatial Direction like volume in the physical system is this directly analogous to the volume in um like parameter space like if it's a regular manifold right sorry regular learning machine then there's this notion of like volume given by the Fisher information metric is this analogous this is like directly analogous uh no so the volume given by the Fisher information would this is just the volume according to the prior right um so those would be different especially near the set of true parameters and be careful when you say spatial Direction uh that's like in the that's in momentum space right so the the energy the the kind of space of parameters here in the physical setting is the space of momenta so it's not it's not volume of the physical system it's like the volume of the foreign state in SRT is actually the instantaneous increase uh rate of change of V of T so it's a derivative respectivity of that
quantity um so there's a derivative missing in the definition on the ball um if I remember correctly um well you can formulate it differently and talk about it as a derivative but I believe the formula for the rlct is correct so Jesse just told me just want to check that is As you move closer to the singularity the density of states uh blows up and the speed with which it encrypt like exponentially blows up that's the rscp correct or the you know sort of the scaling exponent is that what you said yeah that's right cool very cool yeah so that that line of thinking has gone into in much more detail in that earlier seminar um okay so I want to explain semiconductors um of course the hidden message being that uh what um I'm about to do a lot of ugly truncations and approximations and derive a useful fact about electrical properties of materials so then I will not be afraid to do similar ugly things to SLT something okay so uh let's talk about semiconductors so intrinsic carrier concentration so in a semiconductor there's two types of carriers two types of um particles you shouldn't get too attached to like particles as things with like real Mass you know there are holes that have effective masses and quasi particles with effective masses and so on depends on the mathematical description so the carriers here in one case are electrons that's a kind of real thing and then the other case are holes um in the Fermi c those have an effective mass and they also carry a positive charge okay um so let's see where do I start okay so we assume uh that Epsilon minus mu same meaning as before is much larger than KBT so Epsilon here is the energy mu is the as the Fermi level uh you can think about this as halfway between the the sort of in the middle of the band Gap and maybe I should draw a diagram uh so here so remember we're talking about a semiconductor is the energy this is the conduction band that reaches down to energy EC there's the valence band that reaches up to energy v e sorry EV and then the Fermi level is is in here somewhere okay and the energy gap is just the difference the energy that's like the highest energy electrons so yeah this is different from the vehicles but so like think about the free energy guys right here so you're in momentum space okay so the femi to rock distribution in this case uh well maybe I'll write it out so this was one divided by the exponential of Epsilon minus mu on KBT they're like levels it becomes a perfectly dense sphere I think we can we can hear you
guys discussing which is which is fine maybe you want to mute thanks so under this hypothesis that Epsilon minus mu is much bigger than KBT I can just ignore this one is the point all right and so effectively I just get uh this is approximately the exponential of mu minus Epsilon on KBT all right so what's that saying it's a I'm Contin I'm considering an energy say up here or such that this Gap is large compared to KBT and that's not always true but it's like okay this is our first shot at a theory describing semiconductors so this will do okay so that's the probability that a conduction electron orbital is occupied that's an energy that's above EC uh the energy of such an electron is um well Epsilon I guess I was writing E before um yeah forgive me I'm just going to stick with Kittle's notation so uh this is energy there's a lot of Ease on the board so maybe that's not such a bad thing and K is the wave number as before so it's going to be the the energy at the bottom of the conduction band plus some term that comes from the momentum uh you could think about it this way that uh once you get up into the conduction band how much extra energy do you have to actually propagate around to maybe use up all your energy just getting up there that's one way of thinking about why this is a configuration that's um I got plus this term Emmy is the is the mess okay so now we can use the earlier formula for the density of the state so I'm we're just going to treat an electron in the conduction band is a free uh as a free electron that's sort of embedded in that formula for the energy right so the density of states right so you could think about that as how many new States for a conduction band electron open up as I uh increase the energy by some infinitesimal amount so that's d of electron adds some energy I left off these pre-factor terms earlier but if you think about it there the following so this is not that interesting that's just the constant proportionality from earlier and then you have the energy and before it was D on 2 minus one right there was no term here free electron gas I just had e to the D on 2 minus one this is now e and D on2 is three on two D is three so that's just the half so that's just the formula from before okay so now I'm going to see what the concentration of electrons in the conduction band is by integrating F against d and so I'm going to call that in so that's the integral from the bottom most energy you can have to Infinity the density of States times this energy
uh right if you think about it um the density of States was basically n e plus Delta e minus number of states divided by Delta e so that means If I multiply the Delta e over there I have something that looks like this and so what am I doing when I do this integral well I'm just saying how many states are in this particular band times the energy that they have sorry um times the number the number of electrons in that state right times the probability of finding electron in that state so I'm going to go through every one of these um energy levels just multiply the number of electrons uh times the energy that they have sorry I'm going to times the number of electrons times the probability and that'll count how many electrons there are in the conduction band okay and that's this number here n so let's do the integral uh so what's that I get all this junk um and then F had some junk no it's not [Music] uh mu minus Epsilon [Music] the Epsilon okay well that's just that's basically the integral of root x e to the minus X right once we make a change of coordinates to make this x this is going to change to some other thing we can do this by integration by parts we'll get one part it has an earth in it and then we evaluated Infinity that'll go away so I'll just tell you what the answer is [Music] answer is um two m e k b t 2 pi H bar squared to the three and two and then U minus Epsilon C sorry e c on KBT [Music] okay that's the number of electrons in the conduction band what does it depend on well it depends on the effective mass of the electron uh depends on the Fermi level it depends on this gap between the Fermi level and the electron band kind of it around the temperature of course now we can do a similar calculation for the number of charge carriers in the valence band so it's basically the same I'll just sketch how that goes okay the equilibrium concentration of holes the holes have an effective mass that you could derive I won't do it so there's like an m H for whole what is the I mean I need to know what f is the density of states is going to be the same but the the f is just 1 minus the old F right the F for a whole is one minus f from above I think I just wrote I think I just wrote F didn't I yeah just one minus f okay and under the approximation that I'm taking uh so that's one one minus this thing and if you just put them in the same denominator divide through by that exponential and use the same approximation as I did earlier you'll get that this is just the
the density of states for a whole is the same kind of formula it has the effective mass for a whole H bar squared and then it's the the difference between that energy at the boundary of the valence band and the given energy to a half so now I guess in the first calculation my Epsilon was something that lived kind of in this top group right in their conduction band so this was up here for this stuff and for this last board I'm doing uh the Epsilon is now down here so I'm talking about this Gap okay uh so we just do the same thing right the occupation um so the density of whole sedates it's just the same integral p now minus infinity to e v d h e f h e blah blah blah okay and a similar formula we don't really care about this formula independently the upshot is what happens when you take the product of these two and this actually informs how you engineer a semiconductor and that's kind of the point of this little exercise oh thank you yeah thank you all right so basically the same kind of formula so uh when I take n times p the MU disappears right so this is just some junk maybe the fact that it's t cubed you care about I forget right now I don't care so I just get EC minus EV here right and that's e g the energy gap and this was the formula I wanted to get to so the concentration of electrons in the conduction band times the concentration of the holes in the valence band is given by this formula and well the the interactions that cause electrons to jump from the valence band of the conduction band when they interact say with photons is governed by these two numbers so let me just finish this part by uh giving you a little Taste of what's in this section of Kittle okay [Music] um yeah I guess it's a bit out of scope to justify the following statement um okay so there's a physical input here which is that if you look at this quantity the product NP is independent of impurity concentration so remember I said that you create a semiconductor well one way to create them or to modify their properties is to dope them to introduce some other atom into the crystal um and you can show that the product NP under these approximations is independent of the impurity concentration at a given temperature and that's not a hard argument but it's sort of would require a bit more physics to explain okay so that's a physical input NP is independent of the impurity which you control right the introduction of a small amount of impurity right which increases n must decrease p because that product is constant
Kittle says here is this result is important in practice we can reduce the total carrier concentration which is n plus p and that's n plus some constant divided by P sorry divided by n because the product is constant at least in the context of the changes we're considering uh so you can increase that by introducing impurities by raising n right [Music] and this is called compensation I don't know why but anyway this is a basic technique in the design of semiconductors right you want the total carrier concentration to be high and this tells you how to do it as a function of these this basic Assumption of the electron gas being in both of the valence band and the conduction band of free electron gas and this little calculation with the density of States all right so this is something to do with the relationship between bands of this energy function in this material all right and that's that's where I wanted to get to so to summarize um there's an analogy between the energy function of electrons in a material in solid state physics and the KL Divergence in singular learning theory we know in solid state physics that many of the electrical properties of a material can be determined by the structure the geometry and the topology of the level sets of that energy function and that's actually the foundation of the subject right you can't do everything just by understanding density of states right it's it's more complicated than that but uh if you read Kittle's book the density of States and how to reason about it the Fermi surfaces how their topology changes pictures for the Fermi surface how they release to the crystal structure that is like three quarters of the course that I remember as an undergraduate that's the bread and butter of solid-state physics so this really is the foundation of an engineering discipline and you don't need so much Singularity theory for regular materials but for things like the magic angle um potentially and I don't want to over emphasize this because it's still ongoing research right but there are serious people writing papers explaining the Ade classification to solid-state physicists in order to understand the engineering of materials like Twisted by layer graphene so maybe that can serve as a kind of template for how to go about trying to apply SLT to to learning machines in the future so the next talk next week I'll um I'll go a bit more into some of the aspects I discussed today I haven't decided exactly what um if you have opinions then share them and maybe I'll
do whatever you say okay thanks everyone and I'll um take questions so Dan you just mentioned something about the Ade classification um what's the story there yeah so uh when you start so there's a paper actually it's the one I'm citing that I was quoted the second one I was quoting from uh the appendix of that paper is going through not all of the ad classification uh but so yeah I forget exactly maybe the e-types don't show up or something but um there there is a discussion in that literature of well for the same reason Ade shows up in Singularity Theory right they're the the simplest singularities you get uh that have modulus zero so they show up for the same reason and um in this area yeah interesting cool um I don't really know um the last part about the the just the last board so did the formula did derived doesn't depend on impurity but you're saying any impurity would change I mean I I understand the usual um High School level explanation of doping and introducing um different amount carriers but you're saying that that process has something to do with the um n times P formula that is oh oh it's the last part saying that it has nothing to do with that uh it's saying that uh I'm not sure I understand the question so the um the idea of the impurities is he referring to is that they would depending on the type of impurity whether it's an N or P type uh would either increase the available amount of charge carriers or or decrease it um but uh yeah it um it's it's hard for me to connect that last ball with the second last ball that that um highlighted formula that for the formula yeah so what does impurity have to do that that um that distribution I think it's not normalized but not that yeah I'm sorry this this is a floor in the presentation so there's a missing piece here which goes from this formula and then makes the physical deduction that I'm black boxing so yeah it's not very satisfying I I agree um okay yeah uh let me see if I can um I can't have anything to do with uh changing the foaming surface oh or is that like a completely different mechanism um in this case I'm assuming it doesn't right so uh it's it's kind of yeah I suppose it's considering a fixed level of doping and then varying that a little bit um so it's yeah um let me see maybe I'll just see if I can just see if I can explain this um this step here yeah the point is that this product if you change the number of impurities you'll change in and P but the product will stay fixed and that's that's
deduced from thinking about the uh the actual interaction between the the photons and the electrons that are involved here I can try and sketch the arguments I um I'll see if this makes sense to you so here's the here's the missing step here so why why NP is constant so the um the population of electrons and holes is maintained by interactions with photons but some temperature t so the photons generate electron hole pairs at a rate a t and this I had I could not explain right now and so some function of T times n p is the rate of the recomb oh okay no that does make sense um is the rate of the recombination reaction my mission so electron plus a whole goes to a photon let's think about the product as being the probability that uh some electron and somehow meet I guess then there's some term proportional to the temperature so what that means is that DN DT which is a T minus btnp must be equal to t let's see does that make sense uh electron hole pairs yeah that's right okay so in equilibrium T and DP T is zero and that gives you that NP is a t on BT which means at on BT is equal to um the green box equation that which depends only on temperature and the material property like the band Gap yeah so yeah that's okay now that I think about it uh that doesn't seem to actually need this formula in the box right so I suppose you could read this combined as saying here's an argument based on the equilibrium Assumption of the number of holes at photons that suggests that the product NP only depends on the temperature and that is consistent with this formula that's in the box and that more of a computes it explicitly as a function of t uh so you could read these as two separate derivations of the fact that the product NP only depends on the temperature I suppose yeah um sorry for holding up the session I was just um wondering where impurity factors into the whole derivation of the no no you're right this was a bit under thought so maybe the uh in combination with what I just said uh the broader point is that from this formula for NP you can then deduce other things as well as this observation which yeah maybe now that I think about it you could justify uh partly by by other arguments but this particular formula for the product NP and then later on in the section Kittle goes on to use it to do other things so maybe that's actually the stronger argument for the assertion that this calculation in terms of density Estates and concentration and so on is um is telling you something
physical the particular example I chose maybe you yeah maybe this wasn't the right way to motivate it yeah thanks for explaining what other questions what does doping look like in learning machines I suppose it probably means architecture so uh what are we doing when we design an architecture well we're doing something like attempting what we don't know I guess is the first answer but you could conjecture that what we're trying to do is engineer the valence or engineer the bands or the equivalent of the Bands in order to create something that maybe has some properties like a conductor or a semiconductor right I mean when we talk about learning being easy in Transformers maybe we mean that the electron bands don't have large gaps right or the analog of the electron bands don't have large caps and so maybe when we say something like it's easy to train x x being say Transformers actually what we've done is introduce quote unquote impurities into like just a pure feed forward Network or something in a way that brings those bands closer together I don't know um that would be my yeah it's a good question I think one should think about it much more carefully but that is a kind of rough shape of an answer you say it's easy to learn because turns flow easier and so somehow that's related to The Learning Dynamics involved yeah I don't know about the analog of currents but uh you could so learning uh so if you you're descending an energy surface when you learn right say by gradient descent and if there are large energy gaps well what does that mean um that means that the so the the bands the analog of bands will be something like uh collections of local Minima of the energy surface uh in a given energy range um and so if there are large gaps it'll mean it's quite difficult to move from some level set of the energy or band of level sets to some other band so you don't have to posit an electron or some sort of particle to just think about like say the boltzmann distribution and moving between configurations right in this case the relevant boltzmann distribution is over energy states of the electrons or over you know over the Memento themselves perhaps so just being able to easily move between different bands of the energy that have high occupancy would actually have something to do with ease of navigating the energy surface right and one of the reasons that Transformers are successful is that they're easy to train right meaning that you very often just get a predictable
decrease in loss and you don't get stuck somewhere so I I guess you could maybe think about that in terms of structure of the Bands I've seen formulation of the correspondence between thermodynamics and learning patient learning that posit that the impurities are the particular set of data that is drawn um yeah in this analogy then type of doping becomes choice of true distribution from which you draw the data um and I think the idea is that if you um draw a different uh data set um if you choose a different true distribution then you are sort of biasing uh level sets that are closer to that true distribution and does form sort of energy level and if we want to discretize things like that and say it's sort of an order set of um critical sets um or the roughly by the distance from the true parameter yeah I guess it's probably a little over determined in the sense that uh like what does one mean by the analog of impurities I mean in this from one point of view it's the thing you can control in your lab in order to engineer a given electrical property which given that you have these two inputs into the KL Divergence one is the true distribution and one is the model uh you could think about engineering either of them in order to control the uh band structure and thus the sort of analog of the electrical properties so yeah maybe you could try and change the true distribution or you could try and change the architecture I mean I guess I see the the analogy to impurities being to do with the particular like um sample you draw but I don't think that's it's a little bit not quite in the spirit of the reason why you what you do when you introduce an impurity into a semiconductor I think hmm because that would seem to suggest that if you draw a larger sample you just get rid of the impurities or something no I um during a larger sample would kind of just make the uh energy so State distribution concentrate on it like sort of like becomes the true distribution the the engineered State instead of just a perturbation from uh from the uh the pure material yeah but that feels like it's I mean you could also talk about a stochastic uh dispersion relation right where you have these additional stochastic terms in the energy which would be the analog of the difference between K and K in in our case so maybe that's like an another kind of impurity you could consider but I don't know that yeah anyway uh that would be a good thing to think more about yeah another question I thought Richard would