Language Models for Machine Learning

By

Introduction

Introduction to Large Language Models (Google)


Training

On Pre-Training

Training


Tokenization

Tokenization


Course

Language-Modeling from Scratch (Stanford)

Comments

Post a Comment

Popular posts from this blog

What Counts

By
Image
Overview Counting Number Origin(s) of Number Communicating Number (Penn State) History of Writing A Partial History of Counting Counting? Writing and Numbers History of Numeral Systems Construction of the Natural Numbers (MIT) Construction of Number Systems A Partial History of Numbers Number Theory Peano Axioms Peano Difference (Philosophy) Development Counting Bee Numbers and Language Keith Devlin on the History of Mathematics 1. Development 2. Golden Ratio 3. Algebra 4. ? .... ? 5. Ability Timeline  Evolution Overview Timeline The Big Bang: ~13.8 Billion Years Ago The Earth: ~4.5 Billion Years Ago Human Beings: ~3 Million to ~1.9 Million Years Ago Modern Human Beings: ~200,000 Years Ago Human Language; ~150,000 Years Ago Evidence of Counting: ~46,000 to ~10,000 Years Ago The Anthropocene: ~8,000 Years Ago Evidence of Human Language: ~6,000 Years Ago Evidence of Writing (i.e., not Proto-Writing): ~5,200 Years Ago Questions Is Counting Prior to Number? Is There Number without Lang...
Read more

Math Self-Study

By
Image
Overview Overview Understanding Math (GitHub) Awesome Math (GitHub) Learning How to Read a Math Article Mathematical Discovery OSSU Curriculum (GitHub) Guide to Math and Physics Self-Learning (Futurism) Visual Math (YouTube) , Visual Math (Slides per Topic) Michael Taylor   Starter YouTube Videos Basic Notions YouTube Videos (ICTP) Math Maps A Map of Math (YouTube) A Map of Math (Quanta Magazine) Resources Abakcus American Mathematical Society (AMS) Cambridge Digital Library of Mathematical Functions Harold's Tutoring Institute for Advanced Studies Logic Matters Math Book Reviews Math Conferences (Math Meetings) Math Conferences (Hiro Lee Tanka) Mathematics Textbooks for Self-Study Milne Open Textbooks MIT Mathematics MIT Online OER Commons Open Culture: How to Learn for Free Open Culture Courses Open Culture Textbooks Open Library OpenStax Open Textbook Library Project Euclid Real, Not Complex Stitz Zeager Open Source Mathematics (Pre-Calculus) UCLA University of Texas Yale Readin...
Read more

Computing and the Linguistic Turn

By
Image
Enigma (Wikipedia) Symbolic analysis and statistical analysis were both approaches codebreakers took in breaking Enigma. The development of AI has also incorporated both approaches. One historical question for AI is why the dominance of Symbolic AI from 1950 to 1990? Note the development of AI out of the Dartmouth Workshop and especially the contributions of one of its attendees,  Ray Solomonoff , to the statistical approach, an approach taken up by Marcus Hutter , a DeepMind  researcher. Note too that along with the difference between the symbolic and the statistical is the difference between language learning and computer vision. Overview History of Computing (Medium, Mazhar Hassan ) Historical Development of Computing Devices (CSULB) Analytic Philosophy ~ The Linguistic Turn (Oxford, Timothy Williamson) AI History (Weighty Thoughts) Symbolic AI (Deep Code AI, Frank Fisher) Symbolic AI History (Wikipeida)  Symbolic AI and Statistical AI (ANU,  Yoshihiro Maruyama)...
Read more