2024-12-09 at 13:31

i wanna be curious and just explore whatever but like also i dont wanna waste time,, i wanna actually be fast and speedy and actually catch up to things so that i can actually think smart about these things and actually start experimenting on novel things quickly, like ofc it's very useful to just sit around and think about hypotheses and we need that to get New Creative Ideas (re: greatness cannot be planned, etc) and that is very intangible but i also wanna be able to respond to tangible measurements like "what did u get done last week" and i wanna actually move the needle ykwim?

like i wanna get out of hypothesis dream land kinda quickly
but also rn im kinda swamped w finals

okay so i wanna .

• take an overview of my current hypotheses and notes to see what experiments i should do
• oh wait but i need to learn the math first to translate the psychological hypotheses into actual NN experiments
• okay so learn some basic math/code first, and then translate the psych hypotheses into actual experiments
• then also combine and consolidate my hypotheses and see if i'm missing anything immediate/adjacent

how to figure out the math/code

• 3b1b series
• speedrun an intro course or neel nanda's roadmap or smth like that

==========

Why neural networks aren't neural networks
https://www.youtube.com/watch?v=CfAL_cL3SGQ

neural networks are such a very very leaky abstraction and yet somehow the abstraction works in so many cases

===

dude i'm realizing i really dont know much about how 4o works, or just like anything tbh

like . interpolation ? manifold hypothesis ? finding linear/nonlinear transformations ? and then overfitting vs underfitting, how? and then RL,, how would you even RL an LLM?

and then also, how does RL even work lol