2024-12-11 at 03:51

summary of convo w shane today

i met this guy beside the pool tables in faunce while i was talking about Hidden Markov Models w moses and then he chimed in and started teaching us about HMMs and then i asked him what year he was and he said he's a phd student and i asked him what research and he said ai rl and so we started talking about ai

here's his website
https://sparr.io/

double descent

penalize large coeffs

language changes perception not bc of the language but bc the language is a scaffold for facilitating practice of cultural categorizations (and obviously training at smth makes your perception finer, like w chess or tennis)

we used to only talk about nonlinear convex problems bc we could prove that it is the global minimum, now we talk about nonlinear problems w just many many params

you need ALL params to be convex to be minimum
you only need one param to be concave to be a saddle point
more params, probably much lower chance to get All of them to be convex, so it's like you usually always find at least one concave param and then you can gradient descent further --> so if it happens that all ur params r convex, then it's probably The global minimum (???????)

defies the intuition of . more params will overfit ur data

we also talked about how . imagine if theres a perfect solution that does exist in our param space, then surely we must be able to descend to it somehow, and if we descend to some minimum then it is probably The global minimum perfect solution ????
sounds weird + probably also misunderstood some of it

his research is on . four legged ant (who doesnt even know how to walk yet) learning to escape a maze (whose reward is only at the exit)

that has a huge dimensional action space (multiple joints w many degrees of freedom + combinatorial explosion for the maze, like imagine if u didnt know how to walk or navigate it would be near impossible to just randomly move in such a way that u randomly exit the maze)

game controller has a low dimensional space

and this question is particularly motivated and interesting bc . he is sorta trying to figure out a method for unsupervised learning that projects high dimensional action spaces onto lower dimensional generalizable abstractions (or smth like that. there r many places that the word generalizable could go)

work on subtask approaches vs his approach, and the subtaks approaches r actually not doing v well

manifold of the action space (or i forgot what space what manifold blah blah)
cost functions descent towards diff solutions
talked about his approach was sorta like understanding the entire manifold