2024-12-11 at 20:03

i dont think i wrote abt this yet but . i remember when reading the Voyager minecraft thing and how they used embeddings of past solutions to help current/future solutions, i remember i was like "oh i've been thinking of that" (but i forgot what Exactly i was actually originally thinking and how i originally wanted to go about it, so now the earliest version i actually remember is just the voyager implementation)

==========

constitutional ai not simple RL but like nuanced RL
could we implement this w reasoning ??

also i dont know how constitutional ai works and i should read their paper

===

also i wonder how humans process the nuanced RL (re: note "mirror neurons") like when you critique someone's process at a high level and they Somehow process that feedback!
(also humans process feedback in diff ways! one example is taking it in intuitively somehow vs writing it on a cheat sheet. also many other dimensions tho)

related: when we teach humans how to reason
it is like constitutional ai rl (?)
and it is like . the reward is not actually on the correct answer but on smth deeper
like . when i learn physics by going from relatively deeper first principles (at least compared to some peers), it's not exactly bc ive optimized my physics-learning to be like this (although that is part of it) but some of it is also just that i feel like this subgoal is inherently enjoyable for its own sake as well, right? maybe? idk if that's true. but i think there's some truth to that at least in a sense, like it's not just that i've optimized my physics-learning more, but that my reward is actually on smth else (understanding first principles) as a proxy for successful physics-learning

===

also teaching a human how to reason is not even a closed problem itself

also how would we find a solution to teaching salena-esque person (like not just teaching salena herself, but let's say like we also have clones of salena that anyone can attempt to teach)
it's like . if you rly wanted to be optimal about it . how would u do it ? would you just immediately try to teach salena and go blitzkrieg about it ? would you cycle thru diff methods ? maybe have other people try teaching salena clones ? maybe have 100 diff teachers test out 100 diff methods w diff salena clones . maybe have those teachers also reason about reasoning .

an aside: bro im even having a hard time fucking reasoning about this shit lmfao . fuck dude this rly proves reasoning is not that easy and it is not even a closed problem for humans

and let's say there's some good guide/knowledge out there about how to teach humans to reason .
i think it's interesting to ask . how did we discover that solution?

• many diverse hypotheses/attempts/experiments
• communication/debate betw ppl trying to combine their knowledge, w scientific reasoning (or at least an attempt at it, note that even humans are far from perfect at reasoning)

===

also the ARC challenge is not just about teaching salena how to reason, but also designing the brain -- and then those two meet in the middle
like . we have people designing brains (model architecture), training them to some base level intuition+knowledge (e.g. LLMs that are already trained like 4o), and then we can teach them how to reason (either in the prev step or as a next step)

===

hogly moglies

"I know this only happens if they are given a problem that is unsolvable without learning to think differently. If it's possible using their existing knowledge, they won't bother learning something new."

https://www.reddit.com/r/AskAcademia/comments/1ghot1m/comment/luz317b