2024-12-05 at 2018 Untitled 3

ah . god im realizing how new i am to thinking about AI because im making many silly mistakes. but its okay.

playing devil's advocate though: how do we make sure it doesn't optimize for:

• taking shortcuts that look efficient but aren't actually good reasoning
• finding patterns in the training problems rather than learning general reasoning
• memorizing solution templates instead of actually reasoning

oh wow all of these are problems we have with Humans too... (note to self: 2 and 3 are basically the same statement)

let's think about how we get out of this dilemma with Humans.

for #1, we have to check their intermediate steps / line of reasoning. perhaps we can use a model that is known to already be pretty good at math. like 4o. it's not 100% good but it's pretty dang good. just like a human math teacher.

for #2 and #3, i mean it is actually Good to find patterns that create heuristics (and note to self, dont forget - i dont wanna implement this into the LLM, but in the Reasoner). but we just have to make sure it's not limited to only solving those types of problems in its training data. thus, just like humans, we can test their True Reasoning Capability by seeing if they can solve a new type of problem that wasnt in their practice (e.g. the midterm has a diff problem compared to the homeworks)

==========

now can we still optimize for reasoning efficiency/duration while it reasons through a math problem?

i feel tempted to say that we need human RLHF to say like "oh you couldve made this step faster"

oh wait also like . alphazero was trained on human chess data so it was alr pretty good before they did the process of playing by itself. how do we train our model on human reasoning data? it seems kinda difficult bc a lot of the nuanced small important reasoning steps for humans happen inside the brain so it's more intangible or more invisible (compared to a way more tangible thing like moving the chess piece on the chessboard). Wait okay but we taught alphazero how to "reason" through chess with Zero human reasoning data. is that bc researchers implemented search?

okay but the thing is, when youre building from first principles, AT LEAST FOR FIELDS LIKE MATH, there's only like literally one good move and its easily verifiable. i swear thats easier than chess or go. in those games there are like a super amount of possible moves. in math there's like . i mean i guess there's more moves but the right move is pretty easy to approximate (usually) and its pretty easy to verify the right move

also how do u implement smth like . oh be careful for this counterintuitive step!... idk i think alphago actually learns that pretty well tbh.

hmmm i know i wanted to write more abt the above few ideas but idk my brain is fried rn and also i just dont have the knowledge to look at these well

i need to actually make sure im not bogged down by jumping in and learning these things. but also make sure you actually fucking learn these things.

wait also . how can we make novel advances using only intuition but LLMs cant .?