2024-12-07 at 1727 Untitled 5

do we only interpolate?

how do we interpolate on ideas
how limited are LLMs? bc they do not interpolate on ideas

wait but what if they Do interpolate on ideas .
like what if with 170B params it actually interpolates on ideas in a similar way as us
bc like . we are only trained on our output not our hidden layers (ideas) right ...? i mean sorta but also no.

okay but with next word prediction it obviously does not interpolate on an Entire idea
(or maybe it is?? and maybe our sentence generation is not as idea->sentence as we think and it's closer to next-word-generation than we think)

perhaps they are actually interpolating based on ideas (in some nontrivial sense) and perhaps they are Actually good at reasoning in a human way (in some nontrivial sense), but perhaps it is just forced to reason rly immediately with no backtracking, etc.

idk how "conscious" (in the sense of self-reflection type 1 deliberate thinking and reasoning) the LLM mechanism is. maybe i was wrong about it being purely intuition. i keep on dialecting this and im swinging back and forth and refining the dialectical synthesis...

wait um .

when we optimize a model's params and interpolate. is it sorta the same as linear regression predicting smth inbetween existing points?

and also . when humans interpolate . that's only for objective things with a tangible output . like e.g. physics

wait but also music is interpolative in the same sense
(if we take everything as interpolation, even extrapolation)

but like im trying to think of . Ugh it feels like theres something that isnt interpolation + even if everything is interpolation it feels like there are things that arent trained based on interpolating on these kinds of datasets + there are activities that arent trying to be Optimized like humans just do random silly shit sometimes too

god i forgot exactly what iw as thinking about but i remember it was sorta related to humans existing in a world with actual sensory inputs and not just Text input