Model Ablation

I came across this term when reading OpenAI's paper on GPT (Generative Pre-Training), which is one of the references in the book Build a Large Language Model (From Scratch). It's a new word/terminology for me, so I thought I might as well note it down.

According to Wikipedia, ablation is originally a medical term meaning the surgical removal of body tissue. In the context of machine learning, its usage is credited to Allen Newell. An ablation study means removing a component of an AI system in order to study the impact/contribution of said component.

This is sort of similar to a sensitivity analysis - you change part of the input, and you study the changes in the output in order to study the importance of whatever you've just changed.