Nah, you’re not a dirty anti-adaptationist! All these labels are nonsense; all that is important is understanding the math involved — something Gould never really seemed very interested in. The problem with purely verbal arguments is that there is no scorekeeper.: it’s like Olympic ice dancing, or something.
A lot of the formalism in population genetics isn’t that mentally taxing (although the derivations may be!). For example, the famous (or infamous) Hardy-Weinberg Equilibrium, p2 + 2pq + q2 = 1, requires about a middle school level of mathematical knowledge (grade 7 or 8). And one can describe it in words, but by doing so one removes its algebraic utility and clear precision of communication. A non-formal Hardy-Weinberg Equilibrium could never have become one of the pillars upon which population genetic theory was built, to use programming terminology, verbal arguments are often far less extensible than mathematical ones. In the short term verbal description of population genetic theory is much clearer to those who are not used to thinking in terms of variables, but over the long term they leave a lot to be desired in generating a contingent & systematic model. I also think that when scientists communicate and debate it wastes a lot of time when you argue in words because without any magnitude placed upon the implicit variables people easily talk past each other and generate mountains of repetitive prose.
Update: See comments for a clarification.