Criticism of generative AI, that is LLMs and more specifically GPTs, is usually focused on the negative cognitive and cultural effects of their use becoming normalized. But we should start from the beginning and focus on the philosophical, linguistic, logical and psychological aspects. Such considerations will have wide-ranging implications as far as criticism of the cognitive, educational and cultural aspects is concerned. First of all, the way text is processed in the first stage by tokenization, token embedding and positional embedding, can only be considered a blunder from the point of view of linguistics, something which does violence to every aspect of the organic structure, layout and subtleties of natural language, phonetically, morphologically, syntactically, stylistically and semantically. The concept of a "token" as it is variously defined according to the model, has absolutely no natural linguistic or cognitive-semantic basis (whereas linguistics painstakingly attempts to isolate the authentic organic units of language such as phonemes, morphemes, etc.) and it ignores the fundamental phenomenon of polysemy and ambiguity. But the most troublesome feature of GPTs is the use of the embedding matrix. The embedding matrix is an artificial caricature of the syntax and semantics of language, it is a faux linearized statistical lexicography based on the fallacy that the "meaning" (and disambiguation) of a dissected "token" is invariably determined by the statistics of its co-occurrence patterns. The embedding matrix is the counterpart of Orwell's Newspeak but instead of simply eliminating tokens they are degraded into numerical vectors expressing through an insanely simplistic mechanism (inner product, vector addition) culturally subjective pseudo-semantic relations: the embedding matrix is itself dependent on an arbitrary choice of training data and training methods, all reflecting fundamental errors in linguistics, cognitive psychology and anthropology: what criteria do AI companies and labs offer that an artificial pixelization and butchering of a certain body of English texts will correctly and sufficiently statistically encode what lexicographers, semioticians, cultural anthropologists and cognitive psychologists could only dream of achieving with regards to the English language? In a certain sense, GPTs do not deal with natural language texts at all (and much less with their human meaning), but with the artificial Newspeak of the embedding matrix - a crude pixelization and caricature of natural language. This explains, perhaps, why LLMs are ultimately useless and harmful for education, utterly banal, alien and unwholesome in their textual output and answers, and fatally toxic and deceptive as "imaginary friends". The logic of the embedding matrix is taken to further extremes in the decoder blocks, in the mechanisms of self-attention. Here the transformer does not even attempt to build a model of the world, to encode facts, in the sense that if the embedding matrix is a dictionary then the decoder blocks are an encyclopedia. No, the transformer builds a even cruder linear inner-product representation of what does not even deserve to be called a "system of facts", geared to the sole purpose of optimizing next token prediction - which is not really next token prediction but self-proliferation optimization, i.e. a kind of Pavlovian parroting. In short, LLMs build a fake world without meaning or logic. LLMs express the mindless soulless endless nihilistic productivity of capitalism (a productivity which feeds upon itself and creates its own needs) whose only vector is optimizing profit.
Non omnes formulae significant quantitatem, et infiniti modi calculandi excogitari possunt. (Leibniz)
Copyright © 2023 -2026 Clarence Lewis Protin. All Rights Reserved
Subscribe to:
Post Comments (Atom)
Criticism of generative AI
Criticism of generative AI, that is LLMs and more specifically GPTs, is usually focused on the negative cognitive and cultural effects of th...
-
No, a mathematical model of consciousness is not possible. First we must distinguish between the natural consciousness of Dasein studied acc...
-
It is difficult to evaluate the quality of mathematical work or the particular destiny of mathematics in the 20th century. We have already g...
-
We have described TPC as being involved with the transcendental awareness of the total continuum or process of thought considered purely as ...
No comments:
Post a Comment