Another view of TPC

Perhaps it to attempt to attain passadhi (samatha) through vipassana is putting the cart before the horse. Or rather a different kind of insight is called for as a foundation. Yoga citta vrtti nirodha. Understanding the relationship between consciousness and the body - and the existence of a middle subtle body-consciousness field which carries the feedback interaction between both (the neuro-muscular aspect is important). The unified field has some analogy the solutions of Klein-Gordon and Dirac equations (Dirac was perhaps the greatest physicist of the 20th-century).The models of René Thom come very close to the idea of a field of harmonic oscillators over space-time.  The goal of the fundamental stage of TPP is to attain the pax profunda, possibly using psycho-somatic feedback as a support (to dampen or muffle the spectrum of mutually exciting harmonic oscillators). What we ordinarily call 'body' and ordinarily call 'mind' are two complementary modes of the same underlying consciousness-field. It is only in the deep clarity and stillness of the mind-body field that authentic TPC can blossom 孰能濁以靜之徐清. Thus the initial TPC involves perceiving consciousness as an excitation and self-interaction (producing the illusory perception of the individual self) of an underlying psycho-somatic field, and understanding the effective dynamics (and functional stratifications ) and feedback mechanisms to attain the desired goal. This initial TPC is indeed the pure impersonal perceiving of the flux of consciousness as thus, but also the inner first-person experience of the body (i.e. we have nama-rupa): it is also the perception of how the inner body generates a kind of frame of reference or proto-space (proto-topology) for the total sphere of consciousness. Temporality is on one hand transcendent and a condition for thought and consciousness rather than being generated by consciousness, on the other hand it is generated as an illusory excitation. We must find the lost original deeper meaning of Pyrrhonism, regarding belief, thought and agitation - the same underlying TPC insight.

Some things to explore: how can the role of symmetry in physics be transposed to understand the central role of symmetry in consciousness ? And in the symmetries in logic ? Philosophically, how can our theory of a priori computability relate to the obviously rich computational nature implicit in the study of solutions of PDEs ? Radiation, diffusion, harmonic equilibrium. Geometric optics. Singularities of the solutions of PDEs such as the Hamilton-Jacobi equation. Meaning is perhaps described as symmetry invariance in the dynamical system of consciousness.

What are sense and reference ? It seems that we can reconcile psychologism and objectivity by a theory analogous to that of the invariance under group actions used in physics. The conscious content for two different people can differ, but the contents must be related to each other through a well-defined law, a continuous symmetry group, a deformation. Galilean relativity has perhaps an immense overlooked philosophical significance.  It shows that objectivity is relational (all inertial frames of reference are equivalent, there is no absolute frame of rest), but not less objective for that. Once the idea of group action and group invariance had been brought to light - in physics as well as geometry - it is the most natural thing in the world to investigate hyperbolic geometry (Gauss, Lobachevsky) in this light. Why is the connection of Minkowski space (the pseudo-sphere) to hyperbolic geometry obscured (obtained by the analogue of the stereographic projection of a sphere)?

In order to develop these ideas we recall a previous note on computational linguistics. Large Language Models such as ChatGPT-3 use high dimensional ($dim V$ = 12,288) vector-space representations of meanings of certain textual units ('tokens'). These are generated from context in large data sets. The idea of having certain semantic 'atoms' (sememes) from which are combinatorically constructed possible meanings can be found for instance in Greimas (cf. Osgood's semantic differential for studying the variation of connotation across different cultures). Some (such as René Thom) have claimed that the idea that meaning should have a continuous, geometric aspect is found in Aristotle. Leibniz' characteristica used 'primitive terms' but it is not clear if they are combined in a simple algebraic, combinatorical or mereological way, or if complex logical expessions must be involved (or associated semantic networks). But in embedding matrices we have what would seem to be a quantification of meaning, each 'sememe' is given a 'weight' which determines its geometric relation to other meaning-vectors in a crucial way (the weights cannot be dismissed as probabilistic or 'fuzzy' aspects). To us this would correspond to the 'more-or-less' aspect of species in Aristotle. A very interesting aspect of embedding matrices is how they capture analogy through simple vector operations. This suggests another possible formalization of Aristotelian 'difference' , the same difference operating on two different genera. We get a notion of semantic distance and semantic relatedness. This also revindicates Thom's perception of geometry and dynamics in the spaces of genera.

Some questions to ask: are these token-meaning-vectors linearly independent ? If not can we work with a chosen basis ? If the token is ambiguous is the corresponding vector a kind of superposition of possible meanings, as in quantum theory ? How are we to understand the idea of the meaning of complex expressions being linear combinations of the meaning representations of the tokens occuring in the expression ? It would of course be interesting to analyze these questions relative to the other fundamental components of LLMs (attention in transformers, multi-layer perceptrons) - even if these are more practically oriented rather than reflecting actual linguistic and cognitive reality.

Suppose we are given a large text $T$ generated by a set of words $W$ and a context window $S$ of size $n$. Suppose we wished to represent the elements of $W$ as vectors of some vector space $V$ in such a way that given $v,w \in W$ the modulus of the inner product $|\langle v,w\rangle|$ gives the probability of the two words being co-occurrent in contexts S. Consider the situation: it is very rare for words $s_1$ and $s_2$ to co-occur but words $s_1$ and $s_3$ co-occur sometimes as do $s_2$ and $s_3$. But there is also a word $s_4$ which never co-occurs with $s_3$ but has the same co-occurrence frequencies with $s_1$ and $s_2$ as does $s_3$. Then it is easy to see that there is no way to represent $s_1$,$s_2$,$s_3$,$s_4$ in the same plane in such a way that these properties are expressed by the inner product. Thus the dimension must go up by one value. We can define the geometric $n$-co-occurence dimension as the minimal dimension of a vector space adequate to represent co-occurrence frequencies by an inner product. We can ask what happens as $n$ increases, does the geometric dimension also increase (and in what manner) or does it stabilize after a certain value ?

Thus we can think of different people as having semantic vector spaces which must be related in a well-defined way and in such a way that the semantic information remains coherent. Thus the mental content of the term 'horse' for Alice and Bob may be quite different, but each is related to the other through a kind of continuous deformation related to some structure contrasting the background of Alice and Bob. Thus we need to define a kind of relation space for contrasting and comparing different subjects - and in such a way that we have a representation of the algebraic structure of this space in terms of continuous deformations of mental content.

René Thom proposed that concepts were analogous to living beings and that mathematical models of the regulation structures of living beings could be applied to concepts themselves. This is kind of obvious for natural kinds and not very clear for other kinds of concepts. We need a very different approach.  We need to understand representation, the subject's mental and yet objectified representation of the world. The question: what is a world ? Software engineering and the structure of Object Oriented software aiming at creating virtual worlds (such as Unreal Engine 5, Unity or in general RPG games - we are thinking here only of the classical ones such as the Ocarina of Time which were also works of art besides sophisticated puzzles) including automated agents are of some interest though with great limitations. Generative AI is likely to be followed by more sophisticated models which can train in real-time. The run-time process structure of operating systems is also important. The irony here is that these approaches become more interesting once we discard neuro-reductionism - once we abandon the pointless attempt to view the brain as the hardware of the mind. The central hardware of the mind is to be sought elsewhere, the brain itself is a kind of auxiliary cache.

There is much analogy between the structure of a computer program and that of a novel. 

To obtain a mathematical understanding of consciousness we must first bridge the gap between mathematical models of nature and computer systems.

Also we need to take into account altered or higher states and modes of conscious experience (once harmful and falsified approaches to the spiritual life have been discarded - those that hide the truth that a royal path to spiritual realization can consist in a pure love for a real person). 

Do these higher states of consciousness possess a geometry, a topology, a semantics ? It is curious, how many Henads are there is Proclus' system ? Or does cardinality itself not apply to them ? 

Also the entire discipline of lexicology needs to be reformed. Indeed what was the ancient project of the classification and division into genera and species but a lexicological program ? We need to greatly clarify the insight involved in defining a term by its context. It is not only that we need to know the meaning of words to understand a narrative but also narratives themselves give meaning to words.  Being multilingual and practicing translation offers unique insight into the pure semantic universe.

Maybe natural language is a kind of super-mathematics which contains ordinary mathematics as a special case. It is presumptuous to ridicule the concept of an 'ideal language'. Learning other languages and in particular ancient languages is surely on the path of wisdom. In natural language we cannot in general define lexemes in the way we define mathematical or scientific concepts (and the ancient theory of genera and species must have been derived from Euclidean mathematics, law and medicine).  This is polymorphism. Meaning is in an inseparable feedback loop with life and experience, depending on whether we are engaging in solitary discourse or on which person we are conversing with.

Naive dictionaries with obvious circularity in definitions should not be despised as non-scientific. Rather they  express something profound about polymorphism, the circulation, the flow, the dynamics of meaning. Instead of a oriented tree we have a directed graph with cycles. There is an analogy with commutativity and non-commutativity. Meaning circulates like a living current or flow through the whole web or tapestry of language. The name generates a story, the story a name.

Even for mathematical concepts we gain a deeper understanding or apprehension of them through practice, through exercises, through studying proofs in which they are applied. Are these degrees of apprehension - or degrees of meaning?  Formal logic acts as an ultimate arbiter which rarely needs to act, mathematicians with distinct intuitions and apprehensions of a given mathematical concept generally can agree that their concepts are 'the same'.

Thurston's On Proof and Progress in mathematics (1994)

Schopenhauer offers a strikingly alternative theory of consciousness, concepts, intuition and representation as well as super-consciousness. So does Hume. Even Sextus. The problems discussed above are not some kind of puzzle of which one needs to find a solution. Rather they are all the result of delusion and deception, consciousness pulling itself down as an illusion over its own eyes. Only vipassana, only TPC and TPP can break through this illusion. It is foolish to ask about meaning and language without first asking about consciousness and experience. Meaning and language are within consciousness. There is a higher form of non-linguistic cognition. Thus the so-called philosophy of language is not fundamental and does not represent a radical or critical approach to philosophy, rather a dogmatic one (and its arguments against psychologism, against empiricism, against the a priori vs. a posteriori distinction, against the analytic vs. synthetic distinction, all fail). And there is the fact that consciousness can calm itself.

Attention is all you need

Around 2017 I was thinking about the problem of automatic translation and how ambiguity and idioms could be dealt with. One idea was to tag every word $w$ with one of Roget's 1000 categories (represented by a set $C$). Thus we have map $\kappa: W \rightarrow P(C)$.  Roughly speaking an ambiguous word $w$ will have $\kappa(w)$ with cardinality at least 2. Given a context $T$ and a word $w$ occurring in $T$  my idea was to devise an algorithm which functioned a little bit like a Sudoku puzzle using a concept of 'semantic distance'.  We find a word $w$ such that based on the current words $v$ in the context with singleton $\kappa$ we can determine which $c \in \kappa(w)$ is 'closest' to the set of $c$'s inhabiting the singletons of such words. We then make the choice and this should lead to finding further words that can be resolved and so forth. Of course the problem is how to define such semantic distance as well as to guarantee that the process achieves its goal and does not get stuck (but we could introduce random choices). If we view Roget's 1000 categories as organized as leaves (or even nodes) of a binary tree then there is an obvious definition. For instance 'rotation' is semantically closed to 'motion' than it is to 'feeling'.

There is a problem with compositionality for idioms like 'raining cats and dogs' or for a term like 'white rhinocerous'. Thus composition of meanings is in general multi-valued. We have uploaded a small text about this on researchgate and other platforms.

The vector representations used in LLMs suggest the following speculation. Could it be that meaning can be coherently constructed out of complex entities which themselves have no intrinsic (or easily assignable) meaning ? Or like in quantum mechanics the wave function (proto-meaning) is essentially a superimposition of eigenstates corresponding to actual observables (meanings). 

Around 2017 I was thinking about the problem of automatic translation and how ambiguity and idioms could be dealt with. One idea was to tag every word $w$ with one of Roget's 1000 categories (represented by a set $C$). Thus we have map $\kappa: W \rightarrow P(C)$. Roughly speaking an ambiguous word $w$ will have $\kappa(w)$ with cardinality at least 2. Given a context $T$ and a word $w$ occurring in $T$ my idea was to devise an algorithm which functioned a little bit like a Sudoku puzzle using a concept of 'semantic distance'. We find a word $w$ such that based on the current words $v$ in the context with singleton $\kappa$ we can determine which $c \in \kappa(w)$ is 'closest' to the set of $c$'s inhabiting the singletons of such words. We then make the choice and this should lead to finding further words that can be resolved and so forth. Of course the problem is how to define such semantic distance as well as to guarantee that the process achieves its goal and does not get stuck (but we could introduce random choices). If we view Roget's 1000 categories as organized as leaves (or even nodes) of a binary tree then there is an obvious definition. For instance 'rotation' is semantically closed to 'motion' than it is to 'feeling'. There is a problem with compositionality for idioms like 'raining cats and dogs' or for a term like 'white rhinocerous'. Thus composition of meanings is in general multi-valued. We have uploaded a small text about this on researchgate and other platforms. The vector representations used in LLMs suggest the following speculation. Could it be that meaning can be coherently constructed out of complex entities which themselves have no intrinsic (or easily assignable) meaning ? Or like in quantum mechanics the wave function (proto-meaning) is essentially a superimposition of eigenstates corresponding to actual observables (meanings). In this note we present an abstract formal approach to the basic problems regarding texts and meaning.

We start with a non-empty set $W$ of word-forms, expressions which have no meaning-bearing parts. We consider a subset $T \subset W^\star$ of possible meaningful texts. $T$ must satsify the following conditions: \[ W \subset T\] \[ \forall t \in T \, t\notin W \rightarrow \exists s,u \in T \, t = su\] The last axiom means that every text has at least on syntactic decomposition - a binary tree expressing sucessive division into meaningful elements down to the level of $W$. 
We are given furthermore a non-empty set $M$ of possible meanings. We postulate a map \[\sigma : T \rightarrow P(M)\] satisfying $\sigma(t) \neq \emptyset$ for all $t \in T$ and a map \[\lambda : M \rightarrow P(T)\] satisfying $\lambda(m) \neq \emptyset$ for all $m \in M$. The first map gives the possible meanings of a text $t$ and the second map gives the possible ways to express a given meaning $m$.
Recall that for any set we have a map $P(P(X)) \rightarrow P(X)$ obtained by taking unions. Using this map we can compose $\sigma$ and $\lambda$ to obtain maps \[\pi : T \rightarrow P(T)\] and \[\alpha : M \rightarrow P(M)\] which express the ways to paraphrase a given text and the possible linguistic ambiguities (semantic misreadings) conveyed by expressions of a given meaning.

Given $s,t \in T$ we define $s \subset t$ to mean that there exists $s_1,...,s_n \in T$ and $u_2,...,u_{n-1} \in T$ such that (for $ n> 1$, $s_n = t$,  $s_i = s_{i-1}u_{n-1}$ or $s_i = u_{n_1}s$ for $i > 1$ and $s_1 = s$. For $n = 1$ we require that $s = t$.  We postulate that there is a family of maps $\rho_{ts} : \sigma(t) \rightarrow \sigma(s)$ for each $t,s \in T$ with $s \subset t$ which satisfies the properties $\rho_{tt} = id$ and $\rho_{su}\circ\rho_{ts} = \rho_{tu}$. Thus we boldly state that there is a restriction to 'white' of the standard meaning of 'white rhinocerous'. This means that the meaning of a text uniquely determines the meanings of all its meaningful components.

Furthermore $\rho$ must satisfy the fundamental property: Given $t = s_1s_2$ with $s_1,s_2 \in T$ and given $m_1 \in \sigma(s_1)$ and $m_2 \in \sigma(s_2)$ there exists at least one $m \in \sigma (t)$ such that $\rho_{ts_1}(m) = m_1$ and $\rho_{ts_2}(m) = m_2$. An important corollary is that if $s \subset t$ then for every $m \in \sigma(s)$ there exists $n \in \sigma(t)$ such that $\rho_{ts}(n) = m$. We hold that a lawful syntactical combination of meaningful expressions has some meaning. Indeed 'green idea' is just as meaningful as Polonius' 'green girl'. Metaphor should be at the heart of any cogent linguistic philosophy and formal linguistics. But our principle might seem to fail here because the restriction of the meaning of this expression to 'green' has a different meaning from the ordinary perceptive meaning of 'green'. Metaphoric green is a valid meaning for the expression 'green' and thus the meaning of the combination 'green idea' is valid. However even the expression 'green idea' with 'green' in its perceptual sense is just as meaningful as Meinong's 'square circle'.

The new senses of 'green' and 'white' adquired by restriction from 'green girl' and 'white rhinocerous' recall some of the mechanisms for attention in LLMs. We define the set of $t$-contexts for $t\in T$ to be $C(t) = \{(t_1,t_2) : t_1,t_2 \in T \& t_1t_2 \in T\}$. For $c = (t_1,t_2) \in C(t)$ we write $c[t]$ for $t_1tt_2$.

Finally we postulate that, for $t \in T$, there is a map $\kappa : \sigma(t) \rightarrow C(t)$ satisfying the following property for all $m \in \sigma(t)$: \[ \forall n \in \sigma (\kappa (m)[t])\, \rho_{\kappa(m)[t] t}(n) = m \]  This means that given an expression $t$ there is a context such that even if ambiguous still determines the meaning of $t$ uniquely. We can think of contexts for 'white rhinocerous' involving 'stuffed animal' and involving 'species' which determine distinct meanings.

The above theory could be carried out using the set of terms generated freely by a non-associative operation $W^\bullet$ and by postulating that each $t \in T$ has a unique syntactic decomposition tree in $W^\bullet$ - but this is a little too restrictive perhaps. We can rewrite the above theory in probabilistic terms, specially $\kappa$. We can replace contexts with co-occurrences. And turn $\kappa$ into map which from the data of certain $t_i$s co-occuring with $t$ (at certain distances) assigns a certain probability distribution to possible meanings of $t$.

But let us now consider another approach. We suppose that there is a 'metric' defined on $M$, $d: M \times M \rightarrow \mathbb{R}_0^+$. To simplify things we can also consider a finite disjoint decomposition of $M$ into categories $M = \bigcup_{c\in C} M_c$ (for example something based on the 1000 categories of Roget' thesaurus, but these of course only work for $W$ or small texts) and consider the distance only on $C$ and extend it in a trivial way to $M$. And important question: what is the distance of $m \in \sigma(ts)$ to $\rho_{(ts)t}(m) \in \sigma(t)$ ? Note that each $t \in T$ determines a subset $\sigma(t) \subset M$. Suppose we have a context $c \in C(t)$ and consider $\sigma(c) \subset M$. How do we define the distance between a $x \in M$ and a subset $ X \subset M$ ? One possibility is $d(x,X) = min \{d(x,z) : z \in X \}$ but this is not what we want. Rather we need an average over all the distances $d(x,z)$ for $z \in X$. For finite $M$ this is easy. Otherwise we need to introduce a measure on $M$. Now we can consider a map  \[\kappa: C(t) \rightarrow \sigma(t) \]  which associates to each $c \in C(t)$ the element $m \in \sigma(t)$ such that $d(m, \sigma(c))$ is smallest. Or from a practical perspective we can use the distance induced by the finite decomposition into categories. This works best (for the case of Roget's categories) if instead of using $\sigma(c)$ we use the subset of $M$ determined by all of the $\sigma(w)$ for $w$ occurring in $c$. The problem with this approach is that the minimum may not be unique. But for a text $t$ we can decompose it into a context $c[s]$ for different $s \subset t$. Our algorithm would first seek the right $s$ so that $\kappa$ determines a unique minimum. This will allows us to refined the images in $M$ for the other contexts (strickly speaking this is no longer the union of the $\sigma(w)$ but a refined subset). It is to be hoped that this process could be continued until the entire $t$ is disambuiguated. Is this not what the human mind does ?  It may be worth considering that $M$ has a more refined description in terms of subsets of a certain 'semantic space' $\Sigma$ in such a way that the $\sigma$ of elements of $W$ are like 'points', their concatenations like 'lines' and so forth. And we can define the metric on $\Sigma$ rather than directly on $M$.  Note that we must not confuse the mathematical with the philosophical aspect. If we postulate that meaning can be formally analized and studied via context this is not meant to imply that meaning is context, anymore than studying a recursive set as a set means that we are identifying the associated algorithm with the set itself.

Given a $T \subset W^\star$ (for simplicity we assume it is finite) as in section 1, let us consider a map $c: T \rightarrow P(T)$ which associates to each $t \in T$ the set set of all $s' \in T$ such that $t$ occurs in $s$ (contexts). From a practical perspective it is better to restrict $c$ to contexts having a certain limit of length, which we fix to be $n$ (and denote by $T_n$) and denote the resulting map by $c_n$. Of course then $c_n$ becomes empy for $t$ precisely of length $n$. We can define a notion of distance $d(t_1,t_2)$ between $t_1,t_2 \in T$. This is given by  \[\frac{\Sigma_{t \in T_n: t_1,t_2 \in t} dist(t_1,t_2,t) }{|T_n|}\]  where $dist$ is the ordinary distance between $t_1,t_2$ in $t$. There are various ways to define distances of sets. In our case we should use $d(A,B)$ being equal to the average distance. The idea is to decompose $c_n(t)$ into subsets which are far apart, reflecting distinct semantic categories. If for a given radius the elements of $T_n$ in that radius are random and dispersed then this is not possible. Ideally we wish to find 'clusters'. We can view the decomposition into clusters as writing a vector represeting a general element as a linear combination of vecotrs representing the clusters or categories.

Note that if $t_1t_2$ is know to be part of a defined cluster then it is plausible that this cluster also determines one for $t_1$ and $t_2$ thus revindicating the existence of the map $\rho$. Let us look more closely at $T$. It can mean all possible texts in a given language.  But if we consider a person $p$ then we can take $T_p$ to mean the subset generated by the meaningful subsegments of the text $t_p$ consisting of all linguistic material either thought in inner verbal discourse, heard or spoken from the moment of birth to the moment of death. Or we can consider in some sense all the possible such maximal onto-texts. We can also define the same kind of text for communities and their history. $T$ is then like a book whose pages correspond to a possible total text of linguistic material in a certain possible history of a community (cosmo-texts). The set of possible books of the world encompassing the total linguistic-consciousness of all people throughout history.  A text is mainly about certain topics. A biography is about a person. Can we give a purely textual definition of aboutness ?

Since a cosmo-text is finite there will be meaningful texts which will escape it. Is there an English text of 1000 words which will never be read by mankind and yet is highly meaningful (or would be meaningful in its possible encompassing cosmo-text) ? Can we give a purely textual combinatorial definition of 'highly meaningful' ? We give an abstract answer. Consider a set $\Sigma$ of symbols and suppose we have a metric $d$ on $\Sigma^\star$. Let $S \subset \Sigma^\star $. Let $S(w)$ denote all elements in $S$ with initial segment $w$. Then a word $w \in \Sigma^\star$ more impactful (meaningful) than $w' \in \Sigma^\star$ for $s$ in $S$ if the rough 'distance' between $S(sw)$ and $S(sw')$ is significant (again we need a good notion of distance between sets). This would correspond to an 'influential' or 'seminal' work in the global cosmo-text.

Large Language Models such as ChatGPT-3 use high dimensional ($dim V$ = 12,288) vector-space representations of meanings of certain textual units ('tokens'). These are generated from context in large data sets. The idea of having certain semantic 'atoms' (sememes) from which are combinatorically constructed possible meanings can be found for instance in Greimas (cf. Osgood's semantic differential for studying the variation of connotation across different cultures). Some (such as René Thom) have claimed that the idea that meaning should have a continuous, geometric aspect is found in Aristotle. Leibniz' characteristica used 'primitive terms' but it is not clear if they are combined in a simple algebraic, combinatorical or mereological way, or if complex logical expessions must be involved (or associated semantic networks). But in embedding matrices we have what would seem to be a quantification of meaning, each 'sememe' is given a 'weight' which determines its geometric relation to other meaning-vectors in a crucial way (the weights cannot be dismissed as probabilistic or 'fuzzy' aspects). To us this would correspond to the 'more-or-less' aspect of species in Aristotle. A very interesting aspect of embedding matrices is how they capture analogy through simple vector operations. This suggests another possible formalization of Aristotelian 'difference' , the same difference operating on two different genera. We get a notion of semantic distance and semantic relatedness. This also revindicates Thom's perception of geometry and dynamics in the spaces of genera. Some questions to ask: are these token-meaning-vectors linearly independent ? If not can we work with a chosen basis ? If the token is ambiguous is the corresponding vector a kind of superposition of possible meanings, as in quantum theory ? How are we to understand the idea of the meaning of complex expressions being linear combinations of the meaning representations of the tokens occuring in the expression ? It would of course be interesting to analyze these questions relative to the other fundamental components of LLMs (attention in transformers, multi-layer perceptrons) - even if these are more practically oriented rather than reflecting actual linguistic and cognitive reality.  Suppose we are given a large text $T$ generated by a set of words $W$ and a context window $S$ of size $n$. Suppose we wished to represent the elements of $W$ as vectors of some vector space $V$ in such a way that given v,w in W the modulus of the inner product $|\langle v,w\rangle|$ gives the probability of the two words being co-occurent in contexts S. Consider the situation: it is very rare for words $s_1$ and $s_2$ to co-occur but words $s_1$ and $s_3$ co-occur sometimes as do $s_2$ and $s_3$. But there is also a word $s_4$ which never co-occurs with $s_3$ but has the same co-occurrence frequencies with $s_1$ and $s_2$ as does $s_3$. Then it is easy to see that there is no way to represent $s_1$,$s_2$,$s_3$,$s_4$ in the same plane in such a way that these properties are expressed by the inner product. Thus the dimension must go up by one value. We can define the geometric $n$-co-occurence dimension as the minimal dimension of a vector space adequate to represent co-occurrence frequencies by an inner product. We can ask what happens as $n$ increases, does the geometric dimension also increase (and in what manner) or does it stabilize after a certain value ?