This post consists in only some incomplete sketches and is obviously very tentative.
René Thom called quantum mechanics 'the greatest intellectual
scandal of the 20th century'. Maybe this was too harsh, but quantum
theory was meant originally just as to be crude provisional proto-theory
destined to give place to something to better (which has not...due to
political, military, economic and industrial reasons ?). Consider the
double-slit experiment. The 20th century was also the century of
dynamical systems and chaos theory. It is clear to us that the random
aspect of the double-slit experiment must be explained in light of chaos
theory, thus of an underlying deterministic system. In a classical
setting there will also be a pseudo-random aspect for particles
traversing the two slits (but without the interference pattern). Nobody
would think of interpreting this as a probabilistic collapse of a
wave-function. In the non-classical situation it would occur to almost
anyone to see the wave-function as a real physical field associated to
particles (a "pilot-wave"). If we rule out local hidden variables (but
do we really need to ?), then we are lead to non-local yet deterministic
non-linear systems which generate the pseudo-random phenomena of
quantum theory in the standard way of chaotic dynamics. Even numerous
colliding perfectly elastic particles is a deterministic system which
yields Brownian motion. To do: study the argument involving single
photons and half-silvered and full-silvered mirrors described in
Penrose's The Emperor' New Mind p.330 (1st edition). Both the photon
and the wave-function are real existing physical entities and the
randomness of the reflection can be given an underlying deterministic
explanation. Some wave-packets are empty of particles yet still have
physical meaning. We could also consider space as being like a
Poincaré section for some higher-dimensional continuous dynamic. Of
course there is an easy objection to our proposal: what about maximally
delocalised solutions for the free particle Schrödinger equation ? Due
to many other difficulties we could also take A. Hobson's approach that 'there are no particles, only fields'.
Here small-scale irregularities, the fact that we are dealing with
approximations, etc. could well explain the 'collapse of the wave
function' - if we postulate that quantum fields are to have here an
intrinsic holistic nature so that their localized interaction around the
boundary of their support entails an immediate (or very fast)
alteration of the entire field (Hobson gives the analogy to popping a
balloon). In the double-slit experiment if we think of the wave-function
as a single entity then in reality only one small portion of the
wave-front will hit the screen first - which will be determined by
sensitivity to initial conditions and many perturbations and
irregularities in the instruments involved in the experiment. This,
based on Hobson's own analysis, could furnish the missing piece to
eliminate any appeals to probability, even in a field-only
interpretation. The $|ready >$ state is itself complex and
fluctuating (deterministically). Thus the pseudo-randomness of which
$A_i$ region will effect the 'pop'. However there seems perhaps to be a
difficulty in interpreting the apparently random aspect of the
experiment above discussed by Penrose (it would suggest that the result
of 'popping the balloon' must still be considered random). But is this
experiment really so different from the double-slit one ? We need to
find the inner geometric deterministic dynamics of field interactions
that could account for this behaviour. Maybe use the fact of the
interference of the environment (and entanglement) in all experimental
conditions.
Hobson interprets $|\Psi|^2$ as the probability of interaction of the field. We need to add an extra dimension to $\Psi$ and an accompanying deterministic non-linear dynamic field-process (as in nonlinear PDEs) which explains the resulting interaction probabilities in a totally deterministic way. This is where chaos theory is the key to quantum theory. This applies to interaction and to spin-measurement. Consider the classical orbitals of the Hydrogen atom. Some have nodal points which seem to rule out a particle interpretation. Also spin basically involves extending the phase space of the original wave function, for instance for a single particle $L^2(X, \mu) \otimes \mathbb{C}^2$. Thus our proposal is not surprising. On the other hand if we consider the orbitals of the Hydrogen atom is seems natural that they should posses also some kind of dynamic nature in an extended dimension (analogous to spin) related to the amplitude of the original wave function.
In the Penrose experiment considered above consider the detectors in two distant locations in which each spin configuration has a 1/2 percent chance and in which the two measurements are always correlated. We view the electron wave as a single entity even if divided into two packets. The unity is expressed in the phase in the extended dimension which oscillates not as two independent oscillators (one for each packet) but as a single oscillator, thus guaranteeing the correlation of the measurements.
A model: a packet could have a phase oscillating
between UP and DOWN in the extended dimension which determines the
measurements (interaction probabilities). But two coupled packets would
oscillate between UP x DOWN and DOWN x UP globally.
