We present a multimodal paradigm for exploring topological surfaces embedded in four dimensions; we exploit haptic methods in particular to overcome the intrinsic limitations of 3D graphics images and 3D physical models. The basic problem is that, just as 2D shadows of 3D curves lose structure where lines cross, 3D graphics projections of smooth 4D topological surfaces are interrupted where one surface intersects another. Furthermore, if one attempts to trace real knotted ropes or a plastic models of self-intersecting surfaces with a fingertip, one inevitably collides with parts of the physical artifact. In this work, we exploit the free motion of a computer-based haptic probe to support a continuous motion that follows the local continuity of the object being explored. For our principal test case of 4D-embedded surfaces projected to 3D, this permits us to follow the full local continuity of the surface as though in fact we were touching an actual 4D object. We exploit additional sensory cues to provide supplementary or redundant information. For example, we can use audio tags to note the relative 4D depth of illusory 3D surface intersections produced by projection from 4D, as well as providing automated refinement of the tactile exploration path to eliminate jitter and snagging, resulting in a much cleaner exploratory motion than a bare uncorrected motion. Visual enhancements provide still further improvement to the feedback: by opening a view-direction-defined cutaway into the interior of the 3D surface projection, we allow the viewer to keep the haptic probe continuously in view as it traverses any touchable part of the object. Finally, we extend the static tactile exploration framework using a have problems that are simple enough to formulate clearly, yet complex enough that even Einstein would have had trouble working out the details in "images that [one] can reproduce and recombine at will."

In this paper we take the first steps towards this goal by designing and implementing novel multimodal methods for exploring surfaces (two-manifolds) embedded in 4D Euclidean space.