Collaborative augmented reality system prototype developed by the Rekimoto Lab to deliver two-way information, both providing a realtime visual first-person camera feed which is organized spatially and visual instructions of points of interest to a head-mounted headset - video embedded below:
JackIn is a new human-human communication framework for connecting two or more people. With first-person view video streaming from a person (called Body) wearing a transparent head-mounted display and a head-mounted camera, the other person (called Ghost) participates in the shared 1st-person view. With JackIn, people’s activities can be shared and assistance or guidance can be given through other peoples expertise. This can be applied to daily activities such as cooking lessons, shopping navigation, education in craftwork or electrical work, and sharing experiences of sporting and live events. For a better viewing experience with 1st-person view, we developed the out-of-body view in which 1st-person images are integrated to construct a scene around a Body, and a Ghost can virtually control the viewpoint to look around the space surrounding the Body. We also developed a tele-pointing gesture interface. We conducted an experiment to evaluate how effective this framework is and found that Ghosts can understand the spatial situation of the Body.