Extended Intelligence and Mediated Reality Veillametrics in the Space, Time, Frequency, and Amplitude Domains

Date

2019-06

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Sensing is a key part of human life, and also plays a key role in technological systems. Technology and humans share a need to sense the physical world accurately and precisely, preferably with low error rates, low noise, high dynamic range, and high bandwidth. With the rise of surveillance (institutional sensing) and sousveillance (individual grassroots sensing) --- or more generally, veillance --- in the world around us, it is desired to enhance, control, measure, and understand the capacity-to-sense, in the context of extended intelligence, in order that this capacity-to-sense can benefit humanity. This thesis proposes new algorithms, circuits, and devices to extend the sensory capability of machines and humans. This includes sensory enhancement systems, user-interface devices that embody humanistic intelligence (HI) and a proposed new field of veillametrics. Veillametrics --- the measurement of sensing --- is a quantified approach to the sensing of sensing itself, introducing the new concept of “veillance flux.” Veillance flux is the quantified capacity-to-sense “emitted” from sensors, as that capacity-to-sense propagates through space, reflects, refracts and scatters. The mathematics and physics of veillance flux are introduced, along with measurement systems and visualizations of it. Sensory aid systems are further proposed, to expand sensory capability for machines and humans, in the spatial, temporal, frequency, and amplitude domains. A first application is cameras, where the capability to sense extreme-dynamic-range signals is improved by compositing together dynamic ranges from multiple sensors. A new concept called “coupled dynamic dynamic range” is introduced, where sensors themselves are controlled in a feedback loop, to expand their range of sensing. This work is then expanded to a wider class of sensor signals, including audio, high voltages, and seismic vibrations. This has led to newly-published concepts such as the amplitude-frequency transform, and harmonic range density functions. Novel user-interfaces and algorithms are introduced, to sense human input continuously over spatial-temporal-range domains, and provide tactile, audible and visual feedback to the user, for a tightly coupled interaction. Finally, space-time-frequency-amplitude domain signal processing is used to characterize sensing itself, using veillance flux, to help us understand, mediate, and augment the sensory capability of machines and humans.

Description

Keywords

augmented reality, mediated reality, sensing, sensors, sensory enhancement, veillametrics

Citation

DOI

ISSN

Creative Commons

Creative Commons URI

Items in TSpace are protected by copyright, with all rights reserved, unless otherwise indicated.