Reduction of complex dynamic touch information to a single stable perceptual feature

Reviewer #1 (Public review):

Summary:

This manuscript deals with the ability to identify material hardness from the vibrations induced by single light taps on that surface. Psychophysical tests of human perception under varying conditions of modified fingertip compliance and/or externally imposed vibrations demonstrated that total spectral energy was the main determinant of perceived hardness and that perception of increased hardness can be induced by adding external vibration at the time of contact.

Strengths:

The experiments are well-reported and the data potentially useful, but much narrower than is implied by the (provisional) title and abstract. Their potential application to tactile perception in virtual reality seems promising, but the largely unexplored need for synchronization with physical contact and modulation with velocity and force of that contact seems likely to complicate proposed applications to prosthetics and telerobots.

Weaknesses:

(1) The authors have confused discriminability with perception. The sense of touch is derived from several different types of mechanoreceptors and processed into several dimensions of haptic perception. The fact that subjects can rank surface material hardness correctly when requested to focus on that alone does not mean that they rely on total spectral energy normally or that total spectral energy is normally perceived as surface material hardness, as opposed to other aspects of materials, such as their surface texture. They have not considered the effects of more complex features of most surfaces, such as curvature, lamination or other exploratory movement strategies besides light taps.

(2) Discussion section. Lines 262-264 are overstated. Dynamic spectral energy can be used to modify perceived hardness when exploratory movements are limited to taps that are unlikely to generate any other useful cues, such as skin deformation or proprioception. The authors have not explored what happens if there actually are conflicting cues in non-vibratory modalities. There are many different examples from sensory psychophysics of percepts that arise from taking the mean of conflicting cues (e.g. stereophonic sound localization) and others that arise from a dominant modality (e.g. self-motion perception from visual flow fields, vestibular signals and proprioception).

The authors have ignored the substantial literature on artificial tactile sensors and their ability to identify texture, hardness and other haptic properties of materials. These have emphasized the importance of the many types and parameters of exploratory movements, which were loosely specified and not quantified in these studies.

See:

Li, Q., Kroemer, O., Su, Z., Veiga, F. F., Kaboli, M., & Ritter, H. J. (2020). A Review of Tactile Information: Perception and Action Through Touch. Ieee Transactions on Robotics, 36(6), 1619-1634. doi:10.1109/tro.2020.3003230.

Fishel, J. A., & Loeb, G. E. (2012). Bayesian exploration for intelligent identification of textures. Frontiers in Neurorobotics, 6(4). doi:10.3389/fnbot.2012.00004

Fishel, J. A., & Loeb, G. E. (2012). Sensing Tactile Microvibrations with the BioTac - Comparison with Human Sensitivity. Paper presented at the IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, Rome.

(3) Introduction (lines 23-31) and Discussion (lines 296-298). The notion that tactile receptors are "frequency tuned" is something of a straw man. Different receptor types are preferentially sensitive to different broad spectral bands, but it has long been known that they can be driven by larger stimuli outside those bands and that humans have very limited ability to discriminate actual frequency of tactile vibration (as opposed to auditory pitch), particularly for frequencies greater than the maximal one-to-one firing rate of neurons (~200-300 Hz). Conversely, fine onset timing of spikes in tactile afferents appears to be available from brief contact taps to identify features other than hardness; see:

Johansson, R. S., & Flanagan, J. R. (2009). Coding and use of tactile signals from the fingertips in object manipulation tasks. Nature Reviews Neuroscience, 10, 345-359.

Pruszynski, J. A., Flanagan, J. R., & Johansson, R. S. (2018). Fast and accurate edge orientation processing during object manipulation. eLife, 7, e31200.

(4) Methods section. The Lofelt L5 actuator used to apply vibrations to the fingernail is rather large for use on multiple fingers of a haptic display. Do the authors know of any more compact technology with the requisite power and frequency response? One of the most useful contributions of this paper is to suggest that those details matter relatively little, which opens up more compact technologies such as piezoelectric actuators.

(5) Methods section. It is good that headphones were used to block and mask audible tapping sounds, which are known to be capable of generating tactile illusions (Jousmäki, Veikko, and Riitta Hari. "Parchment-skin illusion: sound-biased touch." Current biology 8.6 (1998): R190-R191). But that suggests that hardness might be signalled by precisely timed acoustic stimuli, which would be much easier to deliver than fingertip vibration.

Reviewer #2 (Public review):

This paper aimed to demonstrate that total spectral energy alone is sufficient to drive hardness perception and material identification. Through five user studies, they tested materials ranging in stiffness and with covered fingers to support their claim. Using a spectral energy compensation framework, they concluded that total spectral energy alone, regardless of frequency content, was sufficient to support material hardness percepts. However, it should be noted that all experiments used a tapping procedure, which is not the standard exploratory procedure when judging material hardness. A tapping method also selectively enhances vibratory feedback while limiting others. This fundamentally limits the scope of their work, and assessing their claim on generalizability would require further experimentation.

Some additional clarification and extension on the experiments are also suggested:

(1) According to Lederman and Klatzky (1987), pressure, and not tapping, is the exploratory procedure humans use to judge hardness. And during tapping instead (as used in all experiments), it is expected that the dominant cue available to the user comes from vibrations, as other mechanical cues, such as skin stretch, are limited. These vibrations could serve as a proxy for hardness, as claimed by the authors, but it is unclear if the participants are basing their evaluations on perceived hardness or vibration intensity. A more fundamental question that needs to be answered to support the paper's claim is whether a single tap is sufficient for conveying a material's hardness. To better support their claim, I recommend that the authors include an experiment using participants' bare fingers with materials of the same modulus but different damping coefficients. These materials would produce different vibration signals when tapped, but are equivalent in hardness.

(2) The setup text for experiment 4 does not match the results. Results suggest that a finger covered with a bubble and touching a soft material was used (i.e. dual compliance), but the setup describes otherwise. The authors should clarify this and confirm that this is different from experiment 2.

(3) As silicone, foam, and rubber can have very similar or different hardness depending on the specific material used, please report the hardness of each material tested (Shore or Young's modulus) to better understand the range of stiffness tested.

(4) In the "materials grouping and selection" section, it states that a pilot study suggested hard materials tended to be perceptually similar while softer materials were easily distinguishable. However, this contradicts the results in experiment 1. The authors should expand on the details of the pilot study and address the inconsistency between its findings and experiment 1.

(5) The methods section suggests that individual recordings for each material were performed before the experiment. Please clarify if this is correct, or if a single signal for each texture was used across all participants. Additionally, were the participants' tap pressure controlled during either the recordings or in the experiments? If not, how do the authors account for the difference in intensity that would be generated due to different tapping pressures across participants and trials?