Introduction

When humans interact with objects through touch, they rely on material properties such as hardness, compliance, texture, and friction, which are conveyed through tactile cues including high- and low-frequency vibrations, microslips, skin strain, and slip motion to guide perception and action. A tap on a surface excites a brief transient that propagates through finger tissue as broadband vibrations. The nervous system interprets these vibrations to infer properties such as whether a surface is hard like metal or soft like foam (1). Hard surfaces produce higher-amplitude, higher-frequency transients, whereas soft materials yield attenuated, lower-frequency signals. Classical studies established that mechanoreceptors in the skin are frequency-tuned, with different afferent classes responding preferentially to distinct vibration bands (2, 3). This led to the longstanding view that frequency-selective populations — particularly the Pacinian corpuscles sensitive to ∼40-1000— Hz-encode hardness and have influenced haptic device design that attempts to replay high-frequency content at contact (4).

However, frequency tuning is most relevant at threshold, i.e., when detecting very small vibrations (5). Above threshold, the total energy in the vibration signal may be a more robust indicator of perceived hardness. Neuro-physiological and psychophysical studies show that afferent responses correlate with vibration amplitude and power across a broad frequency range, not with single frequencies (6–8).

Natural textures and materials generate broadband vibration “signatures,” with harder materials imparting greater high-frequency energy to the skin (1, 9, 10). Consistent with this view, Toscani and Metzger’s ViPer database shows that perceptual judgments of natural textures are closely related to the spectral statistics of the corresponding vibratory signals (11). These findings suggest that the tactile system may rely on the total spectral energy — the integrated vibratory power across —rather than specific frequency components, as a compact variable for judging hardness. Similar energy-integration strategies have been observed in other sensory systems, such as vision and audition (12, 13).

When part of this vibratory content is lost, such as when a finger is covered by a soft interface, or when interacting with compliant or damped surfaces, perceived hardness decreases (5, 14–16). Recent work with encountered-type haptic displays shows that the physical hardness of the device’s contact interface itself (e.g., a rigid plate versus a padded surface) systematically biases how stiff the same virtual object feels, consistent with the idea that attenuating high-frequency vibratory energy makes materials feel softer (17). Similarly, in stylus-based haptic augmented reality, contact-triggered vibrations delivered through the stylus can reliably bias judgments of how hard a surface feels (18). Across these studies, increasing vibration amplitude and high-frequency content can be understood as increasing the total spectral energy delivered to the skin. Together, these findings suggest that restoring this missing spectral energy in contact transients can recover, and even systematically alter, perceived hardness.

We hypothesize that by artificially reintroducing the missing spectral energy of contact transients we can restore or alter the perception of hardness. Using this principle, one can determine the transient energy needed for a given “virtual hardness” and inject that energy at contact to trick the sense of touch. This concept shifts the focus from precise waveform replication to matching the power spectral density (PSD: a measure of how vibration intensity is distributed across different frequencies) profile that a real material would generate.

This principle is directly applicable to prosthetics, where silicone padding can dampen natural vibrations (19), to teleoperation, where remote tools often dull tactile feedback (20), and to immersive virtual reality, where neither the user’s finger nor the simulated object provides natural high-frequency transients (4, 21). We tested this hypothesis across five experiments in two degraded scenarios: (i) a soft finger interface, where fingertip stiffness was artificially reduced, and (ii) a soft surface interface, where the object itself was compliant. Figure 1 summarizes these experimental conditions, designed to probe whether spectral energy alone is sufficient to recover hardness perception and material identity.

Figure 1.

To test this hypothesis, we designed five experiments in two degraded scenarios: (i) a soft finger interface, where fingertip stiffness was reduced, and (ii) a soft surface interface, where the object was compliant. These scenarios, common in prosthetics, teleoperation, and immersive VR, allowed us to ask whether restoring total spectral energy alone is sufficient to recover hardness perception and material identity (Fig. 1).

Results: Spectral Energy Compensation Enables Perception of Hardness and Material Identity

We investigated whether restoring high-frequency spectral energy in contact-evoked vibrations is sufficient to recover perception of hardness and material identity when natural mechanical cues are degraded. In all experiments, participants tapped either a rigid wood block or a compliant foam block with the index finger while a nail-mounted vibrotactile actuator (Lofelt L5) replayed brief (∼50 ms) vibration bursts that were time-locked to contact via a photoelectric sensor (trigger-to-output latency < 1 ms; see Fig. 9). Compensation signals were constructed in the frequency domain by subtracting the spectrum of the degraded condition (soft finger or soft surface) from that of a harder reference material and then band-limiting the result to 30–1000 Hz, selectively boosting the part of the power spectral density where hard and soft contacts differed (mainly at higher frequencies).

We tested two degraded-feedback paradigms: (i) contact with a hard surface using an artificially softened fingertip, achieved by mounting an inflatable silicone bubble over the fingerpad, and (ii) contact with a soft foam surface using a bare fingertip. In both paradigms, we asked whether adding these spectrally rich transients to the degraded contact could re-enable relative hardness discrimination (2AFC) and material identification (5-AFC).

Participants experienced three families of compensation signals: (1) data-driven waveforms recorded from real taps on the reference materials, (2) decaying sinusoidal bursts tuned to each material’s dominant frequency, and (3) broadband Ricker wavelet pulses. For each family, signals were matched in total spectral energy (RMS over 30–1000 Hz; Methods, Signal Calibration) so that spectral energy, rather than waveform shape, was the primary manipulated variable. Behavioral performance was summarized with confusion matrices and analyzed using repeated-measures ANOVAs with planned post hoc comparisons (see Methods for full details).

We investigated whether restoring high-frequency spectral energy in contact vibrations enables the perception of surface hardness, even when natural mechanical cues are degraded. Two experimental paradigms were used to test this hypothesis: (i) contact with a hard surface using artificially softened fingers, and (ii) contact with a soft surface using bare fingers. In both cases, we assessed whether brief, spectrally rich vibration bursts could re-enable relative hardness discrimination and material identification.

Participants completed tasks involving three types of compensation signals: (1) a data-driven waveform derived from real contact events, (2) a decaying 200 Hz sinusoid, and (3) a Ricker wavelet pulse. All signals were matched in total spectral energy (RMS-matched; Methods, Signal Calibration) to isolate the effect of spectral energy from waveform. Participants performed either forced-choice or matching judgments. Performance was analyzed using ANOVA and post hoc comparisons (for full details see Methods).

Compensating for Finger Softness: Restoring Hardness Perception through Spectral Energy

Experiment 1: Spectral vibration cues enable hardness discrimination with a soft fingertip

To simulate fingertip compliance relevant to prosthetic and wearable technologies, 10 participants wore an air-filled silicone bubble on each index finger, reducing finger stiffness, and a nail mounted vibrotactile actuator on each index finger. They tapped two identical rigid wood blocks with each index finger, where one of them randomly being delivered a vibration signal and the other silent, and judged which contact felt harder (two-alternative forced choice (2AFC) task). Vibration intensity varied across five discrete levels, each delivered using one of three waveform types (data-driven, decaying sinusoid, or Ricker wavelet).

Participants reliably judged contacts accompanied by higher spectral energy as harder (Fig. 2). Performance was near ceiling for most comparisons: Metal was judged harder than No signal, Foam, Silicone, and Rubber on 100% of trials, and harder than Wood on 85-90% of trials (depending on waveform). Similarly, Wood was consistently judged harder than all softer materials (100%). The only systematic confusions arose at the softest contrast: Silicone was judged harder than Foam on only 20-25% of trials (p > 0.99, binomial test). Rubber was judged harder than Foam on 75-85% of trials (p < 0.01), and reliably harder than Silicone and No signal (p≪ 0.001). These results establish that participants’ hardness judgments were overwhelmingly consistent with spectral energy differences, except at the smallest increment (Foam-Silicone).

Figure 2.

Binomial tests on the four adjacent contrasts confirmed this pattern. Foam-Silicone performance did not exceed chance (20-25%, p > 0.99), whereas Silicone-Rubber and Rubber-Wood were discriminated at or near ceiling (≥95%, all p < 10⁻⁶). Wood-Metal performance was strong but less than perfect: 90% (Data-driven, p < 10⁻⁵), 85% (Decaying Sine, p < 10⁻⁴), and 70% (Wavelet, p = 0.012).

A two-way repeated-measures ANOVA on per-subject accuracies confirmed a robust main effect of difficulty , reflecting the progressive increase in accuracy from Foam-Silicone to Wood-Metal. No main effect of waveform was observed (F (2, 18) = 1.91, p = 0.18), nor any interaction (F (6, 54) = 1.63, p = 0.16), indicating that all waveform types yielded the same difficulty-dependent pattern. One-way ANOVAs within each waveform similarly showed strong effects of difficulty (all p < 0.001). Post hoc paired comparisons confirmed that Foam-Silicone was significantly worse than all other contrasts (Holm-corrected p < 0.01).

To assess global discriminability, we pooled trials where each nominal hardness level was paired against all softer levels. A two-way repeated-measures ANOVA (hardness level x waveform) revealed a strong main effect of hardness level , but no effect of waveform (F (2, 18) = 1.84, p =0.19) and no interaction (F (8, 72) = 1.02, p = 0.43). Binomial tests showed ceiling-level discrimination for Metal and Wood against all softer levels (all p≪ 0.001), high performance for Rubber ( ≥95%), and near-chance performance only for Foam-Silicone. This confirms that hardness perception is preserved across all but the smallest spectral energy increment.

Together, these results demonstrate that high-frequency spectral energy is sufficient to restore hardness perception through a softened fingertip. The somatosensory system appears to prioritize total energy content over waveform details when interpreting tactile transients, allowing reliable discrimination except when energy differences are minimal.

Experiment 2: Synthetic vibration cues support material identity with a soft finger

We next asked whether spectral vibration cues could not only restore perceived hardness but also deliver material-specific qualities. 10 participants tapped a soft foam surface with an augmented vibration signal, then explored five reference materials, foam, silicone, rubber, wood, and metal-using their unaltered index finger. The materials increased in stiffness from foam (E≈ 0.01 GPa) to metal (E≈ 200 GPa). The task was to match the augmented foam tap to the closest material.

Participants’ choices aligned strongly with spectral energy (Fig. 3). The no-vibration condition was almost always matched to foam, intermediate signals to silicone or rubber, and the strongest signals to wood or metal. Confusion matrices confirmed a near-diagonal structure with only nearest-neighbor errors at the soft and mid-levels. Overall, 67% of responses were exact matches, and 95% were within ±1 material rank of the intended target (p < 0.001, binomial), indicating highly systematic scaling. Ordinal correlation between signal energy and perceived material rank was strong across all waveforms (r = 0.82 − 0.84). Mutual information analysis showed that the vibration cues carried 0.63-0.69 bits of categorical information (out of a theoretical maximum of 2.32 bits), indicating substantial transmission of material identity through synthetic vibrations.

Figure 3.

A two-way ANOVA (signal level x waveform) on response ranks revealed a robust main effect of signal level (F (4, 749) = 177.23, p < 0.001, η² = 0.76), no main effect of waveform (F (2, 749) = 2.34, p = 0.158), and a minor but significant interaction (F (8, 749) = 3.55, p < 0.001, η² = 0.009). Tukey post hoc tests confirmed that each increase in vibration level produced significantly harder perceived materials (all adjacent contrasts p < 0.01), with the exception that the two highest levels (wood vs. metal signals) were not reliably different (p = 0.12).This indicates that perceptual scaling was systematic but compressed at the upper end of the stiffness range.

These findings indicate that synthetic vibration cues not only restored a sense of hardness but also conveyed material-specific identity. Even when fingertip mechanics were altered to simulate soft robotic contact, vibration energy was sufficient to drive perceptual categorization along the natural continuum of material stiffness.

Compensating for Soft Surfaces: Vibration Signals Restore Hardness Perception

Experiment 3: Graded hardness perception on a soft surface via spectral vibration cues

Here, we reversed the scenario: 10 participants used bare fingertips to tap two physically identical soft foam blocks, one of which delivered a vibration burst (2AFC). Five energy levels and three waveform types (Data-driven, Decaying Sine, Wavelet) were tested.

Participants reliably judged contacts accompanied by higher spectral energy as harder (Fig. 4). Performance was near ceiling for most comparisons: Metal was judged harder than No signal, Silicone, and Rubber on 100% of trials, and harder than Wood on 75-80% of trials depending on waveform. Wood was consistently judged harder than No signal and Silicone (100%), and harder than Rubber on 85-95% of trials. Rubber was judged harder than No signal and Silicone on 95-100% of trials, and Silicone was always judged harder than No signal. These results establish that hardness judgments closely tracked spectral energy differences across the foam substrate, with the only systematic errors arising in the Wood-Metal contrast.

Figure 4.

Binomial tests on the four adjacent contrasts confirmed this pattern. No signal-Silicone performance was at ceiling (100%, p = 8.3×10⁻²⁵ across all waveforms). Silicone-Rubber and Rubber-Wood were discriminated at 92.5-100% (p ≤ 2.7 × 10⁻²¹). Wood-Metal performance was strong but less than perfect: 87.5% (Data-driven, p = 1.6 × 10⁻¹²), 90% (Decaying Sine, p = 2.7 × 10⁻¹⁴), and 85% (Wavelet, p = 6.0 × 10⁻¹¹).

A two-way repeated-measures ANOVA on per-subject accuracies showed a trend-level main effect of difficulty, no main effect of waveform (F (2, 18) = 0.11, p = 0.893), and no interaction (F (6, 54) = 1.83, p = 0.111). One-way ANOVAs within each waveform showed a significant effect of difficulty for Wavelet (F (3, 27) = 3.55, p = 0.028), and trend-level effects for Data-driven (F (3, 27) = 2.47, p =0.083) and Decaying Sine (F (3, 27) = 2.00, p = 0.138). Thus, although the across-subject variability reduced the omnibus effect, all contrasts were significantly above chance, though Wood-Metal fell below ceiling.

To assess global discriminability in Experiment 3, we again pooled trials where each nominal hardness level was paired against all softer levels. The omnibus ANOVA showed only a trend-level effect of hardness level (F (3, 27) = 2.72, p = 0.064), with no main effect of waveform or interaction (F (2, 18) = 0.046, p = 0.955; F (6, 54) = 1.87, p = 0.102). However, pooled binomial tests corroborated the monotonic scaling, with ceiling performance for Silicone (100%, p = 5.66 × 10⁻⁷³) and near-ceiling for Rubber (99.2%, p = 7.06 × 10⁻¹³⁶), Wood (98.3%, p = 6.81 × 10⁻¹⁹²), and Metal (96.5%, p = 4.99 × 10⁻²²⁷).

Together, these results mirror Experiment 1, demonstrating that the somatosensory system prioritizes vibratory spectral energy over substrate stiffness when inferring hardness. Even on a uniformly soft foam surface, participants’ judgments followed the delivered signal’s spectral energy, confirming that tactile transients can override mechanical compliance cues.

Experiment 4: Material identification on a soft substrate via synthetic vibration

In Experiment 2 we showed that spectral vibration cues on a foam surface could convey material identity with high fidelity. Here we asked whether the same cues remain effective when both the surface and the fingertip are compliant, by attaching a soft bubble to the finger. 10 participants tapped the foam block, received one of four vibration signals, and matched the percept to one of five reference materials.

Perceived material identity continued to scale with vibration intensity (Fig. 5). Low-energy signals were judged as foam or silicone, mid-levels as silicone or rubber, and high-energy signals as wood or metal. Confusion matrices confirmed a strong diagonal structure, with errors predominantly restricted to nearest-neighbor categories. Overall, 52% of responses were exact matches and 94% were within ±1 material rank of the intended target (p < 0.001). Ordinal correlations between signal level and perceived material rank remained significant across all waveform types (r = 0.62 − 0.65), though weaker than in Experiment 2 (r ≈ 0.83). Mutual information analysis showed that the vibration cues carried 0.29-0.31 bits of categorical information, roughly half of the information transmitted in Experiment 2, indicating reduced precision under dual compliance.

Figure 5.

A two-way ANOVA (signal level x waveform) confirmed a robust main effect of signal level (F (3, 599) = 642.01, p < 0.001, η² = 0.55), but no effect of waveform type (F (2, 599) = 2.18, p = 0.194) and no interaction (F (6, 599) = 0.38, p = 0.89). Tukey post hoc tests showed that each increase in signal level produced a significant shift toward harder perceived materials (all adjacent contrasts p < 0.01), except for the highest comparison (wood vs. metal), which was not consistently discriminated (p = 0.21). This confirms that perceptual scaling remained systematic but compressed at the upper end of the stiffness range.

Together, these results demonstrate that while dual compliance reduces categorical precision, spectral vibration cues continue to systematically drive material identity judgments.

Experiment 5: Perceived Material Identity Is Dominated by Rendered Spectral Energy

To directly test whether the somatosensory system prioritizes total spectral energy over frequency when inferring material properties, we designed stimuli for which these cues were intentionally put in conflict, e.g., combining the RMS amplitude of foam with the frequency characteristics of wood. This allowed us to dissociate the contribution of each cue and determine the behaviorally relevant material feature. We conducted a material classification task with a reduced stimulus set (foam, rubber, wood) comprised of such cue conflict stimuli. On each trial, participants tapped a physical foam block with a bare finger while receiving a vibration signal - corresponding to the RMS of one of the three materials and a base frequency of one of the other materials - delivered to the dorsal side of the fingertip. This setup created congruent and incongruent combinations of base and rendered cues.

Participants classified the perceived material from among the three options. Confusion matrices (aggregated across 10 participants, five repetitions per condition) revealed that judgments overwhelmingly aligned with the rendered vibration rather than the actual material of contact (Fig. 6).

Figure 6.

Figure 7.

Figure 8.

A binomial logistic regression confirmed a significant effect of the rendered signal. In particular, the “wood” vibration strongly influenced judgments (β = 1.015, p < 0.001; odds ratio = 2.76, 95% CI [1.70, 4.48]), while “rubber” showed a positive trend (β = 0.407, p = 0.082). By contrast, the physical base material had no significant effect.

Model comparison further supported this interpretation. Including interaction terms between base frequency and RMS rendered materials significantly improved fit (AIC = 572.99 vs. 592.41; likelihood ratio test 𝒳²(4) = 27.42, p < 0.001). Nonetheless, the rendered spectral energy RMS component remained the dominant determinant of perceived material identity.

These results confirm that, when presented with conflicting cues, the tactile system relies more heavily on the spectral characteristics of vibratory input than on the fundamental frequency response of the contacted surface. Total spectral energy - not frequency-specific patterns or physical compliance - drives material classification under ambiguity.

Discussion

Across five experiments, we demonstrate that high-frequency spectral energy serves as a robust primitive or cue for conveying both surface hardness and material identity. These effects are consistent across both finger-compensation and surface-compensation paradigms. Simple synthetic waveforms, such as decaying sinusoids and wavelets, are perceptually equivalent to complex real contact transients when matched for spectral energy. Furthermore, under conflicting cues, the tactile system prioritizes spectral energy over fundamental frequency or compliance in material classification.

Together these findings highlight a core organizing principle in tactile perception: the somatosensory system primarily interprets high-frequency spectral content - rather than detailed frequency composition or contact mechanics - to infer material class and surface hardness. This insight enables principled design of haptic rendering techniques for prosthetics, virtual reality, and robotics, where physical constraints limit the availability of natural contact cues.

Our findings demonstrate that matching the spectral energy of contact vibrations can robustly convey perceptions of material hardness, even under conditions where mechanical cues are degraded or misleading. Across five experiments, we show that participants reliably perceived gradations in hardness - and, in many cases, could identify specific materials - based solely on vibration cues, despite interacting with physically soft or compliant surfaces.

Crucially, this perceptual reconstruction occurred regardless of the waveform’s temporal structure - whether it was a natural recording, a sinusoidal burst, or a wavelet pulse - as long as the spectral energy matched the corresponding hardness level. These results provide strong evidence that the human tactile system prioritizes spectral energy over precise waveform features as a core cue in inferring hardness.

These insights extend current models of tactile perception. Classical frameworks emphasize force-displacement dynamics or sustained pressure, typically mediated by slowly adapting mechanoreceptors, as primary cues for stiffness and compliance (22). However, in dynamic touch contexts such as tapping, brief high-frequency transients dominate the tactile signal (7), suggesting a distinct perceptual mechanism. Our results support the hypothesis that dynamic spectral energy - particularly within the 100–300 Hz band - serves as a proxy for mechanical hardness, interpreted by the nervous system even in the absence of congruent proprioceptive or mechanical feedback.

One of the most striking findings was the perceptual reconstruction exhibited by participants: despite interacting with a physically softer interface, brief high-frequency vibrations were sufficient to evoke the illusion of a hard material. This suggests that the tactile system engages in cue integration, wherein salient high-frequency components can override - or be weighted more heavily than - conflicting proprioceptive or mechanical signals, particularly during transient contact events. The finding that minimal synthetic bursts could still support material recognition underscores the robustness and adaptability of the somatosensory system.

From an engineering standpoint, this perceptual flexibility presents a valuable design opportunity. Rather than relying on complex force-feedback systems - which are often unstable, costly, or difficult to miniaturize - haptic devices can instead leverage computationally efficient, precomputed vibration signals to simulate hardness percepts. In tandem with existing physics engines, rendering can additionally leverage the spectral energy cues that would be computed as a function of a finger interacting with a given material. Importantly, the perceptual fidelity required for realism appears to depend more on accurate spectral energy matching and precise temporal alignment with contact onset, rather than on detailed modeling of mechanical interactions or force dynamics (23, 24). Timing of these haptic feedback cues is critical given the millisecond encoding precision of the AC-coupled Pacinian corpuscles, tuned to transient impacts and their spectral content (25, 26). Accounting for these dynamics dramatically increases skin sensing capabilities as Rongala et al. demonstrate in their neural simulation work (26). Follow-up investigations aimed at quantifying the perceptual limits of these timing constraints could establish concrete design tolerances for haptic systems. Identifying the temporal resolution required for reliable material discrimination would not only refine rendering strategies, but also inform hardware specifications for next-generation tactile interfaces. Future work characterizing the perceptual bounds of these temporal dynamics would provide principled constraints for tactile display design. By mapping the thresholds at which temporal imprecision degrades material percepts, such studies could bridge neural encoding models and practical engineering implementations.

Our results revealed a behaviorally relevant cue space with both shared and distinct components underlying surface hardness perception and material identification. Perceived hardness - our first-order effect - was systematically modulated by spectral energy: higher energy reliably increased perceived hardness ratings across all experiments. In contrast, second-order effects such as material identification were sensitive to the spectral shape. While hardness scaled with energy, subtle differences in spectral shape also contributed to material identity. These findings suggest that haptic systems may flexibly support both coarse hardness categorization (e.g., “hard” vs. “soft”) and fine-grained material-specific rendering, depending on the perceptual demands of the application.

These findings challenge the notion that the somatosensory system operates as a frequency-filter bank akin to the cochlear model in audition. Instead, our data suggest that tactile perception relies on a more integrative, task-dependent extraction of spectral features. In the context of dynamic touch interactions - which generate vibrations that propagate both locally and distally - our findings argue for a conceptual revision of which vibration features are likely recruited by the somatosensory system to reconstruct material properties.

In practical terms, this leads to several design implications:

• Latency is critical. In all experiments, vibrations were triggered within ∼5 ms of contact, aligning temporally with the skin’s mechanical response. Delays could degrade the realism or effectiveness of the illusion.

• Calibration is tractable. We demonstrated that participants can reliably map a vibration signal to a material, implying that users could calibrate their own haptic devices via interactive matching.

• Personalization may help. Participants differed slightly in sensitivity to vibration levels, suggesting future systems could learn user-specific thresholds or preferences.

Of course, our approach has limitations. It focuses on transient, tapping-based hardness perception, not continuous deformation or squeezing, which rely on different cues (e.g., force gradients, skin stretch). In real-world manipulation, hardness perception is multimodal and dynamic, so vibratory cues should be seen as augmentative, not complete. Still, in many practical contexts - such as teleoperation, prosthetics, or VR/AR interfaces - precise mechanical feedback is unavailable. Here, our spectral compensation method provides a compelling alternative.

In summary, this study introduces and validates a spectral energy compensation framework for rendering material hardness through high-frequency contact transients. By controlling the total spectral power, regardless of waveform, we were able to restore or simulate tactile hardness across different degraded interaction conditions. These findings establish high-frequency spectral energy as a reliable proxy for stiffness, supporting both graded hardness discrimination and material identification in the absence of realistic force cues.

This principle offers a path forward for lightweight, low-power haptic systems that rely on transient overlays instead of complex actuators. By aligning artificial transients with the perceptual mechanisms of the skin, haptic experiences can be made both intuitive and adaptable to diverse applications, from robotic touch to prosthetic feedback to immersive virtual environments.

Materials and Methods

Overview

We conducted five experiments to evaluate how spectral energy cues in vibration signals contribute to material perception, particularly in the encoding of hardness via touch. These experiments manipulated either fingertip compliance or surface properties and employed three signal-generation techniques. All experiments used controlled tapping procedures, precise haptic playback, and randomized trial designs.

Participants

Participants (Experiments 1–4)

Forty adults (22–52 years; 13 female, 27 male; 39 right-handed, 1 left-handed) were recruited from the local university/community and compensated [$X/hour or course credit]. Ten participants took part in each of Experiments 1–4 [state whether distinct or overlapping across experiments]. All reported normal tactile sensitivity and no history of neurological or musculoskeletal disorders affecting the hands. Prior experience with haptics was self-reported as: none (2), limited (16), moderate (13), extensive (9). All provided written informed consent under a protocol approved by the Reality Labs Research ethics committee under the AGOI protocol).

Participants (Experiment 5)

A separate cohort of ten adults ([age range/mean]; [# female/male]; [handedness]) completed Experiment 5; none participated in Experiments 1–4. Inclusion/exclusion criteria and compensation matched those of Experiments 1–4. All procedures were approved under the AGOI protocol).

Apparatus

Vibrations were delivered via Lofelt L5 actuators (30-1000 Hz) affixed to the fingernail. Stimuli were triggered by photoelectric sensors and played back on a Bela board (trigger-to-output latency ¡1 ms). Fingertip compliance was manipulated using an air-filled silicone bubble inflated to 5 psi and secured over the index fingerpad. Participants were blindfolded during tasks to eliminate visual cues. To eliminate auditory cues, participants wore over-ear headphones playing continuous pink noise at 65-70 dB SPL throughout all tasks. (See Supplementary Methods for full hardware specifications.)

Material Selection and Vibration Recording

We recorded vibration transients from taps on 10 everyday materials using a tri-axial accelerometer affixed to the fingernail. Recordings were sampled at high resolution and later processed to derive spectral energy profiles (see Supplementary Methods for full apparatus details).

Signal Generation

Three signal types were tested: 1. Data-driven: recorded transients replayed directly. 2. Decaying sinusoid: exponentially decaying bursts capturing resonant modes. 3. Ricker wavelet: broadband pulses simulating impact transients. All signals were RMS-normalized and matched in duration. Equations and parameter values are provided in Supplementary Methods.

Experimental Procedures

Participants completed five experiments: - Exp. 1: Soft finger discrimination - judged which of two wood contacts felt harder (2AFC). - Exp. 2: Soft finger matching - matched augmented foam taps to real materials. - Exp. 3: Soft surface discrimination - judged hardness between two foam contacts. - Exp. 4: Soft surface matching - matched augmented foam taps to real materials. - Exp. 5: Cue conflict classification - judged material identity when spectral energy and frequency cues conflicted. Trial counts, randomization, and timing are detailed in Supplementary Methods.

Signal Calibration

Stimuli were RMS-normalized across methods to ensure that differences in perception reflected spectral energy, not raw amplitude. A pilot study confirmed matched perceptual intensity.

User Studies

Participants completed five experiments grouped into two categories: fingertip modification (Experiments 1-2) and surface modification (Experiments 3-5). All tasks involved either discrimination (2AFC) or material matching, with randomized trial sequences and practice sessions. Fingers were wrapped with tape to minimize confounding cues, and participants were blindfolded to eliminate visual input.

Experiments 1-2 (Fingertip modification)

To simulate reduced fingertip stiffness, participants wore an inflatable bubble actuator on the index finger. In Experiment 1, they tapped identical wood blocks (one with vibration playback) and judged which felt harder. In Experiment 2, they tapped a foam block with vibration and matched it to one of five real reference materials (foam, silicone, rubber, wood, metal).

Experiments 3-4 (Surface modification)

Here, participants tapped soft foam surfaces without fingertip modification. Experiment 3 used a discrimination task (2AFC), while Experiment 4 used the same material-matching task as in Experiment 2.

Experiment 5 (Cue conflict)

A separate cohort of 10 participants completed a classification task in which spectral energy (RMS) and base frequency cues were dissociated. Participants tapped foam with synthetic vibrations designed to conflict in energy vs. frequency signatures, and judged which material was perceived.

Across all studies, vibration signals were delivered within 1 ms of contact, and task structure ensured double-blind conditions. Full trial counts, durations, and detailed procedures are reported in the Supplementary Methods.

Data availability

Data will be uploaded to an Open Science repository upon acceptance.

Supplementary Methods

Apparatus Details

Vibration feedback was delivered using Lofelt L5 actuators (30–1000 Hz) affixed to the fingernail of each index finger using double-sided tape and wax. Actuator placement was informed by Birznieks et al. research that demonstrated that mechanoreceptors around the fingernail borders reliably respond to forces applied to the finger pad (22). A Bela board running a real-time Linux OS (Xenomai) ensured low-latency playback (< 1 ms). Finger presence was detected via Panasonic CX-400 photoelectric sensors positioned directly above the contact surface. Tactile stimuli were played back using one of three signal-generation methods (described below). The soft bubble actuator used to manipulate fingertip compliance was manually inflated to and maintained at 5 psi throughout the experiment. The bubble was secured with elastic tape to the fingerpad of the participant’s index fingertip. Participants were blindfolded during all tasks and wore lightly wrapped tape on their index fingers to minimize non-vibratory tactile cues.

Material Selection and Frequency-Domain Clustering

Vibration Recording

To characterize the vibratory responses of everyday materials, we recorded transients from taps on 20 samples, including natural and synthetic surfaces (see Table S1). Vibrations were measured using a tri-axial piezoelectric accelerometer (Endevco Model 35A, Endevco Corp., NY, USA; sensitivity 10 mV/g, resonance frequency 30 kHz), affixed to the participant’s index fingernail with orthodontic wax to ensure stable coupling. Signals were conditioned using an Endevco 133 signal conditioner and digitized at 2000 Hz via a National Instruments NI-9205 data acquisition module. Each material was tapped five times at a controlled pace (90 BPM metronome), and signals were stored for subsequent analysis and playback.

Fingertip Compliance Interface (Bubble Actuator)

To simulate changes in fingertip stiffness, we fabricated a soft inflatable “bubble” actuator from thermoplastic elastomer. The actuator was secured over the index fingerpad with elastic cloth tape and inflated to 5 psi using a hand pump with inline pressure gauge. This setup reduced effective fingertip stiffness while leaving the fingernail accessible for actuator mounting. Although vibration recordings were collected for all materials with the bubble attached, only the bubble-on-wood condition was used in the experiments. Wood was selected as a mid-range reference material to simplify comparisons across signal types.

Material Grouping and Selection

Spectral analysis was used to identify representative materials for the main experiments. For each sample, the FFT magnitude spectrum was computed, and spectral energy was summed into 50 Hz bands from 0-1000 Hz. Materials were grouped using k-means clustering (k = 4) based on spectral centroid and broadband energy. Pilot perceptual testing with blindfolded participants confirmed that hard materials (e.g., metal, wood, glass, rock) tended to be perceptually similar, while soft materials (e.g., foam, silicone, rubber) were easily distinguished. Based on both spectral and perceptual clustering, we selected five representative materials spanning the stiffness range: foam, silicone, rubber, wood, and metal. These five formed the stimulus set for the main user studies.

Supplementary Methods: Signal Processing and Implementation

Signal Extraction and Windowing

Raw vibration recordings were collected from taps on each material at a metronome-controlled pace (90 BPM) to ensure consistent impact velocity. For each material, the tap with median peak acceleration was selected for further processing. A 20 ms analysis window (5 ms before peak to 15 ms after peak) was extracted to capture the transient response. To reduce onset artifacts and spectral leakage, the signal was zero-padded and tapered with a 5 ms Hanning window.

Compensation Signal Design

To simulate changes in interaction properties (e.g., compliant surfaces or softened fingertips), frequency-domain subtraction was applied to construct compensation signals. Specifically:

where S_target(f) is the FFT of the harder reference material (e.g., wood or metal) and S_reference(f) is the FFT of the baseline condition (e.g., foam or bubble-modified finger). The inverse FFT of C(f) was computed to generate time-domain compensation signals. This approach effectively overlays the missing spectral components of a harder material onto the degraded baseline condition.

Signal Normalization and Upsampling

All signals were RMS-normalized to a target amplitude of 0.35 G to ensure perceptual comparability across materials and signal-generation methods. Signals were then upsampled from the acquisition rate (2000 Hz) to the playback rate (44.1 kHz) using a windowed-sinc interpolation filter. Final signals were saved in 16-bit mono WAV format for playback.

Real-Time Signal Generation

We used three approaches to synthesize material-specific vibration cues:

• Data-driven Model This approach captured real acceleration signals from finger-material interactions and replayed them directly, offering high realism (4, 15, 16).

• Decaying Sinusoidal Model Signals were synthesized as

  

  where A₀ is the initial amplitude, τ the decay time constant, f the carrier frequency, and ϕ the phase. To minimize onset/offset artifacts, each signal was multiplied by a Gaussian window (∼ 50 ms total duration). Parameters (τ, f) were chosen to approximate recorded spectra for each material (e.g., foam: τ = 5 ms, f = 85 Hz; metal: τ = 20 ms, f = 400 Hz).

• Ricker Wavelet Model The Ricker wavelet was defined as:

  

  Peak frequencies ωp were chosen to approximate dominant spectral components of each material ( (27, 28)). The duration of each waveform was fixed at 50 ms with zero-padding.

All signals were within trigger-to-output latency of < 1 ms of contact detection (measured with oscilloscope).

Playback and Actuation

Photoelectric sensors positioned above the tapping surface detected finger contact events. Sensor triggers initiated signal playback on a Bela board running Xenomai real-time Linux. Signals were routed through a TPA3116D2 Class D amplifier and delivered to Lofelt L5 actuators affixed to participants’ fingernails. This ensured both low-latency delivery and high-fidelity reproduction of vibration cues. Signals were band-limited to 30-1000 Hz (8th-order zero-phase Butterworth). Peak acceleration did not exceed 1.2 G and remained below published discomfort thresholds for fingertip vibrotactile stimulation. Participants could request breaks at any time; no adverse events were reported.

Experimental Procedures

Participants’ index fingers were lightly wrapped with tape to reduce confounding cues (e.g., temperature, skin texture). All studies used the same actuation and triggering pipeline (latency < 1 ms) and were run single-blind with randomized trial order. Unless otherwise noted, participants tapped at a natural pace (“as if quickly checking a surface”). We matched RMS acceleration per stimulus (equivalently, total spectral energy integrated over 30-1000 Hz via Parseval’s theorem) and refer to this matched quantity as total spectral energy (RMS-matched) (RMS matching procedure and values are detailed in Supplementary Methods, Signal Calibration).

Fingertip modification experiments (Experiments 1-2)

Experiment 1: Hardness discrimination with softened fingertip (2AFC)

• Goal. Test whether added spectral energy restores relative hardness judgments when fingertip stiffness is reduced.

• Participants & setup. Ten participants wore an inflatable bubble actuator on each index fingerpad (5 psi). Both hands tapped identical rigid wood blocks; one block delivered a vibration transient at contact, the other was silent.

• Stimuli. A 3 × 5 set: three waveform types (data-driven, decaying sinusoid, Ricker wavelet) crossed with five RMS-matched energy levels (No-signal control; foam-, silicone-, rubber-, wood-/metal-like total energy). Signals were brief (∼50 ms), aligned to contact onset.

• Task & design. 2AFC (“Which tap felt harder, left or right?”). For each waveform, all ordered pairs of the five levels that differed in energy were tested once per hand (20 pairwise comparisons per waveform), then repeated once (2 reps/hand). Total per participant: 60 comparisons, randomized. Session duration: 30-45 min.

• Planned analysis. Accuracy per contrast; binomial tests versus chance for adjacent pairs; repeated-measures ANOVA on accuracy with factors difficulty (adjacent pair) and waveform; confusion matrices for descriptive summaries.

Experiment 2: Material matching with softened fingertip (5-AFC)

• Goal. Test whether spectral energy alone conveys material identity when fingertip mechanics are softened.

• Participants & setup. Ten participants, bubble actuator on the non-dominant index finger (taps a foam block with vibration); dominant index finger explores five real reference materials (foam, silicone, rubber, wood, metal) without vibration.

• Stimuli. Fifteen signals (3 waveforms × 5 RMS-matched energy levels), one transient per trial.

• Task & design. 5-alternative forced choice (“Which real material best matches the rendered vibration?”). Each of the 15 signals presented 5 times (75 trials), randomized, after one practice trial. Duration: 30-60 min.

• Planned analysis. Confusion matrix; ordinal (Spearman) correlation between energy level and chosen material rank; mutual information; two-way ANOVA (factors: energy level, waveform) on response rank; Tukey post hocs for adjacent levels.

Surface modification experiments (Experiments 3-4)

Experiment 3: Hardness discrimination on a soft surface (2AFC)

• Goal. Test whether added spectral energy can override a uniformly soft substrate in relative hardness judgments.

• Participants & setup. Ten participants, bare fingertips, two identical foam blocks; one block delivered vibration, the other was silent.

• Stimuli. Same 3 × 5 set as Experiment 1 (three waveforms × five RMS-matched energy levels), transient aligned to contact.

• Task & design. 2AFC (“Which tap felt harder, left or right?”). Total: 60 pairwise comparisons per participant (as in Exp. 1), randomized. Duration: 30-45 min.

• Planned analysis. As in Exp. 1 (binomial tests for adjacent contrasts; rmANOVA on accuracy with difficulty and waveform; confusion matrices).

Experiment 4: Material matching on a soft surface (5-AFC)

• Goal. Test whether spectral energy conveys material identity when the touched surface is compliant.

• Participants & setup. Ten participants, bare fingertip taps a foam block with vibration; dominant hand explores the five reference materials.

• Stimuli. Twelve signals (3 waveforms × 4 RMS-matched energy levels spanning foam→metal; highest two levels can be collapsed if needed based on calibration range).

• Task & design. 5-alternative forced choice. Each signal repeated 5 times (60 trials), randomized, after one practice trial. Duration: 30-60 min.

• Planned analysis. As in Exp. 2 (confusion matrix, ordinal correlation, mutual information, two-way ANOVA with energy level and waveform).

Experiment 5: Cue-conflict (surface compensation) paradigm

• Goal. To test whether perceived material identity is dominated by total spectral energy (RMS) or by dominant frequency, we dissociated these cues in rendered vibrations delivered while tapping a soft substrate.

• Participants and setup. Ten participants tapped a physical foam block with a bare finger. Vibrotactile playback was delivered to the fingernail via the Lofelt L5 actuator (latency < 1 ms), as in prior experiments.

• Stimulus construction. We created a 3 × 3 factorial set by crossing three RMS targets with three dominant-frequency classes, yielding 9 stimuli (3 congruent, 6 incongruent):

  • RMS (“energy”) targets: values matched to the band-limited (30-1000 Hz) RMS acceleration measured from our data-driven recordings for foam, rubber, and wood. RMS was computed in the frequency domain and verified in time domain (Parseval’s theorem).

  • Dominant-frequency classes: peak frequencies characteristic of the same three materials, grouped as ≈ 85 Hz (foam-like), ≈ 180 Hz (rubber-like), and 350-400 Hz (wood-like).

Each stimulus was an exponentially decaying sinusoid, windowed to a brief transient and aligned to contact onset:

with f∈ {85, 180, 350-400} Hz, w(t) a Gaussian/Hann onset window (total duration ∼50 ms), and τ chosen to match the decay constant of recorded transients. After generating s(t) at the desired f, we scaled the time-domain amplitude so that the playback signal’s band-limited RMS (30-1000 Hz) matched the assigned energy target. We refer to this matched quantity as total spectral energy (RMS-matched).

Congruent conditions paired foam-RMS with 85 Hz, rubber-RMS with 180 Hz, and wood-RMS with 350-400 Hz.Incongruent conditions crossed each RMS with the two non-matching frequency classes.

• Task and design. On each trial, participants tapped the foam surface once and received one of the nine stimuli at contact. They then made a 3-alternative forced choice (3AFC): foam, rubber, or wood. Each of the 9 conditions was presented 5 times (45 trials total, 10-15 minutes), in randomized order, following one practice trial. Sessions were conducted single-blind.

• Planned analysis. Primary outcome was the chosen category per trial. We fit a multinomial (or separate binomial) mixed-effects logistic regression with fixed effects ofRMS target and frequency class(and their interaction), and random intercepts for participant. Confusion matrices and energy-rank correlations were computed as descriptive summaries.

Calibration and Control

Energy matching

We matched RMS acceleration per stimulus (equivalently, the total spectral energy integrated over 30-1000 Hz by Parseval’s theorem). We refer to this quantity as “total spectral energy (RMS-matched).” Durations and windowing were held constant across waveforms.

Amplitude Calibration

RMS amplitude was standardized across materials and methods to ensure that perceived intensity did not confound perceived hardness. A pilot perceptual matching task (n = 4) confirmed that amplitude-normalized waveforms were not systematically perceived as more or less intense. Calibration values are provided in Table S2.

Control Signals

Three “null” signals were included for each generation method:

1. A flat-zero waveform (no vibration)

2. A low-amplitude noise burst (sub-threshold)

3. A neutral sine pulse (non-material specific)

These were used to evaluate baseline response rates and exclude participants with response bias.

Trial randomization and data collection

Each trial began when the participant tapped the assigned contact surface (foam or wood) with a prepared index finger. Upon sensor trigger, a vibration cue was delivered within < 1 ms. Participants were free to alternate between stimuli (left/right or physical/rendered) before responding via a button box.

Stimulus order was randomized within participant. For each block, left/right assignment, waveform type, and energy level were independently randomized without replacement, ensuring equal counts per condition. Block order (discrimination vs. matching) was counterbalanced across participants using a balanced Latin square. Sessions were conducted single-blind: participants were unaware of condition and the experimenter did not see trial assignments during data collection.

We did not constrain impact force; participants were instructed to “tap as if quickly checking a surface.” Post hoc checks did not reveal systematic differences in tap intensity across conditions, so no force normalization was applied.

Data Collection Interface

Trials were managed via a custom Python interface using PyDAQmx and PyQt5, with real-time communication to the Bela board over UDP. Participant responses and timing logs were recorded with 1 ms resolution.

Additional Notes and Figures

• Additional setup and equipment photos: Figures 10a, 10b, 10c

• Foam was chosen as the compensation surface due to its low hardness and consistent tactile properties.

• The actuator on the dominant hand during matching tasks served only as a mass balancer.

Figure 9.

Figure 10.

Acknowledgements

This work was funded by Reality Labs Research. The authors declare no competing interests.

Additional information

Funding

Meta (FB)

• Sonny Chan

Meta (FB)

• Roham Padakhtim

Meta (FB)

• Chase Tymms

Meta (FB)

• Benjamin Stephens-Fripp

Meta (FB)

• Jess Hartcher-O'Brien

Meta (FB)

• Nagmeh Zamani