Yuri Mikawa's Website | Softness Estimation Based on <br>Imaging of Pushing Action Using Deep Learning

The hardness of an object is measured by a contact sensor, but there are cases where contact is not desired due to hygienic problems such as food. Therefore, in this research, we propose a method to estimate the hardness of an object from the image of the pushing motion of the target object through the object that has no hygienic problem. In this study, we assumed food as the target object and made an estimation using the pressing motion through chopsticks. Human skin gels of various hardness shown in Fig. 2 were used as flexible objects. Using 30fsp color and depth images (shown in Fig. 1) of the appearance where a subject is pushing the target object (Fig. 3), we created a deep-learned model in association with the hardness data, and confirmed that the hardness can be estimated appropriately for the test data.

Fig. 1: Color and depth images used for deep learning.

Fig. 2: Human skin gel used as the target object, and each hardness value measured with a durometer.	Fig. 3: Experimental environment.

Yuri Mikawa，Yasutoshi Makino，Hiroyuki Shinoda: Softness Estimation Based on Imaging of Pushing Action Using Deep Learning, Transactions of the Virtual Reality Society of Japan, Vol.23, No. 4, pp.239-248 (2018) (Japanese)