中文版 | English
Title

Visual-tactile Sensing for Real-time Liquid Volume Estimation in Grasping

Author
DOI
Publication Years
2022
Conference Name
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
ISSN
2153-0858
ISBN
978-1-6654-7928-8
Source Title
Pages
12542-12549
Conference Date
23-27 Oct. 2022
Conference Place
Kyoto, Japan
Publication Place
345 E 47TH ST, NEW YORK, NY 10017 USA
Publisher
Abstract
We propose a deep visuo-tactile model for real-time estimation of the liquid inside a deformable container in a proprioceptive way. We fuse two sensory modalities, i.e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor without any extra sensor calibrations. The robotic system is well controlled and adjusted based on the estimation model in real time. The main contributions and novelties of our work are listed as follows: 1) Explore a proprioceptive way for liquid volume estimation by developing an end-to-end predictive model with multi-modal convolutional networks, which achieve a high precision with an error of similar to 2 ml in the experimental validation. 2) Propose a multi-task learning architecture which comprehensively considers the losses from both classification and regression tasks, and comparatively evaluate the performance of each variant on the collected data and actual robotic platform. 3) Utilize the proprioceptive robotic system to accurately serve and control the requested volume of liquid, which is continuously flowing into a deformable container in real time. 4) Adaptively adjust the grasping plan to achieve more stable grasping and manipulation according to the real-time liquid volume prediction.
Keywords
SUSTech Authorship
Others
Language
English
URL[Source Record]
Indexed By
WOS Research Area
Automation & Control Systems ; Computer Science ; Engineering ; Robotics
WOS Subject
Automation & Control Systems ; Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Robotics
WOS Accession No
WOS:000909405303133
Data Source
IEEE
PDF urlhttps://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9981153
Citation statistics
Cited Times [WOS]:0
Document TypeConference paper
Identifierhttp://kc.sustech.edu.cn/handle/2SGJ60CL/420617
DepartmentDepartment of Mechanical and Energy Engineering
Affiliation
1.Department of Computer Science, The University of Hong Kong
2.Department of Biomedical Engineering, City University of Hong Kong
3.Department of Mechanical and Energy Engineering, Southern University of Science and Technology
Recommended Citation
GB/T 7714
Fan Zhu,Ruixing Jia,Lei Yang,et al. Visual-tactile Sensing for Real-time Liquid Volume Estimation in Grasping[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2022:12542-12549.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Export to Excel
Export to Csv
Altmetrics Score
Google Scholar
Similar articles in Google Scholar
[Fan Zhu]'s Articles
[Ruixing Jia]'s Articles
[Lei Yang]'s Articles
Baidu Scholar
Similar articles in Baidu Scholar
[Fan Zhu]'s Articles
[Ruixing Jia]'s Articles
[Lei Yang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Fan Zhu]'s Articles
[Ruixing Jia]'s Articles
[Lei Yang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.