Journal of Information Systems Engineering and Management

Journal of Information Systems Engineering and Management, 2022 - Volume 7 Issue 2, Article No: 14039
https://doi.org/10.55267/iadt.07.12011

Published Online: 14 Apr 2022

Views: 962 | Downloads: 461

Interaction and command modes as well as their combination are essential features of modern and futuristic robotic systems interacting with human beings in various dynamical environments. This paper presents a synthetic overview concerning the most command modes used in Human-Robot Interaction Systems (HRIS). It includes the first historical command modes which are namely tele-manipulation, off-line robot programming, and traditional elementary teaching by demonstration. It then introduces the most recent command modes which have been fostered later on by the use of artificial intelligence techniques implemented on more powerful computers. In this context, we will consider specifically the following modes: interactive programming based on the graphical-user-interfaces, voice-based, pointing-on-image-based, gesture-based, and finally brain-based commands.

human-robot interaction teleoperation speech-based commands image-based commands gesture-based command brain-computer interface

• Abiri, R., Heise, G., Zhao, X., Jiang, Y., Abiri, F.A., 2017. Brain computer interface for gesture control of a social robot: An offline study. 2017 Iranian Conference on Electrical Engineering (ICEE) 113–117.
• Afonso, A., Angélico Gonçalves, M., Lima, J., Cota, M., 2014. UsaWeb. A model for usability evaluation web interfaces, in: Iberian Conference on Information Systems and Technologies, CISTI. Presented at the Iberian Conference on Information Systems and Technologies, CISTI, p. 6. https://doi.org/10.1109/CISTI.2014.6876952
• Afonso, A.P., 2013. Proyecto de Evaluación de Interfaces — PAI,. Universidad de Vigo, Departamento de Informática., Vigo.
• Appelstal, M., Michalak, J., Osterberg, M., 2018. Easy to Use Graphical User Interface for Robot Programming.
• Arents, J., Greitans, M., 2022. Smart Industrial Robot Control Trends, Challenges and Opportunities within Manufacturing. Applied Sciences 12. https://doi.org/10.3390/app12020937
• Argall, B.D., Chernova, S., Veloso, M., Browning, B., 2009. A survey of robot learning from demonstration. Robotics and Autonomous Systems 57, 469–483. https://doi.org/10.1016/j.robot.2008.10.024
• Arkin, R.C., 1998. Behavior-based robotics. MIT press.
• Baker, S., Matthews, I., 2004. Lucas-Kanade 20 Years On: A Unifying Framework. Int. J. Comput. Vision.
• Baniqued, P., Stanyer, E., Awais, M., Alazmani, A., Jackson, A., Mon-Williams, M., Mushtaq, F., Holt, R., 2021. Brain-Computer Interface Robotics for Hand Rehabilitation After Stroke: A Systematic Review. Journal of NeuroEngineering and Rehabilitation 18. https://doi.org/10.1186/s12984-021-00820-8
• Berna-Martinez, J., 2011. Robotic Control Based On The Human Nervous System. International Journal of Artificial Intelligence & Applications 2, 107. https://doi.org/10.5121/ijaia.2011.2409
• Bonci, A., Fiori, S., Higashi, H., Tanaka, T., Verdini, F., 2021. An Introductory Tutorial on Brain–Computer Interfaces and Their Applications. Electronics 10, 560. https://doi.org/10.3390/electronics10050560
• Bouchemal, B., Zaatri, A., 2014. Image-based control for cable-based robots. International Journal of Control, Automation and Systems 12, 118–125. https://doi.org/10.1007/s12555-012-0445-y
• Bouchemal, B., Zaatri, A., 2013. Gestural and Image-Based Control Combination for rehabilitation applications using cable-based robots. Presented at the International Conference on Technology for Helping People with Special Needs, Riyadh, Kingdom of Saudi Arabia.
• Buzsaki, G., 2006. Rhythms of the Brain. Oxford university press.
• Ceccarelli, M., 2001. A Historical Perspective of Robotics Toward the Future. Journal of Robotics and Mechatronics 13, 299–313. https://doi.org/10.20965/jrm.2001.p0299
• Chakraborti, T., Kambhampati, S., Scheutz, M., Zhang, Y., 2017. AI challenges in human-robot cognitive teaming. arXiv preprint arXiv:1707.04775.
• Corke, P., 1994. High-Performance Visual Closed-Loop Robot Control.
• Doelling, K., Assaneo, M., 2021. Neural oscillations are a start toward understanding brain activity rather than the end. PLoS biology 19, e3001234. https://doi.org/10.1371/journal.pbio.3001234
• Eakins, W., Rossano, G., Fuhlbrigge, T., 2013. Lead through Robot Teaching. Presented at the IEEE Conference Technology Practice Robot Application, pp. 1–4.
• Elliott, L.R., Hill, S.G., Barnes, M.J., 2016. Gesture-Based Controls for Robots: Overview and Implications for Use by Soldiers. Presented at the Human Research and Engineering Directorate, ARL, US Army Research Laboratory, ARL-TR-7715.
• Feil-Seifer, D., Mataric, M., 2009. Human-Robot Interaction, in: Journal Abbreviation: Encyclopedia of Complexity and Systems Science. Springer, pp. 4643–4659.
• Ferreira, A., Celeste, W., auat cheein, F., Bastos, T., Sarcinelli-Filho, M., Carelli, R., 2008. Human-Machine interfaces base on EMG and EEG applied to robotic systems. Journal of neuroengineering and rehabilitation 5, 10. https://doi.org/10.1186/1743-0003-5-10
• Ford, M., 2015. Rise of the Robots Technology and the Threat of a Jobless Future. Basic Books, New York, NY, US.
• Fukui, H., Yonejima, S., Yamano, M., Dohi, M., Yamada, M., Nishiki, T., 2009. Development of teaching pendant optimized for robot application. 2009 IEEE Workshop on Advanced Robotics and its Social Impacts 72–77.
• Galván-Ruiz, J., Travieso, C., Tejera-Fettmilch, A., Pinan-Roescher, A., Esteban-Hernández, L., Dominguez Quintana, L., 2020. Perspective and Evolution of Gesture Recognition for Sign Language: A Review. Sensors 20, 3571. https://doi.org/10.3390/s20123571
• Gasparetto, A., Scalera, L., 2019. A Brief History of Industrial Robotics in the 20th Century. Advances in Historical Studies 08, 24–35. https://doi.org/10.4236/ahs.2019.81002
• Hans, S., 2018. A Brief Comparative Study of Visual Servoing Systems. IJSRD - International Journal for Scientific Research & Development 6, 2321–0613.
• Harriott, C., Adams, J., 2013. Modeling Human Performance for Human-Robot Systems. Reviews of Human Factors and Ergonomics 9, 94–130. https://doi.org/10.1177/1557234X13501471
• Javaid, M., Khan, I.H., 2021. Internet of Things (IoT) enabled healthcare helps to take the challenges of COVID-19 Pandemic. Journal of Oral Biology and Craniofacial Research 11, 209–214. https://doi.org/10.1016/j.jobcr.2021.01.015
• Joseph, J.S., 1998. Teach Pendant for An Industrial Robot.
• Kim, W.S., Stark, L.W., 1989. Cooperative control of visual displays for telemanipulation. Proceedings, 1989 International Conference on Robotics and Automation 1327–1332 vol.3.
• Krämer, N., Rosenthal-von der Pütten, A.M., Eimler, S., 2012. Human-Agent and Human-Robot Interaction Theory: Similarities to and Differences from Human-Human Interaction. Studies in Computational Intelligence 396, 215–240. https://doi.org/10.1007/978-3-642-25691-2_9
• Lemaignan, S., Warnier, M., Sisbot, E.A., Clodic, A., Alami, R., 2017. Artificial cognition for social human–robot interaction: An implementation. Artificial Intelligence 247, 45–69. https://doi.org/10.1016/j.artint.2016.07.002
• Lichiardopol, S., 2007. A survey on teleoperation, DCT rapporten. Technische Universiteit Eindhoven, Eindhoven.
• Linde, Y., Buzo, A., Gray, R.M., 1980. An Algorithm for Vector Quantizer Design. IEEE Trans. Commun. 28, 84–95.
• Liu, J., Luo, Y., Ju, Z., 2016. An Interactive Astronaut-Robot System with Gesture Control. Computational Intelligence and Neuroscience 1–11. https://doi.org/10.1155/2016/7845102
• Low, K., 2006. Industrial Robotics: Programming, Simulation and Applications. IntechOpen, Rijeka.
• Lucas, B., Kanade, T., 1981. An Iterative Image Registration Technique with an Application to Stereo Vision (IJCAI), [No source information available].
• Makhataeva, Z., Varol, A., 2020. Augmented Reality for Robotics: A Review. Robotics 9, 21. https://doi.org/10.3390/robotics9020021
• Mane, R., Chouhan, T., Guan, C., 2020. BCI for stroke rehabilitation: motor and beyond. Journal of Neural Engineering 17, 041001. https://doi.org/10.1088/1741-2552/aba162
• Marion, P., Fallon, M., Deits, R., Valenzuela, A., D’Arpino, C.P., Izatt, G., Manuelli, L., Antone, M., Dai, H., Koolen, T., Carter, J., Kuindersma, S., Tedrake, R., 2017. Director: A User Interface Designed for Robot Operation with Shared Autonomy. J. Field Robot.
• Martinek, R., Ladrova, M., Sidikova, M., Jaros, R., Behbehani, K., Kahankova, R., Kawala-Sterniuk, A., 2021. Advanced Bioelectrical Signal Processing Methods: Past, Present, and Future Approach—Part III: Other Biosignals. Sensors 21. https://doi.org/10.3390/s21186064
• Mitsi, S., Bouzakis, K.-D., Mansour, G., Sagris, D., Maliaris, G., 2005. Off-line programming of an industrial robot for manufacturing. The International Journal of Advanced Manufacturing Technology 26, 262–267. https://doi.org/10.1007/s00170-003-1728-5
• Muda, L., Begam, M., Elamvazuthi, I., 2000. Voice Recognition Algorithms using Mel Frequency Cepstral Coefficient (MFCC) and Dynamic Time Warping (DTW) Techniques. Journal of Computing 3, 138–143.
• Myers, B., 1995. User Interface Software Tools. ACM Transactions on Computer-Human Interaction (TOCHI) 2, 64–103. https://doi.org/10.1145/200968.200971
• Nam, C., Nijholt, A., Lotte, F., 2018. Brain-Computer Interfaces Handbook: Technological and Theoretical Advances. CRC Press, Taylor & Francis Group, Oxford, UK.
• Nearchou, A., 2011. Innovative gesture-based interaction.
• Nicolas-Alonso, L.F., Gomez-Gil, J., 2012. Brain computer interfaces, a review. Sensors (Basel) 12, 1211–1279. https://doi.org/10.3390/s120201211
• Nicolescu, M.N., Mataric, M.J., 2005. Task Learning Through Imitation and Human-Robot Interaction Dimensions, in: Models and Mechanisms of Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative. Cambridge University Press.
• Oussalah, M., Zaatri, A., 2003. Integration and Design of Multimodal Interfaces for Supervisory Control Systems. Information Fusion 4, 135–150. https://doi.org/10.1016/S1566-2535(03)00018-6
• Pan, Z., Polden, J., Larkin, N., Van Duin, S., Norrish, J., 2012. Recent progress on programming methods for industrial robots. Robotics and Computer-Integrated Manufacturing 28, 87–94. https://doi.org/10.1016/j.rcim.2011.08.004
• Paul, D., Parekh, R., 2011. Automated Speech Recognition of Isolated Words using Neural Networks. International Journal of Engineering Science and Technology 3, 4993–5000.
• Perez, J.A., Deligianni, F., Ravi, D., Yang, G.-Z., 2018. Artificial intelligence and robotics. arXiv preprint arXiv:1803.10813 147.
• Pinto, J., 2010. Multilayer Perceptron Based Hierarchical Acoustic Modeling for Automatic Speech Recognition. EPFL, Lausanne, Switzerland.
• Qi, L., Zhang, D., 2009. A Lead-Through Robot Programming Approach Using A 6-DOF Wire-based Motion Tracking Device, in: IEEE International Conference on Robotics and Biomimetics, ROBIO 2009, December 19-13, 2009, Guilin, Guangxi, China. Presented at the IEEE International Conference on Robotics and Biomimetics, IEEE, Guangxi, China, pp. 1773–1777.
• R. Brooks, 1986. A robust layered control system for a mobile robot. IEEE Journal on Robotics and Automation 2, 14–23. https://doi.org/10.1109/JRA.1986.1087032
• Rabiner, L.R., 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 257–286. https://doi.org/10.1109/5.18626
• Ramos, O.E., 2018. Foundations of Robotics: 2018-1 - Topic 1a - Historical Context, Types, and Applications of Robots,. UTEC, Universidad de Ingenieria y tecnologia.
• Rechy-Ramirez, E.J., Hu, H., 2015. Bio-signal based control in assistive robots: a survey. Digital Communications and Networks 1, 85–101. https://doi.org/10.1016/j.dcan.2015.02.004
• S. Garcıa, C. Menghi, P. Pelliccione, T. Berger, R. Wohlrab, 2018. An Architecture for Decentralized, Collaborative, and Autonomous Robots, in: 2018 IEEE International Conference on Software Architecture (ICSA). Presented at the 2018 IEEE International Conference on Software Architecture (ICSA), pp. 75–7509. https://doi.org/10.1109/ICSA.2018.00017
• S. Mitra, T. Acharya, 2007. Gesture Recognition: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37, 311–324. https://doi.org/10.1109/TSMCC.2007.893280
• Semeraro, F., Griffiths, A., Cangelosi, A., 2021. Human-Robot Collaboration and Machine Learning: A Systematic Review of Recent Research.
• Sheridan, T.B., 2016. Human–Robot Interaction: Status and Challenges. Hum Factors 58, 525–532. https://doi.org/10.1177/0018720816644364
• Sheridan, T.B., 1992. Telerobotics, automation, and human supervisory control. MIT press.
• Sigalas, M., Baltzakis, H., Trahanias, P., 2010. Gesture recognition based on arm tracking for human-robot interaction. Presented at the Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems, p. 5429. https://doi.org/10.1109/IROS.2010.5648870
• Staub, C., Can, S., Knoll, A., Nitsch, V., Karl, I., Faerber, B., 2011. Implementation and Evaluation of a gesture-based Input Method in Robotic Surgery, HAVE 2011 - IEEE International Symposium on Haptic Audio-Visual Environments and Games, Proceedings. https://doi.org/10.1109/HAVE.2011.6088384
• Steyrl, D., Kobler, R., Müller-Putz, G., 2016. On Similarities and Differences of Invasive and Non-Invasive Electrical Brain Signals in Brain-Computer Interfacing. Journal of biomedical science and engineering 9, 393–398. https://doi.org/10.4236/jbise.2016.98034
• Terzopoulost, G., Satratzemi, M., 2020. Voice Assistants and Smart Speakers in Everyday Life and in Education. Informatics in Education 19, 473–490. https://doi.org/10.15388/infedu.2020.21
• Vahrenkamp, N., Wieland, S., Azad, P., Gonzalez-Aguirre, D.I., Asfour, T., Dillmann, R., 2008. Visual servoing for humanoid grasping and manipulation tasks. Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots 406–412.
• W. S. Kim, B. Hannaford, A. K. Fejczy, 1992. Force-reflection and shared compliant control in operating telemanipulators with time delay. IEEE Transactions on Robotics and Automation 8, 176–185. https://doi.org/10.1109/70.134272
• Wallén, J., 2008. The History of the Industrial Robot (No. 14003902 (ISSN)), LiTH-ISY-R. Linköping University Electronic Press, Linköping.
• Wang, Y., 2016. A Visual Servoing Approach to Human-Robot Interactive Object Transfer. Aachen, Germany.
• Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M., 2002. Brain-computer interfaces for communication and control. Clin Neurophysiol 113, 767–791. https://doi.org/10.1016/s1388-2457(02)00057-3
• Yonck, R., 2002. Heart of the Machine: Our Future in a World of Artificial Emotional Intelligence. Skyhorse, Arcade publishing.
• Yong, Y.F., Bonney, M.C., 1999. Off-line programming” Handbook of Industrial Robotics. Wiley, United States.
• Zaatri, A., 2021. Towards a Unified Representation for Human-Robot Control Architectures. Journal of Information Technology & Software Engineering 11. https://doi.org/10.4172/2165-7866
• Zaatri, A., 2000. Investigations into integrated supervisory control systems. Katholieke Universiteit, Leuven, Belgium.
• Zaatri, A., Azzizi, N., Rahmani, F., 2015. Design experiments for voice commands using neural networks. World Journal of Engineering 12, 301–306. https://doi.org/10.1260/1708-5284.12.3.301
• Zaatri, A., Van Brussel, 1997. Investigations in telerobotics using cooperative supervisory modes of control. Presented at the International Conference on Telemanipulator and Telepresence Technologies IV, pp. 41–52.
• Zamalloa, I., Kojcev, R., Hernandez, A., Muguruza, I., Usategui, L., Bilbao, A., Mayoral, V., 2017. Dissecting Robotics - historical overview and future perspectives.
• Zendoui, F., Mahmoudi, M., Zaatri, A., 2018. Development and Experimentation of an Articulated Mechanical System using Internet. Mechanics 24, 462–466. https://doi.org/10.5755/j01.mech.19386

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.