The UB develops an artificial intelligence system to analyse non-verbal communication quality
In academic presentations, to keep eye-contact with the panel while exposing and gesticulating matches with high marks. This is one of the conclusions of a study in which researchers at the University of Barcelona (UB) have created a new system of artificial intelligence to evaluate the quality of non-verbal communication. Nowadays, communication competences are on the top of the most relevant skills for oneʼs professional and personal life. In the case of higher education, “assessment was centred so far on the command of contents and the ability to explain them; however, now, with the implementation of the European Higher Education Area, competences related to cooperation and communication are much more important than knowledge”, says Sergio Escalera, researcher at the Faculty of Mathematics of the UB, member of the Computer Vision Center, head of the Research Group on Human Behaviour Analysis (HuPBA), and leader of the study, developed together with UB postgraduate students Álvaro Cepero y Albert Clapés. The system, described on an article published in the journal AI Communications, is an evolution of a former system developed by the research group together with the PhD student Víctor Ponce.
In academic presentations, to keep eye-contact with the panel while exposing and gesticulating matches with high marks. This is one of the conclusions of a study in which researchers at the University of Barcelona (UB) have created a new system of artificial intelligence to evaluate the quality of non-verbal communication. Nowadays, communication competences are on the top of the most relevant skills for oneʼs professional and personal life. In the case of higher education, “assessment was centred so far on the command of contents and the ability to explain them; however, now, with the implementation of the European Higher Education Area, competences related to cooperation and communication are much more important than knowledge”, says Sergio Escalera, researcher at the Faculty of Mathematics of the UB, member of the Computer Vision Center, head of the Research Group on Human Behaviour Analysis (HuPBA), and leader of the study, developed together with UB postgraduate students Álvaro Cepero y Albert Clapés. The system, described on an article published in the journal AI Communications, is an evolution of a former system developed by the research group together with the PhD student Víctor Ponce.
“Our programme shows that computer vision and artificial intelligence systems can be used to analyse automatically the quality of non-verbal communication”, affirms Escalera. “The system —he adds— is useful to analyse presentations, with teaching and evaluation purposes, and job interviews. However, tools to perform behavioural analyses are not available in the market yet”.
The system —developed within a UB-funded project of the Consolidated Research Group on Teaching Innovation in Mathematics and Computing (INDOMAIN) of the Faculty of Mathematics— was used to analyse fifty-four recorded videos of presentations performed by students of the UB degree in Computer Engineering and the inter-university masterʼs degree in Artificial Intelligence (UPC-UB-URV). All the videos were recorded with a KinectTM device that enables to capture image, but also depth and audio data.
According to Clapés, “first, considering literature about non-verbal communication, a reliable set of behaviour indicators was defined, for example the degree of agitation and hand position in relation to body”.
For instance, to observe whether the speaker faces the audience, the number of frames in which the subject faces the audience is considered. To determine whether speakerʼs arms are crossed, the distance between opposite arm and elbow is measured. The system also assesses speakerʼs degree of agitation by analysing arm movement and voice intensity.
“The number of studies centred on these aspects is increasing. Particularly, they are applied to e-learning programmes, which is one of European Unionʼs prior actions included in Horizon 2020 projects”, points out Escalera.
In the study, presentations were rated by three different instructors; some are UB psychologists in the group Research and Innovation Group on Designs (GRID), led by Professor M. Teresa Anguera. System reliability was proved by comparing systemʼs results with instructorsʼ observations.
Analysis of human behaviour
The HuPBA focuses its activity on two main research lines: one centred on computer vision and based on defining new computational algorithms able to interpret images; and the other devoted to automatic learning, in which a set of data —image, audio or any other type of data with numerical representation—, allows the system producing statistics and making predictions and correlations.
The UB research group is expert on behaviour analysis; they use transversal methods that enable them to apply results to many disciplines. For example, in security, these systems can be used to analyse whether a certain behaviour is dangerous or not, and to develop mechanisms for automatic detection. In health, they can support elderly care (the Care Repite project) or analyse automatically body posture in physiotherapy, rehabilitation and athletic performance (the system ADiBAS Posture developed by the UB spin-off PhyscalTech).
“In short, we use expert knowledge to programme automatic learning in order to get a system that helps experts making decisions, but which is not a diagnosis tool”, concludes Escalera.
Article reference:
Cepero, Álvaro; Clapés, Albert; Escalera, Sergio. "Automatic non-verbal communication skills analysis: a quantitative evaluation". AI Communications, January 2015. DOI: 10.3233/AIC-140617