2016
Five Degrees of Happiness: Effective Smiley Face Likert Scales for Evaluating with Children
In: Proceedings of the The 15th International Conference on Interaction Design and Children
Authors: Lynne Hall, Colette Hume and Sarah Tazzyman
Abstract: This paper focuses on achieving optimal responses through supporting children’s judgements, using Smiley Face Likert scales as a rating scale for quantitative questions in evaluations. It highlights the need to provide appropriate methods for children to communicate judgements, highlighting that the traditional Smiley Face Likert scale does not provide an appropriate method. The paper outlines a range of studies, identifying that to achieve differentiated data and full use of rating scales by children that faces with positive emotions should be used within Smiley Face Likert scales. The proposed rating method, the Five Degrees of Happiness Smiley Face Likert scale, was used in a large- scale summative evaluation of a Serious Game resulting in variance within and between children, with all points of the scale used.
Citation: Hall, L., Hume, C., & Tazzyman, S. (2016, June). Five Degrees of Happiness: Effective Smiley Face Likert Scales for Evaluating with Children. In Proceedings of the The 15th International Conference on Interaction Design and Children (pp. 311-321). ACM.
Download: Five Degrees of Happiness: Effective Smiley Face Likert Scales for Evaluating with Children
Map Reading with an Empathic Robot Tutor
In: The Eleventh ACM/IEEE International Conference on Human Robot Interaction
Authors: Lynne Hall, Colette Hume, Sarah Tazzyman, Amol Deshmukh, Srinivasan Janarthanam, Helen Hastie, Ruth Aylett, Ginevra Castellano, Fotis Papadopoulos, Aidan Jones, Lee J. Corrigan, Ana Paiva, Patrícia Alves Oliveira, Tiago Ribeiro, Wolmet Barendregt, Sofia Serholt & Arvid Kappas
Abstract: In this submission, we describe a scenario developed in the EMOTE project. The overall goal of the EMOTE project is to develop an empathic robot tutor for 11-13 year old school students in an educational setting. The pedagogical domain here is to assist students in learning and testing their map-reading skills typically learned as part of the geography curriculum in schools. We show this scenario with a NAO robot interacting with the students whilst performing map- reading tasks on a touch-screen device in this video.
Citation:Hall, L., Hume, C., Tazzyman, S., Deshmukh, A., Janarthanam, S., Hastie, H., Aylett, R., Castellano, G., Papadopoulos, F., Jones, A. and Corrigan, L.J. (2016, March). Map reading with an empathic robot tutor. In The Eleventh ACM/IEEE International Conference on Human Robot Interaction (pp. 567-567). IEEE Press.
Download: Map Reading with an Empathic Robot Tutor
2015
Engaging Children in Interactive Application Evaluation
In: Enfance, 2015
Authors: Hall, L., Hume, C., & Tazzyman, S.
Abstract: Interactive applications designed specifically for children offer great potential for education and play. However, to ascertain that the aims of applications are achieved, child-centred evaluations must be conducted. The design of any evaluation with children requires significant consideration of potential problems with comprehension, cognitive ability, response biases and study attrition. Multidisciplinary R&D project evaluation requirements are often extensive, requiring an all-encompassing and prolonged evaluation design. Discontinuity between the highly engaging interaction experience and the multitude of measures that form the evaluation poses a major issue for the evaluation of interactive applications. In response, we have developed Transmedia Evaluation, a method that aims to maintain engagement throughout the evaluation process. In this paper, the Transmedia Evaluation process is explained and applied to evaluate a learning application for children, MIXER (Moderating Interactions for Cross Cultural Empathic Relationships). Children aged 9-11 (N = 117) used the MIXER application and completed an evaluation battery including pre- and post- test questionnaires, immediate learning assessment and qualitative evaluation. Using Transmedia Evaluation to develop the MIXER evaluation resulted in complete data-sets (100%) for quantitative data (by self-regulated completion) along with rich, high quality qualitative responses. Transmedia Evaluation transformed the evaluation, with children fully engaging in and enjoying their experience.
Citation: Hall, L., Hume, C., & Tazzyman, S. (2015). Engaging Children in Interactive Application Evaluation. Enfance, 2015(01), 35-66.
Download: Engaging Children in Interactive Application Evaluation
From Non-human to Human: Adult’s and Children’s Perceptions of Agents Varying in Humanness
In: International Conference on Intelligent Virtual Agents
Authors: Eva Krumhuber, Arvid Kappas, Colette Hume, Lynne Hall, and Ruth Aylett.
Abstract: While most interface agents have been designed from an adult perspective, the present paper compares adults’ and children’s views of agents that vary in their degree of humanness. Four synthetic characters ranging in appearance from non-human to very human (blob, cat, cartoon, human) were presented to adult and children perceivers and were evaluated with respect to their cognitive and emotional abilities. The visual appearance significantly influenced participants’ ratings in both age groups. However, the pattern of results was more differentiated for adult perceivers as a function of the human-likeness of the character. The findings suggest that children may rely less on human-like features in inferring agents’ capabilities which are judged along simpler cognitive and social dimensions. Implications for the design of artificial agents are discussed.
Citation: Krumhuber, E., Kappas, A., Hume, C., Hall, L., & Aylett, R. (2015, August). From Non-human to Human: Adult’s and Children’s Perceptions of Agents Varying in Humanness. In International Conference on Intelligent Virtual Agents (pp. 471-474). Springer International Publishing.
Download: From Non-Human to Human: Adult’d and Children’s Perceptions of Agents Varying in Humanness
Learning to Overcome Cultural Conflict through Engaging with Intelligent Agents in Synthetic Cultures
In: International Journal of Artificial Intelligence in Education (, Volume 25, Issue 2, pp 291-317)
Authors: Lynne Hall, Sarah Tazzyman, Colette Hume, Birgit Endrass, Mei-Yii Lim, GertJan Hofstede, Ana Paiva, Elisabeth Andre, Arvid Kappas and Ruth Aylett.
Abstract:Providing opportunities for children to engage with intercultural learning has frequently focused on exposure to the ritual, celebrations and festivals of cultures, with the view that such experiences will result in greater acceptance of cultural differences. Intercultural conflict is often avoided, bringing as it does particular pedagogical, ethical and political dilemmas of which cultures we place in conflict in the multicultural classroom. In this paper we discuss an alternative approach, providing children with an interactive learning experience with synthetic cultures and characters. The agent architecture developed to enable intelligent agents to exhibit culturally appropriate affect and behaviours is outlined. MIXER, an experiential learning application developed for 9–11 year old children on intercultural conflict is described, highlighting the learning goals and approaches. A school-based evaluation of MIXER with 144 UK children is presented. Children demonstrated high levels of comprehension, engagement and enjoyment of MIXER, with MIXER contributing to near and far transfer, supporting children’s cognitive, emotional and behavioural learning and stimulating discussion and debate about how to resolve conflict.
Citation: Hall, L., Tazzyman, S., Hume, C., Endrass, B., Lim, M. Y., Hofstede, G., … & Aylett, R. (2014). Learning to overcome cultural conflict through engaging with intelligent agents in synthetic cultures. International Journal of Artificial Intelligence in Education, 25(2), 291-317.
2014
Engaging with virtual characters using a pictorial interaction language
In: CHI’14 Extended Abstracts on Human Factors in Computing Systems
Authors: Endrass, B., Hall, L., Hume, C., Tazzyman, S., Andre, E., & Aylett, R.
Abstract: Providing fun, engaging child-centric approaches to interaction is challenging. The Pictorial Interaction Language was developed for children to communicate and interact with virtual characters in a serious game, MIXER. The design and development of the Pictorial Interaction Language is briefly outlined. Results highlight that children found interacting fun and were highly positive about the Pictorial Interaction Language.
Citation: Endrass, B., Hall, L., Hume, C., Tazzyman, S., Andre, E., & Aylett, R. (2014, April). Engaging with virtual characters using a pictorial interaction language. In CHI’14 Extended Abstracts on Human Factors in Computing Systems (pp. 531-534). ACM.
Download: Engaging with Virtual Characters using a Pictorial Interaction Language
A Pictorial Interaction Language for Children to Communicate with Cultural Virtual Characters
In: 6th Int. Conf. on Human-Computer Interaction (HCII 2014)
Authors: Endrass, B., Hall, L., Hume, C., Tazzyman, S., & André, E
Abstract: In this paper, we outline the creation of an engaging and intuitive pictorial language as an interaction modality to be used by school children aged 9 to 11 years to interact with virtual characters in a cultural learning environment. Interaction takes place on a touch screen tablet computer linked to a desktop computer on which the characters are displayed. To investigate the benefit of such an interaction style, we conducted an evaluation study to compare the pictorial interaction language with a menu-driven version for the same system. Results indicate that children found the pictorial interaction language more fun and more exciting than the menus, with users expressing a desire to interact for longer using the pictorial interaction language. Thus, we think the pictorial interaction language can help support the children’s experiential learning, allowing them to concentrate on the content of the cultural learning scenario.
Citation: Endrass, B., Hall, L., Hume, C., Tazzyman, S., & André, E. (2014). A Pictorial Interaction Language for Children to Communicate with Cultural Virtual Characters. 16th Int. Conf. on Human-Computer Interaction (HCII 2014), 22 – 27 June 2014, Heraklion, Crete (in press)
Download: A Pictorial Interaction Language for Children to Communicate with Cultural Virtual Characters
2013
TRAVELLER – Intercultural training with intelligent agents for young adults.
In: First International Workshop on Intelligent Digital Games for Empowerment and Inclusion (IDGEI 2013)
Authors: Degens, N., Hofstede, G. J., Mascarenhas, S., Silva, A., Paiva, A., Kistler, F., André, E. Swiderska, A., Krumhuber, E., Kappas, A., Hume, C., Hall, L., & Aylett, R.
Abstract: In this paper we describe Traveller, an intercultural training tool for young adults. Traveller is based on an original theoretical framework which focuses on key concepts of intercultural training. By progressing through a creative story, users are able to engage via a novel interaction paradigm with intelligent virtual characters that incorporate different simulated cultures which can lead to misunderstandings and sometimes conflicts. Through the use of an innovative evaluation approach, users will gain a greater understanding of the behavioural differences between these characters, and thereby learn to become more effective at dealing with misunderstandings due to differences in culture
Citation: Degens, N., Hofstede, G. J., Mascarenhas, S., Silva, A., Paiva, A., Kistler, F., André, E. Swiderska, A., Krumhuber, E., Kappas, A., Hume, C., Hall, L., & Aylett, R. (2013). Traveller – Intercultural training with intelligent agents for young adults. Presented at the First International Workshop on Intelligent Digital Games for Empowerment and Inclusion (IDGEI 2013) held in conjunction with the 8th Foundations of Digital Games 2013 (FDG), Chania, Greece, May 2013
2012
Scaffolding the Story Creation Process
In: Proceedings of the 4th Conference on Computer Supported Education, Oporto, Portugal, 2012.
Authors: Hall, M., Hall, L., Hodgson, J., Hume, C., & Humphries.
Abstract: Comic books provide an appropriate and structured context for education and personal or peer reflection. In this paper we discuss the benefits of comic books and technology in a pedagogical context, including the mechanism of scaffolding and how this interaction impacts upon the child’s environment. Our studies into the educational benefits of comic books have lead to the development of an interactive comic book application. The application is being developed for the purpose of narrative inquiry through the creation and completion of a story scaffold. The analysis of the data will help evaluate the child’s social and cultural interaction with the story
Citation: Hall, M., Hall, L., Hodgson, J., Hume, C., & Humphries, L. (2012). Scaffolding the Story Creation Process, Proceedings of the 4th Conference on Computer Supported Education, Oporto, Portugal, April 2012.
Download: Scaffolding the Story Creation Process
Extending the Story into the Feedback Loop: Transmedia Evaluation
In: 1st Global Conference Immersive Worlds and Transmedia Narratives
Authors: Hall, L. and Hume, C
Citation: Hall, L. and Hume, C. (2012) Extending the Story into the Feedback Loop: Transmedia Evaluation, 1st Global Conference Immersive Worlds and Transmedia Narratives, 13-15 November, Salzburg.
ECUTE: Difference is Good!
In: IADIS eLearning Conference, Lisbon Portugal, 2012.
Authors: Nazir, A., Ritter, C., Aylett, R., Krumhuber, E., Swiderska, A., Degens, N., Endrass, B., Hume, C., Hodgson, J., & Mascarenhas, S.
Citation: Nazir, A., Ritter, C., Aylett, R., Krumhuber, E., Swiderska, A., Degens, N., Endrass, B., Hume, C., Hodgson, J., & Mascarenhas, S. (2012) ECUTE: DIFFERENCE IS GOOD! IADIS eLearning Conference, Lisbon Portugal.
Download: ECUTE: Difference is Good!
Incorporating Multi-Modal Evaluation into a Technology Enhanced Learning Experience
In: IVA, University of California, Santa Cruz, 2012
Authors: Hall, L., Aylett, R., Hume, C., Krumhuber, E; Degens, N.
Abstract: This paper outlines Transmedia Evaluation and its application to the evaluation of TRAVELLER, a Technology Enhanced Learning application that aims to provide 18-25 year olds with improved cultural understanding and sensitivity. TRAVELLER aims to enhance the users’ intercultural communication through engaging users in multi-modal interaction with intelligent characters deployed in emergent narratives. Transmedia Evaluation will enable us to gather and analyse a wide range of complimentary user data in response to diverse R&D questions, whilst embedding the evaluation within the user experience, so that the user is unaware of evaluation instruments and measures. The described approach will enable us to investigate subjective views and physiological responses to the narrative and characters, reflective and immediate reactions to innovative interaction. To achieve this we will use data collection instruments ranging from questionnaires to observation to physiological measurement, all embedded within the context of a Live (Inter)Action Role Play.
Citation: Hall, L., Aylett, R., Hume, C. (2012) Incorporating Multi-Modal Evaluation into a Technology Enhanced Learning Experience. IVA, University of California, Santa Cruz.
Download: Incorporating multi-modal evaluation into a technology enhanced learning experience
2011
Why Numbers, Invites and Visits are not Enough: Evaluating the User Experience in Social Eco-Systems
In: SOTICS, October 2011.
Authors: Lynne Hall and Colette Hume.
Abstract: Social eco-systems are often evaluated through quantitative data that is automatically logged and analysed. However, where the user’s experience of social eco-systems is evaluated, more explicit intervention approaches are typical, with questionnaires, focus groups and user testing widely used, directly asking the user about their experience. User experience evaluation thus ruptures the social eco-system, occurring as a separate, discrete activity outside of that system. In this paper, we propose that evaluation should be part of the social eco-system adding value to the user experience. We outline an evaluation approach that has been applied within games-based learning environments where the evaluation is seamlessly embedded. We briefly outline our approach to generating and analysing data highlighting its potential for social eco-system evaluation.
Citation: Hall, L., & Hume, C. (2011). Why Numbers , Invites and Visits are not Enough : Evaluating the User Experience in Social Eco-Systems. SOTICS, (pp. 8-13) Barcelona, Spain.
Download: Why Numbers, Invites and Visits are not Enough: Evaluating the User Experience in Social Eco-Systems