Understanding How People Approach Constraint Modelling and Solving – University of St Andrews and University of Victoria

Ruth Hoffmann will be presenting the paper on “Understanding How People Approach Constraint Modelling and Solving” at the 28th International Conference on Principles and Practice of Constraint Programming (CP 2022) taking place between July 31 to August 5, 2022 in Haifa, Israel.

This paper is a joint collaboration between SACHI (Human Computer Interaction) and Constraint Programming groups, in both the University of St Andrews, Scotland and the University of Victoria, BC.

Abstract

Research in constraint programming typically focuses on problem solving efficiency. However, the way users conceptualise problems and communicate with constraint programming tools is often sidelined. How humans think about constraint problems can be important for the development of efficient tools that are useful to a broader audience. For example, a system incorporating knowledge on how people think about constraint problems can provide explanations to users and improve the communication between the human and the solver.
We present an initial step towards a better understanding of the human side of the constraint solving process. To our knowledge, this is the first human-centred study addressing how people approach constraint modelling and solving. We observed three sets of ten users each (constraint programmers, computer scientists and non-computer scientists) and analysed how they find solutions for well-known constraint problems. We found regularities offering clues about how to design systems that are more intelligible to humans.

Researchers

• Ruth Hoffmann
• Xu Zhu
• Özgür Akgün
• Miguel Nacenta
  • Personal website: https://nacenta.com/
  • Research website: https://vixi.cs.uvic.ca/

The paper can be found at: https://doi.org/10.4230/LIPIcs.CP.2022.28

Conference

Ruth will be presenting the paper in the main conference and giving an invited talk at ModRef 2022 to raise awareness of the benefits of understanding how people represent, model and solve constraint problems.

CP 2022 Conference link: https://easychair.org/smart-program/FLoC2022/CP-2022-08-03.html#talk:197219

ModRef 2022 link: https://easychair.org/smart-program/FLoC2022/ModRef-2022-07-31.html#talk:197355

More ModRef info: https://modref.github.io/ModRef2022.html#invtalks

The paper is open access: Representational transformations: Using maps to write essays.

Summary of the paper and its findings

We built a tool to study how writers move between map and text to write essays. The main takeaway is that important cognitive work happens in the transformation process between map and text representations.

There are lots of existing tools for building representations to support complex cognitive tasks – e.g. argument maps, text, notes, slides, sketches, and so on. But tool support for the transformations *between* representations is much more neglected – and we think it’s crucial!

We built Write Reason, a tool which combines a text editor and a mapping interface. You can drag parts of the map into the text, and parts of the text into the map, and it helps you keep them in sync.

We then studied how 20 students used Write Reason to write essays. You can interactively explore the maps and essays built by participants. We identified key properties of transformations: change in representation type, cardinality, and explicitness. And we found that most used an all-at-once batch translation, while a few used bit-by-bit interleaving. 

We think understanding transformations is crucial for building the next generation of multi-representational tools. How can we better support multi-transformation pipelines like these? Can automation unlock more complex + powerful workflows, which would be tedious to do manually?

Our findings revealed and falsified some of the key implicit assumptions that we baked into the design of Write Reason. We hope that these reflections will help other designers and researchers start one step ahead of us and avoid these mistakes!

Project page. Paper (open access).

Thrilled to see Iain and Guilherme graduating this week. Congratulations on your well-deserved success Dr. Carneiro & Dr. Carson!

Abstract:

Bio:

Event Details

• When: Friday 06 March 2020, 2-3pm

• Where: JCB:1.33b – Teaching Laboratory

Abstract:

Visual programming environments have long been applied in an educational context for encouraging uptake of computer science, with a more recent focus on blocks-based programming as a means to teach computational thinking concepts.  Today, students in primary, secondary and even tertiary education are learning to code through blocks-based environments like Scratch and App Inventor, and studies in these settings have shown that they ease the transition to ‘real’ programming in high-level languages such as Java and Python.  My question is, do we need to bother with that transition?  Can we accomplish more with blocks than just programming for its own sake?  More ‘serious’ visual programming environments like LabVIEW for engineers, and Blueprints embedded in the Unreal Engine for game developers are testament to visual programming producing more than just toy programs, so how far could blocks go?  In this talk, I’ll give an overview of blocks-based programming and its applications outside education, including its role in my PhD project and current postdoctoral research in allowing end-users with no programming experience to tailor spoken dialog systems.

Bio:

Daniel is a postdoctoral research fellow working in the HCI group in UCD on the B-SPOKE project with Dr Ben Cowan.  The goal of this project is to open up the development of Spoken Dialog Systems to the end-user without programming experience, through techniques from the field of end-user development.  Prior to this, Daniel completed his PhD at the University of St Andrews, focusing on the adoption of an end-user development tool for psychology researchers to create their own data collection apps.  Daniel is especially interested in applying blocks-based programming (the visual approach to learning code used in well-known tools like Scratch) to domain-specific applications, allowing end-users to customise their software experiences without writing a single line of code.

Event Details

• When: Monday 03 February 2020, 11am – 12hrs
• Where: JCB:1.33A – Teaching Laboratory

Abstract: Innovation and creativity are the research drivers of the Human-Computer Interaction (HCI) community which is currently investing a vast amount of resources in the design and evaluation of “new” user interfaces and interaction techniques, leaving the correct functioning of these interfaces at the discretion of the helpless developers.  In the area of formal methods and dependable systems the emphasis is usually put on the correct functioning of the system leaving its usability to secondary-level concerns (if at all addressed).  However, designing interactive systems requires blending knowledge from these domains in order to provide operators with enjoyable, usable and dependable systems.  The talk will present possible research directions and their benefits for combining several complementary approaches to engineer interactive critical systems.  Due to their specificities, addressing this problem requires the definition of methods, notations, processes and tools to go from early informal requirements to deployed and maintained operational interactive systems.  The presentation will highlight the benefits of (and the need for) an integrated framework for the iterative design of operators’ procedures and tasks, training material and the interactive system itself.  The emphasis will be on interaction techniques specification and validation as their design is usually the main concern of HCI conferences.  A specific focus will be on automation that is widely integrated in interactive systems both at interaction techniques level and at application level.  Examples will be taken from interactive cockpits on large civil commercial aircrafts (such as the A380), satellite ground segment application and Air Traffic Control workstations.

Bio: Dr. Philippe Palanque is Professor in Computer Science at the University Toulouse 3 “Paul Sabatier” and is head of the Interactive Critical Systems group at the Institut de Recherche en Informatique de Toulouse (IRIT) in France. Since the late 80s he has been working on the development and application of formal description techniques for interactive system. He has worked for more than 10 years on research projects to improve interactive Ground Segment Systems at the Centre National d’Etudes Spatiales (CNES) and is also involved in the development of software architectures and user interface modeling for interactive cockpits in large civil aircraft (funded by Airbus). He was involved in the research network HALA! (Higher Automation Levels in Aviation) funded by SESAR programme which targets at building the future European air traffic management system. The main driver of Philippe’s research over the last 20 years has been to address in an even way Usability, Safety and Dependability in order to build trustable safety critical interactive systems. He is the secretary of the IFIP Working group 13.5 on Resilience, Reliability, Safety and Human Error in System Development, was steering committee chair of the CHI conference series at ACM SIGCHI and chair of the IFIP Technical Committee 13 on Human-Computer Interaction.

Hui-Shyong Yeo from SACHI was attending MobileHCI 2019 in Taiwan, UIST 2019 in New Orleans, USA and ISS 2019 in Daejeon, South Korea.

He presented 2 papers and a demo at the conferences. He was a student volunteer at ISS.

WRIST: Watch-Ring Interaction and Sensing Technique for Wrist Gestures And Macro-Micro Pointing
Hui-Shyong Yeo, Juyoung Lee, Hyung-il Kim, Aakar Gupta, Andrea Bianchi, Daniel Vogel, Woontack Woo, Aaron Quigley
In Proceedings of International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI’19.

[PDF][ACM DL]

[VIDEO]

Opisthenar: Hand Poses and Finger Tapping Recognition by Observing Back of Hand Using Embedded Wrist Camera
Hui-Shyong Yeo, Erwin Wu, Juyoung Lee, Aaron Quigley and Hideki Koike
In Proceedings of the ACM symposium on User interface software and technology, UIST’19.

[PDF][ACM DL]

[VIDEO]

Presenting Opisthenar

Student volunteering at ISS 2019

Bill Buxton giving keynote at ISS 2019

Erwin Wu demoing Opisthenar at UIST 2019

Different WRIST pointing techniques

Event Details

• When: Thursday 14 November 2019, 2-3pm
• Where: JCB:1.33B – Teaching Laboratory

Title: Toward magnetic force based haptic rendering and friction based tactile rendering

Abstract: Among all senses, the haptic system provides a unique and bidirectional communication channel between humans and the real word around them.  Extending the frontier of traditional visual rendering and auditory rendering, haptic rendering enables human operators to actively feel, touch and manipulate virtual (or remote) objects through force and tactile feedback, which further increases the quality of Human-Computer Interaction.  It has been effectively used for a number of applications including surgical simulation and training, virtual prototyping, data visualization, nano-manipulation, education and other interactive applications.  My work will explore the design and construction of our magnetic haptic interface for force feedback and our surface friction based tactile rendering system through combining electrovibration effect and squeeze film effect.

Bio: Dr XIONG LU is an Associate Professor in College of Control Engineering at Nanjing University of Aeronautics and Astronautics and is an academic visitor in St Andrews HCI research group in the School of Computer Science at University of St Andrews.  He received his Ph.D. degree in Measuring and Testing Technologies and Instruments from Southeast University, in China.  His mainly research interests is Human-Computer Interaction, Haptic Rendering and Tactile Rendering.

Uta Hinrichs, Fearn Bishop and Xu Zhu are representing SACHI this year at the IEEE VIS’19 conference which is held in Vancouver, BC, Canada.

Fearn will present her research on exploring free-form visualization processes of children. Xu will present his work on how people visually represent discrete constraint problems. Uta has been involved on research that introduces design by immersion as a novel transdisciplinary approach to problem-driven visualization. She is also co-chairing the VIS Doctoral Colloquium this year, and is co-organizing the 4th workshop on Visualization for the Digital Humanities (VIS4DH’19).

Design by Immersion: A Transdisciplinary Approach to Problem-driven Visualizations [preprint]
Kyle Wm. Hall, Adam Bradley, Uta Hinrichs, Samuel Huron, Jo Wood, Christopher Collins and Sheelagh Carpendale.

Tuesday, Oct. 22 – 2:35-3:50 PM  [preview video]
Provocations; Ballroom A

Construct-A-Vis: Exploring the Free-form Visualization Processes of Children [preprint]
Fearn Bishop, Johannes Zagermann, Ulrike Pfeil, Gemma Sanderson, Harald Reiterer and Uta Hinrichs.

Wednesday, Oct. 23 – 2:20-3:50 PM
(De)Construction; Ballroom A

How People Visually Represent Discrete Constraint Problems [TVCG paper; PDF]
Xu Zhu, X, Miguel Nacenta, Özgür Akgün and Peter W. Nightingale

Thursday, Oct. 24 – 9:00-10:30 AM [preview video]
Vis for Software and Systems; Ballroom B

Event details

• When: 8th October 2019 09:30 – 15:15
• Where: Byre Theatre
• Series: Distinguished Lectures Series
• Format: Distinguished lecture

Speaker: Stephen Brewster (University of Glasgow)
Venue: The Byre Theatre

Timetable:

9:30: Lecture 1: The past: what is multimodal interaction?
10:30 Coffee break
11:15 Lecture 2: The present: does it work in practice?
12:15 Lunch (not provided)
14:15 The future: Where next for multimodal interaction?

Speaker Bio:

Professor Brewster is a Professor of Human-Computer Interaction in the Department of Computing Science at the University of Glasgow, UK. His main research interest is in Multimodal Human-Computer Interaction, sound and haptics and gestures. He has done a lot of research into Earcons, a particular form of non-speech sounds.

He did his degree in Computer Science at the University of Herfordshire in the UK. After a period in industry he did his PhD in the Human-Computer Interaction Group at the University of York in the UK with Dr Alistair Edwards. The title of his thesis was “Providing a structured method for integrating non-speech audio into human-computer interfaces”. That is where he developed my interests in Earcons and non-speech sound.

After finishing his PhD he worked as a research fellow for the European Union as part of the European Research Consortium for Informatics and Mathematics (ERCIM). From September, 1994 – March, 1995 he worked at VTT Information Technology in Helsinki, Finland. He then worked at SINTEF DELAB in Trondheim, Norway.