James Charles Wen

Faculty of Logistics

Press photo

Norwegian version of this page Position

Associate Professor in IT

Contact

+4771214099

james.c.wen@himolde.no

Study place

Molde

Office nr.

C433

A bit about me...

Hi, there and thanks for dropping by! I am Professor of Information Technology and Human Computer Interaction. I hail from Queens, New York, did my undergraduate degree in Physics and Computer Science at Cornell University and graduate studies at Brown University in Computer Science. Turning down a scholarship to join MIT's Media Lab (partly because I wanted to be an astronaut and NASA didn't accept technology degrees, which was what the Media Lab offered), I chose to join IBM Research doing research in visualization and interaction design before moving to Amazon to build the annotation system for their Look Inside the Book feature. After founding and leading tech startups in New York and Seattle, I decided I really love research and did my PhD under the Augmented Reality expert, Mark Billinghurst, at the University of Canterbury. My research established a quantitative relationship between the usability of AR navigation tools and the retention of cognitive maps. Outside of research, I enjoy playing music, (American) football, watching cartoons, the outdoors, and traveling.

My Courses

IBE320 - eXtended Reality
IBE405 - The Internet of Things and Smart Cities

My Research

My area of interest lies in the intersection between humans and computers, neatly packaged in the area of Human Computer Interaction. I try to understand, on a fundamental level through empirical qualitative and quantitative research methods, how human behavior should affect technology design and how technologies affect us as human beings. Many people have expressed concern about how technology can impact our innate abilities greatly and my research has focused on this area. For example, exploring the relationship between the ease-of-use of digital navigation tools and our natural ability to retain cognitive maps, I built augmented reality tools and analyzed empirical data to balance the utility of navigation tools with our dependency on them.

Switching from a cognitive to a social setting, I investigated how digital technology can distort our judgment of our ability to establish connections with others. I implemented an iPhone app for cooperative image capture to combine online collaboration with in-person interaction. This revealed perceptual discrepancies of our social comfort zones that I reconciled through user interface elements. In another project, I operationalized empathy for robot teammates by using the Unity game engine to build a gamified simulation of a space mining mission to study relationships within human-machine teams. Using a mixed method analysis, I found evidence that people may form strong attachment to high tech tools revealing how we interact with digital technologies in general, from the Internet of Things to autonomous vehicles.

Taken together, my research deals with the impact of emerging technologies on the users they serve as well as the relationship between people and technologies, which capture the research directions I plan to expand and extend upon. My work has been published in peer-reviewed venues of the ACM, IEEE, and APA (see publications list below). I have reviewed for numerous venues including ACM Computer Human Interaction, IEEE Human Robot Interaction, International Conference on Intelligent Robots and Systems, and the Human Factors and Ergonomics Society. I have also helped organize several local and international conferences in the startup space and in the area of HCI including NordiCHI and CHINZ. I served two terms as council member and two terms as president of the New York Association of Phi Beta Kappa, which was founded in 1776 and is the oldest academic association in the United States. My duties included fund raising, donor relations, selection of scholarship recipient institutions, and managing over 3000 active members.

Research groups

ABC-AI: Applied, Basic, and Conscientious Artificial Intelligence

Helton, William S. & Wen, James (2023). Will the real resource theory please stand up! Vigilance is a renewable resource and should be modeled as such. Experimental Brain Research. ISSN 0014-4819. 241(5), p. 1263–1270. doi: 10.1007/s00221-023-06604-x. Show summary
The vigilance decrement or decline in signal detection performance with time on task is one of the most reliable findings in the cognitive neuroscience and psychology literatures. The majority of theories proposed to explain the decrement are limited cognitive or attention resource based theories; the central nervous system is a limited capacity processor. The decrement in performance is then due to resource reallocation (or misallocation), resource depletion or some combination of both mechanisms. The role of resource depletion, in particular, is hotly debated. However, this may be due to a lack of understanding of the renewable nature of the vigilance resources and how this renewal process impacts performance during vigilance tasks. In the present paper, a simple quantitative model of vigilance resource depletion and renewal is described and shown to generate performance data similar to results seen in both humans and spiders. This model clarifies the role resource depletion and resource renewal may play in vigilance in both people and other animals.
Ward, Matthew; Wen, James Charles; Head, James & Helton, William S. (2019). Designing wearable interfaces for people who have hard jobs. In Mouloua, Mustapha & Hancock, Peter A. (Ed.), Human performance in automated and autonomous systems : current theory and methods. Routledge. ISSN 9781138312272.
Wen, James Charles; Stewart, Amanda; Billinghurst, Mark Nathan; Dey, Arindam; Tossell, Chad C & Finomore, Victor S (2018). He who hesitates is lost (...in thoughts over a robot). In ., Ukjent (Eds.), TechMindSociety '18: Proceedings of the Technology, Mind, and Society. Association for Computing Machinery (ACM). ISSN 978-1-4503-5420-2. p. 1–6. doi: 10.1145/3183654.3183703. Show summary
In a team, the strong bonds that can form between teammates are often seen as critical for reaching peak performance. This perspective may need to be reconsidered, however, if some team members are autonomous robots since establishing bonds with fundamentally inanimate and expendable objects may prove counterproductive. Previous work has measured empathic responses towards robots as singular events at the conclusion of experimental sessions. As relationships extend over long periods of time, sustained empathic behavior towards robots would be of interest. In order to measure user actions that may vary over time and are affected by empathy towards a robot teammate, we created the TEAMMATE simulation system. Our findings suggest that inducing empathy through a back story narrative can significantly change participant decisions in actions that may have consequences for a robot companion over time. The results of our study can have strong implications for the overall performance of human machine teams.
Wen, James Charles; Stewart, Amanda; Billinghurst, Mark & Tossel, Chad (2018). TEAMMATE : a scalable system for measuring affect in human-machine teams. In Cabibihan, John-John; Mastrogiovanni, Fulvio; Pandey, Amit Kumar; Rossi, Silvia & Staffa, Mariacarla (Ed.), 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE Press. ISSN 9781538679814. p. 991–996. doi: 10.1109/ROMAN.2018.8525704. Show summary
Strong empathic bonding between members of a team can elevate team performance tremendously but it is not clear how such bonding within human-machine teams may impact upon mission success. Prior work using self-reporting surveys and end-of-task metrics do not capture how such bonding may evolve over time and impact upon task fulfillment. Furthermore, sensor-based measures do not scale easily to facilitate the need to collect substantial data for measuring potentially subtle effects. We introduce TEAMMATE, a system designed to provide insights into the emotional dynamics humans may form for machine teammates that could critically impact upon the design of human machine teams.
Wen, James Charles; Stewart, Amanda; Billinghurst, Mark & Tossell, Chad (2018). Band of brothers and bolts : caring about your robot teammate. In IEEE, IEEE (Eds.), 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-5386-8094-0. p. 1853–1858. doi: 10.1109/IROS.2018.8594324. Show summary
It has been observed that a robot shown as suffering is enough to cause an empathic response from a person. Whether the response is a fleeting reaction with no consequences or a meaningful perspective change with associated behavior modifications is not clear. Existing work has been limited to measurements made at the end of empathy inducing experimental trials rather measurements made over time to capture consequential behavioral pattern. We report on preliminary results collected from a study that attempts to measure how the actions of a participant may be altered by empathy for a robot companion. Our findings suggest that induced empathy can in fact have a significant impact on a person's behavior to the extent that the ability to fulfill a mission may be affected.
Wen, James Charles; Helton, William S. & Billinghurst, Mark (2015). If reality bites, bite back virtually : simulating perfection in augmented reality tracking. In Billinghurst, Mark & Marshall, Stuart (Ed.), CHINZ '13: Proceedings of the 14th Annual ACM SIGCHI_NZ conference on Computer-Human Interaction. Association for Computing Machinery (ACM). ISSN 978-1-4503-2640-7. p. 1–8. doi: 10.1145/2542242.2542246.
Wen, James Charles & Ünlüer, A. Ayça (2015). Fun, cool...and awkward : preliminary reactions to cooperative photography, MobileHCI '15 : Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. Association for Computing Machinery (ACM). ISSN 978-1-4503-3653-6. p. 831–837. doi: 10.1145/2786567.2793708.
Wen, James Charles & Ünlüer, A. Ayça (2015). Redefining the fundamentals of photography with cooperative photography. In Holzmann, Clemens; Mayrhofer, René; Häkkilä, Jonna; Rukzio, Enrico & Roland, Michael (Ed.), MUM '15 : Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia. Association for Computing Machinery (ACM). ISSN 978-1-4503-3605-5. p. 37–47. doi: 10.1145/2836041.2836045. Show summary
Cooperative photography redefines some fundamental notions of the photographic process by using mobile devices to synchronously coordinate multiple parties in a collaborative image capturing task. Using wireless communication and geo-location technologies, the photographer takes on a support role as content enabler rather than as the content creator, opening up creative possibilities. However, unreliable wireless networks and misunderstandings between users can make the coordination challenging and lead to a premature termination of the task. We present the evolution of the iterative design process we used to find a suitable set of interactions to encourage users to complete cooperative photography tasks. We report on our design motivations and their associated user studies, which suggest that user experiences dealing with higher level user behavior rather than lower level system awareness may be central in facilitating the successful completion of cooperative photography tasks.
Wen, James Charles; Deneka, Agnes; Helton, William S.; Dünser, Andreas & Billinghurst, Mark (2014). Fighting technology dumb down : our cognitive capacity for effortful AR navigation tools. In Kurosu, Masaaki (Eds.), Human-Computer Interaction : Applications and Services 16th International Conference, HCI International 2014, Heraklion, Crete, Greece, June 22-27, 2014, Proceedings, Part III. Springer. ISSN 978-3-319-07227-2. p. 525–536. doi: 10.1007/978-3-319-07227-2_50. Show summary
By overlaying virtual guidance information directly over the surrounding environment, Augmented Reality (AR) is seen as an easy alternative to maps for pedestrians navigating in unfamiliar urban environments. It is hypothesized, however, that easing navigation tasks would result in weaker cognitive maps, leaving users more vulnerable to becoming lost should their navigation device fail. We describe an outdoor navigation study that highlighted the gap between theoretical expectations and real world testing with navigation tools. We addressed the issues by creating a simulation system for testing navigation tools and report on the results of a study comparing AR with maps. We then extended the system to support simultaneous secondary tasks to assess relative workload. We present this as a way of objectively measuring relative cognitive effort expended on navigation tool use. Our findings are helpful in the design of mobile pedestrian navigation tools seeking to balance navigational efficiency with mental map formation.
Wen, James Charles; Helton, William S. & Billinghurst, Mark (2013). A study of user perception, interface performance, and actual usage of mobile pedestrian navigation aides. Proceedings of the Human Factors and Ergonomics Society Annual Meeting.. ISSN 1071-1813. doi: 10.1177/154193121357143. Show summary
The proliferation of pedestrian navigation tools has made it challenging for users to avoid being confused and overwhelmed by the choices. Studies comparing mobile pedestrian navigation aides have generally based conclusions on either survey results from separate trials of exclusive interface usage or on performance of the interfaces as judged by the speed with which users are able to complete wayfinding tasks. However, it is not clear if users would mirror their individual trials or find a more strategic mixed-mode approach to using the tools at their disposal when given an option to choose from a set of tools. It is also unclear if users actually care about performance when choosing a navigation tool. We conducted a study to compare actual usage of navigation tools against user perception of the tools and performance with the tools in a series of wayfinding tasks. Results indicate that independent surveys can align well with extreme cases while performance may not actually be a good indicator of usage preferences.
Wen, James Charles; Helton, William S. & Billinghurst, Mark (2013). Classifying users of mobile pedestrian navigation tools. In Shen, Haifeng; Smith, Ross; Paay, Jeni; Calder, Paul & Wyeld, Theodor (Ed.), OzCHI '13: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration. ACM Publications. ISSN 978-1-4503-2525-7. p. 13–16. doi: 10.1145/2541016.2541073. Show summary
Welcome to Adelaide and OzCHI 2013, the annual conference of the Computer-Human Interaction Special Interest Group (CHISIG) of the Human Factors and Ergonomics Society of Australia (HFSA) and Australia's leading forum for a growing international community of practitioners, researchers, academics and students to exchange work in all areas of Human-Computer Interaction.
Dünser, Andreas; Billinghurst, Mark; Wen, James Charles; Lehtinen, Ville & Nurminen, Antti (2012). Exploring the use of handheld AR for outdoor navigation. Computers & graphics. ISSN 0097-8493. 36(8), p. 1084–1095. doi: 10.1016/j.cag.2012.10.001. Show summary
Recently, mobile-phone based outdoor augmented reality (AR) systems have become readily available. One of the most popular applications are AR browsers that show virtual points of interest (POIs) overlaid on top of the phone's camera view. These virtual cues can be used to guide people to the POIs. However, the usefulness of AR systems for guiding users to POI has not yet been evaluated, especially when compared to map interfaces. In this paper we present results of a user study comparing navigation with information typically provided by currently available handheld AR browsers, to navigation with a digital map, and a combined map and AR condition. We found no overall difference in task completion time, but found evidence that AR browsers are less useful for navigation in some environment conditions. We also found that navigation performance differed significantly with gender for the Map and AR+Map interfaces, but is very similar across gender for the AR interface. Users preferred the combined AR+Map condition, and felt that there were significant problems with using the AR view alone for navigation.

View all works in Cristin

Wen, James Charles (2024). Når menneskelig intelligens og kunstig intelligens blander seg.
Wen, James Charles & Colley, Ashley (2022). Hybrid Online Survey System with Real-Time Moderator Chat. In Döring, Tanja; Boll, Susanne; Colley, Ashley; Esteves, Augusto & Guerreiro, João (Ed.), MUM 2022: 21th International Conference on Mobile and Ubiquitous Multimedia, Lisbon Portugal, November 27 - 30, 2022. ACM Digital Library. ISSN 978-1-4503-9820-6. p. 257–258. doi: 10.1145/3568444.3570593. Show summary
Online surveys present a quick and efficient method to collect user experience data. However, they are less effective at extracting qualitative data. Due to response fatigue when answering open-ended questions, the quality of responses may be poor, and participants can skip answering individual questions, or drop out from the survey entirely, potentially introducing bias. To address these challenges, we developed a prototype tool that enables a test moderator to initiate a chat intervention with a participant at any point during the completion of an online survey. Through this approach, participants can be prompted to return to complete unanswered questions or provide clarification to given answers. A functional prototype system has been implemented and, as future work, will be evaluated with a variety of content and question types.
Wen, James Charles; Tossel, Chad & de Visser, Ewart J. (2018). Bridging Sensemaking and Situation Awareness through Human-Machine Teams.
Wen, James Charles (2015). Getting things started in cooperative photography. In Cosley, Dan; Forte, Andrea; Ciolfi, Luigina & McDonald, David Wright (Ed.), CSCW'15 Companion: Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing. Association for Computing Machinery (ACM). ISSN 978-1-4503-2946-0. p. 49–52. doi: 10.1145/2685553.2702684.
Wen, James Charles (2015). Communicating with the outside world through surreptitious wearable systems, WearSys '15: Proceedings of the 2015 workshop on Wearable Systems and Applications. Association for Computing Machinery (ACM). ISSN 978-1-4503-3500-3. p. 59–60. doi: 10.1145/2753509.2753523.
Wen, James Charles; Deneka, Agnes; Helton, William S. & Billinghurst, Mark Nathan (2014). Really, it's for your own good...making augmented reality navigation tools harder to use. In Jones, Matt; Philippe, Palanque; Albrecht, Schmidt & Grossman, Tovi (Ed.), CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing Systems. ACM Publications. ISSN 9781450324748. p. 1297–1302. doi: 10.1145/2559206.2581156.
Wen, James Charles (2003). Post-valued recall web pages : user disorientation hits the big time. IT & Society. 1(3), p. 184–194. Show summary
The growth in complexity of the World Wide Web, matched by the increase in user sophistication, has transformed the notion of user disorientation. While the retrieval of visited Web pages has been studied, the particular class of Web pages not initially recognized as being important enough to return to merits closer examination. Such pages, referred to as Post-Valued Recall (PVR) Web pages, tend to sit at the interface between the solitary act of surfing the Web and the social act of interacting with colleagues, who will often trigger the realization that a particular page is, in fact, of value. Results from a user study illustrate the effect of PVR in a social environment, as well as offer design guidelines for browser utilities that could diminish the negative impact of PVR.
Wen, James Charles (2014). The potential for augmented reality to Bbing balance between the ease of pedestrian navigation and the acquisition of spatial knowledge. University of Canterbury. Show summary
Being completely lost in an unfamiliar environment can be inconvenient, stressful and, at times, even dangerous. Maps are the traditional tools used for guidance but many people find maps difficult to use. In recent years, new tools like outdoor Augmented Reality (AR) have become available which allow virtual navigation cues to be directly overlaid on the real world, potentially overcoming the limitations of maps. However, it has been hypothesized that lower effort invested in processing navigation guidance may lead to diminished spatial knowledge (SK) thereby making users of such navigation tools far more vulnerable to getting lost should the tools fail for any reason. This thesis explores the research question of how AR and maps compare as tools for pedestrian navigation guidance as well as for SK acquisition and if there is a potential for AR tools be developed that would balance the two. We present a series of studies to better understand the consequences of using AR in a pedestrian navigation tool. The first two studies compared time-on-task performance and user preferences for AR and Map navigation interfaces on an outdoor navigation task. The results were not aligned with expectations, which led us to build a controlled testing environment for comparing AR and map navigation. Using this simulated setting, our third study verified the assumption that AR can indeed result in more efficient navigation performance and it supported the hypothesis that this would come at the cost of weaker SK. In our fourth study, we used a dual task design to compare the relative cognitive resources required by map and AR interfaces. The quantitative data collected indicated that users could potentially accept additional workload designed to improve SK without incurring significantly more effort. Our fifth and final study explored an interface with additional AR cues that could potentially balance navigation guidance with SK acquisition. The contributions of this thesis include insights into performance issues relating to AR, a classification of user types based on navigation tool usage behavior, a testbed for simulating perfect AR tracking in a virtual setting, objective measures for determining route knowledge, the capacity that pedestrian navigation tool users may have for performing additional tasks, and guidelines that would be helpful in the design of pedestrian navigation tools.