Forum for Artificial Intelligence
Archive
|
This website is the archive for past Forum for Artificial Intelligence talks. Please click this link to navigate to the list of current talks. FAI meets every other week (or so) to discuss scientific, philosophical, and cultural issues in artificial intelligence. Both technical research topics and broader inter-disciplinary aspects of AI are covered, and all are welcome to attend! If you would like to be added to the FAI mailing list, subscribe here. If you have any questions or comments, please send email to Catherine Andersson. |
[ Upcoming talks ]
Friday, September 24, 2021, 11:00AM
|
Approaches to Grounded Language Acquisition from Human InteractionCynthia Matuszek [homepage]
Abstract: As robots move from labs and factories into human-centric
spaces, it becomes progressively harder to predetermine the
environments and interactions they will need to be able to handle.
Letting robots learn from end users via natural language is an
intuitive, versatile approach to handling novel situations robustly.
Grounded language acquisition is concerned with learning to understand
language in the context of the physical world. In this presentation, I
will give an overview of our work on using joint statistical models to
learn the grounded semantics of natural language describing an agent's
environment. I will discuss the role of speech in grounded language
learning, including introducing a new dataset and results on learning
directly from that speech, and will describe work on learning language
in simulated environments.
About the speaker:Bio: Cynthia Matuszek is an assistant professor of computer science and electrical engineering at the University of Maryland, Baltimore County, and the director of UMBC’s Interactive Robotics and Language lab. After working as a researcher on the Cyc project, she received her Ph.D. in computer science and engineering from the University of Washington in 2014. Her research is focused on how robots can learn grounded language from interactions with non-specialists, which includes work in not only robotics, but human-robot interactions, natural language, machine learning, machine bias, and collaborative robot learning, informed by a background in common-sense reasoning and classical artificial intelligence. Dr Matuszek has been named in the IEEE bi-annual “10 to watch in AI,” and has published in machine learning, artificial intelligence, robotics, and human-robot interaction venues.Watch Online |
Friday, October 1, 2021, 11:00AM
|
Overclaiming Is a Problem. Underclaiming May Be Worse.Sam Bowman [homepage]
Abstract: In an effort to avoid reinforcing widespread hype about the capabilities of state-of-the-art NLP systems, researchers have developed habits in framing and citation that serve to deemphasize the field's successes, even at the cost of making misleadingly strong claims about the limits of our best systems. This is a problem, though, and it may be more serious than it looks: It limits our ability to mitigate short-term harms from NLP deployments and it limits our ability to prepare for the impacts of highly effective future systems, which may be significantly greater. This paper urges researchers to be careful about these claims, and suggests some research directions that will make it easier to avoid or rebut them.
This talk will draw primarily from a work-in-progress paper on this issue and secondarily from a recent NAACL position paper with George Dahl, with additional references to several other evaluation-related projects from my group.
About the speaker:Bio: Sam Bowman has been on the faculty at NYU since 2016, when he completed PhD with Chris Manning and Chris Potts at Stanford. At NYU, he is a member of the Center for Data Science, the Department of Linguistics, and Courant Institute's Department of Computer Science. His research focuses on data, evaluation techniques, and modeling techniques for sentence and paragraph understanding in natural language processing, and on applications of machine learning to scientific questions in linguistic syntax and semantics. He is the senior organizer behind the GLUE and SuperGLUE benchmark competitions and he has received a 2015 EMNLP Best Resource Paper Award, a 2019 *SEM Best Paper Award, a 2017 Google Faculty Research Award, and a 2021 NSF CAREER award.Watch Online |
Friday, October 15, 2021, 11:00AM
|
Forty Years of Machine Learning: Myths, Metaphors, Paradigms, and ChallengesPat Langley [homepage]
The machine learning community declared itself as a new branch
of artificial intelligence at its first workshop in 1980. In this talk,
I review the field's forty-year history, covering developments that
led to its intellectual advances and its application successes. I also
examine some myths that have emerged along the way, metaphors
that have guided research, and paradigms that have drawn attention
in different periods. In closing, I describe some open challenges that,
despite recent impressive successes, suggest the field remains far
from reproducing the breadth and depth of human learning.
About the speaker:Dr. Pat Langley serves as Director of the Institute for the Study of Learning and Expertise and as Research Scientist at Stanford University's Center for Design Research. He has contributed to artificial intelligence and cognitive science for more than 40 years, having published over 300 papers and five books on these topics. Dr. Langley developed some of the first computational approaches to scientific knowledge discovery, and he was an early champion of experimental studies of machine learning and its application to real-world problems. He is the founding editor of two journals, Machine Learning in 1986 and Advances in Cognitive Systems in 2012, and he is a Fellow of both AAAI and the Cognitive Science Society. Dr. Langley's current research focuses on architectures for embodied agents, abductive methods for plan understanding, and learning procedures from written instructions.Watch Online |
Friday, October 22, 2021, 11:00AM
|
Factor Graphs for Perception and ActionFrank Dellaert [homepage]
Factor graphs have been very successful in providing a lingua franca in which to phrase robotics perception and navigation problems. In this talk I will re-visit some of those successes, also discussed in depth in [a recent review article](https://t.co/Xc0RMXyeYY). However, I will focus on our more recent work in the talk, centered on using factor graphs for *action*. In particular, I will discuss our efforts in motion planning, trajectory optimization, optimal control, and model-predictive control, highlighting in each how factor graphs provide an intuitive and natural framework in which to think about these problems and generate state of the art solutions.
About the speaker:Bio: Frank Dellaert is a Professor at Georgia Tech's School of Interactive Computing, and a research scientist at Google AI. He has previously done stints at Skydio, a drone startup, and Facebook Reality Labs. His work is on sensor fusion and the use of large-scale graphical models for robotics sensing, thinking, and acting. With his students and collaborators, he created the popular sensor fusion/SLAM library GTSAM, see gtsam.org.Watch Online |
Friday, October 29, 2021, 11:00AM
|
Designing Emotionally-Intelligent Digital Humans that Move, Express, and Feel Like Us!Aniket Bera [homepage]
The creation of intelligent virtual agents (IVAs) or digital humans is vital for many virtual and augmented reality systems. As the world increasingly uses digital and virtual platforms for everyday communication and interactions, there is a heightened need to create human-like virtual avatars and agents endowed with social and emotional intelligence. Interactions between humans and virtual agents are being used in different areas including, VR, games and story-telling, computer-aided design, social robotics, and healthcare. Designing and building intelligent agents that can communicate and connect with people is necessary but not sufficient. Researchers must also consider how these IVAs will inspire trust and desire among humans. Knowing the perceived affective states and social-psychological constructs (such as behavior, emotions, psychology, motivations, and beliefs) of humans in such scenarios allows the agents to make more informed decisions and navigate and interact in a socially intelligent manner.
In this talk, I will give an overview of our recent work on simulating intelligent, interactive, and immersive human-like agents who can also learn, understand and be sentient to the world around them using a combination of emotive gestures, gaits, and expressions. Finally, I will also talk about our many ongoing projects which use our AI-driven IVAs, including intelligent digital humans for urban simulation, crowd simulation, mental health and therapy applications, and social robotics
About the speaker:Bio: Aniket Bera is an Assistant Research Professor at the Department of Computer Science. His core research interests are in Affective Computing, Computer Graphics (AR/VR, Augmented Intelligence, Multi-Agent Simulation), Autonomous Agents, Cognitive modeling and planning for intelligent characters. His work has won multiple awards at top Graphics/VR conferences. He has previously worked in many research labs, including Disney Research, Intel, and the Centre for Development of Advanced Computing. Aniket's research has been featured on CBS, WIRED, Forbes, FastCompany, Times of India, etc.Watch Online |
Friday, November 5, 2021, 11:00AM
|
Learning Better Ways to Measure and Move: Joint Optimization of an Agent's Physical Design and Computational ReasoningMatthew Walter [homepage]
The recent surge of progress in machine learning foreshadows the advent of sophisticated intelligent devices and agents capable of rich interactions with the physical world. Many of these these advances focus on building better computational methods for inference and control---computational reasoning methods trained to discover and exploit the statistical structure and relationships in their problem domain. However, the design of physical interfaces through which a machine senses and acts in its environment is as critical to its success as the efficacy of its computational reasoning. Perception problems become easier when sensors provide measurements that are more informative towards the quantities to be inferred. Control policies become more effective when an agent's physical design permits greater robustness and dexterity in its actions. Thus, the problems of physical design and computational reasoning are coupled, and the answer to what combination is optimal naturally depends on the environment the machine operates in and the task before it.
I will present learning-based methods that perform automated, data-driven optimization over sensor measurement strategies and physical configurations jointly with computational inference and control. I will first describe a framework that reasons over the configuration of sensor networks in conjunction with the corresponding algorithm that infers spatial phenomena from noisy sensor readings. Key to the framework is encoding sensor network design as a differential neural layer that interfaces with a neural network for inference, allowing for joint optimization using standard techniques for training neural networks. Next, I will present a method that draws on the success of data-driven approaches to continuous control to jointly optimize the physical structure of legged robots and the control policy that enables them to locomote. The method maintains a distribution over designs and uses reinforcement learning to optimize a shared control policy to maximize the expected reward over the design distribution. I will then describe recent work that extends this approach to the coupled design and control of physically realizable soft robots. If time permits, I will conclude with a discussion of ongoing work that seeks to improve test-time generalization of the learned policies.
About the speaker:Matthew R. Walter is an assistant professor at the Toyota Technological Institute at Chicago. His interests revolve around the realization of intelligent, perceptually aware robots that are able to act robustly and effectively in unstructured environments, particularly with and alongside people. His research focuses on machine learning-based solutions that allow robots to learn to understand and interact with the people, places, and objects in their surroundings. Matthew has investigated these areas in the context of various robotic platforms, including autonomous underwater vehicles, self-driving cars, voice-commandable wheelchairs, mobile manipulators, and autonomous cars for (rubber) ducks. Matthew obtained his Ph.D. from the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution, where his thesis focused on improving the efficiency of inference for simultaneous localization and mapping.Watch Online |
Friday, November 12, 2021, 11:00AM
|
Towards Robust HRI: A Quality Diversity ApproachStefanos Nikolaidis [homepage]
The growth of scale and complexity of interactions between humans and robots highlights the need for new computational methods to automatically evaluate novel algorithms and applications. Exploring the diverse scenarios of interaction between humans and robots in simulation can improve understanding of complex HRI systems and avoid potentially costly failures in real-world settings.
In this talk, I propose formulating the problem of automatic scenario generation in HRI as a quality diversity problem, where the goal is not to find a single global optimum, but a diverse range of failure scenarios that explore both environments and human actions. I show how standard quality diversity algorithms can discover surprising and unexpected failure cases in the shared autonomy domain. I then discuss the development of a new class of quality diversity algorithms that significantly improve the search of the scenario space and the integration of these algorithms with generative models, which enables the generation of complex and realistic scenarios. Finally, I discuss applications in procedural content generation and human preference learning.
About the speaker:Stefanos Nikolaidis is an Assistant Professor of Computer Science at the University of Southern California and leads the Interactive and Collaborative Autonomous Robotics Systems (ICAROS) lab. His research focuses on stochastic optimization approaches for learning and evaluation of human-robot interactions. His work leads to end-to-end solutions that enable deployed robotic systems to act optimally when interacting with people in practical, real-world applications. Stefanos completed his PhD at Carnegie Mellon's Robotics Institute and received an MS from MIT, a MEng from the University of Tokyo and a BS from the National Technical University of Athens. His research has been recognized with an oral presentation at NeurIPS and best paper awards and nominations from the IEEE/ACM International Conference on Human-Robot Interaction, the International Conference on Intelligent Robots and Systems, and the International Symposium on Robotics.Watch Online |
Friday, November 19, 2021, 11:00AM
|
David V.S. Goliath: the Art of Leaderboarding in the Era of Extreme-Scale Neural ModelsYejin Choi [homepage]
Scale appears to be the winning recipe in today's leaderboards. And yet, extreme-scale neural models are still brittle to make errors that are often nonsensical and even counterintuitive. In this talk, I will argue for the importance of knowledge, especially commonsense knowledge, and demonstrate how smaller models developed in academia can still have an edge over larger industry-scale models, if powered with knowledge.
First, I will introduce "symbolic knowledge distillation", a new framework to distill larger neural language models into smaller commonsense models, which leads to a machine-authored KB that wins, for the first time, over a human-authored KB in all criteria: scale, accuracy, and diversity. Next, I will introduce a new conceptual framework for language-based commonsense moral reasoning, and discuss how we can teach neural language models about complex social norms and human values, so that the machine can reason that “helping a friend” is generally a good thing to do, but “helping a friend spread fake news” is not. Finally, I will discuss an approach to multimodal script knowledge, which leads to new SOTA performances on a dozen leaderboards that require grounded, temporal, and causal commonsense reasoning.
About the speaker:Bio: Yejin Choi is Brett Helsel Professor at the Paul G. Allen School of Computer Science & Engineering at the University of Washington and also a senior research manager at AI2 overseeing the project Mosaic. Her research focuses on commonsense knowledge and reasoning, language grounding with vision and perception, and AI for social good. She is a co-recepient of the ACL Test of Time award in 2021, the CVPR Longuet-Higgins Prize (test of time award) in 2021, the AAAI Outstanding Paper Award (best paper award) in 2020, the Borg Early Career Award (BECA) in 2018, the inaugural Alexa Prize Challenge in 2017, IEEE AI's 10 to Watch in 2016, and the Marr Prize (best paper award) at ICCV 2013. She received her Ph.D. in Computer Science at Cornell University and BS in Computer Science and Engineering at Seoul National University in Korea.Watch Online |
Friday, December 10, 2021, 11:00AM
|
Systematic Reasoning and Explanation over Natural LanguagePeter Clark [homepage]
Recent work has shown that transformers can be trained to reason *systematically* with natural language (NL) statements, answering questions with answers implied by a set of provided facts and rules, and even generating proofs for those conclusions. However, these systems required all the knowledge to be provided explicitly as input. In this talk, I will describe our current work on generalizing this to real NL problems, where the system produces faithful, entailment-based proofs for its answers, including materializing its own latent knowledge as needed for those proofs. The resulting reasoning-supported answers can then be inspected, debugged, and corrected by the user, offering new opportunities for interactive problem-solving dialogs, and taking a step towards "teachable systems" that can learn from such dialogs over time.
About the speaker:Peter Clark (peterc@allenai.org) is a Senior Research Manager at the Allen Institute for AI (AI2) and leads the Aristo Project. His work focuses on natural language processing, machine reasoning, and world knowledge, and the interplay between these three areas.Watch Online |
Friday, February 11, 2022, 11:00AM
|
Principles and Algorithms for Data-Efficient Assistive Sequential Decision MakingSiddharth Srivastava [homepage]
Can we balance efficiency and reliability while designing assistive AI systems? What would such AI systems need to provide? In this talk I will present some of our recent work addressing these questions. In particular, I will show that a few fundamental principles of abstraction are remarkably effective in designing efficient and reliable AI systems that can learn, plan and act over extended horizons. Our results show that abstraction mechanisms are invaluable not only in improving the data efficiency of sequential decision making, but also in developing AI systems that can explain their own behavior to non-experts, and in computing user-interpretable assessments of the limits and capabilities of Black-Box AI systems. I will also present some of our work on learning the requisite abstractions in a bottom-up fashion. Throughout the talk I will highlight the theoretical guarantees that our methods provide along with results from empirical evaluations featuring AI systems such as decision support systems and physical robots.
About the speaker:Siddharth Srivastava is an Assistant Professor of Computer Science in the School of Computing and Augmented Intelligence at Arizona State University. Prof. Srivastava was a Staff Scientist at the United Technologies Research Center in Berkeley. Prior to that, he was a postdoctoral researcher in the RUGS group at the University of California Berkeley. He received his PhD in Computer Science from the University of Massachusetts Amherst. His research interests include robotics and AI, with a focus on reasoning, planning, and acting under uncertainty. His work on integrated task and motion planning for household robotics has received coverage from international news media. He is a recipient of the NSF CAREER award, a Best Paper award at the International Conference on Automated Planning and Scheduling (ICAPS) and an Outstanding Dissertation award from the Department of Computer Science at UMass Amherst. He served as conference chair for ICAPS 2019 and currently serves as an Associate Editor for the Journal of AI Research.Watch Online |
Friday, February 18, 2022, 11:00AM
|
Causal analysis of the syntactic representations used by TransformersTal Linzen [homepage]
The success of artificial neural networks in language processing tasks has underscored the need to understand how they accomplish their behavior, and, in particular, how their internal vector representations support that behavior. The probing paradigm, which has often been invoked to address this question, relies on the (typically implicit) assumption that if a classifier can decode a particular piece of information from the model's intermediate representation, then that information plays a role in shaping the model's behavior. This assumption is not necessarily justified. Using the test case of everyone's favorite syntactic phenomenon - English subject-verb number agreement - I will present an approach that provides much stronger evidence for the *causal* role of the encoding of a particular linguistic feature in the model's behavior. This approach, which we refer to as AlterRep, modifies the internal representation in question such that it encodes the opposite value of that feature; e.g., if BERT originally encoded a particular word as occurring inside a relative clause, we modify the representation to encode that it is not inside the relative clause. I will show that the conclusions of this method diverge from those of the probing method. Finally, if time permits, I will present a method based on causal mediation analysis that makes it possible to draw causal conclusions by applying counterfactual interventions to the *inputs*, contrasting with AlterRep which intervenes on the model's internal representations.
About the speaker:Tal Linzen is an Assistant Professor of Linguistics and Data Science at New York University and a Research Scientist at Google. Before moving to NYU in 2020, he was a faculty member at Johns Hopkins University, a postdoctoral researcher at the École Normale Supérieure in Paris, and a PhD student at NYU. At NYU, Tal directs the Computational Psycholinguistics Lab, which develops computational models of human language comprehension and acquisition, as well as methods for interpreting and evaluating neural network models for language technologies.Watch Online |
Friday, March 4, 2022, 11:00AM
|
Recent advances in speech representation learningAbdelrahman Mohamed [homepage]
Self-supervised representation learning methods recently achieved great successes in NLP and computer vision domains, reaching new performance levels while reducing required labels for many downstream scenarios. Speech representation learning is experiencing similar progress. This talk will present our recent work on self-supervised speech representation learning from audio and video (AV-HuBERT). Learning such high-quality speech representations also enabled our research on the Generative Spoken Language Modeling (GSLM) tasks. I’ll share our recent advances on that front.
About the speaker:Abdelrahman Mohamed is a research scientist at Facebook AI Research (FAIR). Before FAIR, he was a principal scientist/manager in Amazon Alexa and a researcher in Microsoft Research. Abdelrahman received his Ph.D. from the University of Toronto, working with Geoffrey Hinton and Gerald Penn as part of the team that started the Deep Learning revolution in Spoken Language Processing in 2009. He is the recipient of the IEEE Signal Processing Society Best Journal Paper Award for 2016. His research interests span Deep Learning, Spoken Language Processing, and Natural Language Understanding. Abdelrahman has been focusing lately on improving, using, and benchmarking learned speech representations, e.g. HuBERT, Wav2vec 2.0, TextlessNLP, and SUPERB.Watch Online |
Friday, March 11, 2022, 11:00AM
|
Trustworthy Human-AI PartnershipsGopal Ramchurn [homepage]
Recent advances in AI, Machine learning and Robotics have significantly enhanced the capabilities of machines. Machine intelligence is now able to support human decision making, augment human capabilities, and, in some cases, take over control from humans and act fully autonomously. Machines are becoming more tightly embedded into systems alongside humans, interacting and influencing each other in a number of ways. Such human-AI partnerships are a new form of socio-technical system in which the potential synergies between humans and machines are much more fully utilised. Designing, building, and deploying human-AI partnerships present a number of new challenges as we begin to understand their impact on our physical and mental well-being, our personal freedoms, and those of the wider society. In this talk I will focus on the challenges in designing trustworthy human-AI partnerships. I will explore the multiple elements of trust in human-AI partnerships and discuss the associated research challenges.I will also aim to identify the risks associated with human-AI partnerships and therefore determine the associated measures to mitigate these risks. I will conclude by giving a brief overview of the UKRI Trustworthy Autonomous Systems Programme (www.tas.ac.uk), a £33m programme launched in 2020 involving over 20 universities, 100+ industry partners, and over 200 researchers.
About the speaker:Prof. Sarvapali Ramchurn is a Professor of Artificial Intelligence, Turing Fellow, and Fellow of the Institution of Engineering and Technology. He is the Director of the UKRI Trustworthy Autonomous Systems hub (www.tas.ac.uk) and Co-Director of the Shell-Southampton Centre for Maritime Futures. He is also a Co-CEO of Empati Ltd, an AI startup working on decentralised green hydrogen technologies. His research is about the design of Responsible Artificial Intelligence for socio-technical applications including energy systems and disaster management. He has won multiple best paper awards for his research and is a winner of the AXA Research Fund Award (2018) for his work on Responsible Artificial Intelligence. He has pioneered the development of AI-based disaster response systems using multi-UAV systems, AI-driven large-scale battery management for smart grids, and an AI bot that outperformed more than 5M human players (top 0.7%) in the English Premier League Fantasy Football Tournament. His papers have been cited more than 9000 times (according to Google scholar). He is originally from Mauritius and is interested in promoting applications of AI for environmental sustainability.Watch Online |
Friday, March 25, 2022, 11:00AM
|
Outracing Champion Gran Turismo Drivers with Deep Reinforcement LearningPeter Stone [homepage]
Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world's best e-sports drivers. We combine state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing's important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world's best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.
About the speaker:Dr. Peter Stone is the David Bruton, Jr. Centennial Professor and Associate Chair of Computer Science, as well as Director of Texas Robotics, at the University of Texas at Austin. In 2013 he was awarded the University of Texas System Regents' Outstanding Teaching Award and in 2014 he was inducted into the UT Austin Academy of Distinguished Teachers, earning him the title of University Distinguished Teaching Professor. Professor Stone's research interests in Artificial Intelligence include machine learning (especially reinforcement learning), multiagent systems, and robotics. Professor Stone received his Ph.D in Computer Science in 1998 from Carnegie Mellon University. From 1999 to 2002 he was a Senior Technical Staff Member in the Artificial Intelligence Principles Research Department at AT&T Labs - Research. He is an Alfred P. Sloan Research Fellow, Guggenheim Fellow, AAAI Fellow, IEEE Fellow, AAAS Fellow, ACM Fellow, Fulbright Scholar, and 2004 ONR Young Investigator. In 2007 he received the prestigious IJCAI Computers and Thought Award, given biannually to the top AI researcher under the age of 35, and in 2016 he was awarded the ACM/SIGAI Autonomous Agents Research Award. Professor Stone co-founded Cogitai, Inc., a startup company focused on continual learning, in 2015, and currently serves as Executive Director of Sony AI America.Watch Online |
Friday, April 8, 2022, 11:00AM
|
Unifying segmentation, tracking, and forecasting for 3D embodied scene understandingAljosa Osep [homepage]
The future of AI is embodied. Think Alexa but with a robot body that can actually help out at home, or robotaxis to easily take you wherever, whenever, and however. But how do we get there? From the perception point of view, we first need to answer questions such as where I am or where I need to go, what is around me, and how it moves.
I will start with the question of what is around me and present our recent efforts toward the unification of temporal 3D semantic and instance segmentation for streaming lidar perception. I will then zoom into the temporal aspect of this task, tackling the question of how does it move and talk about how far we can get in 3D object tracking by purely relying on modeling geometric relations using graph neural networks. I will continue with how it could move and discuss how end-to-end forecasting can be posed as future object detection. Finally, I will conclude my talk with the question of where I need to go and talk about city-scale position localization based on textural descriptions of the visual surroundings.
About the speaker:Aljosa Osep is currently working as a postdoctoral fellow at the Robotics Institute of Carnegie Mellon University in Pittsburgh and Dynamic Vision and Learning Group at the Technical University in Munich. When not exploring the world, he is working on cool research problems that lie on the intersection of computer vision, robotics, and machine learning. His current focus is on scaling object detection, segmentation, and tracking methods to the open world, in which future robots will need to operate.Watch Online |
Friday, April 15, 2022, 11:00AM
|
A Dimensional Model of Interaction Style Variation in Spoken DialogNigel G. Ward [homepage]
In spoken dialog people vary their interaction styles, and dialog systems should also be able to do so. Previous work has elucidated many aspects of style variation and adaptation, but a general model has been lacking. We here describe a dimensional model of the space of interaction styles, derived by applying Principal Component Analysis to a large sampling of American English conversations, each represented by 84 novel features that encode the frequencies of diverse interaction-related prosodic behaviors. The top 8 dimensions were meaningfully interpretable, and include aspects previously noted in the literature, such as engaged vs. uninterested, but also new ones, such as positive assessment vs. negative feeling and factual vs. thoughtful. Further, regarding individual differences in interaction style, we find that individual style tendencies were surprisingly weak, with a predictive model based on individual tendencies only barely outperforming a speaker-independent model. There are interesting implications for dialog systems design, tuning, and adaptation.
About the speaker:Nigel G. Ward is Professor of Computer Science at the University of Texas at El Paso. He received his Ph.D. from the University of California at Berkeley. On the faculty of the University of Tokyo for ten years before joining UTEP, in 2015-2016 he was a Fulbright Scholar and Visiting Professor at Kyoto University. He is chair of ISCA’s Speech Prosody SIG and the author of Prosodic Patterns in English Conversation. He is organizing the Special Session on Speaking Styles and Interaction Styles at Interspeech 2022. Ward's research areas are at the intersection of spoken language and human-computer interaction. Current topics include the subtle non-lexical and prosodic signals that enable inference of a dialog partner's needs, intentions, and feelings at the sub-second level; and ways to model and exploit these phenomena to improve dialog system responsiveness, information retrieval from audio, and language assessment and teaching. These projects apply multiple methods: linguistic, corpus-based modeling, systems building, and experimental.Watch Online |
Monday, August 1, 2022, 11:00AM
|
Program Synthesis, Program Semantics, and Large Language ModelsCharles Sutton [homepage]
I will describe our experience with two generations of large language models for code at Google. These models show a range of abilities, including generating small programs from natural language descriptions and engaging in dialog about code, incorporating human feedback to improve solutions. However, in a deeper sense, these models seem not to understand the code that they write, in the sense that they are generally unable to predict the output of a program given a specific input. I will discuss our subsequent efforts to improve the "code understanding" abilities of LMs, by asking them to emit intermediate computation steps as tokens onto a "scratchpad". These same models are able to perform complex multi-step computations when asked to perform the operation "step by step", showing the results of intermediate computations, even operations that the LM could not perform directly.
About the speaker:Charles Sutton is a Research Scientist at Google Brain and a Reader (equivalent to Associate Professor: http://bit.ly/1W9UhqT) in Machine Learning at the University of Edinburgh. He has published over 75 papers in probabilistic machine learning and deep learning, motivated by the demands of a broad range of applications, including natural language processing (NLP), analysis of computer systems, sustainable energy, data analysis, and software engineering. His work in machine learning for software engineering has won two ACM Distinguished Paper Awards. He has served as Director of the EPSRC Centre for Doctoral Training in Data Science at the University of Edinburgh, and has previously been a Fellow of the Alan Turing Institute, the UK’s national research institute for artificial intelligence and data science. |