University of Texas A.I. Lab: Diagrams Research

This page describes several lines of research involving diagrams:

• ISAAC: a program that solves physics problems stated in English.
• VIP: a system to create programs from connections of diagrams that represent physical and mathematical principles.
• Views that describe, using diagrams, how user data implement mathematical concepts.
• The Figure Understander: a system for integrating text and diagram input to a knowledge base.

An on-line demonstration is available for several of these systems. The demos require a Unix workstation running X windows.

Isaac Program: Solving Physics Problems

The ISAAC program solves textbook physics problems stated in English. The problems are in the area of rigid body statics. The program draws a diagram from its understanding of the English problem statement. It makes a symbolic geometric model that is similar to the diagram and uses it in problem solving.

See the papers by Novak and Bulko for work on using English text and a diagram together as input to specify a physics problem.

Isaac Demo: This demo allows the user to solve one of 37 problems whose text is stored, or to enter a new problem. The demo requires a Unix workstation running X windows. Note: the program is not very robust for new problems.

Programming by Connecting Diagrams

The VIP program allows a user to write programs or solve physics problems by connecting diagrams that represent physical and mathematical principles. The diagram above shows a specification for calculating the mass of the sun.

Initially, VIP presents a mostly blank workspace containing only input and output variables. The user can select the Physics command to add boxes that represent physical principles to the workstace. Connections are made by clicking on "buttons" on the diagrams; a connection signifies that the variables represented by the buttons are equal.

A system such as VIP might be useful for teaching physics. While equations and algebraic manipulation are present, they are hidden from the user. The user focuses on conceptualization of the problem in terms of physical principles, rather than focusing on algebra, as students sometimes do.

VIP Demo: The demo requires a Unix workstation running X windows.

Making Views by Connecting Diagrams

The Automatic Programming Server can create specialized versions of generic programs based on views that describe how user data can be viewed as instances of abstract data. For mathematical abstract data, the views are specified by connecting a diagram of the mathamatical object to a menu of the user's data.

Automatic Programming Server: (*Under Construction*) Create Programs in Lisp or C. To see the diagrams, enter a description of mathematical data and then choose Make a View and Mathematical View. The demo requires a Unix workstation running X windows.

Integrating Diagram and Text Input

The following papers describe work by Raman Rajagopalan and Ben Kuipers.

• Picture Semantics for Integrating Diagram and Text Input, to appear in: AI Review Journal, Vol. 10, No. 3-4, 1995.
• The Figure Understander: A System for Integrating Text and Diagram Input to a Knowledge Base, 7th Intl. Conf. on Industrial and Engineering Applications of AI and Expert Systems, 1994.