Required Features for Virtual Mannequin Milestone 2

Due: Apr 4, 2024

This page requires JavaScript to display formulas correctly.

Virtual Mannequin Milesone 2

In this milestone, you will implement a simple animation editor for posing a virtual character into several keyframes, and then automatically interpolating the results.

The assignment is broken down into several features described below. Each feature is worth a number of points and will be graded separately (although later feature may build on and require a working implementation of earlier features). Your code should be correct and free of segfaults, deadlocks, or other buggy behavior. You will not be graded on software engineering (use design patterns and unit tests if they help you) but unreadable code is less likely to receive partial credit. You should not worry too much about performance or optimizing your code, but egregiously inefficient implementations of any of the features will be penalized.

(0 pts) Sanity Check and Build

Your submitted archive should build smoothly when make-skinning.py is run and should yield a web package that executes without fatal errors on common browsers (Chrome; Firefox; etc). Code that does not build or run will lose anywhere between 5 to 100 points, depending on the time and effort it requires the TA to fix your code. It is your responsibility to thoroughly test your submission.

(0 pts) Changes to Your Milestone 1 UI

You will need to make some changes to the Milestone I UI, so that there is enough space for new features.

• Increase the width of your app's canvas by 320 pixels. The new strip of canvas on the right will be where you render keyframe previews.
• Change the OpenGL viewport so that your Milestone I code only renders to the main view region and not the preview area on the right.
• Update your GUI class to make it aware of this new partitioning of the canvas.

(40 pts) Keyframe Animation

For this feature, you need to implement some basic control for adding keyframes, and playing an animation interpolating keyframes. Specifically, you need to implement the following keystroke functions:

• K key: Create a keyframe: save the current skeleton pose to a data structure that can later be used for animation (see below). You will need to maintain an ordered list of keyframes; append to this list each time the K key is pressed.
• P key: The first time this key is pressed, start playing through the keyframes you have recorded: interpolate between the poses recorded in the keyframes, spending one second of wall clock time animating the interval between each consecutive pair of keyframes. Hitting P again during this animation should abort the animation.
• R key: Resets the pose of the skeleton to its rest pose and deletes all keyframes.

Each keyframe consists of a set of joint rotations \(T_i\) (and, optionally, root joint translations, if you have implemented these). To compute a pose in between two keyframes, perform spherical linear interpolation (SLERP) on the rotation at each joint (and ordinary linear interpolation, LERP, on the root translations). If you do this interpolation correctly, joints will rotate with constant angular velocity from one pose to the next. There should be no "jumps" in the positions of the joints (or the vertices of the model)!

More details

There are several reasonable data structures and algorithms for storing and interpolating keyframes. Here's one possible approach:

1. To implement kayframe animation, store the joint orientations at each keyframe. Record the rotations performed by the user in some type of keyframe data structure. If the user rotates a bone, only the orientation of that bone should be changed (and not the descendants). For example, if you rotate your shoulder joint, the entire arm moves, but only the shoulder joint (and not the elbow, wrist, etc) has changed orientation. The set of orientations over the entire skeleton is the data you need to save per keyframe.

2. Code up a function that takes in a set of per-bone orientations and computes the corresponding skeleton configuration, in a way that lets you reuse the skinning algorithm to show the skeleton, and model, for an arbitrary pose specified by these joint orientations.

3. To render the character at a time \(t\) in between two keyframes, use SLERP to interpolate the orientations of the two keyframes straddling time \(t\). Once you compute the interpolated rotation at each joint, you should be able to reuse the function implemented in Step 2 to render the model at any time \(t\). As noted above, your animation should play at a rate of one keyframe/second.

If you are uncertain about how the keyframing is supposed to work, the online reference solution has an implementation of the above keyframe animation features.

(30 pts) Keyframe Preview

Once you have implemented the functions from the previous section, you can record and play animations consisting of an arbitrary sequence of keyframes. But it is difficult to craft an animation without a visualization of the keyframes you have already saved---the features in this section make this task easier.

Implement the following keyframe preview feature:

• Render each key frame to a texture
  • Tip: A common mistake is forgetting to call gl.clear before rendering to the texture. The data in the newly created framebuffer is undefined and it very likely consists of depth data with large magnitude. Hence your depth test will always fail and nothing will be rendered to your texture.
• Draw the sequence of textures vertically in the blank panel you added at the right of the main view. The easiest way to do this is to perform a second rendering pass consisting of a sequence of textured quads (one per keyframe you wish to preview). Ensure the quads appear in the right place on the screen by using an Orthographic Projection Matrix and placing the quads in the right place in the viewing volume.
• Optionally, add a way to scroll through the preview keyframe panel if there are too many keyframes to fit on the screen.

It is well worth becoming comfortable with the technique of rendering to a texture, rather than the framebuffer: is is a key component of many effects including shadow mapping, reflections, and portable anti-aliasing.

(20 pts) Keyframe Management

The basic keyframe controls are unforgiving if you make a mistake, or wish to update/replace an existing keyframe. You will next implement a set of features that provides the user with a more sane UI.

Detect mouse events within the right preview panel, and implement the following functionality:

• Left-clicking on a keyframe preview should select that keyframe. Highlight the currently-selected keyframe.
• When the user presses the U key, update the currently selected keyframe (replace the stored joint orientations with the current ones).
• Delete key: delete the selected keyframe.
• =: set the character's pose (in the main window) to that stored in the selected keyframe.

Optional Features

Implement at least 10 points of optional features. You may implement more for extra credit.

The list below contains pre-approved optional features (worth five points per 🎐 and ten points per 🔔)

All optional features must be fully described in your README file (including instructions for how to invoke the feature behavior) to receive credit.

• 🎐 Extend the GUI to allow not only bone rotations, but also translation of the root joints.

• 🎐 Draw a timeline in the status bar area of the window. Show a slider moving across the timeline as the animation plays. Indicate keyframes with some kind of tick mark on the timeline.

• 🎐 Add the ability to save/load an animation to/from a clipboard string. It should be possible to create an animation, save the string, close and reopen the browser, and completely recover the saved keyframes from the string.

• 🎐 Implement texture-mapping. This will require you to use the provided image loaders to read in a bitmap from file, load the texture into the GPU using the provided RenderPass texture functionality, and then add a texture-mapping fragment shader that samples the bitmap. I've provided a texture map for the simple cube (scene 4) if you'd like a test case.

• 🔔 Find or create your own character model, rig, and skinning weights, and create some images showcasing different poses of the character.

• 🔔 Implement shadow mapping to render the character's shadow on itself, and on the ground.

• 🔔 Research and implement dual-quaternion skinning, a more sophisticated and robust skinning algorithm that does not suffer from the "candy-wrapper" artifact.

• 🔔 🎐 Allow the user to specify arbitrary durations (not necessarily one second) between keyframes. Visualize the time gaps on a timeline as part of the program GUI and allow the user to change the spacing of the keyframes interactively (including reordering the keyframes).

• 🔔 🔔 Allow the user to move the camera and lights, and keyframe the motion of these objects.

• 🔔 🔔 Add an option to interpolate poses using a spline, or another advanced blending technique, rather than piecewise linear interpolation.

• 🔔 🔔 🔔 Create a third-person interactive walking simulator featuring the provided robot. Allow the user to move the robot around the world with the mouse and arrow keys; as the robot moves, animate its legs and arms using a precomputed walk cycle adapted to the user's turning inputs. The third person camera should follow the robot in some reasonable way. Populate the world with objects/obstacles.

Creative Scene (up to 20 pts extra credit)

You may save and submit particularly impressive animations that you create using your software system for extra credit. Put videos of your animations on YouTube and carefully describe any scenes you want considered for extra credit in your README file.