Each of these demonstrations is created through RL-Applet, which is built upon RL-Glue and RL-Library. Source code and installation instructions for RL-Applet can be found here. If you have Java implementations of RL-Glue compatible agents or RL-Viz-compatible task environments and would like to see them up here, please do let me know.
Enjoy!
Museum curated by Brad Knox
(The Java applet code used here is buggy with certain Linux installations. Sorry about that. If it really bothers you, feel free to download the source and get to the bottom of it.)