Status Report 10, or How I Learned to Stop Worrying and Love CV

As of tonight, everything works. Sometimes temperamentally, but I'm 99.9% sure that's the result of imperfect lighting conditions and miscellaneous camera quirks. I'm happy with it, at any rate. I may go back and tweak more later if I feel like it.

A Summary of my Time Spent with the Mitosis Game

I started this game in an attempt to teach children about mitosis in a new way that was more interesting than textbook learning. However, in the process I ended up teaching myself a lot about using other people's code, and using Flash in general. Some of the high points:

- Learning how to use the iHart system. This was obviously the major point of this exercise.
- Learning to code and implement my own events and event dispatchers.
- Implementing the refShot() function.
- Untangling all the iHart code to code my own pseudoserver.

The low points, of course, occured in the process of convincing all this to work; however, I think I've learned that with persistence and planning, things will eventually work out the way you want them to. It's been fun, guys.

Status Report 9.5

Status Report 9

ALL STAGES ARE CURRENTLY FUNCTIONAL WITH THE PSUEDOSERVER!!!!!!!!

Well, that's a relief. Also resolved is one issue in which two DNA bases would sometimes snap to their final positions at once--this occured when two bases whose final positions were adjacent appeared one after the other. To fix this, I moved the bases further apart vertically and changed the hit area so that only the nucleotide itself (not the sugar/phosphate backbone) counts as a "hit." Unfortunately this adds the problem of me not being tall enough to reach the top bases, but assuming we can run this thing from a ceiling-mounted projector that shouldn't be a problem in the long run.

Next week: camera debugging. I do hope we have tall enough chairs.

Status Report 8

I got the reference shot method to work. Turns out the problem was that Flash takes a few seconds to send the data to the camera, so the new frame doesn't actually appear on the screen until after the refShot() method is called. If I set a slight delay between calling the method and actually taking the shot, it works. I'll be experimenting in my spare time to figure out exactly how long it usually takes and how long I should set the delay for.

In other news, everything up to and including metaphase works fine with the psuedoserver, so I'm very happy about that. Another problem that has cropped up lately is that if I enlarge the window to run it on the projector, the positions of certain objects get really messed-up. I think I can get around this by hardcoding the coordinates, but there's probably a better option I haven't thought of yet.

Status Report 7.1 - Eureka!

I have found a solution to one of my reference shot problems. The problem was that Flash's XMLServer, when its send() method is called, adds a zero bit to the end of whatever was sent, which was messing up the String.equals() method in Java. Short of rewriting Flash's XMLServer code, I discovered that if I use the String.startsWith() method instead of the equals() method, it works fine. Success!

The new problem is that since the server is not lightning-fast, the reference shot that gets sent is a shot of the screen a few seconds before the method call. This is problematic since the only time the method gets called is during a frame change. I guess I could program the Flash side to wait a few seconds before asking for a reference shot, but I'm going to speak to Audrey in a few minutes to see what she thinks.

Status Report 7

Still on track, as the G1, S and G2 phases are now done. I am very happy about this. What I am not so happy about is my inability to write a function that triggers the server to take a new reference shot upon a method call to the CVManager. Basically what's happening is that the psuedoserver gets the message and responds (by printing out a specific string), but the general server acts like it hasn't gotten the message. I do not know why this is, but I will continue to plug away at it as I proceed with the rest of my project.

Status Report 5/6

So apparently Status Report 5 didn't actually go through when I hit publish. Oops. In lieu of typing everything again I'll just summarize what Status Report 5 actually said: all the graphics are done and I'm starting on coding now. Yay!

Now for Status Report 6: The G1, S and G2 phases are now totally functional. In addition, I have coded a custom event handler (I'm very proud of myself) to get the game to progress on its own. Unfortunately this also means I can't run it on the projector until I have a convenient way of taking new reference shots. For right now I'm just elated that it works at all.

Status Report 4

On schedule with the mockups for metaphase and anaphase finished. All seems to be going according to plan.

Pictures:

Status Report 3 and some more pictures

I'm finally back on schedule after being sick last week, and the mockups for prophase and prometaphase are finished. In addition, some of the skeleton classes are no longer so skeletal; in particular, I have the G1 phase working ahead of schedule, and the main classes for the game are almost entirely fleshed-out. Below is a brief rundown of the classes I have:

MitosisGame: This is the main class for the program. It initializes the CVManager and creates a new Game instance.

Game: The wrapper class for all the different symbols that make up the game. It takes the CVManager in its constructor and proceeds through the different phases by keeping track of the current phase as an int (1 corresponds to G1, 2 to S and so on). It then uses a switch function to add the appropriate symbol.

Frame: The general parent class for all the symbols that make up the game. Each phase has its own Frame implementation. The Frame class takes the CVManager in its constructor, as passed to it by the Game class. Using this parent class allows me to define a variable in the Game class corresponding to the current symbol.

And now, pictures! Keep in mind that most of these are animated, which you can't see in the screenshots. If anyone knows how to embed a Flash animation into a Blogspot page, such information would be most welcome.

Prophase

This is just an informational frame. In the real frame the nucleus enlarges and the chromosomes slowly appear. I may do something interactive with this if I can figure out how.

Not animated at the moment, but it will be in its final incarnation. How I'm thinking of doing this is having the closest centrosome follow the onscreen Hull's x and y coordinates until it reaches an invisible "hit box" around its final position, at which point it snaps into place. This assumes, of course, that there is only one Hull in the area at the time--another possibility is using some sort of motion tween to make the centrosome move toward its final position if there is a Hull over its hit box.

Prometaphase

Also animated--in the real version, the nucleus really does "dissolve" (i.e. its alpha variable decreases).

This shot actually depicts the end result of the frame. The kinetochores start as little dots near the centrosomes and grow out into those things. Again, I'll either have to figure out how to get the CVManager to recognize the same Hull from shot to shot or have the frame react to a Hull in a specified area.

Teaser pics!

I've recovered from my illness, and now I have mockups of the G1, S and G2 phases, as well as skeleton classes. I'm not posting the skeleton classes, because they're boring, but here are some screenshots I took of the first three phases (scaled down to fit on the blog).

G1:
S (the interesting bit):
G2:

So there you have it: proof that I have been doing something this whole time. More should be forthcoming.

Status Report 2

Not much has gone on this past week due to me being half-dead.

What I do have, however, is a mock-up of G1.

Timeline/milestone document

Tentative timeline:

3/1: General class structure decided on, skeleton classes written

I will decide on the classes I will need for this project and how they will interact. I will also write skeleton classes (classes with all fields and methods declared but no methods filled in) for each class.

Also: mockups for phases G1, S and G2

These mockups will show what each stage will look like, but will not function. Some very basic functionality using a mouse may be implemented if I have time.

3/8: Mockups for prophase, prometaphase

3/22: Mockups for metaphase, anaphase

3/29: Mockups for telophase, cytokinesis

4/5: Main classes filled in and functional

All method bodies in main classes filled in, main classes run without errors

4/12: Functionality added to G1, S, G2 phases

All the named phases should work properly when tested with a psuedoserver. I may have to tweak Cleo's provided psuedoservers to get something suitable for testing.

4/19: Functionality added to prophase, prometaphase, metaphase

4/26: Functionality added to anaphase, telophase, cytokinesis

Testing and debugging with camera, circumstances willing

5/3: Presentation

Status Report 1

A few of the graphics for the Mitosis Game are done now. Mostly what I've been doing is trying to figure out the class structure for this program. I would like all the different minigames to share a single CVManager to put less strain on the processor, but I am not sure how to set the program up to accomplish this. Here is the solution I have so far:

1.) The CVManager will be instantiated in the main MitosisGame class when the game starts.

2.) The MitosisGame class will add a symbol (hereafter referred to as "Game") as a child of itself. This symbol will contain each minigame as a child within a single keyframe. The Game symbol will take the MitosisGame's CVManager as a parameter in its constructor. In this way the Game symbol will be able to share the CVManager with the main class.

3.) The symbols containing the minigames will all be subclasses of another Minigame class, which will take the Game's CVManager as a parameter in its own constructor. Within these symbols the CVManager will perform its minigame-specific functions.

This should work, if I'm thinking about it right... however, I have a nagging feeling that there is an easier way to do this.

Pitch Document

The Mitosis Game will be an interactive lesson to teach children ages 9-14 about mitosis. Mitosis is the process by which living cells divide and reproduce. By interacting with a Computer Vision based Flash movie, children will be able to learn about this process in a more engaging way than with common teaching methods.

A diagram of mitosis taken from https://dbscience3.wikispaces.com/Drew. " The G1, S and G2 phases are part of Interphase.

Core Gameplay

The Mitosis Game will consist of a series of mini-games illustrating the various stages of mitosis. Each game will provide an interactive demonstration of the core function of that stage. The mini-game for each stage will be as follows:

Stage G1: In this stage, the cell grows in size to prepare for mitosis. Children will form holes in front of the projection to "hug" the cell. If the hole intersects the image of the cell on the screen, the cell will enlarge in size. The game will proceed to the next stage once the cell reaches a specific size.

Stage S: In this stage, the cell duplicates its DNA. After a brief animated explanation of the process, children will be shown a half-strand of DNA along with a series of loose nucleotides. The children will be asked to match these nucleotides to their correct partners: adenine to thymine, and guanine to cytosine. This will be accomplished by having the nucleotides react to and move along with hulls. When all the nucleotides have been matched, the game will proceed to the next stage.

A diagram of DNA, illustrating the two base pairings, from http://www.breakitdownblog.com/bush-signs-the-genetic-information-nondiscrimination-act/. Children will construct a similar structure in the Mitosis Game.

Stage G2: This stage is virtually the same as G1. The cell grows further to ensure it is large enough to divide. Children will have to "hug" the cell again until it reaches its final size.

Prophase: During prophase, the centrosomes of the cell move to opposite sides of the cell. Children will have to move these centrosomes to their correct positions in a similar manner to that described for the nucleotides in the S stage. Once the centrosomes are in their correct positions, the game will proceed to the next stage.

A diagram of a cell in prophase taken from http://www.ias.unt.edu/~tpp001/Mitosis_Diagram_Page.main.html. Each pair of centrioles is collectively called a centrosome. Note that the two centrosomes are moving toward opposite ends of the cell.

Prometaphase: Two notable events occur in this stage. First, the nuclear envelope dissolves in preparation for the separation of chromosomes. The game will emulate this by having the child wave his or her hand over the image of the nucleus on the projection, causing the nuclear envelope to dissolve. Secondly, the centrosomes generate kinetochores, which reach toward the center of the cell. Children will be asked to move their hands toward the center of the cell to cause this to occur.

A diagram of a cell in prometaphase from Sparknotes' page on mitosis. Note that the nuclear envelope has dissolved into nuclear vesicles and that the kinetochores are extending toward the center of the cell.

Metaphase: In this stage, the chromosomes line up at the center of the cell. Children will have to align the chromosomes along the center line. Once the chromosomes are all in place, the game will move on.

A diagram of a cell in metaphase, from http://www.uic.edu/classes/bios/bios100/lecturesf04am/lect16.htm. Notice the chrosomes aligned at the center.

Anaphase: In this stage, the kinetochores shorten and pull the chromosomes apart. The children will have to "stretch" the chromosomes toward the opposite ends of the cell. The game will accomplish this by keeping track of the distance between two hulls on the screen.

A diagram of a cell in anaphase from iknow.net. Here the chromosomes are being pulled toward opposite ends of the cell.

Telophase: In this stage, the chromosomes reach their final positions at opposite ends of the cell and new nuclei are formed. Children will be asked to make holes at each end of the cell to "create" the new nuclei. Once each nucleus has been formed, the game will move to the next and final stage.

A diagram of a cell in telophase from http://www.biology.iupui.edu/biocourses/N100/2k2ch8mitosis.html. Notice that the new nuclei are beginning to form.

Cytokinesis: In this stage, the cell "pinches off" at the center and splits into two new cells. Children will be asked to form a "wall" between the two forming cells with their arms. The game will detect this as a hull and split the cell in response. Once the new cells are formed, the game ends.

A diagram of cytokinesis from Stanford University, illustrating the division of the two new cells.

Status Report 0

Looks like the winner for now is the Mitosis Game idea. The Google Maps idea might be implemented if I have time. The Google Maps API for Flash is here.

Pitching ideas

Here are the different ideas I have for a project:

Mitosis Game - A CV-driven game/interactive demo to teach people about mitosis. Each stage of the cell cycle would have a separate "mini-game" to illustrate the concept. Below are the ideas I had for each stage:

G1 - "hug" cell to make it grow to a certain size

S - catch the correct nucleotides to duplicate the DNA

G2 - "hug" cell again

Prophase - move centrosomes to opposite sides of the cell

Prometaphase - wave your hand to dissolve the nuclear envelope
"stretch" kinetochores toward the middle of the cell

Metaphase - line up chromosomes at the center

Anaphase - "stretch" chromosomes apart

Telophase - "stretch" cell some more, create new nuclei

Cytokinesis - "hug" the middle of the cell to pinch the cytoplasm

Google Maps - A Google Maps application that would compute the shortest distance between two points selected by the user. I have already had some experience using Dijkstra's Algorithm to find shortest distances, and I know Jim Teresco has created data sets containing a list of highway intersections. The user could then scroll the map and zoom in and out. The obvious problems here are:

-how to recognize that a user has selected a certain point without selecting every point the user touches

-how to recognize that a hull is moving so that the map can be scrolled and zoomed

Magnets - A game using the 2D physics engine. This would involve a maze with a metallic ball placed at the starting point. The user would control a magnet and try to guide the ball through the maze.