Taking Home the Audience Choice Award!

After two quarters of hard work, we finally arrived at a polished product, complete with rotation, zoom, swiping off layers, and a menu interface. We felt prepared and ready for our moment to shine at the 2014 iSchool Capstone event, hoping to bring home at least an Audience Choice award for our novel application.

With our toughest features implemented, we hauled our 55″ television, prism, and tent over to the HUB Ballrooms for the final capstone event. Set up was quite a busy time, with a tent to assemble, television to stabilize, prism to align, and business cards to print. By 6 pm we were ready to show interested visitors into the dark tent which blocked interfering light from our volumetric display. By 7 pm, we had a line of people waiting to see our capstone project!

The iSchool Capstone event, decorated with our huge black tent and line

The iSchool Capstone event, decorated with our huge black tent and line

We were in disbelief of the popularity of The Ghost Anatomy Project, which we were told caught the attention of the entire room. People we didn’t even know were telling visitors that this was the project to see at capstone. With such a huge line, we split tasks to effectively manage the event, and share our project with as many people as possible. Ted ushered in visitors in groups of three, while Ngoc showed people inside the tent how to use the interface, and Ashish, Connie, and Alyssa talked to judges and visitors in line. It was incredible gaining the support and excitement of project supporters, friends, family, and complete strangers. We received many compliments on how cool and exciting the project was.

The Ghost Anatomy Project Business Cards

The Ghost Anatomy Project Business Cards

Our hard work paid off when we heard “The Ghost Anatomy Project” announced for one of the four Audience Choice Awards! We were proud to join Fidget, MIST, and Waitless on the stage. Being recognized as one of the best capstone projects was a shining moment to end our college careers. With this chapter of The Ghost Anatomy Project successfully closed, we plan on sharing our interface with Trond Nilsen, our collaborator from the UW Structural Informatics Group, and discussing whether or not to continue developing features for the anatomy application.

We couldn’t have accomplished what we did without the help and support of all of our collaborators, teachers, user testers, friends, and family. A huge thanks to everyone who made this insanely awesome project possible, especially Trond Nilsen, Matt Turner, Francis Nguyen, and David Ruiz. And a huge thanks to everyone on the team, who pulled through all-day and all-night work sessions, early morning wake ups and late night deadlines, user testing on every possible corner of campus, and countless trips to Home Depot and Lowe’s for hardware parts. The most successful projects come from the most successful teams, and in this project, not only did we find successful team, but a loving group of friends. It has been nothing but a pleasure to work with this team the entire year. We did it!

 

Signing Off,

The Ghost Anatomy Project

 

Your Audience Choice Award winners, The Ghost Anatomy Project!

Your Proud Audience Choice Award Winners, The Ghost Anatomy Project!

Alyssa (5)

Ashish (5)

Connie (5)

Ngoc (5)

Ted (5)

 

 

Advertisements

We’re Ready for the Poster Event!

With the capstone event underway, we have created a poster to share our hard work with the community. The capstone event will be the accumulation of all the hard work we’ve completed these past two quarters. We are extremely excited to present our work, and look forward to seeing you all at the event on Thursday, June 5th, from 6 p.m. to 9 p.m in the HUB Ballroom!

After hours of work in tinkering in Illustrator, here is the poster we have created for the event! For full resolution, check out the pdf. With a few more touch ups left to do, we are almost fully prepared for the event tomorrow night. A huge thanks to every single person who has put time into helping us develop this interactive 3D application. Special shout outs to Trond Nilsen for providing us the base of our code, and taking so much time to help us with integration, and Matt Turner for helping us laser cut the pieces for our plexiglass prism. We are very proud of our work these past two quarters, and look forward to the event tomorrow!

The Ghost Anatomy Project Poster

The Ghost Anatomy Project Poster

Point Breakdown

Alyssa (5)

Ashish (5)

Connie (5)

Ngoc (5)

Ted (5)

Welcome the Minimum Viable Product

Goals

With the full-sized prism finally built, we were able to test our application on its intended platform. We aimed to achieve a minimum viable product. For our display, that meant holographic-like projections on all four sides of the prism, basic rotation gestures, and basic zooming functionality.

Process

The first step in modifying our web application to make the display look believable was changing the background color of the panels to black. With black backgrounds, the 3D anatomy models appear to be floating. Once we made quick modifications to the code we had been developing all quarter long, we tested our code on the plexiglass prism, and discovered we had a minimum viable product ready to go! We were considering rotating the canvas of the web application to more efficiently utilize the screen realestate, but upon testing decided to stick with our default canvas to maximized image size in the display.

10301276_321493831334129_1141399167565397332_n

Our interface is alive!

Results

While several features are left to be implemented, we have achieved a presentable interface ready for the capstone event. This quick demo video should give a flavor of what we’ll bring to the table these next upcoming weeks!

Next Steps

While we have achieved a presentable product, we are far from being at the point where we would call our application anywhere close to complete. We are currently working on implementing more gesture features to the application, including highlighting body parts and removing organ systems. The interactive Leap Motion menu is also a feature we would like to complete by the capstone presentation. Next week, we will also meet with Trond to see how far we can get with integrating a quizzing function into the interface. There is much work to be done, and we’re hoping to have as much done as possible!

Getting the highlighting function to cooperate

Point Breakdown

Ashish (5)

Ngoc (5)

Late night development parties

Building the Volumetric Display

Goals

The magic of our interface lies within the plexiglass prism which creates the illusion of a hologram when an image from an LCD screen is reflected onto it. In order to bring our interface to life, we needed to obtain a large monitor and build a matching plexiglass prism. The process came with challenges, but in the end, reaped great rewards.

Process

The first thing we had to do was consider our main limitations for prism and monitor size. We bought enormous sheets of plexiglass at the beginning of the quarter, and ideally wanted to maximize the height of the plexiglass for our display. A larger prism would require a larger monitor size, however, and deeper pockets than college students normally have. After some research and consideration, we decided that a 55″ television monitor would be sizable enough for our capstone display. We calculated the corresponding plexiglass piece to be 19″ in height.

10377362_321492934667552_8138987569824061914_n

Performing calculations for the plexiglass prism

While an Xacto knife worked for cutting the plexiglass for our small 5″ tall prototypes, we knew it wouldn’t be the cleanest solution for our large final display. We knew the University had to have laser cutters on campus available to students, so we reached out to the art and engineering departments, and attended their laser cutting orientation. After attending the laser cutting orientation, we unfortunately discovered that our piece of plexiglass was much larger than the laser cutting bed. After asking around for a larger laser cutter, we were put in contact with Matt Turner, a Physics grad student who runs the laser cutter at the Center for Experimental Nuclear Physics and Astrophysics. To our delight, he offered to cut our plexiglass pieces for free with his laser cutter! We were incredibly gracious for his help with the construction of our display.

With perfectly cut plexiglass trapezoids in hand, all that was left was to glue the pieces together into a prism. Using painter’s tape to hold the prism together and prevent silicone glue from getting all over the prism, we assembled the prism. Two pieces were glued together at a time, with a box corner holding the prism to a 90 degree angle during the drying process. After the two pieces dried, we pushed those pieces together and glued the two final edges together. Since the silicone glue dried like flexible rubber, several edges had to be reglued again and again. The silicone glue worked finely for our smaller plexiglass prism, but was not as successful at holding together our larger prism in a stable manner.

A drying half of the prism, complete with painter’s tape and a box corner to hold the 90-degree angle

Results

After several calculations and shopping trips to Lowe’s, we finally brought the Ghost Anatomy Project to life with our 19″ prism and 55″ monitor!

19" tall prism and 55" monitor

19″ tall prism and 55″ monitor

The images from the 1080p monitor project beautifully onto the plexiglass prism, giving the illusion of a 3-dimensional image sizable enough for anatomy students to study.

The Birth of The Ghost Anatomy Project

The Birth of The Ghost Anatomy Project

Next Steps

To fix our wobbly prism, we have decided to buy a large 24″ x 24″  piece of plexiglass to glue on top of our prism to act as a stabling lid. If the lid does not completely stabilize the prism, we are planning to switch to Epoxy instead of silicone glue. We also noticed that our display is most vibrant in the dark. We obtained permission from Tori to bring a tent to the capstone event next week, and are in the process of constructing the tent from pieces of PVC pipe and black fabric.

Hardware Shopping List:

1x 24″ x 24″ square piece of plexiglass for the prism top
3x 6.5′ long pipes
4x 5′ long pipes
4x 7′ long pipes
6x 3-pipe corner connectors
2x 2-pipe corner connectors
Black Fabric/Tarp

Diagram of tent dimensions

Diagram of tent dimensions

Point Breakdown

Ashish (5)

Ngoc (5)

 

Revised Success Definition and Measurements

Goals

Now towards the end of our project, we are trying to see whether if our application has met the initial success definition and also make some revision for our success definition.

Results

Our application has met most of the metrics of success we defined. There are some metrics that we can not measure our success with due to time limitation. And there are also some goals that we haven’t achieved due to other reasons.

Deliverable

We created a document with more detailed explanation. The content of this document is subject to change as development continues. Click here to check out the document.

Point Breakdown

The tasks were equally distributed to all members.

Ashish (5 / 25)
Ngoc (5/ 25)

Alyssa (5 / 25)

Connie (5 / 25)
Ted (5/ 25)

Create App Specification

Goals

Now that we have bridged the majority of the learning gap, we have come to a point in the project in which the focus will be development. In order to proceed in an organized fashion, we decided to write loose application specification, one that would be able to change as our knowledge of the framework grew.

Results

We created a document that details the parts that need to be completed by capstone in detail. It should serve as a guideline so that all developers can stay in sync and be able to gain from each other’s knowledge/learning.

Deliverable

We created a specification linked here. The content of this document is subject to change as development continues.

Point Breakdown

Each group member contributed ideas to the functionality of the application. Ashish, Ngoc, Ted, and Connie focused on the the technical aspects of the application where Alyssa focused on making sure the design principles would hold up in the implementation.

Ashish (5 / 25)
Ngoc (5/ 25)

Alyssa (5 / 30)

Connie (5 / 30)
Ted (5/ 30)

Development Meeting with Trond (2)

Goals

The goal of this meeting was to present the gesture code that we had been assigned to work with in the previous week and integrate it into Trond’s framework.

Results

We were successfully able to add recognition code to the main controller of the application. In addition we were able to integrate pinch-zooming fully into the graphics engine which allowed us to simply make an API call to resize the model on screen.

Deliverable

Gesture recognition code added to handle detection of leap gestures

Gesture recognition code added to handle detection of leap gestures

Point Breakdown

 

Each team member worked on gesture code or conferred with Trond about integrating our leap motion code into the graphics manipulation engine.

All members equally contributed to the reading of Trond’s code base and helped him to integrate gesture recognition.

Alyssa(5)

Ashish(5)

Connie(5)

Ngoc(5)

Ted(5)

1006099_310053135811532_8377368112038668117_n

One big, happy development family.

Design Document

For this milestone, we created our application design document. We want to provide explicit information about the requirements for out project and how the project is put together. Our design spec contains several main parts including problem statement, project scope, platform specifics, software specifics, personas, use case, and application details. These specifications are made in order to help us to have more clear direction when we are developing the application.

 

2menupopup-11

We achieved this milestone by  gathering and formulating what we have done throughout the past 15 weeks. After conducting user research with our hi-fi prototype, we have gathered all the necessary information we need to design our application. And when we were exploring the leap motion and playing with 3D models, we found out some limits of the technologies we are using. For examples, the groping motion to take off layers doesn’t not work with leap motion while the sensor can not recognize users’ fingers when they get too close to users’ palm; and also leap motion can not differentiate the swiping motion and the horizontal movement of cursor.

Therefore, throughout the process of this milestone, we have iteratively redesigned our application. The dev team and the design team have collaboratively discussed about how to implement the most user-friendly design under the technology constrains. While the design of the application eventually has to compromise our limited time and budget, this deliverable helped us to recognize and reassure our project scope and priority tasks.

Point Breakdown

The tasks were equally distributed to all members.

Connie(5): Covered the scope of the project and assumption of the users.

Ashish(5): Covered the software specification.

Ted(5): Covered users and persona.

Alyssa(5): Covered application details.

Ngoc(5): Covered hardware specifications.

Deliverable: Design Document

Tagged

High-Fidelity User Testing

Our design for the 3D interactive interface is on the verge of usable design, but we can’t be sure until we test! To test the high-fidelity prototype and diagnose any possible design flaws, two anatomy students who had never been exposed to the project or interface were selected. Both students, Rebecca and Susan, were in graduate school for a medical-related field, and had taken intense anatomy courses in their studies. Both were students at the University of Washington.

User Testing Method 

Both users were tested at a local library with minimal distractions and a quiet work area. The high-fidelity prototype consisted of two parts: one to test the gestures, and one to test the user interface. The gestures were tested with a Leap Motion web prototype, and the user interface was tested with video simulations. Each user tested each part consecutively during their testing session. Users were compensated with candy.

The Leap Motion gestures were tested with a coded prototype built in JavaScript with the Leap Motion API and three.js Library. Existing tutorial code for Leap Motion gestures, and examples from Robbie Tilton’s Reflective Prism demo were used to hack the prototype together. The gestures tested on the interactive demo were rotation, zoom, and pointing.

To test the sanity of the menu options, users were asked to interact with video prototypes mirroring actions of the interface. The user would point to the screen, as if it were the 3D interface, while the tester moved the mouse to act as the system. The user would be guided through a set script carefully planned with the video. The video would reflect the ideal actions that would be performed by the user. While one tester guided the user and acted as the system, the other tester took notes or recorded the testing session.

User Testing Results

After testing users on the gestures and UI of the interface, several main issues need to be revised in the prototype before the final specifications are drawn for the Ghost Anatomy Project. Users struggled with the harsh rotation gesture, which took an enormous amount of effort and precision to get working. The rotation gesture should definitely be a lot smoother, and more natural to control. Swift interactions, and shortcuts were asked for by the users as well. For example, Rebecca asked for a one-motion shortcut to search for and select body parts. Most users had to be told how gestures worked in order to interact with the interface. Some way to explain the gestures to the user, or indicate the gestures with affordances in the interface, would be most beneficial.

The user interface could also be improved to better suit the users. Users did not notice the menu, and did not look to it to perform functions during testing, such as turning labeling on or off. Both asked for a visible zoom indicator. Since the UI interactions were only simulated, the exact gestures and interactions with the UI will have to be tested and refined as the actual interface is built.

Next Steps

For this step, each member was involved in the user testing and report compilation. With these results, the design team will iterate on the high-fidelity prototype again, and refine it for the final Design Document detailing the design details of the application.

Alyssa (5)

Connie (5)

Ted (5)

Deliverable: High-Fidelity User Testing Report

Development Meeting with Trond

Goals

Accomplishing a task such an incredible technical feat as rendering and displaying a human body and identifying all the individual components is no small endeavor.

For this reason, we requested the help of Trond Nilsen, a quite brilliant software engineer. Trond is a graduate student who’s research is closely aligned with ours. He has already built the model rendering and viewing into his application. We are currently having weekly meetings to work on upgrading his current project to support the Pepper’s Ghost illusion/interface. The process will require splitting the render into multiple views, and passing gestures in as function calls to manipulate his view. 

 

Results

Every Monday, we meet with Trond. After the first session we were able to add all of the math calculations and adjustments to Trond’s engine to allow for 4 separate Three.js cameras of a single model.

After setting up a local web server, the models can be fed in and viewed as such.

Hologram split

View Split for Hologram Use

If one were to place a prism on the screen with this  application running, you would be able to see the illusion of a floating, 3D human body.

 

During the next meeting we able to integrate the Leap Controller and get a data stream from the pinch-to-zoom gesture. Before the next meeting, we hope to be able to get all of the gestures implemented and added to the application.

 

Deliverable

All of the new application code we created was pushed to a GitHub repository.

Open source and available here.

To download and test, use an apache/nginx web server.

Your browser must be compatible with webgl to view.

 

Point Breakdown

Each team member spent some time with Trond throughout each work period and helped integrate our ideas with Trond’s framework.

All members equally contributed to the reading of Trond’s code base and helped him to integrate view splitting and leap controls.

Alyssa(5)

Ashish(5)

Connie(5)

Ngoc(5)

Ted(5)