Motion Capture & VR

I was doing some research for my ETC semester project which involves a motion capture when i came across the two projects I am going to discus.

Map Visibility Estimation For Marge-Scale Dynamic 3D Reconstruction:

This project is tracking movement and then generating the movement path of the objects dynamically. Markers are attached to the objects that what to be tracked so the motion capture cameras can see them and human joints are tracked automatically similar to how the Microsoft Kinect does it. It is a research project here at CMU with its main focus on creating more accurate motion detection by optimal camera detection. In other words, selecting the right cameras for each point (in a very small nutshell).  It was done byHanbyul Joo, Hyun Soo Park, and Yaser Sheikh. I founded this project very inspiring because all the raw data of movement creates beautiful color patterns and shapes and because it solved almost all the issues the Kinect has to encounter when tracking humans. I feel that a very cool installation could be created with this type of technology because the entire human body is being tracked in a 3D cube/space. this would allow for a completely immersive tracking experience!

After finding this project, i decided to try to see what else similar to this is out there and I found this:

NuFormer – Virtual Reality-Video Projection:

This project is trying to combine virtual reality with motion capture to fully engage the user in the experience. apart from this, it generates a projection of the user in the virtual space to show to the audience what he/she is seeing and experiencing. This type of experiences are being explored to look for more possibilities of how far we can push VR and full our brain. This one was made by NuFormer. I really liked the concept of combining virtual reality with motion capture, this is probably what we are going to end up doing in my ETC project, but I feel this project was just a proof of concept. I didn’t find the experience that engaging, yes the art is nice, but with so that power, something more creative would have been better. Something that would really make the user be in the edge!


Last semester I was the technical lead for a project called Marioneta! This project was made with one experience designer, one sound designer, one artist and two programmers (me being one of them).

Marioneta was a project for the Pittsburgh Children’s Museum to create a puppet gesture recognition and mirroring effect with the use of Microsoft Kinect v2. The projects main focus is to create an endless experience in which everything in the world will react to the users actions as they become and impersonate a puppet.

This installation is currently installed in the museum, you can go check it out!


Watch Me Grow – PAEYC’s Hackathon

PAEYC’s Hackathon


What an experience! Starting by mixing with teachers and programmers to make teachers dream apps come true in 48 hours was amazing! I knew there were many unsolved problems in the educational environment that would make a teacher’s day more pleasant; but after listening to all this teachers pitch their ideas I realized the problem was bigger than what I thought. They struggle with day to day activities that only a teacher would understand. Many problems I didn’t know could be problems are impacting kids’ education every da

y. This made me very happy to be part of this hackathon. I knew after the 48 hours I would not only learn a lot and have a lot of fun, but help someone solve some headaches.

During the hackathon, a group of two developers and me were able to make a cross platform app in Unity to help a teacher keep track of its kid’s level of learning. The app was made for 3-5 years old kids that were learning to identify the alphabet, numbers, colors, and emotions.

It had two main features: Teacher side and kid’s side.


  • Add kids to the list and select their level for each of the fields of study
  • See the status of each kid
  • Adjust the kids level in each field of study depending how they are doing


  • Select one of the fields of study and play a learning game
  • Will record the right and wrong answers for the teachers information

The Last Egg

A 3 vs 3, competitive, team based playground game played with PSMoves. Members of 2 teams run around protecting the teammate holding virtual the egg while the other team tries to get it.
The game was a “Jury Selection” in IndieCade’14 and was showcased in Big Game Arena.

The new DJ experience with LEAP MOTION

After one week of playing with leap motion at the Entertainment Technology Center (Carnegie Mellon University) a team of 2 artist 1 sound designer and two programmers (including myself as a programmer); were able to create a new and easy way to experience the thrive of being a DJ in a Unity3D app.

Working with LEAP MOTION we realized its limitations but we were able to work around them to create an engaging and fun experience for naïve and experienced guest. It was a lot of fun working in this project, even though it was only one week I think we were able to do something very cool and unique.

All the art assets are controlled with a sound analyzer (to be accurate we used the Fast Fourier Transform algorithm). This made that each time the user altered the music, all the art assets would react to it as well; giving the user a sensation of power in the world. The lights also played a big part on the experience. They also reacted to the DJ actions; changing the mood of the world as the user experimented with the controlled and DJ its way!

The Beginning of a New Game

After playing the amazing and fun game Joust  , I realize that my next game had to be similar to this great game. When I say similar I don’t mean copy Joust; but in braise the good things of it to make something better.

I got together with some friends from school and we decided to jump in the quest of making a game inspired by Joust. As a developer my first thought was how are we going to connect more than 4 PS Moves to the PlayStation? For the game we have in mind we need minimum 6!!!!!


After some research I found this incredible Open Source Project that you have to check out: UNIMOVE. It is a Unity plugging that sets C# bindings that allow you to connect up to 7 PS Moves to your computer. It works on Mac, Linux and Windows too.

Check it out at:

Joust  Game: