Saving the overheads on Kinect joint data

So I've spent a good part of the last month working on a couple of Python applications to get joint coordinates from the skeletons generated from a Version 2 Kinect and finally have something to show for it. When an XEF file is played back using the Kinect Studio, the Kinect Service interprets it as if the data is being captured live and you can get data out of this using the BodyFrame object, which contains data on all the bodies in the current frame and the location of their joints. The only problem is that the files are in the GigaBytes in size and takes up quite a bit of hard drive space. I've managed to store the location of each body's joints in a database with a concise database schema.

I can easily access which  "performance" I want using a unique ID that I assign when I record Kinect data and store it in a database that can be accessed by a variety of platforms using 0.1% of the original hard drive space. At the moment I am only using motion capture data of the 25 joints the Kinect can automatically detect and no RGB or sound data but I want to have some sort of compromise early in the new year. Using this data I can easily play back the data in a much more human-understandable way in case I need to analyse the data visually or quantitatively, or even just to make sense of the data. I've included a screen shot of how this looks below.

The other great thing about having this data in a database is that I don't need to iterate over the XEF files and extract joint coordinates whenever I want to create a graph, for example. I can just read the data out of the database using a simple SELECT query and plot a graph. Already I'm able to generate graphs tracking joint movement, velocity, and acceleration over time - not something that was particularly easy with motion capture data, even a decade ago. 

I can't wait to get recording ensembles and using the kit to get some real data and see what we can find! I've been getting some ideas for studies after my initial exploratory pilot test - watch this space!

Ryan

Extracting data from Kinect V2 .xef files

The Kinect V2 easily allows users to record and store stream data using .xef (extended event files) but getting any data out of that file once it has been recorded has proven to be quite a task. While traditional video data can be looped through frame by frame at your own leisure, the .xef file does not allow (or at least not for me despite my best efforts) this kind of analysis. At first, it seemed that RGB, Depth, or Body Frame data could only be intercepted at run time and, therefore, if I didn't get the data I wanted I would have to record it all again.

Fortunately, playback of stored Kinect data is actually interpreted by the Kinect Service as a live feed and applications designed to extract data at run time can be used to do so with recorded data. The only problem here is that the applications must be efficient enough to allow my machine to cope with handling the playback and data extraction simultaneously without dropping any frames. So far I have been able to extract the X, Y, and Z values of each of the 25 joints in the Body Frames in metres, which should be enough data for what I need but there is still the issue of working out which joint belongs to which body in the scene so we know who is moving in what way.

I'm not a big C# fan and don't enjoy programming in Visual Studio so I was looking at some alternative ways to access the Kinect Service API and found a really nice Python package that even came with a demo game that showed off its neat implementation:

https://github.com/Kinect/PyKinect2

I've spent the last few days getting my head around it and seem to be getting somewhere and I'm really excited to record a group of singers and seeing if I can get all the data I need. I can then easily store joint co-ordinates in a SQL database (Probably will do one for each recording) and run a few scripts on the these to get some nice pretty graphs. Can't wait to get started!

Ryan

Hands on with the Kinect V2 and Visual Gesture Builder

So we finally managed to get the latest version of the Microsoft Kinect and I'm very very impressed with what, admittedly little, I've seen from it. Within moments of opening up the Kinect Studio you see the skeleton frames overlaying the 3D point cloud image - something that took me hours of tutorials to do with the V1! Within 10 minutes I had recorded myself doing a few "hands up" gestures and trained the gesture builder to recognise these - I can't wait to start collecting some performance data and seeing what interesting stuff we can do with it!

Ryan