📎 Webclip
Elixir, NIFs, Xbox Kinect, and LiveView
The post follows the path from a borrowed Xbox Kinect to a working Elixir integration. It explains why NIFs were needed to reach libfreenect, how opaque C pointers were handled with resources, and why Phoenix LiveView was used for a quick demo UI.
Reading notes#
- The author first became interested in Kinect by seeing what people did with it connected to computers, but did not buy one at the time.
- Years later, a borrowed Kinect led to a project that touches Elixir, NIFs, Kinect, and Phoenix LiveView.
- The post is a journey report, not an in-depth guide to NIFs or Phoenix LiveView.
- OpenKinect’s
libfreenectgives direct C access to Kinect hardware features such as tilt motors, LED, audio, video, and depth sensors. - Elixir cannot call C bindings directly, so Erlang NIFs are used to bridge Elixir and
libfreenect. - NIFs let native code run inside the Erlang VM, but they also carry the risk of crashing the VM.
- Phoenix LiveView was chosen because it was the simplest way to build a small interactive demo app.
- The author validated the Kinect connection by installing
libfreenectand usingfreenect-glview. - The first NIF work centered on
freenect_init, which creates a context for thelibfreenectenvironment. - Because
contextanddeviceare opaque pointers, the project used NIF resources so Elixir could carry them as references and pass them back into later NIF calls. - The Elixir module defines NIF functions that are replaced at load time by the native implementations.
- Video handling uses a callback:
libfreenectsends one frame each time events are processed. - The callback stores the Elixir process PID and sends the frame to that PID.
- Raw RGB data is converted to PNG with
svpng, placed in an Erlang binary, and sent withenif_send. - On the Elixir side, a listener GenServer receives frames and stores the latest one.
- The demo app uses LiveView buttons and
handle_eventfunctions to call Kinext and update assigns. - Frames are embedded in the page as Data URLs by base64-encoding the PNG stored in assigns.
- The author notes that this image approach is inefficient for video, but felt responsive on a local machine.
- The project is intended as a library for the community, and the author invites others to build on it.
