How to control HTML5 slides with your hands via webcam

I had an idea to show kind of human computer interaction during my Fosdem talk. I may not use that, because I will not have time to polish it, but it works quite well if you want to use it.

The Idea
The idea is to control slides only by moving my hand, without any devices in it.

The plan:
0. My web-cam should detect my move
1. After detection an event should be triggered
2. As a final action the slide should move to the next one :)

Motion detect

$ git clone https://github.com/sackmotion/motion.git
$ cd motion
$ ./configure
$ make & make install

Configure it

Motion comes with a great almost working config file, so just copy it to make it “official”:

$ cp  /usr/local/etc/motion-dist.conf /usr/local/etc/motion-dist.conf

Open it, search for those values, edit them and read what are they for:

;on_event_start value
event_gap 60

to become:

on_event_start xdotool key Right /OR xdotool search "Mozilla Firefox" windowactivate --sync key --clearmodifiers Right/
event_gap 1

Event triggering

You may need to install xdotool using your package manager. This tool allows you to emulate keyboard action in different combinations, but I will need to emulate pressing the right arrow key.

How it works:

When web-cam detects movement an event emulating ‘right arrow’ key is pressed. Since I am using HTML5 DZslides from here, this allows me to move to next slide only with moving my hand in front of my webcam.

Run

Run it and move your hand in front of your camera. You may want to turn on the logging to see the motion detection and the event detection if it’s not working at first.

$ motion

If you have any questions, please let me know.

5 Responses to “How to control HTML5 slides with your hands via webcam”

  1. tom says:

    Now if there was a way to capture images from your webcam in Javascript, you could compile (a streamlined version of) motion with emscripten and embed the whole thing in your slides.

  2. Sarah says:

    Haaaa! IT’s working. Awesome!

    The only think I have to do is move your hand and then go out of camera area for 1 sec and then you can repeat it.

    • Bogo says:

      Yes, that’s event_gap 1 settings.

      It will trigger event_start on first move and after ($event_gap value) will trigger event_end

  3. […] Since clicking and double clicking is dead and I think ‘touch’ will follow soon we will implement 2 more interacting interfaces – Voice (Remember my Fosdem talk in 2010 when I made the the demonstration on how to control Firefox with your voice?) and Gestures. […]

Leave a Reply