Gesture controlled Smart Mirror

Datetime:2016-08-22 23:36:40          Topic: Python  OpenCV           Share

SmartMirror

Motion controlled SmartMirror showing time and temperature in background with the possibility of adding any widget to the menu. The camera is behind the two-way mirror to get the best possible look. The screen and camera are managed by a raspberry pi. The motion is managed by a server running in python with OpenCV, the server web is in NodeJS.

Preview

Showing the display of the main menu and a use case on the widget Cinema (movie times of my local theater)

Show full webm preview

Capacities

  • Recognition of gestures : palm, thumbs up/down, slide up/down/right/left
  • Time and outside temperature
  • Widgets :
    • Photo : take a photo and upload it to Dropbox
    • Map : display local Google Map
    • Cinema : movie time of local theater
    • DoodleJump : play the game (hard with the latency)
    • News : show international news

Debugging

The motion server can't recognize gestures on a new environment : lights, hand colors... affect the process, that's why by launching the test.py file and tweak the HSV min/max values and others configs properly you can set it up for your home.

Show webm preview

Dependencies

  • Software :
    • OpenCV 2.x
    • Python 3.x
    • NodeJS
  • Hardware :
    • Two Way Mirror
    • Raspberry pi
    • Camera (I'm using the NoIR but any camera should do)
    • LED Monitor (Preferably that cover the whole surface of your two way mirror)

Building

  • Web server :
  • Motion server :
    • Install OpenCV 2.x and cv2 wrapper for python (should be in the opencv package)
    • pip install numpy
    • pip install tornado
    • python test.py for debug infos or python main.py for silent process

Notes

The motion server was made in python to learn the language, but it should have been made in C/C++ to gain execution speed and flowness in the gestures recognitions. Also, python environment kinda sucks at the moment, the difficulty to set it up for python3 + opencv2 + windows and then linux was exhausting, I don't recommend.





About List