MouseTrap Status
DEVELOPMENT HELP PAGE: http://xcitegroup.org/foss2serve/index.php/MouseTrap_Dev_Help (includes opencv status)
INSTALLED PACKAGES
Please state the versions and OS you are currently working with so that versioning errors can be ruled out.
| Team Member Name | Installed Packages | Team Member Name | Installed Packages | ||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Amber (8/5/2013) |
|
Logan |
| ||||||||||||||||||||||||||||||||||||||||||||||||||
| John |
|
Nicole |
|
STATUS
Please enter the status of the ongoing project below
Stoney Status
Stoney is documenting his efforts on his blog: http://stoney-jackson.blogspot.com/
Amber Status
| [Expand] Status 2012 |
|---|
| [Expand] Status Jan-Aug 2013 |
|---|
Status CURRENT
8.4.2013
I am running into the segfault issue that Logan was talking about.
Logan Status
To make it easier to edit and follow, I have moved my updates to a new page: Logan Status Updates
John Status
12/29/12
- Noted that mousetrap and gnome-mousetrap are in the Ubuntu repository. Ignoring this and following instructions provided by the MouseTrap GNOME Live! project page…
- Installation went as expected. Ran into every issue in the 'Troubleshooting' table, but that seems to be expected behavior.
- Running MouseTrap with command 'mousetrap'. Expected errors due to lack of webcam. Otherwise seems to run without issue (preferences window pops up when button clicked, and so forth).
- Quits without issue.
Thoughts on Install Process:
- Noted that it might be nice if they add the commands for the RPM package manager to the instructions.
- "Troubleshooting" should be a link to the Troubleshooting section for user-friendliness.
- 'Dependency hell' is tedious. Due to the fact that MouseTrap is in the Ubuntu repositories suggests it might be in others as well. It might be useful to include this fact in the install instructions.
- Else it might be useful for the autogen.sh shell file to automatically detect if a package is installed (it already does this) and installs them if they are not. This will run into an issue with different package management systems. A different shell file for the popular ones (APT and RPM)?
- A GUI installer would be the most user-friendly, but that's not the Linux MO.
1/13/13
- Webcam received and set up. Seems to be recognized OOTB on Ubuntu -- the light goes on when in use, etc.
- When running Mousetrap, it shows a blank window (most of the time) or a frozen video frame (only seen this once). In the terminal, it repeatedly displays errors similar to this;
Corrupt JPEG data: 1 extraneous bytes before marker 0xd9
- When running again with webcam unplugged, it runs fine.
- A quick Google got me this, https://bugs.launchpad.net/ubuntu/+source/mousetrap/+bug/685777 which is similar but not exactly (in my case the program gets stuck instantly).
1/25/13
Replaced Ubuntu with Fedora 18 on system. Webcam works out of the box. Everything works nicely. Except for the new UI for Anaconda, but I digress. Had issues installing Mousetrap. Full details below...
Installing Fedora:
I downloaded the ISO image for the Fedora 18 Network Install CD from here http://fedoraproject.org/en/get-fedora-options#formats. This is good if you have fast internet access. This is ideal since a) it's a small(er) download, and b) dd takes a while (see below).
Instead of a CD/DVD, I decided to use a USB stick. I used the Fedora-suggested LiveUSB tool, but it produced a USB stick that either a) is not recognized as bootable, or b) freezes at or after the "Secure boot not enabled" screen. So I used dd instead (a nifty tool, which should already be installed if you're using a UNIX-based OS, that copies a disk image to a destination byte-for-byte). To put the .iso image onto my USB stick, I used the command,
# dd if=./Fedora-18-x86_64-netinst.iso of=/etc/rdisk1
I had to be root to do this (hence the # above). Note that I also put "r" before the device name (in the above example, I used rdisk1 instead of disk1). This enables direct access, which speeds things up considerably. I used
$ fdisk -l
to find out the location of the USB drive.
Once I executed the dd command, it sat there seemingly doing nothing. dd can take a while (hence the reason I didn't use the Live DVD ISO!).
After it was done, booted from the USB drive, followed the (horrid) Anaconda installation process (beware of the terrifying partitioning part!).
After that's done, booted up and installed the b43 driver for my wireless card using the guide here, http://linuxwireless.org/en/users/Drivers/b43#Fedora. Everything works great, including the webcam.
Installing Mousetrap:
I attempted to install all the dependencies listed on the Mousetrap GNOME Live! site using the usual yum install <package>. However pygtk, python-pyspi and libglib2.0 are not in the repos.
Install method Install Dependencies: * gnome-common * glib2-devel * intltool * python-devel * opencv-python * python-xlib Run Commands: -> run git branch fix_install -> run ./autogen.sh -> make -> make install -> run mousetrap from /src (let me know if this works for you --Amber)
2/8/13
Installed Mousetrap successfully via Amber's instructions above. Ran into an error;
ImportError: could not import gobject (error was: ImportError('When using gi.repository you must not import static modules
like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject".',))
2/16/13
Started on learning the Python language using "Learning Python, 4th Edition."
I don't think that you are in the correct branch. You downloaded a clone via git, right? I put up an entire install method, see if you missed something.
4/21/13
NOTES TO SELF:
Worked on getting OpenCV webcam view into GTK+ UI.
Wrote a very simple program using straight OpenCV API. Works fine.
Wrote a GTK+ based program that takes a photo via the webcam and saves it as a file. Also works fine.
With those experiences in my belt, I tried integrating the two, to little success.
Yes, I've tried using the photo program I wrote above to brute-force a display of a video via a loop -- but knew before I begun that it's a horribly bad (and potentially dangerous!) idea.
Googling keeps leading me to GdkPixBuf (previously gtk.gdk.PixBuf in PyGTK). Requires iplimage (used by OpenCV) to be converted into PIL (Python Image Language) before it can be converted into PixBuf via new_from_data. OpenCV used to have adaptors that can do this, but it was, for some reason, deprecated with no equivalent replacement.
There's some nonsense about NumPy and/or Matlibplot as well.
But all of this is moot for the time being due to finding out that the new_from_data API seems to be broken for Python -- http://stackoverflow.com/questions/10284465/gdkpixbuf-pixbufdestroynotify-notimplementederror-python-gtk3
One possible alternative is to use new_from_array, creating an array from the image via Array (or NumPy array, not sure). But I haven't looked too deeply on how to a) convert an iplimage to a PIL then/or to an array, and b) how to use new_from_array.
Another possible alternative -- due to OpenCV using a Qt backend -- is to use Plug() and Socket() in GTK+ to embed the OpenCV window into a GTK+ window. Haven't figured out how to do this due to OpenCV's limited UI API. Maybe a Qt API can interact with windows created by OpenCV?
Another alternative; I checked out Cheese (a open source Photobooth rip off that comes preinstalled with Fedora) -- turns out it uses Gstreamer to capture images from the webcam. As Gstreamer works great with GTK+, perhaps one solution would be to use Gstreamer for the UI, and OpenCV for the background heavy-lifting?
4/30/13
After the discussion on Friday of how MouseTrap was supposed to work, and whether it even worked "in the wild" at all, I decided to compile it and see what I can find out with all that I've learned in the past few months in the hopes of gathering some useful information for future use.
We know that MouseTrap compiles in Ubuntu 10.4, so I installed Ubuntu 10.4 on a virtual machine, and followed the instructions on the MouseTrap GNOME Live! webpage. Additionally, I compiled Mousetrap with the --enable-doxygen and --enable-docbook flags (from the bottom of the installation instructions page) to get the doxygen-generated developer documentation and user documentation, respectively. Those are not obtainable without compiling MouseTrap, and we all know it does not compile in Fedora 18.
The user documentation (docbook) and doxygen-generated documentation are on my GitHub repo; ([1])
Additionally, I ran Mousetrap multiple times, and found three "scenarios", each occurring at random. I have documented the "scenarios" below.
The first scenario is where Mousetrap runs, and displays a frozen captured frame from the webcam, and nothing else. The output in the terminal is as follows:
john@john-desktop:~/mousetrap$ mousetrap
Xlib.protocol.request.QueryExtension
DEBUG: mousetrap.ocvfw.idm -> Starting Forehead idm
INFO: mousetrap.ocvfw.idm -> Forhead Algorithm loaded
DEBUG: mousetrap.ocvfw.idm -> Setting Capture
DEBUG: Commons -> New Singleton Add (mousetrap.ocvfw.dev.camera.Camera)
DEBUG: ocvfw -> cmStartCamera: Camera Started
DEBUG: Camera -> Loaded backend OcvfwPython
DEBUG: OcvfwBase -> Changed lk_swap value to True
INFO: mousetrap - Idm loaded and started
DEBUG: ui.main - Interface Built
DEBUG: ui.main - Addons loaded
INFO: mousetrap - MouseTrap's Interface Built and Loaded
DEBUG: ocvfw -> cmGetHaarPoints: detected some matches
DEBUG: ocvfw -> cmGetHaarPoints: detected some matches
DEBUG: ocvfw -> cmGetHaarPoints: detected some matches
DEBUG: ocvfw -> cmGetHaarPoints: detected some matches
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/dev/camera.py", line 116, in sync
Camera.query_image()
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/_ocv.py", line 140, in query_image
co.cv.cvCvtColor(self.img, self.grey, co.cv.CV_BGR2GRAY)
File "/usr/lib/pymodules/python2.6/opencv/cv.py", line 6759, in cvCvtColor
return _cv.cvCvtColor(*args)
RuntimeError: openCV Error:
Status=Null pointer
function name=cvGetMat
error message=NULL array pointer is passed
file_name=cxcore/cxarray.cpp
line=2370
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/mousetrap/app/main.py", line 238, in update_frame
self.itf.update_frame(self.idm.get_capture(), self.idm.get_pointer())
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/idm/forehead.py", line 128, in get_capture
self.get_forehead()
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/idm/forehead.py", line 148, in get_forehead
face = self.cap.get_area(commons.haar_cds['Face'])
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/dev/camera.py", line 373, in get_area
return Camera.get_haar_points(haar_csd)
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/_ocv.py", line 340, in get_haar_points
co.cv.cvResize( self.img, self.small_img, co.cv.CV_INTER_LINEAR )
File "/usr/lib/pymodules/python2.6/opencv/cv.py", line 6763, in cvResize
return _cv.cvResize(*args)
RuntimeError: openCV Error:
Status=Null pointer
function name=cvGetMat
error message=NULL array pointer is passed
file_name=cxcore/cxarray.cpp
line=2370
Screenshot: 
The second scenario is where Mousetrap runs, with no image displayed. The output in the terminal is as follows:
john@john-desktop:~$ mousetrap
Xlib.protocol.request.QueryExtension
DEBUG: mousetrap.ocvfw.idm -> Starting Forehead idm
INFO: mousetrap.ocvfw.idm -> Forhead Algorithm loaded
DEBUG: mousetrap.ocvfw.idm -> Setting Capture
DEBUG: Commons -> New Singleton Add (mousetrap.ocvfw.dev.camera.Camera)
DEBUG: ocvfw -> cmStartCamera: Camera Started
DEBUG: Camera -> Loaded backend OcvfwPython
DEBUG: OcvfwBase -> Changed lk_swap value to True
INFO: mousetrap - Idm loaded and started
DEBUG: ui.main - Interface Built
DEBUG: ui.main - Addons loaded
INFO: mousetrap - MouseTrap's Interface Built and Loaded
Corrupt JPEG data: 5113 extraneous bytes before marker 0xd2
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/dev/camera.py", line 116, in sync
Camera.query_image()
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/_ocv.py", line 129, in query_image
self.imgSize = co.cv.cvGetSize (frame)
File "/usr/lib/pymodules/python2.6/opencv/cv.py", line 4350, in cvGetSize
return _cv.cvGetSize(*args)
RuntimeError: openCV Error:
Status=Bad argument
function name=cvGetSize
error message=Array should be CvMat or IplImage
file_name=cxcore/cxarray.cpp
line=1227
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/mousetrap/app/main.py", line 238, in update_frame
self.itf.update_frame(self.idm.get_capture(), self.idm.get_pointer())
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/idm/forehead.py", line 128, in get_capture
self.get_forehead()
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/idm/forehead.py", line 148, in get_forehead
face = self.cap.get_area(commons.haar_cds['Face'])
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/dev/camera.py", line 373, in get_area
return Camera.get_haar_points(haar_csd)
File "/usr/local/lib/python2.6/dist-packages/mousetrap/ocvfw/_ocv.py", line 335, in get_haar_points
cascade = co.cv.cvLoadHaarClassifierCascade( haarCascade, self.imgSize )
AttributeError: Camera instance has no attribute 'imgSize'
Screenshot: 
The third scenario, which is rare compared to the above two, is rather interesting. Mousetrap begins with a completely blank window (no widgets, only the title bar) and hijacks the mouse. Most of the time, the mouse is stuck at the lower left corner, but occasionally it would flicker around the screen (no smooth motion, it simply appears and disappears at seemingly random points). This is not in response to the webcam feed as far as I could tell. The behavior does not change even when the webcam is covered. There is also no way to exit the program; I had to do a killall.
The terminal output is:
john@john-desktop:~$ mousetrap Xlib.protocol.request.QueryExtension DEBUG: mousetrap.ocvfw.idm -> Starting Forehead idm INFO: mousetrap.ocvfw.idm -> Forhead Algorithm loaded DEBUG: mousetrap.ocvfw.idm -> Setting Capture DEBUG: Commons -> New Singleton Add (mousetrap.ocvfw.dev.camera.Camera) DEBUG: ocvfw -> cmStartCamera: Camera Started DEBUG: Camera -> Loaded backend OcvfwPython DEBUG: OcvfwBase -> Changed lk_swap value to True INFO: mousetrap - Idm loaded and started DEBUG: ui.main - Interface Built DEBUG: ui.main - Addons loaded INFO: mousetrap - MouseTrap's Interface Built and Loaded DEBUG: ocvfw -> cmGetHaarPoints: detected some matches DEBUG: ocvfw -> cmGetHaarPoints: detected some matches DEBUG: ocvfw -> cmGetHaarPoints: detected some matches DEBUG: ocvfw -> cmGetHaarPoints: detected some matches DEBUG: ocvfw -> cmGetHaarPoints: detected some matches DEBUG: ocvfw -> cmGetHaarPoints: detected some matches DEBUG: ocvfw -> cmGetHaarPoints: detected some matches DEBUG: ocvfw -> cmGetHaarPoints: detected some matches Corrupt JPEG data: 1372 extraneous bytes before marker 0xd2 DEBUG: ocvfw -> cmGetHaarPoints: detected some matches DEBUG: ocvfw -> cmGetHaarPoints: detected some matches ... and so on infinitely.
Nicole's Status
1.25.2013
Fedora 18 was successfully installed. Instructions are now on the "Getting Started" page.
2.5.2013
MouseTrap was fully installed. Instructions are now on the "Getting Started" page. I encountered the same first error:
ImportError: could not import gobject (error was: ImportError('When using gi.repository you must not import static modules
like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject".',))
- Amber, have you put up your latest fork onto your github yet?
Yes, do you have git installed? (Not really sure how much git you know, so I'll just put up instructions for all.) If not, open a terminal, become root cd to /opt mkdir git (just fyi you can put your repo anywhere, this is just how I was taught) run yum install git-core cd into git folder run git clone git://github.com/amberheilman/mousetrap.git now you have pulled down my code.
To start using the code I'm currently working on: run: git checkout fix_install (my branch) git pull * DO NOT MAKE CHANGES HERE (we should all be working in separate branches) git branch INSERT_BRANCH_NAME (this clones my branch and creates a new one) git checkout INSERT_BRANCH_NAME (this opens up your branch that you just created) git branch (now you can see the branch your in with a *, and all others under this git repo) * We should also be making commits on a frequent basis with commit titles that show exactly what was changed * Hope this helps
@Amber from Stoney: It sounds like you're suggesting that Nicole share your repository. If so, she'll need to create a GitHub account, and you'll need to add her as a contributor on your fork.
(cont.) Alternatively, when Nicole has something to contribute, she could make a patch and send it to you.
(cont.) Another alternative, Nicole could fork your fork using GitHub (she'll need to create an account). When she has something to contribute, she can make a pull request to your fork, you can review the changes and accept/reject. Directions/tutorial for this approach are here: https://help.github.com/articles/fork-a-repo .
2.7.2013
Made my own github account and forked off of amber's at: https://github.com/WNEmousetrap/mousetrap
Ran mousetrap and now getting error:
Traceback (most recent call last): File "<string>", line 1, in <module> File "mousetrap/app/main.py", line 42, in <module> import environment as env ImportError: No module named environment
NOTE: Remember even if you download my code you still need to run the install so cd into the git repo /mousetrap and do a ./autogen.sh which checks to make sure your dependencies are there (see what I wrote to john). Then you will have to run "make" after that is done run "sudo make install" Then you should be able to cd to /mousetrap/src and type "mousetrap" to run the program. Otherwise it will give you errors like above because files were not created by the make command.
*Side note:, if you would like to be added to my fork I can do that easily, just send me your username. Plus I think it will be easier for us all to communicate if we're working off of the same repo. (-Amber)
2.12.2013
Started to document OpenCV using the information at http://docs.opencv.org/ . I'm starting right with the Introduction and reading through. Trying to get definitions and history from here as much as I can.
2.13.2013
Continuing documentation. Found a few sources towards the community. Getting into reading about API Concepts currently.
2.15.2013
Started looking at OpenCV 2 documentation. Organized the Dev Help page. Starting to focus on CV 2 documentation for now, most of the general information seems to be the same anyways.
2.19.2013
Found the meeting notes from the OpenCV community; starting to read through them with the first meeting notes starting in 2008 (http://code.opencv.org/projects/opencv/wiki/Meeting_notes)
- stopped at 2009-02-03 for now
2.20.2013
Continued with meeting notes -
2.21.2013
Meeting notes - stopped at 2009-06-02 for now.
3.28.2013
Started to look into what the Makefiles in mousetrap really do; looking through the pdf slides for GNU "autotools" from http://www.lrde.epita.fr/~adl/autotools.html
3.30.2013
Continued reading through slideshow.
4.2.2013
Finished slideshow in preparation for the sprint on Friday. Started researching on Gnome's site to see how they talk about autotools for their projects.
4.4.2013
Found a few things on the Gnome site:
- A basic Gnome project tutorial: https://developer.gnome.org/anjuta-build-tutorial/stable/index-info.html.en REALLY GOOD!!
- A suggestion for a "modernized" autotools: https://live.gnome.org/GnomeGoals/ModernAutotools
- The site does bump me back to the original pdf slideshow that I read through. This is a good sign!!
The basic tutorial is very good to read through. It is a little smaller than the pdf so it would take much less time to get through, but it still has great information. I still recommend reading through at least parts of the pdf if you have time, but the tutorial is an EXCELLENT starting point!
4.5.2013
Mousetrap Sprint all day!! Worked on understanding Makefiles.am's and created a Directory Tree for whole mousetrap folder (not just src) and accompanying Makefile.am's in those directories if they existed.
4.11.2013
Starting to work through unit testing the "app" and GUI section of the project. Reading up on how gtk works and how the code should look. Will then look at the mousetrap code and add in the functionality that they have to see where the program "breaks."
- When only trying to write a file to import gtk I got the error:
Traceback (most recent call last):
File "gtk_test.py", line 1, in <module>
import gtk
File "/usr/lib64/python2.7/site-packages/gtk-2.0/gtk/__init__.py", line 40, in <module>
from gtk import _gtk
ImportError: /lib64/libEGL.so.1: undefined symbol: wl_display_dispatch_queue_pending
- I am now looking into how gtk (especially pygtk) is suppose to work. I am using the tutorial: http://pygtk.org/pygtk2tutorial/
4.12.2013
Still reading through the tutorial about gtk. Going to try to make a small test program for a sample gtk application. Also looking into fixing the gtk import issue.
Fixed the gtk issue! It was a simple version issue. All I needed to do was run:
sudo yum update libwayland-server sudo yum update libwayland-client
The version I was on was 1.0.0. A new symbol was added in 1.0.2. The most recent version out is 1.0.5.
I managed to create a small working gtk application window!!! Below is the code you can put into a test file (such as mine "gtk_sample.py) and run with "python2 gtk_sample.py":
# example gtk program
import pygtk
pygtk.require('2.0')
import gtk
class gtk_sample:
def callback(self, widget, data):
print("Hello again - %s was pressed" % data)
def delete_event(self, widget, event, data=None):
gtk.main_quit()
return False
def __init__(self):
self.window = gtk.Window(gtk.WINDOW_TOPLEVEL)
self.window.set_title("gtk example")
self.window.connect("delete_event", self.delete_event)
self.window.set_border_width(50)
self.box1 = gtk.HBox(False, 0)
self.window.add(self.box1)
self.button1 = gtk.Button("Yes")
self.button1.connect("clicked", self.callback, "yes")
self.box1.pack_start(self.button1, True, True, 0)
self.button1.show()
self.button2 = gtk.Button("No")
self.button2.connect("clicked", self.delete_event, "no")
self.box1.pack_start(self.button2, True, True, 0)
self.button2.show()
self.box1.show()
self.window.show()
def main():
gtk.main()
if __name__ == '__main__': sample = gtk_sample() main()
5.15.2013
I returned to work today since it is the first week of summer break for me. I will normally be in the lab and working on MouseTrap from 1-5pm on Wednesdays and Fridays.
Today I fixed the Ubuntu machine (as per Amber's and Logan's suggestions from a week ago, THANKS AGAIN!) and was able to get past the weird gi.repository issue, though I have an inkling on how to fix it on the Fedora too, I just haven't explored that yet. I was able to debug the code I rewrote from the original source code for the GUI and it doesn't break... but it also doesn't show me an interface.
I moved on to completely writing my own version of the ui.main file and have an interface that shows up with 3 buttons! I am now looking through the ui.settings_gui file and trying to debug that show it shows up when I run my file.
