MouseTrap Dev Help

(Difference between revisions)
Jump to: navigation, search
(→commons.py)
(→backend)
Line 126: Line 126:
 
* Contains three classes:
 
* Contains three classes:
  
−
  OcvfwBase:  
+
====  OcvfwBase: ====
  
−
  OcvfwPython:  
+
====  OcvfwPython: =====
 
   * inherits from OcvfwBase
 
   * inherits from OcvfwBase
 
   * imports global and local variables from Commons.hg (highgui related variables) and Commons.cv(Opencv) variables
 
   * imports global and local variables from Commons.hg (highgui related variables) and Commons.cv(Opencv) variables
Line 135: Line 135:
 
   * get_haar_points: resizes the image by 1.5  
 
   * get_haar_points: resizes the image by 1.5  
  
−
  OcvfwCtypes
+
====  OcvfwCtypes====
 
   * imports global and local variables from Commons.hg (highgui), Commons.cv(which is a CV common lib), and OcvfwBase
 
   * imports global and local variables from Commons.hg (highgui), Commons.cv(which is a CV common lib), and OcvfwBase
  

Revision as of 23:10, 21 February 2013

Contents

Standard Dev Environment

  • Fedora 18
  • OpenCV 2.4.X - whatever recent version is supported
  • Python 2.X for now upgrading to Python 3 when we get the OpenCV issue fixed

Complete Git + Mousetrap Install

Let's get git 

 Open a terminal, become root
 Run Commands:
 -> cd /opt
 -> mkdir git                                                * just fyi you can put your repo anywhere, this is just how I was taught
 -> yum install git-core                                     * Install git
 -> cd /opt/git
 -> git clone git://github.com/amberheilman/mousetrap.git  * Now you have pulled down the git repo
 -> git checkout fix_install                                 * This is my branch
 -> git pull                                                 * DO NOT MAKE CHANGES HERE! We will all be working in separate branches!

Make a branch 

 -> git branch INSERT_BRANCH_NAME      * this clones my branch and creates a new one
 -> git checkout INSERT_BRANCH_NAME    * this opens up your branch that you just created
 -> git branch                         * now you can see the branch your in with a *, and all others under this git repo

Mousetrap Install method 

 Install Dependencies:
 -> yum install gnome-common
 -> yum install glib2-devel
 -> yum install intltool
 -> yum install python-devel
 -> yum install opencv-python
 -> yum install python-xlib
 Run Commands:
 -> cd /opt/git/mousetrap/src          * THIS WILL NOT WORK OUTSIDE THIS DIR!
 -> git branch fix_install
 -> ./autogen.sh
 -> make
 -> make install
 -> mousetrap                          * You may have errors but they should be similar to my own (towards the end)

Make your first commit 

 -> git status                         * Shows all of the modified files
 -> git add *                          * This is to add ALL files to commit list (MAKE SURE YOU WANT THEM ALL FIRST!)
 -> git commit                         * Add a useful title to first line of your commit.
                                         This is in vim so 'i' to insert ':x' to save and quit.
 -> git push origin INSERT_BRANCH_NAME * This must be the branch you created in the git install.
                                         This will ask for your git credentials, so have them ready.

The MouseTrap Program

Here is a complete dissection of the hierarchy done by gnome: http://gnome-mousetrap.sourcearchive.com/documentation/0.3plus-psvn17-2ubuntu1/main.html

Definitions

Haar wavelet
First proposed by Alfred Haar, the Haar wavelet is a series of square-shaped functions that when shown together form a basis otherwise known as a wavelet family and expressed in terms of an orthonormal function basis. For more information, see: [[1]]
Haar-like features
Uses adjacent rectangular regions in a specified detection window, sums up the pixel intensities and calculates the difference between those sums. They are called Haar-like features because they are computed using similar coeffients in the Haar wavelet transforms. These regions can then be concatenated with boosted classifiers into a classifier cascade to cross reference these regions with other positive samples to form a model for object detection. For more information, see [[2]] [[3]]
Boosted Classifiers
The result of increasing the accuracy of a classifier cascade through the means of positive object recognition of scaled images.
Classifier Cascade
Proposed by Paul Viola and refined by Rainer Lienhart, a classifier cascade
ROI (Region of Interest)
This is usually a subset of the original frame represented as a rectangle. It is most often compared to a classifier cascade to determine a positive match.
Ocvfw (OpenCV FrameWork)
A MouseTrap in house framework that manages OpenCV methods and includes functions to initiate the camera and detect further Haar-like features.
Optical flow
Lucas-Kanade method
Singleton
Restricts the instantiation of a class to one object so coordination over program actions can be achieved.

Classes

app

main.py

  • Loads the Image Detection Module (idm)

commons.py

  • Stores global variables for Ocvfw 
 * cv: OpenCV related variables
 * hg: OpenCV.highgui related variables. Set in ocvfw/idm/ color.py to be used in the creation of the Window and Trackbar
 * abs_path: the absolute path to the commons file
 * haar_cds: array with the haar xml file paths
 * colors: array with the types of colors the image can appear in
 * singleton: class instance to be passed through program globally
  • Method to get color channel 
 * get_ch(color): returns channel corresponding to the color given (rgb, bgr, gray)

environment.py

debug.py

  • This is used to display messages to the command line
  • Very helpful to troubleshoot

ui

main.py

lib

addons

OpenCV Framework (ocvfw)

  • This is the wrapper around OpenCV


ocvfw/ _ocv.py

  • Contains three classes: 
 OcvfwBase: direct copy of backends/ OcvfwBase

 OcvfwPython: direct copy of backends/ OcvfwPython

 OcvfwCtypes: direct copy of backends/ OcvfwCtypes

ocvfw/ idm

  • Detects features based on an xml file

ctypesopencv

  • This is used instead of the python bindings for OpenCV

pocv.py

  • Returns an instance of the idm

ocvfw/ haars

  • Used thousands of samples to concatenate an xml file to predict features

dev/ camera

 * first gets a region of interest and then matches it against a haar classifier

backend

  • link to image on how the backend distributes variables ["]
  • Contains three classes:

OcvfwBase:

OcvfwPython: =

 * inherits from OcvfwBase
 * imports global and local variables from Commons.hg (highgui related variables) and Commons.cv(Opencv) variables
 * has the ability to get_motion_points but is not used
 * get_haar_roi_points: finds regions of interest within the entire frame image and returns the matches against the classifier cascade using the HaarDetectObjects() OpenCV function.
 * get_haar_points: resizes the image by 1.5

OcvfwCtypes

 * imports global and local variables from Commons.hg (highgui), Commons.cv(which is a CV common lib), and OcvfwBase

Files affected by OpenCV 2.4.3 upgrade

  • main.py
  • ocvfw
    • _ocv.py
    • ocvfw/ idm
      • eyes.py
      • finger.py
      • forehead.py
      • color.py
    • commons.py
    • ocvfw/ dev
      • camera.py
    • ocvfw/ backends
      • OcvfwBase.py
      • OcfwCtypes.py
      • OcvfwPython.py

Opencv

General Concerns

Here we place all of the concerns when migrating to a newer version of opencv or the repercussions of not upgrading.

Migration to Python 3

  • Opencv 2.4.3 does not support Python 3

Camera drivers

  • Currently, only some camera drivers are supported with the new version of opencv, leading to problems regarding capture. One issue is the inability to cancel the program webcam capture.

Yum repo

  • Right now, only the 2.3.1 version of opencv is available in the yum repository

OpenCV Info

Basics and Source Code

OpenCV (Open Source Computer Vision) is a library of programming functions for the realtime computer vision. OpenCV is released under the liberal BSD license and hence it's free for both academic and commercial use. It has C++, C, Python and Java (Android) interfaces and supports Windows, Linux, Android and Mac OS. The library has more than 2500 optimized algorithms. Adopted all around the world, OpenCV has more than 47 thousand people of user community and estimated number of downloads exceeding 5 million. Usage ranges from interactive art, to mines inspection, stitching maps on the web or through advanced robotics.

To access the OpenCV repository directly: git clone git://github.com/itseez/opencv.git

History

"OpenCV was started at Intel in 1999 by Gary Bradski for the purposes of accelerating research in and commercial applications of computer vision in the world and, for Intel, creating a demand for ever more powerful computers by such applications. Vadim Pisarevsky joined Gary to manage Intel's Russian software OpenCV team. Over time the OpenCV team moved on to other companies and other Research. Several of the original team eventually ended up working in robotics and found their way to Willow Garage. In 2008, Willow Garage saw the need to rapidly advance robotic perception capabilities in an open way that leverages the entire research and commercial community and began actively supporting OpenCV, with Gary and Vadim once again leading the effort." NEED A REF OR LINK

So what is OpenCV?

  • OpenCV is a computer vision library in C++.
  • OpenCV 2 was released in 2009 with additional functionality and increased performance
  • CV is a Python wrapper for OpenCV
  • CV2 is a Python wrapper for OpenCV 2 and includes CV
    • If you want CV functions you have to 
import cv2.cv as cv

The full name is "Open Source Computer Vision Library." It is a library of programming functions aimed at real-time computer vision. It was developed by Intel and now Willow Garage and Itseez support it. It is free for use under the open source BSD license. Also, it can cross-platform.


Where did it come from?

OpenCV emerged from the Intel Research Initiative; it is related to Intel's Performance Library, which today is called Integrated Performance Primitives (IPP). The project launched in 1999. It was looking for CPU-intensive applications. The project goals were to:

  1. advance vision research by open and optimized infrastructure
  2. disseminate vision knowledge with readable code
  3. and advance commercial applications

Version List: 

alpha-release at CVPR 2000 
five beta-releases 2001-2005 
Version 1.0 2006 
Continuation of development by Willow Garage 2008 (pre-release version 1.1) 
Version 2.0 2009 
Versions 2.1, 2.2 2010 
Version 2.3 2011 
Version 2.4.0 May 2012 
Version 2.4.1 June 2012 
Version 2.4.2 July 2012

Application Use: 

2D and 3D Feature Toolkits
Egomotion Estimation
Facial Recognition System
Gesture Recognition
Human-Computer Interaction (HCI)
Mobile Robotics
Motion Analysis
Object Detction and Recognition
Segmentation
Stereo Vision: Depth Perception from 2 Cameras
Structure from Motion (SFM)
Motion Tracking

(http://www.cvl.isy.liu.se/education/graduate/opencv/Lecture1_History.pdf)

Definitions

Modules Available

*core: a compact module defining basic data structures, including the dense multi-dimensional array Mat and basic functions
used by all other modules
*imgproc: an image processing module that includes linear and non-linear image filtering, geometrical image transformations
(resize, affine and perspective warping, generic table-based remapping), color space conversion, histograms, and so on
*video: a video analysis module that includes motion estimation, background subtraction, and object tracking algorithms
*calib3d: basic multiple-view geometry algorithms, single and stereo camera calibration, object pose estimation, stereo
correspondence algorithms, and elements of 3D reconstruction
*features2d: salient feature detectors, descriptors, and descriptor matchers
*objdetect: detection of objects and instances of the predefined classes (for example, faces, eyes, mugs, people, cars, and so on)
*highgui: an easy-to-use interface to video capturing, image and video codecs, as well as simple UI capabilities
*gpu: GPU-accelerated algorithms from different OpenCV modules
*some other helper modules, such as FLANN and Google test wrappers, Python bindings, and others

(http://docs.opencv.org/)

OpenCV - Great Resources

Opencv 2 Info

Some Differences Between cv and cv2

  • single import of all of OpenCV using import cv
  • OpenCV functions no longer have the "cv" prefix
  • simple types like CvRect and CvSca,lar use Python tuples
  • sharing of Image storage, so image transport between OpenCV and other systems (e.g. numpy and ROS) is very efficient
  • complete documentation for the Python functions

OpenCV 2 - Great Resources

cv 2.4.3

A cv2 example

Here is a little program that will capture video from the webcam and display it (you might have to CTRL+C) 

"""
This module is used for testing the opencv2 capabilities
"""
import cv2
#get webcam feed
capture = cv2.VideoCapture(0)
while True:
    #combines VideoCapture.grab() and VideoCapture.retrieve()
    retrieval_value, image = capture.read()
    #shows captured image in a window
    cv2.imshow("webcam", image)
    #will stop capture with capatible webcam
    if cv2.waitKey(10) == 27:
        break

cv 2.1

OpenCV Community

Code Meeting Notes

  • Meetings occur on Wednesday's at noon (starting on 2/20) on irc.gnome.org #mousetrap - holiday and other circumstances permitting.

2.19.2013

Agenda

  1. Status
  2. Go over git+mousetrap installation [[4]]
  3. Address any issues
  4. Go over mousetrap hierarchy [[5]]
  5. Go over opencv integration methods [[6]]
  6. Next Steps


Meeting summary

Status

 * LINK:
   http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/main.html
   (amber, 18:14:27)
 * LINK:
   http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/main.html
   (amber, 18:14:29)
 * LINK: http://code.opencv.org/projects/opencv/wiki/Meeting_notes
   (Dark_Rose, 18:17:39)
 * LINK:
   http://www.xcitegroup.org/foss2serve/index.php/MouseTrap_Dev_Help#The_MouseTrap_Program
   (amber, 18:20:24)

Git and mousetrap installation

 * creating your own branch in git: git checkout -b my_fantastic_branch
   (Stoney, 18:29:24)
 * LINK:
   http://git-scm.com/book/en/Git-Branching-Basic-Branching-and-Merging
   (amber, 18:32:31)

Issues

 * LINK: http://code.google.com/p/pyopencv/   (john, 18:46:37)
 * ctypes-opencv is a package that brings Willow Garage's (formerly
   Intel's) Open Source Computer Vision Library (OpenCV) to Python.
   OpenCV is a collection of algorithms and sample code for various
   computer vision problems. The goal of ctypes-opencv is to provide
   Python access to all documented functionality of OpenCV.  (amber,
   18:46:39)
 * LINK: http://code.google.com/p/pyopencv/   (amber, 18:46:46)
 * http://pythonhosted.org/pyopencv/2.1.0.wr1.2.0/  (amber, 18:51:37)
 * LINK: http://pythonhosted.org/pyopencv/2.1.0.wr1.2.0/   (amber,
   18:51:45)

Integration methods

 * LINK:
   http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/dir_9d66d24675e04ade776f5269c3621ea1.html
   (amber, 18:58:53)
 * LINK:
   http://gnome-mousetrap.sourcearchive.com/documentation/0.3plus-psvn17-2ubuntu1/classmouseTrap_1_1ocvfw_1_1ocvfw.html
   (amber, 19:02:40)
 * ACTION: Create Module documentation for MouseTrap  (amber, 19:14:55)

Meeting ended at 19:16:36 CET.

Action Items

  • Create Module documentation for MouseTrap

People Present (lines said)

  • amber (125)
  • john (39)
  • Stoney (31)
  • logan_h (14)
  • darci (8)
  • Dark_Rose (5)
  • tota11y (2)

Conclusions

  • documentation on Mousetrap's OpenCV FrameWork is needed for further inspection.

2.26.2013

Retrieved from "http://foss2serve.org/index.php?title=MouseTrap_Dev_Help&oldid=464"
Personal tools
Namespaces
Variants
Actions
Events
Learning Resources
HFOSS Projects
Evaluation
Navigation
Toolbox