MouseTrap Dev Help
(Difference between revisions)
(→backend) |
(→Meeting summary) |
||
| Line 329: | Line 329: | ||
===Meeting summary=== | ===Meeting summary=== | ||
--------------- | --------------- | ||
| − | + | '''Status''' | |
* LINK: | * LINK: | ||
http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/main.html | http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/main.html | ||
| Line 342: | Line 342: | ||
(amber, 18:20:24) | (amber, 18:20:24) | ||
| − | + | '''Git and mousetrap installation''' | |
* creating your own branch in git: git checkout -b my_fantastic_branch | * creating your own branch in git: git checkout -b my_fantastic_branch | ||
(Stoney, 18:29:24) | (Stoney, 18:29:24) | ||
| Line 349: | Line 349: | ||
(amber, 18:32:31) | (amber, 18:32:31) | ||
| − | + | '''Issues''' | |
* LINK: http://code.google.com/p/pyopencv/ (john, 18:46:37) | * LINK: http://code.google.com/p/pyopencv/ (john, 18:46:37) | ||
* ctypes-opencv is a package that brings Willow Garage's (formerly | * ctypes-opencv is a package that brings Willow Garage's (formerly | ||
| Line 362: | Line 362: | ||
18:51:45) | 18:51:45) | ||
| − | + | '''Integration methods''' | |
* LINK: | * LINK: | ||
http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/dir_9d66d24675e04ade776f5269c3621ea1.html | http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/dir_9d66d24675e04ade776f5269c3621ea1.html | ||
| Line 373: | Line 373: | ||
Meeting ended at 19:16:36 CET. | Meeting ended at 19:16:36 CET. | ||
| − | + | '''Action Items''' | |
------------ | ------------ | ||
* Create Module documentation for MouseTrap | * Create Module documentation for MouseTrap | ||
| − | + | '''People Present (lines said)''' | |
--------------------------- | --------------------------- | ||
* amber (125) | * amber (125) | ||
Revision as of 23:14, 21 February 2013
Contents |
Standard Dev Environment
- Fedora 18
- OpenCV 2.4.X - whatever recent version is supported
- Python 2.X for now upgrading to Python 3 when we get the OpenCV issue fixed
Complete Git + Mousetrap Install
Let's get git
Open a terminal, become root Run Commands: -> cd /opt -> mkdir git * just fyi you can put your repo anywhere, this is just how I was taught -> yum install git-core * Install git -> cd /opt/git -> git clone git://github.com/amberheilman/mousetrap.git * Now you have pulled down the git repo -> git checkout fix_install * This is my branch -> git pull * DO NOT MAKE CHANGES HERE! We will all be working in separate branches!
Make a branch
-> git branch INSERT_BRANCH_NAME * this clones my branch and creates a new one -> git checkout INSERT_BRANCH_NAME * this opens up your branch that you just created -> git branch * now you can see the branch your in with a *, and all others under this git repo
Mousetrap Install method
Install Dependencies: -> yum install gnome-common -> yum install glib2-devel -> yum install intltool -> yum install python-devel -> yum install opencv-python -> yum install python-xlib Run Commands: -> cd /opt/git/mousetrap/src * THIS WILL NOT WORK OUTSIDE THIS DIR! -> git branch fix_install -> ./autogen.sh -> make -> make install -> mousetrap * You may have errors but they should be similar to my own (towards the end)
Make your first commit
-> git status * Shows all of the modified files
-> git add * * This is to add ALL files to commit list (MAKE SURE YOU WANT THEM ALL FIRST!)
-> git commit * Add a useful title to first line of your commit.
This is in vim so 'i' to insert ':x' to save and quit.
-> git push origin INSERT_BRANCH_NAME * This must be the branch you created in the git install.
This will ask for your git credentials, so have them ready.
The MouseTrap Program
Here is a complete dissection of the hierarchy done by gnome: http://gnome-mousetrap.sourcearchive.com/documentation/0.3plus-psvn17-2ubuntu1/main.html
Definitions
- Haar wavelet
- First proposed by Alfred Haar, the Haar wavelet is a series of square-shaped functions that when shown together form a basis otherwise known as a wavelet family and expressed in terms of an orthonormal function basis. For more information, see: [[1]]
- Haar-like features
- Uses adjacent rectangular regions in a specified detection window, sums up the pixel intensities and calculates the difference between those sums. They are called Haar-like features because they are computed using similar coeffients in the Haar wavelet transforms. These regions can then be concatenated with boosted classifiers into a classifier cascade to cross reference these regions with other positive samples to form a model for object detection. For more information, see [[2]] [[3]]
- Boosted Classifiers
- The result of increasing the accuracy of a classifier cascade through the means of positive object recognition of scaled images.
- Classifier Cascade
- Proposed by Paul Viola and refined by Rainer Lienhart, a classifier cascade
- ROI (Region of Interest)
- This is usually a subset of the original frame represented as a rectangle. It is most often compared to a classifier cascade to determine a positive match.
- Ocvfw (OpenCV FrameWork)
- A MouseTrap in house framework that manages OpenCV methods and includes functions to initiate the camera and detect further Haar-like features.
- Optical flow
- Lucas-Kanade method
- Singleton
- Restricts the instantiation of a class to one object so coordination over program actions can be achieved.
Classes
app
main.py
- Loads the Image Detection Module (idm)
commons.py
- Stores global variables for Ocvfw
* cv: OpenCV related variables * hg: OpenCV.highgui related variables. Set in ocvfw/idm/ color.py to be used in the creation of the Window and Trackbar * abs_path: the absolute path to the commons file * haar_cds: array with the haar xml file paths * colors: array with the types of colors the image can appear in * singleton: class instance to be passed through program globally
- Method to get color channel
* get_ch(color): returns channel corresponding to the color given (rgb, bgr, gray)
environment.py
debug.py
- This is used to display messages to the command line
- Very helpful to troubleshoot
ui
main.py
lib
addons
OpenCV Framework (ocvfw)
- This is the wrapper around OpenCV
ocvfw/ _ocv.py
- Contains three classes:
OcvfwBase: direct copy of backends/ OcvfwBase
OcvfwPython: direct copy of backends/ OcvfwPython
OcvfwCtypes: direct copy of backends/ OcvfwCtypes
ocvfw/ idm
- Detects features based on an xml file
ctypesopencv
- This is used instead of the python bindings for OpenCV
pocv.py
- Returns an instance of the idm
ocvfw/ haars
- Used thousands of samples to concatenate an xml file to predict features
dev/ camera
* first gets a region of interest and then matches it against a haar classifier
backend
- link to image on how the backend distributes variables ["]
- Contains three classes:
backend/ OcvfwBase
backend/ OcvfwPython
- inherits from OcvfwBase
- imports global and local variables from Commons.hg (highgui related variables) and Commons.cv(Opencv) variables
- has the ability to get_motion_points but is not used
- Methods:
get_haar_roi_points: finds regions of interest within the entire frame image and returns the matches against the classifier cascade using the HaarDetectObjects() OpenCV function. get_haar_points: resizes the image by 1.5
backend/ OcvfwCtypes
- imports global and local variables from Commons.hg (highgui), Commons.cv(which is a CV common lib), and OcvfwBase
Files affected by OpenCV 2.4.3 upgrade
- main.py
- ocvfw
- _ocv.py
- ocvfw/ idm
- eyes.py
- finger.py
- forehead.py
- color.py
- commons.py
- ocvfw/ dev
- camera.py
- ocvfw/ backends
- OcvfwBase.py
- OcfwCtypes.py
- OcvfwPython.py
Opencv
General Concerns
Here we place all of the concerns when migrating to a newer version of opencv or the repercussions of not upgrading.
Migration to Python 3
- Opencv 2.4.3 does not support Python 3
Camera drivers
- Currently, only some camera drivers are supported with the new version of opencv, leading to problems regarding capture. One issue is the inability to cancel the program webcam capture.
Yum repo
- Right now, only the 2.3.1 version of opencv is available in the yum repository
OpenCV Info
Basics and Source Code
OpenCV (Open Source Computer Vision) is a library of programming functions for the realtime computer vision. OpenCV is released under the liberal BSD license and hence it's free for both academic and commercial use. It has C++, C, Python and Java (Android) interfaces and supports Windows, Linux, Android and Mac OS. The library has more than 2500 optimized algorithms. Adopted all around the world, OpenCV has more than 47 thousand people of user community and estimated number of downloads exceeding 5 million. Usage ranges from interactive art, to mines inspection, stitching maps on the web or through advanced robotics.
To access the OpenCV repository directly: git clone git://github.com/itseez/opencv.git
- A good source for this is: http://code.opencv.org/projects/opencv/wiki/Working_with_OpenCV_git_repository
- The rest of the history plus matches between git commits and SVN revisions are stored at a separate "OpenCV Attic" repository: git://code.opencv.org/opencv_attic.git.
- Also OpenCV Extra was put to a separate repository: git://code.opencv.org/opencv_extra.git.
History
"OpenCV was started at Intel in 1999 by Gary Bradski for the purposes of accelerating research in and commercial applications of computer vision in the world and, for Intel, creating a demand for ever more powerful computers by such applications. Vadim Pisarevsky joined Gary to manage Intel's Russian software OpenCV team. Over time the OpenCV team moved on to other companies and other Research. Several of the original team eventually ended up working in robotics and found their way to Willow Garage. In 2008, Willow Garage saw the need to rapidly advance robotic perception capabilities in an open way that leverages the entire research and commercial community and began actively supporting OpenCV, with Gary and Vadim once again leading the effort." NEED A REF OR LINK
So what is OpenCV?
- OpenCV is a computer vision library in C++.
- OpenCV 2 was released in 2009 with additional functionality and increased performance
- CV is a Python wrapper for OpenCV
- CV2 is a Python wrapper for OpenCV 2 and includes CV
- If you want CV functions you have to
import cv2.cv as cv
The full name is "Open Source Computer Vision Library." It is a library of programming functions aimed at real-time computer vision. It was developed by Intel and now Willow Garage and Itseez support it. It is free for use under the open source BSD license. Also, it can cross-platform.
Where did it come from?
OpenCV emerged from the Intel Research Initiative; it is related to Intel's Performance Library, which today is called Integrated Performance Primitives (IPP). The project launched in 1999. It was looking for CPU-intensive applications. The project goals were to:
- advance vision research by open and optimized infrastructure
- disseminate vision knowledge with readable code
- and advance commercial applications
Version List:
alpha-release at CVPR 2000 five beta-releases 2001-2005 Version 1.0 2006 Continuation of development by Willow Garage 2008 (pre-release version 1.1) Version 2.0 2009 Versions 2.1, 2.2 2010 Version 2.3 2011 Version 2.4.0 May 2012 Version 2.4.1 June 2012 Version 2.4.2 July 2012
Application Use:
2D and 3D Feature Toolkits Egomotion Estimation Facial Recognition System Gesture Recognition Human-Computer Interaction (HCI) Mobile Robotics Motion Analysis Object Detction and Recognition Segmentation Stereo Vision: Depth Perception from 2 Cameras Structure from Motion (SFM) Motion Tracking
(http://www.cvl.isy.liu.se/education/graduate/opencv/Lecture1_History.pdf)
Definitions
Modules Available
*core: a compact module defining basic data structures, including the dense multi-dimensional array Mat and basic functions used by all other modules *imgproc: an image processing module that includes linear and non-linear image filtering, geometrical image transformations (resize, affine and perspective warping, generic table-based remapping), color space conversion, histograms, and so on *video: a video analysis module that includes motion estimation, background subtraction, and object tracking algorithms *calib3d: basic multiple-view geometry algorithms, single and stereo camera calibration, object pose estimation, stereo correspondence algorithms, and elements of 3D reconstruction *features2d: salient feature detectors, descriptors, and descriptor matchers *objdetect: detection of objects and instances of the predefined classes (for example, faces, eyes, mugs, people, cars, and so on) *highgui: an easy-to-use interface to video capturing, image and video codecs, as well as simple UI capabilities *gpu: GPU-accelerated algorithms from different OpenCV modules *some other helper modules, such as FLANN and Google test wrappers, Python bindings, and others
OpenCV - Great Resources
- http://www.cvl.isy.liu.se/education/graduate/opencv/Lecture1_History.pdf
- http://docs.opencv.org/
- http://docs.opencv.org/opencv_tutorials.pdf (Tutorial)
- http://opencv.org/documentation.html
- http://www.pages.drexel.edu/~nk752/tutorials.html (Tutorials)
- http://www.laganiere.name/opencvCookbook/ (Companion Site to an OpenCV2 Book)
- http://stackoverflow.com/questions/10417108/what-is-different-between-all-these-opencv-python-interfaces (Difference between CV and CV2)
- http://opencv-users.1802565.n2.nabble.com/ (OpenCV mailing list archive)
- http://pr.willowgarage.com/wiki/OpenCVMeetingNotes (OpenCV meeting minutes)
- As of May 22, 2012, the meetings notes moved to: http://code.opencv.org/projects/opencv/wiki/Meeting_notes
- http://opencv.willowgarage.com/wiki/OpenCV%20Change%20Logs (OpenCV Change Logs)
Opencv 2 Info
Some Differences Between cv and cv2
- single import of all of OpenCV using import cv
- OpenCV functions no longer have the "cv" prefix
- simple types like CvRect and CvSca,lar use Python tuples
- sharing of Image storage, so image transport between OpenCV and other systems (e.g. numpy and ROS) is very efficient
- complete documentation for the Python functions
OpenCV 2 - Great Resources
cv 2.4.3
- http://docs.opencv.org -> documentation
- http://docs.opencv.org/opencv2refman.pdf / http://cvhci.anthropomatik.kit.edu/download/visionhci09/opencv.pdf -> pdf documentation resources
- http://fossies.org/dox/OpenCV-2.4.3/index.html -> Complete Hierarchical guide to opencv
- http://docs.opencv.org/doc/tutorials/tutorials.html -> tutorials on opencv
- http://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html#array-attributes -> structure of numpy arrays\
A cv2 example
Here is a little program that will capture video from the webcam and display it (you might have to CTRL+C)
"""
This module is used for testing the opencv2 capabilities
"""
import cv2
#get webcam feed
capture = cv2.VideoCapture(0)
while True:
#combines VideoCapture.grab() and VideoCapture.retrieve()
retrieval_value, image = capture.read()
#shows captured image in a window
cv2.imshow("webcam", image)
#will stop capture with capatible webcam
if cv2.waitKey(10) == 27:
break
cv 2.1
- http://opencv.willowgarage.com/documentation/python/index.html -> documentation
- http://nullege.com/codes/search/cv -> code samples
OpenCV Community
- The #opencv IRC channel on Freenode -- currently 'Un-official'
- http://opencv.willowgarage.com/wiki/FullOpenCVWiki#Welcome.2FPeople.People
- http://tech.groups.yahoo.com/group/OpenCV/ -> User Group on Yahoo
- http://answers.opencv.org/questions/ -> Q&A
- http://code.opencv.org/projects/opencv/issues -> Bugs
Code Meeting Notes
- Meetings occur on Wednesday's at noon (starting on 2/20) on irc.gnome.org #mousetrap - holiday and other circumstances permitting.
2.19.2013
Agenda
- Status
- Go over git+mousetrap installation [[4]]
- Address any issues
- Go over mousetrap hierarchy [[5]]
- Go over opencv integration methods [[6]]
- Next Steps
Meeting summary
Status
* LINK: http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/main.html (amber, 18:14:27) * LINK: http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/main.html (amber, 18:14:29) * LINK: http://code.opencv.org/projects/opencv/wiki/Meeting_notes (Dark_Rose, 18:17:39) * LINK: http://www.xcitegroup.org/foss2serve/index.php/MouseTrap_Dev_Help#The_MouseTrap_Program (amber, 18:20:24)
Git and mousetrap installation
* creating your own branch in git: git checkout -b my_fantastic_branch (Stoney, 18:29:24) * LINK: http://git-scm.com/book/en/Git-Branching-Basic-Branching-and-Merging (amber, 18:32:31)
Issues
* LINK: http://code.google.com/p/pyopencv/ (john, 18:46:37) * ctypes-opencv is a package that brings Willow Garage's (formerly Intel's) Open Source Computer Vision Library (OpenCV) to Python. OpenCV is a collection of algorithms and sample code for various computer vision problems. The goal of ctypes-opencv is to provide Python access to all documented functionality of OpenCV. (amber, 18:46:39) * LINK: http://code.google.com/p/pyopencv/ (amber, 18:46:46) * http://pythonhosted.org/pyopencv/2.1.0.wr1.2.0/ (amber, 18:51:37) * LINK: http://pythonhosted.org/pyopencv/2.1.0.wr1.2.0/ (amber, 18:51:45)
Integration methods
* LINK: http://gnome-mousetrap.sourcearchive.com/documentation/0.4-2/dir_9d66d24675e04ade776f5269c3621ea1.html (amber, 18:58:53) * LINK: http://gnome-mousetrap.sourcearchive.com/documentation/0.3plus-psvn17-2ubuntu1/classmouseTrap_1_1ocvfw_1_1ocvfw.html (amber, 19:02:40) * ACTION: Create Module documentation for MouseTrap (amber, 19:14:55)
Meeting ended at 19:16:36 CET.
Action Items
- Create Module documentation for MouseTrap
People Present (lines said)
- amber (125)
- john (39)
- Stoney (31)
- logan_h (14)
- darci (8)
- Dark_Rose (5)
- tota11y (2)
Conclusions
- documentation on Mousetrap's OpenCV FrameWork is needed for further inspection.
