Project Evaluation Activity V1

Revision as of 16:43, 27 February 2013

Project Evaluation

Preparation:

Background:

This 8 minute video gives you an overview of rubric and how it is used.

[ Watch the video.]

Directions:

Part 1-Evaluate Mission Critical Criteria

Evaluate Viability

1. Size/Scale/Complexity - An ideal project should be neither overly simple nor overly complex. One heuristic to use is the number of contributors as an indicator project complexity.
   1. Go to Ohloh.net, type Mifos into the Search Projects box, Click on Mifos to see the Project Summary page
   2. Scroll down to the Community area and calculate the average number of contributors in the last 12 months. The average was 9 so it passed the minimum average number of contributors metric of 6.
   3. Go to the Mifos web page and choose Tech Overview from the Contributors tab. From examination of the technology stack, the architecture looks modular and further search shows it is documented elsewhere on the site.
   4. Result-Based on the modular design and meeting the minimum average number of contributors metric, the project is scored a 2 for size/scale/complexity.
2. Activity - To support student participation a project should be reasonably active. Number of committers can be used as an indicator of activity.
   1. Return to the Mifos project summary page in Ohloh. Scroll to the Activity area on the page.
   2. Compute the 12-month average of commits. The 12-month average was about 108, much higher than the minimum average level of commits recommended for activity.
   3. Result-Because commits exceed the favorable level of activity for this project it may be a little large/complex. However, still appears manageable, the project is scored a 2 for activity.
3. Community
   1. Examine download activity
      1. Go to Sourceforge.net and enter Mifos into the search box.
      2. Choose Mifos-Microfinance Open Source from the search results.
      3. Click on the number of downloads that is listed on the project page.
      4. Change the date range to give a graph of downloads over the last year.
   2. Examine mailing list activity.
      1. Return to the Mifos web page. From the Contributor tab choose the Join the Mailing List link to get access to the project mailing list.
      2. Once granted access, examine the Developer mailing list for activity.
   3. Examine the IRC channel
      1. Again from the Contributor tab on the Mifos web page, choose the IRC link.
      2. Examine the IRC channel for activity.
   4. Result-Downloads appear steady so the project has a community of users. Developers are responsive on mailing list and developers have a presence on IRC. Project is scored a 3.

Evaluate Approachability

Here you are evaluating a project's on-ramp to contribution, scoring as follows:

1-Insufficient-Few or no pointers on how to become involved.

2-Sufficient-Suggestions about how to get involved other than contributing money with accompanying high-level instructions.

3-Ideal-Obvious link to get started, list of suggestions for things to do and detailed instructions.

1. Link to get started-From the Contributors tab on the Mifos web page there is a Get Started page with links to what Mifos is, how to contribute, community processes, and tools used.
2. List of suggestions for things to do - Clicking the Get Involved link on the Get Started page provides a list of ways to contribute including testing, translation, development and documentation. There is also a volunteer bug queue listed as a good way for developers to get started.
3. Detailed instructions-Instructions are provided in many areas including process, architecture, licensing, product functionality, and developer documentation.
4. Result-Was scored a 3.

Evaluate Suitability

1. Appropriate Artifacts -Since evaluation is dependent on class objectives, in this example we'll assume an objective is to learn the process of working in authentic development project through contributing bug fixes.
   1. Opportunities to contribute bug fixes - Examined the volunteer bite-sized bug queue accessible from the Volunteer page under the Contributors tab on the Mifos web page. There were 10 open bugs for new contributors. There were many more listed for more experienced contributors.
   2. Documentation on how to contribute bug fixes - From the Tech Overview page accessible from the Contributors tab on the web site there is a link to details on the code submission process.
   3. Result - May score a 2 or 3, depends on number of bugs and class size.
2. Contributor Support-Does the project have a high volume of guidance to help students as they learn?
   1. Are communication tools documented?-
   2. Do developers have a web presence?-
   3. Are operating processed documented?-
   4. Do questions posed have timely and supportive answers?-

Decision to move to secondary criteria

Result - No criterion scored lower than a 2 so project passes on mission-critical criteria. Move on to evaluating the project on secondary criteria.

Part 2-Evaluate Secondary Criteria

Evaluate Viability

1. Domain
   1. Does this project require domain knowledge that may be difficult for students to learn? -
   2. Result -
2. Maturity
   1. To have the organization to support student learning, the project should have at least one stable production release -
   2. Result -
3. User Support
   1. The project should have clear instructions for downloading, installing, and using the project -
   2. Result -
4. Roadmap
   1. Student learning is best supported by projects that have a roadmap that includes new feature development, a method for users to submit new feature requests and a process for identifying how new features are prioritized.
   2. Result -

Evaluate Approachability

1. Contribution Types
   1. Does the project contain opportunities for multiple types of contribution and of the type that fits the class?
   2. Result -
2. Openness to Contributions
   1. Acceptance of a student contribution to a project provides valuable affirmation to student learning. Determine whether the project accepts student patches.
   2. Result -
3. Student Friendliness
   1. Do community members moderate the tone of communication? Review mailing lists and IRC to gauge tone.
   2. Result -

Evaluate Suitability

1. Project Description
   1. Students must be able to understand the purpose of the project. Does the project clearly describe the product? -
   2. Result -
2. Platform
   1. What software and hardware platform does the FOSS project run on?
   2. Are there resources to support these platforms?
   3. Are students familiar with the platforms?
   4. Result -
3. Development Features - Is the class dependent on specific development features?
   1. Programming language -
   2. Development environment -
   3. Supporting technologies -
   4. Result -