Evaluate a Project (Activity)

From Foss2Serve
(Difference between revisions)
Jump to: navigation, search
(→Part 2-Walk through of an evaluation of the Mifos project - Use the blank evaluation template to help record your results.)
m (Evaluated for adoption readiness)
 
(78 intermediate revisions by 9 users not shown)
Line 1: Line 1:
−
== Project Evaluation ==
+
__NOTOC__
 +
=<span style="color:#ff0000"> This activity has been revised. The newer version is [http://foss2serve.org/index.php/Project_Evaluation_(Activity) here] </span>=
  
−
=== Preparation: ===
+
{{Learning Activity Overview
 +
|title=
 +
Evaluate a Project
 +
|overview=  
 +
Learners will gain an understanding of the breadth of available HFOSS projects. Learners will also gain an understanding of the identifying characteristics of HFOSS projects including pattern of contributions, patterns of commits, programming languages used, and more. 
 +
|prerequisites=
 +
* Completion of [[FOSS Field Trip (Activity)]] or understanding of SourceForge and OpenHub; Understanding of the course in which students will be participating in an HFOSS project.
 +
|objectives=
 +
* Identify likely HFOSS projects.
 +
|process skills=
 +
}}
  
−
{| border="1"
+
=== Background ===
−
|-
+
−
|'''Description''' || Learners will gain an understanding of the breadth of available FOSS projects. Learners will also gain an understanding of the identifying characteristics of FOSS projects including pattern of contributions, patterns of commits, programming languages used, and more. 
+
−
|-
+
−
|'''Source''' ||
+
−
|-
+
−
|'''Prerequisite Knowledge''' || Completion of Browsing a Forge Activity or understanding of SourceForge and Ohloh; Understanding of course in which students will be participating in an HFOSS project.
+
−
|-
+
−
|'''Estimated Time to Completion''' || 60-90 minutes
+
−
|-
+
−
|'''Learning Objectives''' ||Ability to utilize the rubric to identify likely HFOSS projects.
+
−
|-
+
−
|'''Materials/Environment''' || Access to Internet/Web and web browser, [http://xcitegroup.org/softhum/lib/exe/fetch.php?media=g:evalfossprojects.doc SIGCSE paper on evaluating FOSS projects],  [http://www.foss2serve.org/images/foss2serve/0/0c/Blank_Evaluation_Template.xlsx Blank evaluation template]
+
−
|-
+
−
|'''Additional Information''' || [http://www.foss2serve.org/index.php/HFOSS_Communities List of HFOSS projects]
+
−
|-
+
−
|'''Rights''' || Licensed CC BY-SA
+
−
|-
+
−
|'''Turn In''' || Wiki posting of evaluation of a project from the [http://www.foss2serve.org/index.php/HFOSS_Communities list of HFOSS projects]
+
−
|}
+
  
−
=== Background: ===
+
This activity is intended to give you an overview of what to consider when evaluating an HFOSS project for student participation and for you to gain experience using the rubric.
−
This activity is intended to give you an overview of what to consider when evaluating a FOSS project for student participation and for you to gain experience using the rubric.
+
  
−
=== Directions: ===
+
=== Directions ===
−
====Part 1-Learn about the rubric====
+
−
#[http://youtu.be/MAGet2D5o2c Watch the video describing mission critical criteria]
+
−
#[http://youtu.be/e4lnIXjqczU Watch the video describing secondary criteria]
+
  
−
====Part 2-Walk through of an evaluation of the Mifos project - Use the [http://www.foss2serve.org/images/foss2serve/0/0c/Blank_Evaluation_Template.xlsx blank evaluation template] to help record your results. ====
+
==== Walk through of an evaluation of the OpenMRS project ====
−
:'''Mission Critical criteria-Viability'''
+
  
−
#Size/Scale/Complexity - An ideal project should be neither overly simple nor overly complex.  One heuristic to use is the number of contributors as an indicator of project complexity.
+
There are many criteria which should be looked at when determining if a project is appropriate to use in your class.  These criteria are broken into two groups - mission critical and secondary.
−
##Go to the Mifos web page (http://www.mifos.org) and choose Tech Overview from the Volunteers tab.  From examination of the technology stack, the architecture looks modular and further search shows it is documented elsewhere on the site.
+
For this exercise, the secondary criteria are optional.
−
##Based upon the results from Ohloh (gathered in the FOSS Field Trip activity) and the information from the Mifos page, what would you score this project for size/scale/complexity.
+
−
#Activity - To support student participation a project should be reasonably active.  Number of commits can be used as an indicator of activity. 
+
−
##Based upon the number of commits (gathered i the FOSS Field Trip activity) how would you rate the activity of the project?
+
−
#Community - A suitable project has an active user community.  While it is difficult to quantitatively evaluate the activity of a user community, some indicators include a regular history of project downloads and documentation updates over time, current activity on user mailing lists, and testimonials on the project web site.
+
−
##Examine download activity
+
−
###Go to Sourceforge.net and enter Mifos into the search box. 
+
−
###Choose Mifos-Microfinance Open Source from the search results.
+
−
###Click on the number of downloads that is listed on the project page.
+
−
###Change the date range to give a graph of downloads over the last year. 
+
−
##Examine [https://groups.google.com/forum/#!forum/mifosusers user mailing list] activity
+
−
##Examine the [http://ci.mifos.org/irclogs/%23mifos/ IRC logs]
+
−
##Result:  Downloads appear steady so the project has a community of users.  Developers are responsive on mailing list and have a presence on IRC.  Project is scored a 3.
+
  
 +
:'''Mission Critical Criteria - Viability'''
  
−
:'''Mission Critical criteria-Approachability'''
+
# Size/Scale/Complexity - An ideal project should be neither overly simple nor overly complex.
 +
## Go to the OpenMRS web page (http://openmrs.org/), scroll to the bottom and choose ''OpenMRS Wiki'' (under ''Other OpenMRS sites''). From the menu on the left expand the ''Developer Guide'' and the ''Getting Started as a Developer'' options and then choose ''Technical Overview''. From examination of the technology stack, the architecture looks modular and further search shows it is documented elsewhere on the site. This provides a first look at the complexity of the application and the number and various different technologies involved.
 +
## Based upon the results from OpenHub (gathered in the FOSS Field Trip activity) and the information from the OpenMRS Technical Overview page, think about the size of the code base and how many different technologies and layers are involved in the application. What would you score this project for size/scale/complexity on a scale of one to three where one is "low" and three is "high".
 +
# Activity - To support student participation a project should be reasonably active.  Number of commits can be used as an indicator of activity. 
 +
## Based upon the number of commits (gathered in the FOSS Field Trip activity) would you consider this project active?  Why or why not?
 +
# Community - A suitable project has an active user community.  While it is difficult to quantitatively evaluate the activity of a user community, some indicators include a regular history of project downloads and documentation updates over time, current activity on user mailing lists, and testimonials on the project web site.
 +
## Examine download activity
 +
### Go to [http://sourceforge.net/ sourceforge.net] and enter OpenMRS into the search box. 
 +
### Choose OpenMRS from the search results.
 +
### Click on the number of downloads that is listed on the project page.
 +
### Change the date range to give a graph of downloads over the last year. 
 +
## OpenMRS has begun migrating legacy mailing list activity to OpenMRS Talk. Examine [https://talk.openmrs.org/ discussion] activity
 +
## Examine the [https://botbot.me/freenode/openmrs/ IRC logs]
 +
## Based upon the download history, discussion activity, and IRC activity, do you feel this project has a good community?  Why or why not?
  
−
::Here you are evaluating a project's on-ramp to contribution, scoring as follows:
+
:'''Mission Critical Criteria - Approachability'''
  
−
::1-Insufficient-Few or no pointers on how to become involved.
+
:Here you are evaluating a project's on-ramp to contribution, scoring as follows:
  
−
::2-Sufficient-Suggestions about how to get involved other than contributing money with accompanying high-level instructions.
+
::1 - Insufficient - Few or no pointers on how to become involved.
  
−
::3-Ideal-Obvious link to get started, list of suggestions for things to do and detailed instructions.
+
::2 - Sufficient - Suggestions about how to get involved other than contributing money with accompanying high-level instructions.
  
−
#Link to get started-There is a [http://mifos.org/contributors/get-started Get Started page] with links to what Mifos is, how to contribute, community processes, and tools used. 
+
::3 - Ideal - Obvious link to get started, list of suggestions for things to do and detailed instructions.
−
#List of suggestions for things to do - [http://mifos.org/contributors/volunteer-projects The Volunteer Project page] provides a list of ways to contribute including testing, translation, development and documentation.  There is also a volunteer bug queue listed as a good way for developers to get started. 
+
−
#Detailed instructions- On the web site instructions and information are provided in many areas including process, architecture, licensing, product functionality, and developer documentation.
+
−
#Based upon the resources you looked at, how would you score how approachable the Mifos is as a project?
+
  
 +
# Examine project on-ramp.
 +
## Link to getting started - The website has a [http://openmrs.org/help/ Get Involved page] with links to ways you can contribute and share your ideas. 
 +
## Each of the links (Develop, Test, Document, Translate) contain more detailed information about what and how you can contribute.
 +
## The [https://wiki.openmrs.org/display/docs/Getting+Started+as+a+Developer Getting Started as a Developer] page contains a detailed list of how to get started including a list of introductory issues.
 +
## Detailed instructions - The [https://wiki.openmrs.org/display/docs/Developer+Guide Developer Guide] contains instructions and information in many areas including process, architecture, tools, and developer documentation.
 +
## Based upon the resources you looked at, how would you rate the approachability of the OpenMRS project?
  
−
:'''Mission Critical criteria-Suitability'''
 
−
#Appropriate Artifacts -Since evaluation is dependent on class objectives, in this example we'll assume an objective is to learn the process of working in authentic development project through contributing bug fixes.
 
−
##Opportunities to contribute bug fixes - Examined the [https://mifosforge.jira.com/issues/?filter=10305 volunteer bite-sized bug queue].  There were 10 open bugs for new contributors.  There were many more listed for more experienced contributors.
 
−
##Documentation on how to contribute bug fixes - From the [http://mifos.org/contributors/tech-overview Tech Overview page] there are links to details on the code submission process. 
 
−
##Based upon the number of bugs suitable for students to tackle and class size, how would you rate Mifos?
 
−
#Contributor Support-Does the project have a high volume of guidance to help students as they learn?
 
−
##Are communication tools documented?-Communication tools are documented under the Collaboration and Communication section of the [http://mifos.org/contributors/development-tools Development Tools page].  Instructions on how to access the mailing lists with tips on how to participate are available from the [http://mifos.org/community/communications Communications page].
 
−
##Do developers have a web presence?-Examination of [http://ci.mifos.org/irclogs/%23mifos/ IRC logs] shows scattered activity over the last week. 
 
−
##Are operating processed documented?-Links to information about coding standards, code submission process, and commit privileges process can be found on the [http://mifos.org/contributors/tech-overview Tech Overview page].  The process for making feature requests and for prioritizing feature request is available on the [http://mifos.org/product/roadmap Roadmap page].
 
−
##Do questions posed have timely and supportive answers?-Responses to [https://groups.google.com/forum/#!forum/mifosusers user mailing list] and [https://groups.google.com/forum/#!forum/mifosdeveloper developer mailing list] over the last month have timely and supportive responses.
 
−
##How would you orate the communications tools of Mifos?
 
  
 +
:'''Mission Critical Criteria-Suitability'''
 +
 +
# Appropriate Artifacts - Since evaluation is dependent on class objectives, in this example we'll assume the objective is to learn the process of working in an authentic development environment by contributing bug fixes to OpenMRS.
 +
## Opportunities to contribute bug fixes - Examine the issues found at the bottom of the [https://wiki.openmrs.org/display/docs/Getting+Started+as+a+Developer getting started as a developer page].  How are the introductory issues categorized?  How many issues are listed?
 +
## Documentation on how to contribute bug fixes - On the [https://wiki.openmrs.org/display/docs/Tickets Tickets page] there is information on how to create and work on an issue, including links to coding standards and the code submission process. Review this information.
 +
## Based upon the number of bugs suitable for students to tackle and information on the process of how to submit bug fixes, do you think this would be an appropriate project for your students?  Why or why not?
 +
# Contributor Support - Does the project have a high volume of guidance to help students as they learn?
 +
## Communication Tools - Communication tools are directly available from any of the Wiki Spaces (Documentation, Projects, Resources). The [https://wiki.openmrs.org/display/RES/Home Resources page] contains links to OpenMRS Talk and IRC Chat, as well as links to group meetings (under Events), and training opportunities. 
 +
## Web Presence - Examine the [https://botbot.me/freenode/openmrs/ IRC logs]. Has there been activity during the last week? 
 +
## Operating Processes - Links to information about coding standards, the code submission process, and commit privileges can be found on the [https://wiki.openmrs.org/display/docs/How-To+Submit+Code How-To Submit Code page].  The process for making feature requests is available on the [https://wiki.openmrs.org/display/docs/Tickets Tickets page]. Are these processes well documented?
 +
## Response to Questions - Review a few of the posts on the OpenMRS [https://talk.openmrs.org/ discussion platform]. Do posts to this forum receive timely and supportive responses?
 +
## How would you rate the support that newcomers to OpenMRS receive?
 +
 +
 +
:'''On your wiki page, write a summary of why you think the OpenMRS project would be suitable for your course.  Be sure to include information about the course and reasons why OpenMRS would be a good/poor match.
 +
 +
 +
:'''Overall evaluation for Mission Critical criteria''' - If the mission-critical criteria seemed reasonable, then you may want to evaluate the following secondary criteria.  If the mission-critical was not reasonable then the project would not be considered suitable for student participation.
 +
 +
 +
:'''Secondary Criteria - Viability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment'''
 +
 +
# Domain
 +
## Does this project require domain knowledge that may be difficult for students to learn? OpenMRS is a medical records system. Students should be able to grasp it well enough to contribute a bug fix, which is the learning objective assumed in this example.
 +
## How would you rate the understandability of OpenMRS?
 +
# Maturity
 +
## To have the organization support student learning, the project should have at least one stable production release. The [https://wiki.openmrs.org/display/RES/Platform+Release+Notes Platform Release Notes page] lists releases.
 +
## Does OpenMRS have enough of a stable base to support student learning?  Why or why not?
 +
# User Support
 +
## The project should have clear instructions for downloading, installing, and using the project. As noted previously, the [https://wiki.openmrs.org/display/docs/Getting+Started+as+a+Developer Getting Started as a Developer page] provides detailed information about setting up and using the required tools, in addition there are detailed instructions related to installation, configuration, system requirements, and troubleshooting, including videos.
 +
## Does the documentation seem sufficient for getting students started?
 +
# Roadmap
 +
## Student learning is best supported by projects that have a roadmap that includes new feature development, a method for users to submit new feature requests and a process for identifying how new features are prioritized. Feature requests are made through JIRA, the OpenMRS issue tracker. Road map planning and the process for prioritizing feature requests is available on the [https://wiki.openmrs.org/display/docs/Technical+Road+Map+Planning Technical Roadmap Planning page].  Here you will find information about the planning process and how to participate in the planning process. The [https://wiki.openmrs.org/display/docs/Technical+Road+Map Technical Road Map page] identifies features, their current status, and a point of contact, in addition to expected dates of completion.
 +
##Does the roadmap provide you with enough information to make a decision about using it in your course?
 +
 +
 +
:'''Secondary Criteria - Approachability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment'''
 +
 +
# Contribution Types
 +
## Does the project contain opportunities for multiple types of contribution and of the type that fits the class? There are multiple projects for testers, tech writers, and developers.  These can be seen on the [http://openmrs.org/help/ Get Involved page].  Are the number of and type of bugs available suitable for students given your course and the class size? Are there other ways that students could contribute?
 +
# Openness to Contributions
 +
## Acceptance of a student contribution to a project provides valuable affirmation to student learning.  Determine whether the project accepts student patches. The process for contribution is documented on the [https://wiki.openmrs.org/display/docs/Tickets Tickets page].  Does this project seem likely to accept student contributions?
 +
# Student Friendliness
 +
## Do community members moderate the tone of communication?  Review the discussion platform and IRC to gauge tone. Review the [https://talk.openmrs.org/ discussion platform] and [https://botbot.me/freenode/openmrs/ IRC logs].  What was the tone of the communication?
 +
 +
 +
:'''Secondary Criteria - Suitability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment'''
 +
# Project Description
 +
## Students must be able to understand the purpose of the project.  Does the project clearly describe the product? Can students understand the intended uses of the product? - The [http://openmrs.org/about/ About page] provides an overview of who, where, and what OpenMRS is, including a downloadable PDF file and a video.
 +
# Platform
 +
## What software and hardware platform does the FOSS project run on? Development environment can be built on Windows, Linux or Mac OS X completely with FOSS software.  (Project development information found [https://wiki.openmrs.org/display/docs/Step+by+Step+Installation+for+Developers here])
 +
## Are there resources to support these platforms?
 +
## Are students familiar with the platforms?
 +
# Development Features - Is the class dependent on specific development features?  (Project development information found [https://wiki.openmrs.org/display/docs/Step+by+Step+Installation+for+Developers here])
 +
## Programming language - What is the primary language?
 +
## Development environment - What environments are supported? 
 +
## Supporting technologies - What technologies are suggested/required?
 +
 +
=== Deliverables ===
 +
 +
POSSE: On your user wiki page, a section describing your evaluation of OpenMRS as a suitable project for your course.
 +
 +
= Notes for Instructors =
 +
 +
The remaining sections of this document are intended for the instructor.  They are not part of the learning activity that would be given to students.
 +
 +
=== Assessment ===
 +
 +
* How will the activity be graded?
 +
* How will learning will be measured?
 +
* Include sample assessment questions/rubrics.
 +
 +
{| border="1" class="wikitable"
 +
! Criteria
 +
! Level 1 (fail)
 +
! Level 2 (pass)
 +
! Level 3 (good)
 +
! Level 4 (exceptional)
 +
|-
 +
| '''The purpose of the project'''
 +
|
 +
|
 +
|
 +
|
 +
 +
|-
 +
| '''Why the project is open source'''
 +
|
 +
|
 +
|
 +
|
 +
 +
|}
 +
 +
=== Comments ===
  
−
:'''Overall evaluation for Mission Critical criteria''' - If no mission-critical criteria were scored lower than a 2 the project should be then evaluated on secondary criteria.  Otherwise, the project would have been considered not suitable for student participation.
+
* What should the instructor know before using this activity?
 +
* What are some likely difficulties that an instructor may encounter using this activity?
  
 +
=== Variants and Adaptations ===
  
−
:'''Secondary criteria-Viability - Secondary criteria sections are OPTIONAL'''
+
[https://github.com/ChrisMurphyOnline/open-source-software-development-course/blob/master/activities/foss-evaluation-activity.txt POGIL-style combined FOSS Field Trip and Project Evaluation] used by [[User:Cmurphy|Chris Murphy]] in his [[FOSS Course, UPenn, Murphy]].
−
#Domain
+
−
##Does this project require domain knowledge that may be difficult for students to learn? - As a domain microfinance students should be able to grasp it well enough to contribute a bug fix, which is the learning objective assumed in this example.
+
−
##How would you rate the viability of Mifos?
+
−
#Maturity
+
−
##To have the organization to support student learning, the project should have at least one stable production release - The [http://mifos.org/product/roadmap roadmap page] lists releases.
+
−
##Does Mifos have enough of a stable base to support student learning?  How would you rate it?
+
−
#User Support
+
−
##The project should have clear instructions for downloading, installing, and using the project - There is a demo server, video, and slide presentation that explains system functionality.  This information can be found looking at pages listed under the Product tab that can be used to learn about the system.  There is also a [http://mifos.org/support/user-manual user manual] available. On the [http://mifos.org/product/download-mifos Download Mifos page], there are detailed instructions related to installation, configuration, system requirements, and troubleshooting.
+
−
##Rate the documentation for Mifos.
+
−
#Roadmap
+
−
##Student learning is best supported by projects that have a roadmap that includes new feature development, a method for users to submit new feature requests and a process for identifying how new features are prioritized - The process for making feature requests and for prioritizing feature request is available on the [http://mifos.org/product/roadmap Roadmap page].  The roadmap has features that were implemented in the last release, but no listing for the next release.
+
−
##Based upon the roadmap provided, how would you rate Mifos?
+
  
 +
=== Additional Information ===
  
−
:'''Secondary criteria-Approachability - Secondary criteria sections are OPTIONAL'''
+
* Explore this list of [[HFOSS Projects]] that may be of interest.
−
#Contribution Types
+
* Read the [http://foss2serve.org/images/foss2serve/a/ac/Evaluating_FOSS_Projects.docx SIGCSE paper on evaluating FOSS projects]
−
##Does the project contain opportunities for multiple types of contribution and of the type that fits the class? - There are multiple projects for testers, tech writers, and developers.  These can be seen on the [http://mifos.org/contributors/volunteer-projects Volunteer Projects page].
+
* Watch these videos introducing the FOSS project evaluation criteria:
−
##Result - May be a 1, 2 or 3 depending on whether the number of bugs is suitable for students is enough given the class size.  
+
*# [http://youtu.be/MAGet2D5o2c Mission critical criteria]
−
#Openness to Contributions
+
*# [http://youtu.be/e4lnIXjqczU Secondary criteria]
−
##Acceptance of a student contribution to a project provides valuable affirmation to student learning.  Determine whether the project accepts student patches. - The process for contribution is documented on the [http://mifos.org/contributors/tech-overview Tech Overview page].
+
−
##Result - Score a 3 because the contribution process is documented.
+
−
#Student Friendliness
+
−
## Do community members moderate the tone of communication?  Review mailing lists and IRC to gauge tone - Review of [https://groups.google.com/forum/#!forum/mifosusers user mailing list] and [https://groups.google.com/forum/#!forum/mifosdeveloper developer mailing list] and [http://ci.mifos.org/irclogs/%23mifos/ IRC logs] during evaluation of contributor support showed a positive tone during communication.
+
−
##Result - Score a 3, no inappropriate or demeaning messages. 
+
  
 +
{{Learning Activity Info
 +
|acm unit=
 +
|acm topic=
 +
|difficulty=
 +
|time=
 +
60-90 minutes.
 +
<span style="color:#FF0000">This activity can take a significant amount of time. We only expect you to spend 60-90 minutes exploring.</span>
 +
You may not complete the activity within this time. Of course you are welcome to spend more time if you wish.
 +
|environment=
 +
* Access to Internet/Web and web browser
 +
* [[Media:Evaluating_FOSS_Projects.docx | SIGCSE paper on evaluating FOSS projects]]
 +
* [http://www.foss2serve.org/images/foss2serve/0/0c/Blank_Evaluation_Template.xlsx Blank evaluation template referred to in the SIGCSE paper]
 +
|author=
 +
Michele Purcell
 +
|source=
 +
|license=
 +
{{License CC BY SA}}
 +
}}
  
−
:'''Secondary criteria-Suitability - Secondary criteria sections are OPTIONAL'''
 
−
#Project Description
 
−
##Students must be able to understand the purpose of the project.  Does the project clearly describe the product? Can students understand the intended uses of the product? - The [http://mifos.org/about About tab] on the web page has links to the vision for the product and how it is used by microfinance institutions.
 
−
##Result - Score a 3, how the product is used and the vision for it is well documented and should be understandable by students. 
 
−
#Platform
 
−
##What software and hardware platform does the FOSS project run on? - Development environment can be built on Windows, Ubuntu or Mac desktop completely with FOSS software.  (Project development information found [https://mifosforge.jira.com/wiki/display/MIFOS/Workspace+2.0 here])
 
−
##Are there resources to support these platforms? - In this example, yes.
 
−
##Are students familiar with the platforms? - In this example, yes.
 
−
##Result - Score a 2, assumption in this example is students all have newer personal computers and given the ability to set up a development environment on different operating systems that makes the availability of student resources greater.  However, there is some risk because machine requirements for setting up developer environment are not provided and some documentation may be out of date. 
 
−
#Development Features - Is the class dependent on specific development features?  (Project development information found [https://mifosforge.jira.com/wiki/display/MIFOS/Workspace+2.0 here])
 
−
##Programming language - Is primarily Java.
 
−
##Development environment - Can be built on Windows, Ubuntu or Mac completely with FOSS software. 
 
−
##Supporting technologies - Suggested IDE is Eclipse, requires Maven, Jetty, and mySQL.
 
−
##Result - Need to gauge this on knowledge of students and requirements of class.  Assumption here is students know Java and are familiar with mySQL.  While students are not familiar with Maven and Jetty this may not be necessary for intro bug fix plus the community is very supportive so assistance can be found there.  Given there is some risk, score a 2.
 
  
 +
=== Suggestions for Open Source Community ===
 +
Suggestions for an open source community member who is working in conjunction with the instructor.
  
−
:'''Overall evaluation for secondary criteria''' - Add up your scores to determine the overall score.  If the total score for criteria is over 20, the project passes.  However, criteria scoring below 1 and criteria for which there was some risk noted should be reexamined to see if steps can be taken to mitigate risk.
 
  
−
[[Category: Learning_Activity]]
+
[[Category:Instructor Activities]]
 +
[[Category:Learning Activity]]
 +
[[Category:Use and Evaluate]]
 +
[[Category: Good Draft]]

Latest revision as of 17:44, 8 March 2017

This activity has been revised. The newer version is here

Title

Evaluate a Project

Overview

Learners will gain an understanding of the breadth of available HFOSS projects. Learners will also gain an understanding of the identifying characteristics of HFOSS projects including pattern of contributions, patterns of commits, programming languages used, and more.

Prerequisites
  • Completion of FOSS Field Trip (Activity) or understanding of SourceForge and OpenHub; Understanding of the course in which students will be participating in an HFOSS project.
Learning
Objectives 		After successfully completing this activity, the learner should be able to:
  • Identify likely HFOSS projects.
Process Skills
Practiced


Background

This activity is intended to give you an overview of what to consider when evaluating an HFOSS project for student participation and for you to gain experience using the rubric.

Directions

Walk through of an evaluation of the OpenMRS project

There are many criteria which should be looked at when determining if a project is appropriate to use in your class. These criteria are broken into two groups - mission critical and secondary. For this exercise, the secondary criteria are optional.

Mission Critical Criteria - Viability
  1. Size/Scale/Complexity - An ideal project should be neither overly simple nor overly complex.
    1. Go to the OpenMRS web page (http://openmrs.org/), scroll to the bottom and choose OpenMRS Wiki (under Other OpenMRS sites). From the menu on the left expand the Developer Guide and the Getting Started as a Developer options and then choose Technical Overview. From examination of the technology stack, the architecture looks modular and further search shows it is documented elsewhere on the site. This provides a first look at the complexity of the application and the number and various different technologies involved.
    2. Based upon the results from OpenHub (gathered in the FOSS Field Trip activity) and the information from the OpenMRS Technical Overview page, think about the size of the code base and how many different technologies and layers are involved in the application. What would you score this project for size/scale/complexity on a scale of one to three where one is "low" and three is "high".
  2. Activity - To support student participation a project should be reasonably active. Number of commits can be used as an indicator of activity.
    1. Based upon the number of commits (gathered in the FOSS Field Trip activity) would you consider this project active? Why or why not?
  3. Community - A suitable project has an active user community. While it is difficult to quantitatively evaluate the activity of a user community, some indicators include a regular history of project downloads and documentation updates over time, current activity on user mailing lists, and testimonials on the project web site.
    1. Examine download activity
      1. Go to sourceforge.net and enter OpenMRS into the search box.
      2. Choose OpenMRS from the search results.
      3. Click on the number of downloads that is listed on the project page.
      4. Change the date range to give a graph of downloads over the last year.
    2. OpenMRS has begun migrating legacy mailing list activity to OpenMRS Talk. Examine discussion activity
    3. Examine the IRC logs
    4. Based upon the download history, discussion activity, and IRC activity, do you feel this project has a good community? Why or why not?
Mission Critical Criteria - Approachability
Here you are evaluating a project's on-ramp to contribution, scoring as follows:
1 - Insufficient - Few or no pointers on how to become involved.
2 - Sufficient - Suggestions about how to get involved other than contributing money with accompanying high-level instructions.
3 - Ideal - Obvious link to get started, list of suggestions for things to do and detailed instructions.
  1. Examine project on-ramp.
    1. Link to getting started - The website has a Get Involved page with links to ways you can contribute and share your ideas.
    2. Each of the links (Develop, Test, Document, Translate) contain more detailed information about what and how you can contribute.
    3. The Getting Started as a Developer page contains a detailed list of how to get started including a list of introductory issues.
    4. Detailed instructions - The Developer Guide contains instructions and information in many areas including process, architecture, tools, and developer documentation.
    5. Based upon the resources you looked at, how would you rate the approachability of the OpenMRS project?


Mission Critical Criteria-Suitability
  1. Appropriate Artifacts - Since evaluation is dependent on class objectives, in this example we'll assume the objective is to learn the process of working in an authentic development environment by contributing bug fixes to OpenMRS.
    1. Opportunities to contribute bug fixes - Examine the issues found at the bottom of the getting started as a developer page. How are the introductory issues categorized? How many issues are listed?
    2. Documentation on how to contribute bug fixes - On the Tickets page there is information on how to create and work on an issue, including links to coding standards and the code submission process. Review this information.
    3. Based upon the number of bugs suitable for students to tackle and information on the process of how to submit bug fixes, do you think this would be an appropriate project for your students? Why or why not?
  2. Contributor Support - Does the project have a high volume of guidance to help students as they learn?
    1. Communication Tools - Communication tools are directly available from any of the Wiki Spaces (Documentation, Projects, Resources). The Resources page contains links to OpenMRS Talk and IRC Chat, as well as links to group meetings (under Events), and training opportunities.
    2. Web Presence - Examine the IRC logs. Has there been activity during the last week?
    3. Operating Processes - Links to information about coding standards, the code submission process, and commit privileges can be found on the How-To Submit Code page. The process for making feature requests is available on the Tickets page. Are these processes well documented?
    4. Response to Questions - Review a few of the posts on the OpenMRS discussion platform. Do posts to this forum receive timely and supportive responses?
    5. How would you rate the support that newcomers to OpenMRS receive?


On your wiki page, write a summary of why you think the OpenMRS project would be suitable for your course. Be sure to include information about the course and reasons why OpenMRS would be a good/poor match.


Overall evaluation for Mission Critical criteria - If the mission-critical criteria seemed reasonable, then you may want to evaluate the following secondary criteria. If the mission-critical was not reasonable then the project would not be considered suitable for student participation.


Secondary Criteria - Viability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment
  1. Domain
    1. Does this project require domain knowledge that may be difficult for students to learn? OpenMRS is a medical records system. Students should be able to grasp it well enough to contribute a bug fix, which is the learning objective assumed in this example.
    2. How would you rate the understandability of OpenMRS?
  2. Maturity
    1. To have the organization support student learning, the project should have at least one stable production release. The Platform Release Notes page lists releases.
    2. Does OpenMRS have enough of a stable base to support student learning? Why or why not?
  3. User Support
    1. The project should have clear instructions for downloading, installing, and using the project. As noted previously, the Getting Started as a Developer page provides detailed information about setting up and using the required tools, in addition there are detailed instructions related to installation, configuration, system requirements, and troubleshooting, including videos.
    2. Does the documentation seem sufficient for getting students started?
  4. Roadmap
    1. Student learning is best supported by projects that have a roadmap that includes new feature development, a method for users to submit new feature requests and a process for identifying how new features are prioritized. Feature requests are made through JIRA, the OpenMRS issue tracker. Road map planning and the process for prioritizing feature requests is available on the Technical Roadmap Planning page. Here you will find information about the planning process and how to participate in the planning process. The Technical Road Map page identifies features, their current status, and a point of contact, in addition to expected dates of completion.
    2. Does the roadmap provide you with enough information to make a decision about using it in your course?


Secondary Criteria - Approachability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment
  1. Contribution Types
    1. Does the project contain opportunities for multiple types of contribution and of the type that fits the class? There are multiple projects for testers, tech writers, and developers. These can be seen on the Get Involved page. Are the number of and type of bugs available suitable for students given your course and the class size? Are there other ways that students could contribute?
  2. Openness to Contributions
    1. Acceptance of a student contribution to a project provides valuable affirmation to student learning. Determine whether the project accepts student patches. The process for contribution is documented on the Tickets page. Does this project seem likely to accept student contributions?
  3. Student Friendliness
    1. Do community members moderate the tone of communication? Review the discussion platform and IRC to gauge tone. Review the discussion platform and IRC logs. What was the tone of the communication?


Secondary Criteria - Suitability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment
  1. Project Description
    1. Students must be able to understand the purpose of the project. Does the project clearly describe the product? Can students understand the intended uses of the product? - The About page provides an overview of who, where, and what OpenMRS is, including a downloadable PDF file and a video.
  2. Platform
    1. What software and hardware platform does the FOSS project run on? Development environment can be built on Windows, Linux or Mac OS X completely with FOSS software. (Project development information found here)
    2. Are there resources to support these platforms?
    3. Are students familiar with the platforms?
  3. Development Features - Is the class dependent on specific development features? (Project development information found here)
    1. Programming language - What is the primary language?
    2. Development environment - What environments are supported?
    3. Supporting technologies - What technologies are suggested/required?

Deliverables

POSSE: On your user wiki page, a section describing your evaluation of OpenMRS as a suitable project for your course.

Notes for Instructors

The remaining sections of this document are intended for the instructor. They are not part of the learning activity that would be given to students.

Assessment

  • How will the activity be graded?
  • How will learning will be measured?
  • Include sample assessment questions/rubrics.
Criteria Level 1 (fail) Level 2 (pass) Level 3 (good) Level 4 (exceptional)
The purpose of the project
Why the project is open source

Comments

  • What should the instructor know before using this activity?
  • What are some likely difficulties that an instructor may encounter using this activity?

Variants and Adaptations

POGIL-style combined FOSS Field Trip and Project Evaluation used by Chris Murphy in his FOSS Course, UPenn, Murphy.

Additional Information

ACM BoK
Area & Unit(s)
ACM BoK
Topic(s)
Difficulty
Estimated Time
to Complete

60-90 minutes. This activity can take a significant amount of time. We only expect you to spend 60-90 minutes exploring. You may not complete the activity within this time. Of course you are welcome to spend more time if you wish.

Environment /
Materials
Author(s)

Michele Purcell

Source
License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

CC license.png


Suggestions for Open Source Community

Suggestions for an open source community member who is working in conjunction with the instructor.

Retrieved from "http://foss2serve.org/index.php?title=Evaluate_a_Project_(Activity)&oldid=9475"
Personal tools
Namespaces
Variants
Actions
Events
Learning Resources
HFOSS Projects
Evaluation
Navigation
Toolbox