Revisions Since Design Report 2

Design Report 3

Goals Objectives and Task Analysis

See Design Report 2

Plan for one-to-one formative evaluation

  • Purpose

Improve course materials, particularly the usability of the Teach Web 2.0 Wiki
  • Audience

Primary audience includes K12 teachers and administrators.
  • Issues

    1. After receiving instruction, will the consortium participants be able to effectively enter content into the Teach Web 2.0 wiki?
    2. Will participants be motivated to continue to contribute as new social networking tools are introduced?
    3. Will participants utilize the discussion section of the wiki to voice concerns or add comments?
    4. How much time will be required between face-to-face meetings to effectively contribute content to the Teach Web 2.0 wiki?
    5. Will participants return to the wiki to add content once others have had a chance to update the wiki?
  • Resources

    1. Teach Web 2.0 Wiki populated with example content
    2. Instructor facilitators
    3. Interview Questions for individual and group participants
    4. Computer with access to Internet
  • Evidence

Acceptable evidence of ease of use of the Teach Web 2.0 Wiki will include:
    1. Individual and small focus group success in contributing to all sections of the wiki.
    2. Individual suggestions for future social networking sites or tools to be included in the review process.
    3. Participation in the focus group of teachers and/or administrators from each level of comfort with technology.
    • Level A: Early Web 2.0 adopters
    • Level B: Uses technology regularly in the classroom and/or for professional use
    • Level C: Minimal experience with technology
  • Data-gathering Techniques

    1. Observations of small group navigating the Teach Web 2.0 Wiki
    2. Questionnaire/Interview
  • Analysis

    1. Track number of and type of questions asked by small focus group while navigating the Teach Web 2.0 Wiki.
    2. List comments made by individual and small group participants.
  • Reporting

Results will be reported here, in the Web 2.0 ID wiki. Adjustments to the design will be made as needed based on results.

Results of one-to-one evaluation

The one-to-one evaluation was conducted via email. (Whereas the group evaluation was conducted face-to-face.) The rationale was to determine how new members of the group might get up to speed in the Teach Web 2.0 wiki without necessarily having the benefit of a face-to-face learning session. We identified an early adopter to participate with the understanding that she would have a general comfort with technology and be equipped to troubleshoot any potential issues with the wiki tool. Two major issues were uncovered as part of the evaluation.
  1. We learned that Wikispaces works best with the Firefox browser. Firefox allows for WYSIWYG editing. Safari users must use HTML editing. A portion of our target audience uses the Safari browser. Our plan is to migrate the consortium participants to Firefox.
  2. Our user ran into editing difficulties within the SWOT table. The bullet formatting did not remain consistent as new users edited the content. We removed the table and changed the bullets to numbers. This seems to solve the problem.

See the Formative Evaluation for specific responses to the Wiki Questionnaire.

Materials and assessments for small group evaluation

  1. Teach Web 2.0 Wiki populated with example content
  2. Instructor facilitators
  3. Computers with access to Internet

Instruments for small group evaluation

  1. Observations
  2. Interview / Questionnaire

Procedures for small group evaluation

Instructor facilitators will:
  1. Explain purpose of consortium.
  2. Introduce Teach Web 2.0 Wiki.
  3. Share purpose for participation in small group evaluation.
  4. Show video on how to use wikis.
  5. Allow for general questions about social networking and wikis.

Small Group participants will:
  1. Explore Teach Web 2.0 Wiki independently.
  2. Ask questions as needed during exploration.
  3. Use edit process to add new content to the wiki.
  4. Participate in final interview.

Summary of small group evaluation

The small group evaluation consisted of four faculty members/administrators representing each user level (1 - Level A, 2 - Level B, 1 - Level C). Participants of the evaluation were first introduced to the concept of the consortium via the Teach Web 2.0 Wiki. The goals of the consortium along with the purpose for the evaluation were presented. Next, the video introducing wikis was shown to explain what a wiki is and how it will be used. After the video, questions were asked by the group about how the wiki is maintained, the levels of privileges allowed, and how to follow the changes made to the wiki. Further explanation was given about the various tabs and functions available on a wiki such as the history and discussion tabs along with using the sidebar for navigation. Participants were then invited to join the wiki, review the resources already posted, and asked to make at least one contribution to the SWOT analysis. Instructor facilitators observed participants as they worked, answering any questions along the way. Individual participants were then interviewed to determine the usability of the wiki for the consortium, suggestions for improving the process, and any other general comments or questions about the experience.

Discussion of small group data

During informal interviews with each participant, it was determined that the wiki was very user friendly once the basics had been introduced. It was easy to navigate and edit, and participants could easily see the potential use of this tool in an educational context. Issues did arise when participants attempted to edit pages simultaneously. The following table represents conversation that took place during the evaluation:


General Comments
User 1 (Level A)
clean look,
more user-friendly in Firefox

liked the concept that the consortium would
be democratically led; liked that it invited
User 2 (Level B)
easy to use, a bit tricky when trying to edit
with multiple users, need to save changes

concerns about one person doing all of the
work; need to setup expectations for use
User 3 (Level B)
very easy to use and navigate

can see incredible potential in using this tool
User 4 (Level C)
need more time to explore and get comfortable
with navigation and content; liked the look of it

concerns about editing and losing changes made

Revisions for instruction and assessment

In both the one-on-one interview and small group evaluation two main issues arose that need to be addressed. First, it was discovered that Safari was not the best browser to utilize all of the wiki features. Firefox is the recommended browser to use. At this time, many potential K12 participants use Safari as their main browser. When the consortium begins at the start of the school year, additional Instruction will need to take place to make sure all participants are able to download Firefox on their computers and navigate it easily. Second, it was discovered that while many people can attempt to edit the Teach Web 2.0 Wiki at once, only the person who actually saves their changes first can make the actual edits at that time. All other potential changes are saved as drafts and do not change the wiki. Participants will need to be able to identify when others are editing pages (pop-up windows), understand what will happen to their suggested changes, and learn how to recover drafts of their edits.

Safari Message:

Editing Pop-up Windows:
Picture_2.png draft.png

Relevant current references