SLENZ PROJECT EVALUATION, VLENZ 165, Mar 10, 2010

SLENZ PROJECT EVALUATION RELEASED

SL Research project ‘successful’ but

marred by technical difficulties  …

Full PDF of Evaluation Report available here

Education conference delegates view a Birthing Room

The SLENZ team achieved “a remarkable success in creating and delivering” the core of the SLENZ Project, according to the official evaluation report of the $NZ500,000, Second Life  research programme released today.

This was despite the fact that both the project and student/educator learning  and engagement  had, at times,  been hampered  by  technical  challenges and difficulties during the 18-month long project, said Michael Winter, the independent evaluator of the project. He is  a veteran educator and senior researcher with  CORE Education, of Christchurch, New Zealand.

At the core of the project were two builds – a Foundation Learning Centre and a Birthing Centre – which were used for virtual world teaching as part of the project to determine the benefits or otherwise of virtual world education, and how these benefits, if any, could best be harnessed.

“The creation and effective employment of the two builds is a great success of the team,” Winter said. “In the process, the project team established a valuable corpus of experience in developing and using virtual world resources for tertiary education.”

The pilot programmes, Winter said, had both increased the engagement of learners with a familiar game like environment where learning may intentionally be a product of serious play; had been successful in creating experiential learning situations not available in ‘real life’;  had had only limited success with providing the  opportunity to learn the skills necessary to operate socially, technically, and ethically in an online global virtual world;   but had  provided an opportunity to “experience and practice collaborative, cross-cultural problem solving in social networking environments.

Although the report   appears a little short on hard facts, figures and comparative measurements it contains much  interesting anecdotal evidence from both students and educators and the conclusions reached  by Winter should be helpful to all  contemplating  or launching a new educational programme  within a Multi User Virtual Environment (MUVE) such as Second Life.

Michael Winter

Winter said  that  the “success of the project”, which incorporated Foundation (Bridging) Learning and Midwifery Education pilot programmes, had  largely been due to the project management skills of the real life Project Manager (Terry Neal, of BlendedSolutions) who  had been responsible for managing the budget, interpersonal communications, and ensuring that timelines were adhered to and deadlines met.

Winter  also singled out the  Project Developer (Aaron Griffths, of Fxual  Education Services)  for special mention.  “As far as the design and development of the builds are concerned, the Project Developer almost single-handedly achieved a remarkable success with both builds,” Winter said. “He was able to incorporate the differing needs of both the Foundation and Midwifery educators, and in many cases produced aesthetically pleasing and functional designs. The ongoing process of development generally met the required deadlines, although sometimes at some cost in terms of stress.”

Terry Neal

Winter  made a  number of recommendations for future projects exploring the use of virtual worlds in education, which should be useful to MUVE developers and researchers around the world.  These included:

  • The Project Team, and Lead Educators at each institution should become familiar with the IT processes, and strike up an ongoing working relationship with members of the IT team. This could be greatly facilitated by the identification of a senior manager to act as a champion of the project within the institution.
  • Clear identification of the hardware and network needs required effectively to run the virtual world on the multiple computers within a particular institution. This would include addressing network security and fire walling issues.
  • Clear communication with students who will be using the virtual environment off-site regarding hardware and software requirements to access and make use of all the features of the virtual world.
  • An indication to students of the broadband requirements and hardware specifications to run the virtual world effectively on their own home computers, including connection speed, and the likely impact on broadband usage.

    Aaron Griffiths

  • Give more attention and time to ensure that users become fully familiar with using Second Life. This could involve a more thorough orientation process including working with buddies experienced in Second Life.
  • Clear indication to students of expectations in terms of their participation and learning outcomes when using the builds.
  • Ensure that each stage of the build actively involves and engages learners, and avoids them spending time “just looking”.

Summaries

Foundation Learning pilot

In his  summary of his findings on the Manukau Institute of  Technology Foundation Studies pilot programme, led by lead educator,  Merle Lemon,   Winter noted that  the Foundation build had provided a rich environment for learners to develop their job-hunting skills, despite criticism of its external appearance.

“It provided the opportunity for students to review material they had learned in face-to-face sessions, and to practice dressing appropriately for, and taking part in, interviews,” he said, but “for many participants, the experience was marred by technical difficulties, which highlighted the need for careful planning and good collaboration with the IT department before introducing virtual world learning into on-site programmes.”

The  MIT staff, Winter said,  were generally keen to be involved in future work with virtual worlds, but stressed the need to resolve technical issues.

Midwifery Education pilot

In his summary of the Otago Polytechnic Midwifery pilot programme, led by lead educator Sarah Stewart,  Winter said the two completed stages of the Midwifery build had represented “a significant success of the project”.

He, however,  again noted technical problems involving both hardware and connections to the internet  –  as well as navigation difficulties within Second Life by both  students and tutors -as hampering the results in this distance education part of the SLENZ Project.

On the question of user navigation,  he said,  “It is likely that a longer and more careful period of orientation might help participants overcome these difficulties.”

But he said, “Those students that accessed the build, and who were confident with the environment, reported a high degree of engagement and enjoyment of the experience, especially in working through the scenario with a buddy. They found this experience removed some of the stress, compared with face-to-face role-play. “

Visitors look over the Foundation learning build

Advertisements

SLENZ Update, No 148, November 4, 2009

SLENZ PROJECT – final F2F

SLENZERs celebrate ‘completion’

of virtual world  work …

now  team awaits official evaluation

IMG_1116The SLENZ Project team … final face-to-face meeting and debriefing.

The SLENZ Project team celebrated its successes last week at a real life face-to-face meeting in Wellington, New Zealand.

The meeting, which  included a warts-and-all debriefing of all team members, was marked by an unanimity of views on project outcomes in a team which  has occasionally been rift by  differences of nuance and interpretation over the  16 months of its scheduled 18-month life span.

The NZ$500,000 Second Life/Real Life project, which was funded by the Tertiary Education Commission of New Zealand, has been designed  to determine  whether and how multi-user virtual environments (MUVEs) can benefit New Zealand education and, and if they are of benefit,  how the benefits can best be harvested.

Despite the fact the  formal evaluation  has not been completed  team members appeared in no doubt that most, if not all, of the  objectives of the three pilot programmes – Midwifery, Foundation (Bridging) Learning and Orientation – had been met.IMG_1127

The lead evaluator, Michael Winter (pictured right), of CORE Education,  who attended the meeting – although not pre-empting his formal evaluation, due before year end  –  seemed upbeat about  the project and said he  had been impressed with communication skills displayed by the team.  .

“I was really impressed with the level of communication and  the way people were working together,” he said. “It was a pretty tight ship in terms  of communication.

He added, however, that the project might have been somewhat hindered by a number of technical issues, including  bandwidth (Ed note: Possibly perculiar to New Zealand); institutional technology and firewall issues. He  added that there had been some resistance to what was perceived as “gaming” by some students;  and that there was a necessity for designing the e-learning experiences properly to  increase engagement. He also cautioned about an underlying concern about the “sleezier side of Second Life” which  the press has focused on.

Summing up her feelings about the project, joint project leader Dr Clare Atkins said, she was “incredibly proud of what we have done.

IMG_1121“I’ve learned some amazing lessons how not to do many things,” she said to laughter.

Despite the barriers to adoption of MUVEs for education in New Zealand,  Atkins said, she now  “absolutely believed”  that “the use of these types of environments and kinds of education are  going to change  the way everyone teaches, how they teach  and the way we think about teaching within 20 years.

“I believe its really important  not only to look for the next project,” she added, ” but also to offer everything (that we have learned) we can to others in education.”

In the debriefing team members  agreed the staged approach to the SLENZ Project had been one of the major keys to the success of the project.

“In fact,” Atkins said, ” I would recommend next time that we should go for even shorter stages – each with its own discrete documentation. For example we could perhaps have broken Midwifery Stage 1 down further into a) the build of the Birth Unit b) the ‘fitting out’ of the birth unit with information.”

Other things that  had worked well had included  the regular team meetings with voice in  Second Life and the face-to-face meetings for getting acquainted and determining agendas for further Project Stages.

Barriers or obstacles to development of the pilot programmes chosen for implementation, included, according to a list compiled from the discussions by  joint project leader, Terry Neal (pictured lower left):

  • Communication: Not having a one-stop shop for all documents from the start of the project. This was implemented when problems arose  after the project  had been launched.
  • Immersion: A lack of pre-project immersion by some tutors, team members. It was felt by some team members that for education to succeed in virtual worlds it is essential that promoters/champions/teachers and tutors be “immersed” in virtual worlds rather than just being “active” before launching into  educating students. This was coupled with a the lack of educator release time for immersion in world.
  • Learning Designer: The need for a Learning Designer or Educator  to be fully  “immersed” so that he/she could specify exactly what was needed based on their own knowledge.
  • Roleplaying Experience:  At launch a lack of MUVE roleplaying experience on the part of tutors, preventing them from having a complete understanding of what could and could not be done in a virtual environment.
  • Clarity: More clarity was needed around the setting of pilot  objectives/initial learning design specifications and the expected/required outcomes.

Things that were seen as an aid to project development included:

  • The use of “immersed” mentors/helpers for new tutors and students.
  • The employment of a professional MUVE builder/scripter rather than attempting to get teachers/tutors up to speed in this area. It was observed that teaching  should be left to teachers/facilitators, and building and facility development to MUVE building/scripting professionals.

Summing up the consensus feeling and her feelings at the debriefing, Neal said, she  thought the team could have done a lot worse, but it could have done a better job too.
Team members were all given a Taonga ( treasure) at the end of the session.