Category Archives: Assessment

Consistent Policy in K-12 Online Education

The world of online education is expanding at an incredible rate. Each year, more and more K-12 students are enrolling in online courses to supplement, or at times, replace traditional coursework.  Even though the phrase used is “K-12”, most K-12 online providers offer courses only for secondary students.  By law, however, if a student is eligible to attend school, then online schools must accept them as well.  The expansion of online education requires further study of sound policy, implementation of such policy and the impact online education has on enrolled students.

Please take a moment to review the work of Bonnie Magnuson and Greg Spahn, two students from Bemidji State University who explored the need for consistent policies in K-12 online education as part of their graduate work.  The paper can be viewed here.

2 Comments

Filed under Assessment

Online PBL – The new standard?

Different definitions of PBL in the literature address three basic principles.  Triggering learning, PBL is not an isolated instructional technique but a holistic approach, and finally PBL is student-centered.  These three approaches trigger prior knowledge and have meaningful content, research and organized information.  Schmidt (1993) maintains that PBL is based on information processing theory, in which learning requires learners to participate actively in the process of retrieving, constructing and using information and then relating it to their prior knowledge.  Norman and Schmidt (2000) explain PBL through a constructivist approach where information is acquired and general principles are learnt during problem solving practices to be used for similar future problems.  Hoffman and Richie (1997) consider PBL as a student-centered strategy that has significant contextualized, real-world, ill-structured situations while providing resources, guidance, instruction and opportunities for learners to develop content knowledge and problem solving skills.

PBL activities are designed in a way that is possible for instructors to offer guidance in developing problem solving and critical thinking skills.  Ill-structured problems are similar to those that students will encounter in real life situations.  By providing these situations to students you are sparking their critical thinking skills.  PBL is shown to positively influence learning outcomes as well as the learners higher order thinking skills.  Research has shown that students who learn in PBL settings have higher levels of achievement.

Because the education trends have moved from the traditional classroom setting to using more of a distance learning setting, PBL has to transition as well.  Online learning environments are flexible, attractive and interactive, it is more convenient to implement constructivist and PBL practices through online learning environments.  Online PBL applications should support access to information and collaborative learning and sustain interaction among learners.  It is also suggested though that before an instructor decides to implement the PBL method in their online course, that the course be appropriately structured with logical timeframes or the method becomes cumbersome.  Overall PBL is an excellent instructional methodology and when used correctly is extremely effective and beneficial for the students and their academic success.

Gibson and Gibson (1995) describe an alternative approach in which a learner is engaged with a problem individually and prepares a written analysis of the problem, in preparation for group interaction.  The PBL Instructional Model includes five phases

  • Presentation Phase – where the learner is situated in the problem context and to begin the process of activating relevant prior knowledge.
  • Exploration Phase – provides opportunity for recall and reconfiguration of prior knowledge relevant to the specific problem and exploration of additional, content specific knowledge and experience gained during problem solution.
  • Integration Phase – emphasizes relevant knowledge transfer, analysis, integration, synthesis and evaluation of selected, content specific knowledge and problem based experience.
  • Solution Phase – encourages learners to further integrate knowledge experience and artifacts gathered through the problem solving process into their cognitive structures as through products of real experiences
  • Reflection Phase – learners are encouraged to conduct self-assessment of their artifacts assessing the content and organization of the learning modules according to the particular domain of technology integration

This instructional model is just one of many possible implementations of PBL.  As technologies and methodologies advance there will be future instructional design models that are even more innovative.  In a survey constructed by my group members and I we found out the following information on what students think about PBL and how they have used it in their courses.  The results were as follows –

  • 7 students took the survey.
  • 6 out of 7 online students have utilized problem-based learning in their online courses.
  • All six of those students utilized wikis and blogs for the problem-based learning.  Using email and student-created media were the next most popular tools.
  • 5 out of the 6 students agreed that wikis were the most beneficial.
  • All 7 students agreed that using problem-based learning would aid them in learning concepts in their courses.
  • All 7 students thought they would like this form of learning.
  • 6 of the 7 students thought that problem-based learning prepared them for their career fields.

Overall PBL seems to be an accepted approach to teaching “real life” problem solving skills.

References –

Malopinsky, L., Kirkley, J., Stein, R., & Duffy, T. (2000). An instructional design model for online problem based learning (pbl) environments: The learning to teach with technology studio. National Convention of the Association for Educational Communications and Technology, 2,

Sendag, S., & Odabasi, H. (2009). Effects of an online problem based learning course on content knowledge acquisition and critical thinking skills. Computers & Education, 53, 132-141.

1 Comment

Filed under Assessment

Online Learning and the At-Risk Student

Can online learning be a successful model for at-risk learners?

Early articles regarding successful online learners describe students who were motivated, self-directed, and able to “teach” themselves. These are characteristics not often associated with at-risk students.  Recently, there has been an increase in the use of online programming for at-risk students.  Is this a feasible option or are we setting students up for failure?

As with any environment, a carefully planned online program or course can optimize learning for the at-risk learner.  The online platform offers some tremendous opportunity to address some of challenges at-risk learners face.

Successful online learners report the following as some of the components of their online course that supported learner success:

1.     Flexibility

2.     Social presence

3.     Quality interaction and feedback

At-risk learners report the following as some of the aspects of a learning environment that help them become successful:

1.     Flexibility

2.     Relationships

Research also indicates that a blended learning environment support at-risk learners most effectively, as they have access to assistance as needed.

There are clearly many more factors that need to be considered when working with at-risk learners or in designing an online course than those listed here.  However, if careful consideration is given to the areas where the “characteristics” overlap, a program that will help at-risk students succeed can be developed.

Works Referenced

Hart, C. (2012). Factor associated with student persistence in an online program of study:

A review of the literature. Journal of Interactive Online Learning, 11(1), 19-42.

Watson, S. (2011). Somebody’s Gotta Fight for Them: A Disadvantaged andMarginalized Alternative School’s Learner-Centered Culture of Learning. Urban Education, 46(6), 1496-1525.

www.auditor.leg.state.mn.us/ped/pedrep/alted.pdf

 

http://kpk12.com/reports/

Leave a comment

Filed under Assessment

A Need for Consistent Online Education Policy

While the U.S. Department of Education has outlined recommendations for states, districts and schools in their National Education Technology Plan (2004), Kerry Lynn Rice (2006) states that only a few states have policies in place for K-12 development, and “further, (Watson, et al, 2004) found that in most cases, online learning is little understood by policymakers.”  Policy created for physical schools may need to be adjusted for online programs in order to better serve students.  There is a need not only for research necessary to form consistent policy for distance learning but best practices associated with them. This current lack of available research is a problem for policy making that is compounded by numerous complex variables that make conclusions regarding distance education difficult to make.

In his Phi Delta Kappan article, author Wayne Journell (2012) cautions school districts to consider more than cost savings when considering development of online courses, stating, “Online learning may be cheaper than traditional schooling, but it isn’t necessarily equivalent ot face-to-face instruction, nor is it an appropriate substitute in every case” for teens. Schools should not assume that a successful classroom or technology teacher will be able to transfer skills to teach effectively online. “Online structure requires a different skill set and dispositions” (Garrison & Anderson, 2003; Journell, 2008; Quinlan, 2011).

Amy Murin, interviewed by Sheila Regan (2012) of the Twin Cities Daily Planet, is a researcher for Evergreen Ed Group and reviews online policy and practice on a yearly basis.

According to Murin, Minnesota is thoughtful in its approach to offering programs online, districts are collaborating and a legislative task force is “promoting a statewide discussion.”  A 2011 state audit, however, showed that students taking online courses were less likely to finish the classes they started, full time online students were more likely to drop out and math scores on the MCA II were lower than for students in traditional school settings.

Several strategies are offered for the improvement of online classes.  Teachers must get used to using technology in different formats.  Instant messaging and virtual meetings to connect with students who use emoticons to help express their understanding help bridge the lack of face to face interaction. Building a constructivist community of trust, facilitating discussion and giving timely, caring student feedback are essential elements for student success in online learning.

A Time (2012) article suggests that teachers need to be trained to adjust to communicating without the advantage of communicating in real time.  Some schools of education and online schools have begun to teach how to instruct online.  Online teachers know that students need to be motivated to self-reflect and become independent.  Teachers also need to connect with other online teachers in order to learn from one another.

This is another article that emphasizes the crucial importance of appropriate training for the teacher and communication between teacher and student.  The courses must be planned in great detail to clarify expectations for learning.  Again, student motivation is a clear factor in whether students will be successful online. They will not succeed if they do not even “come to class,” which indicates that online classes are not a viable option for some students.  More research is needed to find out what constitutes a good candidate for online learning.

Due to scarcity of research in the K-12 distance learning realm, it is tempting to generalize research of adult learners.  It is vitally important that researchers begin the gargantuan task of collecting and analyzing data from K-12 sources not only in order to set consistent policy for improving outcomes but to screen prospective students to determine whether they are good candidates for online learning.  We are obligated to use technology advances to benefit every student in both the cognitive and social formation of knowledge to prepare them to evolve with the changes of an unknown future world.

References

Journell, W.(2012). Walk,, don’t run. Phi Delta Kappan 93(7), 46-50.  Retrieved from http://web.ebscohost.com.bsuproxy.mnpals.net/ehost/pdfviewer/pdfviewer?vid=10&hid=7&sid=55c881a9-48e7-4250-8941-2f884136090d%40sessionmgr10

Pandolfo, N. (2012, June 13). The teacher you never met: inside an online high school class. Time. Retrieved July 5, 2012 from http://www.time.com/time/printout/0,8816,2117085,00.html1#

Regan, S. (2012, April 16). Kids online: best practices for teaching and learning. Twin Cities Daily Planet.  Retrieved July 5, 2012 from http://www.tcdailyplanet.net/news/2012/04/16/kids-online-best-practices-teaching-and-learning?print=1

Rice, K.L. (2006). A comprehensive look at distance education in the k-12 context. Journal of Research on Technology in Education, 38(4), 425-448.  Retrieved from http://www.iste.org/Content/NavigationMenu/Publications/JRTE/Issues/Volume_38/Number_4_Summer_2006_A_Comprehensive_Look_at_Distance_Education_in_the_K-12_Context.htm

Leave a comment

Filed under Assessment

Assessing Student Readiness for the Online Learning Environment

Recent studies have suggested that generally online students are more dissatisfied with online learning compared to traditional classroom satisfaction levels. Student dissatisfaction not only affects the student’s success, it also affects attrition rates and institutional funding so it is important to define the qualities that are necessary for student success in an online learning environment.

Students who are ready to successfully interact in an online environment have excellent time management skills, are personally motivated to succeed and have incorporated their respective learning style into the online learning experience. In addition, “qualities that may explain individual differences in academic achievement, completion rates and levels of satisfaction with online learning, fall into the categories of technical skills, computer self-efficacy, learning preferences and attitudes towards computers” (Hitendra Pillay)

So, how do we assess a student’s readiness for online learning based on these four principles: technical skills, computer self-efficacy, learning preferences and attitudes towards computers? Pillay, Irving and Tone suggest “The Tertiary students’ readiness for online learning survey (TSROL)” which is an attempt to consolidate several diagnostic tools into a single, more inclusive mechanism of assessing the student’s alacrity to engage in online classes.

Traditional online student preparation ranges from testing to a certain proficiency skill level to allowing any student who has Internet access to register for an online class. In the paper “Coming and Going in all Directions: Preparing Students for Online Learning”, Lujean Baab suggests that we adapt a sort of “treasure hunt”, in which the instructor sets “a list of required tasks that must be completed by a date prior to the start of the class” (Baab) These tasks can include interacting with a listserv item, sending emails with an attachment, participate in a chat room session, answer technical specification questions about their computer using an online form, or have the student simply organize their email messages by creating folders and sending the screenshot as an attachment in an email. Realistically, if the student can perform these simple technical tasks, they should be computer literate enough to successfully navigate through an online class.

According to Martinez, Torres and Giesel, while many educational institutions recognize the need to assess technical skills as a prerequisite to participating in online classes, “the need at this point in the development and expansion of online instruction is for identification of best practices in determining student readiness and in facilitating student success in an online environment. “ (Martinez, Torres and Giesel) There is a plethora of Best Practice information on the Online Student Support’s “Determining Student Readiness” web site.

The University of Minnesota’s Technology Enhanced Learning at the University of Minnesota blog hosts several interesting articles about assessing student readiness and enhancing online curriculum. The blog “More dropout in online classes: What should we do?” which refers to the results of the Online and Hybrid Course Enrollment and Performance in Washington State Community and Technical Colleges study that tracked enrollment patterns and academic outcomes in online, hybrid, and face-to-face courses for nearly five years.

Three main insights came from that study according to Hwang:

• Online students carry more pressure from outside obligations
• High drop-out rate is a results of poor online curriculum design
• Increased amount of self-discipline is required for an online student

Three action items are needed to reduce online class drop out rates:

  • Faculty Training
  • Assess Student Readiness
  • Additional Student Support

In addition, a few Minnesota colleges are using a tool called SmarterMeasure to assess student’s readiness for the online learning environment with great success. The SmarterMeasure web site has a Research page that shares results of research results, assessment details, item reliability, usage patterns and case studies. The 2011 Student Readiness Report showed three-year trending analysis to help online programs understand what makes a successful online learner. The white paper released with Noel-Levtiz showed a direct correlation between measured areas of SmarterMeasure and student retention. (SmarterMeasure)

No matter the tool(s) we use, it is imperative to provide a means for students to assess whether or not they are prepared to participate in an online class and succeed. “The dynamic nature of the online environment precludes any final word on the subject. The only definite word at this point is that institutions must continue to assess student readiness for learning in the online environment in order to develop appropriate and timely strategies to promote student success.” (Martinez, Torres and Giesel)

Works Cited
Baab, Lujean. “Coming and Going in all Directions: Preparing Students for Online Learning.” 00 04 1999. Teaching in the Community Colleges Online Conference. 02 07 2012 <http://leahi.kcc.hawaii.edu/org/tcon99/papers/baab.html&gt;.
Hitendra Pillay, Kym Irving and Megan Tones. “Validation of the diagnostic tool for assessing Tertiary students’ readiness for online learning.” 00 06 2007. Higher Education Research & Development Vol. 26, No. 2. 03 07 2012 <http://www.informaworld.com/openurl?genre=article&id=doi:10.1080/07294360701310821&gt;.
Hwang, Seogjoo. More dropout in online classes: What should we do? 08 09 2011. 03 07 2012 <http://blog.lib.umn.edu/tel/blog/2011/09/more-dropout-in-online-classes.html&gt;.
Martinez, Helen Torres and Giesel. Determining Student Readiness for Online Instruction. 00 00 2006. 03 07 2012 <http://www.onlinestudentsupport.org/Monograph/readiness.php&gt;.
SmarterMeasure. SmarterMeasure Research. <http://smartermeasure.com/research/&gt;.

1 Comment

Filed under Assessment

Increasing Success of Online Students: Evaluating Student Readiness

A major movement in education in the United States is the growth of online learning.  In the most recent Sloan Consortium’s report, the number of students taking online courses in 2008 grew by 17% when compared to the previous year, while traditional face-to-face course enrollment only increased by 1.2% (Allen, 2010).  As enrollment in online courses continues to expand, the issue of student retention takes on significance.  Efforts to improve success of college students enrolled in online classes either through improved Grade Point Averages or by program/degree completion are becoming widespread. 

Student success in online course has been linked to three factors: 
1. Student readiness
2. Student orientation
3. Student support (Harrell, 2008) 

This report focuses on some of the strategies institutions are utilizing to predict student readiness for and success in the online environment.

A large body of research involves the development of assessment surveys.  These surveys may be administered by the institution or may be self-assessments. One such survey, the LASSI for Online Learning (LLO) was investigated by Carson (2011).  His findings indicated that the LASSI Motivation subscale provided the most efficient predictor of student success with time management being the second most important predictor.  His report is noteworthy as the sample size was quite large, including 8112 students. 

On a smaller scale, Robyler and Marshall (2002) developed an Educational Success Prediction Instrument (ESPRI) for use with high school students.  After completing a literature review and compiling a list of qualities of successful online students, the 70-item instrument was designed.  This instrument predicted student success, defined as a grade of “C” or better, with 100% confidence. 

The authors identified three factors as being the strongest predictors of student success: 
1. Study environment
2. Motivation
3. Computer confidence 

They conclude with the recommendation that this instrument be used in a larger study to determine if it is an effective predictor in the widespread population.

A more targeted study was completed by Fisher and Tague (2001).  These authors undertook to develop a readiness scale for self-directed learning for nursing students.  The final scale included 40 items that reflected three components: self-management, desire for learning (motivation) and self-control. 

Self-assessments are also widely used.  When investigating the institutions within the Minnesota Colleges and Universities system (MnSCU), it was found that 15 of 30 two-year community and technical colleges offered students the opportunity to self-assess their readiness for online learning while 4 of 7 four-year institutions linked students to a similar survey (compiled by Sprangers, 2012)

Another strategy used by some institutions is the development of a course specifically designed to augment student success in the online environment.  One course, “Online Student Success (OSS)” is offered by Consomes River College in Sacramento, California.  After implementation of this course, it was found that 78% of students achieved a grade of “C” or better in their online courses if they had been successful in OSS.  This compares to only 38% of students achieving a passing grade in an online course prior to taking OSS (Beyrer, 2010).

An important finding of this review is the key role of motivation in online student success.  Yet, these investigations lead to additional questions.  For example, is it possible to develop a single, standarized tool that would accurately predict student success across a wide variety of disciplines?  Are population demographics important in assessing the validity of these tools?  How well do these tools correlate with the “success” characteristics that have been identified in the literature?  With the rapid rise in enrollment for online courses, it is clear that there is much work to be done if we, as educators, wish to promote student success in online classrooms.

References:

Allen, I. S. (2010).  Learning on Demand: Online Education in the United States, 2009. Retrieved July 5, 2012 from

http://sloanconsortium.org/publications/survey/learning_on_demand_sr2010

 Beyrer, G. (2010).  Online Student Success: Making a Difference.  Journal of Online Teaching and Learning, 6(1), 89-108.

 Carson, A. D. (2011). Predicting Student Success from the LASSI for Learning Online (LLO). Journal Of Educational Computing Research, 45(4), 399-414.

 Fisher, M., King, J., and Tague, G. (2001).  Development of a self-directed learning readiness scale for nursing education. Nurse Education Today, 21, 516-525.

Harrell, I.L. (2008). Increasing the Success of Online Students.  Retrieved July 5, 2012 from Virginia Community College System: http://www.vccaedu.org/inquiry/inquiry-spring-2008/1-13-Harrell.html

Roblyer, M. D., & Marshall, J. C. (2002). Predicting Success of Virtual High School Students: Preliminary Results from an Educational Success Prediction Instrument. Journal Of Research On Technology In Education, 35(2), 241.

Information on MnSCU institutions compiled by J. Sprangers (2012)

Leave a comment

Filed under Assessment

Self-Regulated Leaning Skills for Students Online: What are they and how are they measured?

Online learning asks more of students than traditional classroom settings. The classroom brings students and instructors together in a common room and time enabling direct communication and direction from the instructor. The students can be passive in that once they are in class, the information is delivered. Online students must be proactive in bringing themselves to the lessons and recognizing that they are entirely responsible for their success.

Hung et al. (2010) described five personal skills of successful online students. Self-directed learning is a measure of a student’s ability to establish personal goals, identify the necessary resources, schedule their work, and in general, take personal responsibility. Motivation for learning includes intrinsic characteristics like personal interest or curiosity, and extrinsic motivations to receive rewards. Learner control refers to the inherent flexibility of asynchronous online learning where students can choose specific activities, the time spent on activities, or the activity sequence. Computer and Internet self‑efficacy measures the ability to use the Internet to accomplish tasks rather than just technical ability. Online communication self‑efficacy is the ability to effectively use the technology to participate in intellectual discourse with classmates and instructors.

Yang and Park (2011) described a study in which the instructional design embedded self‑directed learning skills and communication self‑efficacy skills into an online class. They reported that based on post course surveys, students’ ability in self‑directed learning skills improved but not so for communication self‑efficacy skills. They postulate that these students may not value peer interactions as an important aspect of individual performance.

Higher education institutions try to alert students to the demands of online learning through documents or websites describing the skills of online learners, and by surveys that ask students directly or indirectly about these skills. Both of these methods are common but have limited success. Poorly designed surveys may not ask the right questions of fully impress upon students the importance for self‑regulated learning skills (Hall, 2008). Indiana University offers students white paper describing personal, academic, and technical skill needed to succeed online but again (Alford & Lawson, 2009) but these need to be read and fully understood by students to make a difference.

Given that many students will enter online classes unprepared, researchers attempt to understand student learning in courses through surveys and interviews. Usually these surveys are given before or after the class, and therefore miss the details of student progress through specific lessons (Winne, 2012). A properly designed computer based learning environment can trace student activity and infer the decision process through the lesson. In this way, instructors can assess both what the student learned, and how the student learned it. This can lead to better instructional design tools that help students adopt the skills of self‑regulated learners.

 

Works Cited

Alford, P, & Lawson, A. (2009). Distance Education Student Primer: Skills for Being a Successful Online Learner. Indiana University. Retrieved June 30, 2012 from http://ittraining.iu.edu/free/DESPR.pdf

Michael Hall. (2008). Predicting Student Performance in Web-Based Distance Education Courses Based on Survey Instruments Measuring Personality Traits and Technical Skills. Online Journal of Distance Learning Administration, 11(3). Retrieved from http://www.westga.edu/%7Edistance/ojdla/fall113/hall113.html

Hung, M.-L., Chou, C., Chen, C.-H., & Own, Z.-Y. (2010). Learner readiness for online learning: Scale development and student perceptions. Computers & Education, 55(3), 1080–1090. doi:10.1016/j.compedu.2010.05.004

Winne, P. H. (2010). Improving Measurements of Self-Regulated Learning. Educational Psychologist, 45(4), 267–276. doi:Article

Yang, Y.-C., & Park, E. (2011). Applying Strategies of Self-Regulation and Self-Efficacy to the Design and Evaluation of Online Learning Programs. Journal of Educational Technology Systems, 40(3), 323–335. doi:Article

Leave a comment

Filed under Assessment