Use of multi-modal media and tools in an online information literacy course: College students' attitudes and perceptions

This project studies the use of multi-modal media objects in an online information literacy class. 162 undergraduate students answered seven surveys. Significant relationships are found among computer skills, teaching materials, communication tools and learning experience. Multi-modal media objects and communication tools are needed to strengthen course interactions and student engagement.


INTRODUCTION
With the development of information and communication technologies (ICT), networked learning has become popular at higher education institutions for reasons including institutional advances, student enrollment, and instructional demands. Studies have shown that the use of ICTs by instructors and students is increasing both in and out of classroom. 1,2,3,4,5,6,7 The increasing usage is a result of university investments in campus information infrastructure and technological implementation as well as studies of pedagogy; however, in the meantime, the increase also demonstrates continuing demands on campus. 8,9,10 These demands are accompanied by high costs. Therefore, university administrators must determine whether investment in ICTs has improved the quality of teaching and learning. The goal of this project is to determine student preferences over multi-modal media and tools for online interaction in web-based classes and to investigate how these 4 preferences inform navigation and performance within such courses. The investigators are interested in learning what impact students feel media variety and interaction type have on how they work within the web-based environment and how their expectations and preferences in such an environment relate to preferences for other online activities.

Effectiveness perceived by students
Evaluation of networked learning often focuses on attracting new students, generating new revenues, providing students flexible and convenient educational opportunities. Some researchers have identified disadvantages in networked learning such as low self-motivation and discipline, minimal interaction with instructors and peers, and lack of a learning framework. 11,12 However, Hara and Kling 13 point out that most studies fail to address students" difficulties, and the quality and effectiveness of online distance education courses. Due to the rapid development of ICTs and their applications to online education, it is important to re-examine those issues and see whether the findings from Hara and Kling"s study are still applicable. Bouhnik and Marcus 14 present a model promoting students" interactions with course content, instructors, and systems.
Many public universities are required by state legislators or the U.S. Congress to justify their budgets and accountability. 15 Effectiveness is one aspect of accountability measurement of education, as universities invest enormous amount of money on technologies for instruction. 16,17,18 As the pedagogical focus moves from teacher-5 centered to student-centered, instructional effectiveness should include students" feedback on the use of technology. 19,20,21 Whether or not students perceive the same value in approaches to online instruction as their instructors is an area that requires further study. 22

Multi-modal learning objects
Multi-modal learning objects in this study are identified in both visual and auditory modes. These objects are text, graphic, audio, video, and instant messaging. The instructors and students use these objects to communicate with each other. The use of ICTs can strongly influence the presentation and organization of course content. 23 Additionally, it can have great impact on in-class communication and interaction among students and between instructors and students in both synchronous and asynchronous forms.
Instructional technologists have promoted the use of multimedia in classrooms, believing that multimedia enriches the learning process and that students can perform better with visual images and words than just words alone. 24, 25 However, some learning scientists doubt the effectiveness of graphical presentation on learning opportunities. 26,27 Mixed results in students feedback indicate that multi-modal learning objects may have no influence on magnitude of students" learning judgment 28 ; and some students still prefer face-to-face lectures which can be more animated than the Web format. 29 These different findings intrigue the investigators to study the effectiveness from a student perspective. 6 Therefore, determining the appropriate multi-modal learning objects for synchronous and asynchronous instructional settings is an important topic for course content development and student-centered learning.

Students' attitudes
Student attitudes toward instructional media are related to motivation and learning outcomes. 30 Sims 31 advocates the importance of aligning student perceptions and expectations regarding interactive multimedia in the networked learning environment.
According to his study, sixty-eight Australian undergraduate students considered that effective interactivity should consist of engagement, control, communication, design, the individual, and learning. Bruce, Dowd, Eastburn, and D"Arcy 32 also find similar responses from college students in an online agricultural Web site over a six-year period.
Regarding resistance, Thompson and Lynch 33 discover that people with weaker Internet self-efficacy beliefs would be inclined to resist Web-based instruction. Therefore, students" attitudes and expectations are essential factors to the success of networked learning environment.

Studied online course: INF 312 Information in Cyberspace
The course examined in this study, Information In Cyberspace, is an online course with an enrollment over 150 students at the University of Texas. It has been evolving since 1998; it began as a face-to-face classroom course, but due to space constraints and student demand, it has evolved into a course that it taught completely online. The content of the elective course covers the basics of technology and information literacy, and is taught by students and staff of the UT School of Information. In this course, students learn new skills for research and communication online, consider the history and future of the networked society, and regularly engage with new technologies. The course emphasizes a hands-on, critical approach to finding, using, and sharing information on In order to expose students to the variety of synchronous collaborative technologies available, the instructors alternate between the tools they use to present the group chat session. These tools include Blackboard"s "Office Hours" (a text-only group chatroom), the more robust Blackboard "Virtual Classroom" (which includes a virtual 9 whiteboard and other tools), and the group chat feature of Skype, a popular voice-over-IP client.
The instructors of the course, who regularly share their teaching experiences with one another, have found that communicating with and maintaining students" awareness of others can be a challenge for such a course. In order to meet the challenge and to address the varying levels of experience with technology present among students, the instructors chose to offer students a variety of communication options to ensure that students remain informed and feel their voices will be heard. 34 The instructors have also incorporated enhancements to course materials based on response from students, and seek to include a wide spectrum of technologies for content delivery. Based on the instructors" informal interaction with students, such enhancements have contributed to student excitement about the course, and also have helped to identify some areas in which student attitudes indicate the limitations of some instructional technologies. However, a systematic student-oriented instructional evaluation of the class is needed to ensure the quality of the class. In creating the course content and delivery strategies, the instructors need to understand students" perceptions of the effectiveness of different approaches (collected through discussion and surveys) and strive to create an instructional environment in which students have multiple paths and multi-modal arrangements for engaging with the instructional modules and among themselves and with their instructors.

PROJECT OVERVIEW
This project is an exploratory study on the use of multi-modal media and tools for an online information literacy course. The goal of this project is to establish a framework for developing, designing, and evaluating the course. The investigators plan to report findings in three parts. This paper is the first part of the project focusing on identifying meaningful variables which may have impact on students" online learning experiences based on students" feedback and self-evaluation at three different learning stages (before, during, and upon completion of the course). The second part will cover students" learning experiences from the beginning of the course to the end. Based on the variables and connections among variables, the investigators will discuss design and evaluation principles for the class and implications for online education as a whole in the third part ( Figure 5).

RESEARCH QUESTIONS
The investigators consider that identifying meaningful variables is the first step for evaluating quality online courses. Few studies focus on identifying such variables, particularly, at different learning stages. Therefore, the investigators used seven online surveys to collect data from students, one at the beginning of the semester, five during the semester (one for each of the different course modules), and one at the end of the semester. The investigators proposed the following three research questions:  What are the relationships among participants" demographic characteristics, computer skills and usage, and their expectations about the online class?
 What are the relationships between the media employed in each course module and participants" learning experiences and satisfaction?
 What are participants" perceptions of the overall learning experiences and satisfaction levels in this online class?

RESEARCH METHODS
Data collection included seven different surveys corresponding to course content.
Online surveys were conducted at the beginning of the course, immediately after each of the five webcast sessions, and at the end of the course. The investigators" survey items were integrated within regular surveys designed by instructors to elicit student feedback on the design and content of the course ( Table 1). The investigators did not have access to the survey data until after the class concluded and final grades for the semester were submitted.
As part of the course orientation, students were required to complete the incoming student survey, which was presented on a web page they accessed in completing initial course requirements. The real-time webcast session conducted during the second week of each of the five core instructional modules served as the setting for the five interstitial surveys. Toward the end of each of these webcast sessions, a hyperlink to an online survey form was provided to students in the online chat session. As discussion was winding down in each webcast session, students were given time to complete the each of the surveys, response submission was closed one hour after the webcast ended. Webcast 12 sessions were not mandatory, but most students did, however, attend more than one webcast session. A link to the web-based exit survey was shared with students upon their completion of the course"s final examination. The data were analyzed with two multiple regression analyses. The first multiple regression analysis used as predictors: students" academic status (freshman, sophomore, junior with senior as a reference category), gender (female with male as a reference category), computer skills, frequency of computer use, tendency to procrastinate, and frequency of instant messaging use, and the dependent variable was the students" rating of their expectation for the online course. In this analysis, only students" computer skills and frequency of instant messaging use reached a significant level. Therefore, for the second multiple regression analysis, the investigators used the two significant predictors (students" computer skills and frequency of instant messaging use) and students" rating of their expectations for the course as the dependent variable. The regression is (R 2 = 0.097) and the overall relationship was significant (F 2, 159 = 8.50, p=0.000). Students" expectation scores are positively related to their computer skills (t=2.76, p=0.006, Beta=0.33), and to the frequency of their instant messaging use (t=2.05, p=0.043, Beta=0.16). Table 4 presents the mean and SD for each variable from 5 course modules. The survey was conducted immediately after the webcast session of each course module. <Table 4. Participants" feedback on 5 course modules" webcast sessions (mean±SD)> The investigators used multiple regression analyses to analyze the following variables. The procedure described above for the first research question was also used for the second research question. Students" learning experiences and satisfaction were used as the dependent variables, and the predictors were: audio quality, video quality, particular tools used for webcast, ability to follow the webcast program, class engagement, and comparison with a physical class.

Course module 1
As for students" learning experiences, the regression was (R 2 = 0.491) and the overall relationship was significant (F 3,100 = 32.21, p=0.000). Students" ratings of the experience were positively related to the audio quality they reported (t=3.74, p=0.000, Beta=0.33), to Blackboard"s performance as a synchronous classroom environment during the session (t=2.69, p=0.008, Beta=0.20), and to comparison with a physical class (t=4.98, p=0.000, Beta=0.45).

15
Course module 2 For the second course module, the regression for student reports that the webcast session was a worthwhile learning experience was (R 2 = 0.289) and the overall relationship was significant (F 2,78 = 15.87, p=0.000). Students" experience scores were positively related to their reports of class engagement (t=2.98, p=0.004, Beta=0.31) as well as their ability to follow the webcast program (t=3.30, p=0.001, Beta=0.34).
In terms of overall satisfaction with the webcast session, the regression was (R 2 =0.254) and the relationship was significant (F 2,82 = 13.98, p=0.000). Students" satisfaction scores were positively related to the level of audio quality reported (t=2.57, p=0.012, Beta=0.25), and to their ability to follow the webcast program (t=3.77, p=0.000, Beta=0.37).

Course module 3
For student scores of the overall learning experience in the third course module, the regression was (R 2 = 0.236) and the relationship was significant (F 2,84 = 12.99, p=0.000). Again, students" experience scores were positively related to the level audio quality they reported (t=2.61, p=0.011, Beta=0.25), and their ability to follow the webcast program (t=3.81, p=0.000, Beta=0.37).
In terms of students" overall satisfaction with the webcast session for this module, the regression was (R 2 = 0.3) and the overall relationship was significant (F 2,84 = 18.02, p=0.000). Students" satisfaction scores were positively related to audio quality (t=4.11, p=0.000, Beta=0.39), and to their comparison of the potential of webcasting versus a traditional large face-to-face class (t=2.93, p=0.004, Beta=0.28).

Course module 4
The regression for students" learning experience scores for module four"s webcast was (R 2 = 0.396) and the overall relationship was significant (F 2,60 = 19.67, p=0.000).
Once again, students" experience scores were positively related to the level audio quality they reported experiencing (t=3.50, p=0.001, Beta=0.37), and to their ability to follow the webcast program (t=3.87, p=0.000, Beta=0.41).
Regarding overall satisfaction with module four"s webcast session, the regression was (R 2 = 0.438) and the overall relationship was significant (F 2,58 = 22.64, p=0.000).
Students" satisfaction scores were positively related to their reported levels of class engagement (t=5.37, p=0.000, Beta=0.53), and their ability to follow the webcast program (t=3.70, p=0.000, Beta=0.37).

Course module 5
The regression for students" learning experience scores in the fifth module was (R 2 = 0.338) with an overall relationship that was significant (F 2,74 = 18.85, p=0.000).
Students" experience scores were positively related to their comparison of the webcast session with a traditional large course a physical class (t=3.41, p=0.001, Beta=0.37), as well as to class engagement (t=2.71, p=0.008, Beta=0.30).
In terms of students" overall satisfaction with the webcast for the module, the regression was (R 2 = 0.421) and the overall relationship was significant (F 3, 76 = 18.41, p=0.000). Students" satisfaction scores were positively related to video quality (t=2.88, p=0.005, Beta=0.29), to Skype"s performance as a synchronous classroom environment during the session (t=2.98, p=0.004, Beta=0.26), and to their ability to follow the webcast program (t=3.50, p=0.001, Beta=0.36).
Research Question #3: What are participants' perceptions of the overall learning experiences and satisfaction levels in this online class? Table 5 presents the mean and SD for each variable from the exit survey. The survey was conducted immediately after students finished the last online section.

<TABLE 5. Participants" feedback on exit survey>
Again, the same procedure for multiple regression analyses was used in this case.
The dependent variables were: a student"s likelihood of recommending the course to others and undertaking other web-based courses in the future. The predictors were: the students" perception of convenience of INF 312 over face-to-face courses, usefulness of video tutorials, comparison of workload with other courses, reported need for printing online materials, personal contact, use of IM, and reports of fewer technical problems.
As for taking other web-based courses, the regression is (R 2 = 0.332) and the overall relationship was significant (F 2,43 = 10.70, p=0.000). The scores are positively related to instant messaging use (t=3.10, p=0.003, Beta=0.39), and to fewer technical problems (t=2.95, p=0.005, Beta=0.37). Additionally, the audio-video quality of the multi-modal objects used in the synchronous parts of the class serve as a proper variable to predict reports of their learning experiences and satisfaction within the five course modules. In terms of predicting student ratings of overall learning experience in this class, the convenience of the online class and the number of technical difficulties students encountered with the class are suitable variables. As of one project objectives is to identify meaningful variables for evaluating quality online courses, the investigators will develop an evaluation matrix based on these variables in future studies.

CONCLUSIONS
Regarding students" demographic characteristics and their computer skills and usage, female participants reported lower levels of computer skill and indicate a tendency to procrastinate in course work. Meanwhile, students" familiarity with computing technologies appears to influence their expectations for the course.
In general, the investigators found the quality of online AV materials varied in conjunction with student learning experiences in the five course modules. When students rated the quality of the AV materials high, they reported that they were able to follow online course activities and to engage with their fellow students and instructors. This additionally led higher overall satisfaction about the class. This finding suggests that the structure of smooth-running technologies and direction within which students can easily orient themselves contributes to a more satisfying online learning experience.
Findings also show reports of positive learning experiences as the result of experiencing fewer technical problems as well as use of multiple media and IM. Students with fewer technical problems indicated that they would recommend the course to their fellow students and will also likely take other online courses. On the other hand, students who experienced more technical problems reported a preference for face-to-face lecture courses. More frequent use of multiple media and IM seems to have prompted studentinstructor communications.
In order to reach instructional goals of online courses, educational institutions must work to prepare students, particularly female students, with relevant and necessary computing skills. Additionally, based on the range of student reports on skills, 20 confidence, and expectations, the creation of a self-assessment tool for skills and study habits might serve both students and the investigators in terms of understanding the requirements of online courses and managing expectations for success. The results of this study suggest that online courses should provide a rich array of online media and communication tools to strengthen course interactions and student engagement.
Additionally, this array of media and tools can expose students to the benefits and challenges of dealing with information and information technology in a networked world.
The investigators have identified several factors to contribute to the design and evaluation of web-based courses, specifically a course on the subject of technology and information literacy. This study, which focused on a non-mandatory activity within an elective class, may not reflect the impressions of students with low levels of confidence with technology, as they may be less likely to take online classes, or, once enrolled in online classes, may shy away from real-time collaboration that involves multiple technologies.
For future studies, a detailed framework of course design and evaluation and alternative data collection methods are needed. The framework will help educational institutions and course designers implement high quality courses and evaluate course outcomes. In addition to online surveys, the investigators plan to integrate other methods such as observations and personal journals in future studies.