Salma Jan
Institute for Educational Development–Aga Khan University, Karachi, Pakistan
In the contemporary world, the use of information and communication technology by students in and outside schools is on the increase due to penetration of technology in teaching and learning processes. This penetration has brought about many changes in teaching and learning activities. For instance, information and communication technology (ICT) provides the element of interactivity, communication, and access to technology that did not exist before (Umar & Jalil, 2012). However, there are differences in usage of ICT among students in how they take advantage of ICT related elements due to their exposure and expertise in digital literacy. Today, students grow up in the knowledge society. Being digital natives, digital literacy is a survival skill, they need digital literacy skills in every aspect of their life. To be literate in the modern sense, it is important to be digitally literate (Ba, Tally & Tsikalas, 2002; Eshet-Alkali, 2004; Hargittai, 2010). The Pakistan National Curriculum of Computer Science at the secondary level (2000) acknowledges the role of information technology in the cultural and socio-economic development of society, and it emphasizes the development of computer skills. Likewise, Pakistan National Education Policy (1998-2010) insists on introducing computer classes and providing internet facilities in secondary schools and aligning the curriculum toward recent developments in information technology. Accordingly, within the Pakistani context, many organizations and schools prepare students for the current information age through the implementation of ICT integration programmes.
Despite availability and integration of ICT in many schools, students' interaction with ICT and their level of digital literacy is unknown in many developing countries like Pakistan. Firstly, it seems important to measure students' level of digital literacy in order to help them develop the knowledge and skills required for attaining 21st century skills. Secondly, most studies related to digital literacy have been conducted mainly in countries having higher levels of ICT integration and ICT resources than those countries which are on initial stages of ICT integration. In order to properly inform education policy, research is needed in a context like Pakistan where ICT is integrated rarely and irregularly in schools. Context-based studies will not only inform policy makers about students' digital literacy, but they will also present wide range of digital literacy skills in low to moderate levels of technological contexts.
The original meaning of literacy is reading, writing and the capability to communicate among each other, but literacy is also used in teaching ICT through different electronic mediums, such as graphs, illustration, computer graphics and hypertexts (Loveless, Taylor & Millwood, 2001). According to Wilhelm (2010), literacy is generally the ability to communicate or receive information using a culture's most common means of expression. If we relate this to current trends of communication, literacy is technology centred. Gilster (1997) defines digital literacy as "the ability to understand and use information in multiple formats from a wide range of sources when it is presented via computers" (p. 1). Likewise, other researchers (Aviram & Eshet-Alkalai, 2004; Ilomäki, Kantosalo & Lakkala, 2011) describe digital literacy as social, emotional, cognitive and technical skills. Kartz (2007) considers accessing, managing, evaluating, integrating, creating, and communicating information as the most important components of digital literacy. Indeed, digital literacy has become a life skill (UNESCO, 2016), and students need to develop this skill. Neither the availability of technology in schools nor the adoption of technology through a one- size fits all approach can ensure the development of these life skills.
Brindley (2000) explains,
The new literacy however, cannot be contained in the same way. There clearly exists a need to reconsider literacy as an ongoing development referred to earlier lifelong literacy. The acquisition of the new literacy cannot be picked off at eleven. Instead, while acknowledging the need for basic extended and advanced literacy skills, reflecting changes made in reading and writing demands through ICT. (p. 16)
For its National Curriculum (2006), Norway increased the status of digital literacy to be the fifth basic skill in the Norwegian schools with the aim to develop skills required for the digital world (Nusche, Earl, Maxwell, & Shewbridge, 2011). Likewise, in many developing countries, digital literacy is becoming a national priority; yet, the UNESCO's annual world report (2016) and UNICEF (2011) highlights the lack of improvement in the area of digital literacy in the developing countries.
In order to understand students' "skills and knowledge for tomorrow's world" (OECD, 2011, p.45), international organisations, such as ETS-ICT Literacy Assessment, Digital Competence Assessment and OECD Programme for the International Assessment of Adult Competencies, among, others have attempted to find students' digital literacy. These organisations have set indicators to describe digital literacy and digital competence. Moreover, some researchers (e.g., Hargittai, 2005; Zhong, 2011) have tried to measure the digital literacy with self-reported questions, and others have used multiple-choice questions (Berge, Hatlevik, Klovstad, Ottestad & Skaug, 2009) to assess digital literacy.
Nevertheless, assessing students' level of skills using digital technologies seems to be a big challenge as digital knowledge and skills are increasingly important (Young, 2015). A global measure of digital and ICT literacy skills (2016) declare that the level of digital literacy skills assessed by PISA and OECD differs from country to country. The higher scores in digital literacy can be attributed to environment and region in which students live. For instance, it is believed that students from developing countries possess low to moderate level of digital literacy due to lack of ICT infrastructure and resources. Lee, Kim and Lee (2015) in their study verified that students' environmental factors are associated with their test scores and ICT literacy levels.
The purpose of this study was to research secondary school students' level of digital literacy and investigate gender, class and school wise differences in digital literacy skills. The rationale for conducting this research is as follows. Firstly, educational technology is an emerging trend; teachers, policy makers and even parents need to be aware of their children's digital competencies in order to help them become lifelong learners. Secondly, there is a shift in educational processes from traditional literacy skills to the incorporation of current technological skills. Being digital natives, current students need digital literacy as survival skills, and it is their right to be digitally literate. Third, there is a lack of sufficient empirical studies (OECD, 2012; Rhema & Miliszewska, 2014) regarding students' levels of digital literacy. It is assumed that this study will be one of the few studies in the Pakistani context that investigates students' level of digital literacy.
The following research question guided the research presented in this article: What are gender, class and school wise differences in levels of digital literacy among secondary school students in Pakistan?
Data for the study was collected through a survey instrument, and it was analysed using the Statistical Package for the Social Sciences (SPSS). This study examined differences in the level of digital literacy among secondary school students. Descriptive statistics were used to summarise and describe the data. In addition, Mann-Whitney U test and Kruskal-Wallis tests were used to examine the difference between the variables.
This survey study involved students from three private secondary schools in Karachi, Pakistan. The sample size for the study was (n=344), representing 40.85% of students from the entire population (N=842) within the school. The sample includes male (n=169) and female (n=175) students of grade 9 and 10, ages 13-16 years. The sample was selected through simple random sampling.
This study used an online survey questionnaire to collect data. The questionnaire was adapted from an online European Commission digital literacy (ECDL) survey. The ECDL test was used because it was more relevant to the context, grade level and students' exposure to ICT. Besides, much of the work in the ICT literacy to date has been self-reported and theoretical. Consequently, the results of the study do not necessarily represent the actual ICT skills of students. The ECDL test seems to be an appropriate tool because it has multiple-choice questions designed to measure the students' actual skills in some basic concepts of ICT. Moreover, this tool appears to control for the possibility of students overestimating or underestimating their own ICT skills.
The survey questionnaire consisted of two sections. The first section sought information related to the participants' demographic characteristics, such as age, gender, class and experience of using technology. This section had 23 questions which determine and analyse factors that could influence students' digital literacy skills. The second section consisted of 11 skill areas related to commonly used ICT tools and applications, such as MS word, mobile phones and malware prevention. It tests the actual skills of students through multiple choice questions consisting of four options. Participants had to identify the correct application for each statement.
In order to ensure the quality of the survey tool, pilot testing was done in a secondary school within the same school system. The tool was piloted in order to determine content validity and establish reliability within the Pakistani context. The overall reliability of the tool was 0.8, which indicates good level of internal consistency for the scale.
The questionnaire was administered to the students in the computer lab of each school. Before administering the tool with the target sample, the purpose and process of research was briefly explained to the students. Social desirability (Bryman, 2004) was also tried to minimised by emphasising that the study aimed at exploring their digital literacy skills; it did not aim at assessing correct answers. Afterwards, the survey link was written on the white board and students were asked to open the survey and start filling the form on their computers. The participants were given 40 minutes to complete the survey. Participants filled out the survey presented via Google form in the presence of the researcher.
On completion of the field study, formal analysis began. Students' answers on digital literacy test were first checked. The answers were marked with 1 for every correct answer and 0 for every incorrect answer. Next, the students' marks were categorized into 'poor', 'fair', 'good', 'very good', and 'excellent' on the basis of marks obtained. The total score of the test were 11. A template was developed for data entry on SPSS 22.0. Marks of below 30% were labeled poor and coded 1, marks between (30-50%) were labeled fair and coded 2, marks obtained between (51-70%) were labeled good and coded 3, marks obtained between (71-90%) were labeled very good and coded 4 and marks above 90% were labeled excellent and coded 5. Frequencies were run to detect errors. Characteristics of data were explored by computing central tendency (mean, median, mode, and standard deviation). Total marks obtained, percentages, mean scores and standard deviation were calculated to find the difference across class, School And gender. Further properties of data were examined through exploring overall distributions and formation of tables and whisker plots. Furthermore, the Mann Whitney U test and Krushkal-Wallis test were employed to identify the differences in levels of digital literacy in terms of gender, class and school.
In order to present students' overall performance on digital literacy test, the percentages of mean scores were calculated for the whole sample. Results show that a large majority of students (97%) got more than 50% marks. One third of participants (n=112; 32.6 %) showed excellent digital literacy skills. A little less than half of the participants (n=163; 47.4 %) showed very good, and a quarter (n=79; 17.2%) of participants displayed good level of the digital literacy skills. Few participants (n=8; 2. 4%) showed fair and a very few participants (n=2; 6%) showed poor level in the digital literacy test. These results indicate that a high number of students possess good level of digital literacy. They have a good understanding and command on basic digital literacy. They have the ability to access, interpret and communicate through the basic ICT tools and applications.
A further analysis was conducted to discover the performance of participants on each question. Overall mean scores (M=8.6; SD=1.52) indicated that students' achieved high scores on the digital literacy test. Almost all questions are scored higher (M=0.5; SD=0.5) except question 6 (preventing computers from becoming infected by viruses) which scored the lowest (M=0.3; SD= 0.4). The highest score (M=0.9; SD=0.1) was observed in question 11; (sending messages through mobile phone). This indicates that students have good digital literacy skills in using mobile devices and lack skills in preventing their computers from viruses and malwares
In order to explore how males and females scored on the digital literacy test, the total mean score and standard deviation was calculated.
It is evident from whisker plot shown in Figure 1 that, on average male students scored (M= 8.6; SD=1.53) slightly higher than female students (M=8.5; SD=1.52).The difference in mean scores is 0.1. Moreover, four outliers in boys group indicate that four cases scored less than the lowest score in the group. Even so, the overall results are satisfactory with boys slightly more literate (78.18%) than girls (77.27%).
To see whether this difference in mean scores is significant or not, further analysis was carried out. Descriptive statistics showed that females (Median = 9.00; Mean rank=173.57) scored slightly higher with regard to digital literacy than males (Median =9.00; Mean rank =171.39). In order to understand the statistical difference between both groups, the Mann Whitney U test was used. A Mann-Whitney U test showed that male and female participants do not appear to differ in their score on digital literacy test, U=14600.5, Z=-.697, p > 0.05, with a negligible effect size of (r = 0.03). This means there is no significant difference in male and female digital literacy skills. They are almost at the same level.
To explore the difference in performance between the 9th and 10th grade students, mean scores of both groups on the digital literacy test were compared.
Results of the comparison presented in Figure 2 showed that, on average, the 10th graders achieved slightly higher scores (M=8.6; SD=1.5), which is 78.18% of the total score, as compared to 9th graders (M=8.5; SD=1.5), with 77.27%. Overall, it seems that the 10th graders performed slightly better than the 9th graders on the digital literacy test. However, the standard deviations show students of both classes have performed consistently. Four outliers in the 9th grade indicate the four cases scored less than the average lowest score in the group. Similarly, in the 10th grade, three outliers indicate the three cases that scored less than the average lowest score in the group.
The Mann Whitney U test was used to learn whether the difference between two groups is statistical different. Figure 2 presents the results of the comparison. The difference between medians scores of both groups was statistically insignificant, U=14754.5, Z=-.093, p > 0.05, with negligible effect size (r = 0.005), showing that there is no difference between grades 9th and 10th in terms of their level of digital literacy. This indicates that students of both classes have similar level of digital literacy.
School wise differences in overall mean Score of digital literacy
Figure 3 presents the difference in performance between Schools A, B, and C on digital literacy test. Overall mean scores of the schools were compared.
The results show that mean score of School C is (M= 9.1; SD=1.3) is higher than School A (M=8.3; SD=1.0) and School B (M=8.3; SD=1.9). Smaller value of standard deviation in School A indicates that the scores of School A are more consistent as compared to School B and C. The highest value of standard deviation in School B shows that the scores are least consistent as compared to School A and C. Interestingly, there is no outlier in School B, whereas there are three outliers in School A and eight outliers in School C. This suggests that three cases in School A have achieved lower scores than lowest score in the range of that school. Likewise, eight outliers in School C suggest that there are eight students who scored below the lowest score in their group.
The Kruskal-Wallis test was used to compare the difference between medians of dependent variable. The results showed that the difference in mean scores of schools was statistically significant χ2 (2, n=344) = 27.34, p <0.01 with a mean rank of 142.81 (Median =8.0) for School A, 163.77 (Median =9.0) for School B and 207.82 (Median =9.0) for School C. Because the overall test was significant, pairwise comparisons among the three groups were computed using the Mann-Whitney U test. A post-hoc test using Mann-Whitney U tests with Bonferroni correctionshowed the significant differences between Group A and C (r = 0.35, p < 0.01) and between Group B and C (r = 0.20, p < 0.05,). No significant difference was observed between groups A and B. The results on comparison of schools confirm the significant difference between School A and C (p < 0.01), B and C (p < 0.05) but not in School A and B.
Results of digital literacy test showed that almost all students possess adequate levels of digital literacy. The majority of students appear to achieve higher than 50% marks on the digital literacy test. On the whole, students seem to be most knowledgeable in the area of using mobile phones and most deficient in Internet and computer safety. This means students lack skills in preventing computers from becoming infected with viruses, including malware hazards and cyber-attacks. It is surprising that the students representing schools known to use ICT in their teaching and learning processes, are still unable to develop Internet and computer safety skills. Eshet-Alkalai (2004) fears that "Activity in cyberspace may be risky for immature and innocent users who do not understand the 'rules of the game'"(p.10). Based on these findings, educational policy makers need to be alerted to the fact that secondary school students require training related to computer and internet safety. Moreover, findings of this study indicate that students are more proficient in using mobile phones; therefore, it is important to consider the role of emerging gadgets such as mobile phones, smartphones and tablets in developing students' digital literacies. Moreover, it is essential to investigate the use of mobile phones and their effect on students' digital literacy. UNESCO Global Education Monitoring Report (2016) insists on understanding and exploring the set of competencies that are required to measure digital literacy applied in latest technologies.
In terms of differences in digital literacy, no statistical differences were observed for gender and class. However, the results of the comparison between schools show that students of School C are more literate in digital literacy skills as compare to A and B.
Previous research (Lee, Kim & Lee, 2015; Teck & Lai, 2011) indicated no difference in digital literacy or ICT literacy levels between male and female students. However, Umar and Jalil (2012) declared that male students demonstrate a significantly superior computer skill than female. Computers are historically considered to be the subject of study for males and males are considered to be more digitally literate than female. On the contrary, the findings of this study and other such studies (e.g.Varank, 2007) challenge the stereotype thinking and findings of the studies (e.g. Umar & Jalil, 2012) that computer and computer skills are the related of male. The world is changing drastically, and computers, which were once considered as luxury and masculine, are now available to many females, not only in the context of developed world but also in developing countries. Hence, in the context of this study and other similar studies, the similar levels of digital literacy between male and female students may be due to the equal opportunities given to female students by school systems and families to interact with technology.
Furthermore, this study found no significant difference between digital literacy and the class in which students are studying. No difference in digital literacy was observed between students of grades 9 and 10. In the context of this study, students of both classes do not differ so much in terms of age. The majority of these students belong to the same age group. It is possible that students possess similar levels of digital literacy because there is no significant difference in their use of computer in and outside the schools. Hargittai (2005) confirms that the differences in digital literacy are due to usability and user experience. Nevertheless, digital literacy can be associated with large differences in age groups or class level. Previous studies (Eshet-Alkalai & Hamburger, 2004) observed differences in digital literacy in different age groups. It can be anticipated that the larger differences in age and class level may lead to differences in level of digital literacy, but slight difference in age and class may not necessarily influence level of digital literacy.
Furthermore, the results show a smaller percentage (7.27%) of the variation in the digital literacy scores of students across schools. This variation in digital literacy score can be explained by differences at the School A and individual level. However, Hatlevik (2009) explains, "how students attain digital literacy is connected to systematic factors at the individual level, and the characteristics of the students, rather than factors at the school level" (p. 170). Conversely, in this study, the school seemed to be an important factor contributing to enhancing digital literacy because School C was likely to place more emphasis on ICT and is more resourceful as compared to the other two schools. This school has three ICT experts, and digital studio along with a functional computer lab. On the contrary, School B had no ICT expert or teacher (at the time of data collection) and School A has no digital studio. It seems that teacher competency and efforts contribute in developing digital literacy of students. Shuel and Farber (2001) conclude that the instructor's use of technology sets the stage for learning. Hence, the difference between the infrastructure and availability of resources might provide the reasons for the differences in the level of digital literacy among the three schools. On the basis of these findings, it can be assumed that ICT related factors contribute to developing students' digital literacy.
Based on the findings, it is recommended that education systems help students in developing advanced digital literacy skills so that they can take maximum benefit from using computer and other technological devices. Moreover, educational policy makers need to be alerted to the fact that secondary school students require training and proper guidance related to computer and internet safety. The findings from this study support the need for further research on digital literacy to have in-depth understanding of factors associated with digital literacy of students, but contextual digital literacy assessment tools and frameworks are not available. Therefore, contextual digital literacy assessment tools are needed in order to accurately measure the digital literacy of students who belong to the developing nations context where ICT is used at low to moderate levels.