An App to Teach With: A Case of Mobile Learning Non-adoption in Higher Ed

Sidneyeve Matrix

Queen's University

I wish all my courses had an app.
-Anonymous student questionnaire

In 2011, with seed funding from my university, I designed a smartphone application. The app seemed a logical extension for my mass communication course, which was already using several social media sites to distribute study aids, course content and research links, such as Facebook, Twitter, Moodle, Scribd, and iTunesU. In 2011 course enrollment totaled 700 students, and eventually expanded to 1,400 students by 2012 in a single massive course section. While some students appreciated the plugged-in nature of the course circa 2009 and 2010, others complained that the content was too widely dispersed across the Web. This made it difficult for all but the most dedicated students to keep up and cover all of the course content-and so the idea for ClassCaddy was born.

ClassCaddy was thus intended to solve a pedagogical problem: my class had become a Frankencourse, comprising content housed on various social sites, a collection of online tools, the campus learning management system (LMS), and a custom website (Kuhlmann, 2012, para. 2). The course website attempted to bring course content from the LMS and the cloud together on a single page, but the effect was overwhelming. While students were engaged and enthusiastic, in their end-of-term evaluations they often reported feeling a sense of information-overload.

Vision for ClassCaddy

Developed in response to this student feedback, I envisioned ClassCaddy as a next-generation PDA (personal digital assistant): a cleaner, more inspiring and elegant visual experience packaged in a native smartphone app. ClassCaddy would serve as a content aggregator, enabling students to access mobile-optimized versions of the course's myriad multimedia instructional materials on one handheld screen. When launched in 2011, Class Caddy inspired some professional interest: professors contacted me from other departments on campus and far beyond. I received inquiries from educators and administrators across North America, the UK and Europe, many of whom expressed a strong interest in buying and using Class Caddy or in designing their own version of it. This is not surprising in light of the current rampant enthusiasm about mobile learning for higher education (Biggs & Justice, 2011; Clough, Jones, McAndrew, & Scanlon, 2009; Elias, 2011; Matias & Wolf, 2013; Mueller, Wood, De Pasquale, & Cruikshank, 2012; Wong, 2012). Stephen Baldridge of Abilene Christian University has observed that "Mobility, combined with things like social media, has already shown us the potential it has to engage students" (Bengfort, 2013, para. 7). Research also suggests that the "instant access to information" afforded by a smartphone "promotes both referential and reflective activities" (Clough, Jones, McAndrew, & Scanlon, 2008, p. 365). These were exactly the pedagogical goals I had in mind for ClassCaddy.

ClassCaddy is a bespoke app, a custom-designed mobile learning tool configured specifically for my class and launched in the fall of 2011. It remains available for free download from Google Play and iTunes stores. Updated each semester, the app includes links to audio podcasts, videos, copies of lecture presentation slides and flashcards, as well as status updates from class-related content posted on social media streams of Facebook and Twitter. Lecture outlines, links to online readings, biographical and contact information for the professor and teaching assistants are also available in-app, along with a wall for comments and ratings from other fellow app student users who have downloaded ClassCaddy.

Between September 2011 and June 2013, it was used in courses with a total enrollment of 2,700 students. During that same period, the app was downloaded just over 2,200 times. At a glance, the similarity between these numbers might suggest near-universal adoption by students-but this is not the case. To date, fewer than one third of my students have downloaded the app.

Student feedback, together with research into the mobile technology habits of Generation-Y (Gen-Y), suggests various reasons for this low adoption rate. This article will outline several such reasons. In addition, I will document unexpected outcomes from my classroom experience. Finally, I will offer lessons learned from the design, development and implementation of ClassCaddy, and share ideas for faculty and instructional designers interested in using bespoke mobile applications for college-level courses.

Challenges

Device Difficulties

I need to get a smartphone asap [sic]!
-Anonymous student questionnaire

Prior to and after creating the ClassCaddy app, I conducted informal surveys of my students using online polling and clickers to identify which smartphones, if any, they were using. I learned that approximately 20% of my students did not have smartphones. Of those who did, roughly 60% used BlackBerry devices. From 2010 to 2013 these percentages did not shift significantly, although I had expected they would. The prevalence of BlackBerry devices among these student cohorts is potentially a key reason for the low adoption rates of ClassCaddy. ClassCaddy was developed using MobileRoadie (Mobile Roadie Inc., 2013), a do-it-yourself (DIY) mobile content management system that offers native apps for iOS and Android but not for BlackBerry or Windows phones. My research identified no DIY solution that would have allowed me to create a BlackBerry application without a working knowledge of several programming languages in addition to HTML.

BlackBerry Device Usage

In 2013, 80% of Canadians belonging to the Gen-Y cohort (18-24) owned a smartphone, as compared to 62% of the general adult population (CBC News, 2013). As I set out to design an app to teach with I did an informal survey of my students, and found that in 2011 approximately 60% of those enrolled in my class were using BlackBerry phones. Compared to statistics for smartphone adoption in Canada as a whole, today 40% of smartphone users have Android devices, 35% have iPhones and 20% are using BlackBerrys (comScore, 2013). The disproportionately high percentage of BlackBerry users on my campus can be traced partly to the popularity of the BBM chat messaging application among Canadian students. Until October 2013, this app was available exclusively on BlackBerry devices (Fingas, 2013).

My decision to go ahead with developing an application unusable on BlackBerrys was not unusual considered in the context of industry trends in mobile app design. Because mobile application developers follow demand, fewer and fewer are designing apps for BlackBerry phones (Carney, 2012). When asked if a BlackBerry or Windows version of his company's popular photo sharing app is in the works, Instagram CEO Kevin Systrom explained they were not because "the way we evaluate where to expand to is really simply where our users are, and where the growth is" (Hardy, 2013, para. 2) – a response echoed by Netflix (para. 3).

In 2013, the learning management system (LMS) Blackboard released Mosaic, a mobile app development tool for iOS and Android (Blackboard, 2013). Blackboard has announced no plans to extend functionality to BlackBerry or Windows phones, having previously ended support for native BlackBerry apps (Blackboard, 2012). Mosaic is comparable to MobileRoadie insofar as it allows relatively unskilled developers (like myself) to design and distribute applications rapidly. Mosaic, MobileRoadie and other DIY mobile development tools enable app creators to include custom multimedia as well as ready-made features such as chats and campus maps, using code-free Web-based interfaces and what-you-see-is-what-you-get (WYSIWYG) development environments. Application updates can be made via the website and pushed to users' mobile devices without waiting for app store approval (Blackboard, 2013). If other LMS companies follow suit and campus apps proliferate on Apple and Google platforms exclusively, Windows and BlackBerry smartphone sales on college campuses are unlikely to benefit.

The shift from BlackBerry devices to competing platforms did not occur as rapidly as I imagined it would, however-at least not on my campus. Polls taken as late as the summer of 2013 showed that a majority (roughly 60%) of my students continued to use Blackberry phones. There is some possibility that the smartphone landscape in Canada will change more quickly in the future: in 2013, the Canadian Radio Telecommunications Commission passed regulations ending the mandatory 3-year contracts favored by all major Canadian cell phone providers (CRTC, 2013).

After excluding BlackBerry users and those without smartphones, fewer than one-third (approximately 850) of the students in my multi-year sample owned devices compatible with the ClassCaddy app. Simply having the hardware was not enough: students also needed to download ClassCaddy from the Google Play Store (for Android devices) or iTunes Store (for iOS devices). My assumption that smartphone-owning students would customize their devices with dozens of mobile apps, including mine, turned out to be mistaken.

Data Dilemmas

The app is great, I just have a very small amount of data on my phone, so I could not use it often.
-Anonymous student questionnaire

To download and operate ClassCaddy away from a Wi-Fi network, students needed to have a cellular data plan on their phone. I had assumed students would have such a plan, but many reported they did not, and therefore were not able to easily download or reliably use the application. While the average U.S. smartphone user had 41 apps downloaded to his or her device in 2012 (The Nielsen Company, 2012), mobile application and Internet usage in Canada lags behind the US. Although almost half (47%) of Canadians owned a smartphone in 2012 (Ipsos, 2013), only 6 of 10 Canadian smartphone owners surfed the Web or downloaded apps on their devices (eMarketer, 2012). Students could use Wi-Fi to download ClassCaddy and access content within the app, but many complained that outside the library campus wireless connectivity was painfully slow and unreliable. Moreover, the terrifying specter of "bill shock" was too great to risk using the application without a robust data plan (CBC News, 2012). Canada ranks among the three most expensive countries in the world for standard mobile data plans (Geist, 2013).

Canadian telecoms provide social networking plans to suit Gen-Y consumers, which include minimal mobile Web access, but unlimited Tweeting and Facebook access. Students with limited or no cellular data access through their smartphones may thus still have full access to Facebook and Twitter. Consequently, these mobile-optimized sites could be leveraged as part of a mobile learning content strategy. Integrating these popular social media platforms with teaching introduces other issues, including privacy concerns, the management of distractions, and student resistance to using Facebook for school purposes (Fife & Orjuela, 2012; Darling & Foster, 2013; Gikas & Grant, 2013; Gehlen-Baum & Weinberger, 2012).

Sustained Engagement

I'm not sure about other students, but I preferred using a computer a lot more.
-Anonymous student questionnaire

On average, 20% of those students who downloaded the ClassCaddy application reported they did not use it regularly over the course of the 12-week term. Reported reasons for abandoning the app included forgetting that it existed, experiencing technical glitches while trying to use it, or both. This is not uncommon: market research on the mobile media ecosystem shows that about one quarter (26%) of applications downloaded and installed on smartphones are used only once or not at all. Up to 95% of installed apps are abandoned within a month (Empson, 2011; Gary, 2011). An app may end up in the mobile dustbin for many reasons, including stale content, a confusing user interface (UI), technical glitches, frequent or irrelevant push notifications, misleading app store descriptions, and privacy violations such as requests for access to Facebook or locational data (Perez, 2013; Fuming, 2013). The abundance of free apps available to smartphone users also encourages impulse acquisitions that may be quickly forgotten.

Out of memory.

Several of my students reported forgetting about ClassCaddy: "Just downloaded it now," wrote one student during the last week of the course. "Forgot this existed. You could have plugged it a little more!" another student suggested. "Although I haven't used it regularly throughout the semester," a third student stated, "I am hoping to really make full use of this app for exam studying since it's so convenient for on-the-go learning." This problem could be solved with more frequent integration of the app into weekly lectures, thereby reminding students of this study tool.

Redundancy and user experience (UX). 

Forgetfulness aside, by far the most frequently cited reason for discontinued use was a perception that ClassCaddy content was redundant. "Whenever I intend to do homework, I always have my laptop on me anyway," one student wrote on the end-of-term questionnaire, "I use that device as it's more comfortable and has faster processing." "I found the ClassCaddy app to be a great idea," another student responded, "but in the end I used it minimally because I just find it much easier to access info needed for this course on a full website." Even though the ClassCaddy application featured mobile-optimized versions of all instructional materials, students reported a strong preference for accessing content on the larger screens of their laptops. In a sense, I was back where I started, as students apparently preferred to watch the videos on YouTube, listen to the podcasts via iTunes and access the Facebook page via a Web browser-replicating the fractured Web-distributed content model that had inspired my mobile aggregator plans in the first place.

This feedback suggests both a UX issue (small smartphone screens are bothersome to use for extended periods) and a content issue (redundancy). The latter problem could be addressed by designing app-exclusive content, thus providing strong encouragement for students to adopt the app. Unless faculty could be assured that all students had access to the necessary hardware and wireless data plans, however, such a solution would be inadvisable.

Student concerns about the user experience also resist easy answers. The ClassCaddy app was configured primarily as a portal for instructional content, rather than as a social or entertainment tool. It therefore contained a lot of reading material: lecture outlines, online articles, PowerPoint slides and flashcards for exam prep. A recent study by Foasberg suggested that millennial students consider computer screens and e-books suitable for "less serious work," but still prefer print when they "really wanted to get work done" (as cited in Grossman, 2013).

Potential Improvements

How then to design an educational application that will be downloaded and used, rather than deleted or forgotten? What kinds of apps perform consistently well in iTunes? Some of the most popular educational apps are, in fact, books and study guides. Cross-category comparisons show that utilities top even games in popularity: a 2011 survey by the Pew Internet & American Life Project identified a preference for informational apps (e.g., weather and news updates) above all others (Purcell, 2011). Of those users who regularly download apps to their phones and tablets, Purcell found that 64% were motivated to do so in order to "learn about something in which they are interested" (p. 3). Likewise, Bellman, Potter, Treleaven-Hassard, Robinson and Varan (2011) found that informational smartphone applications with informational or utilitarian goals, such as "on-line [sic] shopping or banking" (p. 193) were "more effective at shifting purchase intention" (p.198) than were experiential apps like games or digital magazines. Moreover, best practice guidelines for app developers contend that an application should feature "constantly changing or updated information" to encourage downloads and continued use (Clancy, 2012, para. 9).

For an educational app developer, these insights are encouraging and valuable. Cumulatively, they indicate that designing an information-rich app for course use merits further investigation. Together with student feedback, research into informational apps suggests that course applications should deliver content in smaller chunks, and in formats suitable for small screens. Supplemental materials should also be provided for motivated students who may wish to go beyond the required readings. Finally, if regular updates of in-app content inspire sustained use, as the research suggests, this would work especially well for educators who are willing to schedule weekly updates. Posting just-in-time course information and resources via mobile platforms would inspire students to get in the habit of checking the app regularly, helping them stay on-track and in-sync with the course progression.

Conclusion

It is a great app. I used it regularly even when I am on a bus, I can check the posted slides and the topics. That's really helpful. Thank you!
-Anonymous student questionnaire

The 2012 Horizon Report for Higher Education ranked as its number one technological trend that "People expect to be able to work, learn, and study whenever and wherever they want to" (Johnson, Adams, & Cummins, 2012, p. 5). Developing mobile technologies to support learning will only become more attractive to educators as this and other trends continue: more students are adopting smartphones and tablets; connectivity is becoming ubiquitous; and data is becoming increasingly portable (Biggs, 2011; Engel, 2011; Kukulska-Hulme et al, 2011; Kukulska-Hulme, 2012; Wong, 2012). As Murphy and Farley observe, however, "there are still a number of barriers that influence the adoption of m-learning initiatives in education, both at an institutional and at a user level" (2012, p. 1). From device wars to overpriced data plans, too-small screens, information overload and technical glitches, mobile learning instructional design is a complicated initiative. Yet the experience of building an app to teach with is exceptionally valuable: it reveals key insights into learner preferences, student technology skills and daily media use habits, and identifies the digital learning assets students find most engaging and useful.

Acknowledgements

The author would like to thank Max Lieberman and the anonymous reviewers at Issues and Trends in Educational Technology for their helpful feedback and editorial assistance. Queen's University PARTEQ Innovations generously provided funds for the development of the Class Caddy app.

References

Bellman, S., Potter, R. F., Treleaven-Hassard, S., & Robinson, J. A. (2011). The Effectiveness of Branded Mobile Phone Apps. Journal of Interactive Marketing, 25(4), 191-200.

Bengfort, J. (2013, June 27). Mobile Learning on Campus Is Hands-On, Engaging and Here to Stay. EdTech Magazine. Retrieved from http://www.edtechmagazine.com/higher/article/2013/06/mobile-learning-campus-hands-engaging-and-here-stay

Biggs, B. & Justice, R. (2011, April 12). Mobile Learning: The Next Evolution. Chief Learning Officer, 10(4), 38-41. Retrieved from http://clomedia.com/articles/view/mobile-learning-the-next-evolution

Blackboard. (2012, November 15). Ending Mobile Central Support for Native BlackBerry Apps [blog post]. The Mobile Garage. Retrieved from http://www.mobilegarageblog.com/2012/11/ending-mobile-central-support-for-native-blackberry-apps.html

Blackboard. (2013, June 21). Blackboard Launches Mosiac, A New Approach to Mobile Apps for Schools and Universities [press release]. Retrieved from http://www.blackboard.com/About-Bb/News-Center/Press-Releases.aspx?releaseid=122698

Canadian Radio-television and Telecommunications Commission. (2103). Canadians can cancel their wireless contracts after two years under new CRTC wireless code. Online: http://www.crtc.gc.ca/eng/com100/2013/r130603.htm#.Uq9yjWRDv1Y

Carney, John. (2012, December 5). Who-If Anyone-Is Developing BlackBerry Apps? CNBC. Retrieved from http://www.cnbc.com/id/100280568

CBC News. (2012). 4 tips to avoid smartphone 'bill shock': Understand the application before downloading it, telecom experts say. Retrieved from http://www.cbc.ca/news/canada/4-tips-to-avoid-smartphone-bill-shock-1.1261342

CBC News. (2013). Millennials' mobile tech savvy leaves boomers behind. Retrieved from http://www.cbc.ca/news/technology/millennials-mobile-tech-savvy-leaves-boomers-behind-1.2449252

Clancy, H. (2012, March 12). Three considerations guiding SMB mobile app development [blog post]. Small Business Matters. Retrieved from http://www.zdnet.com/blog/small-business-matters/3-considerations-guiding-smb-mobile-app-development/1818

Clough, G., Jones, A. C., McAndrew, P., & Scanlon, E. (2008). Informal learning with PDAs and smartphones. Journal of Computer Assisted Learning, 24(5), 359-371. DOI: 10.1111/j.1365-2729.2007.00268.x

Clough, G., Jones., A., McAndrew, P., & Scanlon, E. (2009). Informal learning evidence in online communities of mobile device enthusiasts. In M. Ally (Ed.), Mobile learning: Transforming the delivery of education and training (99-112). Edmonton, Canada: Athabasca University Press. http://oro.open.ac.uk/id/eprint/23850

Darling, J, & Foster, M. (2012). Preparing Students to Join the Global Public Sphere. International Studies Perspectives 13(4), 423-436.

Elias, T. (2011). Universal Instructional Design Principles for Mobile Learning. International Review of Research in Open and Distance Learning, 12(2), 144-156. http://www.irrodl.org/index.php/irrodl/article/view/965/0

Fuming Shih, Julia Boortz. (2013). Understanding People's Preferences for Disclosing Contextual Information to Smartphone Apps. Human Aspects of Information Security, Privacy, and Trust. Vol. 8030: 186-196.

eMarketer (2012, September 4) PCs Preferred to Smart Devices for Web Use in Canada. Retrieved from http://www.emarketer.com/Article/PCs-Preferred-Smart-Devices-Web-Use-Canada/1009318#3mpRTjQC43PPqPtm.99

Empson, R. (2011, March 15). Mobile App Users Are Both Fickle And Loyal: Study [blog post]. TechCrunch. Retrieved from http://techcrunch.com/2011/03/15/mobile-app-users-are-both-fickle-and-loyal-study

Engel, G., Palloff, R. & Pratt, K. (2011). Using Mobile Technology to Empower Student Learning. Paper presented at the 27th Annual Conference on Distance Teaching and Learning, Madison, WI. Retrieved from http://www.uwex.edu/disted/conference/Resource_library/proceedings/46307_2011.pdf

Fife, E., & Orjuela, J. (2012). The Privacy Calculus: Mobile Apps and User Perceptions of Privacy and Security. International Journal of Business Engineering Management, 4(11). DOI: 10.5772/51645

Fingas, Jon. (2013). BlackBerry resumes BBM launch for Android and iOS [blog post]. Engadget. Retrieved from http://www.engadget.com/2013/10/21/blackberry-resumes-bbm-launch-for-android-and-ios/

Gary, R (Interviewee). (2011). Why 95% of Mobile Apps are Abandoned [corporate whitepaper]. Retrieved from Nuance Communications, Inc.: http://www.nuance.com/ucmprod/groups/enterprise/@web-enus/documents/collateral/nc_020218.pdf

Gehlen-Baum, V., & Weinberger, A. (2012). Notebook or Facebook? How students actually use mobile devices in large lectures. In A. Ravenscroft, S. Lindstaedt, C. D. Kloos, & D. Hern‡ndez-Leo (Eds.), 21st Century Learning for 21st Century Skills (103-112). Proceedings 7^th European Conference of Technology Enhanced Learning. SaarbrŸcken, Germany. Springer: Heidelberg, Germany. Retrieved from http://link.springer.com/chapter/10.1007%2F978-3-642-33263-0_9

Geist, M. (2013, July 15). OECD Report Confirms What Canadians Have Long Suspected: Wireless Pricing Among Highest in the World [blog post]. Michael Geist's Blog. Retrieved from http://www.michaelgeist.ca/content/view/6905/125/

Gikas, J., & Grant, M. M. (2013). Mobile Computing Devices in Higher Education: Student Perspectives on Learning with Cellphones, Smartphones & Social Media. The Internet and Higher Education, 19, 18-26.

Grossman, S. (2013, July 17) Students Prefer Print for Serious Academic Reading. The Chronicle of Higher Education. Retrieved from http://chronicle.com/blogs/wiredcampus/students-prefer-print-but-not-books-for-serious-academic-reading/44871

Hardy, I. (2013, June 27) Instagram's CEO says BlackBerry and Windows Phone apps are not in the immediate plans. MobileSyrup. Retrieved from http://mobilesyrup.com/2013/06/27/instagrams-ceo-says-blackberry-and-windows-phone-apps-are-not-in-the-immediate-plans/

Ipsos Reid. (2013, February 21) Close to Half of Canadians Now Own a Smartphone [press release]. Retrieved from http://www.ipsos-na.com/news-polls/pressrelease.aspx?id=6005

Johnson, L., Adams, S., & Cummins, M. (2012). The NMC Horizon Report: 2012 Higher Education Edition. Retrieved from http://www.nmc.org/pdf/2012-horizon-report-HE.pdf

Kuhlmann, Tom. (2012, June 26) Avoid the Curse of the Frankencourse [blog post]. The Rapid E-Learning Blog. Retrieved from http://www.articulate.com/rapid-elearning/avoid-the-curse-of-the-frankencourse/

Kukulska-Hulme, A., Pettit, J., Bradley, L., Carvalho, A. A., Herrington, A., Kennedy, D. M. & Walker, A. (2011). Mature Students Using Mobile Devices in Life and Learning. International Journal of Mobile and Blended Learning, 3(1), 18-52. DOI: 10.4018/jmbl.2011010102

Kukulska-Hulme, A. (2012). How should the higher education workforce adapt to advancements in technology for teaching and learning? The Internet and Higher Education, 15(4), 247-254.

Matias, A., & Wolf, D. F. (2013). Engaging Students in Online Courses Through the Use of Mobile Technology. Cutting-Edge Technologies in Higher Education, 6, 115-142. http://dx.doi.org/10.1108/S2044-9968(2013)000006D007

Mobile Roadie Inc. (2013) MobileRoadie [web application]. Available from http://mobileroadie.com/

Mueller, J. L., Wood, E., De Pasquale, D., & Cruikshank, R. (2012). Examining Mobile Technology in Higher Education: Handheld Devices In and Out of the Classroom. International Journal of Higher Education, 1(2), 43-54. http://dx.doi.org/10.5430/ijhe.v1n2p43

Murphy, A., & Farley, H. (2012). Development of a framework for evaluating the impact and sustainability of mobile learning initiatives in higher education. In M. Brown, M. Hartnett & T. Stewart (Eds.), Future challenges, sustainable futures (678-680). Proceedings ascilite 2012. Wellington, New Zealand. Retrieved from http://ascilite.org.au/conferences/wellington12/2012/images/custom/murphy%2c_angela_-_development.pdf

Perez, S. (2013, March 12). Users Have Low Tolerance For Buggy Apps - Only 16% Will Try A Failing App More Than Twice [blog post]. TechCrunch. Retrieved from http://techcrunch.com/2013/03/12/users-have-low-tolerance-for-buggy-apps-only-16-will-try-a-failing-app-more-than-twice

Purcell, K. (2011, November 2). Half of adult cell phone owners have apps on their phones. Pew Research Center's Internet & American Life Project. Retrieved from http://pewinternet.org/Reports/2011/Apps-update.aspx

The Neilsen Company. (2012, May 16). State of the Appnation-A Year of Change and Growth in U.S. Smartphones. Retrieved from http://www.nielsen.com/us/en/newswire/2012/state-of-the-appnation-%C3%A2%C2%80%C2%93-a-year-of-change-and-growth-in-u-s-smartphones.html

Wong, W. (2012). Tools of the Trade: How Mobile Learning Devices Are Changing the Face of Higher Education. Community College Journal, 82(5), 54-61.