/Garcia_fig1.png)
Abstract
In June 2015, the Colombian government via the Labor Ministry announced a project for young workers called 40.000 Primeros Empleos. In the framework of this project, the Ministry of Labor signed an alliance with the language platform Duolingo as a strategy to provide participants with English learning opportunities and a free language certificate. With the help of a monitoring and evaluation perspective, this study describes Colombian English language learning policies and their relationship with the labor market. The discussion presented here intends to maximize the outcomes of these kinds of agreements and to provide insights for researchers and national stakeholders willing to carry out similar projects in their countries. Certification is also thoroughly analyzed as a means of estimating the possible impact of this partnership.
Keywords: Computer-assisted language learning, Duolingo, foreign language learning policies, labor market, mobile-assisted language learning.
1. Introduction
The perspective of economics, globalization and internationalization have brought along the creation of open trading practices that require interactions in different languages for economic achievement. In governmental policies across the world (e.g., The European Union, Japan, China, The Middle East) an economic interest in foreign language education has been added to the ideas of cultural understanding and development of communication skills. Furthermore, current geopolitical issues have pushed governments to extend their language policies to the general population that is lacking in foreign language skills. Wagner (2005) insists that equality-oriented projects should be clear on what they are actually bridging. We see this in projects such as Odysseus, a European project to introduce a second language at the workplace, matching the language needs of migrant workers with the vocational workplace context (Grünhage-Monetti, Halewijn & Holland, 2003). The relationship between foreign language learning and equality of opportunity is already being discussed (e.g., Byram, 2008).
In this vein, promoters of information and communications technologies (ICT) in education consider that ICT will lead to a breakthrough in learning and contributes to social change and economic development (Wagner et al., 2005). In particular, mobile technologies can allow the development of policies that enable equal access for all. These technologies maximize informal and non-formal learning opportunities, expanding diverse educational settings (West & Vosloo 2015). Mobile learning projects such as the English in Action project in Bangladesh (2012) and the Samsung SDS Multi-Campus project (2011) are examples of pro-equity mobile learning initiatives. In Latin America, these kinds of efforts are still in their infancy. In 2012, there were 17 identified mobile learning initiatives resulting from pilot projects driven by non-profit organizations or universities; that same year, Colombia was the only country reporting active government support for mobile learning initiatives through public funding (Lugo & Schurmann, 2012). Until the publication of this paper, there have not been any more updated reports on mobile learning projects in Latin America. This implies that the mobile learning projects being carried out in different Latin American countries are still in the phases of piloting and experimentation.
2. A mobile language solution that attracts governments
Despite the need for concrete and credible results showing positive societal changes caused by mobile technology use (Jara, Claro & Martinic, 2012), the mobile language platform Duolingo is gradually becoming a foreign language learning solution for governments in Latin America. In 2014, Duolingo reached an agreement with the Guatemalan government to facilitate English learning in 97 public schools (Orellana, 2014). In 2016, Duolingo reached an additional agreement with Mexico. More particularly, the Jalisco Bilingüe program intends to provide 100,000 people with English skills in 2017 (El Universal, 2016). Similarly, Duolingo will begin training 350 primary education teachers on the pedagogic use of Duolingo after having reached an agreement with the government of Costa Rica (Presidencia de la República de Costa Rica, 2016).
In June 2015, the Colombian Ministry of Labor received a donation from Duolingo. The direct beneficiaries are 40000 young adults enrolled in a major program called 40.000 Primeros Empleos. The aim of this program is to facilitate the integration of young professionals into the labor market. The Colombian government presents the participation of Duolingo as a strategy to provide high-quality English education for free with the possibility of obtaining a free language certificate (Ministerio de Trabajo, 2015a).
Given that there have been different operational issues related to the 40.000 Primeros Empleos program, the implementation of Duolingo in the framework of the project has been delayed. Therefore, the objectives of this study are twofold: to provide a general perspective of language learning in the Colombian labor context, and to discuss the potential benefits of the Duolingo-Colombian government partnership. By doing so, this paper positions itself as a reference document for national and international educational stakeholders who consider Duolingo a language learning solution in their educational plans.
3. Method
Assessing governmental projects
One approach to measuring the impact of national initiatives around ICT in education is through Monitoring and Evaluation (M&E). M&E is a process that provides government officials, development managers and civil society with better means for learning from past experience (Grant et al., 2002, p.5). Without this approach, “ it would be impossible to judge if work was going in the right direction, whether progress and success could be claimed, and how future efforts might be improved” (UNDP, 2002, p.5). Even though M&E should be an integral component of any planned ICT for education program (James & Miller, 2005), and even though it can play a significant role in supporting equity approaches to ICT for education (Wagner, 2005), it is only starting to gain popularity in developing countries.
A descriptive approach to M&E provides fundamental insights that helps estimate the success of national initiatives. When dealing with ICT for education projects, Kozma and Wagner (2005) refer to four core aspects in long term M&E. One first important factor is the national context, which determines the potential of the project. A second factor is input, which comprises the analysis of the ICT resources (such as the software) and the role they play in students' knowledge and skills. A third factor is the outcomes, which are synonymous with the impact of the project regarding students' increased knowledge. A final aspect is cost, where cost-benefit and cost-effectiveness relationships are analyzed to measure the return on the investment.
Based on the core elements highlighted by Kozma and Wagner (2005), the Duolingo-Colombian government partnership in the project 40.000 Primeros Empleos is analysed. Furthermore, the discussion considers ICT in a conceptual education framework as proposed by Wagner, Day, James, Kozma, Miller & Unwin (2005). The adaptation of this framework is presented in figure 1.
Figure 1. Monitoring and evaluation discussion framework adapted from Wagner, Day, James, Kozma, Miller & Unwin, (2005).
4. Discussion
4.1. Development context (National educational context)
Colombian foreign language policies and their relationship with the labor market
Despite speaking a language that is shared by 400 million people around the world, Colombia has traditionally considered foreign language learning as an important aspect in the integration, understanding, and construction of ties beyond the borders of the Spanish language(1). Not only does Colombia have the mission to preserve the 65 indigenous languages spoken in its territory, it also needs to respond to the demands of a globalized world where economic, political and social issues are at stake (Ministerio de Educación Nacional, 2014). The importance of having contact with a foreign language was recognized in the General Education Act of 1994, which highlights the value of "acquiring conversation and reading skills in a foreign language" (Ley N°115 del 8 de Febrero de 1994). The act was reformed by the Ley N° 1651 del 12 de Julio del 2013, which gave priority to English. While it is true that English had already taken prevalence in bilingualism plans such as the National Bilingualism Plan of 2004, the educational reform officially legitimized English as the priority foreign language in Colombia.
The privileged position given to English is not agreed upon in its entirety. There are concerns about the situation of indigenous languages (Guerrero Nieto, 2008) and the teaching of other foreign languages necessary for Colombia (Valencia Giraldo, 2005). Similarly, some views question the idealization of English (Vélez-Rendón, 2003) and its promotion as a panacea against poverty (Torres-Martínez, 2009). Authors such as González (2007) go one step further and call into question the interests of institutions such as the British Council or Cambridge Language Assessment for an English-oriented bilingualism plan.
Even though these claims should not be disregarded, it is also true that bilingualism in English is an asset that allows people to benefit from globalization. The English language empowers individuals as it provides a more widespread access to knowledge. Not only is English the most common language in websites(2), English is also largely used in the spreading of scientific knowledge, to such an extent that its use is discussed in terms of lingua franca (e.g., Tardy, 2004). English also facilitates social mobility and is positively correlated with quality of life (Education First, 2015). These aspects add to the general advantages of bilingualism and bilingual education(3), which in turn have a positive impact on social capital. Social capital is understood as "the links, shared values and understandings in the society that enable individuals and groups to trust each other and, as a consequence, work together" (Brian, 2007:102).
An additional criticism towards foreign language learning plans in Colombia is that they convey a philosophy of employability and job training as the sole drive for education (Bonilla Carvajal & Tejada-Sánchez, 2016). Certainly, having skills in English is likely to have a positive impact on the economy of Colombia. The 500 biggest companies in Colombia require 100000 people with English skills, while the tourism sector has an immediate need for 110000. There is also a potential need for 40000 English qualified employees in the tourism sector by 2019 as well as 300000 employees in ICT and Business processing outsourcing (BPO) (Ministerio de Educación Nacional, 2014).
In Colombia, companies would be willing to pay more to employees having English language skills (Ministerio de Educación Nacional, 2014). This is also the case at an international level where employees with an exceptionally high level of English, as compared to their country’s level, earn a 30%-50% higher salary (Education First, 2015). Thus, English encourages competitiveness and social development (Seargeant & Erling, 2011) and it is positively correlated with per capita income and economic growth (Education First, 2015).
The above reasons have pushed the Colombian government to encourage English language learning beyond formal education, specifically in the labor context, and it has developed synergies between employers and employees by facilitating resources and subsidizing English language learning related projects (Ministerio de Educación Nacional, 2014). In 2012, for example, the Colombian national learning service, SENA, implemented English for Work, a continuing education project that allowed participants to learn English and be recruited by the companies participating in the program. As a result of the recent agreement with Duolingo in the frame of the 40.000 Primeros Empleos project, the beneficiaries will study in the Duolingo platform for six months and they will be provided with a language learning certificate after receiving a passing grade on the Duolingo test. The following sections describe Duolingo and its potential impact on the beneficiaries of the 40.000 Primeros Empleosprogram.
4.2. Software
Duolingo is composed of three main pedagogic tools: the Duolingo platform, the Duolingo for schools dashboard and the Duolingo test center (that provides a language certificate). Because the 40.000 Primeros Empleos program does not take into account the Duolingo for Schools dashboard, this feature is not widely addressed in this study.
4.2.1. The Duolingo platform
Duolingo is a popular language learning platform, with over 100 million users registered around the world(4). Users can access the platform from any web browser as well as through an app that is available for devices running with Android or iOS. Duolingo is free of charge and users can choose from a variety of languages to learn according to their source language. If the language of the user is Spanish, for instance, he/she can take a Duolingo language course to learn English, French, Portuguese, Italian, and German, Catalan, Guaraní, Esperanto, and Russian(5).
One first remark about Duolingo comes from its origin as a platform. In 2013, Duolingo co-founder Louis Von Ahn wrote a conference paper entitled " Duolingo: Learn a language for free while helping to translate the web." This article clearly states that the primary objective of Duolingo is breaking the language barrier in the web by translating its content: "Our goal is to encourage people, like you and me, to translate the Web into their native languages." (Von Ahn, 2013). This is achieved by Human Computation in which people "can be engaged to perform meaningful tasks through some other activities that they are already deeply interested in" (Law, 2011). In human computation, humans use computers to do something while computers will use that input to achieve something else (Garcia, 2013). This concept, though very innovative, puts into question the real educational aim of Duolingo. When asked about the main criticism of Duolingo, Mr. Von Ahn brings up the extended idea that Duolingo's central purpose is not to teach a language, but that users translate material for the company (Entrevista interactiva Duolingo, 2014).
While Duolingo does use crowdsourcing to translate documents on the web (including for CNN and Buzzfeed), use of this feature is not mandatory, and it is only presented through a tab in its laptop version. Eighty percent of Duolingo users practice languages only through the Duolingo app, and thus they are not able to see the to-be-translated texts (Entrevista interactiva Duolingo, 2014). Figure 2 shows how Duolingo presents the translation option to the students in the laptop version.
/Garcia_fig2.png)
Figure 2. Duolingo translation option in the laptop version.
Having said this, a user who accesses Duolingo is likely to perceive the educational outlook of the platform. Duolingo presents a language skill tree that is to be discovered through gamified language exercises. It offers a considerable degree of personalized language learning through elements that promote self-directed study and engagement through motivational features. Furthermore, in 2015 Duolingo launched Duolingo for Schools, a free dashboard which offers educators the possibility of integrating Duolingo into their language instruction. Teachers can see and use the complete language curriculum provided by Duolingo, and they can also assign homework and track the activity of students in the app. Figures 3 and 4 show the educational outlook of Duolingo.
/Garcia_fig3.png)
Figure 3. Duolingo language tree.
/Garcia_fig4.png)
Figure 4. Duolingo for schools dashboard.
One of the reasons why Duolingo has been given several awards in the industry(6) is its gamified method of instruction. In addition to Human Computation, Duolingo's success also relies on its Game with a Purpose method (Garcia, 2013) in which users "play" while learning a language. The visual outlook of the platform, the way to advance in the language course, the use of "lingots" as a fictional currency to help the user attain his or her learning goals and the possibility of competing against friends are all available options to keep the learning experience amusing and motivating.
Duolingo, alluding to “two languages” in many romance languages(7), uses translation at the level of words and sentences as its primary delivery method. Whenever a user decides to learn a language with Duolingo, the platform presents him/her with a variety of exercises. These practices include the translation of sentences from the source and target language of the user; the transcription of words or phrases in the target language; the pairing of words in the two languages; the choosing of the correct translated phrase through a multiple choice exercise; practice with flash cards in the two languages; and the translation of unknown words just by clicking or tapping on them. Figure 5 shows some types of exercises proposed by Duolingo.
/Garcia_fig5.png)
Figure 5. Duolingo exercise types.
4.2.2. The Duolingo test center
When advertising its language certificate, Duolingo points out what job seekers and students around the world have to go through to get a language certification. They have to pay a considerable amount of money to take the test, travel to an examination facility that can be far from where the person lives, and wait for weeks to receive the results (Duolingo Test Center, language certification for all, 2014). To tackle those issues, Duolingo created an app called Duolingo Test Center. This app allows people to take a language test using their mobile devices for a cost of only twenty dollars, one tenth of other tests (Duolingo Test Center, language certification for all, 2014).
4.3. Impact
4.3.1. The effectiveness of Duolingo on the beneficiaries of the 40.000 Primeros Empleos program
As part of its marketing strategy, Duolingo claims that 34 hours of studying on the platform equals a semester of language study at university. This claim is based on the Duolingo effectiveness study funded by Duolingo and carried out by Vesselinov and Grego (2012). In the website of Duolingo(8), it is suggested that the platform is effective because it covers the content of a university language course in less time. By this reasoning, the effectiveness of Duolingo is interpreted regarding the time taken to complete a language course and not considering actual learning outcomes. If this train of thought is followed in the case of the beneficiaries of the Duolingo-Government partnership, it should be noted that none of them is a real beginner in English. On average, a beneficiary with a high school diploma would already have studied a total of 720 hours of English(9). Considering that the English course in Duolingo begins from scratch, it is likely that the beneficiaries of the 40.000 Primeros Empleos program take shortcuts in the application that allow them to progress rapidly on the course. Therefore, it is expected that time engagement (and hence learning gain) with Duolingo will be considerably reduced.
The hypothesis of minimal learning gain of the beneficiaries of the 40.000 Primeros Empleos program is also supported by the delivery techniques Duolingo implements to provide language learning. Duolingo shares the characteristics of the Grammar Translation Method(10). Advocates of translation in foreign language learning argue that translation activities build self-confidence and reduce foreign language anxiety (Cook, 2010). They also claim that translation protects students’ linguistic and cultural identities (Hall, Graham, and Cook, 2012, p. 283) and concentrates on structures that build on what learners can already do (Campbell, 2002). Conversely, other views highlight that translation is an outdated method (Cook, 2002) that encourages a false equivalence between two languages and impedes automatic fluent language use (Jordan, 2015). Duolingo focuses on corrective feedback and measures language progress by assessing knowledge of the structure of the language(11). From studies about translation in language learning (e.g., Kobayashi & Rinnert, 1992; Uzawa, 1996) it can be derived that Duolingo could have a positive impact on the reading and writing skills of users. Nevertheless, grammar becomes so important in Duolingo that it frequently asks users to translate sentences that are detached from the real world, e.g., "The birds are reading the newspaper” or "Sorry, I am an apple." Therefore, Duolingo neglects other paramount aspects of language learning which are essential for the beneficiaries of the Duolingo-government partnership, namely communicative and intercultural competence.
For these reasons, we argue that Duolingo’s primary learning focus is vocabulary acquisition. According to Cook (2013), knowledge of words requires more than just knowing their meaning. It requires a complex range of information about their spoken and written form, the ways they are used in grammatical structures and diverse aspects of their meanings. In Duolingo, many of these aspects are addressed. A Duolingo user is faced with different kinds of exercises that make him/her conscious of the meaning, spelling, and pronunciation of a word. The platform also provides a syllabus consisting of grammatical points, vocabulary items and a list of vocabulary with the grammatical categories per word. Once again, these features fall into the assumptions of traditional language learning (Cook, 2013) which focus on conscious learning that does not produce true language competence (Krashen, 2014).
By looking deeper into the Duolingo effectiveness study by Vesselinov and Grego (2012), it becomes clear that language gain is equated to vocabulary learning. The study used the WebCAPE test as a learning outcomes instrument. WebCAPE is a multiple choice test that is used as a placement tool for students in their first two years of language instruction, and that is clearly form-based (Krashen, 2014). In other words, this test does not evaluate communicative ability as it is often assessed in a face-to-face setting .
The results of the study by Vesselinov and Grego show that the average effectiveness (gain) was 8.1 WebCAPE points per one hour of study (Vesselinov and Grego, 2012). Taking into account the WebCAPE score scale, the interpretation of the scale by higher education institutions(12), and the false beginner level of the beneficiaries of the 40.000 Primeros Empleos program(13), it is likely that the beneficiaries will not experience a major improvement in their learning outcomes.
Through this partnership, the Colombian government expects participants to achieve a B1 level of English proficiency in the Common European Framework of Reference for Languages, or CEFRL(14) (Ministerio de Trabajo, 2015a). The CEFRL is based on competences in listening, reading, spoken interaction, spoken production, and writing. Furthermore, it considers language learners as agents who have tasks – not exclusively language-related – to accomplish (Council of Europe, 2002). Since Duolingo is exclusively language related, it is likely that the B1 benchmark is achieved regarding vocabulary(15). Nevertheless, it is less likely that the beneficiaries of the partnership acquire communicative competence at level B1 by using Duolingo only. At this level, a language learner should possess the capacity to understand the main points on familiar matters encountered at work or school, produce texts on familiar topics, describe experiences and events by giving reasons or explanations, and deal with situations happening in the country where the language is spoken (Council of Europe, 2002).
An additional issue regarding the Duolingo-Colombian government agreement is that it is planned for a beneficiary to spend six months studying with Duolingo before taking the Duolingo language test. While Duolingo manages its own language platform as well as its own test center, there are differences in how Duolingo provides and tests language learning. Despite the fact that a learner completes the Duolingo language course by translating words and sentences, translation in the test center is non-existent. Furthermore, not only is the vocabulary learned in the platform uncertain to be tested by the test center, the exercises provided by the test center (i.e., cloze exercises) require a much higher cognitive effort than that experienced by studying in the Duolingo platform.
Regarding the Duolingo language certificate, it is true that the Duolingo test center uses the latest testing technology to generate a CEFRL-aligned score(16), and that it has validity and reliability (Ye, 2014), correlating substantially with the TOEFL (Ye, 2014) and IELTS (Bézy & Settles, 2015) total scores(17). Nonetheless, similar to the WebCAPE test, the Duolingo language test relies heavily on language form. Focus on language form is seen from the instructions given in the test, e.g. "Select the real English words in the list", "fill in the missing words using the drop-down menus", "type in English the statement that you hear." According to the Duolingo test center, a beneficiary successfully completing the Duolingo language test will be able to "fulfill most communication goals, even on unfamiliar topics, understand the main ideas of both concrete and abstract writing, and interact with native speakers fairly painlessly” (Duolingo test center, 2014). Nevertheless, the focus on form in the Duolingo language test does not really assure that the person having the certificate will actually be able to communicate in English effectively. A final remark about the language certificate is that Duolingo claims that it is recognized by "leading institutions around the globe." A closer view reveals that there is still a long way to go for the certificate to be widely accepted. The certificate is only recognized by ten institutions, including the government of Colombia and platforms like Uber and Linked In. It remains to be seen how the Duolingo language test will gain its place in the context of language certification.
5. Conclusion
During the discussion about the impact of Duolingo in the frame of the 40.000 Primeros Empleos program, different concerns about the Duolingo-Colombian partnership were addressed. The prevalent role of language structure and the possible (limited) learning gains obtained by the beneficiaries of the partnership were emphasized. Furthermore, the paper questions the role of the Duolingo language certificate as a means of demonstrating foreign language competence. Other factors that might have a negative impact on this agreement relate to the risk of the beneficiaries dropping out of the language course. Duolingo does not provide information on the percentage of students who finish its language courses, but what we can see from the Duolingo effectiveness study is that about 60% of the participants drop out of the course. Studies on commercial software similar to Duolingo, such as Auralog’s TELL ME MORE and Rosetta Stone, show severe participant attrition (Nielson, 2011). Data from similar content delivery methods such as MOOCs reveal that although many thousands of participants enrol in these courses, the completion rate for most courses is below 13% (Onah, 2014).
5.1. Maximizing the impact of the agreement
While there are issues that call into question the impact of the Duolingo-Colombian government partnership, there are still actions that can be implemented to maximize the positive impact on the beneficiaries. In the frame of the 40.000 Primeros Empleos program, the participants should follow a 40-hour training course on transversal and key competences to face their first job experience (Ministerio de Trabajo, 2015b). Transversal competences include critical and innovative thinking, interpersonal skills, intrapersonal skills(18), and global citizenship (UNESCO, 2015). Given that participants are expected to self-direct their language study, we suggest that the Ministry of Labor should allocate a few hours of training to intrapersonal skills to ensure successful Duolingo use. Research indicates that, by making a specific emphasis on self-regulation(19) and self-regulatory sub-processes(20), participants are more likely to make decisions on how to better use stand-alone language solutions such as Duolingo (Fischer, 2007; Garcia-Botero & Questier, 2016; Nielson, 2011; Onah & Sinclair, 2017). Therefore, some participants would likely be willing to cover all the units in the language course, while others would prefer to do a placement test or study on the to-be-translated documents provided by Duolingo.
We also recommend that the activity of the participants in Duolingo be followed through the Duolingo for Schools dashboard. Having the possibility of data tracking allows the understanding of the actual use of the platform and its real impact. The dashboard can be a channel of communication between the participants and the Ministry of Labor, which in turn could use the data obtained from the dashboard to create strategies to prevent and minimize dropping out.
Although this paper is not optimistic about the learning gain resulting from the use of Duolingo for the beneficiaries of the agreement, we do think that it is important to consider Duolingo at more basic levels of language knowledge, as is shown in the study by Vesselinov and Grego (2012). We believe that when participants' knowledge of English surpasses what Duolingo offers, they should be encouraged to explore the other languages provided by Duolingo that are also of interest for the Colombian labor market. As with English, the Ministry of Labor could provide participants with a certification for their "excellence and perseverance" for the completion of any other language course in Duolingo. This certification would have a positive impact on the participants' resume, and it would be proof of their commitment to complete goals.
5.2. Final comments
The active participation of Duolingo is an interesting add-on in the efforts to spread foreign language learning among the Colombian population. Duolingo encourages lifelong learning and it can be used with devices that people already have, representing a cost-effective strategy to learn a language. However, within the conditions established by the Government-Duolingo partnership, we have reservations about the real contribution of such a project for the targeted population. Governments interested in developing similar ICT projects should realize that simply having access to software does not ensure success. Cooperation between different branches of the government should be promoted (Vosloo, 2012), as well as a combination of input factors that positively influence impact (such as human resources, assessment, and teacher and student training). When a partnership is reached by means of a donation, M&E should not only be carried out by the donor, but also by all implied stakeholders in order to maximize the benefits to the beneficiaries.
Language projects, like the one analyzed in this study, should consider language in terms of language competence and not only regarding language form. It should be noted that learning a language should also encourage the development of 21st-century skills such as critical thinking, teamwork and problem-solving, which are necessary for human and economic development. Tasks should be similar to situations where users apply their knowledge in the real world (Wagner et al., 2005). We encourage the Colombian government to assess the considerations stated in this paper, and we call for policy makers and governments to have an M&E approach prior, during and after the implementation of ICT projects for education.
Acknowledgements
We thank the Colombian Ministry of Labor for their updates about the project.
Funding
Gustavo García Botero's research is funded by the European Commission's Erasmus Mundus Action 2-Eureka SD project. Grant number: 2013-2591/001-001.
References
Baker, C. (2014). A parents' and teachers' guide to bilingualism (Vol. 18). Multilingual Matters.
Bézy & Settles. (2015). The Duolingo certificate of English and East Africa: Preliminary linking results with IELTS and CEFRL. Duolingo research report. DRR-15-01, August 4, 2015.
Bonilla Carvajal, C. & Tejada-Sánchez, I. (2016). Unanswered Questions in Colombia’s Foreign Language Education Policy. Profile, 18(1), 185–201.
Brian, K. (2007). OECD Insights Human Capital. How what you know shapes your life. OECD Publishing.
Byram, M. (2008). From foreign language education to education for intercultural citizenship: Essays and reflections. Multilingual Matters, 17.
Campbell, S. (2002). Translation in the context of EFL – The Fifth Macroskill? TEFLIN, 13, 1.
Cook, G. (2002). Breaking taboos. Readings in Methodology, 46.
Cook, G. (2010) Translation in language teaching: An argument for reassessment. (Oxford: Oxford University Press
Cook, V. (2013). Second language learning and language teaching. Routledge.
Council of Europe (2002). Common European Framework of Reference for Languages: learning, teaching, assessment. Cambridge University Press.
Duolingo Test Center, Language certification for all, 2014. https://englishtest.duolingo.com/en
Education First (2015). English Proficiency Index. Comparing English skills between countries. EF EPI. Available from https://www.ef.edu/epi/.
El Universal. (2016). Duolingo presenta Jalisco Bilingüe. El Universal, retrieved from http://www.eluniversal.com.mx/articulo/techbit/2016/03/14/duolingo-presenta-jalisco-bilingue#imagen-1.
English in Action. 2012. Welcome to English in Action. Dhaka, Author. http://www.eiabd.com/eia/
Entrevista Interactiva Duolingo: Louis Von Ahn (2014) Retrieved from: https://www.youtube.com/watch?v=t0vpQ57bBSY.
Fischer, R. (2007). How do we know what students are actually doing? Monitoring students' behavior in CALL. Computer Assisted Language Learning, 20(5), 409-442.
García, I. (2013). Learning a language for free while translating the web. Does duolingo work? International Journal of English Linguistics, 3(1), 19.
García Botero, G. & Questier, F. (2016). What students think and what they actually do in a mobile assisted language learning context: new insights for self-directed language learning in higher education. In S. Papadima-Sophocleous, L. Bradley & S. Thouësny (Eds), CALL communities and culture – short papers from EUROCALL 2016, (pp. 150-154). Research-publishing.net. https://doi.org/10.14705/rpnet.2016.eurocall2016.553.
González, A. (2007). Professional development of EFL teachers in Colombia: Between colonial and local practices. Íkala, Revista de Lenguaje y Cultura, 12(18), 309-332.
Grant, A., Roldán, P., Peñate, H., López, L., Teruel, R., Gómez, M. E. & Mora, L. (2002). Monitoring & evaluation: some tools, methods and approaches (No. C10-61). World Bank.
Grünhage-Monetti, M., Halewijn, E. & Holland, C. (2003). ODYSSEUS – Second language at the workplace: Language needs of migrant workers: Organising language learning for the vocational/workplace context. Strasbourgh: Council of Europe Publishing.
Guerrero Nieto, C. H. (2008). Bilingual Colombia: What does it mean to be bilingual within the framework of the National Plan of Bilingualism? PROFILE Issues in Teachers’ Professional Development, 10, 27-45.
Hall, G. & Cook, G. (2012). Own-language use in language teaching and learning. Language Teaching, 45(3), 271-308.
James, T. & Miller, J. (2005). Developing a monitoring and evaluation plan for ICT in education. Monitoring and Evaluation of ICT in Education Projects, 57.
Jara, I., Claro, M. & Martinic, R. (2012). Mobile learning for teachers in Latin America: Exploring the potential of mobile technologies to support teachers and improve practice. United Nations, Educational, Scientific & Cultural Organization.
Jordan, G. (2015). Geoff Jordan vs. Duolingo. Retrieved from: http://eltjam.com/geoff-jordan-vs-duolingo.
Kobayashi, H. & Rinnert, C. (1992). Effects of First Language on Second Language Writing: Translation versus Direct Composition. Language Learning, 42(2), 183-209.
Kozma, R. B. & Wagner, D. A. (2005). Core indicators for monitoring and evaluation studies in ICTs for education. Monitoring and evaluation of ICT in education projects, 35.
Krashen, S. (2014). Does Duolingo “Trump” University-Level Language Learning? International Journal of Foreign Language Teaching, 9(1), 13-15.
Larsen-Freeman, D. & Anderson, M. (2013). Techniques and Principles in Language Teaching. 3rd edition. Oxford University Press.
Law, E. (2011). Defining (human) computation. CHI 2011: Workshop on Crowdsourcing and Human Computation.
Ley N° 1651 del 12 de Julio del 2013. (2013). Retrieved from http://www.colomboworld.com/academico/resoluciones/Ley1651de2013-LeydeBilinguismo.pdf.
Lugo, M. T. & Schurmann, S. (2012). Turning on mobile learning in Latin America: Illustrative initiatives and policy implications. Paris, UNESCO.
Milton, J. (2006). French as a foreign language and the Common European Framework of Reference for Languages. In Proceedings from the Crossing Frontiers: Languages and the International Dimension Conference, pp. 1-6.
Ministerio de Educación Nacional, (2014). Colombia very well! Programa nacional de inglés 2015-2025 Documento de Socialización, 41. Retrieved from http://www.colombiaaprende.edu.co/html/micrositios/1752/articles-343287_recurso_1.pdf.
Ministerio de Trabajo. (2015a). ¿Qué ofrece el programa 40 mil primeros empleos? Retrieved from http://unidad.serviciodeempleo.gov.co/soy-joven-y-quiero-participar-en-el-programa.
Ministerio de Trabajo. (2015b). Cartilla de presentación alianza Duolingo. Retrieved from http://www.mintrabajo.gov.co/40000-primeros-empleos.html.
Nielson, K.B. (2011). Self-study with language learning software in the workplace: What happens? Language Learning & Technology, 15
Onah, D.F.O., Sinclair, J.E. & Boyatt, R. (2014). Dropout Rates of Massive Open Online Courses: Behavioural Patterns. EDULEARN14 Proceedings, pp. 5825-5834.
Onah, D.F.O. & Sinclair, J.E. (2017). Assessing Self-Regulation of Learning Dimensions in a Stand-alone MOOC Platform. International Journal of Engineering Pedagogy (iJEP), 7(2), 4-21.
Orellana (2014). Estudiantes de Guatemala aprenderán inglés con Duolingo. La prensa. Retrieved from http://www.laprensa.hn/vivir/tecnologia/705787-98/estudiantes-de-guatemala-aprender%C3%A1n-ingl%C3%A9s-con-duolingo.
Presidencia de la República de Costa Rica, (2016). 350 docentes de inglés se capacitan para utilizar plataforma Duolingo dentro del Aula. Presidencia de la República de Costa Rica. Retrieved from http://presidencia.go.cr/comunicados/2016/03/docentes-de-ingles-se-capacitan-para-utilizar-plataforma-duolingo.
Samsung SDS (2011). Samsung SDS Multi-Campus. http://www.multicampus.co.kr.
Seargeant, P. & Erling, E. J. (2011). The discourse of ‘English as a language for international development’: Policy assumptions and practical challenges. Dreams and Realities.
Tardy, C. (2004). The role of English in scientific communication: lingua franca or Tyrannosaurus rex? Journal of English for Academic Purposes, 3(3), 247-269.
Torres-Martínez, S. (2009). Las vicisitudes de la enseñanza de lenguas en Colombia. Diálogos Latinoamericanos, 15. Retrieved from http://www.redalyc.org/pdf/162/16220868004.pdf.
UNDP (2002). Handbook on Monitoring and Evaluating for Results. New York: UNDP Evaluation Office.
UNESCO (2015). 2013 Asia-Pacific Education Research Institutes Network (ERI-Net) Regional Study on transversal competencies in education policy and practice: Phase 1. Regional synthesis report. UNESCO Bangkok office.
Uzawa, K. (1996). Second language learners' processes of L1 writing, L2 writing, and translation from L1 into L2. Journal of Second Language Writing, 5(3), 271-294.
Valencia Giraldo, S. (2005). Bilingualism and English language teaching in Colombia: A critical outlook. Presentation delivered at the Conference on English Language Teaching in Colombia, Universidad del Quindío, October, 2005.
Vélez-Rendón, G. (2003). English in Colombia: A sociolinguistic profile. World Englishes, 22(2), 185-198.
Vesselinov, R. & Grego, J. (2012). Duolingo effectiveness study. City University of New York, USA.
Von Ahn, Luis (2013). Duolingo: learn a language for free while helping to translate the web. Proceedings of the 2013 International Conference on Intelligent User Interfaces. ACM.
Vosloo, S. (2012). Mobile learning and policies: Key issues to consider.
Wagner, D., Day, B., James, T., Kozma, R. B., Miller, J. & Unwin, T. (2005). Monitoring and evaluation of ICT in education projects. A Handbook for Developing Countries. Washington DC: InfoDev/World Bank.
Wagner, D. A. (2005). Pro-equity approaches to monitoring and evaluation: Gender, marginalized groups and special needs populations. Monitoring and Evaluation of ICT in Education Projects, 93.
West, M. & Vosloo, S. (2015). Policy guidelines for mobile learning. UNESCO Publishing.
Ye, F. (2014). Validity, reliability, and concordance of the Duolingo English Test. Pittsburgh, PA. Retrieved from https://s3.amazonaws.com/duolingo-certifications-data/CorrelationStudy.pdf.
Zimmerman, B. J. (1986). Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology, 11(4), 307-313.
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329.
[1] A detailed timeline displaying the history of foreign language policies in Colombia is included in Bonilla Carvajal & Tejada-Sanchez (2016:188).
[2] Usage of content languages for websites. W3Techs.com.
[3] Baker (2014) recapitulates the advantages of bilingualism: Creativity, raised self-esteem, wider communication, literacy in two languages, greater tolerance of differences, sensitivity to communication, security in identity, as well as economic and employment benefits.
[4] Source: 100M users strong, Duolingo raises $45M led by Google at a $470M valuation to grow language-learning platform.
[5] As of July, 2017.
[6] iPhone app of the year 2013, Google's Best of the Best 2013 and 2014, TechCrunch's Best Education Startup, 2014.
[7] Duolingo: Design guidelines. https://www.duolingo.com/design.
[8] https://www.duolingo.com/effectiveness-study.
[9] http://www.mineducacion.gov.co/1621/article-97498.html.
[10] A synthesis of the characteristics of the "Grammar translation" method is found in Larsen-Freeman (2013).
[11] In the Duolingo skill tree, users pass through grammar units such as present simple, prepositions, adverbs, past simple, infinitives, present perfect, future or conditionals.
[12] Institutions such as the New York University interprets the scores of the Webcape test as follows: Beginner level: from 0 to 379 points; intermediate level: from 380 to 624 points; advanced: from 625+.
[13] According to our calculations, the current number of words existing in the Duolingo English course reaches 1914. Assuming that a Colombian high school graduate already had about 720 hours of English instruction, it is likely that he/she already knows a considerable number of the words offered by the platform. For instance, in the first unit Duolingo users learn the following words: Man-woman-I-am-a-boy-girl-not-from-Mexico-you-Spain-are-he-she-is-my-what-your-name. These words are already learned in primary school.
[14] The Colombian government decided to adopt the CEFRL as an instrument in the assessment of foreign language programs.
[15] Research on vocabulary in the CEFRL shows that the wordlists of languages come to an average of 1,000 words at the level of A2 and 2,000 words at the level B1 (Milton,2006). The Duolingo English course has 1,914 words.
[16] https://englishtest.duolingo.com.
[17] These tests are also aligned with the CEFRL.
[18] Intrapersonal skills include self-discipline, enthusiasm, perseverance, self-motivation, and commitment (UNESCO, 2015).
[19] Zimmerman (1989) defines self-regulation as the degree students are metacognitively, motivationally and behaviorally active their own learning process.
[20] Zimmerman (1986) refers to self-observation, self-judgement, and self-reaction as the key sub processes in self-regulation. Duolingo provides tools that, if used efficiently, encourage self-regulatory processes which are positively correlated with academic performance.