THE EFFECTIVENESS OF DUOLINGO MOBILE APPLICATION IN TEACHING TOEFL PREPARATION OF STRUCTURE AND WRITTEN EXPRESSION
Mohammad Sugeng Rahman Hakim
English Education and Literature, Post-Graduate Program, Surabaya State University
This research examines how the use of mobile educational application influences the learner’s understanding and motivation through process of learning in classroom and home environment. This research focus on study the effectiveness of the application in teaching TOEFL preparation of structure and written expression. The application of mobile devices has become common part of our daily lives, they are used for many kinds of purposes, including as educational objectives in learning foreign language which in this case the researcher uses the application of Duolingo. The Methodology of This study population included the researcher’s twelfth grades students of Madrasah Aliyah Atqia Bondowoso as the group who is obliged to use the application on their mobile device in classroom and home for two weeks. The second group consisted to the other class in the same school which uses the traditional course. Both of the group is tested using pre-test and post-test. The findings were analyzed use IBM SPSS version 22, and model was examined with partial Least Squares Structural Equation Modeling. Thus, the application of Duolingo can be considered effective.
Keywords: mobile educational applications, foreign languages, Duolingo, TOEFL, structure and written expression.
Nowadays there are highly developed communication and information technologies are engaged in almost all human activities. The application of mobile devices has become common part of our daily lives, they are used for many kinds of purposes. The learning process of the young generation todays is particularly important to go with technologies that became the common knowledge of their daily live. This kind of process hope to creates a significant motivational aspect to the students and leads them to the successful learning process both in formal and informal area of learning. For this reasons e-learning is increasingly used in course, school or even in college degree. The online learning is an excellent opportunity to learn because it creates a suitable, enjoyable and entertaining learning environment, which creates a positive mood to the learner.
TOEFL is one of the standardized tests which have been applied by institutions and universities all around the world to test the English language proficiency of their students. It also becomes one of the graduation or even registration standard requirements in some universities for the students. However, many students encountered the difficulty in answering the TOEFL especially in section two which grant them with the Structure and Written Expression.
This paper is mainly focused on research examines how the use of mobile educational Duolingo application in teaching TOEFL preparation of structure and written expression. It also discusses the influences the learner’s understanding and motivation through process of learning in classroom and home environment. Duolingo application, which enables free online language learning.
Duolingo is explored, critically considered and evaluated in terms of appropriateness and quality of an educational application tool in the classroom. The goal of this paper first is to know the effectiveness of using Duolingo in teaching TOEFL preparation of structure and written expression on twelfth grade. Second is to know the learner’s respond through the application. Third is to examine the student’s motivational impact from the use of Duolingo. The research findings hope to assist foreign languages teachers and students who participate in foreignlanguages courses which combining the use mobile learning educational application in their process of learning.
The previous research conducted by Norshahila Ibrahim, Wan Fatimah Wan Ahmad, and A’fza Shafie about the Effectiveness Study on Multimedia Mobile Application for Children: Mfolktales Application, University of Sultan Idris, Tanjung Malim, Perak, Malaysia. Their research paper discusses the effectiveness testing for MFolktales Application to measure effectiveness level in students’ comprehension performance after using the MFolktales mobile application. It has a significant improve in Experimental group which mean that MFolktales application can be considered as effective.
There is a lot of Mobile Education application available on the internet. This study focus on examines the effectiveness of one of Mobile Assisted Language Learning: Duolingo Application in teaching TOEFL preparation of structure and written expression. More specifically, how the language learning process through the mobile Duolingo application is perceived by the students, and which attributes that motivate them to use it more in their process of language learning. In order to understand the effectiveness of the application, three research questions were presented.
1. What is the effective aspects of using Duolingo application in language learning process, especially in teaching TOEFL preparation of structure and written expression?
2. How are the students respond to Duolingo application through their process of language learning?
3. Is there motivational impact from use Duolingo mobile application language learning?
There are some researchers explain about the extent of the effectiveness of mobile technologies help and improve student language learning skills especially in learning structure text. Mobile technologies know that gives the students to have high usability on learning, be easy and simple to work with and it has been observed that mobile technology improve the satisfaction and desire to learn useful (Viberg & Gronlund, 2013). They also found that the disadvantages of mobile devise for learning are the small keypad and screen of mobile devices. It also very dependence on the use of internet.
2. Theoritical Background
The Common European Agenda defines six levels of proficiency in a language (Council of Europe, 2001). All level that explained includes following skills: reading and listening comprehension as receptive skills, writing and speaking as productive skills. There are three fields, which should be considered when describing mobile assisted language learning: (1) physical or touchable, (2) pedagogical, and (3) psychosocial areas (Bachore, 2015). Physical issues in this research related to screen size, storage capacity, processor speed, and battery life of the mobile phone. Pedagogical issues in this research refer to confirming the tasks are suited to the affordances of the mobile phone or devices used. Psychosocial issues in this research refer to the social environment. While laptops and desktop are used mainly for work or research purposes, it also can be the substitutes devices to the mobile devices. Mobile devices designed for personal or social purposes.
Any procedure in which a learner uses a computer and as a result improves the learner’s language skills, is counted as a Computer Assisted Language Learning (CALL) (Jarvis & Archileos, 2013). CALL has become established in the education of language in the early 1980s. At first, the domain was limited only to desktop computers with a few basic programs of mechanical software. Though, during the years, this area has developed to include applications, virtual learning environments, online blogs and vlogs (video blogs), online courses, etc. The advance of mobile devices, and their common use, their widespread availability has directed to the creation of a new area termed Mobile Assisted Language Learning (MALL). The term of MALL is used both in formal and informal of foreign languages learning (Chen, 2013). MALL is different from CALL as of its personal use, the function of portable devices that allow new ways of learning. MALL is a quite new research area, although the fact that people nowadays have been using personal mobile or portable devices (Gafni R. , 2009). The main gap in MALL to traditional process of language learning is the mobility and connectivity of the learner, in addition to the potentials of spatial and time shifts yielding improved learning opportunities.
Figure 1. Some screenshoot of Duolingo Mobile Application.
The application of Duolingo languages learning is a free, created by Luis von Ahn and Severin Hacker in November 2011 (Von Ahn, 2013). It offers the several languages learning for English speakers as well as for non-English speakers. It has versions for desktop and mobile. Duolingo is used to leverage both productive and perceptive skills of language such as reading, listening, writing, speaking, also translating and understanding. The most usual activities to accomplish these skills are by writing a word after perceiving a picture that represents it, translating a word and sentence from the native language to the foreign language or vice versa, writing a word or sentence after hearing it, matching pairs of word and also pronouncing a sentence. Duolingo includes several subjects, such as modal verbs, plurals, food, animals, which are further separated into several lessons that are adjusted to the learner’s level. Each subject matter in this application is represented by a node that changes its grey color which showing that the subject has not been started, to gold color, indicating the best or optimum control of the subject matter. The gold color can downgrade or turn back to other color if the algorithm of Duolingo resolves that some of the learners’ vocabulary has been forgotten. In this application the students are rewarded with points, according to their success, so they can compete against their friends to see who has the highest amount of points.
In general, the objectives in this paper is to know the effectiveness of mobile Duolingo mobile application in teaching TOEFL preparation of structure and written expression. Pre-test was conducted to evaluate the level of existing knowledge that respondents learnt before getting any treatment by using the application while post-test was conducted to identify significant difference between two different groups of participants after undergoing a specific particular treatment. In the other hands, the objective of this paper also to evaluate and to summarize the relationship between the samples that have been used in the testing.
3.2 Effectiveness Testing
The effectiveness testing was conducted to measure effectiveness level in students’comprehension performance after using the Duolingo mobile application. This testing used a quasi- experimental, pre-test/post-test control group design (T. D. Cook & D. T. Campbell, 1979). The pre-test and post-test testing was done to observe the student’s performance before and after using Duolingo application as compared to TOEFL preparation test in structure and written expression. Effectiveness testing used Quasi Experimental Design involved two groups of respondents from twelfth grade students of Madrasah Aliyah Atqia Bondowoso: (i) control group (CG), (ii) experimental group 1 (EG. The study population includes students who participated in researcher two class English language courses, one group (EG) consist of 20 students used the educational mobile application Duolingo on a mobile device, but the other group (CG) consist of 20 students during the same period just learning the subject matter with traditional language learning process. Table 1 shows the sample number involved in the effectiveness.
Figure 2. Students playing and challenge their friends in Duolingo.
Table 1 Sample Number of Effectiveness Testing
Number of students (N)
Control group (CG)
Experimental group (EG)
As mentioned, the twelfth grades students of MA ATQIA are divided into two groups, which are control group (CG) and experimental group (EG) The objective of conducting the pre-test of Quasi experiment is to evaluate the level of existing knowledge that respondents acquired before getting any treatment as the preparation for post-test. During pre-test evaluation, a TOEFL preparation test focused only in Structure and written expression in section 2 about 20 questions in 20 minutes devided into 10 questions of structure and 10 questions of written expression was delivered to all groups of students. A teacher was assigned to give a brief introduction about what should students do and face from the test. Then, all groups of students were asked to answer questions by themselves with limited of time. The answers and score from the test were then recorded.
After conducting the pre-test, the preparation for post-test was made. The objective of conducting post-test Quasi experiment is to identify significant difference between two different groups of participants after undergoing a particular treatment. Again, students in (CG) went through the same activity as in the pre-test. Students in (EG) were given a mobile application called Duolingo. In general, Duolingo application is used to leverage both four major skills in languages learning such as reading, listening, writing, speaking, it also teaches the students about translating and understanding skills. Duolingo includes several subjects, such as modal verbs, plurals, food, animals, which are further separated into several lessons that are adjusted to the learner’s level. Interactive social compete is provided in this application the students in the form of rewarded points. They can collect points according to their success, so they can compete against their friends to see who has the highest amount of points. The students in (EG) are given a limited time to use the Duolingo application for 2 weeks. In this group have experience using smart phone and tablets. As first motivation to the group (EG) to use Duolingo in their daily life of learning languages, the teacher was given a reward for student with highest score in Duolingo application. After two weeks passed, post-tests are given to all groups with the same format with pre-test but different question in same degree of difficulty. The result of the effectiveness testing will also be discussed in the next section.
Hypotheses evaluation was applied to summarize the relationship between the samples that have been used in the testing. For evaluating the hypotheses, one-way analysis of variance (ANOVA) had been applied (Coakes & Ong, 2011). The calculation of one-way ANOVA produced ? value. If the calculated ? is higher than 0.05, the null hypothesis is accepted. Meanwhile, if the calculated ? value is lesser than 0.05, the null hypothesis is rejected (Sekaran & Bougie).
There are two null hypotheses had been constructed:
1. Hypothesis (H01) – There is no significant difference between the pre-test mean scores of the students in control group 1 (CG) and experimental group 2 (EG)
2. Hypothesis (H02) – There is no significant difference between the post-test mean scores of the students in control group 1 (CG) and experimental group 2 (EG)
4. Result and Discussion
4.1 Pre-Test Result
As mentioned above, Quasi experiment had been designed to include pre-test and post-test on the selected samples. Table 2 shows the descriptive analysis of pre-test scores for CG and EG. Based on Table 2, students in EG obtained slightly better mean score as compared to the students in CG with small difference. Apart from pretest evaluation, hypothesis testing to evaluate the level of existing students knowledge has been performed. The variables were tested to examine whether the relationship that had been stated is, in fact, true. The null hyopthesis is:
Hypothesis (H01) – There is no significant difference between the pre-test mean scores of the students in control group (CG), and experimental group (EG)
Table.2 Descriptive analysis of pre-test scores.
In evaluating the hyphotesis (H01), one-way ANOVA had been performed to analyse the significant difference of all groups. According to (Sekaran & Bougie) and (Coakes & Ong, 2011), the p-value (Sig.) determine the acceptance or rejection of the hypothesis. In the case of the presented result in Table 3, the p value is 0.720, which is greater than ? value 0.05.
Therefore, the null hypothesis (H01) is accepted. Thus, it could be concluded that there is no significant difference between the pre-test mean scores of the students in CG and EG. The findings of this evaluation indicated that the same level of knowledge was acquired by the students in all three groups. This finding also fulfills the aim of conducting pre-test where students in all groups should have similar level of knowledge.
Table 3: One-way ANOVA result for pre-test
Sum of Squares
4.2 Post-Test Result
The purpose of conducting the post-test of Quasi experiment is to identify the significant difference between two different groups of students after undergoing a particular treatment. Table 4 shows the descriptive analysis of post-test scores for CG and EG. Based on Table 4, students in EG obtained better mean scores as compared to the students in CG. In making the conclusion on the difference between the effect of using MFolktales mobile application and traditional learning approach, hypothesis evaluation was considered.
Table.4 Descriptive analysis of pre-test scores.
Similar to pre-test evaluation, hypothesis testing had been conducted to compare students’ performance for all groups after getting the treatment. The null hypothesis is:
Hypothesis (H02) – There is no significant difference between the post-test mean scores of the students in control group (CG) and experimental group (EG).
In evaluating the hypothesis (H02), one-way ANOVA had been performed to analyse the significant difference between the groups. In Table 5, the p value is 0.000, which is lesser than ? value 0.05. Therefore, the null hypothesis (H02) is rejected. Thus, it could be concluded that there is a significant difference between the post-test means scores of the students in CG and EG. The findings of this evaluation indicated that different levels of knowledge were acquired by the students from all groups. One of the groups obtained the highest mean scores as compared to other groups.
Table 5 Descriptive analysis of post-test scores.
Sum of Squares
4.3 Students Respond and Motivation
At the end of the study the participants (EG) were asked to complete an exit survey with questions mostly related to their experience with Duolingo and their motivation through the application. Overall 20 people completed the exit survey.
Table . 6 Students respond and Motivation
Do you agree with the following statements?
Neither Disagree nor Agree
“Duolingo was easy to use”
“Duolingo was helpful in studying English”
“I enjoyed learning English with Duolingo”
“I am satisfied with Duolingo”
“I want to learn more languages with Duolingo”
If we combine the “Agree” and “Strongly Agree” answers we can say that 95% of group (EG) that use Duolingo mobile application consider it easy to use, 95% think that Duolingo application helps them in learning English language, 70% enjoy learning English with Duolingo and 80% of responder are satisfied with Duolingo. In the last statement the researcher stated the statement of “I want to learn more languages with Duolingo” in order to know the motivational of students in using Duolingo as their daily leaning language application, the answer of this statement is show that 90% of the student want to learn more languages using Duolingo.
In general, the educational Duolingo mobile application was developed based on the proposed conceptual model that has been validated to see the correlation between the elements included in the model. Based on the validated conceptual model, Duolingo mobile application has been successfully design and developed.
As a conclusion, this paper presented the effectiveness testing for Duolingo mobile application. This testing is conducted to observe the application’s performance among students in teaching TOEFL preparation of structure and writing expression before and after using the mobile application. 40 children were divided into two groups. Quasi experimental design was used to evaluate the pre-test and posttest results while ANOVA was used to evaluate the hypotheses. The results from the testing shows that: (i) there is no significant difference between the pre-test mean scores of the students in CG, EG, (ii) there is a significant difference between the mean scores for the students in CG, EG, (iii) their responds and motivations are 90% agree to use Duolingo to learn more languages in their daily learning process.
Learning a new language can bored to repeating task to remember vocabulary, exercise same structure again and again. Therefore, playing with the words and competing versus others friends can be very encouraging, thus, stimulating the learning process. Based on these findings of the effectiveness testing proves that Duolingo mobile application is an effective tool for students to improve their knowledge and score in TOEFL preparation test of structure and writing expression through the application where the students who utilised the application obtained better marks as compared to the students who did not.