Improve Test Scores? Forget OLPC, Just Teach to the Test!

   
   
   
   
   

Hi, my name is Bryan Berry. I live in Nepal and I volunteer most of my free time to OLPC Nepal. I am not an educator by profession yet I was not surprised by the examples cited in the recent article in the New York Times "Seeing No Progress, Some Schools Drop Laptops." I was interested to read Amir Kaeger's response to the article.

new york times olpc
The shirt says it all

As the article title proclaims, many American schools have adopted and since dropped laptops in the classroom after failing to find any improvement in test scores. I believe that the laptops did not produce measurable results primarily for the following reason: The best way to raise test scores is teach to the test. There are also three additional lesser reasons:

  1. Most educational software today is sub-par
  2. American schools already so many resources that laptops only offer marginally more access to information
  3. Teachers need familiar material when they first begin to incorporate laptops into the classroom
Teach to the Test!
The best way for the schools to raise test scores is to gear all classwork to the tests and for parents to pressure their kids to study hard. That's how it works throughout Asia. I believe that testing has an important place in education but it is only part of education. Furthermore, years of intense exam cramming are not a recipe for producing adults with
  1. creativity
  2. strong problem-solving skills and
  3. a love of learning.
However, one of the nations that disproportionately produces singular creative talents, Israel, does not focus heavily on examinations. When I lived in Tel Aviv, I recall many delegations of East Asian educators seeking to find out the secret to the Israel's success in creating great scientists, artists, and entrepreneurs. Israeli culture prizes creative problem-solving and a love of learning. That isn't something you can easily distill into a multiple-choice exam.

After Israel, I lived in Hong Kong. Students in Hong Kong schools consistently rank near the top in international test scores for math. Hong Kong schools emphasize rigorous testing from first grade on. This system also produces people that often don't enjoy learning. I can only provide anecdotal evidence to support this. I observed that a much lower portion of Hong Kongers read in their free time and the attitudes of the Hong Kongers I knew. Further, Hong Kong companies are known for their ability to maximizing profit, be it in manufacturing operations or real estate development but not for developing new and innovative products.

In a 2000 OECD education survey, Hong Kong scored 1st worldwide in Mathematical Literacy while Israel came in at #33. Whoah! If Israelis have such low test scores, why did Google put a large research unit there? Why does Intel design all its notebook processors there? Golly, Israel also puts out a ton of entrepreneurs, musicians, and writers. I don't mean to knock Hong Kong, I just want to draw attention to the fact that Israelis excel in technical fields despite their low test scores.


Creativity device for students

But I digress, I am trying to explain why I believe OLPC will improve education despite a lack of hard evidence (higher test scores). The only evidence that we can get in the short term are higher test scores. The real proof will be if the OLPC generation grows up to be an incredibly creative and successful group of people. The delegations of Korean teachers were looking for an educational silver bullet for creativity and innovation during their trips to Israel but they never found it.

Amir's questions whether that "if" is too much of a risk. I argue that it isn't. Kids in developing countries will get access to at least a thousand times more information than they do currently and the auto-didactic/genius kids will teach themselves amazing things. If OLPC just does those two things but fails to help the average kid "learn learning," it will still be a huge success. I do think a lot of those so-called average kids will find a program like TamTam or eToys that really inspires them.

The week prior to the NYT article on laptops, Thomas Friedman published an article "China Needs an Einstein. So do we." The article examines the connection between political freedom and creativity. I argue that OLPC is the first of two steps that will unleash an immense amount of creativity. Which leads me to my second point . . .

Most Educational Software is Sub-Par
When teachers receive a new textbook, it is a completely static document. They can't edit it, rearrange it, or provide feedback to the authors on how effective it is. Proprietary educational software is the same. Educational consultants design it, programmers write it, and X months later it arrives at schools. I have been really, really impressed with Squeak, pygames, and TamTam. I have also tested some proprietary packages that are very expensive and not impressive.

We need to democratize the process of educational content development. We need an online repository of open source learning content that allowed teachers and students to rate the materials and provide feedback. Teachers and students could also modify the materials and upload them as a separate version. Their would be official versions and unofficial versions just like Ubuntu has official and unofficial repositories of software. This explosion of open learning content is the second step after OLPC that will release an amazing amount of creativity and quite possibly revolutionize education.

American Schools Already Have Tons of Stuff
The laptops deployed to the American schools in the NYT article were not the first computers the students had worked with. A large majority of them probably have computers at home and Internet access. This means they already have access to virtually limitless information in addition to the large number of textbooks -- large compared to developing countries -- their school issued them. American kids are spoiled with information access compared poor countries and they already have access to computers. This dulls the impact of laptops on education when it comes to test scores.

Nepal olpc art
Limbu script on OLPC XO

Give the Teachers Something They are familiar With
The leader of OLPC Nepal's education group, Christine Stone, has instructed us to develop learning activities for Nepali classes that are similar to the ones Nepali teachers currently use. The teachers will not embrace something radically different from what they currently do.

Similarly, the NYT article cites a math teacher that grows frustrated with the math package on the laptops and reverts to having his kids use graphing calculators ($80-120 on Amazon.com). This is quite telling since a graphing calculator is just a special purpose PDA. The math teacher just happened to be more comfortable with the graphing calculator's interface than the laptops'.

We intend to give the kids super cool learning activities and games that they can do in their own time. Those activities will be truly "constructivist" activities.

By the way, OLPC Nepal is looking for volunteer engineers. We will be working very intensely this summer and fall to implement as many learning activities as we can. Volunteers will work very closely with our Education Director Christine Stone, Nepali teachers, and a lot of kids. You can read the full notice on our blog.

Related Entries

15 Comments

"We need to democratize the process of educational content development. We need an online repository of open source learning content that allowed teachers and students to rate the materials and provide feedback. Teachers and students could also modify the materials and upload them as a separate version. Their would be official versions and unofficial versions just like Ubuntu has official and unofficial repositories of software. This explosion of open learning content is the second step after OLPC that will release an amazing amount of creativity and quite possibly revolutionize education."


A wikipedia of sorts for the classroom? No, thanks.

BTW, even if your idea (above) had merit, it wouldn't need a crippled computer like the OLPC machine.

I think we've all stopped worrying about your comments Troy.
You obviously have a vested interest in seeing initiatives like the OLPC fail. Keep up the rhetoric though, its entertaining.

Meanwhile, education will go through a revolution - most likely with OLPC Nepal leading the way with education content instead of hot air.

Bryan,
thank you for your inspiring article. This is the first coherent and comprehensible description of an implementation plan concerning OLPC I heard of. Is only OLPC Nepal following this plan? Or is this OLPC's plan everywhere?

In the short term you intend to have learning content close to the existing one in order to have the teachers feel comfortable with it. From then on the content will further evolve including the feedback of the teachers and students. That way both teachers and students will be involved in that development and "grow" with it. In such a way over some time it may gradually evolve to a revolutionary level but by an evolutionary process instead of a revolutionary one causing refusal.

Since you're looking for multimedia and interactive content a wiki consisting mainly of textual content will not do. It will have to be python programs in the sugar environment. Python is easy enough to be quickly learned by many students and teachers. Given some time they can actively participate in the development.

They could mutually develop such content using public version control servers containing multiple parallel branches of software versions as they are generally used by most open-source projects today. And they could collect the software packages in a repository for easy download like debian, ubuntu or gentoo do it using different maturity levels like stable, testing, unstable.
That plan sounds feasible.

There remains the old problem: how to prove success. When you imitate Israel's model you will have to wait for one generation of students to show the increasing number of extraordinary individuals. And you would accept in the meantime that the test scores might drop or at least do not significantly increase. This is risky. The indicators of success and failure could not be distinguished for many years. If the proof failed in the end, a huge investment of money and work over many years would have been wasted. How are you going to deal with this risk?

Your plan looks intriguing but its risks are high. How could the risk be limited?

Excellent article. I am glad that someone is standing up for creatity as a key goal for education.

In the case of olpc, however, I think we can have both creativity and higher test scores. That is because, as I have mentioned before, the priority group for olpc is students in areas where there is a gross shortage of teachers (see Negroponte transcripts at olpctalks.com). In these areas schools have split sessions and a typical student gets only 2 1/2 hour instruction a day, if that.

Olpc hardware and software is designed to overcome this problem. Instead of hiring more teachers, which developing countries can't afford to do, the idea is for students to become creative, self-directed learners.

If this idea works, then the students will become much better at reading, writing and arithmetic, so test scores will go up, but they will become more creative at the same time.

Bryan, thanks for the excellent article!

After reading through your comments I'm now convinced that OLPC Nepal leads in really understanding the "educational" part of the OLPC equation.

Good luck for your work, I'll keep an close eye on it via your blog... :)

Troy: The XO is not a crippled machine. It has some very impressive technical features and it impresses me the more I work with it. The 2 km wi-fi range is far from crippled.

Roland:You are correct, a wiki would not be sufficient for the open content repository. For Nepal, the risk is actually quite low, Nepali kids are quite motivated and the XO would grant them access to the information they are hungering for. Math and English are seen as the key to success in Nepal even among the poorest people. Unfortunately, few villages have an English teacher and most math instruction is taught by the same teacher who teachers every other subject. Also, it is often recited rather than taught. Nepali kids will seize upon the XO to learn those subjects.

Eduardo: Thanks for support. I do agree that OLPC could raise test scores in developing countries. I can't speak of Latin America having never been there, but Asian students are very motivated and hard working. They will take advantage of the learning features in the XO. Unfortunately, the same cannot be said for the average American high schooler.

I forgot to write this in the article, "You can give American kids more tools but that won't make them study." More tools don't compensate for lack of effort.

I have traveled quite widely in Asia and studies are taken very seriously. The creative element of education, as I note in my article, has very much been missing. OLPC can help bridge that gap and compensate for the lack of resources.

In the case of Nepal, even if the Ministry of Education had funds to double the number of teachers it would take many years to find and convince those teachers to move to the rural areas where they are most needed.

For countries in Asia, I think the risk of OLPC is actually quite low. Kids are eager to learn but resources are scarce.

correction: The caption for the picture of the XO should read "Limbu script on OLPC XO" not "Nepali script on the XO." Limbu is an indigenous language in Nepal.

Bryan, allow me to congratulate you on a very compelling article. Besides all that we could talk about (Hong Kong being a trading post makes for creativity to be seen as less critical as profit-making, for instance), the main ideas are worthy of a long discussion, as the one it's starting here.

The creative side of education is indeed critical. One could argue that creativity finds better expression at the second half of school education (secondary or high school) and that a fundamental set of skills has to be developed at primary or elementary school; also, that schools have to work towards socialization and "grounding" of social practices, and that a computer may not be the best way to do this.

But this is precisely the issue here: to argue. The issue about motivation, for instance, maybe quite important here in Latin America, where the educational ethics of the students is not that great. On the other hand, the idea that having a lot of exposure to computers makes you less prone to focus on education when you receive one is not something I'd second; it may precisely the opposite, especially if you consider that many kids in Latin America, even in the most remote regions, are somewhat aware of what you can do and find with a computer.

But again, it's a great post. Thanks for writing it.

Eduardo V.: I didn't mean to imply that ready access to computers have made American kids less prone to education. I strongly imply that the work ethic of American students isn't that strong and merely adding laptops won't make them work harder. Also, Americans kids's heavy exposure to regular computers blunts the possible impacts of integrating laptops into education.

I think computers could be used for "grounding" of social practices but we need to develop the software and teachers to do that.

I really don't know much about the education system in Latin America or work ethic of latin American students or that of Africa. I do feel I have a pretty good understanding of the system in South and East Asia. Eduardo V., I welcome your thoughts in this and other regards.

Brian,

Now we are scrutinizing educational efforts and ideas from Nepal. I always love to read about the Nepal project. Is there something similar from the Nigerian and Brazilian deployments?

Isn't this exactly what the OLPC is all about, allowing disadvantaged and "isolated" communities to be part of the world?

We try to understand the great ideas and needs of teachers and children in Nepal. These teachers and children get to hear about our ideas and needs. So if you would need some adaptation or script for the Nepalese participants, you could ask, and someone might just be able to post it to you.

And only a few XOs have been delivered yet!

Btw, Brian, I have a more "professional" question.

Socio-linguists would expect that having access to writing scripts and speech technology for minority languages on a computer (ie, elite technology) will increase the standing or status of that language. I am not sure about the status of the local languages. But I would like to ask you whether you see a change in attitude towards local (minority) languages present on the XO?

Winter

Topic A) Creativity or Score Oriented Education?
************************************************
For Asian students the high work ethics and - in some places - the low quality of present education facilities is going to reduce or even eliminate the risk of decreasing scores when introducing laptops and a creativity focused education method. Apart from providing a generally better education most Asian students need more development of their creative skills since in the current, very score oriented education systems creativity suffers. So Bryan's description of Nepal's OLPC implementation plan might work in Nepal and might become a role model for whole Asia.

Now how about students in the Americas and Africa? There the work ethics seems not as high as in Asia. In wealthy countries of those continents (e.g.USA) even getting laptops will not boost motivation that much. How can these countries reduce the risk of laptop projects reducing or not increasing scores? If the same creativity focused content would be used as planned for Asia this risk might be very high.
If you focus on scores instead then you might end up with underdeveloped creativity as well.

a) Can there be a dual strategy? Score AND creativity oriented education in parallel?
b) Or could there be a motivation oriented education to let kids feel the joy and satisfaction of learning and to reawaken their natural curiosity?
c) What would have to be different among those education contents to account for the different situations in Asia, Africa, Americas, rich, poor?

I would like to caution against mixing up work-ethics of students with expected economical benefit of investing in education.

To a large extend, (ambitious) young, teenage, people and their family will try to invest the children's time in activities that will benefit their perceived future prospects.

In Asia, getting high scores in school is a golden road into a good job (and mariage) and a bright future.

In the US, having very high grades in high-school is not perceived as distinctive. Getting into a ivy leage college/university is. The old, "It is not what you know that is important, it is who you know". We can see that these expectations affect how US children spend their time.

In Israel, creativity is clearly rewarded, so it will pay off for a child to "go for" creativity instead of high grades. In many Asian societies, the reverse is true, so it wouldn't pay off for a child to invest in creativity. In some really disadvantaged communities in the world, it doesn't even matter whether or not a child is educated. Sometimes, the child doesn't even expect to live long enough in freedom to actually benefit from any education.

Children and their families react to these expectations, and this affects their "work ethos".

It is sad, but being lazy can be a very rational strategy for a child.

Rob

Topic B) Elitarian versus Egalitarian Education
**********************************************
Elitarian System:
Bryan's description showed that a rather creativity oriented education in Israel produced higher numbers of extraordinary performers but it showed also lower average scores. I.e. in this system the elite students are fostered better than the average or weak students. Therefore it is also an elite oriented education. Its goal is to best develop the elite at the cost of the average and weak. Then those countries will have e.g more successful entrepreneurs but the average people are not educated enough to take advanced and well paid jobs in those enterprises. So the social gap will increase.

Egalitarian System:
In Europe it is the other way around where the focus is more on the average student. In many cases Europe's education systems invest more effort not leave any weak student behind than to foster the highly talented ones. Therefore this is an egalitarian system. It best develops the average at the cost of the elite. Then you have masses of well educated average people that could do more advanced jobs but you lack the elite to create enough of those jobs. So you end up with well educated people out of work or taking jobs below their skills.

Utopian System:
Why not to give each student the best development of his individual skills and talents. To achieve this you had to individualize the education system to the individual needs of every student or at least to the needs of a larger number of different kinds and levels of skills. This is still the unrealized utopia which no school system could perform so far. Will it be possible one day?

Which system is better?

I think the beauty of OLPC is that teachers can offer kids and individualized education so we can combine the best of the egalitarian and elitist systems. Teachers can spend more time helping those kids that need it most while the more talented kids can race along on their own or in concert with some of their peers.

Also, some students may be average in some subjects but excel in others. Joe Average that struggles with reading and writing may go crazy w/ a music application and do something really great. Before OLPC such a kid might not find a particular subject or application that excites him or where he can excel.

I think you can't compare the OLPC model in the US and in the third world.

Close