Until the recent Research from the Organization for Economic Cooperation and Development (OECD) showed up, the United States education system was unquestionably the best in the world. Not only was the investment of highest order, the overall results and international standing was highly rated as well.
For years OECD has conducted the Program for International Student Assessment (PISA) to set a benchmark for incoming students in this country. The basis for this research was raw data collected from over 65 countries which comprise nearly 90% of global economy, showing a direct correlation between excellence in Mathematics and Science to financial growth and economic prosperity.
So what exactly is PISA, and how does it work?
In a nutshell, PISA is a critical thinking examination in math, science, and reading for 15 year olds on a scale of 1000. The tests are repeated triennially. It is structured to test applied knowledge in solving real-world problems, instead of memorization skills. The test is administered to students from a wide variety of socio-economic backgrounds, regardless of their academic achievements or grades. The United States, contrary to popular belief has a fairly high ratio of students from financially advantaged backgrounds, already giving them a bit of head-start against students from other OECD nations.
Reading tests include both short answers, multiple-choice questions and full answers adding up to a two-hour assessment time.
As a broader representation of literacy of a nation and their economic impact, PISA 2015 is supposed to include a questionnaire for teachers as well. Germany and France, which reacted sharply to their average performance in 2000 PISA had a broad overhaul of their schooling process, but the results in 2003, 2006 or 2009 didn’t quite fly thick and fast, which points to a gradual growth as the process.
Data scaling
Given the wide spectrum of student backgrounds and test booklets, the raw data from scores are scaled using Rasch model of item response theory (IRT) to arrive at meaningful comparison statistics. A Gaussian prior probability distribution of competences is used to include students who have solved all or none (scores 0 and 1000) on the scale.
Evaluation
The primary testers are all school-goers. The standard or board of education doesn’t matter, but home-schoolers are not included. This is done to gauge if respective participating nations are doing enough to groom their wards for 21st century challenges in terms of imparting holistic education.
There has been some criticism of the pattern though, given that only one African nation (Tunisia) was represented. Other countries like Germany implemented additional level of language competence tests like PISA-E (E=Ergänzung=complement). China doesn’t participate as a single country, splitting up the system into provinces like Macau, Shanghai and Hong Kong instead.
Others have wondered if the ‘predicted conditional’ model is capable of evaluating relative proficiency in an effective manner, given that difficulty level for students of same age group across the world may not be the same. Language experts have also wondered if the same reading questions in Chinese for Chinese students can be assumed to be of similar difficulty as in Danish for a student from Denmark. The relative difficulty of an expression in Chinese versus Danish or a typographical versus a logographical language can at best be regarded as an approximation of reality.
Another aspect which pundits have cited is the measurement of overall educational experience rather than test results as a marker of whether educational system is doing the job. South Korea, Shanghai and Macau have ranked near the top of PISA scores, but the students are generally ‘unhappy’ at school.
By contrast UK and the US, have respondents who agree that they are “happy” at school, despite average PISA ratings.
Results
Asian countries have been strong winners at PISA tests. Shanghai (China), Hong Kong (China), Japan, Singapore, Taiwan and South Korea have consistently finished within the top five – Shanghai being the top educational system per OECD for the entire decade.
