质疑中国在 PISA 测试中名列第一

中国在 PISA 测试中名列第一——但这就是为什么它的测试成绩令人难以置信

https://www.washingtonpost.com/education/2019/12/04/china-is-no-pisa-heres-why-its-test-scores-are-hard- believe/
作者：瓦莱丽·施特劳斯 2019 年 12 月 4 日

在新发布的国际学生评估项目分数中，中国大陆是最大赢家，该项目每三年对数十个国家的 15 岁学生进行数学、阅读和科学测试。 2018 年，来自 79 个国家和学校系统的 60 万名学生参加了考试，中国的四个省份（在 PISA 中构成中国大陆）在所有三个科目中均排名第一。

但我们有充分的理由对中国大陆的分数持怀疑态度，这就是学生成绩、考试、教育政策和 K-12 学校改革专家 Tom Loveless 这篇文章的主题。

洛夫莱斯曾是六年级教师和哈佛大学政策教授，曾是治理研究高级研究员，也是华盛顿非营利性布鲁金斯学会布朗教育政策中心主任。 他撰写了 16 卷《布朗中心美国教育报告》，这是一份分析教育重要趋势的年度报告。

这并不是 Loveless 第一次对中国的 PISA 分数发表评论。 2013年，我曾写过一篇文章质疑上海在2012年PISA中的第一名。 在那次考试中，美国学生的表现并不高于 65 个国家和教育系统的平均水平（与往常一样）。

2013年底公布2012年成绩时，PISA的赞助者经济合作与发展组织表示，上海样本中使用的学校代表了该市15岁的人口。 当时在布鲁金斯学会任职的洛夫利斯和一些中国问题专家表示，流动儿童经常被排除在上海的学校之外，而上海比中国其他地区更富裕。 经合组织坚持这一结果。

顺便说一句，在 2018 年 PISA 成绩中，新加坡在所有三个科目上均排名第二。 美国学生在阅读方面排名第八，在科学方面排名第11，在数学方面排名第30，自几十年前PISA开始以来，这些分数没有发生显着变化。

作者：汤姆·洛夫莱斯

2018 年 PISA 成绩已出炉。 一般来说，根据过去的记录，各国的得分都在预期范围内。 除了一个。 B-S-J-Z 的得分令人震惊，B-S-J-Z 是参赛的中国四个省份的缩写：北京、上海、江苏和浙江。 在 77 个国际体系中，B-S-J-Z 在阅读、数学和科学这三个科目中均排名第一。

2015年至2018年，参加PISA的中国四个省份发生变化，浙江取代了广东。 2018 年组的分数显着高于 2015 年组（恰当地称为 B-S-J-G）。 事实上，差异如此之大，必然会引起人们的注意。

B-S-J-Z 的阅读成绩高出 61 分（494 比 555），数学高出 60 分（531 比 591），科学高出 72 分（518 比 590）。 像这样的差异有多罕见？ 为了回答这个问题，我检查了 2006 年至 2015 年的 PISA 数据。

对于每个三年的测试间隔，我计算了每个国家/地区在三项 PISA 测试中的变化，并将其转换为绝对值。 结果产生了 497 个观测值，平均值为 9.5 分，标准差为 8.6。

因此，一个国家得分的典型变化约为 10 分。 2015 年和 2018 年中国参与者之间的差异至少是这个数字的六倍。 这些差异也至少是所有间隔变化的标准差的七倍。 非常不寻常。

一个合理的假设是，改变参与 PISA 的省份，即使只是四个省份中的一个，也会影响测试成绩。 事实上，当我最初在 Twitter 上就此主题撰写帖子时，我忽略了这一变化，并将 2015-2018 年的分数差异视为参与省份相同。 我对这个错误表示歉意。 然而，我的错误确实凸显了一个更大的问题：中国的 PISA 分数应该受到怀疑。

为什么中国人口第一大省广东被剔除，而加入浙江？ 改变后分数飙升只是巧合吗？

由于户口对 15 岁人口的剔除效应，以及经合组织允许中国批准哪些省份可以进行测试，中国各省份过去的 PISA 分数受到了质疑（我和其他人）。 2009年，PISA测试在中国12个省份进行，其中包括几个农村地区，但仅公布了上海的分数。

三年后，英国广播公司报道称，“中国政府迄今不允许经合组织公布实际数据。” 截至目前，该数据尚未公布。

经合组织通过攻击批评者和闭门进行数据审查来回应过去的批评。 中国各省份的 PISA 分数笼罩在阴云之上。 我敦促经合组织尽快公布对 2018 年数据进行的质量检查结果，以及 2015 年和 2018 年参与者按省分列的分数。

国际评估的可信度取决于测试程序的透明度，包括如何选择参与者以及报告测试结果的规则。 经合组织不公开其在中国进行的评估，可能会损害 PISA 的可信度。

瓦莱丽·施特劳斯 (Valerie Strauss) 是一位教育作家，运营着“答卷”博客。 1987年，她加入《华盛顿邮报》，担任亚洲地区的助理外国编辑，此前曾在路透社担任国家安全编辑和国会山军事/外交事务记者。 她此前还曾在 UPI 和《洛杉矶时报》工作过。

China is No. 1 on PISA — but here’s why its test scores are hard to believe
https://www.washingtonpost.com/education/2019/12/04/china-is-no-pisa-heres-why-its-test-scores-are-hard-believe/

By Valerie Strauss December 4, 2019 

Mainland China was the big winner in the newly released scores on the Program for International Student Assessment, which tests 15-year-old students in dozens of countries in math, reading and science every three years. With 600,000 students from 79 countries and school systems taking the exam in 2018, four provinces in China — which for PISA constitutes mainland China — were collectively ranked No. 1 in all three subjects.

But there is good reason to view the scores from mainland China with skepticism, and that’s the subject of this post by Tom Loveless, an expert on student achievement, testing, education policy and K-12 school reform.

A former sixth-grade teacher and Harvard policy professor, Loveless was a senior fellow in governance studies and director of the Brown Center on Education Policy at the Washington-based nonprofit Brookings Institution. He wrote 16 volumes of “The Brown Center Report on American Education,” an annual report analyzing important trends in education.

This isn’t the first time Loveless has commented on PISA scores from China. In 2013, I wrote a post questioning the No. 1 ranking of Shanghai in the 2012 PISA. In that test administration, U.S. students performed no better than average among 65 countries and education systems (like usual).

When the 2012 scores were released in late 2013, the Organization for Economic Cooperation and Development, which sponsors PISA, said the schools that were used in the Shanghai sample represent the city’s 15-year-old population. Loveless, then at the Brookings Institution, and some China experts said that migrant children were routinely excluded from schools in Shanghai, which is wealthier than the rest of China. The OECD has stood by the results.

Incidentally, in the 2018 PISA results, Singapore was second in all three subjects. U.S. students ranked eighth in reading, 11th in science and 30th in math, with scores that have not significantly changed since PISA began a few decades ago.

By Tom Loveless

The 2018 PISA results are out. Generally, countries scored within an expected range given their past records. Except one. The scores are astonishing for B-S-J-Z, an acronym for the four Chinese provinces that participated: Beijing, Shanghai, Jiangsu and Zhejiang. Out of 77 international systems, B-S-J-Z scored No. 1 in all three subjects: reading, math, and science.

The four Chinese provinces taking PISA changed from 2015 to 2018, with Zhejiang taking the place of Guangdong. The 2018 group’s scores are dramatically higher than those of the 2015 group (which appropriately is called B-S-J-G). In fact, the differences are so large that they are bound to raise eyebrows.

B-S-J-Z’s scores are 61 scale score points higher (494 versus 555) in reading, 60 points higher (531 versus 591) in math, and a whopping 72 points higher (518 versus 590) in science. How uncommon are differences like these? To answer that question, I examined PISA data from 2006-2015.

For each three-year test interval, I computed the changes for each country on the three PISA tests and converted them to absolute values. That produced 497 observations, with a mean of 9.5 points and standard deviation of 8.6.

So the typical change in a nation’s scores is about 10 points. The differences between the 2015 and 2018 Chinese participants are at least six times that amount. The differences are also at least seven times the standard deviation of all interval changes. Highly unusual.

A reasonable hypothesis is that changing the provinces participating in PISA, even if it was just one out of a group of four, influenced the test scores. Indeed, when I originally composed a thread for Twitter on this topic, I overlooked the change and treated the 2015-2018 score differences as if the participating provinces were the same. I apologize for the error. My mistake does underscore, however, the larger issue: that PISA scores from China should be viewed skeptically.

Why was Guangdong, China’s most populous province, dropped from participating and Zhejiang added? Is it only a coincidence that scores soared after the change?

The past PISA scores of Chinese provinces have been called into question (by me and others) because of the culling effect of hukou on the population of 15-year-olds — and for the OECD allowing China to approve which provinces can be tested. In 2009, PISA tests were administered in 12 Chinese provinces, including several rural areas, but only scores from Shanghai were released.

Three years later, the BBC reported, “The Chinese government has so far not allowed the OECD to publish the actual data.” To this day, the data have not been released.

The OECD responded to past criticism by attacking critics and conducting data reviews behind closed doors. A cloud hangs over PISA scores from Chinese provinces. I urge the OECD to release, as soon as possible, the results of any quality checks of 2018 data that have been conducted, along with scores, disaggregated by province, from both the 2015 and 2018 participants.

The credibility of international assessments rests on the transparency of test procedures, including how participants are selected and the rules for reporting test results. The OECD risks undermining the credibility of PISA by not being open on its conduct of the assessment in China.

By Valerie Strauss
Valerie Strauss is an education writer who runs the Answer Sheet blog. She came to The Washington Post as an assistant foreign editor for Asia in 1987 and weekend foreign desk editor after working for Reuters as national security editor and a military/foreign affairs reporter on Capitol Hill. She also previously worked at UPI and the L.A. Times.