The initial policy defined a quality course as being a model course with “first-class teaching teams, first-class teaching content, first-class teaching methods, first-class course materials, first class teaching and first-class teaching management” (MoE 2003). These “six first-classes” are mentioned in almost all literature on the project, and were also repeated by most of my interview participants.
The specific aspects listed in the first policy paper specified that the courses should be taught by top teachers, who have long experience and influence in their fields. This was partially to address the desire, also expressed in the 2001 policy paper, to attract full professors at top universities into spending time teaching core undergraduate courses. The content of the course should be up-to-date and reflect the cutting edge of the academic field. The course should be taught with advanced and innovative teaching methods.
Teaching materials should either be award-winning nationally approved materials, materials developed by the teachers themselves, or material from abroad. More experiential and practical learning was encouraged, for example experiments and internships. Finally, the university was to set up effective incentive and evaluation mechanisms for rewarding participation in the project, and effectively selecting and improving appropriate courses.
In 2004, a more detailed policy note appeared, which provided further information about the requirements for courses. Both regular undergraduate courses, and courses from vocational schools, that had already been taught for at least three years, were accepted. The head of the teaching team should be a full professor. To evaluate that person’s teaching capability, a minimum of one 50 minute video recording had to be made available on the website, but universities were encouraged to put up recordings of all lectures. In addition, the syllabus, lesson plans, exercises, laboratory guides, and reference material also had to be made available online (MoE 2004).
In the previous chapter, a number of course evaluation systems that preceded the Top Level Courses project were discussed. One of these, at Tsinghua University, actually gave its name to the Top Level Courses project (Han Xibin and Ju Feng, personal communication). According to Mr. B0, most of the categories listed, such as teaching teams, teaching contents, teaching methods, teaching principles, and the academic level of teachers, were similar to earlier evaluation methods. The most important innovation was the focus on educational technology, ie. developing internet resources and then sharing them openly.
Development of selection criteria
As the Ministry of Education has gained experience, there have been continuous changes from year to year of the evaluation criteria for Top Level Courses. These range from minor variations in the rating rubrics used, to more large-scale changes in priorities. The actual rubric used to evaluate courses has five comprehensive indicators: teaching teams, teaching content, teaching conditions, teaching methods and practice, and instructional outcome. These five are subdivided into 15 secondary indicators (one was removed in 2008) as listed by Wang Peng (2008):
teaching team, including responsible person and main lecturer
the composition and quality of teaching team
educational reform and educational research
the course content
the organization and planning of the course content
the practical components (removed in 2008)
teaching materials and other resources
conditions for practical components
online teaching environment
instructional outcomes – evaluation by peers, by university supervisor,
evaluation of recorded materials
When it comes to large scale changes in focus, Professor B3 commented that initially the online resources were not that important, because the Internet was not very developed at the time that the Top Level Courses Project started. As the web grew in importance and sophistication this changed, and in 2004 and 2005, requirements about the functionality and design of the course website increased.
Likewise, Mr. B0 stated that the provinces and the Ministry of Education have focused increasingly on the sharing of resources and providing a service to others. This is a marked change from the first few years, when the application process was the main focus. Initially there were not many courses, and attracting a number of high-quality courses was the focus. Later, as the number of courses increased, it became important to offer better access to these resources, which led to the creation of the Jingpinke.com portal (see the later section on this).
Wang Peng (2008) who studied how the evaluation criteria had changed from 2003-2008, also identified a number of trends. He reaffirmed the added focus on instructional design, pedagogy, use of technology, impact and sharing, and the building of local teams. When evaluating the changes, it is often more interesting to look at the explanations given for the different criteria, than the weight assigned. For example, the explanation for the criteria “evaluation of recorded materials” changed from specifying “dignified bearing, clear sound, full of teaching enthusiasm, lively class atmosphere, high student participation rate, and effective student-teacher interaction” in 2003 to asking for "influential teaching which attracts students attention, enlightens and inspires thinking, association and creativity” in 2004 (ibid. 39).
As new categories of courses were added, new rubrics also had to be developed. For example, the evaluation of online courses put more focus on course resources for self-learners, the teaching and learning process, and learning support design. Zeng, Zeng and Fan (2007) believe this to be natural, and mention high student numbers, and a focus on adult learners that need practical and vocationally oriented training as key features distinguishing distance education from teaching programs in traditional universities.