The opportunity to reconsider allows you to refine your programming skills and develop them as a personal strength.
How is an automated grading system different from the mark-sensing system of the Common University Test?
In the Common University Test (formerly the National Center Test), for example, a part of an English text may have blanks, and students must select the correct word from multiple-choice options to fill in the blanks. Since programming is also based on text, one way of structuring the questions is to provide part of a correct program with blanks and have students choose the correct syntax and words from given options.
This is like the Common University Test. A key aspect of programming is that students can test their own code to see if it works. If there’s a spelling mistake, the program won’t run, and even if the syntax is correct, it may not work as intended. Testing the code students write is a unique feature and will be extremely important for educational purposes.
At first, teachers might give multiple-choice options, but eventually, students should be able to figure out the right answers on their own by going back to the lecture notes. They should start by filling in single words, then move on to filling in short paragraphs, and finally, they should be able to write full programs from the beginning.
An automated grading system makes it easy to evaluate programming problems as they change from simple word blanks to free-response paragraph-based coding exercises.
In the programming world, there has always been a way to check if it works correctly, right?
Yes, that’s true. The automated grading system essentially runs test commands to verify if the submitted code functions as expected.The main idea of the system is to make the tests easy to understand and use. It gives a score based on which tests the program passes or fails. If a student’s code passes certain tests, they receive points. If it fails critical tests, they may get zero points.
The system enables efficient grading while also allowing students to check their own answers under certain settings. Without this system, teachers would have to grade the programs manually, which is very time-consuming. Automation makes this much easier.
What educational benefits can we expect from this system?
Our organization, the ‘Center for DS&AI Education,’ offers courses to all students across all disciplines, including those without a background in computer science. Some students struggle with programming, but they still need to learn data science and AI because of the current academic and industry demands. We need to think about how these students can learn programming efficiently. For students who find programming challenging, even with peer discussions, completing assignments on time can be difficult. It is unavoidable because some people are not good at programming, but it is important to provide extra support to those who need it.
Even when students are studying on their own, it is helpful to have access to hints and help. Students can use buttons or commands to check their code to see if it is correct, and to notice if it is wrong. Usually, they wouldn’t know the results until they submit their code to the teacher and get it back a week or two or six months later. But if they can immediately identify mistakes, they have a chance to fix their code before submission. In education, rethinking is fundamental. I think that being able to improve their programming skills by rethinking is a highly effective way of learning.
Creating a system is easy, but maintenance requires tremendous effort.
I understand that UC Berkeley pioneered automated grading systems for programming in Python.
Automated grading systems aren’t new. They use test command structures that already exist. Some teachers on campus are actually creating these systems. However, creating the system alone isn’t enough. It’s also important to maintain it continuously. Programming languages change over time, so syntax rules need to be updated regularly. Building a system is possible, but maintaining it is very labor-intensive. UC Berkeley not only provides the system, but they also maintain and support it, which is a big advantage.
They offer their program as open-source software on the Internet, which makes it freely accessible to anyone. They also focus on data science education. In this regard, we are engaged in inter-university exchanges, including the use of an automatic grading system.
Can you tell us when and how the automated scoring system was introduced?
At the former Tokyo Institute of Technology, we explored various ways to introduce the system in the first half of 2023. By October, we started using it mostly in my courses, with the help of other faculty members. In 2024, we plan to expand its use further. Fortunately, DS&AI is now attracting a great deal of attention in society. In 2022, ChatGPT and a generative AI for images were released, and by 2023, AI was a common topic on TV talk shows. As a result, more students enrolled in our DS & AI courses. To meet this growing demand, we introduced an automated grading system.
I used to manage servers at a research institute, so I could figure out how much computing resources would be required to implement a new system and whether it would be sustainable. Through collaboration with UC Berkeley, we found that it is sufficient to have a machine with a certain level of performance with a GPU for teachers to grade, and that there are no costs for student use.
Developing teachers and considering incorporating them into the resources of other universities and technical colleges.
Since the system is open source and does not require infrastructure investment, could it be expanded to other universities?
Yes, cost is a critical factor. We have also tested the feasibility of using commercial cloud services for students working from home to ensure accessibility for everyone. The social focus on DS&AI is not limited to our universities, and the number of students taking DS&AI-related classes is increasing at all universities. By making our materials available online, we aim to contribute to education throughout Japan.
We have two main initiatives. One is providing educational materials, including the introduction and use of the automated grading system I mentioned earlier, as well as support for introducing the system through Zoom interviews and other means. The other is related to teaching assistants (TAs). We have initiated a special program on the training of TAs in DS&AI education. Regardless of whether they pursue a career in academia or industry, the ability to teach others will become increasingly important. By training one person to teach another, we aim to spread DS & AI knowledge exponentially. In addition, we are considering incorporating the personnel we have developed into the educational resources of other universities and technical colleges, and we have already started negotiations in some cases.
How do you plan to introduce the system to the former Tokyo Medical and Dental University?
It is not so different from other universities, but ensuring the quality of education is very important. As enrollment increases, the workload of faculty becomes heavier. We hope to strike a balance between reducing faculty workload and maintaining high educational standards through automated grading. We welcome feedback from faculty as they gradually adopt the system.
If we really care about Japan's future, education is the only answer.
Finally, what motivated you to get involved in DS & AI education?
I grew up hearing phrases like the “Lost Decade” – which later became the “Lost 20 Years” and then the “Lost 30 Years”. We have grown up with the term “lost years” which refers to Japan’s prolonged period of economic stagnation and slow growth following the asset bubble collapse in the early 1990s. Living with that terminology felt like a constant negation of my generation. It is frustrating, to say the least. Now that I could act as an adult, I am seriously considering how we can prevent the phrase “Lost 40 Years. I believe that education is the only way. Strengthening education is the key to Japan’s development.
Fortunately, there was an opportunity to be involved in DS&AI education at a top-level university, which is also my alma mater. I applied for the position, thinking that this is precisely what I could do to contribute significantly to Japan in the future. It may sound formal, but I really mean it.
Off-record talk
I have not been conscious of education since I was a student. However, I have had a strong interest in social issues since I was a child, and naturally I would think about what I would do in response to headlines like those in the newspaper.
I come from a physics background, not computer science, but the lab I belonged to was filled with like-minded individuals who shared a similar outlook. Of course, we do research, but we also discuss various social issues outside of research. And the conclusion we always came to was that education is important after all. In that sense, I guess I was aware of the importance of education.