ASys is a semi-automated assessment system for Java exercises and exams. ASys helps in assessing Java exercises/exams in an exhaustive, assisted, and fair way. The system uses the JavAssess assessment library that provides reflection capabilities together with an API to introspect, modify, and test a Java program. To hide the complexities inherent to the use of this advanced library, we have developed a graphical domain specific language (DSL) that uses the library to generate self-assessment exercises.

Thanks to the reflection capabilities, our system goes beyond output comparison and it can not only assess the final output produced by the code, but it can also inspect the structure of the source code itself and the language resources used in the performed implementation (e.g., the existence of a particular field, the definition of a concrete class hierarchy, the implementation of a particular set of interfaces, etc.). This allows the technique to check any property of the code, producing an intermediate mark when the code is partially correct. Moreover, the system is designed in such a way that all automatable tasks such as opening the students code, compiling it, detecting the errors, executing test cases and comparing the results with a teacher’s solution, storing the marks, etc. have been completely automatised, potentially saving a lot of time. In particular, a correct student’s solution is always marked automatically. A student’s solution with errors, can be also automatically marked because the system can correct the errors with reflection (intercession), which allows the system to automatically modify the students code.

The main functionalities of ASys include the following:

• Automatic detection of the assessment order. The system uses Algorithmic Debugging (Caballero et al., 2017) to detect the order of assessment.
• Collaborative assessment. Exercises are divided into atomic properties. When the system detects an error in a property, it records the error, the mark assigned to this error, and the comment introduced by the teacher for the student. When the same error is detected, the system tells the teacher that this error was penalized with mark M. This avoids penalizing the same error in a different way. This is also applied between different teachers, so that the assessment knowledge and work is shared.
• Individual learning record and suggestions. Student’s errors are recorded and classified (e.g., in Java, as polymorphism, abstract classes, interfaces, etc.). Hence, the system can recommend exercises specific to solve the weak points of the student.
• Personalized output. Besides the solution of the teacher, our system also outputs the student’s code corrected.