1. 2010 CRC PhD Student Conference
An Investigation Into Design Diagrams and Their
Implementations
Alan Hayes
alanhayes725@btinternet.com
Supervisors Dr Pete Thomas
Dr Neil Smith
Dr Kevin Waugh
Department/Institute Computing Department
Status Part-time
Probation viva After
Starting date 1st October 2005
The broad theme of this research is concerned with the application of information
technology tools and techniques to automatically generate formative feedback based
upon a comparison of two separate, but related, artefacts. An artefact is defined as a
mechanism through which a system is described. In the case of comparing two
artefacts, both artefacts describe the same system but do so through the adoption of
differing semantic and modelling constructs. For example, in the case of a student
coursework submission, one artefact would be that of a student-submitted design
diagram (using the syntax and semantics of UML class diagrams) and the second
artefact would be that of the student-submitted accompanying implementation (using
java syntax and semantics). Both artefacts represent the student’s solution to an
assignment brief set by the tutor. The design diagram describes the solution using one
set of semantic representations (UML class diagrams) whilst the implementation
represents the same solution using an alternative set (Java source code). Both artefacts
are describing the same system and represent a solution to the assignment brief. An
alternative example would be that of a student submitting an ERD diagram with an
accompanying SQL implementation.
This research aims to identify the generic mechanisms needed for a tool to be able to
compare two different, but related, artefacts and generate meaningful formative
feedback based upon this comparison. A case study is presented that applies these
components to the case of automatically generating formative assessment feedback to
the students based upon their submission. The specific area of formative feedback
being addresses is based upon a comparison between the submitted design and the
accompanying implementation. Constituent components described within each
artefact are considered to be consistent if, despite the differing modelling constructs,
they describe features that are common to both artefacts. The design (in diagrammatic
format) is viewed as prescribing the structure and function contained within the
implementation, whilst the implementation (source code) is viewed as implementing
the design whilst adhering to its specified structure and function. There are several
major challenges and themes that feed into this issue. The first is how the consistency
between a student-submitted design and its implementation can be measured in such a
way that meaningful formative feedback could be generated. This involves being able
to represent both components of the student submission in a form that facilitates their
comparison. Thomas et al [2005] and Smith et al [2004] describe a method of
reducing a student diagram into meaningful minimum components. Tselonis et al
Page 30 of 125

2. 2010 CRC PhD Student Conference
[2005] adopt a graphical representation mapping entities to nodes and relationships to
arcs. Consequently, one component of this research addresses how the student
submitted design and its source code representation can be reduced to their constituent
meaningful components.
The second challenge associated with this research addresses the problem of how to
facilitate a meaningful comparison between these representations and how the output
of a comparison can be utilised to produce meaningful feedback. This challenge is
further complicated as it is known that the student submission will contain errors.
Smith et al [2004] and Thomas et al [2005] identified that the student diagrams will
contain data that is either missing or extraneous. Thomasson et al [2006] analysed the
designs of novice undergraduate computer programmers and identified a range of
typical errors found in the student design diagrams. Additionally, Bollojou et al
[2006] analysed UML modelling errors made by novice analysts and have identified a
range of typical semantic errors made. Some of these errors will propagate into the
student implementation whilst some will not.
This research investigates how such analysis and classifications can be used to
support the development of a framework that facilitates the automation of the
assessment process. This work will be complemented by an analysis of six data sets
collated for this research. Each data set is comprised of a set of student diagrams and
their accompanying implementations. It is anticipated that this work will be of interest
to academic staff engaged in the teaching, and consequently assessment, of
undergraduate computing programmes. It will also be of interest to academic staff
considering issues surrounding the prevention of plagiarism. Additionally, it will be
of interest to those engaged in the field of software engineering and in particular to
those involved in the auditing of documentation and practice.
References
[1] Higgins C., Colin A., Gray G., Symeonidis P. and Tsintsifas A. 2005 Automated
Assessment and Experiences of Teaching Programming. In Journal on
Educational Resources in Computing (JERIC) Volume 5 Issue 3, September 2005.
ACM Press
[2] Thomasson B., Ratcliffe M. and Thomas L., 2005 Identifying Novice Difficulties
in Object Oriented Design. In Proceedings of Information Technology in
Computer Science Education (ITiCSE ’06), June 2006, Bologna, Italy.
[3] Bolloju N. and Leung F. 2006 Assisting Novice Analysts in Developing Quality
Conceptual Models with UML. In Communications of the ACM June 2006, Vol
49, No. 7, pp 108-112
[4] Tselonis C., Sargeant J. and Wood M. 2005 Diagram Matching for Human-
Computer Collaborative Assessment. In Proceedings of the 9th International
conference on Computer Assisted Assessment, 2005.
Page 31 of 125

3. 2010 CRC PhD Student Conference
[5] Smith N., Thomas, P. and Waugh K. (2004) Interpreting Imprecise Diagrams. In
Proceedings of the Third International Conference in Theory and Applications of
Diagrams. March 22-24, Cambridge, UK. Springer Lecture Notes in Computer
Science, eds: Alan Blackwell, Kim Marriott, Atsushi Shimomnja, 2980, 239-241.
ISBN 3-540-21268-X.
[6] Thomas P., Waugh K. and Smith N., (2005) Experiments in the Automated
Marking of ER-Diagrams. In Proceedings of 10th Annual Conference on
Innovation and Technology in Computer Science Education (ITiCSE 2005)
(Lisbon, Portugal, June 27-29, 2005).
Page 32 of 125