From Ideas to Impact: Verification, Validation, and Evaluation of Design Artifacts

Prof. Jolita Ralytė

Abstract

Design-oriented research in digital systems increasingly produces artifacts—methods, models, languages, frameworks, and tools—intended to address real-world problems. Yet, a recurring weakness in doctoral research lies not in the creativity of the proposed artifacts, but in the rigor and clarity of their assessment. Too often, verification, validation, and evaluation are treated as interchangeable terms, reduced to a single “evaluation section,” or postponed to the very end of a PhD project.

This talk will argue that every scientific contribution must undergo multiple, distinct levels of assessment, and that verification, validation, and evaluation (VVE) should be explicitly planned as part of a PhD research agenda, not added as an afterthought. Drawing on extensive experience in method engineering and the development of modeling methods, the talk will clarify the meaning, purpose, and complementarity of verification (“am I building the artifact right?”), validation (“am I building the right artifact?”), and evaluation (“is the artifact worthwhile in practice?”). Building on a recent expert-voice article and related work in design science research and quality evaluation frameworks, the talk will illustrate how VVE activities evolve over time, how they align with iterative and incremental research, and how they can be realistically conducted within doctoral constraints. Examples from the development of modeling and method artifacts will be used to highlight common pitfalls, trade-offs, and good practices.

The talk aims to equip PhD students with conceptual clarity, practical guidance, and realistic expectations regarding assessment, empowering them to argue more convincingly for the scientific value and credibility of their research contributions.

About Speaker

Jolita Ralyté is a Senior Lecturer and Researcher at the University of Geneva, Switzerland, where she also directs the continuing education program MATIS (Management and Technology of Information Systems). She holds a PhD in Computer Science from the University of Paris 1 Panthéon–Sorbonne, France. Her research interests include method engineering, conceptual modeling, requirements engineering, digital transformation, and transdisciplinary and smart services. She is a co-author of a book on situational method engineering. She currently serves as Associate Editor of the Springer book series IFIP Advances in Information and Communication Technology, and of the journals Business & Information Systems Engineering (BISE) and Requirements Engineering (REEN). Since the early 2000s, Jolita Ralyté has been actively involved in the International Federation of Information Processing (IFIP), where she served as Secretary, Vice-Chair, and Chair of IFIP Working Group 8.1 “Design and Evaluation of Information Systems”. She currently serves as Chair of IFIP TC8 “Information Systems.” Jolita is a member of the Steering Committees of RCIS (Chair), PoEM (Chair), ER (Vice-Chair), INFORSID, and CAiSE (2012–2020) conferences. She has served as General Chair of RCIS 2026, PoEM 2025, PoEM 2021, and ME 2011, and as Program Chair of Baltic DB&IS 2024, IESS 2023, ER 2022, RCIS 2022, RCIS 2016, PoEM 2015, IEEE CBI 2014, INFORSID 2014, CAiSE 2012, and ME 2007.