Delphi study: Asking the Experts

As a PhD candidate in the Netherlands, we have to develop and execute four studies and publish them in journals listed on SSCI index. Some PhD when they start now exactly the content of the four studies, whereas for others every study is built up on findings of the previous one. In my case, I had an idea of what I want to do in the 4 studies, but as the context proved to be too problematic for data analysis I had to switch several times1.It taught me flexibility. Conducting a Delphi study was never my plan back in 2011, thus I approached it with excitement: Something new to learn! Currently I’m in the middle of it and in the next posts I will reflect upon my experience (Educational scientists will view this as a form of portfolio – a perfect tool for learning)

Social identity configuratonThe origin of this Delphi study began around November 2012. I met my supervisors at the end of the day and we discussed for 1h30 hours the first insights we gained from data. The data were in the form of knowledge sharing networks and the level of social identity of participants measured at three points in time. The big matrix depicted above was the starting point for our discussions. After the lengthy discussion it was decided that we don’t know enough and I should start reading. Some months later after still having no real good idea of what kind of change is a good change, one of the supervisors thought about doing a Delphi study: The experts out there should know and help us. A Delphi study is a mix of qualitative and quantitative data collection. Having only quantitative researchers on my supervision team, I was left to my own to figure out how to best do it. Later on I became aware that one supervisor had some reservation about the research method, but I swept it away as simply reservations towards qualitative research.

The question we tried to answer originally was: What type of change in team and/or professional identity is good for team performance and adaptive expertise. The design of the Delphi study went through 3 stages:

  1. Provide a lot of information to the experts, but kept the questions very open. The aspect of change was prominent in the questions.
  2. After asking colleagues to answer it, it was made simpler by including my thoughts of what change in the level of identification is good/bad for team performance and adaptive expertise. This required that I needed to provide a very long introduction.
  3. I took the drastic decision to not mention change in the questions. Thanks to my daily supervisor who made the excellent suggestions to have experts compare different social identity complexity 2 in a best-worst question format, the change aspect didn’t need to be addressed explicitly. Now the aim was not to say what change is good/bad but to get a ranking of what social identity complexity is best/worst. Change was integrated by a movement up/down the ranking.
Footnote
1 Originally the research should have focused on effectiveness of a master honors course in stimulating several competencies in students, but the program creation took longer than expected and enrolments are lower than expected. Thus not enough data (in time) for me to write my dissertation. 
2Social Identity Complexity is the mixture of team identity and professional identity. I made the distinction between no, some or high level of identity. This gave me 9 different mixes. More information on social identity complexity can be found in this article by Sonia Roccas and Marilynn B. Brewer.

Leave a Reply

Your email address will not be published. Required fields are marked *