Research Paper: Qualitative Approach of Evaluating the Effectiveness of the Coaching Process

Research Paper By Olga Bassai
(Life and Career Coach, INDIA)

Have you ever been in this situation? You are receiving a client for a discovery session and she/he asks you “What are the measurements that you use to assess the result of your coaching? How do you know if coaching actually works? What is your value? “

Personally I have heard those questions many times and I was not able to give a clear answer to my client, except “Well…it works because…”

How as a professional life coach can we start to develop an effective evaluation of the coaching process which will meet the clients’ requirements? And most importantly, how can we help clients to understand and to acknowledge the major changes and progress they have made with the support of a coach?

These questions have initiated my idea for this research paper. The purpose of this paper is to provide a structured qualitative methodology for measuring results of the coaching relationship as well as to demonstrate the way coaching brings about change from the perspective of individual clients rather than that of the organization.

Most of the research that has been done on coaching is from executive coaching perspective working with organizations that mainly focus on the impact and return on investment of executive coaching within the organization (quantitative approach).

However, there is limited research done on life coaching and its impact on life satisfaction from qualitative point of view. This leads to the idea that coaching cannot claim a positive impact on individuals if coaches cannot prove its value, which may make the life coaching profession vulnerable.

Considering that the means to build up a professional evaluation for an independent life coach are much more challenging than for an organization, we want this paper to be a small stone for building up the wall of an effective qualitative approach for life coaching assessment.

In the world of coaching both quantitative and qualitative outcomes matter. The balance between numbers vs. feelings and perceptions should be improved to give more insight into qualitative measures.

I. Two Main Coaching Specificities

1. Qualitative Approach

An increasing number of people are using life coaches to identify what they want personally or professionally and to get support in achieving a life they really want and love.

Coaching is becoming a very popular people development tool – “coaching is a personalized development process that engages every client in a unique way” (Anderson & Anderson, 2005, p.41). At the same time it is seen as an abstract activity when the outcomes rarely can be quantified.

Indeed, the challenge is that many of the shifts experienced by a client are qualitative rather than quantitative: clear vision and life purpose, better work/life balance, increased self-awareness and increased clarity of thought, better relationship with family, etc. As the authors of Co-Active coaching state, “We believe that coaching is about discovery, awareness, and choice…”

How to do to measure this discovery, awareness and choice?

In 2009 the Harvard Business Review conducted a survey in which they asked coaches ‘What should a client look for when hiring a coach?’  32% of the coaches ranked the ability to measure success as a ‘very important’. 70% of coaches surveyed said they provide a qualitative assessment of progress because results are intangible, so they have to measure only in qualitative terms vs. financial terms.

In this context, for an independent life coach it’s very important to establish a good set of qualitative measures so to ensure that the right things get done well.

2. Long-Term Vision

The second specificity of coaching is a long-term vision where the coaching is viewed as a journey rather than a destination. So the impact that the client will experience will be long-term.

The impact of small shifts in attitude and behaviour are cumulative over time continuing way beyond the boundaries of the actual coaching session.

Some of the key features of coaching are: goal-specific and action-oriented. These elements all lend themselves to some element of measurement framework and hence evaluation.

However, these goals and objectives emerge as part of the process (journey), rather than exist at the beginning as part of the current situation.

How can a coach, by taking into consideration these two main coaching specificities, fit the client’s requirement regarding the value she/he wants to get from this relationship?

II. Different Perspectives of Evaluation

We propose to build up our approach in two different perspectives of evaluation: one is the evaluation of client’s progress and another one is the evaluation of coach’s performance.

1. Client Progress Evaluation

There are four phases to a coaching relationship that support the coach while building the evaluation methodology, so to transform some of the abstract goals into a concrete form:

1. Planning – Both coach and coachee independently reflect on their roles and approach and what they want to achieve through the relationship.
2. Contracting – The objective of this phase is to establish an agreed foundation from which a productive and successful relationship will be developed. At that stage coach and coachee agree on:

Key aspects will be recorded in the coaching contract.

Practical evaluation tips for Contracting:

Contract skilfully or do pre evaluation – client wants to know whether coaching accomplishes the goals outlined in the objectives of the engagement.

Attempting to define outcomes while contracting will significantly facilitate the evaluation process and align coaching with desired results from the beginning.

Be sure to make clear what coaching is and what is not. This will help to manage expectations.

Some questions that can help to design the pre evaluation:

Coaching – The key aspect of this phase is to clarify the development needs of the coachee and start coaching. An action plan aligned to the coachee’s goals is developed. Together they should review the relationship against the goals and desired outcomes to ensure they remain focused on the coachee’s identified development needs.

The main strength of the coaching lays in its process. While coaching is about fluid conversations, a truly effective coaching relationship maintains some structure to ensure that the goals of the coaching are met. The coach will maintain a focus on the overall coaching process and agreed outcomes.

Practical evaluation tips for Coaching:

Assess goal – as says Carol Wilson,

A goal without a measure is a dream, not a target.

That is not to say we don’t measure it at all, we just need to frame it differently and pick different measures. It’s hard to attach a measure to some types of goal and it’s more difficult to identify the specific point when the goal has been reached.

What are different measures of success that coach can use to work with?

Once goals are made within the coaching process these then lead to definitions of success criteria, which could and should be linked to evaluation measures.

Success criteria – there’s the client’s definition of success involved in a coaching relationship and it is up to the client to define what success means for them and to own that vision and take the action to create the success. When creating the coaching agreement with the client we:

Some questions that can be used when defining success criteria:

Measure it – Measurable objectives – by helping the client to define what success looks like for them at the beginning of a coaching relationship, the coach will be able to set some measurable objectives.

Sample questions that can be used when measuring a goal (Carol Wilson):

Another solution is to ask for a comparative measure:

Percentage method or scaling:

Important: translate the number the clients give to us to qualitative assessment. Help them describe what they feel different in a positive way and be able to describe some of those differences. “My children comment that Dad and I are no longer snapping at one another during dinner”. Or, “ I think about how to frame questions to my husband so he better understands what I am asking him for.”

4. Evaluation – This phase occurs at the end of the coaching relationship and also earlier if agreed. Evaluating the coaching relationship is relevant at any point in the process, to ensure it remains on track and continues to be beneficial.

Evaluation involves reflection on the coaching relationship, reviewing progress, goals and desired outcomes and acknowledging progress and gains.

Practical evaluation tips for Evaluation phase:

Document it – This is a particularly important step to review and celebrate progress made over a series of coaching sessions. It allows us to identify and to change any aspects of the coaching agreement that can be improved. Generally we can prepare a list of questions and ask the client to reflect on them between sessions and to email responses before the next session.

The use of an open-ended questions method is and important point because it allows the clients to determine which issues they considered to be of most benefit.

Here we will mix up the different questions personally used in daily practice as well as some of them used by Dale Schwarz and Anne Davidson in the ‘ Facilitative coaching’ approach:

Middle term evaluation questions:

Close of coaching contract questions:

Those evaluations should be used to develop a shared assessment that both client and coach believe to be an appropriate picture of current reality.

To summarise this approach:

Coaching is a time-intensive engagement so as a coach we should give regular and formal progress reviews, even if they are only qualitative.

We should be also aware of too frequent evaluation. This can lead to negative responses if people don’t feel they are making enough short-term progress.

2. Coach Performance Evaluation

The second part of evaluation is the coach’s performance evaluation. As a coach we need to assess our success and the quality of coaching relationship.

Practical evaluation tips for coach performance evaluation:

The first point for coach personal evaluation will be the feedback from client. One way to evaluate effectiveness is ask clients for feedback. It’s widely recognized good practice to do a mini-evaluation after each session.

Open ended questions that target this approach need to be developed. We can call it « session feedback form » or « post session form » where we would just ask client for constructive feedback.

This practice builds trust and encourage transparency of the coaching relationship. It gives the coach immediate feedback which he/she can use to adapt his/her approach in the next session.

The second tip that might be a very useful tool in building up the coach performance evaluation is the self- reflection.

Coaching is inherently a reflective process. Throughout the study (Peter Jackson, 2004), “reflection” and “reflective practice” are intended in the broad sense to describe any approach that generates individual self-awareness of behaviour or performance. Reflective practice might be seen as combining the monitoring function of self-reflection and the evaluative function of insight.

Here are some techniques that coach can use for practicing the self-reflection:

The first part might address questions, such as:

In the second part the coach ask him/ herself questions about:

1. How did I help?
2. What insightful questions did I ask?
3. What did I learn?
4. What would I do differently another time?
5. Where did I struggle?

The author advises to create a list of questions that personally works for coach so it becomes a part of continuous improvement and personal and professional growth.

Coaches need to know how to best learn through their experiences. Reflective practice is a major learning tool in this regard. We can learn to develop our self-reflective skills and gain most from our experience if we integrate this practice in our daily routines.

III.Critical Components to Consider in Client Progress Evaluation

1. Client’s preferred representational system:

“Each human being processes information differently from another, by creating their own internal representation of an outside event. These representations are the way the individual has interpreted events based on personal experiences of life and social conditioning.” (Curly Martin, The life Coaching Handbook, 2001).

When we are working with client, asking them to describe the outcome indicators we need to take in consideration their preferred

representational system: visual, auditory or kinaesthetic. When coaching we need to identify which system each client is using. When the coach matches the client preference, he will be more effective in bringing clarity about the goals.

Visual

People with visual preference will tend to:

1. use visualization for memory and decision making
2. be more imaginative and may have difficulty putting their ideas in words;
3. have pictures in their mind;
4. want to see a big picture;
5. draw a map or picture for them- they can see what you are saying.

Auditory

People with an auditory preference will tend to:

1. memorize by steps, procedures;
2. pay attention on the tone of your voice;
3. prefer to communicate through spoken language rather than the written words;
4. need to be heard.

Kinesthetic

People with kinesthetic preference will tend to:

2. Preferences for planning style

In creating a development plan with clear goals for the client, there are two components to consider: planning style and learning style.

In terms of planning styles, Annie McKee, co-author of ‘Primal Leadership and Resonant Leadership’, found in her research that people use different approaches to thinking about the future. Most of us fall into one of these three camps:

Goal-oriented planners like to set specific goals and outline a series of action steps. These folks love check-lists and the sense of accomplishment that comes from finishing a task.

Direction-oriented planners know the general path they wish to pursue, but stop short of getting too specific. They are guided by a sense of purpose and can adapt their specific actions as they go.

A surprising number of us really don’t think much about the future. Action-oriented planners live largely in the moment, avoiding long-term considerations and choosing their next action based on the last one.

Each of these styles has wisdom for the other. The main idea here is for the coachees to know their preference in planning and to be aware of the others as they draft their steps.

For coaches, it’s important to guide clients to find out their preference in planning while creating personal goals.

3. Preferences for learning style

The second component of development plan to consider is the learning style. Many learning and talent specialists are familiar with instruments such as the Kolb learning styles approach.

It is important to consider the coachee’s learning style. David Kolb suggests that people have different preferences for how they learn, relying upon experienced, reflection, theory, or action.

One mistake people often make is to include activities in their plan that don’t fit their preferred approach of learning. The result is they don’t complete the activities, and the plan falls apart.

Helping coachees consider their preferences for learning will enable them to select goals and activities that fit them and their lifestyle.

Conclusion – Recommendations:

Here we would like to summarize different points to take in consideration for those life coaches that are looking for ways to significantly improve their qualitative approach of evaluation:

1. Pre-evaluation: Encourage the client to share the expectations of benefits before the start of the coaching. Record them. The coach needs to understand the answers before the coaching begins.
2. Record the individual client’s instant reaction directly after each session.
3. Record the individual client’s qualitative view of the value of the coaching.
4. Document your work with clients. Create 2 lists of questions for middle term evaluation and end of contract evaluation.
5. Be adaptable to clients’ specific measurement and feedback. Use measure questions while assessing goals.
6. Identify preferences: representational systems, planning styles, learning styles.
7. Create a self-reflection list of questions for personal and professional growth as a coach.
8. Keep a journal that can help you to record your thoughts, feelings, questions or concerns related to your work performance.
9. Encourage and promote a focus on quality.

I would like to finish this research paper with the last sentence of one of the first module I have accomplished with ICA: Coaching Models – “It is important to remember… that evaluation instruments are there to support the coach, but they shouldn’t drive the coaching process.”

Be prepared and trust the process!

References:

David A. Kolb and Richard E. Boyatzis “Goal Setting and Self-Directed Behavior change”, Human Relations Volume 23 No.5
Kirkpatrick, D. and Kirkpatrick, J. (2005) Transferring learning to behavior: using the four levels to improve performance. San Francisco, CA: Berrett-Koehler
Chartered Institute of Personnel and Development, Guide “Real-world coaching evaluation A guide for practitioners” (2010)
Carol Wilson Best Practice in performance coaching (2011)
Curly Martin ‘The life coaching Handbook Everything you need to be an effective life coach’ (2001)
Anderson, M., & Anderson, D. (2005). Coaching that counts: Harnessing the power of leadership coaching to deliver strategic value. Jordan Hill, Oxford: Elsevier Butterworth-Heinemann.
Peter Jackson Understanding the experience of experience: a practical model of reflective practice for Coaching. Jackson Personal Development Coaching, Chalford, Gloucestershire, UK International Journal of Evidence Based Coaching and Mentoring Vol. 2, No. 1, Spring 2004 Page 57
Rosinski, P. (2003). Coaching across cultures. London: Nicholas Brealey.
Dale Schwartz and Anne Davidson Facilitative coaching, Pleiffer, 2009
Henry Kimsey-House, Karen Kimsey-House, Phillip Sandahl and Laura Whitworth (2011). Co- Active Coaching Changing Business Transforming lives
David Clutterbuck (2010). Using the seven conversations in supervision.
http://www.davidclutterbuckpartnership.com/wp-content/uploads/Using-the-seven-conversations-in-supervision.pdf