Paired comparison method example. Hello student

We'll tell you what the paired comparison method is and explain how HR can use it in their work. Let's consider its pros and cons. Here is an example of the paired comparison method.

From the article you will learn:

Related documents:

What is the “paired comparison method”

The paired comparison method is a tool for selecting one option from several. The author of the method is American psychologist Louis Leon THURSTONE. He developed the method in the first half of the twentieth century. Since then it has been used in mathematics, physics and psychology. In the HR field, the method of paired comparisons has been used for the last 15-20 years. It works successfully when you need to select one or more employees to solve a work problem.

The paired comparison method is used not only to evaluate existing employees, but also when selecting personnel. The method is especially effective during a group interview. It allows recruiters and HR people to quickly make coordinated decisions. Of course, the paired comparison method will not allow you to predict how a candidate will perform; for this you need to use other, more accurate assessment tools.

Hint card: how to determine if a candidate is suitable for a position

The paired comparison method is used when selecting employees for the personnel reserve or when introducing a new remuneration system. For example, when the task is to rank positions and combine equivalent ones into groups. In HR practice, there are also unusual options for using the paired comparison method. For example, when developing company values. Employees are offered valuables in pairs, and in each pair they must choose the most important one. The values ​​that score the most points are approved as corporate values.

How to use the paired comparison method

Let's look at how to use the paired comparison method to evaluate personnel.

Step 1. Make a list of employees you will evaluate. Determine by what criterion you will rank specialists. For example, knowledge of a certain program, professional or personal quality. For example, let’s take the quality “punctuality”.

Step 2. Generate all possible pairs of employees for comparison. To do this, enter the employee’s name in a specially prepared matrix.

Step 3. Compare the first employee with the second. In our example - Arbuzova and Baranova. Which one is more punctual? Who always completes tasks on time and is never late? If it is Baranov, then he gets 1 point. Then Arbuzov will receive 0 points.

An example of a pairwise comparison matrix for the “Punctuality” criterion

Step 4. Compare the first employee with the third. In the example - Arbuzov and Vinogradov. Rank the points according to the same scheme.

An example of paired comparison using the “Punctuality” criterion

Download a sample paired comparison matrix

Step 5. Repeat the process until you compare all employees with each other.

Step 6. Consider the employee who scores the most points as the best according to the selected criterion. Next, rank by points.

Assessment form using the paired comparison method according to the “Punctuality” criterion

Determine the number of comparisons to estimate a certain number of employees using the formula:

Number of comparisons = Number of employees x (Number of employees - 1) : 2

In our example, we used the minimum number of employees - 3. The number of comparisons is determined by the formula: number of comparisons = 3 x (3 - 1) : 2 = 3

For 4 employees there will be 6 such comparisons, for 5 - 10, for 10 - 45. Further, this figure will rapidly increase. This is the main disadvantage of the method - it can be used to evaluate relatively small groups of people . Ideally, up to 15 people.

What to do if you need to evaluate employees according to several criteria? For each criterion, prepare a separate matrix similar to the one above. For example, if you are assessing professionalism, punctuality and honesty, then prepare three matrices. In each matrix, rate employees according to one criterion and calculate the total score. Then create a pivot table. Enter the final scores from the three tables into it and sum them up for each employee.

Who should evaluate employees?

There is no point in using the services of external consultants: they are not familiar with the people who will be assessed and do not know their professional qualities. The head of a department who knows his subordinates well can evaluate employees. However, even the most experienced manager may not know everything about his employees.

Why do managers resist evaluating subordinates?

Involve ordinary department employees as appraisers. Prepare them for this process and explain the nature of the task. There should be no difficulties with the procedure; the method is simple and clear. The main requirement for appraisers is that they must know those who will be assessed and could observe them in their daily work.

Pros and cons of the paired comparison technique

The pros and cons of the paired comparison method are presented in the table.

Difficulties in using the paired comparison method: examples

Let's look at the main difficulties that HR may encounter when using the paired comparison method and give recommendations on how to correct them.

Difficulty 1. The evaluator cannot choose the better of the pair. In his opinion, they both deserve the highest rating. The reason for the difficulty: it is difficult for him to remember examples from life when an employee showed his qualities.

How to fix: ask leading questions, ask them to focus on the last months of the employees’ work and think about who deserves the highest rating.

Difficulty 2. The expert evaluates employees without thinking and tries to complete the task faster. The reason for this attitude is a lack of understanding of the purpose and meaning of the assessment. Most likely, the expert considers it a quirk of HR.

How to fix: explain how the assessment can affect the employee's fate. Make the expert feel that his opinion is important to management. Recognize the expert's experience and authority.

Difficulty 3. The expert evaluates employees with whom he is almost unfamiliar. The reason is that the groups for evaluation were formed incorrectly.

How to fix: groups should be formed anew and the main principle of the paired comparison method should be observed - the evaluator personally knows the person he is evaluating.

Plan:

1. Description of the method of paired comparisons.

2. Advantages and disadvantages of the paired comparison method.

1. Description of the paired comparison method

Paired comparison method- one of the tools for assessing and selecting solutions, widely used in expert assessments when it is necessary to set priorities in the process of any activity or ranking various objects.

The idea of ​​the method is that every two objects are compared in pairs and the primacy of one of them is determined, hence the name - “pairwise (or paired) comparison.” It is believed that when solving a problem it is much easier to make a qualitative comparison of two objects, relying on the opinion of experts, than to establish quantitative criteria. Strictly speaking, the PS method is a method of obtaining initial data, a method of conducting a kind of survey of respondents. Based on the data obtained, you can solve various problems, which do not necessarily include the construction of a rating scale. Let's assume that we are interested in how respondents in the population being studied evaluate certain objects - professions, political leaders, radio programs, certain types of goods, etc. Let us denote these objects by a1, a2, ..., an (n is the number of objects being assessed).

Table 1. An example of using the PS method

The method under consideration allows us to obtain an answer to this question in a rather unique form. Each respondent is offered all possible pairs made up of the objects in question. For each pair, he must say which object of this pair he likes best. Let's say, if we consider some professions as our objects - for example, turner, baker, doctor, etc. - we ask each respondent which profession he likes best: turner or baker (record the answer), turner or doctor (record the answer), baker or doctor (record the answer), etc. for all possible pairs of objects under consideration.

2. Advantages and disadvantages of the paired comparison method

The main advantages of the method are as follows:

It is possible to measure the unevenly changing importance of indicators, which is so necessary for solving most practical economic problems;

During the analysis process, the expert does not focus on all indicators at once, but only on two, compared at any given moment, which facilitates the work and, therefore, helps improve its quality;

It is possible to obtain a large number of comparisons of each indicator with others, which increases the accuracy of the assessment and opens up the opportunity to study the quality of more aspects of the research object than when using other methods;

It is possible to obtain not only the average assessment of the indicator given by each expert, but also the dispersion of this assessment, which makes it possible to further conduct a more in-depth economic and mathematical analysis

The paired comparison method is the simplest of the existing classification tests, since it involves comparing only two samples of a product

The advantage of the paired comparison method over ranking is that it is easier to make a judgment since the supervisor only has to compare two people at a time. The second advantage is that it provides the opportunity to put people with the same abilities on the same level.

The method of paired comparisons makes it possible to conduct a rigorous, statistically based analysis of the consistency of expert opinions and to identify whether the estimates obtained are random or not. Undoubtedly, the procedure of the paired comparison method is more complicated than the simple ranking method, but simpler than the sequential comparison method.

This method is very simple and it allows you to study a larger number of objects (compared, for example, with the rank method) and with greater accuracy.

Main disadvantages methods are as follows:

- the need to perform a huge number of pairwise comparisons if large groups need to be assessed. A supervisor with 60 employees will have to perform 1,770 comparisons! If the comparison is carried out according to five separate parameters, then the number of pairwise comparisons will increase fivefold. Typically, the paired comparison method is used in two cases: either when assessing small groups, or when assessing only one parameter - the overall efficiency of production activities

One disadvantage of paired comparisons as an experimental method is that comparing n stimuli requires making (n - 1) x (n/2) judgments. For example, for 10 stimuli we need to obtain 45 judgments, and if we wanted to scale a set of 50 stimuli, we would need 1225 judgments.

The main disadvantage of the paired comparison method is that there are no ready-made software developments.

To use this method, you will have to instruct competent company employees to develop a special program or order it from a third-party developer. Additionally, there are no proven business valuation solutions based solely on pairwise comparisons, but rather a mathematical half-baked product that serves to create scaling tools. In addition, the test procedure itself looks rather monotonous, which affects the objectivity of research conclusions. And the quality of the results directly depends on the number of assessments and indicators assessed, as well as the correct selection of pairs and their unambiguous interpretation.

The paired comparison method is generally considered to be superior to direct ranking. This point of view is not entirely correct, and here's why.

The choice of method is always determined by the research situation and the objectives of the study. Naturally, if the basis for ranking can be unambiguously formulated, then the method of paired comparisons gives an excellent result and this method should be chosen. But there are situations when a clearly understood ranking basis is impossible and not really necessary.

The disadvantage of the method is the increase in the labor intensity of the procedure as the number of objects increases: already with 12-15 objects the procedure becomes labor intensive. In addition, different pairs of objects are sometimes compared by respondents according to different criteria, which leads to intransitivity of preferences. The method is widely used in expert assessments.

Questions for self-control

Define the pairwise comparison method.

What is the idea of ​​the method?

Describe the comparison process?

What are the advantages of the paired comparison method?

Name the disadvantages of the paired comparison method.

The method of pairwise comparison of employees is one of the methods of comparative personnel certification.

The specificity of this technique involves comparing workers of the same or similar professions with each other. Comparison is carried out according to one or more parameters for assessing the competence of employees. These parameters are selected depending on the specific content of the work performed by the employees being assessed, since different types of work place different and sometimes diametrically opposed requirements on employees (for example, communication skills are an ideal quality for a sales consultant, but a clear hindrance in work for a worker working on an assembly line).

So, for example, to evaluate sales consultants, evaluation parameters may include communication skills, politeness, ability to persuade and other qualities that help to interact effectively with people. But for an enterprise accountant, these qualities are no longer so important; here accuracy, punctuality, and mathematical abilities come to the fore.

To conduct a comparative assessment of employees, a separate assessment table is compiled for each assessment parameter, the number of rows and columns of which corresponds to the number of employees being compared. The technology for filling out the table is as follows: when comparing two employees, you need to distribute 2 points between them in one of the following ways:

• if one of the employees is better at the assessed skill than the other, then he is assigned 2 points, and the “losing” employee is assigned 0 points (in the table: employee “A” is more sociable than employee “B”);
• if employees have equal proficiency in the assessed skill, each of them is awarded 1 point (in the table: employee “A” is as communicative as employee “D”).

1 point is assigned along the diagonal of the table.

Example - assessment of the communication skills of five sales consultants

For self-test, remember that the points in the table are distributed correctly if the equality is satisfied:

where Point i is the total number of points assigned to the i-th employee (the amount in the corresponding line);
N – number of employees being assessed.

Publications

Lityagin A. Effective certification
The typical mistakes made by HR managers when conducting certification are discussed, as well as its purpose, international standards and ways to protect against subjectivity.

Tkachenko S., Zharkov A. How to measure competence. About the personnel assessment method “assessment center”
Data on the accuracy of various methods of personnel research are provided, a definition of competence is given, and the possibility of assessing personnel using the “assessment center” method is determined.

Borisova E. Certification. Is the game worth the candle?
The article compares the concepts of certification and personnel assessment, formulates the procedure for its implementation, and gives a brief description of the methods of personnel certification.

Malinovsky P. Methods of personnel assessment
Various methods of assessing an organization's personnel are considered: the questionnaire method, the descriptive assessment method, the classification method, the pair comparison method, the specified distribution method, the decisive situation assessment method and others.

Gorsky P. Personnel assessment. Mathematical tools
This article discusses the possibilities of correctly obtaining an employee's assessment, often called a "rating". Methods for assessing the reliability of the rating and setting the problem of estimating the monetary costs of assessment work at a given level of reliability are also proposed.

The paired comparison method is one of the most common methods for assessing the relative preference of alternative options.

When using this method, the expert is sequentially offered pairs of alternative options, of which he must indicate the more preferable one.

If an expert finds it difficult to do this regarding any pair of objects, he has the right to consider the compared alternatives to be equivalent or incomparable.

After sequentially presenting all pairs of alternative options to the expert, their comparative preference is determined according to the expert’s assessments.

As a result of paired comparisons, if the expert turns out to be consistent in his preferences, all evaluated alternative options may be ranked according to one or another criterion, indicator or property. If the expert recognizes some alternative options as incomparable, then the result will be only a partial ordering of them.

In the practice of using the method of paired comparisons, one often has to deal with inconsistency and even contradictory assessments of experts. In these cases, it is necessary to conduct a special analysis of the examination results.

We also note that with a sufficiently large number of alternative options being evaluated, the procedure for pairwise comparison of all possible pairs of them becomes labor-intensive for an expert. In this case, it is advisable to use appropriate modifications of the method of paired comparisons.

For example, if we assume the consistency of the expert’s assessments, then a single presentation of each alternative option paired with some other option is practically sufficient.

3. Ranking of alternative options.

A fairly common procedure is also the direct ranking by an expert of the preferred alternatives being evaluated. In this method, the expert is presented with alternative options selected for comparative assessment, but preferably no more than 20 ÷ 30, in order to order them by preference. If there are more alternative options, then it is advisable to use appropriate modifications of the ranking method.

For example, the ranking of alternative options may be preceded by their division into classes ordered by preference using the expert classification method.

The expert can rank the compared objects in various ways. Let's give two of them.

In accordance with the first, the expert is presented with the entire set of alternative options, and he indicates the most preferable among them. Then the expert indicates the most preferred alternative among the remaining ones, etc., until all the evaluated alternative options are ranked by him.

In the second method, the expert is initially presented with two or more alternative options, which he is asked to order according to preferences. If the expert is initially offered several alternative options for ordering according to preferences, then at this stage he can use the first ranking method.

After the initial ranking, the expert is successively offered new, not yet evaluated, alternative options. The expert must determine the place of the newly presented alternative option among those already ranked.

The procedure ends after the last alternative is presented and evaluated.

The proposed methodology is a derivative of the comparative analysis method. It is based on the method of pairwise comparisons, well known in matrix algebra. Like any product, it has advantages and disadvantages. The disadvantages include its relative labor intensity, and the advantages include uncriticality of the strict selection of analogues, because allows you to obtain fairly accurate results even in the absence of analogues of the evaluated object that are similar in their characteristics. To facilitate calculations, a computer version of this technique is used.

Sequence of determining cost using pairwise comparison method 4:

Selection of analogue objects (at least 3, the optimal choice is 4-6 analogues, but this increases the complexity of calculations). To describe the method, assume that (n-1) analogue objects have been selected. Thus, n real estate objects are involved in the process of pairwise comparison, including the subject of assessment.

property number, .

Selection of comparison parameters that significantly affect the cost (the optimal number is 6-10, while other parameters are considered similar to the parameters of the object being assessed). The comparison is made by m factors.

factor number,

Formation of a scale of qualitative assessments (Table 28) according to the following principle:

Table 28

Qualitative rating scale

Each parameter is assigned an importance according to the selected scale. The parameters are compared in pairs. The diagonal of the matrix will have a value of 1, because in this case the object is compared with itself and, naturally, must have equal importance. If one cell contains an integer number, then the mirror cell relative to the diagonal will contain a fractional number.

Table 29

Importance Index Matrix

Translation of the result of comparison of objects and, expressed in a qualitative discrete scale with a maximum number of gradations equal to 9, into a numerical value is carried out as follows.

Determining the weight of each importance index:

Finding Hadamard vectors:

Calculation of consistency spreads:

The consistency index should not exceed 0.1. Otherwise, it is necessary to change the importance indices in this matrix.

Determination of the reduced normalized deviation of each object by m factors:

where is the reduced normalized deviation of the evaluated object;

– cost of a unit of comparison of an analogue object.

An example of determining cost using the comparative approach using the pairwise comparison method

Initial data: the object of assessment is a store building located in the village. Pasad. Additional data and construction of matrices are reflected in Table 30 (see p. 76).

Calculation of the weight of the importance index (using the first matrix) (formula 26):

Table 30

Pairwise comparison table

We normalize the resulting weights (formula 27):

We have a vector of priorities

Finding the Hadamard vector:

Hadamard vector,

divide the Hadamard vector by the priority vector

and we obtain the consistency vector of the first matrix Z 1 .

and consistency index

If it does not exceed the value of 0.1, then the matrix is ​​constructed correctly. Such operations should be performed for all matrices (in the example there should be six of them).

We calculate the reduced normalized deviation of each object using formula (32):

We determine the cost of the desired object: