Test - Psychometric Test - Inventory - Psychological Test - Survey - Questionnaire - Instrument - etc. etc.  

All of these terms are used to describe a series of questions, designed to help measure knowledge, skills, aptitudes, behaviours, attitudes and personality traits.  

Here are two other definitions given by the British Psychological Society.

'A psychological test is any procedure on the basis of which inferences are made concerning a person’s capacity, propensity or liability to act, react, experience, or to structure or order thought or behaviour in particular ways.'

The word ‘test’ is the term most widely used by people other than psychologists.  That is why we chose to use that term in this Guide.  However, we do like the words ‘questionnaire’ and ‘inventory’.  So when we think that it is appropriate we will use these descriptions.  You will find that we tend not to use the term 'psychometric' too much.

Hope all this does not confuse you!     

Common Sense Rule:

You judge the quality of a test in much the same way as you would judge a ‘good interviewer’.

A good test, like a ‘good interviewer’, will pose many questions rather than just a few.

Asking more questions =
more answers = more information.

Common Sense Rule

Like a good interviewer, a good test needs to be realistic. People completing questionnaires as part of a selection or assessment programme often try to ‘beat the system’. 

Many people frequently and naturally seek to impress. Some even, on rare occasions, attempt to make a ‘bad impression’.
Like a good interviewer, a well-constructed test anticipates this behaviour. 

Be ‘street-wise’ rather than ‘ivory tower’.   

Common Sense Rule

Good tests use interrogation techniques. For example, they use a technique called ‘masking ‘.  This involves embedding significant and critical questions amongst non-relevant questions.

Questions are also posed in an unexpected rather than an anticipated sequence.

Do not be a textbook interviewer!

Change your pace, change your style, and try not to let the interviewee know what is coming next.
Remember it’s a battle of wits.
Study your 'enemy'.
Read all the books giving hints about - 'How to Get Through a Tough Interview.' Good candidates will!   

 Return to Index

A good test only promises to provide ‘clues’. Clues to help you structure your interview/s and assessment investigation.

A clue does not become a fact until you have very carefully searched for a lot of evidence to confirm – or contradict - the test result.    

WYGIWYS - Never trust a test result (clue) until you have at least six ‘bits’ of supporting evidence.

As previously stated, we vigorously stress that no selection, development or training decisions should be made solely on the basis of a test result.

We recommend that you consider using the Rudyard Kipling Technique - the RKT:

‘I keep six honest serving–men
(They taught me all I knew);
Their names are

What 

and Why

and When

and How

and Where

and Who.
I send them over land and sea,
I send them east and west;
But after they have worked for me,
I give them all a rest.’

Rudyard Kipling

 Return to Index

‘Good tests' compare 'like with like'

Common Sense Rule

A good test should compare the person being tested against a large number of other people very similar to him or her.

For example.
You are very rich - You are very poor
You are very tall - You are very short
You are very young - You are very old.

Any of these statements could be true about you.
It just depends against whom you are being compared.

WYGIWYS and Rudyard Kipling Technique (RKT)  will help you find out which is right. (See above section.)

Always ensure that your people are being compared against a large number of people as much like themselves as possible.

Remember – ‘like with like’ increases accuracy. 'Like compared with unlike' distorts the score and reduces accuracy.

 Return to Index

A ‘good test' never asks you to keep the results a secret

Common Sense Rule

The authors of this Guide strongly recommend and encourage disclosure of test results. We believe that people have the right to this information. We also believe that they should be given an opportunity to express their opinions on the accuracy of test results.

Always disclose and discuss a person’s test scores.

Let them know who you are comparing them against. This should either be your own company norm or an appropriate occupational norm group.

Tell them why you consider the areas you are measuring to be important.

Obviously, you refrain from doing this until you have carried out most of your other investigations. You don’t want to affect their behaviour or answers to interview questions.

People are more likely to trust you if you do not keep scores a secret and if you tell them why you are measuring a particular aspect of personality or behaviour. If the test you are using is 'good' - and the group against which test takers are being compared is appropriate - there is absolutely no reason why you should not give the test taker his or her results. 

Indeed, in some countries, the law does require you to disclose test results. Hopefully this law will soon be universal. We believe that everyone has the right to have access to the results of any test they complete.  

 Return to Index

Common Sense Rule

They need a good test because they are only human.

As a good interviewer, you will know that during an interview it’s virtually impossible to ask identical questions, in an identical manner, in the same sequence and elicit a definite response. 

You are only human. You may be a good judge of character but, unless you are Mr. Spock, we doubt that you can ever be as clinical as a good test.   

 Return to Index

Anyone can design a ‘good test'

Given the right set of circumstances and budget, anyone can design a good test. There should not be any mystique surrounding the creation of a good test. 

It is easier and less costly to develop tests to measure knowledge, skills, aptitudes and attitudes than it is to measure personality traits.      

Tests relating to personality can take a long time to develop and validate. This can be very costly. Consequently, you have to consider the cost-effectiveness of creating such tests very carefully.

In a recent article* Dr. Robert McHenry, Chairman of OPP - one of Britain's largest test publishers - said "developing a test could cost up to £100,000".  He also "... urged HR practitioners to use tests in conjunction with other selection methods such as role play, in-tray exercises and structured interviews."  We totally agree with that advice.  Indeed, we vigorously stress that no selection, development or training decisions should ever be made solely on the basis of a test result.

 *'People Management' published by the Chartered Institute of Personnel Development

 Return to Index

A recent article in the Daily Telegraph reported on various selection methods and their accuracy in predicting job success.

Selection Method:

Normal interviews 

Structured interviews

Personality test 

 Ability/skill tests 

Assessment Centres

Note:  Assessment Centres almost always include Ability and Personality tests.

Degree of accuracy at
predicting job success:

17% 

33%

42%

54%

65%

Assessment Centres

If tests are not being used, you might ask the organiser of the assessment centre ... Why not?

Reliability and Stability

A good test should produce consistently similar results when administered to the same group of people over different time periods.

For example, the reliability coefficients for the tests that we use and recommend range from .86 to .92. 
In every day language this means an 86% to 92% test to re-test accuracy.

This is considered to be a very high correlation between test and re-test results.

 Return to Index

Validity

Questionnaires are shown to be valid if they:

1. Correlate with results obtained by other questionnaires – which have already been proven to be valid - measuring the same skill or personality area. 

2. Accurately predict how someone behaves or performs.

Predictive Validity

Data relating to predictive validity is always rather difficult to obtain for the following reasons.

1. Companies frequently have difficulty in accurately and objectively measuring the competencies of their work force.

2. Often there is almost total reliance on opinions expressed by the individual’s manager, supervisor or peer group.

Obviously such opinions can be very subjective and highly influenced by recent or immediate work performance rather than taking into consideration an individual’s competency over a longer period of time.

Criterion Validity

The best situations involve validating the questionnaire against specific criteria. For example:

1.  A group situation where individual’s competencies are being objectively measured and compared in a controlled environment. For example – an Assessment Centre, Training or Development Course.

2.  Performance within a group that has common objectives. For example – a sales force where earnings are directly related to results and where factors such as compliance to defined regulations, activity level and quality of ‘sale’ are measured.

In situations such as these all of the tests we use and recommend indicate a very high validity coefficient between test results and actual job competency or assessment centre conclusions. Obviously the correlation will differ from organisation to organisation depending on their competency requirements.

 Most personality test instruments appear to be somewhat more efficient at identifying those who cannot do the job (and yet who are often hired), than predicting success in the job. 

 Consequently, organisations that follow the soundest selection method - process by elimination – should find these selection and assessment tools very cost effective.

 Return to Index

Assess the validity of a test and its value to your organisation:

 First, remember, tests are a product. Just like other products there are branded, generic and own-label tests. 

New ones are being introduced all the time - some become dated, some fall out of fashion and, like other products, some just never make it.

Just because a test has worked well elsewhere there is no guarantee that it will be of value to your organisation. So don’t base your decision solely on a sales presentation. Do your own market research and be an informed consumer.

Concurrent Validation Study

We recommend that, if possible, you validate all tests within your own organisation. You can do this by conducting what is described as a Concurrent Validation Study.

Conducting a Concurrent Validation Study will also provide you with localised ‘norms’ – benchmarks against which future applicants can be considered.

A Concurrent Validation Study is carried out by administering tests to a cross section of a workforce who have, as objectively as possible, been ranked in order of job competency or against specific criteria. Results are then analysed to identify the relationship between test results and performance levels.

By way of example, one recent Validation Study conducted by one of Britain's largest assurance companies, established that certain personality traits correlated highly with success or non-success as a Financial Adviser.

Test results indicating that an individual would be in the top half of the sales force - were 88% correct.

Test results indicating that an individual would be in the lower half of the sales force - were 95% correct.

These results are similar, in terms of predictive validity, to many other Concurrent Validation Studies.

Please remember – we vigorously stress that no selection, development or training decisions should be made solely on the basis of a test result.

 Return to Index

Predictive Validation Study

This takes longer but is probably even more effective than a Concurrent Validation Study. However, you can only do this if you are hiring a reasonably large number of people – to do the same job – over a short period of time.

In a Predictive Validation you administer tests as part of your normal selection procedure but you do not disclose the results to anyone involved in the selection, training or day to day management of those who are hired.

Over a period of time you compare test results against actual job performance criteria. This will identify any positive/negative correlation and so assess the predictive value of the test questionnaire.

One slight disadvantage of this method is that you cannot give chosen candidates ‘feedback’ until you have completed your predictive validation study.

All authors and sellers of tests should be willing to help you conduct your concurrent or predictive validation study. They will normally do this for a reduced cost. They may even consider doing it for free if you are a big enough fish! Remember there is nothing mystical about tests, they are a ‘product’ like every thing else.   

Norm Groups

These are the Control Groups against which you can compare people.

Norms are sets of scores derived from a clearly defined sample. The importance of norms is that it gives a 

test score (raw score) a meaning, making accurate comparisons possible.

Remember, your test score (raw score) is a constant. Your norm score will vary depending upon the norm group against which you are being compared. 

For example: if you are a 6’ 2" male adult you would have a high norm score for height when compared against a cross section of the UK/USA adult male or female population...

However, you would only have an average norm score for height if you were compared against African Watusi male adults (who average around 7'!)

We recommend that you create your own norms wherever possible.

However, if you do not have sufficient numbers to create your own norm - or if you are in the process of change and are looking for people who are different from your current workforce - the authors of this Guide can recommend an appropriate occupational norm group. 

These norm groups include:

This range is constantly expanding to include other occupational groups.

• UK general population norms.

• Sub-groups – demographic, socio-economic, working/non-working, age range and sex, etc.

• Occupational norms – primarily white collar. These include senior, middle and junior management and supervisory.

• Financial services (high net worth sales, direct sales, home business, IFA and tied agents).

• Call centre – both reactive and pro-active.

• Retail – white goods, fashion, food, video rental, etc.

• Catering – managerial, waiting and production staff.

 Return to Index

What if a Validation Study proves nothing significant?

What if you spend money on a validation study and the results are inconclusive or establish that there is no correlation between areas measured and job performance?

Isn’t that money wasted?

No, it will probably be the best money you have ever spent ! 

Its better to invest in a validation study than to simply accept ‘sales speak’ and use a test that is valueless. 

For example, we know of one major supermarket chain that wasted much money and time testing checkout assistants for numeracy skills. Think about it! They did that because someone did not conduct a validation study.
After taking our advice, they conducted a concurrent validation study and now invest money on visual comprehension, dexterity and colour recognition testing.

 Return to Index

A word of warning about validation studies

Beware of the sales person who offers you the ‘Sales Prospect Validation Study’, aka the ‘Barnum Bait Validation Study’. This is where they offer you one free test to do personally. They then try to get you to base your decision to buy their tests on the accuracy of your personal test report. You can imagine how good that report is likely to be!

Barnum - please refer to:
Expert Reports are
created in this way

You already use a test - why consider change?

Are you continually validating the relationship between the test scores you are currently obtaining and job performance?

If you are doing that - and you are happy - then only consider changing your routine if you want to achieve one of the following:

- Get the same, or better, results and information for less money.

- Improve your measurement of personality traits that are not being measured by your current test/tests.

- The protection and comfort of a second opinion.

By the way, if you have people in your organisation who tend to resist change you might like to ask them to consider the following:

  The title of a Harvard marketing case history relating to people’s resistance to and fear of new products, innovations and change. The title refers to a common reaction to a new product launched in the 1930’s – Sellotape!

 For example, one supermarket chain still uses a numeracy test when selecting check-out personnel. Surely sight, dexterity, dyslexia and colour blindness tests would be more appropriate?

 Return to Index

Do you need ‘expert reports’? Answer the following question to find out:

Who do you think is likely to be more reliable, dependable, committed and hard working?

A. Someone who scores high on a test for Conscientiousness.
- or
B. Someone who scores low on a test for Conscientiousness.

Did you choose 'A'?  

Congratulations - you are an expert !  You are unlikely to need an Expert Report to understand the implications of a comparative test score. However, the following may still be of interest.

The term Expert Report usually refers to a computer generated narrative report on a person’s test results. Some are good and some are worthless - even dangerous. They can be dangerous because some have in-built discriminatory biases and use inappropriate ‘look-up tables’ (norm groups). Many perpetuate occupational stereotypes.

Some employers become over-dependant on such reports - they treat the reports as gospel. They become lazy and fail to practice basic test-user skills and disciplines. They neglect to search for information that supports or negates test scores.

Expert Reports are created in this way                                            [Back to A word of warning about validation studies]

The software relates a test score to a ‘look-up table’. This, in turn, triggers off a particular standard sentence or paragraph. These standard sentences and paragraphs describe and predict how the ‘expert’ expects the individual being tested will behave in the future.

As we have said, some Expert Reports are good and may well help the less experienced person to understand and interpret comparative test scores. If you do choose to use one, you must ensure that there is a facility to change the norm group (look-up table).  If you don't have this facility, you may be using a look-up table which is totally inappropriate.

The worst Expert Reports are simply packaging and are nothing more than some rather sweeping assumptions and a rephrasing of the candidate’s answers to test questions.

For example:  A ‘strongly agree’ response to the test statement 

"I enjoy meeting new people" triggers the following sentence in the ‘expert report’ - 

"Mr. Example will be good at dealing with customers and building new business relationships.  He is not likely to experience difficult in establishing social and interpersonal relationships or working as a member of a team."

Sometimes, whole series of paragraphs and assumptions are created based on responses to two or three questions!

Many so called Expert Reports have a high Barnum Effect* content - so take care!

*To find out more about the Barnum Effect please

see the Potential for Change and Self-Improvement Test.

To measure the value of an Expert Report we recommend that you conduct the following quality check:

• Ask the person selling you the Expert Report what norm group forms the basis of their ‘look-up table’. Is that norm group appropriate to you?

• Ask if that norm group, the ‘look-up table’, can easily be altered to meet your particular and changing needs.

• Underline the first 10 statements in the report that refer to the future behaviour of the person who completed the test. Then, ask the report seller what specific questions and scores triggered-off those assumptions. Does it seem a realistic and reasonable number?

• Read the Expert Report carefully. Does it have a Barnum Effect feel about it? Is there just too much copy? Does it go on and on out of all proportion to the number of answers given by the candidate? Are too many fancy graphics used to illustrate simple data?

• Ask yourself, do you really need an Expert Report to understand the implications of a comparative test score?    

 Return to Index

Look-a-Like Tests

 It can be completed anonymously and it is scored online. If you would like to see how you score go to:
The Ubiquitous DISC Test

If you would like more information about this type of test, please contact us: DISC@mtselect.co.uk 

Do you need to send your people on a training course?

So you have to ask yourself - what sort of business are they in? 

Are they providing a service that helps you to choose and use the test/tests most appropriate to your specific and changing needs?  

Or are they just training you to use their test?

Will they help you validate the usefulness of their test before you spend time and money on training?

Remember – if a person who has been licensed or registered to use a particular test leaves your organisation, you could be left ‘high and dry’. You are then restricted to hiring someone who is also licensed or you will have to pay up to £4,500 for the replacement to be trained.

Will a couple of good books, and some study time, be a more cost effective way of learning about how to use tests properly?

In our opinion, virtually all HR specialists and most managers who are experienced at interviewing and developing people, should be able to interpret and use test scores without the need for special training.

If they do need help most good test suppliers should provide coaching and have a client support program.

 Return to Index

1 of 4.  What’s New?

The Big Five

Comparatively new in terms of understanding and describing personality is the Big Five. Over recent years, the views of many personality psychologists have converged regarding the structure and concepts of personality. Generally, researchers agree that there are five robust factors of personality which can serve as a model for accurately classifying and describing personality.

The Big Five factor model has important implications in the field of personnel psychology. These five, relatively independent factors, provide a model that can be used to accurately study, describe and understand differences between individuals.

You might find the mnemonic OCEAN helpful in remembering the Big Five.

The Big Five

Openness to Experience

Sometimes called intellect, culture, even intellectence.

Traits commonly associated with this factor include: being imaginative, cultured, curious, original and sensitive.

Conscientiousness

Common traits associated with this factor include: being careful, thorough, responsible, hardworking, achievement striving and persevering.

Extraversion

Traits frequently associated with extraversion include: being sociable, gregarious, assertive, talkative and active.

Agreeableness

Some psychologists have labelled this factor differently.  They use such terms as likeability, friendliness, social conforming, compliance versus non compliance. Traits associated with this factor include: being courteous, flexible, trusting, good natured, co-operative, forgiving, soft hearted and tolerant.

Neuroticism

Sometimes referred to as Emotional Stability.

Common traits associated with this factor include: being anxious, depressed, angry, embarrassed, emotional, worried, and insecure.

 Return to Index

OCBT

(Don’t you just love those abbreviations !)

OCBT is the abbreviation used for ‘online computer based testing’.

Online computer based testing offers many advantages - 

• Reduced testing time.

• Increased security.

• Immediate scoring and feedback.

• Increased options for scheduling testing time.

• A better standardisation of testing conditions. 

• More flexible options for test construction; etc. 

• OCBT offers a very significant reduction in testing cost.

As a recent HR article reported - 

"Online testing is proving to be a wonderful logistical benefit, particularly in national and international recruitment campaigns. But the quality of assessment and the accuracy of prediction still depends on the instruments themselves. Online administration doesn’t turn a poor test into a good one. The danger is that people may be fascinated by the digital technology at the expense of the psychometric quality."

 Return to Index

3 of 4.  What’s New?

Measuring the difference

One personality factor has been clearly established - by a recent meta-analysis of almost 30,000 test results and performance judgements - to impact on all work activities - on all competencies - at all levels.

This personality factor has been described as the difference between ‘Can Do’ and ‘Will Do’.

Most companies still neglect to measure this area of personality in depth !
[Thousands of people visit this site - including job seekers.  Therefore we cannot be more specific about this personality factor here and now!]

4 of 4.  What's New?

'Psyc info'

Competitive edge and life-style news.

An occasional email newsletter for people who are interested in keeping up-to-date with the world of psychology and how it can impact positively - and negatively - on business and every day living.  

Join our emailing list and keep up-to-date 
psych@mtselect.co.uk

 Return to Index

Now find out how to assess and develop people more effectively: 

testmore@mtselect.co.uk

Back to MTS Home Page