Skip to main content
All CollectionsTalkScore
TalkScore User Guide
TalkScore User Guide

Step-by-Step Guide: Setting Up and Understanding TalkScore

Dario Gallo avatar
Written by Dario Gallo
Updated over a week ago

Audio or Video pre-screening questions, enhanced with Play Media question types, provide a powerful method to evaluate candidates' language proficiency. We've developed TalkScore, a rapid and automated English proficiency test, to streamline this process. It seamlessly integrates with any Audio or Video question where candidates respond in English.

How TalkScore Works

When candidates answer an audio or video question, TalkScore analyzes and scores a set out of 6 possible English linguistic parameters:

  • Fluency: Scores the rhythm, rate, and smoothness of the user’s speech.

  • Grammar: Evaluation of the grammatical range and accuracy in the user’s response, including phrase and clause complexity.

  • Pronunciation: Measure of intelligibility and accuracy of the sounds of each word, syllable, and phoneme.

  • Vocabulary: Score for the lexical range, sophistication, and correct usage of the diverse words in the user’s response.

  • Comprehension: Measures the relevance of the candidate’s answer relative to the given question or transcript.

  • Active Listening: Evaluates the candidate’s ability to retain and repeat information given on a transcript.

These are given scores from 0 to 10:

Which parameters are scored depends on the TalkScore Question Type:

  • Open-ended: Evaluates the response given to a text-based question. Needs at least 30 seconds of audio.

  • Listen and Repeat: Evaluate the capacity of a candidate to repeat words from a previous Play Media question.

  • Listen and Speak: Rates a response given to a previously played media that includes a scenario or question.

Check out our article on TalkScore Question Types to learn more about the three types of questions.


TalkScore Setup

Once your Account Manager has activated TalkScore for your company, you can start using it:

  1. Go to your desired Audio or Video type question

  2. Check the TalkScore box below the Response Language dropdown list.

  3. Choose the TalkScore Question Type.

  4. If you chose a Listening question, include the transcript of the audio your candidates heard.

  5. Click the Timer button, and set the minutes for candidates to answer.

  6. Hit Save. Any upcoming answers will be scored!


TalkScore Candidate Attributes

Applications that answer TalkScore-enabled questions will have certain Candidate Attributes linked to them. Here’s the full list and what each of them means:

  • TalkScore Question Count: The number of questions with TalkScore activated.

  • TalkScore Answer Count: The number of questions where TalkScores were generated.

  • TalkScore Overall: An average of all the TalkScores in the application.

  • TalkScore CEFR: A conversion of the Overall TalkScore to the CEFR format, according to the following scale:

TalkScore CEFR

TalkScore Overall

A0

0 - 4.3

A1

4.4 - 4.9

A1+

5.0 - 5.5

A2

5.6 - 6

A2+

6.1 - 6.6

B1

6.7 - 7.1

B1+

7.2 - 7.7

B2

7.8 - 8.2

B2+

8.3 - 8.8

C1

8.9 - 9.3

C2

9.4 - 10

The scores are generated on a scale from 0-10 and converted to CEFR (Common European Framework of Reference).



You will also find the following attributes, which are the average of a given parameter across all the scored answers:

Candidate Attribute

Description

Type of Data

Values

TalkScore Fluency

Measures how smoothly and quickly the candidate speaks

Numerical

0 to 10

TalkScore Grammar

Evaluation of the candidate's proper use of English grammar.

Numerical

0 to 10

TalkScore Pronunciation

Evaluation of the candidate's pronunciation accuracy.

Numerical

0 to 10

TalkScore Vocabulary

Assessment of the variety and relevance of the candidate's word choices.

Numerical

0 to 10

TalkScore Active Listening

Gauges the candidate’s attentive listening skills, comprehension, and information retention.

Numerical

0 to 10

TalkScore Comprehension / Spoken English Understanding

Assessment of the candidate's comprehension of spoken English.

Numerical

0 to 10

Other additional TalkScore-related Candidate Attributes include:

Candidate Attribute

Description

Type of Data

Values

TalkScore Question Count

The count of TalkScore questions given to the candidate.

Numerical

0 to x

TalkScore Answer Count

The count of valid responses from the candidate resulted in a TalkScore rating.

Numerical

0 to x

TalkScore Generated At

The date the candidate finished the TalkScore questions and when the TalkScore was finalized.

Date

Date

You may use these attributes to set up Autoflows or to Filter your candidates.


TalkScore Q&A

Q. What are the criteria for generating the TalkScore Overall rating, and when is a response considered valid?

A: The TalkScore Overall rating will only be generated if there are at least two (2) valid responses from the candidate at the question level. A response is considered valid if a score is assigned to it.

Q. What are the response length criteria for "Listen and Repeat" and "Open-ended" or "Listen and Speak" questions, and when will a response be disqualified from scoring?

A: "Listen and Repeat" questions do not have a response length requirement. "Open Ended" or "Listen and Speak" questions require a 30-second response length; otherwise, the response will be disqualified from scoring.

Q. Where are data points generated, either at the individual question level or at the overall application level, for TalkScore Overall, TalkScore CEFR, and TalkScore Dimensions (including Fluency, Grammar, Pronunciation, Vocabulary, Active Listening, Comprehension / Spoken English Understanding)?
A: This will depend on the data points.

  • TalkScore Overall: Generated at the Application Level (i.e., saved as a candidate attribute).

  • TalkScore CEFR: Generated at the Application Level.

  • TalkScore Dimensions (Fluency, Grammar, Pronunciation, Vocabulary, Active Listening, Comprehension / Spoken English Understanding): Generated at both the individual Question Level and the Application Level.

Q: What happens if a candidate records an audio for TalkScore questions that is less than 30 seconds?

A: If the recording is less than 30 seconds, an Insufficient Audio Length error will be displayed, and the score will be 0.

Q: What happens when candidates reach the time limit for a TalkScore audio question?

A: Once the time is up, candidates will automatically move to the next question. If there are no more questions, they will see the completion screen.

Q: What if candidates don't start recording before the time runs out?

A: If candidates don't start recording, a No Answer error will be displayed, and the score will be 0.


So you still don't have TalkScore?

To enable it, contact your Account Manager and start the conversation.


🤔 Do you have any questions? We want to help! 💚

You can chat with our Product Support team by clicking the intercom button on the lower section of your Talkpush screen. We're one click away.

Need more help? 📖

Did this answer your question?