Audio or Video pre-screening questions, enhanced with Play Media question types, provide a powerful method to evaluate candidates' language proficiency. We've developed TalkScore, a rapid and automated English proficiency test, to streamline this process. It seamlessly integrates with any Audio or Video question where candidates respond in English.

How TalkScore Works

When candidates answer an audio or video question, TalkScore analyzes and scores a set out of 6 possible English linguistic parameters:

Fluency: Scores the rhythm, rate, and smoothness of the user’s speech.
Grammar: Evaluation of the grammatical range and accuracy in the user’s response, including phrase and clause complexity.
Pronunciation: Measure of intelligibility and accuracy of the sounds of each word, syllable, and phoneme.
Vocabulary: Score for the lexical range, sophistication, and correct usage of the diverse words in the user’s response.
Comprehension: Measures the relevance of the candidate’s answer relative to the given question or transcript.
Active Listening: Evaluates the candidate’s ability to retain and repeat information given on a transcript.

These are given scores from 0 to 10:

Which parameters are scored depends on the TalkScore Question Type:

Open-ended: Evaluates the response given to a text-based question. Needs at least 30 seconds of audio.
Listen and Repeat: Evaluate the capacity of a candidate to repeat words from a previous Play Media question.
Listen and Speak: Rates a response given to a previously played media that includes a scenario or question.

In the table below, note the parameters being evaluated based on the type of question selected:

Check out our article on TalkScore Question Types to learn more about the three types of questions.

TalkScore Setup

Once your Account Manager has activated TalkScore for your company, you can start using it:

Go to your desired Audio or Video type question
Check the TalkScore box below the Response Language dropdown list.
Choose the TalkScore Question Type.
If you chose a Listening question, include the transcript of the audio your candidates heard.
Click the Timer button, and set the minutes for candidates to answer.
Hit Save. Any upcoming answers will be scored!

TalkScore Candidate Attributes

Applications that answer TalkScore-enabled questions will have certain Candidate Attributes linked to them. Here’s the full list and what each of them means:

TalkScore Question Count: The number of questions with TalkScore activated.
TalkScore Answer Count: The number of questions where TalkScores were generated.
TalkScore Overall: An average of all the TalkScores in the application.
TalkScore CEFR: A conversion of the Overall TalkScore to the CEFR format, according to the following scale:

TalkScore CEFR	TalkScore Overall
A0	0 - 0.9
A1	1.0 - 2.3
A1+	2.4 - 3.6
A2	3.7 - 4.5
A2+	4.6 - 5.9
B1	6.0 - 6.8
B1+	6.9 - 7.3
B2	7.4 - 8.2
B2+	8.3 - 8.7
C1	8.8 - 9.4
C2	9.5 - 10

The scores are generated on a scale from 0-10 and converted to CEFR (Common European Framework of Reference).

Note: Each time we receive a new response for a TalkScore question, we score the answer and recalculate the average shown in the candidate’s profile, which includes the CEFR score attribute.

You will also find the following attributes, which are the average of a given parameter across all the scored answers:

Candidate Attribute	Description	Type of Data	Values
TalkScore Fluency	Measures how smoothly and quickly the candidate speaks	Numerical	0 to 10
TalkScore Grammar	Evaluation of the candidate's proper use of English grammar.	Numerical	0 to 10
TalkScore Pronunciation	Evaluation of the candidate's pronunciation accuracy.	Numerical	0 to 10
TalkScore Vocabulary	Assessment of the variety and relevance of the candidate's word choices.	Numerical	0 to 10
TalkScore Active Listening	Gauges the candidate’s attentive listening skills, comprehension, and information retention.	Numerical	0 to 10
TalkScore Comprehension / Spoken English Understanding	Assessment of the candidate's comprehension of spoken English.	Numerical	0 to 10

Other additional TalkScore-related Candidate Attributes include:

Candidate Attribute	Description	Type of Data	Values
TalkScore Question Count	The count of TalkScore questions given to the candidate.	Numerical	0 to x
TalkScore Answer Count	The count of valid responses from the candidate resulted in a TalkScore rating.	Numerical	0 to x
TalkScore Generated At	The date the candidate finished the TalkScore questions and when the TalkScore was finalized.	Date	Date

You may use these attributes to set up Autoflows or to Filter your candidates.

Q. What are the criteria for generating the TalkScore Overall rating, and when is a response considered valid?

Score will not be calculated unless ALL questions are answered
For answers with errors (No speech is detected, Not answered within time limit, and The answer appears irrelevant to the context of the question), they will have N/A scores and will not be calculated in the overall score.
If ALL Talkscore responses have errors, the overall will be 0, CEFR A0 and all categories overall will be N/A

Q. What are the response length criteria for "Listen and Repeat" and "Open-ended" or "Listen and Speak" questions, and when will a response be disqualified from scoring?

A: "Listen and Repeat" questions do not have a response length requirement. "Open Ended" or "Listen and Speak" questions require a 30-second response length; otherwise, the response will be disqualified from scoring.

Q. Where are data points generated, either at the individual question level or at the overall application level, for TalkScore Overall, TalkScore CEFR, and TalkScore Dimensions (including Fluency, Grammar, Pronunciation, Vocabulary, Active Listening, Comprehension / Spoken English Understanding)?
A: This will depend on the data points.

TalkScore Overall: Generated at the Application Level (i.e., saved as a candidate attribute).
TalkScore CEFR: Generated at the Application Level.
TalkScore Dimensions (Fluency, Grammar, Pronunciation, Vocabulary, Active Listening, Comprehension / Spoken English Understanding): Generated at both the individual Question Level and the Application Level.

Q: What happens if a candidate records an audio for TalkScore questions that is less than 30 seconds?

A: If the recording is less than 30 seconds, an Insufficient Audio Length error will be displayed, and the score will be 0.

Q: What happens when candidates reach the time limit for a TalkScore audio question?

A: Once the time is up, candidates will automatically move to the next question. If there are no more questions, they will see the completion screen.

Q: What if candidates don't start recording before the time runs out?

A: If candidates don't start recording, a No Answer error will be displayed, and the score will be 0.

So you still don't have TalkScore?

To enable it, contact your Account Manager and start the conversation.

TalkScore Powered by Talkpush

Prescreening Question Types

TalkScore Question Types: Listen, Speak & Evaluate

Navigating the TalkScore Report

Fast Assessment Tools for Quick and Effective Candidate Evaluation