Audio or Video pre-screening questions, enhanced with Play Media question types, provide a powerful method to evaluate candidates' language proficiency. We've developed TalkScore, a rapid and automated English proficiency test, to streamline this process. It seamlessly integrates with any Audio or Video question where candidates respond in English.
How Talkscore Works
When candidates answer an audio or video question, TalkScore analyzes and scores a set out of 6 possible English linguistic parameters:
Fluency: Scores the rhythm, rate, and smoothness of the user’s speech.
Grammar: Evaluation of the grammatical range and accuracy in the user’s response, including phrase and clause complexity.
Pronunciation: Measure of intelligibility and accuracy of the sounds of each word, syllable, and phoneme.
Vocabulary: Score for the lexical range, sophistication, and correct usage of the diverse words in the user’s response.
Comprehension: Measures the relevance of the candidate’s answer relative to the given question or transcript.
Active Listening: Evaluates the candidate’s ability to retain and repeat information given on a transcript.
These are given scores from 0 to 10:
Which parameters are scored depends on the TalkScore Question Type:
Open-ended: Evaluates the response given to a text-based question. Needs at least 30 seconds of audio.
Listen and Repeat: Evaluate the capacity of a candidate to repeat words from a previous Play Media question.
Listen and Speak: Rates a response given to a previously played media that includes a scenario or question.
Check out our article on TalkScore Question Types to learn more about the three types of questions.
TalkScore Setup
Once your Account Manager has activated TalkScore for your company, you can start using it:
Go to your desired Audio or Video type question
Check the TalkScore box below the Response Language dropdown list.
Choose the TalkScore Question Type.
If you chose a Listening question, include the transcript of the audio your candidates heard.
Click the Timer button, and set the minutes for candidates to answer.
Hit Save. Any upcoming answers will be scored!
TalkScore Candidate Attributes
Applications that answer TalkScore-enabled questions will have certain Candidate Attributes linked to them. Here’s the full list and what each of them means:
TalkScore Question Count: The number of questions with TalkScore activated.
TalkScore Answer Count: The number of questions where TalkScores were generated.
TalkScore Overall: An average of all the TalkScores in the application.
TalkScore CEFR: A conversion of the Overall TalkScore to the CEFR format, according to the following scale:
TalkScore CEFR | Talkscore Overall | Remarks |
C2 | 90 | Proficient |
C1 | 85 | Advanced |
B2+ | 80 |
|
B2 | 75 | Upper Intermediate |
B1+ | 70 |
|
B1 | 65 | Intermediate |
A2+ | 60 |
|
A2 | 55 | Elementary |
A1+ | 50 |
|
A1 | 45 | Beginner |
A0 | 40 | Failed/Irrelevant Answer |
You will also find the following attributes, which are the average of a given parameter across all the scored answers:
Candidate Attribute | Description | Type of Data | Values |
TalkScore Fluency | Measures how smoothly and quickly the candidate speaks | Numerical | 0 to 10 |
TalkScore Grammar | Evaluation of the candidate's proper use of English grammar. | Numerical | 0 to 10 |
TalkScore Pronunciation | Evaluation of the candidate's pronunciation accuracy. | Numerical | 0 to 10 |
TalkScore Vocabulary | Assessment of the variety and relevance of the candidate's word choices. | Numerical | 0 to 10 |
TalkScore Active Listening | Gauges the candidate’s attentive listening skills, comprehension, and information retention. | Numerical | 0 to 10 |
TalkScore Comprehension / Spoken English Understanding | Assessment of the candidate's comprehension of spoken English. | Numerical | 0 to 10 |
Other additional Talkscore-related Candidate Attributes include:
Candidate Attribute | Description | Type of Data | Values |
TalkScore Question Count | The count of TalkScore questions given to the candidate. | Numerical | 0 to x |
TalkScore Answer Count | The count of valid responses from the candidate resulted in a TalkScore rating. | Numerical | 0 to x |
TalkScore Generated At | The date the candidate finished the TalkScore questions and when the TalkScore was finalized. | Date | Date |
|
TalkScore Q&AQ. What are the criteria for generating the TalkScore Overall rating, and when is a response considered valid? A: The TalkScore Overall rating will only be generated if there are at least two (2) valid responses from the candidate at the question level. A response is considered valid if a score is assigned to it.
Q. What are the response length criteria for "Listen and Repeat" and "Open-ended" or "Listen and Speak" questions, and when will a response be disqualified from scoring? A: "Listen and Repeat" questions do not have a response length requirement. "Open Ended" or "Listen and Speak" questions require a 30-second response length; otherwise, the response will be disqualified from scoring.
Q. Where are data points generated, either at the individual question level or at the overall application level, for TalkScore Overall, TalkScore CEFR, and TalkScore Dimensions (including Fluency, Grammar, Pronunciation, Vocabulary, Active Listening, Comprehension / Spoken English Understanding)?
Q: What happens if a candidate records an audio for TalkScore questions that is less than 30 seconds? A: If the recording is less than 30 seconds, an Insufficient Audio Length error will be displayed, and the score will be 0.
Q: What happens when candidates reach the time limit for a TalkScore audio question? A: Once the time is up, candidates will automatically move to the next question. If there are no more questions, they will see the completion screen.
Q: What if candidates don't start recording before the time runs out? A: If candidates don't start recording, a No Answer error will be displayed, and the score will be 0. |
So you still don't have TalkScore?To enable it, contact your Account Manager and start the conversation.
|
🤔 Do you have any questions? We want to help! 💚
You can chat with our Product Support team by clicking the intercom button on the lower section of your Talkpush screen. We're one click away.
Need more help? 📖