Can ChatGPT crack SHRM-SCP?
At the intersection of Artificial Intelligence and Human Resources!
Most of us are aware of ChatGPT. It’s a revolutionary AI model which can reason over world’s knowledge until 2021 and provide answers over a chat-like interface. In the last few weeks, this model was able to pass the Wharton MBA exam, US medical licensing exam and a few others listed here.
Having recently been certified as an SHRM-SCP professional, I ran ChatGPT through quizzes from the SHRM curriculum and it was able to get 85% accuracy within no time. What’s even better is ChatGPT not only provided the correct answer but also provided a reasoning why a choice was correct.
Experiment Setup
As an initial experiment, I considered the chapter - “Competencies - Leadership” for testing. SHRM online material provides quizzes for each chapter and each quiz typically contains around 15-25 questions. I manually copy each question from the SHRM portal, paste it on the ChatGPT OpenAI portal and ask the chatbot to pick one of the choices for each question.
Here are a few examples:
Example 1
Question from SHRM Quiz
Answer from ChatGPT
It’s impressive to see ChatGPT giving such an elaborate reasoning here.
Example 2
Question from SHRM Quiz
Answer from ChatGPT
It’s so damn impressive that ChatGPT is able to reason why the other options are incorrect. This is massive for test takers to develop critical thinking and improve their test performance.
Now, let’s look at an example where ChatGPT got it wrong.
Example 3
Question from SHRM quiz
Answer from ChatGPT
I would argue that this is a close call between option 2 and option 4. This example is a reminder that human - AI model is a better model than just AI itself.
For those who are curious, you can find all the questions and answers here.
Takeaways
This is game changing for a few reasons:
For SHRM test takers: SHRM is notorious for not providing reasoning for situational judgment questions and I personally had a difficult time debugging my approach to problem solving. ChatGPT would have been a lifesaver had it been released in early 2022.
For HR professionals: The incredible accuracy of ChatGPT means it is a valuable copilot to all the HR professionals. With the rapid advances we are seeing in Artificial Intelligence, some of our current skills in the AI Copilot world are being commoditized (Yes, I do mean the present-continuous tense!). We will need to discover new ways of working together with AI and do more in faster and cheaper ways.
For HR leaders: They need to acknowledge that these tools can dramatically improve efficiency of an HR organization and should encourage the usage of these tools in our daily work. (What exactly do I mean? That’s for another post!) This allows the organization to move from operational excellence to adding strategic value to business objectives.
For SHRM test administrators: With the changing HR landscape in an AI Copilot world, new skills will need to be tested to upskill a generation of HR professionals and prepare them for real-world situations.
Next Steps
Improve performance: I did not prompt the model to behave in any particular way and still ended up with 81% performance. As the topic is “Competencies - Leadership”, we could use prompts which direct the model to behave in ways that are expected of leaders, I anticipate improved performance.
Full SHRM mock test: Instead of using one quiz, I’m curious to see how ChatGPT would perform on a full length mock test.
It’s an exciting time in the technology world where almost every domain and sector are being accelerated by new AI capabilities. 2023 is going to be an amazing year!
Lastly, this post is manually written but edited by ChatGPT. It has already become my Copilot. Thank you, OpenAI!