The popularity of ChatGPT, a language model developed by OpenAI, has been on the rise among MBA applicants for its remarkable ability to produce persuasive and coherent text. In this episode, John, Maria, and Caroline delve into the advantages and disadvantages of using this technology in the admissions process, while also addressing the potential ethical concerns that may arise.

Our hosts also provide a broader perspective on the changing landscape of MBA admissions, highlighting the role that technology is playing in shaping the future of the industry.

Whether you’re an experienced business professional, a prospective MBA applicant, or simply intrigued by the intersection of technology and education, this episode of Business Casual is a must-listen. Tune in to join an engaging and informative conversation on the use of ChatGPT in MBA admissions.

Episode Transcript

[00:00:07.210] – John

Well, hello, everyone. Welcome to Business Casual, a weekly podcast of Poets and Quants. I’m John Byrne, the editor of P and Q. And with me are my co hosts, Maria Wich Vila and Caroline Diarte Edwards. Maria, of course, is the founder of Applicant Lab and Caroline is the board former admissions director of INSEAD and the co founder of Fortuna Admissions. You will have been buried under a rock. If you haven’t heard about ChatGPT, there has been a tremendous number of stories written on this artificial intelligence ChatGPT, and we can tell you, we can do really amazing and incredible things. On a recent visit to Goizueta Business School, I sat down with the admissions director, Melissa Rapp, who, out of curiosity, started typing into ChatGPT the leadership question that Goizueta asked of all its MBA applicants. Her conclusion what came back was pretty good. Sure, it felt a bit canned and there was nothing personalized about it, but it was pretty darn good. It was organized well, it was well written, and it’s going to be a problem, I think, for a lot of admissions officers who have weighed essays in an important way to judge the qualifications of a candidate for business school.

[00:01:38.750] – John

So one of the things that we’ve done, we’ve asked Maria and Caroline to play with ChatGPT. I’ve played with it as well, and we’re going to tell you what we discovered. Maria, why don’t you go first? What question did you ask ChatGPT?

[00:01:55.050] – Maria

Sure. So I decided to ask it what I think is probably one of the hardest, if not the hardest, essay question in MBA admissions, and that is Stanford’s. What matters most to you and why? Admittedly, I started a bit too philosophical. I asked ChatGPT to tell me what should matter most to me, and it very rightfully pointed out that I am simply an AI. But it actually gave me he’s like it. It actually said, well, here are some ways that some people think about prioritizing things in their lives. And I was like, wow, this is very therapeutic. I think that admissions officers and therapists maybe should start to be concerned about this potential. But so anyway, I then asked it more specifically, okay, how should I write the essay for the Stanford thing? When I first gave it just very vague information, I got back what I would call a cotton candy answer. It was very sweet, but ultimately very empty, so it sure sounded nice. It had a lot of good things. I told it, what if community matters most to me? And so it came back with only let me see if I can find it.

[00:03:02.170] – Maria

We are all interconnected when one person suffers, we all suffer when one person thrives. We all benefit. By investing in our communities, we create a ripple effect of positivity. So, hey, look, ripple effects of positivity. Sounds great. The thing is that there were no actual examples there was nothing concrete. Like I said, it was essentially cotton candy. Like it looks sweet from far away and it is at first, but then when you dig into it you’re like, well there’s nothing here, it’s just air. So then I decided to give it more details about me, sort of a hypothetical me. And I created a composite of a sort of a typical applicant to Stanford, someone being raised in a military dictatorship, who worked with the government and protested the government and now is a sustainability expert and has done all this stuff with sustainability. And then I asked it to write the essay around that. What it did. It did some things well and it did some things that obviously it couldn’t possibly do. What it did do well was it took the information that I gave it about myself and it did structure an essay around those things.

[00:04:06.000] – Maria

It actually put together a few connections that even I had not told it to make. So for example, one of the things I told it was that I had led protests against the government when I was in college, but I didn’t say any, and that I risked getting arrested for doing so, but I didn’t say anything else. And then ChatGPT filled in the details of I knew that silence would only perpetuate the problems that plagued my community and through this I inspired others to join the fight for change. So I had not said anything in my prompt about inspiring others and silence was not an answer, but it somehow was able to make that conclusion on its own and I thought that was actually really impressive. And the other thing was that the details I gave it, the original thing I said to it was what matters most to me is my community. But then the details I gave it were more aligned with someone who is committed to a career in sustainability. And then at the end what it did is it very smartly tied together those two things, that creating positive change in my community is what matters most to me.

[00:05:16.840] – Maria

And the way I think I should do this is through sustainability because without the environment, whatever the environment is really important for all of us. And so I thought that was really smart. Now obviously what it couldn’t do is it didn’t provide any deeper information beyond what I had given it. The other thing I will say, so that’s sort of a downside. But the other thing I will say also is that when I asked it just to just rewrite it and rewrite it and rewrite it every time it said the same things, but it was able to phrase it in slightly different ways. So I thought that that was really interesting, that it’s very fluent with taking a concept and writing one sentence that expresses that concept well and then writing a completely different sentence that also expresses that same concept well, just with rearranging words and choosing synonyms and what have you. So overall I actually think it’s a very good tool for actually writing something out. Where I think it fails is it doesn’t really provide any inner, deeper insight into someone’s thought process or their values or anything like that. But of course it can only work with what it’s given.

[00:06:32.990] – John

Nonetheless, were you surprised at how good it was or disappointed?

[00:06:39.970] – Maria

I have been surprised. I’ve been playing with it for a couple of weeks and the first time I used it for something I thought, oh wow, within moments it comes back with a fully fleshed out, grammatically correct, well worded essay.

[00:07:00.150] – John

You wonder if an admissions official who spends I guess the average time spent in a first read is only like, I think twelve minutes or so. So an admissions official with a pile of application essays in front of them, having to get through, all of them rushing through. If you provide enough answers and background to ChatGPT, would that person really be able to tell that it was created by a ChatGPT, do you think?

[00:07:31.170] – Maria

I mean, I think the thing it definitely lacked was sort of a personal or personality or a voice, so it was pretty dry. So I do think that applicants going forward should try to even harder to incorporate some aspects of their personality into it because it was a little bit sort of academic in its writing. I don’t know if an admissions officer will be able to tell. I think this is why the interviews are so important and I think it’s a shame that the new GMAT is getting rid of the analytical writing assessment because I just feel like when you actually see how someone writes under a time pressure you’re going to get a much better sense of who they actually are. Whether or not the admissions officer can tell or not, I don’t know. But I also think that it’s not, as you yourself have quoted, Dee Leopold, the former head of admissions for Harvard has said, and we’ve all repeated amongst ourselves many times, it’s not an essay writing contest. The admission does not go to the person who writes the most beautiful prose. The admission goes to the person who has accomplished the most impressive things in the best way.

[00:08:39.820] – Maria

So whether or not we can tell if it’s written by the person or written by their cousin or written by ChatGPT, at the end of the day I think it’s going to be what they’ve accomplished that’s going to matter most in terms of making that assessment.

[00:08:53.910] – John

Yeah, very true. Now Caroline, you fed it one of the INSEAD questions, right?

[00:08:58.990] – Caroline

Yes, I did. So I fed it the candid description essay from INSEAD. So the school asks the candidate to give a candid description of yourself, stressing the personal characteristics you feel to be your strengths and weaknesses and the main factors which have influenced your personal development. So what I found is very similar experience to Maria, so I only got a sensible response once I started giving it more and more details. Right. So you have to have figured out what are the key elements of your story and what are the key points you want to get across before it’s able to draft anything that is usable. So the point where I found it most useful was when I took a rough draft, fed that in, and asked it to improve the style and reduce the word count. And then it spat something out that had INSEAD had a smoother style and it had reduced the word count, and it did that very quickly. I also found that the style was very bland, as you’ve both said. So definitely it read like something that lacked the individual personality. And so that’s not something that I would want to submit to a school by any means.

[00:10:14.150] – Caroline

So I think it can be useful tool at a certain point in the process. But until AI can stare into your soul and tap into your memories and your life experience, it’s not going to be able to tell you. What are the things that you should be telling Business School and what is relevant about your past experience and what are the key achievements that Harvard Business School or INSEAD are going to be particularly interested in. And that’s one of the things that, as coaches, we spent a lot of time on, is understanding a candidate’s background and delving into that and helping them understand what are their key strengths, what are their weaknesses, how do they showcase their strengths, and how can they effectively mitigate their weaknesses. And once you have that understanding yes. Then maybe this can play a role in helping you develop some of those drafts. And it could be useful in that process. But I still think garbage in, garbage out, right? You have to know what you’re doing, and there has to be some intelligence in the prompt, I think, that you’re giving it. And you have to understand the context of what sort of response you’re looking for, because there are all sorts of disclaimers on this tool.

[00:11:35.760] – Caroline

Right. There’s no guarantee that what it’s generating is accurate, or there could be all sorts of issues with it. You’ve got to bring your own intelligence to it, especially if the stakes are high, which they are when you’re preparing your application for business school.

[00:11:58.490] – John

Very true. I’m interested in the difference that you pointed out between editing and writing. So when you asked it to improve an existing draft, you thought it did a pretty decent job.

[00:12:12.170] – Caroline

Yes, I did. Although it was good at cutting down the word count, it made some awkward phrases less awkward. It had some nice turns of phrase. So I did find that useful, I think, particularly for a non native English speaker, that could be quite useful in the process. But then again, it did read the word that came to mind when I read it was bland. Right?

[00:12:42.770] – John

Yeah, exactly.

[00:12:44.630] – Caroline

And as Maria said, it’s not an essay writing competition. So the admissions officer and if you’re not a native English speaker and you make the occasional grammar mistake, or you have a turn of phrase that is not perfect, that’s fine. As Maria said, what they’re concerned about is your track record and your potential. Right. And they’re less concerned about whether you are able to write a sparkling essay. And so I don’t think people should get too hung up on the expression of how that’s put down on the paper. The key thing is what are the important elements of your story that you want to get across to business school and how do you want to convey that?

[00:13:27.970] – John

Now, since you have read, I would guess, tens of thousands of essays, both as the director of admissions at INSEAD and as a consultant, I wonder if you put your admissions director hat back on. Do you think you could be fooled by ChatGPT?

[00:13:47.130] – Caroline

Well, the application has various elements. Right. And that’s one of the reasons why schools have not just essays, but they also have interviews and now they have video questions, and there’s the GMAT and other recommendations. And one of the reasons they have these different elements is they are like pieces of puzzle that come together, and there needs to be some coherence in that. And so if the essays are perfectly written, but then the verbal GMAT is poor, very poor. Right. Or there are concerns expressed by the interviewer, or there is a video response that isn’t fantastic, they will be looking at how that cross checks. It may be feasible that, as you say right. You cited the admissions director from Emory and how it was able to spew out pretty impressive career goals. So it could be that for some element of the application, it would be able to come up with something that would be a useful addition to your application, but it can’t fake the whole thing, I think. And there are cross checks in that process that I think will still be valuable. Having said that, I think the schools are struggling to figure out how to deal with this.

[00:15:07.940] – Caroline

And we’ve reached out to a number of schools and they’ve said, we’ll figure it out. We’ll get back to you later. I think they’re all not quite sure how this is going to play out right now, so we’ll wait and see how they respond. But I think there is a lot of concern about this, and I don’t think they have figured out their policies yet.

[00:15:29.520] – John

Yeah. And Melissa Rapid quizueta said that of course they’re going to still look at essays and count them, but she is inclined to put more weight than has been put in the past on the face to face interview and their verbal essay question, which has to be answered on the spot in 1 minute, in front of a camera, where obviously ChatGPT can play no role at all. I assume. Although one would think you could just type the darn question in and out. You could read it off the screen in another window. But clearly I think that would also run into the same problem that we’ve been talking about. You’re going to get a bland cotton candy kind of answer that’s not very personalized and as soon as someone recognized that it’s that canned, it’s going to get your application tossed in the waste basket pretty quickly. Now, just like Caroline and Maria did, I picked an essay question as well to see what would happen and I decided I wasn’t going to give the ChatGPT any information about me and I was going to ask it my favorite MBA application essay question, which is Duke’s Fuqua’s goal of Business 750 Word question.

[00:16:55.910] – John

Share with us important life experiences, your hobbies, achievements, fun facts or anything that helps us understand what makes you who you are and list them in 25 different bullet points. So what was kind of interesting is ChatGPT just instead of refusing to answer and saying I don’t know enough about you to answer the question properly, it made stuff up. It created an entirely fictional portrait of who I am. And I have to say I kind of admired this person. I’m going to read some of the things that it wrote. Growing up in a multicultural household, I learned to appreciate and respect different cultures and ways of life. Now that’s really right up the alley and business school admissions these days. Another one. One of my biggest hobbies is hiking. I enjoy exploring new trails and pushing myself physically. Or how about this one? I have a strong interest in sustainability and have worked on several projects to promote eco friendly practices in my community. Or this one I am a certified scuba diver and love exploring the underwater world. Now of course, if anyone submitted something like this without having given ChatGPT any information about yourself and your interest, I think that you would pretty quickly be discovered as a fake.

[00:18:33.330] – John

There are so many incredible things here like I am a licensed pilot and enjoy flying small planes in my spare time where I am fluent in three languages english, Spanish and Mandarin. I imagine if you submitted that and then you showed up and were admitted, it would be pretty damn embarrassing, wouldn’t it?

[00:18:54.330] – Maria

Especially when that plane is starting to crash and everyone turns to you to save them and you can’t do it.

[00:19:03.150] – John

The other thing to consider here is that this is really like the first iteration of this ChatGPT. There is a competitive race now going on among Google, Microsoft. The company that obviously put this out will be Facebook as well, and other players. And that competitive race will likely result in fairly dramatic and quick improvement in what a ChatGPT can produce. And so I wonder a year from now, while we may be thinking that these essays answers are bland and cotton candy and kind of boring and almost academic, I wonder a year from now if you include just a few sentences about yourself, how well they might be structured and portrayed. And I wonder if, frankly, some of them could be quite compelling. It doesn’t mean we’re not endorsing the view that a candidate should use this for applying to business school. I think that would be a huge mistake, but it is surprising how good it is already and I think it’s only going to get better. And then the issue is, what impact will it have on admissions long term? Caroline, what do you think? Do you agree with Melissa that schools are going to not eliminate essays, but maybe weigh them a little bit less and maybe over index things like the face to face interview video questions, which may become more apparent and more common across the board, instead of just a few schools leaning on your recommendations more your undergraduate transcript, your work experience, and just putting a little less weight on this.

[00:20:58.180] – John

Or do you think it’s not going to make a difference?

[00:21:00.770] – Caroline

I think that the video questions could become more widespread. I think the schools that use those already find them very useful and we’ve discussed that in the past, that they give a wonderful glimpse into the candidate and how they think and how they communicate, and that’s difficult to fake for the time being. So I wouldn’t be surprised if those become more widespread. Perhaps we will see fewer essays in the future. I mean, many of the schools don’t have that many essays already in Sierra has quite a few essays, but it’s one of the outliers. So the schools already are relying on multiple elements to assess candidates, so I think that they will continue to do that. They’ll just have to be very conscious that they need to cross check things. And it’s a very interesting point that Maria made about how perhaps in the context of ChatGPT and so on, it’s a shame that the GMAT has dropped the essay element, because that could have been a very useful cross check as well for the schools.

[00:22:10.650] – John

Yeah, true. And I bet you at the time that the new test is being created, there was no ChatGPT and this was not an issue, and now suddenly it can be an issue.

[00:22:24.320] – Caroline

Yeah.

[00:22:25.060] – John

Maria, what’s your takeaway on this and how ultimately admission directors will evaluate candidates?

[00:22:32.190] – Maria

Yeah, I agree with Caroline. I have long been a champion of the video essays. I mean, even if you do type into ChatGPT, like, oh, quick, I’ve just been out, I have to give a 32nd answer about a teamwork thing. It’s probably not going to know enough details about you to come up with a good answer in time. And also most people, unless they’ve had significant theater training, I mean, if there’s a ChatGPT answer right in front of them and their webcam, they are reading it, it’s going to be pretty obvious. You can see their eyeballs moving across the page from side to side. And so it’s really obvious when someone is reading something on screen, right? Again, unless they have had a lot of training with theatrical script memorization and performance. So I actually think I would almost advise people right now to almost avoid the temptation to use this because let’s say you are a non native English speaker and you submit a perfectly flawless ChatGPT polished essay, it’s going to look fake. And so, on the contrary, I would almost think that admissions officers might give some points for genuine authenticity if the essay really does reflect what this person communicates like in real life.

[00:23:47.150] – Maria

And so I would actually avoid it, I think, because my worry is that it’s just everyone’s going to sound so sanitized that it will at least at best you might sound bland, but at worst it might be sort of suspicious, like, wait a minute, is this person cheating by using something like ChatGPT? Because like Caroline said, their verbal score on the GMAT wasn’t that high or they only took a few classes in English in college and they didn’t get very good grades or things like that. So I don’t know, I would actually avoid the temptation to use it even for editing.

[00:24:26.470] – John

Yeah, interesting. One of the other things that I did here was to ask the Harvard Business School question, because that’s pretty tricky question in the sense that they’re only asking you for more information that’s not already in your application. And I only gave it a little detail. I said I was an entrepreneur of a digital media company and that’s all I said. And it’s kind of interesting. It even had a salutation it’s a Dear Admissions committee on the answer. And after that it said, I am thrilled to submit my application for the Harvard Business School MBA program and I appreciate the opportunity to elaborate on what I believe makes me a strong candidate for the program. And then it goes on and it says, one of the most formative experiences of my life was growing up in a family of entrepreneurs. From a young age, I was exposed to the challenges and rewards of running a business, and I was inspired by the dedication and hard work that my family put into their work, blah, blah, blah. So it just extrapolated that from me telling that I was an entrepreneur of a digital media company and that I wanted to apply to Harvard and then gave Harvard’s question.

[00:25:54.770] – John

It’s a worrisome tool. You’re right. It’s bland, it’s canned, it’s cotton candy. But the more information you give it, the more specific it tends to get. And I would think that other iterations are going to make essays a very difficult thing to evaluate by admission committees and admission directors in schools. I totally agree with Maria. Don’t use it. It will probably ruin the spontaneity and the genuineness of what you do want to write, so you might want to play with it for other reasons, like tell me what I should eat tonight, or give me a recipe that you’ll think I’ll enjoy or where should I travel on my next vacation. But don’t ask it to answer an essay question where your answer could determine whether or not you actually get into your dream school. Maria and Caroline, you both agree with that?

[00:26:54.470] – Caroline

Sounds good.

[00:26:56.210] – John

And we should ask it okay. If you’re interested in going to business school, what podcast should you listen to? I hope it says Business Casual. All right, everybody. Thanks for listening. This is John Byrne with Poets and Quants. You have been listening to Business Casual, our weekly podcast.