MCATSavior The Home of FREE MCAT Strategy Courses

How often do you find that your scores on third party practice passages fluctuate wildly? Even worse, how often do you notice your practice scores diverge from the real exam? 

For once it isn’t the AAMC’s fault. It’s actually just hard for any company to create material that matches AAMC closely and statistically benchmark it. These are the primary reasons why students' scores fluctuate like crazy on practice passages despite consistent performance.

Issue #1 Passage + Question Creation

CARS is subjective and the AAMC is very good at creating subjective questions (their writers are highly trained professionals). Test writers for third party companies(typically not highly trained professionals) have to mimic AAMC as closely as possible.

If they make it too challenging, students will “hate” the company because it's impossible to study their material. 

Too easy and the company will get dinged for not being “representative” of the AAMC. 

The writers have to strike a fine balance and that balance is dictated by their own company policies, their individual CARS skill level, and their experience teaching the MCAT.(We won’t get into details, let's just say many of our co-workers lacked more than one of the above traits….)

As a result, you have practice passages that are created in a wildly different style from company to company, and even passages within a certain company's prep material can all appear stylistically different. 

This tends to throw students off as strategies that worked previously now seem wrong.(This is also why students tend to get the wrong answer but with the correct reasoning). 

There is no easy way to avoid this inherent subjectivity in third party question creation unless students exclusively do AAMC practice. Unfortunately, the AAMC does not release that much material so third party materials are necessary for training. 

Since every company has a distinct style, students who do practice passages from a single source become too accustomed to that style and get better at that company's material over time.

 BUT remember, you want to train to become better at the AAMC

You don’t necessarily  want to train to ONLY become better at third parties

We recommend that to avoid exclusively being exposed to a single third party company's style(and therefore inaccuracies) students rotate practice frequently between different companies to expose themselves to a wider variety of passage/question writing styles.

This can be prohibitively expensive for some students so we recommend using as many free resources as you can( like free FL tests) and purchasing online Q banks to split with your friends as that is often the most cost effective method to get a lot of practice for very little money.

That said, remember that doing a high volume of practice does not directly equal higher scores so make sure you are combining practice with a very thorough review process.  

Issue #2 No statistical benchmark

The AAMC has numerous ways to play with the exam curve and actually grade your raw score. This can include throwing out questions that don’t meet their statistical benchmarks. Those benchmarks aren't public knowledge, but their existence makes AAMC’s scaled scores for practice exams fairly accurate. 

Third party companies don’t use similar benchmarks. The scaled scores they give you at the end of FL tests are relative ranges and vary from company to company. Regarding individual practice passages, those only undergo basic quality measures and rudimentary testing before being released.

Those frustrating questions that you KNEW you got right but the answer key says you're wrong, and all your friends agree with you but now you're just confused?

It's just a badly written question, your reasoning can still be correct. 

Without proper statistical benchmarking, it's hard to remove questions so they will pass through inspection and end up as part of a third party FL test or Q bank. 

There are certainly a lot more badly written questions in Q banks than students would imagine. Some companies are certainly worse than others but on the whole, everyone is guilty to some degree. 

As a student, it's always a good rule of thumb to not only cycle through resources from different companies but also try your best to purchase and use the latest material produced. 

A good deal of the third party material on the market was either written in early 2015 or is recycled pre 2015 CARS material( hey even AAMC uses this recycled material in their Q packs). The test has changed significantly since 2015 and the newer material on the market does a better job of reflecting that. 

In summary:

There's no easy way to avoid third party practice passages. We know third parties aren’t the best but the four foundations of CARS won’t save you without proper practice. 

In addition, no matter how many cool strategy tricks you learn from us regarding 

   Timing skills

    Passage reading skills

    Or question answering skills

If you don’t do sufficient practice you won’t gain the experience needed to use them on test day. 

We believe that until the AAMC tightens up and releases a humane amount of  post 2015 CARS material, third party practice passages will be a necessary evil so students can maximize their growth in the following ways:

1. Make sure you work on CARS fundamental skills such as reading/answering questions untimed before doing timed passages

2. Save the AAMC material for 1-3 months prior to your exam. This material is most indicative of the real exam so you want to “tuned” to it leading up to your exam. Use third party practice passages before that time.

3. Cycle practice companies relatively frequently(every 1-2 weeks) to expose yourself to different passage styles

4. Just “let go” of questions that you SWEAR you were correct on but the answer key disagrees. If you are consistent with reviewing questions, you don’t need to learn from EVERY question. Remember bunk questions do exist.

5. Track trends in progress, not just raw scores. I.e are you applying reading, timing, and question answering skills appropriately and are you improving?

6. When you transition to AAMC material, switch back to doing untimed practice to get used to AAMC’s very distinct “style” then slowly return back to timed passages. This has helped many students during their transition in the last few months of studying.

CARS is always tricky but hopefully this helped shed some light on why your CARS score just seems to fluctuate like crazy. If you are interested in this topic you can check out the full lesson as part of module 0 in our CARS course. 

Be sure to go through all our other free strategy courses as well and hopefully they help give you the edge on test day!