MATHEMATICS PLACEMENT AND ITS
RELATIONSHIP TO RETENTION
DEAN DE COCK
2.5 Models 48
2.5.1 ACTM 48
2.5.2 ACTM, FG, & LI 48
3.1 Initial Mathematics Course 49
APPENDICES 60
|
Table 1 |
Retention rate for the 1995-2002 cohorts based on first generation status |
47 |
|
Table 2 |
Retention rate for the 1995-2002 cohorts based on family income status |
47 |
|
Table 3 |
Retention rate for the 1995-2002 cohorts based on first math enrollment |
49 |
|
Table 4 |
GORE% (number in cell)- Retention rate for the 1995-2002 cohorts based on ACTM and first math enrollment |
50 |
|
Table 5 |
GORE% (number in cell)- Retention rate for the 1995-2002 cohorts based on ACTM and first math grade. Audits and redundant withdraws have been removed |
52 |
|
Table 6 |
Percentage of students retained by grade achieved in first math class. Note “W” represents students who have no other math grades |
53 |
|
Figure 1 |
Retention rate versus cumulative ACT for the 1995-2002 cohorts |
44 |
|
Figure 2 |
Retention rate versus math ACT sub-score for the 1995-2002 cohorts |
45 |
|
Figure 3 |
Retention rate versus math placement test score for the 1995-2002 cohorts |
46 |
|
Figure 4 |
Cumulative ACT score versus retention rate for the 1995-2002 cohorts separated by students who qualify for scholarship renewal |
54 |
|
Figure 5 |
Cumulative ACT score versus percent of students not returning to Truman for the 1995-2002 cohorts separated by students who qualify for scholarship renewal |
55 |
|
Figure 6 |
Ratio of the percent of students leaving who have a GPA below a 3.25 to the percent of students leaving who have a GPA above a 3.25 for each ACT score |
56 |
CHAPTER 1 INTRODUCTION
The initial goal of this study was to determine if a link exists between retention and the mathematical experiences of Truman students. Recently there has been a large amount of discussion on campus about the Essential Skills and Mode of Inquiry requirements in the area of mathematics. An integral part of successful completion of these requirements is the placement of students into their first mathematics course at Truman. Currently students are placed using a system based on a combination of four pieces of information: ACT math score, placement test score, high school transcript, and self-placement recommendation. This analysis attempted to investigate the relationship between their first mathematics class and their likelihood of staying at Truman.
While no significant relationship was found between placement and retention, due to a large number of confounding factors, several other interesting relationships were discovered. A strong linear relationship exists between ACT and retention, with higher ACT scores coinciding with greater retention. Poor grades in the first math class typically indicate a substantial drop in retention, and this trend is accentuated with rising ACT scores. While these relationships are probably not unexpected, this report gives quantifiable evidence to back common beliefs.
CHAPTER 2 PRE-ENROLLMENT
This section of the report will involve looking at retention based on information that can be obtained about the student before they begin instruction on the Truman campus. This includes ACT scores, math placement test scores, and personal information on the student.
The data set used in this study consists of students enrolling at Truman State University from Fall 1995 through Fall 2002. The information on these approximately 11,000 students was obtained from the Truman ITC department.
Retention in this report will be defined slightly different than the traditional freshman-sophomore retention rate. As this study includes both cohorts that have graduated and others that are currently enrolled, a new measure representing whether a student left the university is required. This measure will indicate that a student has either graduated or enrolled for the next semester (GORE). A student who has not graduated and is not enrolled for the Fall2003 semester, will be considered to have left the university.
2.1 ACT Scores
The analysis begins by comparing retention to the cumulative ACT score. The ACT scores are probably the most reliable data used in this study, as it is a nationally standardized test administered in a controlled environment.
The plot of retention rate versus cumulative ACT (figure 1) indicates a strong relationship between an incoming freshman’s ACT and their likelihood of staying at the university. There is almost a linear relationship from a low of 60% for an ACT of 21 to the high of 100% for an ACT of 36. The trend in the graph is not surprising as one might intuitively believe that students with a higher ACT are going to be more successful in their classes and would be more likely to remain in school.
Figure 1 – Retention rate versus cumulative ACT for the 1995-2002 cohorts.
Back To Top
2.2 ACT sub-scores
An alterative measure of a student’s ability is the ACT sub-scores which indicate strengths in particular fields: Math, English, Reading, and Science. Of these areas, Math (ACTM) is the most interesting for this study.
From Figure 2, we can see that there is once again a very strong linear relationship between the students math score and their likelihood of remaining at the university. It is interesting to note two substantial drops in retention at ACTM’s of 33 and 36. The drop at 36 is most likely due to the relatively small sample size but the drop at 33 may require more investigation as there is a relatively large sample size (n=179). The math ACT appears to be as good at predicting retention as cumulative ACT and possibly may be even better at differentiating student retention as the slope of the relationship appears slightly steeper than that of cumulative ACT.
Figure 2 – Retention rate versus math ACT sub-score for the 1995-2002 cohorts.
As the focus of the analysis is on math, I will only briefly discuss the other ACT sub-scores without displaying their graphs. English has a fairly strong relationship to retention, with better students tending to remain at Truman. It does not differentiate as well as ACT or ACTM, as the relationship is slightly less linear with a range of 55-85%. The science and reading sub-scores show similar relationships, but it is interesting to note a relatively flat slope to their lines. Students with poor ACT scores in these areas have only a slightly lower retention rate than students who have strong ACT scores.
Of interest is why Math seems to differentiate students so much better than Reading or Science (and even slightly more than English) for retention. The most likely reason for this phenomenon is due to the nature of the subjects. Mathematics strongly builds on material learned in previous courses. A student who has a poor/weak high school background in mathematics is likely to struggle in an introductory (or upper level) math course at Truman. In contrast, a student who had a poor quality education in the sciences may be able to catch up in an introductory science course at Truman. While reading comprehension level seems to have almost no relationship to retention, it doesn’t mean that a reading ability is unrelated to their success in college. It may only indicate that students with lower scores have abilities above the threshold necessary to succeed in our classes, or possibly they are able to gain the required skills early in our curriculum.
2.3 Placement Test
A second measure of mathematical ability used is the mathematics placement exam.
This test is administered to all high school seniors via mail. The students are
asked to allocate themselves 2 hours to complete the exam and are requested to
not use books or additional assistance.
Figure 3 – Retention rate versus math placement test score for the 1995-2002 cohorts.
The test is primarily used to determine if the students have mastered the essential skills requirement and are ready for calculus. As this test is measuring approximately the same abilities as the ACTM, we would expect this test to mimic the ACTM results. A review of the graph indicates that this is true, with the PT results showing a strong linear relationship to retention.
2.4 Supplementary material
Though not in the original scope of this grant, it became evident from speaking with several individuals that additional variables may warrant consideration when evaluating retention. The two factors considered are whether a student is a first generation (FG) college student and whether a student is low income (LI).
The data was taken from the CIRP data set and the number of individuals examined is slightly different (n=7600), as these two pieces of information are not available on all students within the cohorts.
|
|
Left |
Retained |
|
First Generation |
34% |
66% |
|
Non-FG |
26% |
74% |
Table 1 - Retention rate for the 1995-2002 cohorts based on first generation status.
The table indicates a substantial difference in retention between first generation students and those whose students who have at least one parent with an advanced degree. The FG students have a one in three chance of leaving while the non-FG students have only a one in four chance of not being retained.
|
|
Left |
Retained |
|
Low Income |
37% |
63% |
|
Non-LI |
26% |
74% |
Table 2 - Retention rate for the 1995-2002 cohorts based on family income status.
A very similar trend is found by looking at the low-income factor. The LI individual is also more likely to leave before graduation with LI having approximately a one in three chance while non-LI have a one in four chance of leaving.
2.5 Models
2.5.1 ACTM
Using the
Math ACT score a simple logistic regression model can be created to predict the
GORE probability of a student. The probability they remain (
) is given by:
.
The coefficients on the equation indicate that a higher ACTM results in a higher probability for retention. Using the equation, we would predict a student with a Math ACT score of 35 to have an 85.6% chance of completing their degree at Truman, while a student with a 15 ACTM would have a 51.8% chance of remaining at Truman.
2.5.2 ACTM, FG, & LI
A slightly more complex logistic model can be created using the FG and LI data along with the ACTM.
All of the coefficients in this model are significant and their signs indicate that retention increases with higher ACTM, while retention decreases if a student is first generation or low income. Using the equation (with 0 or 1 indicating if a student is LI/FG), we would predict a first generation student (who is not low-income) with an ACTM of 35 to have an 83.4% chance of completing their degree at Truman, while a first generation low-income student with a 15 ACTM would have a 39.3% chance of remaining at Truman.
CHAPTER 3 POST ENROLLMENT
This section of the report will involve looking at retention based on information obtained after the student has arrived on the Truman campus. In addition to the ACT and placement test scores are the classes and grades received for all math classes taken at Truman.
3.1 Initial mathematics course
One of the intents of this study was to compare the retention rate for the various classes within the math department. Below are the retention statistics for the courses that have initial enrollments of approximately 100 students or greater. Note that the total GORE rate is 69.1%, which does not represent Truman’s total retention as this data set excludes math courses with minor enrollments and any student who transfers all math requirements.
|
course |
left% |
stayed% |
count |
|
57 |
46.7 |
53.3 |
711 |
|
156 |
34.4 |
65.6 |
4612 |
|
157 |
27.3 |
72.7 |
499 |
|
186 |
30.2 |
69.8 |
1778 |
|
192 |
28.4 |
71.6 |
95 |
|
194 |
20.1 |
79.9 |
463 |
|
198 |
25.3 |
74.7 |
1899 |
|
263 |
19.5 |
80.5 |
548 |
|
264 |
12.9 |
87.1 |
93 |
|
Total |
30.9 |
69.1 |
10698 |
Table 3 - Retention rate for the 1995-2002 cohorts based on first math enrollment.
In general, the retention rate increases with the higher level of math class first attempted by the student. This would be expected from the ACT data reviewed earlier, as students with a higher ACT [and a greater retention rate] will tend to be enrolled in higher-level math classes.
To investigate the effect of placement upon student retention the data must be broken down so that students of similar abilities can be compared. Assuming that the ACTM is the best measure of mathematical ability, the retention rate for each course has been calculated for each ACTM score in the table below.
|
|
57 |
156 |
157 |
186 |
192 |
194 |
198 |
263 |
264 |
|
14 |
100 (1) |
-- |
-- |
-- |
-- |
-- |
-- |
-- |
-- |
|
15 |
25 (4) |
100 (2) |
-- |
-- |
-- |
0 (1) |
-- |
-- |
-- |
|
16 |
50(16) |
56 (9) |
-- |
-- |
-- |
-- |
-- |
-- |
-- |
|
17 |
55 (44) |
55 (40) |
50 (2) |
0 (1) |
-- |
-- |
-- |
-- |
-- |
|
18 |
50 (86) |
57 (103) |
60 (15) |
75 (4) |
100 (1) |
-- |
-- |
-- |
-- |
|
19 |
51 (98) |
58 (199) |
71 (21) |
67 (15) |
-- |
-- |
50 (2) |
0 (1) |
-- |
|
20 |
55 (98) |
65 (268) |
67 (30) |
63 (27) |
0 (2) |
100 (3) |
50 (4) |
100 (2) |
-- |
|
21 |
56 (108) |
64 (385) |
66 (35) |
61 (36) |
100 (2) |
100 (4) |
81 (16) |
100 (2) |
-- |
|
22 |
42 (79) |
64 (451) |
63 (30) |
74 (46) |
100 (1) |
86 (14) |
67 (12) |
-- |
-- |
|
23 |
62 (45) |
64 (565) |
70 (64) |
66 (128) |
75 (4) |
86 (21) |
60 (35) |
100 (3) |
-- |
|
24 |
56 (43) |
68 (564) |
77 (64) |
69 (154) |
60 (10) |
80 (35) |
72 (61) |
80 (10) |
-- |
|
25 |
66 (32) |
69 (575) |
73 (51) |
72 (231) |
73 (11) |
76 (33) |
75 (100) |
79 (14) |
100 (1) |
|
26 |
79 (14) |
67 (474) |
79 (57) |
70 (211) |
50 (14) |
76 (55) |
73 (165) |
70 (23) |
50 (4) |
|
27 |
0 (4) |
69 (336) |
78 (45) |
75 (270) |
73 (11) |
82 (60) |
73 (221) |
85 (33) |
100 (6) |
|
28 |
50 (6) |