Enhancing the College Algebra Advising Process Through a Dynamic Interpretation and Application of the ACT Sub-Scores

Colleges are encountering increasingly difficult circumstances when placing students in introductory, college-level mathematics courses. Attempts to revise the process have met with limited success. Furthermore, colleges are facing thousands of dollars in lost revenue due to student attrition in developmental and college-level mathematics courses. Many of these courses carry attrition rates that are higher than their counterparts in the humanities by factors of two or more. As a result, college administrators are looking for new ideas to reduce the financial losses to the institution and mitigate adverse academic consequences contributing to students failing to return to school due to lost financial aid and low GPAs. Advising students for college algebra coursework can be problematic when a student’s placement indicator is near the cutoff criterion for institutions using the American College Testing (ACT) results for placement. The cutoff criterion for most institutions using this method of placement is 18 or 19 on the mathematics sub-score. During the placement of students based on a quantitative interpretation of the ACT mathematics sub-score alone (i.e., the math-only model), the accuracy of an outcome prediction sharply decreases when the sub-score drops below 19. The falloff occurs in the number of failing predictions which turn out to be erroneous when the score is below 19. Utilizing the currently accepted methodology, 87% of students predicted to pass college algebra have outcomes consistent with this prediction of the model. However, when the model predicts a failure for the student, only 31% of these cases have failing outcomes (McEwen, Mohn, Herron, & Shelley, 2018). This study presents two considerations as an alternative to relying solely on the math sub-score and is based on an expanded indicator which utilizes two ACT sub-scores collectively to assist the adviser in considering placement of borderline students into developmental/intermediate algebra or college algebra. Furthermore, the adviser may use the results as a numerical indicator to potentially enhance the confidence of the student in their mathematical ability.

The expanded indicator focuses on students placed into remedial coursework by the standard placement model through a categorical classification of the mathematics and English ACT sub-scores, based on previous literature findings pertaining to the English sub-score (Case, 1987; Hatch, 1981). It capitalizes on the predictive capacity of the English sub-score as noted by Hatch (1981). In conjunction with the current primary placement mechanism which utilizes the ACT mathematics sub-score, this expanded indicator may demonstrate considerable potential benefit. Students entering their first college-level mathematics course are often accompanied by extreme anxiety. The adviser may find difficulty in guiding a student beyond a preconceived notion of mathematical inability. Numerous secondary placement parameters exist; however, the student and adviser may feel overwhelmed trying to interpret multiple reported statistics. The advising process may be further complicated by factors such as the time constraints placed on the advising process due to faculty schedules and the student’s perception of the adviser as a mathematical expert. This only serves to reinforce a feeling of mathematical inability for the student. This study provides the adviser with a rapidly obtained secondary parameter to guide placement and reduce errors in settings where other known secondary placement criteria are not readily accessible. As an additional benefit, the expanded indicator also provides a single number, easily interpreted by the student, which may potentially reinforce a positive outlook on mathematical ability. Because percentages are readily understood at an introductory level by the general populace, their utilization in the advising process may capitalize on this understanding to potentially promote student success in the college algebra course.

The use of numerous indicators in advising students for college algebra coursework is not a new idea. Much of the recent literature has centered on an ever-increasing list of variables to improve the predictions of the college algebra course outcome (McEwen, 2016). Efforts to produce a viable, multivariate method of outcome prediction for these students began in the early twentieth century with the work of Orleans (1934) and his attempt to correlate IQ, arithmetic grades, and prognosis test results of late 1920s undergraduate students to the course outcome in college algebra. Since that time, numerous attempts have produced a wide range of reported results regarding the ability to predict the outcome of the college algebra course, to determine the probability of success in the course, to generate comparative classification equations, or to simply determine the identity of various indicators of success or non-success in college algebra. These efforts have resulted in an extensive list of similar equations and variables. A minimum of 84 different equations, composed of 93 variables, have been identified in the literature (McEwen, 2016). This number does not include the extensive list of variables and equations presented in Graybeal’s (1958) dissertation or Cauthern’s (1979) equations containing 65 variables each. When advising a student, the use of exhaustive lists of variables and equations is impractical due to time and information constraints. Furthermore, during large student enrollment periods, time available for the advising process is typically limited and updated computer records with secondary parameters, as reviewed in McEwen (2016), could be difficult to access. Student recollection and recitation of these parameters is a subjective source as the number of high school mathematics courses completed and the subsequent outcomes are not readily recalled by most students, and the self-assessment of mathematical ability is not objective evidence.

A source of readily available, objective, easily accessed information exists in the reporting of the ACT sub-scores during college admissions. The acquisition of standardized testing scores such as the ACT are typically a prerequisite to admission and advising. However, the general trend is to apply one sub-score to a specific subject area. Furthermore, the mathematics and English ACT sub-scores have received uneven attention in the literature. Though the English sub-score has been identified numerous times as being related to outcomes in college algebra coursework, the predominant focus in mathematics placement has continued to be the mathematics sub-score. The combination of the English and mathematics sub-scores has also been shown to be correlated to success in college algebra (Case, 1987; Hatch, 1981). Overall, this supports a previously reported relationship of college algebra outcomes with the American Council on Education English Test (Seigle, 1954). The remaining sub-scores, science reasoning and reading comprehension, have neither been identified nor addressed in the literature despite some results suggesting the reading comprehension sub-score is a related indicator of college algebra coursework success (Byrd, 1970; Seigle, 1954). This study focuses on the pairing of the mathematics and English sub-scores as an expanded indicator of readiness for the college algebra course to assist advisers in placing students and to reduce the need to access substantial amounts of information or consume unreasonable quantities of increasingly depleted faculty time, while simultaneously attempting to reinforce students’ self-perception of mathematical ability.

Sample

The total sample comprised 1,941 student records from community college students taking a college algebra course during the 2014-2015 and 2015-2016 academic years. This sample was subject to the restrictions that the course was taught in a traditional format and conducted during the fall or spring semester and that the student was not a course repeater. Of these records, 1,266 records showed that students completed the course and received a grade on the academic transcript. Since the goal of this interpretation is to provide readily accessible, objective information, the incorporation of other demographic information is not indicated. The incorporation of this information links to current issues with multiple parameter placement, which have demonstrated little informative benefit to the students, advisers, or institutional processes. The inclusion of other demographic information may also erode the goals of developing a placement methodology that is independent of social constructs and exhibits improved student outcomes while reducing the necessary time allotments. Based on these goals, demographic data beyond the formatting of the course are not included.

Method

The sample of students completing the course was analyzed using a categorical pass/fail prediction based on the mathematics and English ACT sub-scores and the actual course outcome. Classifications of the independent variable were based on the minimum ACT sub-scores necessary for placement in the corresponding beginning college-level course, consistently regarded to be college algebra. In this study, the college readiness levels of the sub-scores were 19 for mathematics and 18 for English. The dependent variable was categorized based on the grade necessary for acceptable transfer of the course to a university. In this sample pool, university transfer required a minimum grade of C to award transfer credit. A pass is denoted as obtaining a minimum grade of C in the course. Using the binary pass/fail classifications, four sub-classes were developed based on combinatorial groups. The nomenclature for the expanded indicator classes was standardized as two-letter variables of the ACT sub-scores read left to right as Math-English. In this case, PP (M ≥ 19, E ≥ 18) would indicate college ready ACT sub-scores in mathematics and English. The primary emphasis of the analysis centers on the FP sub-class as this is the source of a large percentage of the placement errors in the standard placement model. Sensitivity and specificity comparisons between the math-only model and the expanded indicator will be used to analyze the utility and context applicability of the two models.

General Results

The interpretational basis for the results arises from the standard placement method, the math-only model. The general classification results of the math-only model shown in Table 1 are mathematically equivalent to the numerical math-only model.

Table 1

The probabilities are equivalent to previously reported results (McEwen et. al, 2018) and the probabilities are consistent with expectations currently used in advising college algebra students. Each parent class from the math-only model contains a sub-class based on the assignment of the English sub-score as college ready (P) or non-college ready (F). A clear demarcation line in the classes is seen in Table 2 corresponding to the parent classes of the math-only model. Within the context of the non-college ready class, the benefit of applying the English sub-score as a secondary placement indicator becomes apparent. Comparisons between the parent classes shown in Table 1 are not possible due to the presence of an intermediate algebra pre-requisite requirement which is not present in both parent classes. These results, which are consistent with the numerical math-only models found in the literature, potentially have applicability in improving the low specificity of the math-only models. Since the proposed expanded indicator is not recommended for usage in the college ready classifications, no effect is discussed regarding these sub-classes, PP and PF. These students would be directly placed into the college algebra course based on the current placement model.

Table 2

Within the non-college ready classification, two sub-classes, FP and FF, are present as shown in Table 2. Incorporation of the FP sub-class into the placement guidelines has positive implications within the current sample where 572 students were classified as non-college ready in mathematics and placed into remedial coursework with 413 of these subsequently passing the college-level course. The inclusion of these students in the college-level placement criteria results in a decrease of remedial placements from 62% to 17% of incoming students. Within this sample, the redefinition of college readiness to include the English sub-score increases the direct placement of students in the college-level course from 483 to 1055 while maintaining a 79% sensitivity rate in the correct placement of students which is comparable to the 87% sensitivity of the standard placement model. In the current sample, the number of erroneous failing predictions decreases from 542 to 129 constituting a 74% reduction when considering the expanded indicator. Furthermore, it has the added benefit of only impacting students previously predicted to be non-college ready in mathematics. The loss of sensitivity is comparatively negligible and outweighed by the magnitude of the 74% reduction in erroneous failing predictions incurred by the math-only model. This method also improves the specificity of the model to 38% by focusing the occurrence of failing predictions on the FF sub-class which carries a more equitable distribution of passing and failing outcomes as shown in Table 2. As other non-academic factors are expected to impact this sub-class of students, the expanded indicator is not indicated to address the FF sub-class.

Application of the Expanded Indicator

The current methodology for placing college algebra students is a static application of a single placement score. Dynamic considerations, while reasonably complex in the statistical basis, are considerably simple in application within the context of the probabilities and percentages reported. As the inclusion of the English sub-score has been previously noted in the literature (Case, 1987; Hatch, 1981), it is likely that the lack of current usage in college algebra advising stems from reasons unrelated to the literature base. Two likely initial objections to the use of this model would be that it does not save much in the way of adviser time and is not simple to apply. However, each institution would develop the corresponding criteria for utilization prior to the advising period for the given term. Table 2 is a development resulting from the analysis of the sample at hand. Each institution would develop a program for implementation and generate the appropriate analysis criteria from their database to ensure the systematic accounting of any variables which the model was not intended to include. Considering the prime range of applicability is projected to be students with mathematics sub-scores in the 17-18 range, an institution would develop a protocol by which the dynamic process is activated, produce the appropriate interpretations by the aforementioned method for advising processes, and generate the interpretational guidelines by which the student would be subsequently placed. An institution that progressively interprets the “Remediation” guidelines (Complete College America, 2012) might develop a process whereby students with a non-college ready mathematics sub-score and a college ready English sub-score would be advised to proceed on to college algebra with a reinforcement lab, rather than to complete the intermediate algebra sequence, expending potentially unnecessary time in a remedial course at the possible expense of limited financial aid. Caution would be necessary in determining the exact cutoff point for the mathematics sub-score to limit erroneous placements of students despite English proficiency. Conversely, a student with a college ready mathematics sub-score should be advised in the context of other non-numerical advising tools regarding the completion of coursework to supplement the outcome in college algebra. These tools would likely be dependent on an assessment of perceived mathematical ability. Each institution would freely determine the role, benefits, and methods of application for the expanded indicator in their college algebra advising process.

Influence of Develpmental Courses

Even though the inclusion of intermediate algebra or other developmental coursework is not a factor in this model, previous literature results are inconclusive regarding the benefits of remedial coursework. Despite the mixed basis in the literature, the general position among educators is that the presence of developmental coursework (Hatch, 1981), specifically the successful completion of intermediate algebra (Doyen, 2011), is a positive, significant predictor of college algebra success due to the acquisition of the necessary fundamental mathematics skills (Gray, 1976). However, the presence of time gaps in the completion of developmental coursework (Gonzales, 2012) and an increasing overall number of developmental courses prescribed during advising (Gupta et al., 2006) are associated with a negative outcome. To this end, some current initiatives emphasize placement reinforced by a co-requisite laboratory experience over remedial coursework (Complete College America, 2012); the addition of supplemental predictive tools, including those presented, would be beneficial to the adviser. Further study on the reasons for this apparent relationship between mathematics, English, and reading are necessary; however, it is likely that the similarities between grammatical structures and equation formats are a key connection. Many instructors cite the ability to solve equations as fundamental to success in college algebra coursework (McEwen, 2016). Students with satisfactory grammatical and applied reading skill sets are acquainted with non-mathematical forms of equation agreement. In grammatical structures, the placement of nouns, verbs, objects, and their related tenses are analogous to the structures present on both sides of the equal sign. As such, depending on sub-class membership, as noted in Table 2 remediation and its specific form can have a more customized design based on the needs of the student. One benefit of the expanded indicator approach is in the area of enhanced student perceptions of outcome. Regardless of the presence of developmental coursework, the student currently being placed in college algebra can be shown a favorable outcome prediction, despite the circumstances that have led to them arriving at the placement discussion. This could be particularly influential on students with an external locus of identity, where the perception that external factors guide the outcomes of their experience is a powerful motivator or, conversely, a suppressant of the drive to achieve.

Conclusion

The overall benefit of an expanded indicator lies in the enhancement, not replacement, of the math-only model in advising potential college algebra students and a possible reduction in the number of erroneous predictions of failure based on the math-only model. The results are based on readily available data and are an easily understandable way for students to be presented with evidence of favorable probabilities for successfully completing college algebra when they fall into the FP sub-class. This may be helpful in increasing student motivation to invest more effort into the course. When using the combination of the two models, an adviser should feel empowered to identify potentially at-risk students and focus their advising time on correcting erroneous failing predictions instead of continuing a less apt method of placing students in remedial coursework based on the ACT mathematics sub-score, absent of secondary parameters. The authors do not suggest all students should be placed in the college algebra course without consideration of the predictions of the math-only model, especially considering the accuracy with which that model predicts success (McEwen et. al, 2018). However, there is academic benefit in using secondary placement indicators as it may allow students to move expeditiously toward the college-level course and result in a more appropriate allocation of funding dedicated toward truly necessary remedial coursework. Research is needed to identify the influential variables associated with the results presented here. Accounting for the low specificity of the current math-only model solves part of the problem. The question of why this problem exists remains to be answered. A partial explanation may exist in the context of student retention. The ability to easily drop a course results in many student withdrawals due to attitudes about mathematics and a student’s self-perceived ability. Many institutions are currently revising the timelines for dropping a course with the intention of improving completion and success rates by refining and narrowing the circumstances allowing a student to receive a withdrawal (W) on their transcript for a course. While the numerical results presented previously can be used to encourage a student to persist, it is impractical to present hundreds of students with statistical results and provide counseling during limited advising time. The current models, while demonstrating satisfactory sensitivity, indicate a distinct difference between predicting success and predicting failure. Due to these considerations and the results presented here, combined with the literature base, college algebra placement should no longer be considered a one-dimensional problem. Regardless of the rationale for limiting the placement criteria in the past, the realization of college algebra placement as a two-dimensional problem calls for the expansion of our placement indicators. Further targeted research in this area is suggested and should include: 1) a deeper analysis of the relationship between the four ACT sub-scores as an expanded indicator, 2) a post-course, double-blind analysis regarding the change-in-ability perception as a result of the inclusion in the FP sub-class, and 3) a double-blind, categorical analysis with the collective group of sub-scores to determine which group provides the best probability of success for the student. This will likely determine the full applicability of the expanded indicator. Meanwhile, administrators can consider the results of the expanded indicator as a possible method for improving their accuracy of placement to conserve and better utilize an ever-shrinking pool of government funding sources, faculty time, and student motivation to succeed in entry level college mathematics.