Evaluating Curriculum Materials for Gifted/Talented Students
Dr. E. Jean Gubbins' notes in a paper for the National Research Center on the Gifted and Talented (Gubbins, 1998), "Whether your district's program is relatively new or fully established, it is important to revisit why you developed
specific programs and services and determine how these programs and services promote high-end learning opportunities." Thus, evaluation of curricula offered as part of a district's array of Gifted/Talented (G/T) services, including
Advanced Placement (AP) and International Baccalaureate (IB) course materials, should hinge on how to address the needs of gifted learners and should align with the Principles of a Differentiated Curriculum for the Gifted and Talented put forth at the National Conference on Curricula for the Gifted/Talented.
According to the National Association for Gifted Children (1994), differentiation
of curriculum and instruction for gifted learners should include not only acceleration of content materials (or pacing), but "in-depth study, a high degree of complexity, advanced content, and variety in content and form." Finally,
the State Goal for Services for Gifted Students mandates that students who receive gifted services in Texas demonstrate advanced knowledge and skills through the development of professional-quality products or performances.
Thus, evaluating course materials and instructional strategies requires both formal and informal processes defined by local committees and/or teams of educators throughout a district,
The State Plan, the "Principles of a Differentiated Curriculum for the Gifted and Talented",
as well as the Scoring Scale for programs such as the Texas Performance Standards Project might serve as guides for developing evaluation rubrics or checklists. However, the evaluation
effort should be ongoing and may include data collection ranging from interviews, portfolios, formal observations, program records, logs, journals, test scores, and other assessment tools.
The task force designed the Advanced Academics Curriculum Evaluation System (AACES) by adapting indicators from three sources: the "Principles of a Differentiated Curriculum for the Gifted and Talented",
Facilitating the Understanding of Depth and Complexity, and the original Texas Performance Standards Pilot Project Scoring Scale. The task force also employed measures specific to concerns related to gifted students from poverty. The original purpose of the tool was to guide
the task force in their AP course evaluation work. Modified over the years, the tool now includes six sections:
- Content
- Process
- Product
- Affect
- Talent Development
- Program Design and Administration
For reference, the complete printer-friendly version of this evaluation tool is available for download.
Educators often debate how best to evaluate curriculum materials. The ways each evaluator defines criteria, decides rules, and intuits or forms judgments can present significant challenges when the end goal is a consensus related to the
applicability of course materials to a particular group of students' needs. However, educators agree that evaluation should be deliberate and relate to student learning.
Four steps described by Deborah Fournier (1995) can provide a starting point for any evaluation:
- Establishing criteria of merit. On what dimensions must the student do well?
- Constructing standards. How well should the student perform?
- Measuring performance and comparing with standards. How well did the student perform?
- Synthesizing and integrating data into a judgment of merit or worth. What is the merit or worth of the student's work?
The AACES addresses steps one and two. The evaluator should complete steps three and four as they relate to the learning needs of the school community following the self-assessment evaluation process. Furthermore, through the process of
"
Naturalistic Generalizations"(Stake & Trumbull, 1982), educators can facilitate change and improve practice. At its
core, in "naturalistic generalizations," the evaluator combines all he/she knows with all that he/she can learn through a formal process in order to judge the value of the curriculum for the learners. The evaluator collects data
on an observed value and compares that to a standard (Stake, 1973).
Task Force History
During the 73rd Session of the Texas Legislature, lawmakers created the Texas Incentive Program for Advanced Placement (TEC ยง28.051-28.058) to reward students, campuses, and teachers for high performance on College Board Advanced Placement
(AP) or International Baccalaureate (IB) program examinations. Following these efforts, the 77th Texas Legislature promoted increased access and enrollment in these programs for underserved and economically disadvantaged students. Today,
the overall goal remains to increase participation and raise achievement levels in AP/IB classes as measured by numbers of students taking AP exams and scores received on those exams.
The Advanced Academics Curriculum Evaluation (AACE) Task Force was developed to address this goal of the 77th Legislature with regard to implementing the Texas Incentive Program for Advanced Placement, and the work of this collaborative
was supported by additional funding designated for Gifted/Talented (G/T) services in the state. Additionally, the task force's work focused on the retention of students identified for G/T services in the secondary school years.
In order to enhance program services and retain G/T students, the task force evaluated current AP coursework for its relative strengths or limitations in addressing the needs of G/T learners through differentiated activities/strategies.
As the State Plan places emphasis on the four core areas of mathematics, science, social studies, and language arts, the task force chose to evaluate four corresponding
courses widely tested in Texas:
- AP Biology
- AP Calculus AB
- AP English Language and Composition
- AP U.S. History
Although the pool of evaluators comprising the task force consisted of a relatively small group of educators, the work of this group serves as an example and resource for AP teachers and consultants working through their own curricula evaluation
process.
Task Force Purpose and Methods
The Advanced Academics Curriculum Evaluation (AACE) Task Force evaluated the courses using documents made available through the College Board to all Advanced Placement (AP) teachers, specifically the course descriptions, commonly called
ACORN books, lab manuals (for AP Biology), and teacher's guides. Within these documents, each content area team focused on the following:
- Finding examples of differentiation within the AP objectives and guidelines
- Discovering explicit descriptions within the course overviews that are appropriate for gifted learners
- Summarizing the strengths and limitations of the courses in meeting the needs of Gifted/Talented (G/T) learners
- Making recommendations for areas where teachers should differentiate for G/T learners
2006 Advanced Academics Curriculum Evaluation Task Force Examples
Overall, the Advanced Academics Curriculum Evaluation (AACE) Task Force identified both strengths and limitations inherent within the Advanced Placement (AP) course materials with regard to differentiation for Gifted/Talented (G/T) students.
Their findings indicate that the rigor and accelerated pacing of the content are present. However, differentiation for G/T learners must be directly addressed.
Using the evaluation checklist, the task force specialists analyzed each AP course according to the following scale:
- Indicates that the element was not present
- Suggests that there is potential in the lesson, but the inclusion of the element is not overt
- Signifies that the element is found
The AP Calculus AB course provided the least evidence of overt differentiation strategies in the course materials and would subsequently need the most modifications. Conversely, AP English Language and Composition provided the most overt
opportunities for differentiating for G/T students. Among the four domains measured in the Advanced Academics Curriculum Evaluation System (AACES) evaluation, elements in the Affect area received low scores, while the Content area
scored highest.
| AVERAGE SCORES FROM THE AACE TASK FORCE EVALUATION | ||||
| Category | AP Biology | Calculus AB | English Language and Composition | U.S. History |
| CONTENT | 2.6 | 1.9 | 2.6 | 2.1 |
| PROCESS | 2.7 | 1.9 | 2.7 | 2.6 |
| PRODUCT | 2.3 | 1.5 | 2.7 | 1.7 |
| AFFECT | 1.7 | 1.0 | 2.7 | 2.7 |
| AACE TASK FORCE FINDINGS | ||||||
| Category | # | Descriptor | Bio | Calc | ELC | USH |
| CONTENT | 1 | Presents content related to broad issues, themes, or problems | 3 | 3 | 3 | 3 |
| CONTENT | 2 | Integrates multiple disciplines | 3 | 3 | 3 | 2 |
| CONTENT | 3 | Presents comprehensive, related, and mutually reinforcing experiences | 3 | 3 | 3 | 2 |
| CONTENT | 4 | Allows for in-depth learning of a self-selected topic | 2 | 1 | 2 | 2 |
| CONTENT | 5 | Addresses dilemmas, controversies, biases, and ethical questions involved in the discipline | 3 | 1 | 3 | 3 |
| CONTENT | 6 | Addresses "unanswered questions" or issues still not understood about this discipline | 3 | 1 | 3 | 1 |
| CONTENT | 7 | Requires students to formulate questions, predict and hypothesize about these questions, and then judge the credibility of these explanations | 3 | 3 | 3 | 3 |
| CONTENT | 8 | Fosters connection between content and potential career fields and/or leadership opportunities | 3 | 2 | 2 | 1 |
| CONTENT | 9 | Offers opportunities for students to engage in activities aligned with students' individual strengths, preferences, or interests | 2 | 1 | 3 | 2 |
| CONTENT | 10 | Pulls information from varied resources and media (such as texts, books, articles, and primary source material) | 3 | 2 | 3 | 3 |
| CONTENT | 11 | Allows for the acceleration of content in an area of strength | 2 | 1 | 2 | 1 |
| CONTENT | 12 | Integrates fine arts with content learning | 1 | 1 | 1 | 2 |
| CONTENT | 13 | Accelerates learning of basic skills through complex presentation | 3 | 3 | 3 | 2 |
| CONTENT | 14 | Links emotion and experience to learning | 2 | 2 | 3 | 2 |
| AVERAGES | 2.6 | 1.9 | 2.6 | 2.1 | ||
| Category | Descriptor | Bio | Calc | ELC | USH | |
| PROCESS | 1 | Develops independent or self-directed study skills | 2 | 1 | 3 | 3 |
| PROCESS | 2 | Focuses on open-ended tasks | 3 | 1 | 3 | 2 |
| PROCESS | 3 | Integrates basic skills with higher-level thinking skills | 3 | 3 | 3 | 3 |
| PROCESS | 4 | Develops complex, abstract, and/or higher-order thinking skills | 3 | 3 | 3 | 3 |
| PROCESS | 5 | Develops a research-oriented model for acquiring and synthesizing information | 3 | 2 | 2 | 3 |
| PROCESS | 6 | Encourages the acquisition of skills, methodologies, and dispositions of professionals practicing in the field | 3 | 2 | 3 | 3 |
| PROCESS | 7 | Offers an array of learning opportunities that include whole-class, small-group, and individual instruction and activities | 3 | 2 | 3 | 2 |
| PROCESS | 8 | Offers out-of-school learning options | 2 | 1 | 2 | 1 |
| PROCESS | 9 | Offers assignments that include visual and verbal components | 3 | 3 | 3 | 3 |
| PROCESS | 10 | Uses graphic organizers to develop concepts | 2 | 1 | 2 | 3 |
| AVERAGES | 2.7 | 1.9 | 2.7 | 2.6 | ||
| Category | Descriptor | Bio | Calc | ELC | USH | |
| PRODUCT | 1 | Develops products that challenge existing ideas and produce "new" ideas | 3 | 1 | 3 | 1 |
| PRODUCT | 2 | Develops products that use new techniques, materials, and forms | 3 | 1 | 3 | 1 |
| PRODUCT | 3 | Evaluates student outcomes through self-appraisal, criterion-referenced, and/or standardized instruments | 3 | 2 | 3 | 3 |
| PRODUCT | 4 | Develops products related to real-world applications or problem solving | 3 | 2 | 2 | 2 |
| PRODUCT | 5 | Offers opportunities for students to interact with role models, community resources, or professionals in the field | 1 | 1 | 2 | 1 |
| PRODUCT | 6 | Offers opportunities for in-class projects and homework | 1 | 2 | 3 | 2 |
| AVERAGES | 2.3 | 1.5 | 2.7 | 1.7 | ||
| Category | Descriptor | Bio | Calc | ELC | USH | |
| AFFECT | 1 | Encourages the development of self-understanding | 2 | 1 | 3 | 2 |
| AFFECT | 2 | Encourages growth and change in the student's abilities and personal outlooks | 2 | 1 | 3 | 3 |
| AFFECT | 3 | Includes mentors/tutors who share common interests and talents with students | 1 | 1 | 2 | 3 |
| AVERAGES | 1.7 | 1.0 | 2.7 | 2.7 | ||
Cross-Disciplinary Strengths and Weaknesses
After assessing the curriculum per the scoring dimensions of the evaluation checklist, Wichita Falls High School educator Ward Roberts compiled the data and summarized the areas where the four courses were strongest in meeting the needs of gifted learners
and the areas where additional instructional strategies were needed. Overall, strengths occurred only in the Content and Process domains. Weaknesses were found in the Product, Process, and Content domains.
| STRENGTHS Areas in which all four content area programs scored "3" | ||
| CONTENT | 1 | Presents content related to broad issues, themes, or problems |
| CONTENT | 7 | Requires students to formulate questions, predict and hypothesize about these questions, and then judge the credibility of these explanations |
| PROCESS | 3 | Integrates basic skills with higher-level thinking skills |
| PROCESS | 4 | Develops complex, abstract, and/or higher-order thinking skills |
| PROCESS | 9 | Offers assignments that include visual and verbal components |
| STRENGTHS Areas in which three of the four content area programs scored "3" |
||
| CONTENT | 2 | Integrates multiple disciplines |
| CONTENT | 3 | Presents comprehensive, related, and mutually reinforcing experiences |
| CONTENT | 10 | Pulls information from varied resources and media (such as texts, books, articles, and primary source material) |
| CONTENT | 13 | Accelerates learning of basic skills through complex presentation |
| PROCESS | 6 | Encourages the acquisition of skills, methodologies, and dispositions of professionals practicing in the field |
| WEAKNESSES Areas in which three of four content area programs scored "1" |
||
| CONTENT | 12 | Integrates fine arts with content learning |
| PRODUCT | 5 | Offers opportunities for students to interact with role models, community resources, or professionals in the field |
| WEAKNESSES Areas in which half of the content areas scored "1" |
||
| CONTENT | 4 | Allows for in-depth learning of a self-selected topic |
| CONTENT | 6 | Addresses "unanswered questions" or issues still not understood about this discipline |
| CONTENT | 11 | Allows for the acceleration of content in an area of strength |
| PROCESS | 8 | Offers out-of-school learning options |
| PRODUCT | 1 | Develops products that challenge existing ideas and produce "new" ideas |
| PRODUCT | 2 | Develops products that use new techniques, materials, and forms |
With regard to criteria in each domain, only AP Calculus scores indicated a lack of potential for differentiating according to Content. Scores from all four courses showed potential for differentiating according to Process. Further, scores from AP English
Language and Composition and AP Biology indicated some potential for addressing criteria in the Product domain. In the Affect domain, neither the AP Biology nor AP Calculus AB courses scored well, indicating that educators need to supplement
with additional strategies in these areas.
Overall, the AP English Language and Composition course consistently provides the most evidence of differentiation strategies in each category. The AP Calculus AB course consistently provides the least evidence of differentiation in each category. The
AP Biology course scores sufficiently in every category except Affect, and the AP U.S. History scores sufficiently in every category except Product.
Overall Implications for Gifted/Talented Students from Poverty
The Advanced Academics Curriculum Evaluation (AACE) Task Force compiled scores for the criteria of significance for addressing the needs of Gifted/Talented (G/T) students from poverty and found varying results. For example, the 2.0 average of scores for
Advanced Placement (AP) Biology in the criteria pertinent to G/T students from poverty indicates that the course has potential to meet the needs of these learners, if educators supplement the instructional materials and strategies.
However, this score does not address enough of the needs of G/T students from poverty to be considered suitable without modification. Missing elements include integrating the fine arts with content learning. For AP Calculus AB, fine arts integration is
also missing, in addition to a lack of support for use of graphic organizers and relationship building with mentors/tutors. Acceleration through basic skills and visual/verbal assignments are the strengths present in the AP Calculus AB
course. The average indicates that there is potential for this population's needs to be met through the AP Calculus AB course, but that the special needs of this group are not built into the curriculum and need to be added by the individual
educator.
Additionally, the task force found that for AP English Language and Composition, the only weak measure occurred under Content #1 (integrates the fine arts with content learning). Otherwise, this AP course indicated strong potential for addressing the
needs of the Gifted/Talented students from poverty. Finally, the average of 2.4 showed positive results for AP U.S. History. The only element the task force found lacking for AP History was work with mentors/tutors.
| ELEMENTS PARTICULAR TO STUDENTS FROM POVERTY | ||||||
| ELEMENT | # | DESCRIPTOR | BIOLOGY | CALCULUS AB | ENGLISH L & C | U.S. HISTORY |
| CONTENT | 12 | Integrates fine arts with content learning | 1 | 1 | 1 | 2 |
| CONTENT | 13 | Accelerates learning of basic skills through complex presentation | 3 | 3 | 3 | 2 |
| CONTENT | 14 | Links emotion and experience to learning | 2 | 2 | 3 | 2 |
| PROCESS | 7 | Offers an array of learning opportunities that include whole-class, small-group, and individual instruction and activities | 3 | 2 | 3 | 2 |
| PROCESS | 9 | Offers assignments that include visual and verbal components | 3 | 3 | 3 | 3 |
| PROCESS | 10 | Uses graphic organizers to develop concepts | 2 | 1 | 2 | 3 |
| PRODUCT | 6 | Offers opportunities for in-class projects and homework | 1 | 2 | 3 | 3 |
| AFFECTIVE | 3 | Includes mentors/tutors who share common interests and talents with students | 1 | 1 | 2 | 1 |
AP Biology Course Evaluation
The Advanced Academics Curriculum Evaluation (AACE) Task Force found specific evidence of strategies in Advanced Placement (AP) Biology aiding differentiation in the Process and Content domains. The lowest scores for AP Biology occurred in the Affect
domain. Due to these low scores, the task force concluded that without modification, products developed in AP Biology do not meet the needs of Gifted/Talented (G/T) learners, especially with regard to developing personal interests and
expressing creativity. Additional concerns included the amount of in-class time allowed for course projects, which could pose obstacles to differentiation in depth and complexity. Furthermore, there were no embedded strategies for mentoring
students. Thus, teachers should modify the curriculum to allow for individual, creative project work and establish mentoring relationships with professionals from the community. Addressing these specific issues would satisfy the task force's
primary concerns and would help differentiate the materials for G/T learners.
AP Calculus AB Course Evaluation
According to the Advanced Academics Curriculum Evaluation (AACE) Task Force, Advanced Placement (AP) Calculus AB scored best in the Content and Process domains. Some items in the Product domain indicated potential for differentiation, but the Affect domain
yielded no overt strategies for meeting the needs of gifted learners. Furthermore, career connections and varied resources were not explicit in the course materials. Including connections to math-related career fields and providing more
varied resources and manipulatives could increase Gifted/Talented (G/T) students' understanding in the content and process domains, as well as develop personal interests as noted in the Affect domain. Further recommendations by the task
force's mathematics specialists included providing gifted learners with alternative upper-level mathematics courses, like AP Statistics, which have a greater potential for differentiation strategies, such as AP Statistics.
AP English Language and Composition Course Evaluation
Of the courses evaluated by the Advanced Academics Curriculum Evaluation (AACE) Task Force, Advanced Placement (AP) English Language and Composition scored highest in terms of providing evidence of explicit differentiation strategies across all domains.
Thus, this course came closest to meeting the needs of Gifted/Talented (G/T) learners without further modification. However, the course did not provide for mentoring opportunities or necessarily allow students to pursue individual creative
pursuits. The course also failed to integrate fine arts into content learning, a strategy particularly relevant to G/T learners from poverty. Thus, the task force recommended that educators:
- organize the content thematically around a broad-based idea or contemporary issue to add complexity,
- allow for student choice in reading and/or writing assignments to help learners connect with their individual interests,
- strengthen individual ideas through partnerships with mentors involved with English Language and Composition-related careers,
- provide students the opportunities to work with mentors in publishing works, and
- integrate the unit with fine arts content learning and skills.
Additionally, the task force's AP English Language specialists recommended that AP teachers attend G/T training sessions in order to best understand how to meet the needs of G/T learners, including G/T students from poverty, in their classrooms.
AP U.S. History Course Evaluation
Advanced Placement (AP) U.S. History appeared strong in the Process and Affect areas, with weaknesses in the Product and Content areas. The content in AP U.S. History focused on dilemmas, controversies, and ethical questions, yet allowed little opportunity
for development of innovative products, according to the Advanced Academics Curriculum Evaluation (AACE) Task Force. Furthermore, the curricular focus on human and social systems helped students develop self-understanding and fostered
personal growth, as required in the Affect domain. However, the task force recommended that educators organize the content around interdisciplinary themes to strengthen the differentiation potential in the Content domain. Also, educators
must address the constraints of limited class time and expertise in aiding students who plan to develop innovative products. AP U.S. History products could form a foundation for students to build on for the Distinguished Achievement Program
(DAP) or Texas Performance Standards Project (TPSP). If students, especially those from poverty, are provided the opportunity to work with mentors on a TPSP in AP U.S. History, the educator can strengthen elements in the Affect domain.
Additionally, educators could greatly enhance the applicability of the Content and Process domain items in AP U.S. History. Professional development related to implementing differentiation strategies for Gifted/Talented (G/T) learners,
especially gifted learners from poverty, would faciitate this.