|
|
|
Education Policy Analysis Archives
Scholastic Achievement and Demographic Characteristics
Abstract
This report presents the results of the largest survey and testing program for students in home schools to date. In Spring 1998, 20,760 K-12 home school students in 11,930 families were administered either the Iowa Tests of Basic Skills (ITBS) or the Tests of Achievement and Proficiency (TAP), depending on their current grade. The parents responded to a questionnaire requesting background and demographic information. Major findings include: the achievement test scores of this group of home school students are exceptionally high--the median scores were typically in the 70th to 80th percentile; 25% of home school students are enrolled one or more grades above their age-level public and private school peers; this group of home school parents has more formal education than parents in the general population; the median income for home school families is significantly higher than that of all families with children in the United States; and almost all home school students are in married couple families. Because this was not a controlled experiment, the study does not demonstrate that home schooling is superior to public or private schools and the results must be interpreted with caution. The report clearly suggests, however, that home school students do quite well in that educational environment.
By current estimates, there are between 700,000 and 1,200,000 students enrolled in home schools in the United States. Further, by all accounts, the movement has been growing steadily over the past few years (Lines, 1998). Yet, there is very little scientific literature concerning the population of home school students or even large samples of home school students.
This study describes the academic achievement levels and some basic demographic characteristics of a large sample of students and their families. While the academic levels of home school students are described in terms of public and private school norms, this study is not a comparison of home schools with public or private schools. Such comparisons would be fraught with problems. Home schooling is typically one-on-one. Public schools typically have classes with 25 to 30 students and an extremely wide range of abilities and backgrounds. Home school parents are, by definition, heavily involved in their children's education; the same, unfortunately, is not true of all public or private school parents. Home schools can easily pace and adapt their curriculum; public and private schools typically have a mandated scope and sequence. The list of differences could continue.
This study seeks to answer a much more modest set of questions: Does home schooling tend to work for those who chose to make such a commitment? That is, are the achievement levels of home school students comparable to those of public school students? Who is engaged in home schooling? That is, how does the home school population differ from the general United States population?
Methods
Bob Jones University Press Testing and Evaluation Service provides assessment services to home school students and private schools on a fee-for-service basis. In Spring 1998, 39,607 home school students were contracted to take the Iowa Tests of Basic Skills (ITBS; grades K-8) or the Tests of Achievement and Proficiency (TAP; grades 9-12). Students were given an achievement test and their parents were asked to complete a questionnaire entitled "Voluntary Home School Demographic Survey." A total of 20,760 students in 11,930 families provided useable questionnaires with corresponding achievement tests. The achievement test and questionnaire results were combined to form the dataset used in this analysis.
This section provides descriptions of the achievement measures, the questionnaires, the Bob Jones University Press Testing and Evaluation Service, and the procedures used to develop the dataset.
Iowa Tests of Basic Skills (ITBS)
Home schooled students in Grades K-8 took the Iowa Tests of Basic Skills (ITBS) Form L, published by Riverside Publishing Company, a subsidiary of Houghton Mifflin. Developed by University of Iowa professors, the tests were designed and developed to measure skills and standards important to growth across the curriculum in the nation's public and private schools.
The ITBS reflects more than 50 years of test development experience and research on measuring achievement and critical thinking skills in Reading, Language Arts, Mathematics, Social Studies, Science, and Information Sources. The scope and sequence of the content measured by the ITBS were developed after careful review of national and state curricula and standards, current textbook series and instructional materials, and research (Riverside, 1993).
All items were tried out and tested for ethnic, cultural, and gender bias and fairness prior to the development of the final form of the tests. Data on a nationally representative sample of public and private schools were collected in 1992 and used to form the initial national norms. The norms were updated in 1995 by Riverside. This study used these 1995 spring norms.
Tests of Achievement and Proficiency (TAP)
Home schooled students in Grades 9-12 took the Tests of Achievement and Proficiency (TAP), Form L, also published by Riverside Publishing Company. The TAP was designed and developed to measure skills and standards important to growth across the high school curriculum. Like the ITBS, the TAP scope and sequence were developed after careful review of national and state curricula and standards, and current textbook series and instructional materials. Developed as an upward extension of the ITBS, the specifications, format, and design of the TAP tests are similar to that of the ITBS. TAP is fully articulated with the Iowa Tests of Basic Skills (ITBS) Form L (Riverside, 1993).
Background Questionnaires
Background questionnaires were designed by the staff of the Home School Legal Defense Association (HSLDA). Questions were determined by reviewing the questions in previous surveys, prioritizing them, and selecting only those that were most germane to the objectives of the study. Where possible, questions and responses were made to match those used by the U.S. Census, U.S. Department of Labor and the National Assessment of Educational Progress to facilitate comparisons of home school students with students nationwide.
HSLDA designed the survey to be much shorter than previous survey instruments. They also sought to pose all questions in an objective format, rather than a constructed response format. In keeping with this approach, HSLDA worked with National Computer Systems to design forms to be computer scanable, thereby removing the need for manual data processing.
Bob Jones University Press Testing and Evaluation Service
The Bob Jones University (BJU) Press Testing and Evaluation Service is the largest and oldest of four organizations providing home school families access to standardized achievement tests. The Testing Service began offering the Iowa Tests of Basic Skills and Tests of Achievement and Proficiency in 1984. In subsequent years they added other helpful tools including practice materials, a personality inventory, and diagnostic tests. In 1993, the Stanford Achievement Test series was added as BJU Press assumed the testing that the Home School Legal Defense Association had been providing for its members. Since that time, a full range of writing evaluations (grades 3-12) and a career assessment have been added to the growing number of evaluation tools offered by the Testing Service.
Just as home school families were the impetus behind the start of the Testing Service, home school families continue to be the largest sector utilizing the service. However, there are also a number of private schools that have chosen to use the services provided. Testing is provided for students throughout the United States and Canada, as well as many foreign countries.
The BJU Press Testing and Evaluation Service sends testing materials to qualified testers who administer the tests and return them to the Testing Service for scoring. The results are then returned to the parent. Many parents test primarily for their own information to verify that their home schooled students are progressing academically at a normal pace. Other parents use the results to meet a state testing requirement or to provide documentation when they choose to return their students to a public or private school setting.
Data Generation Procedures
The following steps were followed to produce the data set:
Characteristics of Home School Students and Families
This section provides a description of home school students and their families based on the 20,790 respondents to our questionnaire. The distribution of students by state, gender, age, race, parent marital status, family size, mother's religion, parent education, family income, television viewing, money spent on educational materials, and other demographic characteristics are identified and, where possible, compared to national figures.
State
As shown in Table 2.1, respondents came from each of the fifty states. Several states, including Ohio, Georgia, and Virginia, have exceptionally high representation given their size. This is probably due to the fact that these states require testing of home school students. To reduce the effects of these and other overrepresented states, the data were weighted in all subsequent analyses by the number of public school students in each state. While we would have preferred to weight by the number of home schooled students in each state, such data are not available for all 50 states (Lines, 1998).
Student Age and Gender
Table 2.2 shows the distribution of the respondents by gender and age. About 50.4% or 10,471 of the respondents were females; 49.6% (10,319) were males. These figures are comparable to that of the population of 3 to 34 years old enrolled in school (see U.S. Bureau of the Census, 1998, Table A-2). Some 51.4% of school enrollees nationally are male. The percentages are comparable at all age levels.
Student Grade
Home school student grade placement was identified by their parents, presumably based on the grade level of the instructional materials. That grade was used by BJU to determine the test levels and used in this report as a grouping variable. Tables 2.3 shows the distribution of respondents and the nation by grade. There is a large difference in the proportions of high school (grades 9-12) home school students and the nation. Compared to the national data, a relatively small percentage of home school students are enrolled in high school. Possible reasons for this lower participation for high school students may be the relative newness of the home school movement, early graduation from high school, and possibly a desire on the part of some home school parents to enroll their children in a traditional high school. The distributional differences for students in grades 1 through 8 are minor.
National data: US Census, 1997b, Table 254.
Student Race
Table 2.4 shows the racial distribution of home school students in 1998 and for the students enrolled in elementary and secondary public and private schools nationally in 1994. The distributions are quite different. The vast majority of home schooled children are non-Hispanic White. The largest minority groups for home school students (not shown in the table) are American Indians and Asian students who comprise some 2.4% and 1.2% of the home school students, respectively.
National data: USDE, 1996; Indicator 27.
Marital Status
The great majority of home school students are in married couple families. In contrast, only 72% of the families with at least one child enrolled in school nationwide are in married couple families (Bruno and Curry, 1997, Table 19).
Children at Home
Table 2.6 shows the distribution of children in home school families and families with children under 18 nationwide. On average, home school students are in larger families. Nationwide, most families with school-age children (79.6%) have only 1 or 2 children with a mean of about 1.9 children per family. Most home school families (62.1%) have 3 or more children with a mean of about 3.1 children per family.
National Data: US Census, 1997a, Table 77.
Mother's Religion
We asked the home school families to identify the religious preference of each student's mother by selecting from a list of 27 religions. As shown in Table 2.7, the largest percentage of mothers identified themselves as Independent Fundamental, Baptist, Independent Charismatic, Roman Catholic, Assembly of God, or Presbyterian. The religious preference of the father was the same as that of the mother 93.1% of the time.
Parent Academic Attainment
As shown in Table 2.8, home school parents have more formal education than the general population. While slightly less than half of the general population attended or graduated from college, almost 88% of home school students have parents who continued their education after high school.
National data: U.S. Census (1996; Table 8).
Family Income
National data on family income are available for 1995. As shown in Table 2.9, home school families span all income levels. On average, home school families have a higher income level than do families with children nationwide and all families nationwide. The median family income level for home school families in 1997 is about $52,000. The median income for families with children in 1995, nationwide, was about $36,000.
National data: Bruno and Curry (1997, Table 19).
Television Viewing
The National Assessment of Educational Progress collects information on the television viewing habits of fourth-graders. Home school fourth-graders and fourth-graders nationally differ markedly in terms of television viewing. Home school students rarely watch more than 3 hours of television per day; nearly 40% of the students nationwide watch that much television.
National data: NAEP Math 1997.
Computer Use
The Condition of Education provides a tabulation of the percent of students nationwide who report using a computer by frequency of use for 4th, 8th, and 11th graders in 1996. At each grade level, the distribution of computer use in 1998 by home school students is different from that of the nation in 1996. At each of these three grade levels, much larger percentages of home school students never use a computer. At the fourth-grade level, a much larger percent of home school students use a computer every day.
National Data: Snyder and Wirt, 1998, Indicator 3.
Money Spent on Educational Materials
The amount of money spent in 1997 on home school education for textbooks, lesson materials, tutoring and enrichment services, and testing ranged from less than $200 to more than $2000. As shown in Table 2.12, the median amount of money spent was about $400.
Other Demographic Characteristics
Compared to the nation, a much larger percentage of home school mothers are stay-at-home mothers not participating in the labor force. Some 76.9% of home school mothers do not work for pay. About 86.3% that do work do so part time. Nationwide, in 1996, only 30% of married women with children under 18 did not participate in the labor force (US Dept of Census, 1997a, Table 632).
A very large percentage of home school parents are certified to teach. Some 19.7% of the home school mothers are certified teachers; 7.1% of fathers. Almost one out of every four home school students (23.6%) has at least one parent who is a certified teacher.
Only 7.7% of the respondents were enrolled in a full-service curriculum program, i.e., a program that serves students and their parents as a "one-stop" primary source for textbooks, materials, lesson plans, tests, counseling, evaluations, record keeping, and the like for the year's core required subjects such as language, social studies, mathematics, and science.
Academic Achievement
The complete batteries of The Iowa Tests of Basic Skills (ITBS) and the Tests of Achievement and Proficiency (TAP) were used to assess student achievement in basic skills. The ITBS was used for home school students in Grades K-8; the TAP for students in grades 9-12. Almost all students took Form L; a handful took parallel Form K.
Achievement test batteries like the ITBS and TAP are a collection of tests in several subject areas that have been standardized and normed. Norms for all tests within these test batteries are based on the same group of students at each grade level. Such norms allow students to be compared with other students and groups to be compared with other groups.
The primary purpose of the ITBS and TAP is to assess the academic achievement of students in public and private schools. Consequently, much of the test development effort is devoted to identifying the content to be covered by these batteries. Riverside Publishers follow a four step process: 1) content specifications, 2) editorial review, 3) pilot testing, and 4) national norms development and updating.
The first and most critical step is developing content specifications and writing test items. This step involves the experience, research, and expertise of a large number of professionals representing a wide variety of specialties in the education community. Specifications are developed which outline the grade placement and emphasis of skills. These specifications draw heavily on an analysis of textbooks, research studies, nationally developed subject matter standards, and national curriculum committees.
Once the items have been developed and pilot tested, the final forms of the tests are developed and administered to large standardization samples to gather normative data and to develop scales.
The spring standardization sample for the 10 levels of the ITBS consisted of approximately 137,000 students from public schools, Catholic schools and private non-Catholic schools. The public school sample was stratified to assure adequate representation based on geographic region, district enrollment, socioeconomic status of the district. The Catholic school sample was stratified on geographic region and diocese enrollment. The non-Catholic private school sample was stratified on region and school type. The spring standardization sample for the four levels of the TAP consisted of approximately 20,000 students stratified on the same variables. National norms were developed based on the combined weighted distributions of all three school types: public, Catholic and non-Catholic private. Catholic/private school norms were developed based on the combined weighted distributions of the latter two groups. For simplicity, the combined public, Catholic and non-Catholic private school norms are referenced in this report as national norms or public/private school norms.
The data from the standardization sample are used to develop a variety of reporting scales, such as percentiles and grade equivalent scores. The analyses in this report rely primarily on the Developmental Standard Score (DSS) scale developed by Riverside Publishers. The DSS is a number that describes a student's location on an achievement continuum that spans grades K through 12. Table 3.1 shows the median DSS and median age that corresponds to each grade level in the national standardization sample. The DSS scale shows that the average annual growth in DSS units decreases each year.
Source for age medians: Drahozal (1998, personal communication).
This same DSS scale is used for all tests and levels of the ITBS and TAP. The main advantages of the DSS are that it mirrors reality well, spans all grade levels, and provides a quasi equal interval scale which has a variety of attractive statistical properties. Most importantly, DSS scores can be compared to each other and can be meaningfully averaged.
The main disadvantage of DSS scores is that they have no built-in meaning. Reference points are needed to interpret DSS scores. "Grade level" is one possible reference point. A DSS score of 170 in reading, for example, is about equal to the typical reading score for second-grade students in public and private schools in the spring of the year. A more refined reference is the percentile score that corresponds to each DSS score. The 170 in reading, for example, corresponds to the 54th percentile of second graders. That is, this score is better than the score received by 54 percent of the second graders using the 1995 spring norms.
The reader should note that while all tests of the ITBS/TAP have the same median DSS score at each grade level, the distributions within each subject area vary. A DSS score of 310 for a tenth grader in reading, for example, corresponds to the 87th percentile. A DSS score of 170 in mathematics for a tenth grader would place the student at the 79th percentile.
Percentiles are always defined in terms of a grade level. This can be problematic when analyzing data for home school students. In this study, 24.5% of the home school students were one or more grades above the grade usually associated with that student's age (see Table 3.2). A strong case can be made that rather than using the percentile corresponding to the enrolled grade, as we did in this study, one should use the percentile associated with the student's nominal grade, i.e., the grade usually associated with the student's age. The argument is that a 10-year-old home school student enrolled in 5th grade should be compared to his age peers in 4th grade. The counter argument is that the percentiles already consider the fact that students are not always in their nominal grade since the standardization sample had students above and below grade level. We initially analyzed the data both ways. Rather than expose our analysis to criticism, we chose to take the more conservative route by employing the enrolled grade.
While very meaningful, percentiles do not provide a complete picture of a student's or group's academic performance. In this study, we used grade equivalent scores as an additional reference point for interpreting DSS scores. A grade equivalent score approximates a child's development in terms of grade and month within grade. A DSS reading score of 170 can be viewed as the typical DSS score earned by students in the ninth month of the second grade or a GES score of 2.9. Just as the percentile associated with a DSS scores varies by subtest, so do the properties of GES scores vary across subjects.
Grade Equivalent Scores are particularly useful for estimating a student's developmental status in terms of grade. But, these scores must be interpreted carefully. An GES Score of 6.3 in reading for an 9 year old in the 3rd grade, for example, clearly indicates that the third grader is doing well. This does not, however, mean that the third grader belongs in the 6th grade. It only means that the third grader can read as well as a sixth grader.
The usual interpretation of a Grade Equivalent Score of 6.3 for a third grader is that this third grade student can read third grade material as well as a sixth grader can read third grade material, not that he or she can read sixth grade material. The DSS of the ITBS/TAP, however, is unique. The DSS scales were developed by administering the same special scaling test to students in grades K-3, another common scaling test to students in grades 3 to 9, and another to students in grades 8-12. Thus, in the scaling study, the third graders did take the same test as the sixth graders in each subject area.
Grade Placement
Home school students are able to progress through instructional material at the student's rate. Thus, it is easy for home school students to be enrolled one or more grades above their public and private school-age peers. To evaluate the frequency of advanced placement, we compared students' enrolled and nominal grades. The enrolled grade was identified by the parents and used to determine the ITBS/TAP level. The nominal grade is the public school grade in which the student would normally be enrolled in based on the child's month and year of birth.
As shown in Table 3.2, almost one fourth of the home school students (24.5%) are enrolled one or more grades above their nominal grade. While comparable figures nationally do not exist, one research director in a large school district estimated that less than 5% of their students are enrolled above grade level.
Percentages do not sum to 100% due to a small percentage of students outside this range.
Overall Achievement
Table 3.3 shows the median scaled score (DSS score) for home school students on the Composite with Computation, Reading Total, Language, Mathematics Total with Computation, Social Studies, and Science subtest scores by grade. The corresponding percentiles shown in the table are the within grade percentile scores for the nation that correspond to the given scaled scores. For example, home school students in Grade 3 have a median composite scaled score of 207 which corresponds to the 81st percentile nationwide. The median home school student in third grade out- performs 81% of the third graders nationwide. As an additional comparison, we provide the national median for each grade in the last column. By definition this is the 50th percentile of students nationwide.
It is readily apparent from Table 3.3 that the median scores for home school students are well above their public/private school counterparts in every subject and in every grade. The corresponding percentiles range from the 62nd to the 91st percentile; most percentiles are between the 75th and the 85th percentile. The lowest percentiles are in Mathematics Total with Computation subtest (labeled Math in the tables); the highest in Reading Total. While the grade-to-grade increase in national medians is 13 DSS points in the lower grades, the annual increase for home school students is about 16 points. These are exceptional scores and exceptional grade-to-grade gains.
As shown in Table 3.4, the same superiority of median scaled scores holds when comparing home school students to students enrolled in Catholic/Private schools. The Catholic/Private school percentiles corresponding to median scaled scores range from the 53rd percentile to the 89th percentile; most are between the 65th to 75th percentile. In every area and every grade, the median scores for home school students exceed the median scores of students enrolled in Catholic/Private schools.
Education Policy Analysis Archives,
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
