Other Pages in this Section

VALUE

Frequently Asked Questions about VALUE

 

What does VALUE stand for?

It stands for Valid Assessment of Learning in Undergraduate Education.

What does it mean to evaluate student work using a VALUE Rubric?

The VALUE rubrics are 16 templates, each of which help evaluators assess the level of proficiency represented in a collection of exemplary student work products (papers, performances, community service projects, etc.), or can be adapted for assessment of achievement demonstrated in an individual student work product. Each rubric addresses five to six key criteria for a proficiency, eg., quantitative literacy, and the evaluator chooses from among the four descriptors ( a benchmark level, two milestone levels, or the highest capstone level), which level of proficiency the student’s piece of work demonstrates.

Ideally, more than one evaluator assesses the piece of work, and the level of mastery is determined by the group. E-Portfolios, although not necessary for VALUE use, are one of the most promising approaches to collecting and assessing student proficiencies over time and to document growth and development in learning.

The VALUE rubrics are aligned with the Degree Qualifications Profile (DQP) proficiencies for achievement across the associate and baccalaureate levels.

Did AAC&U create the VALUE rubrics?

In 2009, AAC&U brought teams of faculty experts and others from member institutions together to envision, draft, and refine the 16 VALUE Rubrics. Through review of the literature, extant rubrics, and their own experience, rubric teams identified key criteria broadly shared regarding critical dimensions of achievement for each student proficiency. AAC&U did additional validity and reliability testing of the rubrics to ensure they measure what they are intended to measure. AAC&U staff and fellows maintain the VALUE website on www.aacu.org, offer workshops on the use of the rubrics, and have created several publications containing data analysis and case studies.

What is an e-portfolio?

An electronic portfolio is a digital collection of a student’s best work (oral, written, artistic). It can hold work, for example, from one course, from all required courses in the first two years, all courses taken, co-curriculum, and prior life experiences. It should include the student’s written or oral reflection on and analysis of the learning achievement demonstrated by the problem-centered work and activities. E-portfolios can follow a student as she/he transfers from one institution to another, and typically includes a digital resume with links to examples of the students’ best applications of their proficiency for employers or graduate school.

When the student’s work in the e-portfolio is evaluated using rubrics, it is possible to chart improvement of the student’s knowledge, capacities, and skills over time and to note skills that still need improvement. The assessment becomes part of the e-portfolio as well.

Not only can individual students know their progress and proficiency levels, but institutions can aggregate assessment data and get a sense of the learning that is happening in a particular course, department, or school. Over time, the institution can chart learning outcomes improvement as a whole using the evaluation data from one or more VALUE rubrics. Are students in general becoming clearer writers? Is the student body better able to think critically about a complicated problem than the student body did three years ago?

Studies show that in 2012, about half of all colleges and universities in the United States were using e-portfolios with students. Some use them in particular courses, in general education, or in certain majors; others use them to document and evaluate a student’s full experience.

Some e-portfolio platforms are designed and maintained by for-profit companies or open source providers that build many features into the e-portfolio product and into the interface, allowing institutions to aggregate data easily and use it for continuous improvement and reporting, e.g. during accreditation visits. Some institutions, however, have created their own e-portfolio system using WordPress, Google Docs, or another readily available open application.

Does the e-portfolio itself add to the student’s proficiency, or is it simply a place to store a student’s oral and written work?

AAC&U has begun to study this question, as has AAEEBL (the Association for Authentic Experiential, and Evidence--Based Learning).  Early evidence from the field shows us that the intentional collection of, and reflection upon, one’s college work makes students more proficient in the overarching learning outcomes (critical thinking, integrative learning, oral and written communication, quantitative reasoning, etc.) they are trying to achieve, and makes them more capable of knowing their own strengths and weaknesses in these key proficiencies over the span of their educational pathways.

There is exciting evidence emerging from a national FIPSE-funded e-portfolio project, involving broad access and selective institutions (2-year and 4-year) that when students reflect on and analyze their learning as a whole, and are encouraged to improve certain skills, they are more likely to persist in college and to graduate. AAC&U hopes to undertake more studies to discover if this finding holds true across additional experiments and demonstrations.

 What about standardized tests? Are you saying they are ineffective at measuring learning? 

A standardized test such as the Collegiate Learning Assessment (CLA) takes a snapshot of a sample of students at a particular time and is voluntary, so it measures some learning of some students. Because these tests are almost always taken by volunteers, and carry no consequences, research shows that students are not motivated to do their best work on them. Moreover, good psychometric practice rejects the idea of using any single measure as a proxy either for individual student proficiency or for institutional evaluation. Finally, information from a particular test, because it is disconnected from specific curricula, provides little help for students or faculty to identify specific areas in which to focus their own efforts to achieve higher levels of mastery.

The VALUE rubrics were developed to answer the need to measure the development and application of the essential learning proficiencies that college graduates need in order to be productive in work and in citizenship. By assessing students’ best work done in their curricula, those who evaluate the level of achievement get a fuller picture of how much a student’s knowledge and skills have grown and matured during college.

If a university or college decides it wants to improve the proficiency of all undergraduate students, how can it use VALUE rubrics to do that?

An institution can undertake a study focusing on key proficiencies. It can decide, for example, to measure the development of students’ critical thinking and written communication through the general education curriculum. A team of faculty members and others can assess authentic, problem-centered student work at the beginning, middle and end of that series of courses, measuring the aggregate improvement in those two skills over time. If institutional leaders and faculty decide the level of development is lower than expected, they now can target where interventions--e.g. evidence-based high-impact teaching and learning practices that tend to lead to better learning outcomes--can be included in courses and assignments, and assess the learning again after those changes take place.

Is this system designed to judge publicly the effectiveness of individual faculty members?

VALUE has one goal: to help all students achieve the levels of proficiency necessary for success. It takes a faculty and program working collectively to help students achieve high levels of demonstrated accomplishment. As an institution gathers solid evidence of what teaching and learning practices consistently lead to required proficiency, faculty will be more likely to adopt those evidence-based practices. The process of continuous improvement built into the VALUE project, in other words, is based on carrots and not sticks. AAC&U's on-going initiative with the Multi-State Collaborative (MSC) and select private institutions is developing a process for establishing nationwide benchmarks for learning based on the VALUE rubrics, collected from 2- and 4-year campuses across the country and scored by faculty to create a landscape of learning on all of the Essential Learning Outcomes.

Will assessment using the VALUE rubrics, with or without e-portfolios, save students money in the long run?

AAC&U is engaged in a nationwide project involving sets of states, two- and four-year public colleges and universities and private liberal arts colleges to test the feasibility of the VALUE rubric approach to direct assessment of student learning. The largest component of the current VALUE project is the Multi-State Collaborative to Advance Learning Outcomes Assessment (MSC) in partnership with the State Higher Education Executive Officers association (SHEEO). Currently, twelve states – Hawaii, Oregon, Utah, Texas, Minnesota, Missouri, Indiana, Kentucky, Connecticut, Rhode Island, Massachusetts and Maine – and almost one hundred 2- and 4-year colleges and universities in those states are collecting samples of student work from students nearing graduation for three Essential Learning Outcomes – written communication, quantitative literacy and critical thinking. Faculty and other educational professionals from the participating institutions are using the respective VALUE rubrics to score the collected student work through a project digital database to assess demonstrated levels of learning on each dimension of the learning outcomes. 

Results from the pilot year of the MSC suggest that it is possible within existing resources for all types of campuses in several states to develop sampling designs representative of their students, to collect student work from courses, to upload the student work into a database, to engage faculty in scoring the work using VALUE rubrics, and to utilize the results of the scoring to build campus capacity around direct assessment and improvement of learning. 

Additional states and institutions have indicated a desire to join the MSC, suggesting that cost and feasibility of the VALUE rubric approach are within the capacity of both states and institutions. Faculty report that the rubrics are useful and reflective of the key dimensions of learning they are seeking from their students. The rubric approach involves no additional cost to students since it relies solely on the work students already generated in their courses and assignments regardless of program of study. 
    
At this time, higher education institutions seldom work together to gather the large-scale data we must analyze to discover if assessing proficiency this way leads to smoother transfers between institutions and leads to a faster time-to-degree, which would be a huge cost savings. Initial individual campus findings indicate that a VALUE rubric/e-portfolio approach is actually cheaper than when they use standardized tests.