Assessment in the Russian Language and Literature Program at the University of Denver

Below is an overview of assessment in the Russian Language and Literature Program at the University of Denver. Assessment seeks to examine the three areas of student and faculty endeavor in the Program, which are language ability, cultural fluency, and personal self-definition. Students completing a Bachelor of Arts degree in Russian Language and Literature are expected to graduate with an ability to speak and write -- as well as understand written and spoken -- Russian at a level appropriate to their experience in class and abroad, and to attain a basic linguistic understanding of how the Russian language functions. The BA major also provides the student with a broad overview of Russia's literary and cultural tradition as well as in-depth exposure to the work of select writers and artists, a fundamental grasp of Russia's history, and an understanding of Russia's place in the world today. The Russian Program's assessment protocol includes traditional tools such as portfolios and course evaluations as well as more innovative measures designed to gauge the less tangible results of the study of Russian at DU. Assessment is intended to provide a continual exchange of information between the faculty and student partners in the Program. Assessment includes:

GOALS:

The Russian Language and Literature Program at the University of Denver seeks to give its students the ability to understand how language shapes their lives, and how culture seeks to express it -- differently according to the language being studied.  To achieve this, it seeks to integrate the learning of Russian in three areas of endeavor: the student's language ability, cultural fluency, and personal self-definition. Each area interacts to form a whole: language ability is strengthened in classroom instruction and, when possible, study abroad; cultural fluency is learned through exposure to literary traditions, translation skills, and the media; personal self-definition is attained through work in the Russian community, the informal use of Russian, and internships in Denver and/or abroad. Each student equally serves to vitalize the faculty in the Russian Language and Literature Program by bringing new and fresh perspectives on the assimilation and the integration of the three areas of endeavor. The Program is thereby a partnership between student and faculty and is assessed as such.

DIRECT ASSESSMENT:

    •     RUSS 1003 and RUSS 2100/2110:  standardized exit conversations about class topics in Russian based on modified ACTFL criteria for reading, writing, speaking, listening, and cultural ability, utilizing real-life situations such as telephone conversations (to demonstrate cultural fluency and expand the use of Russian beyond the strictly academic setting)
    •     RUSS 3500 and third-fourth-year sequences: (1) RUSS 3500:  detailed analysis of Russian text with reference to linguistic study or help in designing efficient RUSS 1001-03 grammar drills; (2) cross-graded final project for RUSS 3901 (effective AY 2008-2009); portfolio with standardized grading criteria for one non-language-specific class of the student’s choice for every non-language-specific class after RUSS 2110/2110 (for AY 2007-2008)
    •     Honors Thesis for exiting honors majors

INDIRECT ASSESSMENT:

    •     Course evaluations
    •     Graduate follow-up studies (includes job placement data)

ASSESSMENT TIMETABLE:

1st-year: RUSS 1003 (end of language requirement)
2nd-year: RUSS 2100/2110 (end of second-year language sequence)
3rd-/4th-year: RUSS 3500 (offered every second year), RUSS 3901 (effective AY 2008-2009), or one additional course of the student’s choice before graduation (only AY 2007-2008)

ASSESSMENT TO DATE:

    •     Faculty in charge: Luc Beaudoin
    •     Faculty responsible for developing assessment: Luc Beaudoin
    •     Assessment has been utilized to improve the 1st-year and 2nd-year Russian language curriculum as well as the 3rd-/4th-year literature curriculum
    •     Faculty assessment: class visits
    •     Distribution of assessment and feedback loop: returns to Luc Beaudoin and is distributed to the faculty in Russian.  A summary of assessment objectives is posted at www.du.edu/langlit/russian/assess.html.

RESULTS OF ASSESSMENT:

  • Direct assessment:
    •  The assessment plan in the Russian Language and Literature Program was formulated officially during Winter Quarter 1999, was put to test at the beginning and the end of the quarter, and continued in subsequent quarters as per the outline above (modified in Fall Quarter, 2001, and Spring Quarter, 2007). By comparing the entrance conversation and the exit assessment, the faculty in the Russian and Literature Program have been able to coordinate learning objectives across courses in order to better meet student needs.
  • Indirect assessment:
    •  Student evaluations consistently give high marks to courses in the Russian Language and Literature Program, showing a high level of satisfaction in the courses offered. Post-graduate assessment has shown that all graduates are employed or in graduate school (either in the United States or in the Russian Federation). The small graduating classes of recent years do not necessitate regular graduating exit surveys, since our majors tend to keep in touch. In general, of the graduating seniors, some travel before studying in graduate school, some find employment in the United States, and some either work or study in the Russian Federation, frequently beating students from Harvard, Stanford, and Yale University, and so on, for language ability and preparation. This reflects the information the program has obtained anecdotally about the other students who have graduated with majors in Russian.

ASSESSMENT STANDARDS:

Below are summaries of the modified ACTFL (American Council of Teachers of Foreign Languages) standards. These standards were modified by the ACTR (American Council of Teachers of Russian) to better reflect the unique challenges Russian poses to the American university student. The cultural standards were developed by Russian Program faculty of the University of Denver to parallel the language ability standards. "Beginner" is the level below that of "Intermediate Low" and reflects a decreased ability proportionally. "Advanced Plus" is one level higher than Advanced; ability increases proportionally. Native speakers can be Advanced Plus or higher. Standard university Russian programs have the following goals for the end of each level (cultural fluency is not rated):

  • 1st-yr level: between Beginner and Intermediate Low
  • 2nd-yr level: between Intermediate and Intermediate High
  • 3rd-yr level: between Intermediate and Intermediate High
  • 4th-yr level: Intermediate High

DU's Russian program aims for the following after each course sequence level (including cultural fluency; study in Russia on the Cherrington Global Scholars ACTR program is factored into these expectations):

  • 1st-yr level: Intermediate Low
  • 2nd-yr level: Intermediate - Intermediate High
  • 3rd-yr level: Intermediate High
  • 4th-yr level: Advanced

Below are the descriptions of the modified ACTFL outcomes:

SPEAKING: INTERMEDIATE LOW: able to handle successfully a limited number of interactive, task-oriented and social situations, responding to simple statements and maintaining restricted, face-to-face conversation with much linguistic inaccuracy. Difficulty getting understood.

INTERMEDIATE MID: able to handle successfully a variety of basic communicative tasks and social situations. Can talk about self and family, and participate in conversations beyond the most elementary. Frequent pauses and struggle with grammatical forms.

INTERMEDIATE HIGH: able to handle successfully most uncomplicated communicative tasks and social situations and use linguistic strategies to ensure understanding. Circumlocution still required to circumvent limited vocabulary. Simple narration.

ADVANCED: ability to converse in participatory fashion, initiate and sustain conversation on a variety of topics, satisfy school and work situations, use paragraph-length discourse.

WRITING: INTERMEDIATE LOW: able to meet limited practical needs: short messages, postcards, and simple notes. Frequent errors in grammar and vocabulary.

INTERMEDIATE MID: Able to meet a number of practical needs, writing short everyday letters with good control of indexual time and the imperfective aspect. Little conscious organization.

INTERMEDIATE HIGH: Able to meet most practical writing needs and limited social needs. Can respond in writing to personal questions. Able to use aspect with some accuracy. No habitual use of cohesive elements.

ADVANCED: able to write narratives and descriptions of a factual nature of at least several paragraphs in length on familiar topics.

LISTENING: INTERMEDIATE LOW: able to comprehend the most basic of social and practical situations. Frequent repetition needed to overcome misunderstandings, particularly at native-speaker speed.

INTERMEDIATE MID: able to understand discourse in a variety of social situations with repetition required for comprehension.

INTERMEDIATE HIGH: able to understand most basic social situations and the majority of simple interactions with native speakers speaking at normal speed. Still some need for repetition.

ADVANCED: able to function in situations with native speakers on topics ranging from the familiar to the more esoteric. Little need for repetition--circumlocution can overcome most vocabulary difficulties.

CULTURAL: INTERMEDIATE LOW: able to conform to some of the cultural needs presented by familiar situations (use of correct wording and intonation when addressing people in formal situations) though there is frequent interference from the native culture, some understanding of cultural differences (stereotypes), little familiarity with either high or low cultures.

INTERMEDIATE MID: able to negotiate typical intercultural situations (generally correct use of polite forms and intonation, though there is still misunderstanding about in-group versus out-group), able to navigate familiar social situations (store etiquette, drinking in social situations), limited knowledge of some familiar cultural icons (Pushkin, Dostoevskii).

INTERMEDIATE HIGH: able to correctly handle polite forms of address and intonations with little interference from the native culture, able to grasp some humor and navigate more unfamiliar social situations such as impromptu interactions in the public sphere, though mistakes may be made in this instance. Some knowledge of cultural icons and their achievements (Pushkin, Dostoevskii, Tolstoi, Akhmatova, Tsvetaeva), some knowledge of low culture (pop music, film, music videos), some acquaintance with the most important political players and issues (Westernization, Mafia and crime, the Duma).

ADVANCED: able to handle appropriately confrontational situations, some knowledge of mat (Russian obscenities, though unable to use any but the most rudimentary forms correctly), able to grasp humor in a variety of situations and navigate most unfamiliar social situations correctly. General knowledge of cultural icons and their work (most periods of Russian culture), as well as a general knowledge of low culture and popular influences. Able to formulate Russian vocabulary on-the-spot using linguistic rules of combination, though mistakes may occur. Able to understand influence of the Soviet period and World War II. Able to discuss current political and economic situation with some ease, though misunderstandings may occur.


Return to the Russian Language & Literature Home Page


Return to the University of Denver Home Page