Student feedback: A guide to interpreting survey results to improve teaching and learning

Introduction

Receiving student feedback can be daunting experience, and the prospect of analysing and interpreting data provided by surveys can send shivers down the spines of teachers. This guide has been designed to help you interpret the results of your Student Feedback on Subjects (SFS) and Student Feedback on Teaching (SFT) surveys – demystifying educational research to help you better understand what this data is telling you about your subject, your teaching, and your students' learning, and most importantly how these can be enhanced.

Full information about Student Feedback on Subjects (SFS) and Student Feedback on Teaching (SFT) can be found through the Planning and Institutional Performance Unit (PIPU) who administer these surveys.

For questions about survey administration or support, please contact surveys@latrobe.edu.au.

Data collection and interpretation

Collection

Student feedback surveys at La Trobe adopt a mixed methods approach to data collection, blending qualitative responses and quantitative ratings, drawing on multiple data types to build as complete a picture as possible of the student learning experience.

The SFS survey seeks students' perceptions of their learning experience at a subject-level: these ask students to respond to eight quantitative questions by rating a series of statements on a five-point scale, and also gives them the opportunity to respond to two open-ended quantitative questions.

The SFT survey seeks students' perceptions of teaching, and simply asks students to respond to twelve quantitative questions, again by rating a series of statements on a five-point scale.

A major advantage of a mixed methods approach to educational research is that it helps to capture the richness and complexity of human behaviour. Generally, the validity and reliability of data can be enhanced by the use of two or more methods of data collection; this is known as triangulation.

In this case, the quantitative data allows us to see a broad picture of the student learning experience in your subject; the qualitative data provides a more in-depth picture, can help us to better the specific issues and concerns influencing that experience. In both cases, to best maintain the validity and reliability of data, you should be aware to look out for common interpretive errors, such as confirmation bias, causal error, illusory confirmation, and observer bias.

Interpretation

Interpretation is subjective. Throughout this guide, questions of how we choose interpret this data remain deliberately open to accommodate diverse approaches to educational research, your own disciplinary, epistemological, ontological, and methodological perspectives, and above all the specific context of your subject.

Teaching and learning are contextual. Remember that student feedback is provided in relation to a particular teaching context, and should therefore be interpreted in relation to that context. Generally, you are the one person who understands that context better than anyone else.

Further resources

  • Watch: "What is educational research?" by The Education and Training Consortium, University of Huddersfield. 29 July 2013. https://www.youtube.com/watch?v=5ucLcy_3jZo
  • Read: For a good introduction to the broader context of educational research, see Louis Cohen, Lawrence Manion, and Keith Morrison. 2013. Research Methods in Education. 7th edition. Taylor and Francis. 1-30.
  • Watch: "What is Mixed Methods Research?" by John W. Creswell, University of Nebraska-Lincoln. 19 February 2013. https://www.youtube.com/watch?v=1OaNiTlpyX8
  • Read: For an overview of mixed methods research as it applies to educational research, see: John W. Creswell. 2008. "Mixed Methods Research." In The Sage Encyclopaedia of Qualitative Research Methods. Ed. Lisa M. Given. Thousand Oaks, CA: SAGE Publications. 527-530. http://dx.doi.org/10.4135/9781412963909.n269

Response rates

On the cover page of both your Qualitative and Quantitative Reports you will see three significant numbers.

  1. The first is "Project Audience," which indicates the total number of students invited to complete the survey.
  2. The second is "Responses Received," which indicates the actual number of students who completed the survey.
  3. The third is "Response Ratio," which indicates the percentage of students who completed the survey out of the total sample.

Low response rates present a significant problem for the validity and reliability of data. Generally, the minimum response rates required for validity vary depending on the number of students enrolled in the subject:

  • <25 – 70%
  • 25-50 – 50%
  • 50-100 – 40%
  • >100 – 25%

In cases where response rates fall too low, alternative data sources can be useful: learning analytics, classroom-based surveys and evaluation activities, and focus groups can all provide invaluable information and should be considered in the evaluation of subjects and teaching.

Quantitative feedback: Numerical ratings

While some aspects of education are simply irreducible to quantitative analysis, this data can be quite useful. The SFS asks students to respond to eight quantitative questions, and the SFT twelve questions. In both cases, students rate a series of statements on a five-point Likert scale (strongly agree, agree, undecided, disagree, and strongly disagree). From this data, we can extract a number of descriptive statistics to build a broad picture of the student learning experience.

The first thing you'll encounter inside your quantitative reports is a series of bar graphs. The first graph summarises the survey results across all questions (sampled below), and the subsequent graphs display a breakdown of results question-by-question.

In order, the five columns of each bar graph represent:

  1. The average rating for your subject;
  2. The average rating for all subjects at La Trobe University.
  3. The average rating across all subjects in your School;
  4. The average rating for all subjects at your campus; and
  5. The average rating for all subjects in your School at your campus.

This allows you to see your own student feedback in the broader context of trends within your School and across the entire university. Typically, any ratings around and above 4 are fine, but anything that starts to dip too far below you should try to address. However, you should always consider the representativeness of the responses.

The report then goes on to provide a percentage breakdown of responses to each question, as well as a number of purely descriptive statistics (sampled below). Averages do not always tell a meaningful or useful story. If a reasonable number of students responded, then have a look at the mean, median, mode, and standard deviation.

Mean refers simply to the average rating across the cohort; median refers to the rating given by the middle respondent in the cohort; and mode refers to the rating given by the greatest number of respondents in the cohort. The range of responses on quantitative questions (that is, distance between the highest and lowest ratings), as indicated in the bar graph above, tells a richer story than the average alone. Standard deviation is a measure of the dispersal or range of ratings, calculated as the square root of the variance (that is, a measure of how far scores are from the mean). You will need to check the students' qualitative comments to enable you to interpret each issue if the deviation is large.Firstly look at each item separately, and then consider all items holistically. Target your highest and lowest rated items to focus on areas for specific improvement, or to help identify the strengths of your subject and/or teaching. 

Rating explanations

Student Feedback on Subjects (SFS)

1. The learning outcomes of the subject were made clear to me

A low rating on this items typically indicates that students are unclear about what is required of them in your subject, and exactly what they are learning from your subject. Clearly state your intended learning outcomes from the outset, and ensure students are aware how these are reflected in your assessment tasks, as well as in the learning activities and resources.

It is also a good idea to have students complete a short pre-semester survey to prompt them to reflect on their reasons for enrolling in your subject, to gather more information about what they know, what they expect from the subject, and how they think they learn best. Teaching practice may need to be adapted based on some of this information.

2. The subject enabled me to achieve the learning outcomes

Students tend to agree with this statement when they feel that the high expectations you set out for them matched their own reasons for taking the subject, and when they have had opportunities to assess their success toward achieving those intended learning outcomes. If you rate low on this item, consider how clearly aligned your ILOs are with your assignments, learning activities, and resources.

3. I found the subject to be intellectually stimulating

To best engage students in the subject material, set high but achievable standards, and reward every student who meets them. This is a matter of finding a balance in your subject between students' current knowledge and skills, and realistically challenging those in order to maintain engagement and motivation. Students feel most engaged in learning when they have lots of opportunities and promptings to reflect on the content of your subject. 

4. I found the resources provided for the subject to be helpful

Do not include a lot of readings or other resources that might be relevant but not central to the subject. Your syllabus can help students see the underlying structure of your subject. Coordinate resources with classes (whether lectures, seminars, tutorials, labs or pracs). Tell students why you have assigned particular readings, videos, or podcasts, and how they relate to specific activities and assessment tasks, and support intended learning outcomes.

5. I received constructive feedback on my work

Feedback has a very significant impact on learning. It is not only a tool used to justifying to students how their mark was derived, but also about guiding students on what steps they can take to improve their work. It is an integral part of the learning process.

Feedback is valuable when it is received, understood and acted on. To benefit student learning, feedback needs to be constructive, timely, and meaningful. As well as highlighting the strengths and weaknesses of a given piece of work, it should set out ways in which the student can improve the work. It should also target individual needs, be linked to specific assessment criteria, and be received by a student in time to benefit subsequent work. How students analyse, discuss and act on feedback is as important as the quality of the feedback itself.

If you rate low on this item, you may need to consider amending your assessment and feedback strategy to provide more opportunities for the provision of and reflection on constructive and meaningful feedback.

6. The feedback I received was provided in time to help me improve

The timing of feedback is crucial. Give feedback while the assessed work is still fresh in a student's mind, before the student moves on to subsequent tasks, and structure your assessment strategy so that students are given time to improve the work based on timely feedback.

La Trobe University's Assessment Policy states that: "Feedback, including marks or grades, on all assessment tasks other than final examinations or honours theses should normally be provided within three (3) weeks of the submission date. The nature of feedback should be consequential, bearing in mind how it will inform future subject or course requirements. It should provide specific information, against the relevant assessment criteria, about what has been done well, what has not and how work could be improved."

7. The overall amount of work required of me for this subject was appropriate

Set realistic expectations. Temper your own usually high expectations with the reality of students' likely workloads across three or four subjects, and/or external factors such as part-time or even full-time work, or other personal and family commitments. Was the non-contact or out-of-class workload for the subject what you had planned? How much variation is there in students' responses? A good strategy is to sit down and clearly map out exactly what you're expecting your students to do each week, and over the semester, both in and outside of class – including the time you're expecting them to devote to the subject. Is this realistic?

8. Overall I was satisfied with the quality of this subject

Students generally give high marks for overall quality when they feel that they understood very clearly the objectives or outcomes of the subject, and these coincided with their own reasons for taking the subject. Unclear or mismatched sets of expectations generate frustration and inhibit learning.

Student Feedback on Teaching (SFT)

1. The teacher helps me to achieve the subject's learning outcomes

This item looks specifically at how well your teaching practice aligned to the intended learning outcomes, and how clearly these are reflected in your learning activities. It is of paramount importance to set clear objectives for each class session. These key points are not necessarily specific pieces of knowledge; they can be themes or ideas that provide a framework for understanding and use specific pieces of knowledge.

In your preparation for class, outline how each key point will be covered, and how each key skill will be demonstrated and explained. Before class, take five or ten minutes to mentally walk through your outline, focusing on key transitions, explanations, and demonstrations. At the beginning of each session, share your outline with students or preview the main ideas. During the session, highlight the key points as you cover them. Summarise these points at the end of class. Make sure students also understand how to use the information you have given them, and give them opportunities to practice key skills.

2. The teacher clearly communicates ideas and information

Clear and effective communication is no always easy. If you rate lowly on this item, consider one or more of a number of strategies:

  • Rehearse the main points before class, and practice talking through complex concepts or demonstrations.
  • Clearly signal shifts in topics and explain the connections between various topics.
  • Don't assume that students will understand critical links intuitively. Talk students through important reasoning.
  • Take into account the students' background knowledge (or lack of background knowledge), and try to anticipate common confusions or questions.
  • Leave adequate time to cover the material without rushing.
  • Explain key points in more than one way. Use memorable examples, analogies, or anecdotes to illustrate important points.
  • Link new information to previous concepts to help students distinguish ideas and contextualise information.
  • When relevant, use your own research or professional experience to explain or discuss course material.
  • Check for understanding.
  • If a number of students approach you after class or during office hours regarding a particular concept, review it in class.
  • Encourage students to ask you questions face-to-face or online, so that you can clarify any confusion.

3. The teacher provides constructive feedback on my work

Students are motivated by clear expectations and timely feedback. Feedback is most valuable when it is received, understood and acted on. As well as highlighting the strengths and weaknesses of a given piece of work, it should set out ways in which the student can improve the work. It should also target individual needs, be linked to specific assessment criteria, and be received by a student in time to benefit subsequent work. How students analyse, discuss and act on feedback is as important as the quality of the feedback itself.

Balance critical comments with genuine positive feedback. Recognise students' efforts and achievements, whether it is an insightful comment in class, an outstanding paper, or an improved test score. Use incorrect answers as opportunities to examine misconceptions, and encourage students to develop their original responses. Avoid correcting students in ways that embarrass them, such as suggesting that a concept is "clear," "simple," or "obvious."

Return and review assignments in class. Spend time summarising what you learned from marking students' assignments. Help students understand how to improve future performance, if necessary.

4. The teacher provides feedback in time to help me improve

The timing of feedback is crucial. Give feedback while the assessed work is still fresh in a student's mind, before the student moves on to subsequent tasks, and structure your assessment strategy so that students are given time to improve the work based on timely feedback.

5. The teacher inspires intellectual stimulation and interest in the subject

Demonstrate your enthusiasm about the subject matter. Let students know why you love your field. Show students how the course material can be applied to real life. Begin classes with thought-provoking questions or recent news events that relate to the topic of discussion.

Model your thinking processes and problem-solving by working through actual examples during class. Follow up by having students tackle similar problems. Understand enough to generate questions; if students don't volunteer questions, you can ask them questions to make sure they understand the material.

Encourage students to explore their specific interests in your field by allowing students to choose between readings or assignment topics.

Encourage students to interact outside of class, by forming study groups or suggesting relevant out-of-class activities (talks, demonstrations, events, etc.).

6. The teacher is available and helpful when asked

Encourage students to contact you by email, or using some of the options available through the LMS, such as the Forum or Chat activities. These conversations are often the easiest way for a student to contact a teacher. They are less intimidating, give students an opportunity to plan their thoughts, and can be written at any time. Let students know at the beginning of the semester how often/rapidly you are likely to respond.

Advertise your office hours. Encourage students to visit your office hours. Do not schedule student meetings "by appointment only" – this creates a barrier to students.

Try to anticipate students' concerns and possible questions, based on your previous teaching experience or conversations with students and colleagues. Prepare thoughtful responses to the most likely questions. If you are unable to answer a student's question in class, consider researching the answer and reporting back to the student privately or in class. When you talk to students outside class, note any common questions or confusions. Address those issues in class.

7. The teacher organises and uses time effectively to promote learning

Preparation is the foundation of an organised and logical class. Prepare an introduction to each class that sets a clear and engaging agenda for the day. Prepare and practice a short conclusion that will tie the strands of the lecture together and place the lecture in the wider context of the course. Be sure that any materials you need for the class are organised and working properly, and rehearse any demonstrations.

8. The teacher assesses, marks, and grades fairly

Be consistent when evaluating papers, projects, essays, or exams. Give students explicit guidelines, and consider preparing marking rubric based on these guidelines. Share your marking criteria with students, and provide them with opportunities to engage with and reflect on this in practice. Students always appreciate (and deserve) some comments explaining the mark.

9. The teacher treats students and their ideas with respect

Students are more strongly motivated to succeed in a class when they feel respected. Students especially appreciate teachers who are approachable, show genuine concern about their success, and seem to truly enjoy teaching. Students will invest more in their learning, and be more willing to address learning difficulties, if the student-teacher relationship is strong. For some students, these interpersonal elements of your teaching will determine the difference between success and failure. Demonstrated concern about whether students were learning.

If you are teaching students who are new to the university, take time during the first class to tell them about relevant academic advising services and university policies. As the semester progresses, continue to let your students know about departmental resources and events.

10. The teacher uses resources and materials effectively

The purpose of learning resources is to provide a basis for learning experiences for students. This includes not only textbooks, workbooks, and audio-visual content, but also digital and online learning materials, resources in the natural environment, people, libraries, etc. All of these should be drawn upon to help students to learn, broaden their learning experiences and meet different learning needs. If used effectively, they will help students to construct knowledge for themselves, and develop the learning strategies, generic skills, values and attitudes they need, thus laying a solid foundation for lifelong learning.

11. The teacher requires an appropriate amount of work of me for this subject

Set realistic expectations. Temper your own expectations with the reality of students' likely workloads across three or four subjects, and/or external factors such as part-time or even full-time work, or other personal and family commitments.

12. Overall, the teacher provides good teaching in this subject

Students generally give high marks for overall quality when they feel that they understood very clearly the objectives or outcomes of the subject, and their teachers has helped them to achieve these. Unclear or mismatched sets of expectations generate frustration and inhibit learning.

Further resources

  • Read: For a more detailed overview of approaches to quantitative data analysis in the specific context of educational research, see Louis Cohen, Lawrence Manion, and Keith Morrison. 2013. Research Methods in Education. 7th edition. Taylor and Francis. 604-621.

Qualitative feedback: Written comments

Qualitative data is only collected in the SFS survey, in the form of written comments. This survey provides students the opportunity to respond to two open-ended questions:

  1. What were the best aspects of this subject?
  2. What aspects of this subject were most in need of improvement?

The qualitative report simply collates all responses. These responses can provide extremely helpful elaboration or explanation of your quantitative data.

The analysis of qualitative data typically involves a lot of organisation and explanation. The aim here is to clarify and summarise the data – to note patterns and trends, generate themes, discover regularities, differences and similarities, while maintaining the integrity and wholeness of the data, and to not taking it out of context.

  1. Start by reading through all responses to each question separately, highlighting any common themes or trends, issues, problems, or concerns. Look over all the comments and try to identify any patterns before discounting individual comments.
  2. Feel good about glowing comments you receive about your teaching, but keep them in perspective. Keep unfavourable comments in perspective, too.
  3. Determine how you will organise your data. You may like to start tabulating your data, or even coding your data.
  4. Group the comments by teaching components (i.e., curriculum design, assessment, feedback, teaching delivery, learning activities, resources and materials, communication, interaction, etc.) or by categories that are most meaningful to you.
  5. Try to summarise each theme in one or two sentences, and perhaps extract a good example of typical student perceptions concerning this theme.
    1. We're looking simply for at what works in the current context, giving you a clearer understanding of how your students are learning. This data can also be very useful in applications for promotions, grants and awards, particularly if you can track trends in data over time and display evidence of clear improvement.
    2. We're looking in particular for any points of pressure, problems or concerns, which are negatively affecting student achievement, engagement, or satisfaction.

Positive and negative comments

Students who are either very satisfied or very dissatisfied generally provide written comments. It is helpful to determine the proportion of negative to positive comments for interpretative purposes. This will assist you in determining if the comments are representative of the entire class or a small minority of students.

Comments that reflect positively on your teaching effectiveness can usually be considered genuine. Since the course evaluation is anonymous, students do not usually write positive comments unless they mean them.

Negative reactions from one or two disgruntled students may not accurately reflect the perceptions of the entire class. However, some negative comments are meant to be constructive. Obviously, not all of these comments are constructive. Pressures unrelated to you or your course may also underlie some of these comments. Keeping this in mind may help to limit overreaction to certain comments. Negative comments need to be interpreted with caution. Normal human behaviour often causes one to take the negative comments to heart, regardless of how small the number.

Further resources

Read: For a more detailed overview of approaches to qualitative data analysis in the specific context of educational research, see Louis Cohen, Lawrence Manion, and Keith Morrison. 2013.Research Methods in Education. 7th edition. Taylor and Francis. 537-558.

The next steps

Carefully consider what the student feedback is telling you. Try to write a short summary of your feedback (1-2 paragraphs), in the case of the SFS synthesising what you have learned from both the quantitative and qualitative reports. You should have a good picture of what works, and what needs to be improved.

The next step is to evaluate which of the issues identified (if any) should be acted upon, and how. This could mean making minor or major changes to an aspect of curriculum design and structure, assessment and feedback strategies (particularly timing of assessments), modes of delivery, learning resources, student workloads, etc.

Remember that you are the one person who understands the context of your subject better than anyone else, but also think about having a peer of colleague (from within or outside of the same discipline) look at the data with fresh eyes, to bring a new perspective to any issues – it is always useful to have a critical friend on hand.The Educational Development team are also available for individual or group consultation to help you interpret your student feedback data, advise you on which actions you can take – contact your LTLT College Partner to organise a consultation (see below).

Responding to feedback

We recommend that you communicate to students at least some of the things you have learned and at least some of the actions you will take in response to their feedback, and this qualitative data will provide the basis for your response. One basic framework for responding to feedback is to tell students:

  • Which suggestions will be acted upon and how;
  • Which suggestions you would like to act on but are unable to and the reasons why; and
  • Which suggestions you will not be acting on and why.

It is mandatory to provide some response to Student Feedback on Subject (SFS) surveys. Ideally, a summary of SFS results should be posted in the top section of the subject's current LMS page sometime after the release of results, and should be included on the LMS page and in the 'Student Feedback on Subject Survey' section of the Subject Learning Guide the next time the subject is delivered.

Contact your LTLT College Partner