Viewing Job Reviews Data
Job Reviews Tab
Once your school has activated the job review survey, you will be able to view the results through the Job Reviews Tab. In order to see this data, ensure that you have the "Manage Students" security role. Contact your team's Handshake power user in order to get access if you do not.
- Click Job Reviews on the left navigation bar
- Filter to a specific set of reviews
- Note: Filter to Public and Approved Reviews to reviews that students have opted into being public
- Click on the review to see the details, including:
- What the student liked
- What they wish had been different
- Interview process
- Interview questions
Once your school has activated the internships ratings and reviews survey, you will be able to view the results through Insights. In order to see this data, ensure that you have the "Explore Insights" security role. Contact your team's Handshake power user in order to get access if you do not.
Through Explore Insights, you will now see an additional explore called Job Reviews:
You can create your own custom reports of your job ratings and reviews, or you can use the common reports we've created for you below! Simply click the link below while you are logged into Handshake in another window and this report will begin running:
- Count of job reviews by school year
- Count of job reviews by major
- Employers with the most job reviews
- Employers with the hardest interviews
- Employers with an overall rating of 5
- All job reviews submitted in the last year
Activating Internship Ratings & Reviews within Handshake
To learn more about collecting internship reviews at your school contact your account manager!
- Why collect review data?
- While sites like Glassdoor are already popular with students, career centers get no insight into this data. Handshake can bridge this gap, and in so doing bring tremendous value to both career centers and students:
- Student benefits to internship review data:
- More informed career exploration and decision-making
- Get advice specific to the companies and roles you’re interested in
- Gain insight into company culture, mentorship, learning etc. that is often harder to gauge
- Having access to internship review data will allow career services to:
- Get comprehensive data on internship and co-op outcomes, enhancing student lifecycle reporting
- Build relationships with new employers that your students love
- Enhance your career counseling efforts by referencing tangible stories of student experiences
- How will this data be presented to students?
- The pilot is not focused on displaying reviews and ratings back to students yet. We’ve heard good questions about how we will present these reviews back to students, how we will monitor reviews and how we will take employer concerns into consideration, among others. We have some answers to these questions, but not all. We first need to understand the scale, quality and depth of responses we’ll get from students.
- After we gather reviews in this pilot, we will analyze the student data together with our career services partners before proceeding. No matter what, you will have access to all the data gathered from your school’s students (positive or negative), through our reporting tools in Handshake later this fall.
- During the pilot, we will not display any negative reviews or any numerical / star ratings (positive or negative) back to students. For now, students will only see positive text reviews from other students (free text stories, not numerical ratings).
- Importantly, students can participate but still opt-out of sharing this data with other students. If a student does opt-out of sharing with students and employers, that student’s data will only be shared with their career center.
- What if employers get upset about reviews they consider negative?
- During the pilot, we will not display any negative reviews or any numerical / star ratings (positive or negative) back to students. For now, students will only see positive text reviews by other students (free text stories, not numerical ratings).
- Long-term, we are exploring how to balance employer concerns. At minimum, we will introduce some way for other students, career services and employers to flag reviews to our support team for violating our TOS and content guidelines. Beyond that, we are focused mostly on helping more students by providing more transparency and information to them around the job process.
- Will you allow anonymous reviews?
- During the pilot, we are not allowing any anonymous reviews like Glassdoor. The survey will take place within Handshake, so we can tie every review to an actual student. This will also help to ensure the validity of the information. If, in the future, an employer complains about a review and questions whether the student even worked at the company in question, we can verify the information directly.
- Will students be able to see data just at his or her school, or across all Handshake schools?
- We have not yet finalized how we will present data back to students. We will do so after the pilot data gathering. However, our draft plan today is that students would see reviews from both the students at their school and from students across the Handshake network, so that everyone can get value. Students will be able to see aggregate data for just their school, as well as the reviews of that employer across all of Handshake.
- If my school doesn’t participate, can my students still see this data from other schools?
- No. For now, the pilot data will only be accessible to students at schools that are participating in the pilot.
- Can students choose to just share this data with their career center, and not other students and employers?
- Yes, each student will be able to opt-out of sharing the data with students and employers. If they choose to opt-out, only Handshake and the student’s career center will see their responses.
Collection of Reviews
- Can we see the form you used to collect the initial round of reviews?
- For Phase I of the pilot (focused on collecting reviews), we emailed students inviting them to share a review of their internship experience. The email contained a link to a survey within Handshake.
- You may find images of the form here.
- 2. Will you continue collecting reviews?
- Initially, this spring, there will not be a way for students to submit additional reviews in Handshake. At this point, we are focused on bringing reviews into the product and seeing how students react to reviews, before we introduce the ability to collect more.
- We will circle back in April or May with further details about how future collection will work. We expect Fall 2018 collection will include some combination of email survey, prompts on the profile and the dashboard. We welcome your feedback.
Content Moderation Process
- Can you provide an overview of how the review process works?
After students submit their review, their submission undergoes a two-step moderation process:
- Automatic text analysis: highlights any questionable language for review
- Moderator manually scans reviews, decides whether to publish content
- Reviews 100% of reviews flagged by text analysis
- Spot-checks 20% of all reviews against other content guidelines
- Anything that violates guidelines is still visible to Career Centers
There are also mechanisms post-publication for career centers, other students, and employers to flag reviews that do not meet our content guidelines. After a review is flagged:
- The review is taken down pending secondary moderation
- Flagged content is reviewed again, with notes from user who flagged it.
- Review is still judged against content guidelines, not employer “wishes”
- Aiming for SLA of 3 business days
- Are Career Centers a part of the approval process?
- Handshake will handle the entire approval/moderation process, with the aim to put no additional burden on schools.
- Career Centers have the ability to flag any approved reviews they feel do not meet our Content Guidelines for secondary moderation, in the process described in the question above.
- Will schools be expected to take over moderation responsibilities?
- No, Handshake has (and will continue to have) responsibility for the moderation process, with the goal of not adding any additional burden on schools.
- Where can we find the Content Guidelines?
- You may find our Content Guidelines below. Once reviews are live in Handshake, the guidelines will be easily accessible for students, employers and career centers within the product and the Help Center.
- All reviews must be:
- Authoritative: The reviewer must have worked for the employer being reviewed
- Unique: Only one review per internship or job experience
- Original: All reviews should be original
- Comprehensible: All reviews should be literate and comprehensible
- Not Personal: No negative reviews that identify individuals or contact info
- Not Profane: No profanities, threats of violence or discriminatory language
- Not Confidential: Cannot reveal confidential company information
- Not Spam: No advertisements or irrelevant content
- What is your timeline for approving or declining reviews?
- We hope to have all existing collected reviews moderated before reviews appear in Handshake in early March. Following launch, we hope to have an SLA of 3 - 4 business days. Most reviews should be approved within a few hours.
- What about industries with higher standards of confidentiality? How do you catch sensitive but not “inappropriate” content?
- This provision is covered in our Content Guidelines, under the “Not Confidential” clause. With feedback from several employers in these more “sensitive” industries, our employment and privacy counsel developed extensive guidelines for our moderators to ensure we are not publishing confidential company information.
- What is your process for handling reviews that report harassment or illegal activity in general?
- Our process is as follows:
- Handshake will report any review that mentions illegal activity directly to the Director of the Career Center (and main POC)
- Handshake WILL tell the student the review was referred to the Career Center.
- Handshake will NOT show the employer the review, or tell them it has been referred to a Career Center.
- Handshake will NOT publish the review until we hear back from the Career Center.
- How are you building the auto-flag terminology lists? So for example "got high with coworkers" doesn't have obviously inappropriate words but is coded language for inappropriate activity.
- Our text analysis is compiled from industry-standard lexicons for this type of user-generated content. The lexicon also includes hundreds of phrases in addition to isolated words to ensure maximum coverage.
Career Services View of Reviews
- If reviews are being shown across schools, how will reviews be divided by school in the Career Services view?
- For now, Career Services will only be able to see reviews from your own students in the “Job Reviews” tab.
- Can staff receive alerts when reviews are flagged?
- We do not currently have this functionality, but will circle back with our plans in this regard.
- For multi-office campuses, can we get the ability to filter by schools?
- We do not currently have this functionality, but will circle back with our plans in this regard.
- Can career services see pending reviews? Or only once they are approved/declined?
- Career Services can see all reviews from your students, regardless of the status of the review.
- What does “pending” mean?
- The “pending” status means that a review has yet to be moderated by Handshake. This is the default status for all reviews until they have been through our moderation process. Currently only Handshake moderators have the ability to change the status of a review from “pending” to “approved” or “declined.”
- We hope to have all existing collected reviews moderated before reviews appear in Handshake in early March. Following launch, we hope to have an SLA of 3 - 4 business days.
- Who determines the status of a review?
- Handshake moderators determine the status of each review, after it has been through our moderation process.
- The majority of a reviews are still pending. Are you still processing them?
- Yes, we are. We hope to have all existing collected reviews moderated before reviews appear in Handshake in early March. Following launch, we hope to have an SLA of 3 - 4 business days.
- Where do employers see their reviews in Handshake?
- Employers will see their reviews on their company pages, exactly as students see them. They will also be able to flag and reply to reviews on that same page.
- Will all contacts at an employer be notified when they get a new review, or just one point of contact?
- By default, all recruiters for a given employer will receive in-app (not email) notifications every time a new review is published.
- They will have the option to opt-in to email notifications on an individual basis if they so choose. We will make this clear to them in our communications to employers and Help Center resources.
- What happens if a company has chronic bad reviews? Does this impact their trust score?
- We want to address instances where it appears that students are consistently having bad experiences at a given employer. Currently, we do not have a default consequence for chronic bad reviews. We also have no plans for the quality of reviews to impact employer trust scores at this time.
- What if reviews are left for employers not currently on Handshake?
- If reviews are left for employers without a Handshake account, we currently have no way of displaying that review in product.
- However, with 230k+ employers currently on Handshake, the vast majority of reviews are attached to Handshake employers.
- What if reviews upset an employer?
- We’ve found the vast majority of submitted reviews to be thoughtful, constructive and high-quality. None of the employers we’ve spoken to have raised issues with example approved reviews we’ve shown them. On the contrary, they’re excited about the ability to engage with students in new ways.
- If an employer believes a review does not meet our Content Guidelines, they are able to flag that review for secondary moderation. Employers also have the ability to reply to reviews, a great opportunity to proactively address feedback and show students they are engaged and listening.
- Will only students at pilot schools be able to see reviews?
- For the pilot this spring, reviews will only be available for schools in the pilot. Our goal of the fall is that reviews are available at every Handshake school this fall.
- Is there any way for a student to write an anonymous review?
- Students can choose to share their reviews publically, or keep it private to only their career center.
- For students who share publicly, they have two options:
- Fully visible: Display full name, photo, school, school year and major
- Semi-anonymous: Hide full name + photo, but display school, school year and major
- After talking to students, the overwhelming feedback was that totally anonymous reviews were significantly less valuable. This is because some context around the reviewer is important for understanding if a review is relevant to a student. For example, an English major at a small liberal arts college may find a review from an engineering student at a large state school interesting, but less relevant to them personally.
- Students will also get a chance to change their review before publishing. For this pilot, we will let students edit their review or change their privacy before publishing.
- For students who share publicly, they have two options:
- Can students make their reviews visible only to other students and not employers
- We are making it clear to students that if they choose to share their reviews publicly, they should expect that employers will be able to see it. Because even if Handshake limited access to “just students,” employers could well still see their review (for example, by asking one of their interns to show them reviews via the intern’s student account). We are optimizing for ensuring we do not create a false expectation of privacy by a student.
- Do students at my school only see reviews from other students at my school?
- Students will be able to see reviews from all pilot schools across the Handshake network. This is a core principle of our Reviews feature. We believe that harnessing the power of the network will provide a significantly more valuable experience, for students to be able to see reviews from a far broader set of employers and job functions. For example, if you are a student at a technical school in Arizona, but are interested in working in design in New York, you will likely benefit from reviews shared by design students at a school in the New York area.
- In the reviews collection process, you collected star ratings from students. How are you planning on using these ratings?
- After analyzing the ratings we collected, we found them to be unhelpful, as originally constructed on a one-to-five-star scale.
- Ratings were overwhelmingly positive - the average rating was 4.5 out of 5. They therefore provided very little signal to differentiate employers and provide value to students.
- We will instead show “highlights,” labels on employer pages that will appear for employers with high ratings (rather than a numerical score).