Skip to end of metadata
Go to start of metadata

Problem

Users are appearing with the incorrect rating in the Matrix Grid Report or they are appearing in the Unrated section. 

Solution

Identifying if a given user has ratings and, if it has, what is the latest rating are key steps in troubleshooting rating issues with the Matrix Grid Report. This article explains how to understand the process that you have configured for your Matrix Grid Report and how to find the ratings for a given user based on the process configuration. 

The first step is to understand your Matrix Grid Report configuration. For this, please review this page: Matrix Grid Report - Understanding the source of ratings

The next step will be reporting on the ratings from Employee Profile, Performance Management and Calibration as needed, based on the process configuration of your Matrix Grid Report. 

Employee Profile Domain

The first step of reporting on the ratings involves using the Employee Profile Ad Hoc Report domain, since it will display ratings from the Live Profile, Scorecard, Performance Forms and completed Calibration Sessions, but it will not show Unrated ratings. If the process of your Matrix Grid Report only includes Performance Forms (i.e. if you are not using “All Data Sources” and the Trend Element is not included under “Succession Data” in the process configuration), you can skip this step and only run the Performance Management report. 

  1. Go to Reporting > Report Center > click “New” > select “Report – Table” > click “Select” (or, if you don’t have the new Report Center enabled, go to Ad Hoc Reports and click “Create New”).
  2. Select “Single Domain” and select “Employee Profile” in the domain dropdown > click “Select”.

  3. Give a name to the report.
  4. In the “People” tab, click “Refine Criteria” > select “Other Filters” > click “Ok”.
  5. In the “Columns” tab, click “Select Columns” and select the following:
    1. Employee Information – Username
    2. Employee Information – First Name
    3. Employee Information – Last Name
    4. All Ratings Information – Select All
  6. Still in the “Columns” tab, click “Sort Columns” > Add another sorting level > select “Rating End Date”.
  7. In the “Filters” tab, click on “Refine Criteria” and filter based on:
    1. Employee Information - User Sys ID or Username: filter by the user that you are using to troubleshoot the issue.
    2. All Ratings Information – Rating Type: here, you’ll filter by the Trend Elements that you identified in the Matrix Grid Configuration (a. What is the Trend Element selected for each axis?). For example: Performance.

    Tip: Filter by one rating type at a time for easier readability.

  8. Save and generate the report.
  9. Identify the latest rating (based on the End Date) that the report shows.
  10. Export the results for reference. Next, you will run Performance Management and Calibration reports, to identify Unrated or in-progress ratings coming from those sources.

Performance Management Domain

  1. If you have Performance Forms included in the process of your Matrix Grid Report, you’ll need to run this report to identify any Unrated ratings coming from Performance Forms.
  2. Go to Reporting > Report Center > click “New” > select “Report – Table” > click “Select” (or, if you don’t have the new Report Center enabled, go to Ad Hoc Reports and click “Create New”).
  3. Select “Single Domain” and select “Performance Management” in the domain dropdown > click “Select”.
  4. Give a name to the report.
  5. In the “People” tab, click “Refine Criteria” > select “Other Filters” > click “Ok”.
  6. In the “Data Sets” tab, select only the relevant templates, according to the process configured in your Matrix Grid Report. If the process is “All data sources”, you need to select all. If it’s another process, then select only the forms that are included in your process configuration.
  7. In the “Columns” tab, click “Select Columns” and select the columns outlined below. Instead of “Trend”, select the columns related to the Trend Elements configured for your Matrix Grid Report (for example, “Performance”).
    1. Form – Form Template Name
    2. Form – Form Title
    3. Form – Status
    4. Form – Form Start Date
    5. Form – Form End Date
    6. Subject – Username
    7. Subject – First Name
    8. Subject – Last Name
    9. Trend – Overall Trend Rating
    10. Trend – Overall Trend Rating Description
    11. Trend – Overall Trend Rating Scale Min (Competency and Objective only)
    12. Trend – Overall Trend Rating Scale Max (Competency and Objective only)
    13. Trend – Trend Scale Min (Performance and Potential only)
    14. Trend – Trend Scale Max (Performance and Potential only)
  8. Still in the “Columns” tab, click “Sort Columns” > Add another sorting level > select “Form End Date”.
  9. In the “Filters” tab, click on “Refine Criteria” and filter based on:
    1. Subject - User Sys ID or Username: filter by the user that you are using to troubleshoot the issue.
    2. Form – Form Status:

      1. If “Exclude in-progress forms when retrieving the most recent ratings” is enabled: filter by “Completed” forms only.

      2. If the above setting is disabled: filter by all statuses, except “Deleted”. For example, include “Modify Stage” and “Completed”.

  10. Save and generate the report.

  11. Identify the latest rating (based on the End Date) that the report shows.

  12. Export the results for reference. Next, you will run the Calibration report, to identify Unrated or in-progress ratings coming from Calibration.

Calibration Domain

If the process of your Matrix Grid Report only includes Performance Forms (i.e. if you are not using “All Data Sources” and the Trend Element is not included under “Succession Data” in the process configuration), you can skip this step and only run the Performance Management report. Otherwise, this report is necessary to understand any Unrated or in-progress ratings coming from Calibration. 

  1. Go to Reporting > Report Center > click “New” > select “Report – Table” > click “Select” (or, if you don’t have the new Report Center enabled, go to Ad Hoc Reports and click “Create New”).
  2. Select “Single Domain” and select “Calibration” in the domain dropdown > click “Select”.
  3. Give a name to the report.
  4. In the “People” tab, click “Refine Criteria” > select “Other Filters” > click “Ok”.
  5. In the “Columns” tab, click “Select Columns” and select the following:
    1. Cal Session Subject – Subject User Name
    2. Cal Session Subject – Subject First Name
    3. Cal Session Subject – Subject Last Name
    4. Cal Session – Session Name
    5. Cal Session – Session Status
    6. Cal Template – Select All (Template Name, Cycle Start Date and Cycle End Date).
    7. Cal Audit Trail – Audit Data Type
    8. Cal Audit Trail – Old Value
    9. Cal Audit Trail – Old Value Label
    10. Cal Audit Trail – New Value
    11. Cal Audit Trail – New Value Label
  6. Still in the “Columns” tab, click “Sort Columns” > Add another sorting level > select “Cycle End Date”.
  7. In the “Filters” tab, click on “Refine Criteria” and filter based on:
    1. Cal Session Subject - Username: filter by the user that you are using to troubleshoot the issue.
    2. Cal Session – Session Status:
      1. If “Show In-Progress Calibration Ratings In Live Profile” is enabled, filter by all statuses, except “Deleted”. Include “Approved” and “In Progress”.
      2. If the above setting is disabled: filter by status “Approved” only.  
    3. Cal Audit Trail – Audit Data Type: filter by the Trend element that is configured in your Matrix Grid Report configuration. For example, “Performance”.
  8. Save and generate the report.
  9. Identify the latest rating (based on the End Date) that the report shows. The rating to be considered is the “New Value” and “New Value Label”.
    1. Unrated ratings will display as empty values for New Value and New Value label.
    2. Too New To Rate ratings will display as “Too New To Rate”.
    3. Regular ratings will display the rating number (value) and label.
  10. Export the results for reference. 

Analyzing the results

After running the above reports, we can review the three of them (or the ones you had to run according to the configured process and configuration) and identify the latest rating for the user.

For the reports you ran, identify the rating with the latest End Date (Rating End Date, Form End Date, Cycle End Date) and this is what is considered by the Matrix Grid Report.

If you have the checkbox “Use average of all found ratings, instead of latest” enabled, you’ll need to calculate the average of all found ratings (for the date range you are running the report with, always based on the end date of the rating).

Common Issues

  • The latest rating is an “Unrated” rating coming from an in-progress Performance Form or Calibration Session. In this case, you should consider enabling the checkbox “Exclude in-progress forms when retrieving the most recent ratings” if it is from a Performance Form or disabling the checkbox “Show In-Progress Calibration Ratings In Live Profile” if it is from a Calibration Session. 

  • The latest rating is an “Unrated” rating coming from a completed Performance Form. In this case, there are two possibilities: 
    • The user incorrectly completed the form without putting an Overall Rating: In this case the form should be corrected or relaunched. 
    • The form should not create an Overall Rating (i.e. should not have a Summary section): in this case, the form should be removed from the process used for the Matrix Grid Report and you should also consider removing the Summary section of the Performance Form, so it does not generate Overall Ratings going forward. 
  • The latest rating is the expected rating, but the user is appearing in an incorrect placement in the Matrix Grid Report: This could happen if you have different Rating Scales for the rating (usually 1-5) and for the Matrix Grid Reports (usually 1-3). In this case, the Matrix Grid Report will apply a process, called normalization, to convert the original rating value to the corresponding value in the Matrix Grid Rating Scale. In addition, the Normalization process may be using Custom Weights configured in the Matrix Grid Report. For more information on this, please refer to Wiki Page Matrix Grid Reports: Normalized Ratings and Custom Weights

Related Content

Related Documents

Matrix Grid Report - Understanding the source of ratings