An in-basket exercise assesses a candidate's ability to perform a manager’s job from an administrative perspective. In the exercise, the candidate is confronted with issues and problems that have accumulated in the manager’s "in-basket" after returning to work from an extended absence. A sample of in-basket items might include memos, correspondence, e-mails, directives, requests, reports, forms, messages, minutes, hand-written notes, etc., from management, supervisors, staff members, inmates, and other stakeholders. The candidate's task is to review the in-basket items and then take action on these varied issues and problems using action forms to record notes, comments, and responses. These actions are then assessed and rated based on job related competencies through a formal question and answer session by a group of trained raters. Standardized criteria and predefined rating scales are used to assess the candidate.
What does an in-basket exercise measure?
The in-basket exercise measures administrative skills that are critical for effective performance in supervisory and managerial jobs.
The in-basket is a standardized, behaviorally based exercise. Multiple trained raters assess and score candidates according to defined competencies of performance specific to the in-basket exercise. After each candidate completes the in-basket exercise, a group of trained raters conduct a question and answer session with the candidate to assess performance on the exercise using a set of predefined rating scales based on job related competencies (e.g., planning and evaluating, problem solving and decision making, etc.). The end product of the in-basket exercise is that each candidate receives a combined score from multiple raters based on standardized criteria.
The in-basket exercise is designed to test the candidate’s adaptive thinking, problem analysis, judgment, administrative abilities, planning, organizing, delegating, and integrative skills while under pressure dealing with memos, e-mails, requests, messages, handwritten notes, etc.
What competencies are measured in the in-basket exercise?
Like all tests, the in-basket exercise measure certain attributes or qualities. These attributes or qualities are commonly referred to as competencies.
To provide an example, all competencies have both a title (e.g., oral communication) and a definition (e.g., ability to communicate orally in clear, understandable, and polite tones at the level of the listener giving clear instructions and detailed information). The definition will list behaviors appropriate for that competency based on a job analysis. Some examples of common oral communication behaviors expected are listed as follows: clear and easy to understand, speaking at the level of the listener; fluent and articulate, without frequent pauses; spoke at an appropriate rate (not too fast or slow); information was in a logical order; volume of voice was appropriate, and used appropriate inflection to emphasize main points, etc.
In any given in-basket exercise, various definitions may be combined or modified. While the exact competencies will vary, the following competencies will be tested in the in-basket exercise:
- Planning and Evaluating:
Organizes work, sets priorities, determines resource requirements; determines objectives and strategies to achieve them; monitors and evaluates progress against goals.
- Problem Solving and Decision Making (judgment):
Identifies problems; gathers, interprets, and evaluates information to determine its accuracy and relevance; generates and evaluates alternatives; makes sound and well-informed decisions; and commits to action to accomplish organizational goals.
- Managing and Organizing Information:
Identifies need for information; analyzes and interprets data in complex situations involving conflicting demands, needs, and priorities; determines its importance, accuracy, and communicates it by a variety of methods.
- Self-Management (initiative):
Show initiatives; sets well-defined and realistic personal goals; monitors progress and is motivated to achieve; manages own time and deals with stress effectively.
How is an in-basket exercise developed?
Development of an in-basket exercise is a very expensive and time consuming process. The following is a typical way an in-basket exercise is developed.
Identify critical job tasks and competencies
- Job analysis
- Training Bureau’s core competencies
- Job code specifications (state job descriptions identifying knowledge, skills, and abilities)
- ADC position description questionnaires (PDQs)
- Subject matter expert (SME) development sessions, e.g., input, discussions, data collection, item development, review, feedback, etc.
Collect source materials
- Variety of documents from all levels and locations
- Related to competencies: problem solving, planning, evaluating, delegating, etc.
- Kinds of information to collect: memos, reports, messages, e-mails, calendars, forms, organizational charts, policies, news articles, plans, staff schedules, training schedules, requests, flyers, etc.
Develop/write the scenario
- Information about position, issues, timeframes, etc.
- Set-up organizational charts for use in the in-basket
- Calendars for scheduling, planning, etc.
Develop document pool
- Reports, e-mails, memos, incident reports, phone messages, etc.
- Information from multiple sources (all levels and locations)
- Representative sample of issues dealt with across locations
- Ideas about what is the same, what is different across the job class (by level, by geography, etc.)
- Actual documents related to real events that demonstrate the competencies
- Participation from the field subject matter experts
Conduct first SME session
- Introduction to the in-basket
- Competency review
- Individual review of scenario and documents
- Individual evaluation of scenario and documents
- Group review and evaluation of documents
- Solicitation of competency-based responses to documents
Select documents and prepare item development plan
- Revise documents based on SME consensus and group discussion
- Revisions consistent with the evaluation criteria (more job-related, clearer, more accurate, formatted correctly, etc.)
- Responses now can be focused on competency-based issues
- "Clean" documents--formatted appropriately, technically correct, and job-related
- Competency-based reactions to each document provides justification for writing items, response options, and justification for each
Write items and conduct internal technical review
- Prepare item development plan (e.g., item writing, items linked to competencies, etc.)
- Write items (questions)
- Conduct internal technical review (e.g., some items deal with multiple documents, equal number of items across competencies, etc.)
Conduct second SME session
- Review competencies and definitions
- Review in-basket scenario and documents
- Individual and group evaluation of scenario and documents
- Review of questions, response options, and justifications
- Evaluate items (questions)
- Determine or verify key(s) for items
- Suggest revisions to items/documents as needed
- Review assessment materials (e.g., applicant instructions, etc.)
Pre-test the in-basket exercise
- Plan and schedule raters and test subjects
- Conduct an actual in-basket simulation
- Record results and feedback
Refine and conduct final review
- Adjust and refine in-basket exercise
- Conduct final review
- Proofread in-basket materials
- Print in-basket exercise
Prepare test documentation file