OIS Project Documents
BLS FY 2013 Test Summaries
- Why are you developing a new occupational information system (OIS)? Why can’t the Department of Labor (DOL) update the Dictionary of Occupational Tiles (DOT), or why can’t you use the Occupational Information Network (O*NET)?
The Department of Labor (DOL) developed the DOT in the late 1930s to match jobseekers to jobs. For almost 50 years, the DOT has been our primary source for occupational information. The DOL discontinued updating the DOT in 1991, and replaced it in 1998 with another job placement tool, the Occupational Information Network (O*NET). We studied whether O*NET could take the DOT’s place in our disability adjudication process but found it does not describe the physical requirements of occupations at the level of detail needed for claims adjudication.
- What kind of expert input have you received in developing the OIS? Will the public have access to methodological and survey design information?
When we started developing an OIS in 2008, the agency formed a federal advisory panel, the Occupational Information Development Advisory Panel (OIDAP). OIDAP members included industrial and organizational psychologists, occupational therapists, claimant representatives, vocational experts, physicians, and others. The OIDAP held quarterly public meetings throughout the country from 2008 until 2012, providing a forum for OIS stakeholders to share input about OIS development. SSA disbanded the OIDAP in 2012. In Fiscal Year (FY) 2012, we entered into an interagency agreement with the Bureau of Labor Statistics (BLS) to test the feasibility of using the National Compensation Survey platform to collect updated occupational data similar to what we use from the DOT. Since Social Security’s disability programs are rooted in the concepts found in the DOT, the data elements that describe the physical demands, environmental factors, and skill level of occupations in the new OIS will be similar to the information currently in use. In FY 2013, BLS conducted three phases of data collection testing. Our disability policy experts joined BLS field economists on several data collection appointments. Following each test phase, our disability policy experts met with BLS staff to discuss successes, problems, and ways to improve the survey questions. Testing continued throughout FY 2014. Summaries are available on this website and on the BLS Occupational Requirements Survey website (http://www.bls.gov/ors/).
To develop data elements that describe the mental and cognitive demands of work, OIS project staff compiled a comprehensive list of all of the possible elements that may be useful in disability adjudication as suggested by the OIDAP and external experts who gave public comments to the OIDAP and from requirements rooted in agency regulations and policy. OIS project staff also reviewed the results of an internal study that gathered information from 5,000 disability claims decided at the initial and hearings levels at steps 4 and 5 of sequential evaluation. Results from this study allowed us to identify the most commonly cited mental and cognitive limitations found in our disability claims.
Disability program experts then followed a methodical process of reducing the list to a workable number of data elements that are most important to disability adjudication. Once OIS project staff had narrowed the list to a collectable number of data elements, they worked with BLS staff to develop the initial survey questions. The data elements in the final list resembled the elements that BLS collects through the NCS occupation leveling process. BLS tested the collection of these data elements in FY 2014. OIS project staff then worked with BLS to analyze the test results. BLS experts, who have experience collecting similar mental and cognitive occupational demands through the NCS leveling procedures, provided fundamental input that helped refine the mental-cognitive questions. OIS project staff members continued to work with BLS during FY 2017 to improve the mental and cognitive survey questions.
Why did you select BLS to perform the survey? Are BLS economists qualified to gather and tabulate data and draw conclusions about the data they obtain?
We are confident that BLS is the ideal partner for collecting the occupational data our disability programs need. We decided to collaborate with BLS primarily because of BLS’s reputation for collecting quality job data that produces the Employment Cost Index and helps set Federal pay scales. In addition, through the platform of the National Compensation Survey (NCS), BLS field economists (FE) were already collecting occupational data similar to some of what SSA needs such as physical demands, job complexity, and personal contacts. To collect this data, BLS has in place a nationwide network of FEs who are experienced and effective in encouraging employers to participate in a government survey. BLS economists and statisticians are experienced and capable of analyzing, advising, and drawing conclusions about complex data sets.
- Will you need to change policy to implement OIS?
Yes. Initially OIS development started with the goal of supporting current policy with minimal change. However, as SSA learned more about the data, we determined that larger changes to policy would be necessary to support the data. This coincides with the Office of Disability Policy’s plans to update the vocational regulations used to make steps 4 and 5 decisions. The two projects are working together to ensure that OIS data will support updated vocational policy.
- SSA has received the complete ORS Wave 1 results– why aren’t you using the survey results to adjudicate claims?
We are currently revising our vocational policies to be consistent with the new OIS data and the SOC taxonomy. After we revise our policy, we will need to implement systems changes to accommodate those policy changes. We will begin using OIS data once we have finalized our policy changes, and we have made conforming changes to our computer systems.
Why is development taking so long? Can’t BLS collect data faster?
BLS and SSA have established a large-scale national survey on a remarkably short schedule. We signed our first IAA with BLS in July 2012. BLS started testing in FY 2013. During that year they conducted three stages of testing which started with a small proof-of-concept collection in the Washington, DC, Metro Area and expanded to a broader-scale collection across 6 cities. In FY 2014, BLS conducted further testing to refine the survey questions and methodology based on the outcome of the previous year’s tests, and BLS began testing new questions to collect the mental/cognitive requirements of work. In FY 2015, BLS conducted a nation-wide pre-production test to prepare for production collection.
- Will you regularly update the OIS to ensure that the data remain current?
Yes, based on BLS research on the shelf life of occupational data, we are planning a 5-year data refresh cycle. For more information about the shelf life of occupational data, please see https://www.bls.gov/ors/research/sample-design/pdf/dynamics-occupational-change-2016.pdf
My employer makes special arrangements or accommodations for employees with physical and mental limitations. Is BLS collecting information about the way people with physical and mental limitations perform my job in addition to the way employees without limitations perform my job?
No. The new OIS must meet SSA’s program needs, and under current policy, we do not consider accommodations when determining whether a person can perform work as it is generally performed in the national economy.
How will occupational descriptions in the new OIS differ from those in the DOT (and the Selected Characteristics of Occupations)? Will OIS add, remove, or alter occupational characteristics?
We tailored the new OIS for our disability programs. Since we based disability policy on the DOT’s concepts and definitions, the new OIS will mirror many of the data elements from the DOT. In some cases, the new OIS will provide more detail about occupational requirements. For instance, the new OIS will assign exertional levels to occupations similar to the DOT’s, but will also describe the amount of standing/walking, and information about whether jobs require driving, using a keyboard, and reaching overhead. In 2024, OIS will also include new data elements that our adjudicators and stakeholders requested, such as descriptions of the mental and cognitive requirements of occupations. A few DOT elements will be excluded, such as color vision and balancing. OIS project and BLS staff made slight revisions to some of the data elements and measures based on the input of disability program experts, stakeholders, and the results of testing.
Is there a possibility that from time to time a claimant may have a limitation that the agency cannot assess using the new OIS occupational descriptions?
Yes. There will be situations when impairments cause uncommon limitations that are not included in the new OIS’s job descriptions. Adjudicators experience this situation now when determining whether claimants with severe mental impairments can work. The OIS will not eliminate the need for adjudicators or vocational experts to use judgment in these situations.