Over the Counter Data in Edtech

Over-the-Counter Data Is the Next Frontier for Data in Edtech

Over the Counter Data in Edtech

Educators have widely accepted the importance of using data to inform their treatment of students’ needs. This is a good thing, as research touts the benefits of effective data use. Thus many educators worldwide have embraced data use with gusto. Unfortunately, educators’ widespread data use is not quite a good thing.

A significant portion – and some research claims most – of educators analyzing and using data are doing so incorrectly. For example, a study conducted by the U.S. Department of Education (2009) at districts with reputations for strong data use found only 48% of teachers correctly interpreted given data. Much research supports this trend: educators are using data to inform decisions, but they do not always understand the data they are using. Since their data-misinformed decisions impact students, this is a significant problem. Fortunately, edtech can do much to help if it improves in key areas.

Consider an Analogy

Which offers you better medical care: 1) treatment or medicine administered via a personal visit with a doctor, or 2) over-the-counter medicine administered by yourself? The answer is easy: a personal visit with a doctor, as he or she can ensure the treatment or medicine is used appropriately. However, this does not supplant the need for over-the-counter medicine, as it is not always possible to visit a doctor for every cough or sniffle. While a doctor is not present to explain the over-the-counter medication’s purpose, ingredients, dosage instructions, and dangers, any remedy you buy in a store comes with a detailed label and other supports outlining these matters. Just as it would be wrong to board up pharmaceutical aisles in stores and withhold basic medicine from people who are not receiving it directly from doctors’ hands, it would be equally ludicrous to rip the labels off over-the-counter medications, leaving people with no way to use them wisely.

So, how does this relate to schools and edtech? Educators use data to treat students, and they use edtech data systems (or reports generated from those systems) to analyze this data. However, these educators are usually operating without the data-equivalent to over-the-counter medicine. They are essentially opening data bottles with no labels, and swallowing the contents nonetheless. This is because supports that commonly accompany over-the-counter medicine to ensure it is used properly (thorough labeling, supplemental documentation, online access to a help system, effective package/design, and appropriate contents) are not utilized in most of the edtech data systems or reports educators use to analyze data. Thus this highly utilized sector of edtech typically denies educators the analysis supports that could significantly improve educators’ use of the data.

How Has Edtech’s Data Role Progressed?

Before the rise of technology and user-friendly data systems, school staff did not have site-wide access to data like they typically do today. Staff and students could not benefit from extensive, timely data such as classroom assessment data that could be used to closely monitor student progress and customize instruction accordingly. Perhaps staff received blocks of data by the government, a consultant, or a district-based trainer once or twice per year, but this could not help their students on an ongoing formative basis, and it only offered educators isolated points in time during which students’ needs could be assessed and remedies determined. With this arrangement, educators were like patients with limited access to doctors and no access to over-the-counter medication.

Now most schools are using a data system with an intelligible interface that puts data at the fingertips of most staff members. However, the data reports contained therein are generally not accompanied by guidance on their purpose, how to properly read them, how to properly interpret the data, and analysis misconceptions to avoid. Essentially, educators are like patients using medication in unmarked – or marginally marked –containers. A data system might label a report (“Growth Report,” “Year-to-Year Comparison,” etc.), but relying on mere titles is akin to taking pills in a bottle marked only as “Cold” or “Flu.” Anyone using such data to inform decisions is assuming as much risk as someone opening a pill bottle that reads “Flu” and chancing ingestion of the wrong number of pills, at the wrong time, for the wrong type of flu, and in direct opposition to dangers. Imagine the ramifications for students and staff affected by the decisions made in such a way.

With educators making so many data analysis errors even at districts known for their data prowess, current data analysis supports are clearly not enough. Professional development (PD) is needed and effective, as are good leadership and staff to assist with data use, but we further these supports’ impact when we prescribe an Over-the-Counter Data (OTCD) format for the data we are using.

What Is Over-the-Counter Data?

There is no price tag tied to OTCD. Rather, it is a design approach any edtech data system or report is free to utilize to help educators make more accurate analyses of data. While most data systems are simply “showing the data,” OTCD addresses key ways in which they can better help educators use that data correctly. Inspired by the varied ways in which over-the-counter medication supports the successful use of its contents, OTCD involves embedding data analysis supports directly within reporting environments and adhering to best practices within 5 component categories:

Label – The label functions as an appropriate report title (clear, concise, consistent with logic of report suite titling) and appears as a footer and/or annotation on each report to provide the most important information needed for analysis.

Supplemental Documentation – Since you should not cram all information needed for analysis onto the report itself, explanatory information accompanies each report via links to an abstract and/or interpretation guide providing more analysis guidance specific to the report and its data.

Help System – An online help system built within the product and accessible via an easy-to-spot link offers lessons on using the system (tech lessons), which is common, but also on data analysis (specific to the data and assessments used in the users’ region), which is uncommon.

Package/Display – Report format (how the data is displayed) assists proper analysis by maintaining credibility, offering key features, and adhering to principles of good design that assist analysis ease and accuracy. Reports are accessible (e.g., it’s easy to find a particular report) in a manner that offers easy navigation.

Contents – Reports operate with input controls that specifically facilitate recommended data investigation practices, feature expiration- and audience-sensitive contents, and comprise a suite of reports covering all key needs without overwhelming users.

Specific attributes contained within these OTCD components are detailed at www.overthecounterdata.com/otcd. These recommendations summarized by OTCD are based on research in education and edtech, as well as research in a variety of other fields (e.g., behavioral economics, design, business analytics, technology, etc.). OTCD assumes an edtech data system or data component already meets basic requirements widely accepted and discussed for at least a decade (students tied to unique identifiers, online access for all staff, longitudinal reporting capabilities, etc.), which so many data systems already include, and summarizes what their next milestones should be.

What Can We Do?

OTCD holds great potential to help educators and – most importantly – students, but only if it is implemented in areas of edtech and reporting that communicate data to education stakeholders. Everyone involved in education and edtech can play a role in this endeavor:

Educators – You are the edtech customer and thus edtechs respond to your requests and want to meet your needs. Communicate your need for OTCD to your edtech vendor and work to see your OTCD requests implemented in any area where data is being communicated. Knowing most educators are misusing data in its absence, OTCD is an urgent request.

Edtech Staff – If your system includes a component that communicates data, adjust your system to adhere to OTCD. This has a significant impact on students and thus is a matter of moral responsibility. Fortunately, OTCD also creates more cost-effective efficiency in your product and improves your product’s success and marketability.

Edtech Investors – When determining which edtech startups or projects in which to invest, check edtech candidates for OTCD. Many edtech products contain a feedback component where they are communicating data, and poor communication is not cutting edge. Rather, a firm understanding and implementation of OTCD allows edtechs to perform better and thus compete better in the market, all while being more efficient with design and resources. Products adhering to OTCD are more worthy of your investment than those failing to implement research-based practice.

OTCD is the next frontier for data in edtech. It is time for conversations about edtechs that display data to hold higher standards and reflect OTCD best practices ascertained by the research community. When Over-the-Counter Data is realized in products educators use to view, understand, and use data, it will be a great victory for students and all roles seeking to help them.

Jenny Grant Rankin is a former award-winning teacher, school site administrator, district administrator, and Chief Education & Research Officer at Illuminate Education (a data systems and edtech company). Her close-to-completion Ph.D. in Education features a specialization in School Improvement, and the topic of her dissertation & quantitative study is improved edtech data system/report design through Over-the-Counter Data (OTCD) to improve educators’ analyses and use of data being presented. Jenny's site is overthecounterdata.com, where she promotes her passion for OTCD’s potential to help students. You can follow her on Twitter, Facebook, LinkedIn and Google+.

  • bnleez

    I’d like to offer a slightly alternative perspective. Perhaps the problem is that we (educational stakeholders) fail to find practical ways to close the “knowing-doing gap” (Pfeffer & Sutton, 2000) – http://amzn.to/102sbAs

    Let’s find connective ways to close this gap by applying some “preventative medicine” as opposed to waiting until we are sick (or dead). Let’s stop blaming teachers and realize that it takes a village. Finally, let’s work on solutions using local data by cooperative teams already internal to the education system.

    Just imagine what schools could achieve if they could implement what they already know can improve student achievement. Implementation (change) cannot come solely from educators, so I take exception to the ease in which educators are to blame. Complex solutions require a cooperative attitude from all educational stakeholders: teachers, students, parents, administrators, civil leaders, etc.

    • Jenny Rankin

      Hi there. This piece is 100% in no way meant to blame teachers. Rather, it is calling for edtech to take more responsibility in doing a better job in helping teachers. Like you say, it is a cooperative effort, and while educators are doing what they can to improve data analyses (PD, staffing, leadership, PLCs, etc.), there is a disconnect between what research says edtech can do to help data analyses vs. what most edtech offers. This is a call for edtech to adhere to research showing ways it can better help educators.

      • bnleez

        Hello Jenny. When you say, “…edtech [should] take more responsibility…” is sounds like you mean that the technology needs to be better (please correct me if I’m wrong). What I’m saying is that the way in which individuals (educational stakeholders) cooperate intentionally needs to improve. Sure, there is bad data, and there are things we educators need to learn, but I think we can get more “bang for our buck” if we (again, all educational stakeholders) first find ways to work together connectively and intentionally with what we already know. This then becomes the precursor to understanding how to use good data to spark systemic change.

        I think we need to use caution when integrating educational data throughout a school so that it does not appear to be as a directive. I say, we create a healthy environment of cultural change first, then integrate qualitative, quantitative internal, and external assessments (formative and summative) in ways that best seek to improve student achievement.

        • Jenny Rankin

          Hi there. Yes, that’s totally correct. The technology needs to be better, but all the factors you mention on the education side and more (good leadership, vision, support, collaboration, quality assessments, PD, etc.) are needed, too. They’re different sides of the same coin, and every bit helps. If one is instituted without the other (e.g., if there’s a great culture of support on the ed. side but they’re using a product that misrepresents data, to where educators are led into drawing false conclusions without knowing it) the benefits of data crumble.