Modernizing the Way States Collect and Store Data
© Kittipong Jirasukhanont |


Modernizing the Way States Collect and Store Data

Most states need to restructure their technology and data policies.

In 2018 most of us have heard, at least in some way, about “big data” and its purported benefits. But there are a number of misplaced concerns and doubts about technology policy among state legislators holding back statewide enterprise software technology upgrades that would vastly improve the organization, efficiency, and performance of state government.

The first obstacle is a misunderstanding between open data and shared data, the background technologies needed to develop both, and the potential benefits of both.

In open data, government releases its data in raw format to the public. This is often done via dashboards and open data portals. However, every database represented is created, structured and stored differently.

Shared data means that data from multiple applications and databases are all stored in one placed and structured so that comparisons and analysis can be done instantly. Currently, states tend to only to link specific databases and applications on a custom, resource intensive, one-by-one basis.

The goal for state governments should be to have a statewide enterprise software development strategy that structures data creation so that it is automatically processed and stored, often referred to as “machine readable” format data. While shared data initiatives in Indiana, Missouri, and, Florida highlight the benefits of data sharing, they are custom designed, resource intensive projects that do not represent a comprehensive statewide technology solution for sharing data. The benefits of big data are not fully captured by just open data or one off data sharing solutions.

In order for the major benefits to begin accruing, state agencies should be required by state policy to operate within a defined software as a service (SaaS) development environment, meaning new software creation is done under strict development protocols which apply to all state agencies and partnerships with the state.

For example, when a given agency goes to create new software, they would no longer be allowed to manage the project independently — hiring developers, creating custom applications, and structuring new databases. They would be required to develop applications in a predetermined environment so that there is a conscious, planned, intelligent strategy about what data is collected, how that data is stored, and ultimately that data is shared across state agencies to facilitate better use of that data.

Understandably, a centrally stored and operated database triggers several concerns regarding privacy, security, and cost, but these concerns are mostly misguided.

First, there is a concern that creating a statewide database architecture will enable government to collect more data than it already does on citizens, and in doing so violate their privacy. The creation of the technological structure does not necessitate that it violates privacy. As current state applications already are required to be, any application developed by government on the new system would be legally and constitutionally accountable to privacy standards. The change in technology policy does not require a change in privacy policy.

The second concern is that a centralized database will create the “target on the back” effect, meaning that thieves and hackers will be drawn to a centralized database that contains valuable information, thus putting citizens at greater risk than they already are. Again, it is useful to analyze current state practices and compare them to what is being proposed.

In many states, it is common to use Social Security numbers, passports, birth certificates, and other highly sensitive personal information to register for government services in person or online, which is then typically stored in the aforementioned disparate data systems. Unfortunately, There are ample examples of state government databases being hacked that contain enough sensitive information for hackers to easily realize their ambitions. No one knows how much this has already cost the US taxpayer, but it certainly has cost them something.

From a technology perspective, protecting multiple databases all running on different protocols is immensely more difficult than securing a centrally-managed database. If both database methods contain enough information for hackers to commit crime, where is the logic in choosing the less secure method? State governments are trying to meet their agencies where they are and build a security outpost around each one them, when they could consider consolidating all those resources and building a centrally strong fortress for everyone to live in.

The third concern is cost—that this transition to a statewide enterprise would cost more than it is worth. This thinking deeply misunderstands the future character and creative power of non-cognitive human labor. Deloitte consulting estimates that up to 10 percent of total state government worker hours go towards documenting and recording information, while just 2 percent goes towards analyzing data and information, and another 2 percent goes towards thinking creatively.

Tax dollars are not better spent documenting and recording data than analyzing it and thinking creatively.

For state governments, adopting enterprise SaaS environments and shared data are the path towards reversing these ratios. For example, it would no longer require a state child adoption case manager to collect and analyze applicant data to determine if they are a fit. A simple data check that is done automatically upon application could occur, or perhaps ineligible adopters would be flagged as they began applying. State adoption services would be improved as state employees became freed up to think creatively about how to use data to cut down on the amount of bad outcomes that happen when placing children in adoptive homes.

This is one small example of what could be happening at magnitude across state government if statewide software was adopted.

Lastly, the shift towards SaaS development does not affect the mission or function of state agencies. It tells them how to build applications, not which applications to build. State agencies would also all have a stakeholder seat in a data governance board to ensure a balance of power. In fact, those concerned with the perpetual sprawling of government should consider that this could reduce duplicative and unnecessary expenditures and activities.

In conclusion, while nearly everyone has awoken to the arrival of big data, state governments have major technology policy changes they need to make in order to capture the full potential range of benefits it brings. This constitutes a complete restructure of state technology and data policy. A centralized data policy has not shown to create greater risk than siloed data policy and concerns about security can be addressed. Concerns over costs fail to view technology as a long-term investment that generates continued opportunities to cut costs and improve services, something the private sector has known for 20 years. States should overcome these concerns and move their technology policies forward.