The Open Data City of the Future

"The first step in becoming an Innovative "Smart City" begins with the collection, acquisition and analyses of all municipal data into a centralized data platform by a communal Innovate Team. The huge potential of an Innovative Team is that a municipality can use the analyzed data to address common problems or challenges faced by the constituency while inspiring entrepreneurship as well as new innovative solutions for civic improvements." Yakov Marks Chief Data Officer *MTDAAT (Ma'a lot-Tarshiha Data Acquisition and Analysis Team

"The data that an Innovative Team can acquire through open and free access from across all the municipal sources, needs to be analyzed and shared openly with free access across all city departments. Thereby unlocking the barriers that once blocked the municipality's potential to solve problems, instill change, foster enterprise and inspire the constituency.

Byanalyzing existing and incoming data an Innovative Team can provide new solutions, create ways to implement them and measure the results." Yakov Marks Chief Data Officer *MTDAAT (Ma'a lot-Tarshiha Data Acquisition and Analysis Team

• In most cities the varied sources of municipal data are as though they come from "Tower of Babel" of feudal lands. 
• Information in most municipalities is not shared causing "Data Stagnation".
• The issue is how to formalize, accumulate, analyze and interpret the different forms of data-sets from city wide departments and sources into one finite set.

"Cities are composed of various departments with software-intensive systems that support the operation of modern municipal departments. Information systems help manage the various day by day processes in the departments. However, many departments are not linked to a central data bank or city-wide hub. The information from one department that may be essential for the answer to a problem in another department may not be realized due to the non-communication and sharing that could be realized through "open-data"."

The true value of data is measured by the positive action taken as a result of understanding the data. A Joint Municipal Data platform will help governments identify specific, measurable goals founded on actual data to help them become more operationally efficient, effectively meet constituent needs, and create economic growth in their communities.

New administrations are often elected based on specific improvement platforms. Sharing Municipal Data openly helps ensure that those promises are being addressed appropriately and proactively. Since all programs and projects have milestones or goals that can be measured and tracked an Open Data Dashboard could be instigated as an important part to display progress of staying on track, on budget, and promoting the success of any project. Additionally, by providing transparency into how goals are progressing, it demonstrates to the constituency that progress is being made and money is being invested wisely.

 What is "Big Data"

Big data is often characterized by 3Vs

• the extreme volume of data,
• the wide variety of data types and
•  the velocityat which the data must be processed .

Although big data doesn't equate to any specific volume of data, the term is often used to describe terabytes, petabytes and even exabytes of data captured over time.

“Voluminous data” can come from myriad different sources, such as business sales records, the collected results of scientific experiments or real-time sensors used in the internet of things.

Data may be raw or preprocessed using separate software tools before analytics are applied.

In General:

• Open data has become a hallmark of good government because of its well documented return on investment for the public.
• Open Data is a valuable information resource helping local small businesses compete with large companies.
• Unlike the previous Industrial Revolutions, which were topped by how much information we could transmit and receive, we were limited by how much information we can process and act upon.
• This digitalization of everything implies that we are increasingly reliant on analytics to enhance productivity and recognize the fundamental fact that at the heart of all smart-infrastructure, is data.
• What used to take months or weeks now occurs in real time.
• Today’s level of decision-making requires information that is current in the hands of every user when they need it.

"The digital telegraph of the 21st century is analytics built directly into IoT processes."

We are at the cusp of a brand-new fourth Industrial Revolution; which is building upon the third. It is characterized by a fusion of technologies that blurs the lines between physical, digital, and biological worlds.

Key areas that can be significantly impacted by the opportunities big data offers; include the health, safety, movement or traffic flow, and revenue systems that sustain inhabitants.

Cities striving to become more responsive must have a "stack" of essential building blocks, said Crawford.

• First municipalities need fiber Internet connections; so that large data files can be shared securely, quickly and seamlessly among city staffers,along with
• data sensors and
• CCTV for live "real -time" monitoring positioned at key locations like city bridges, tunnels, and major roadways to help identify problems and repairs

Then, they should publish collected public data and share it across city government and with the public, and have data scientists study the numbers to discern trouble spots that will drive government action and perhaps policy.

Making the collected data open is a critical component of responsiveness. "It's important because open data—well visualized—allows employees to see other agencies, it allows residents to hold their city hall responsible, but also because it provides data that can lead to breakthroughs and solutions" from inside and outside government

Because cities may have thousands of datasets across multiple servers, databases, and computers, it’s helpful to narrow down which datasets should be included in the inventory overall and how to plan for inventory updates in the future.

This collection of data from varied sources and departments is made possible by a thoroughly trained Municipal staff.  The Municipal Analytic Team then utilize the acquired data and analytic systems and processes to recommend and facilitate projects for the good of the constituents (residents).

The changing nature of the technologies themselves and our urban environments are turning into landscapes populated by more and more connected “Things”. The "Thing" are equipped with many different sensors for data capture and analytics. This is driving a growing need for inter-operable platforms and standards that give more players wider access to city data.

This digitalization of everything implies that we are increasingly reliant on analyticsto enhance productivity and recognize the fundamental fact that at the heart of all smart-infrastructure, is data.

The first step to treating your city’s data as an asset is to create a comprehensive data inventory with consistent metadata and to establish a clear authority body to oversee the data inventory process is key to success.

Knowing what data your city collects leads to efficiency, and increases accountability. It also eases citywide reporting, decision making, and performance optimization.

Managing a data inventory reduces risk and uncertainty by creating a checklist for security and compliance requirements and improves a city’s ability to designate accountability for the quality of the data collected and created.

A municipality gains the ability to deliver results by its acceptance and creation of a culture of using its data assets., enhanced abilities to properly and efficiently access the information collected from archived municipal files, in various city department and from the public sector in a centralized data platform.

Because cities may have thousands of datasets across multiple servers, databases, and computers, it’s helpful to narrow down which datasets should be included in the inventory overall and how to plan for inventory updates in the future.

This collection of data from varied sources and departments is made possible by a thoroughly trained Municipal staff.  The Municipal Analytic Team then utilize the acquired data and analytic systems and processes to recommend and facilitate projects for the good of the constituents (residents).

Today's digitalization of everything implies that we are increasingly reliant on analytics to enhance productivity and recognize the fundamental fact that at the heart of all smart-infrastructure, is data.

Because cities may have thousands of datasets across multiple servers, databases, and computers, it’s helpful to narrow down which datasets should be included in the inventory overall and how to plan for inventory updates in the future.

The datasets worth inventorying (collecting and inputing) are those which are considered assets to employees, departments, executive leadership, and the general public.

Data assets can range from individual datasets that are connected to forms that people fill out, to integrated databases that track a city’s operations in any given field (building permits, public safety responses, etc.)

Just as it is important for cities to know what data they have, it’s equally important to know what data a city does not have. With a complete picture, cities can begin to collect and use city data to better align mission goals, increase consistency and confidence in decision making, and build performance intelligence.

Managing a data inventory is crucial to better information sharing and integration and a sustainable comprehensive open data program. Providing a public data inventory will make city employees’ jobs easier when they need information from another department - they will know what exists and how to find it. The same benefits apply to the public regarding its search for city information. Having a complete inventory is also important when determining which datasets to release publicly.

It’s not feasible to release all of a city’s public datasets at once, so decisionmakers need a prioritization strategy. The data inventory can be used to prioritize the release of data according to strategic priorities, public interest, etc.

Cities striving to become more responsive must have a "stack" of essential building blocks, said Crawford.

·         First, they need fiber connections so that large data files can be shared quickly and seamlessly among city staffers,

·         along with data sensors positioned at key locations like city bridges, tunnels, and major roadways to help identify problems and repairs.

Then, they should publish collected public data and share it across city government and with the public, and have data scientists study the numbers to discern trouble spots that will drive government action and perhaps policy.

“Understanding the pulse of life through the use of sensors to improve quality of life in cities.”

Making the collected data open is a critical component of responsiveness. "It's important because open data—well visualized—allows employees to see other agencies, it allows residents to hold their city hall responsible, but also because it provides data that can lead to breakthroughs and solutions" from inside and outside government.

 Advantages of a Data Platform:

A Municipal Data platform enables City Managers and Mayors to extract maximum value from their available budgets.

A Data platform provides real-time access for operational staff to the repository data, thereby providing department heads the ability to intervene or modify plans on the fly if circumstances require.

Through the use of a Municipal platform the City Constituent Care Service and Response center has real-time access to constituent residential and service data to allow incoming queries to be handled on-the-spot, thereby minimizing call-out costs and improving customer service levels.

Furthermore, the Municipal Data Platform can rapidly assimilate, assess and act on data thereby showing the constituents that the Municipality is listening and is responsive.

A Municipal Data platform is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.

Predictive Analytics SaaS (Software as a service) solutions are based on proprietary Machine Learning Big-Data algorithms, guaranteeing real time accurate and reliable predictions, in a fully automated, plug-and-play manner. The data and predictions are presented with advanced visual tools, enabling end users to self-explore, gain insights and comprehend the data, without requiring any statistical background.

The Big Data engine provides real-time accurate load forecasts at the highest level of granularity - the meter / appliance and sub-hour levels. Due to the engine's self-learning capabilities, it models and monitors each meter separately, learns its patterns and behavior, automatically fits its appropriate model and senses its early warnings for irregularities, guaranteeing real-time accurate energy forecasts and actionable insights.

The data and predictions are presented with advanced visual tools, enabling the end user to self-explore, gain insights and comprehend the data, run simulations and impact analysis, view correlations, create 'what if' scenarios and more, without requiring any technical or statistical background.

While most Predictive Analytics solutions require extensive services of data mining experts for the designing, implementation and support, Grid4C provides a fully automated solution that is easy to implement, and does not require on-going support of analysts. The Grid4C distinctive self-learning engine enables to implement the products in a plug-and-play manner.

By freely collecting and combining data from municipal departments Open Data can provide valuable insights into how any city works and how departments may better serve the constituency as well as those that live and work in the city.

Over the past decade, the scope and content of data related to government activities has changed dramatically. The sheer quantity of data available for public consumption, the way in which it is structured and how datasets are used has the potential to impact program planning, analysis and evaluation at the local government level

"That is why our research focus is the intersection of government and data, ranging from open data and predictive analytics to civic engagement technology. We seek to promote the combination of integrated, cross-agency data with community data to better discover and preemptively address civic problems."

The steps to becoming a "Smart City" begins with "Big Data".

First steps:	·         Data Collection and Acquisition ·         Establish the sharing of data and laying the groundwork for inter-departmental cooperation ·         Creating a trained internal work force.
Second Step:	Data Analyses and processing by a coordinated data analytic team
Third Step:	Recommendations for solutions based on established data by a Chief Analytic Officer.
Final Step:	A Smart City

How to Create Your Data Catalog (Inventory)

The first major task for the Data Coordinators is to create a data catalog (or inventory) of your department’s data.

Follow the 3 major steps below to conduct your data inventory:

1.      Identify data sources

2.      Brainstorm and identify datasets in each data source

3.      Complete dataset inventory template (for each dataset)

Step 1: Identify data sources

Your data may be housed in a variety of places - from inside information systems or databases to shared drives and folders. This can also include 3rd party vendors and data hosted on vendor systems. Step 1 is about identifying the major data sources in your department.

·         Questions to help identify and discover data sources:

·         What information systems does your department use?

·         What databases does your department use?

·         What applications capture information or are used in your business processes?

·         Are some data resources kept in spreadsheets (on shared or individual drives)?

·         What information are we already publishing and where did that information come from?

For each of the data sources:

·         Provide a name and brief description of the data sources

·         Capture any technical details and point of contacts.

Regarding Smartphone Applications and Sympathetic Constituent Call Service Centers:

Tom Saunders, a researcher at England-based research and innovation foundation Nesta stated that residents can also be tapped as walking repositories of useful data.

"Cities are covered in a network of people who all have smartphones, it’s a fruitful way to make the cities smarter."

To run the smartest cities, residents need to not only be informed but to also be ready to lead the charge. "It has to start from the people up," said David Gershon, founder of the Empowerment Institute.

Advanced big data technologies can provide municipal governments unprecedented opportunities to proactively change the lives of their citizenry, all through the use of data.

A "responsive" city is one that uses the information generated by its interactions with residents to better understand and predict the needs of neighborhoods, to measure the effectiveness of city agencies and workers, to identify waste and fraud, to increase transparency, and, most importantly, to solve problems

"Cities are organized vertically, and people live horizontally. Data-Smart City Solutions focuses on local government efforts to improve citizen-city engagement through technology.”

“By using its own data and social media, a city "should learn what its citizens are saying about their needs and the issues in their communities; it should learn across agencies about the solutions to problems; it should learn from the data about good actors and bad actors,"

“Data drives decision-making and it drives a lot of the services we all consume… Publishing the data allows everyone inside the city and outside to go in and add intelligence and services on top of it.” Peter Marx, Chief Technology Officer for the City of Los Angeles

Regarding Internet access:

"Without citizen access to reliable, high speed broadband and/or WIFI, the participation rates in studies to determine what gaps smart cities technologies can fill may not be accurately identified. Smart city technology should not happen in ivory towers, but must foster better citizen engagement. “Jim Kurose, assistant director of the National Science Foundation’s (NSF) Computer and Information Science and Engineering Directorate.

"Broadband is “a critical utility” like power or water and citizens need it to advance." Hugh Miller Chief Technology Officer, San Antonio, Texas

Data-Driven Performance Indicators:

Local governments strive to represent and serve their constituents well. Data-driven metrics ensure that everyone is speaking the same language as they continue to solve problems efficiently. By publishing and tracking key metrics on an Open Data Dashboard, residents can respond accordingly by making their voices heard through direct feedback and voting at the polls.

• Create goals based on the latest data uploaded to the system and reflected on the Open Data Dashboard across as many departmental or functional areas as needed
• Use data applications to easily reflect fresh uploaded data with minimal human intervention.
• Organize your goals from a single Open Data Dashboard that shows the status of all goals from a single view
• Through the use of the Open Data Dashboard you can scale your performance program up or down as needed, expanding to additional areas, or by focusing on the top priorities for your government. 

The roles and general responsibilities in support of open data.

Role	General Responsibilities
Data Coordinators	Data Coordinators are designated for each department as the main point of contact and accountability for open data in their department. General responsibilities include: ·         Inventorying department data sets ·         Establishing a plan and timeline for publishing them ·         Serving as a key point of accountability for timelines and questions about data sets ·         Implementing privacy, data licensing, metadata and other standards and practices ·         Providing quarterly reports on progress in implementing the open data plan
Chief Data Officer	The Chief Data Officer is designated by the Mayor and is accountable for the city’s overall implementation of the open data policy. General responsibilities include: ·         Creating processes, rules and standards to implement the open data policy, including but not limited to: o   Prioritizing data sets for publication o   Determining what datasets are appropriate for public disclosure o   Creating data licensing and metadata standards and guidelines o   Providing guidance and assistance to City departments in releasing open data o   Providing guidance and assistance to City departments in assessing and, where appropriate, improving the accuracy, completeness, interoperability and other quality dimensions of data o   Facilitating creation of department implementation plans and reporting ·         Maintaining the open data website ·         Presenting an annual citywide implementation plan for open data ·         Assisting departments with analysis of city datasets.
Data Stewards	Data Stewards are individuals in charge of individual databases, datasets, or information systems. In general, a data steward has business knowledge of the data and can answer questions about the data itself. General responsibilities likely include: ·         Managing the dataset or source and authorizing changes to it ·         Managing access to and use of the data, including documentation ·         Managing accuracy, quality and completeness of the data.
Data Custodian	Data Custodians are individuals that assist with the technical implementation of individual databases, datasets, or information systems. Not all systems or data sources will have a data custodian. General responsibilities likely include: ·         Implementing technical changes requested by the data steward ·         Administration and maintenance for the database or system.

Notable Quotes:

"Residents can also be tapped as walking repositories of useful data since cities are covered in a network of people who all have smartphones. It’s a fruitful way to make the cities smarter." Tom Saunders, a researcher at England-based research and innovation foundation Nesta.

Businesses want to locate in smart communities. Why? Because being a part of a progressive city says good things about their brand. Plus, smart cities attract technical professionals and members of the creative class, a boon for recruiting qualified candidates.

 “We know that using data and technology has the ability toimprove the quality of lives of our residents,”“I am proud of the work we have done, and we will continue our focus onbettering city services through new and innovative approaches. ” Mayor of Boston Massachusetts, Martin J. Walsh

A municipality gains the ability to deliver results by its acceptance and creation of a culture of using its data assets., enhanced abilities to properly and efficiently access the information collected from archived municipal files, in various city department and from the public sector in a centralized data platform.

The importance of sharing "Big Data" (Government data) is an asset whose value otherwise is capped at the operational value it produces internally.

Opening "Big Data" to the public redeploys this asset to encourage entrepreneurialism and innovation outside the four corners of city hall.

Advanced big data technologies can provide municipal governments unprecedented opportunities to proactively change the lives of their citizenry, all through the use of data.

A "responsive" city is one that uses the information generated by its interactions with residents to better understand and predict the needs of neighborhoods, to measure the effectiveness of city agencies and workers, to identify waste and fraud, to increase transparency, and, most importantly, to solve problems.

"Cities are organized vertically, and people live horizontally. Data-Smart City Solutions focuses on local government efforts to improve citizen-city engagement through technology.”

“By using its own data and social media, a city "should learn what its citizens are saying about their needs and the issues in their communities; it should learn across agencies about the solutions to problems; it should learn from the data about good actors and bad actors,"

There is a high importance in the need for the sharing of collected data, whether locally produced or nationally produced. “Sharing” of collected data should be carried out to encourage not only savings in expenditures on the; municipal, state and National level but to encourage entrepreneurialism and innovation as well.

Unshared data is an asset whose beneficial value and potential monetary or budgetary gains to the local and national government, as well as public, is “unrealized” if it is "capped or restricted" at the source. By restricting access to this pool of data. its actual or true value will never be realized.

By opening the acquired "Big Data" to the managers of government departments and offices, they can gain unknown knowledge that was previously overlooked which could be beneficial to them on several levels.

"Yes, open data should be a big part of smart cities policies but there's also need to create the demand for it, the smarter cities are the ones that are able to transparently dish this data out well in advance before discussing future infrastructure projects." Tom Saunders, a researcher at England-based research and innovation foundation Nesta.

Cities are sitting on masses of data. Exposing open data for citizens, developers and businesses can unleash innovation and city efficiency and new monetization opportunities.

Eventually, data could even be shared across cities to support wider innovations, a smart region and a smart nation