蘑菇视频

Is your federal organization struggling to use data to affect business outcomes and drive insight into day-to-day decision making? You鈥檙e not alone. Research indicates that organizations , while poor data quality .

To move from data chaos to actionable insights, more organizations are turning to big data platforms, creating 鈥渄ata lakes,鈥� or centralized repositories that store large amounts of structured and unstructured data. By bringing together vast amounts of data, organizations can now correlate activities in one part of the organization to effects in another, aggregate disparate siloed databases, and reduce overall operations and maintenance costs.

Implementing a big data platform is not without its challenges. For example, using proprietary analytical tools can mean organizations get locked into a one-off solution that doesn鈥檛 integrate naturally with their big data platform. As they navigate this and other challenges, federal agencies may have misconceptions about what big data platforms are, what they aren鈥檛, and how to use them effectively.

Here are 5 myths of big data platforms for government organizations鈥攁nd the reality.

Myth No. 1: If we combine all our data, we鈥檒l have full insight into what our organization is trying to accomplish.

The Reality: Some data can鈥檛 be easily combined, and policies, restrictions, and security need to continue to be enforced. To protect sensitive data such as personally identifiable information or protected health information, organizations traditionally duplicate data in multiple data lakes or restrict access to specific users, driving up storage costs and reducing insights. Focus first on using metadata and attribute authorization schemes to enable a successful single data lake implementation and protect data at the source, record, or field level.

Myth No. 2: Once all our data is in one place, we can focus on analytics.

The Reality: You鈥檒l need to catalog stored data to enable your analysts. Information such as data set name, formats, tagging, release-ability, retention, and more must be available and continually managed. In most organizations, data scientists spend 80% of their time 鈥渄ata wrangling鈥� and only 20%performing real analysis. Focusing on data governance lets organizations begin to turn these statistics around. Data management is key to gaining value from data.

Myth No. 3: If we use industry technologies, we鈥檒l be successful in implementing our big data platform.

The Reality: It鈥檚 not just about the technology. Successful organizations also adopt an agile culture鈥攖rying new ideas and pivoting quickly based on lessons learned. Big data platforms let organizations experiment and vet open source tools coupled with DevOps capabilities in sandbox environments before including them in the operational environment.

Myth No. 4: The more data we have, the more questions we can answer and the more knowledgeable we鈥檒l be.

The Reality: It鈥檚 about the right data, combined with the right tools, for the right problem鈥攏ot how much data is stored in the data lake. Instead of collecting all data and then asking questions, start by having a conversation about what problem you鈥檙e trying to solve and what data you need. The implementation team can gather information to capture metadata, better understand security concerns, and develop use cases enabling them to ingest and store data more effectively.

Myth No. 5: We don鈥檛 have enough data scientists to analyze the data and ask the right questions.

The Reality: Data-driven solutions need to create capabilities that democratize analytics to a wider audience. With simple designs, like a drag-and-drop user interface, more people on your team can analyze data. This allows your experienced data scientists to focus on new models while allowing a larger community to use previously built models and analytics against new data sets.

What鈥檚 holding organizations back from gaining value from their data is not technology, but rather a combination of processes, culture, and the right technology choices for the right problems. With modern data management platforms and best practices鈥攊ncluding security鈥攆rom the outset, organization could start flipping the 80-20 rule of data science. By adopting an open, standards-based big data platform and institutionalizing an agile culture, organizations can accelerate successful big data lakes and avoid falling victim to the big data swamp.