Exploring data observability’s limits in data migration, integrity audits, and the need for specialized tools for reliability.Published at:https://www.eckerson.com/articles/the-shiny-allure-of-data-observability-its-limits-in-data-migration-integrity-audits-and-certificationDoor Eckerson Group

A strategic approach to data monetization covering key processes, risks, and organizational shifts needed for success.Published at: https://www.eckerson.com/articles/unlocking-data-monetization-successDoor Eckerson Group

A fresh perspective on data architecture, advocating for an 'anti-monolith' approach inspired by engineering best practices.Published at: https://www.eckerson.com/articles/good-data-architecture-anti-monolithDoor Eckerson Group

Many practitioners view data mesh and data fabric as mutually exclusive approaches to data strategy. However, these paradigms complement each other. Data mesh focuses on decentralization and autonomy; Data fabric ensures centralized integration and governance. Let’s dive into how blending elements of both can offer flexibility and control to create…

As organizations grapple with data spread across various storage locations, solutions like Coginiti Hybrid Query offer a much-needed alternative to fragmented tools.Published at:https://www.eckerson.com/articles/a-novel-approach-for-reducing-cloud-data-warehouse-expenses-from-coginitiDoor Eckerson Group

This blog post explores the evolving landscape of data catalogs, highlighting ten key market trends driving the adoption of next-generation solutions.Published:https://www.eckerson.com/articles/ten-key-market-trends-in-next-generation-data-catalogsDoor Eckerson Group

Data teams must filter, blend, and refine raw data inputs to create the high-octane fuel that drives innovation with artificial intelligence and machine learning (AI/ML).Published at:https://www.eckerson.com/articles/refining-the-right-fuel-how-data-integration-drives-the-ai-ml-model-lifecycleDoor Eckerson Group

With numerous data catalog options available, all claiming to be the best, how do you make an informed decision without exhaustive research?Published at:https://www.eckerson.com/articles/unveiling-the-future-of-data-catalogsDoor Eckerson Group

This blog describes the need for data teams to establish a flexible yet well-governed data architecture to support dynamic AI/ML projects.Published at:https://www.eckerson.com/articles/multi-style-data-integration-for-ai-ml-three-use-casesDoor Eckerson Group

Many data leaders want to implement self-service, but don’t realize that they first have to implement the right architecture, governance, operating model, project delivery approach, data, and change management plan.Published at:https://www.eckerson.com/articles/self-service-is-the-outcome-not-the-driver-of-a-data-driven-organization…

Explore the essential characteristics to choose the right conversational query tool for your needs and environment.Published at:https://www.eckerson.com/articles/modernizing-analytics-with-conversational-query-tools-five-must-have-characteristicsDoor Eckerson Group

Data analytics is a balance of flexibility for innovation and governance to control risks. This blog discusses its implications for artificial intelligence (AI), including machine learning (ML) and generative AI (GenAI).Published at:https://www.eckerson.com/articles/ai-ml-innovation-requires-a-flexible-yet-governed-data-architecture…

Non-profit organizations are more mission-driven, consensus-driven, and resource-constrained than commercial organizations. As a result, it’s imperative that non-profits develop a data strategy before plunging into building data solutions. It will save them time, money, and burnout in the long run.Published at: https://www.eckerson.com/articles/why…

Explore the reasons for data engineers to collaborate with data scientists, machine learning (ML) engineers, and developers on DataOps initiatives that support GenAI.Published at:https://www.eckerson.com/articles/dataops-for-generative-ai-data-pipelines-part-iii-team-collaborationDoor Eckerson Group

This blog explores three criteria to evaluate tools that manage unstructured data pipelines for GenAI.Published at:https://www.eckerson.com/articles/data-engineering-for-genai-three-criteria-to-evaluate-pipeline-toolsDoor Eckerson Group

If your data team wants to implement data products, it would be wise to avoid these 12 pitfalls that can torpedo an initiative.Published at:https://www.eckerson.com/articles/12-pitfalls-to-avoid-when-implementing-data-productsDoor Eckerson Group

This article compares data catalogs and data marketplaces and argues that you need both and will soon have both as vendors add data marketplace extensions.Published at:https://www.eckerson.com/articles/why-do-i-need-a-data-marketplace-when-i-have-a-data-catalogDoor Eckerson Group

This blog defines conversational BI, why companies should consider it, and how their power and casual users can best get the desired results.Published at:https://www.eckerson.com/articles/driving-results-with-conversational-bi-best-practices-for-power-and-casual-usersDoor Eckerson Group

Data engineering is now considered a crucial job in IT as Generative AI, the hottest technology of this decade, relies on data engineers to provide accurate inputs.Published at:https://www.eckerson.com/articles/data-engineering-for-genai-how-to-optimize-data-pipelines-and-governanceDoor Eckerson Group

Data engineers and data scientists must manage pipelines for unstructured data to ensure healthy inputs for language models.Published at:https://www.eckerson.com/articles/why-and-how-data-engineers-will-enable-the-next-phase-of-generative-aiDoor Eckerson Group

Companies that adopt DataOps increase the odds of success by making GenAI data pipelines what they should be: modular, scalable, robust, flexible, and governed.Published: https://www.eckerson.com/articles/dataops-for-generative-ai-data-pipelines-part-ii-must-have-characteristicsDoor Eckerson Group

Most data leaders want to deliver data products, but few are doing it. Let's face it: most data teams today function as internal service bureaus that fulfill customer requests that arrive via ticketing systems, email, handwritten notes, or calls from colleagues looking for a favor. Most work double time to keep their request backlogs from balloonin…

GenAI can help data engineers become more productive, and data engineering can help GenAI drive new levels of innovation.Published at:https://www.eckerson.com/articles/achieving-fusion-how-genai-and-data-engineering-help-one-anotherDoor Eckerson Group

Discover how master data management (MDM) provides language models with high-quality enterprise data to improve their response accuracy.Published at:https://www.eckerson.com/articles/improving-genai-accuracy-with-master-data-managementDoor Eckerson Group

Explore our four primary criteria for evaluating conversational BI products.Published at:https://www.eckerson.com/articles/genai-driven-analytics-product-evaluation-criteria-for-conversational-biDoor Eckerson Group

The success of Generative AI depends on fundamental disciplines like DataOps.Published at:https://www.eckerson.com/articles/dataops-for-generative-ai-data-pipelines-part-i-what-and-whyDoor Eckerson Group

With the increasing adoption of Generative AI, learn how data governance will add value to and benefit from Generative AI.Published at:https://www.eckerson.com/articles/data-governance-in-the-era-of-generative-aiDoor Eckerson Group

"Meet the business where it is." If you're on the data team, that's what you're expected to do to empower stakeholders with data. But how far should you go to meet the business? And shouldn’t the business be expected to move a little toward meeting the data where it is?Published at:https://www.eckerson.com/articles/meeting-the-data-where-it-is-time…

The European Union recently passed the first of its kind legal framework on the development, use, and governance of artificial intelligence. It lays out rules and standards with the aim of ensuring technologies are safe and transparent, and do not violate the fundamental rights of an individual.Published at:https://www.eckerson.com/articles/the-eu-…

Most organizations are committed to responsible and ethical use of AI. Yet anticipating unintended consequences before designing and implementing AI can be challenging. This framework and process helps evaluate short-term and long-term impacts across multiple dimensions so you can mitigate AI’s unintended consequences.Published at:https://www.ecker…

It's not easy being the head of data & analytics at a large organization. You must align a large team across multiple disciplines; you must deal with oodles of legacy systems and tools that hamper innovation; and you must deliver business value fast to keep executives at bay and your job intact. You also need to recruit dynamic managers who can pus…

Adopting community of practice principles, along with coaching and mentoring, is a practical approach to fostering and cultivating data literacy.Published at:https://www.eckerson.com/articles/a-people-first-approach-to-developing-data-literacyDoor Eckerson Group

This blog examines the upcoming trend of domain-specific LLMs and evaluates three different methods of implementation.Published at:https://www.eckerson.com/articles/the-next-wave-of-generative-ai-domain-specific-llmsDoor Eckerson Group

Many machine learning (ML) use cases center on real-time calculations. This article defines streaming ML and its architectural components.Published at:https://www.eckerson.com/articles/machine-learning-and-streaming-data-pipelines-part-i-definitions-and-architectureDoor Eckerson Group

Companies need to invest heavily in teams and people, both at corporate and in the field, if they want to become a data-driven organization.Published at:https://www.eckerson.com/articles/organizing-for-success-part-iii-how-to-organize-and-staff-data-analytics-teamsDoor Eckerson Group

Data management practices have changed substantially since the early 1990s and the dawn of data warehousing.Published at:https://www.eckerson.com/articles/the-continuing-evolution-of-data-managementDoor Eckerson Group

Conventional data governance conflicts with today’s world of self-service analytics and agile projects. Published at:https://www.eckerson.com/articles/modern-data-governance-problemsDoor Eckerson Group

Let's reflect on the events of the past year and prognosticate on what may transpire in the months ahead.Published at:https://www.eckerson.com/articles/trends-for-2024-our-team-gazes-into-the-crystal-ballDoor Eckerson Group

Data leaders must prepare their teams to deliver the timely, accurate, and trustworthy data that GenAI initiatives need to ensure they deliver results. They can do so by modernizing their environments, extending data governance programs, and fostering collaboration with data science teams.Published at:https://www.eckerson.com/articles/the-data-lead…

Data modeling is a core skill of data engineering, but it is missing or inadequate in many data engineering teams. These teams focus on moving data with little attention to shaping the data. They engineer processes, not products. Full data engineering is both process and product engineering, and that calls for data modeling.Published at:https://www…

The hardest part about implementing data products is fostering a product mindset among the people responsible for defining, governing, building, and shipping data products. It’s also important that an organization establish processes to facilitate the work of the product team and review boards.Published at:https://www.eckerson.com/articles/data-pro…

Many organizations abandoned data modeling as they embraced big data and NoSQL. Now they find that data modeling continues to be important, perhaps more important today than ever before. With a fresh look you’ll see that today’s data modeling is different from past practices – much more than physical design for relational data.Published at:https://…

Data democratization is the buzzword to describe empowering enterprise stakeholders with data. While there have been advances in data management, governance, and analytics, something keeps getting in the way of achieving data democratization.Published at:https://www.eckerson.com/articles/data-democratization-and-the-duties-of-data-citizenship…

Our industry’s breathless hype about generative AI tends to overlook the stubborn challenge of data governance. Data catalogs address this challenge by evaluating and controlling the accuracy, explainability, privacy, IP friendliness, and fairness of GenAI inputs.Published at:https://www.eckerson.com/articles/generative-ai-needs-vigilant-data-catal…

The need for an independent semantic layer continues to rise as data science gains traction in the enterprise. Its five primary elements—metrics, caching, metadata management, APIs, and access controls—support AI/ML use cases as part of data science projects.Published at: https://www.eckerson.com/articles/why-and-how-to-enable-data-science-with-an-…

Business leaders can address AI bias and use it to have rational discussions about management and human bias.Published at: https://www.eckerson.com/articles/weighing-the-risk-and-reward-of-ai-a-non-technical-guide-for-business-leadersDoor Eckerson Group

Most organizations view data as an asset to be actively managed with standards, controls, and discipline. Yet, they are passive and casual about metadata. Data is managed. Metadata happens. As data management becomes more complex, metadata management is becoming an essential discipline. It is time to think about metadata management from an architec…

Kevin Petrie, the Vice President of Research at Eckerson Group, and Dan O’Brien, research analyst, discussed large language models (LLMs), which are neural networks that analyze text to predict the next word or phrase. These models use training data, often from the internet, to understand word relationships and provide accurate answers to natural l…

Generative AI initiatives require new data pipelines that prepare text files for querying by language models. Data engineers, scientists, and other stakeholders collaborate to design and implement these pipelines, which span text sources, tokens, vectors, vector databases, and LMs.Published at:https://www.eckerson.com/articles/the-new-data-pipeline…

Dan and Wayne discussed the concept of data and analytics operating models, which refers to how organizations organize their data and analytics resources for alignment and efficiency.Door Eckerson Group