Data catalog tools for Apache Spark
List of data catalogs tools
Data catalog is a structured collection of data used by an organization. It is a kind of data library where data is indexed, well-organized, and securely stored. Most data catalog tools contain information about the source, data usage, relationships between entities as well as data lineage. This provides a description of the origin of the data and tracks changes in the data to its final form.
Collibra Catalog
Collibra Catalog empowers business users to quickly discover, understand, contribute, and govern the data that matters so they can generate impactful insights that drive business value. It also allows data stewards to certify datasets so that business users can trust the data that they use in their analysis.
Automated Cataloging: |
|
---|---|
Business Glossary: |
|
Commenting/Community: |
|
Commercial: | Commercial |
Data Classification: |
|
Data Lineage: |
|
Data Profiling: |
|
Export: | MS Excel |
Free edition: |
|
Rating of assets: |
|
Dataedo
Dataedo is an on-premises data catalog & metadata management tool. It allows you to catalog, document, and understand your data with a data dictionary, business glossary, and ERDs. It reads your schema and lets you easily describe each data element with descriptions, business-friendly aliases, and custom fields. It features a data community module, which allows you to crowdsource knowledge about data from everyone in your organization.
Automated Cataloging: |
|
---|---|
Business Glossary: |
|
Commenting/Community: |
|
Commercial: | Commercial |
Data Classification: |
|
Data Lineage: |
|
Data Profiling: |
|
Export: | HTML,MS Excel,PDF |
Free edition: |
|
Rating of assets: |
|
Alation Data Catalog
Alation pioneered the data catalog market and is now leading its evolution into a platform for a broad range of data intelligence solutions including data search & discovery, data governance, stewardship, analytics, and digital transformation. Thanks to its powerful Behavioral Analysis Engine, inbuilt collaboration capabilities, and open interfaces, Alation combines machine learning with human insight to successfully tackle even the most demanding challenges in data and metadata management.
More than 250 enterprises realize business outcomes with Alation, including Salesforce, Cisco, Docusign, Finnair, Pfizer, Nasdaq, and Albertsons.
Automated Cataloging: |
|
---|---|
Business Glossary: |
|
Commenting/Community: |
|
Commercial: | Commercial |
Data Classification: |
|
Data Lineage: |
|
Data Profiling: |
|
Export: | MS Excel |
Free edition: |
|
Rating of assets: |
|
Informatica Enterprise Data Catalog
Informatica Data Catalog is a machine learning-based data catalog that lets you classify and organize data assets across any environment to maximize data value and reuse, and provides a metadata system of record for the enterprise. It automatically scans and catalogs data across the enterprise, indexing it for enterprise-wide discovery using simple, Google-like search.
Automated Cataloging: |
|
---|---|
Business Glossary: |
|
Commenting/Community: |
|
Commercial: | Commercial |
Data Classification: |
|
Data Lineage: |
|
Data Profiling: |
|
Export: | CSV,JSON,MS Excel,Plain text,XML |
Free edition: |
|
Rating of assets: |
|
Ataccama Metadata Management & Data Catalog
Ataccama ONE Data Catalog is an AI-powered metadata management module. It’s a central storage for all of your metadata—imported from other sources, crowdsourced, or automatically captured in continuous data discovery processes.
Automated Cataloging: |
|
---|---|
Business Glossary: |
|
Commenting/Community: |
|
Commercial: | Commercial |
Data Classification: |
|
Data Lineage: |
|
Data Profiling: |
|
Export: | CSV,JSON,MS Excel,XML |
Free edition: |
|
Rating of assets: |
|
IBM Watson Knowledge Catalog
IBM Watson® Knowledge Catalog is an open and intelligent data catalog for managing enterprise data and AI model governance, quality and collaboration. It enables you to organize, define and manage enterprise data to provide the right context to drive value across imperatives like regulatory compliance to data monetization.
Automated Cataloging: |
|
---|---|
Business Glossary: |
|
Commenting/Community: |
|
Commercial: | Commercial |
Data Classification: |
|
Data Lineage: |
|
Data Profiling: |
|
Export: | JSON,MS Excel,XML |
Free edition: |
|
Rating of assets: |
|
Talend Data Catalog
Talend Data Catalog automatically crawls, profiles, organizes, links, and enriches all your metadata. It makes easy to search and access data, then verify its validity before sharing with peers. Up to 80% of the information associated with the data is documented automatically and kept up-to-date through smart relationships and machine learning, continually delivering the most meaningful data to the user.
Automated Cataloging: |
|
---|---|
Business Glossary: |
|
Commenting/Community: |
|
Commercial: | Commercial |
Data Classification: |
|
Data Lineage: |
|
Data Profiling: |
|
Export: | CSV,JSON,MS Excel,XML |
Free edition: |
|
Rating of assets: |
|
Alteryx Connect
Alteryx Connect is a social data cataloging and data exploration platform for the enterprise. The powerful data cataloging provided by Alteryx Connect centralizes business terms and definitions, metrics, and information assets for maximum consistency, discoverability, and collaboration.
Automated Cataloging: |
|
---|---|
Business Glossary: |
|
Commenting/Community: |
|
Commercial: | Commercial |
Data Classification: |
|
Data Lineage: |
|
Data Profiling: |
|
Export: | CSV,MS Excel,PDF |
Free edition: |
|
Rating of assets: |
|
Data catalogs are part of data management tools. They enable automatic metadata management with user-friendly form that makes data easy to understand even for non-IT members of the organisation.
The key feature of data catalogs is to provide metadata context to the user in a way that allows different teams within the organization (both IT and Non-IT) to discover and understand relevant data.
From the organization's perspective, the important functions of data catalog tools are also:
• storage of data resources from different repositories as well as from different engine systems - compatibility with multiple connectors,
• automation of data management processes,
• advanced resource search by name, type, date of change, owner, etc.
• data lineage,
• automated data Classification,
• Discovering data relationship and dependencies between objects,
• Business Glossary, unifying nomenclature and definitions of terms,
• Data Profiling,
Data stewards, business teams, and data analysts often struggle with the problem of what specific data means, where it comes from, and which elements it is directly related to. These are just a few problems for which Data catalog tools have been created. Based on the imported repositories, data catalogs enable automated cataloging and organizing of data, solving the problem of time-consuming querying of the resources.
To avoid misunderstandings data catalog tools provide a Business Glossary, through which the nomenclature is systematized. It contains business terms along with their definition, relationship to each other, as well as its location in the hierarchy of all data assets.
There are many apps for data catalog tasks on the market. We have listed complex data cataloging software that can also solve data profiling, data lineage, and data classification problems, as well as open-source data catalog tools.