Apache Spark SQL

Data catalog tools

List of data catalogs tools

Data catalog is a structured collection of data used by an organization. It is a kind of data library where data is indexed, well-organized, and securely stored. Most data catalog tools contain information about the source, data usage, relationships between entities as well as data lineage. This provides a description of the origin of the data and tracks changes in the data to its final form.

Data dictionary tools

List of tools that enable design and building of data dictionaries.

Data Dictionary is a set of important information about data used within an organization (metadata). This information includes names, definitions, and attributes about data, owners, and creators of assets. Data Dictionary tools provide insights into meaning and purposes of data elements. They add useful aliases about the scope and characteristics of data elements, as well as the rules for their usage and application.

Business Glossary tools

List of tools that help build, maintain an share business glossaries.

Business Glossary is a collection of definitions and terms used within a business unitorganization. It is not used to define data, but rather to define what a particular term means from a business perspective. The definitions are mapped and logically linked to entities in the database. This enables structured communication between teams and allows them to reveal potential errors resulting from incorrect nomenclature.

Data democratization tools

Data democratization tools enable the access of data by anyone in the organization, regardless of their role or technical knowledge, in order to make data-informed decisions. It's a crucial element which eliminates gatekeepers and bottlenecks that they create in the company.

Data discovery tools

Data enables companies to make informed decisions and understand needs of the customer and the business. In order to leverage data, data discovery tools help organizations visualize, analyze and get insights from it. They start by collecting data from multiple sources to then consolidate, classify, and catalog it into a single repository to simplify its evaluation.

Data Governance tools

List of data governance tools

Data governance is a strategy of handling data within an organization. It is a set of rules, policies, standards, practices etc. which main purpose is to ensure data has a high quality and integrity, is safely stored and there are no ambiguities in meaning of common terms. Applying this strategy is a long process, engaging whole organization, especially IT and data consuming departments. There are certain data governance tools which helps applying these theoretical plans in real life.

Data intelligence tools

Data intelligence tools refer to the artificial intelligence and machine learning tools used by companies in order to analyze and transform data into information that is valuable and relevant for improving the company's operations.

Data lineage tools

Data lineage tools are software that allows to extract, view and analyze data lineage. Data lineage is the process of understanding and visualizing data flow from the source to different destinations. It allows to create a map of the data journey through the entire ecosystem.

Data literacy tools

Data literacy tools empower employees to read, work with, analyze, and communicate with data. It enables them to ask the right questions and build knowledge in order to communicate meanings through data. Data skills are now essential for any role and the lack of data literacy within the company can lead to misinterpretation and misrepresentation of the facts.

Data observability tools

Data observability tools help the company track and understand the state of its data at any given time and provide it with full insight into their data pipelines. They also allow them to identify, monitor and troubleshoot errors in order to minimize data issues and improve high data quality.

Data profiling tools

Data Profiling tools allow analyzing, monitoring, and reviewing data from existing databases in order to provide critical insights. Data profiling can help organizations improve data quality and decision-making process by identifying problems and addressing them before they arise.

Data quality tools

Data quality tools measure how good and useful a data set is to serve its intended purpose. High quality data can lead to better decision-making and faster insights as well as reduce the costs of identifying and dealing with bad data in the system. It can also save time and allow companies to focus on more important tasks.

Database design tools

List of tools that allow for the design of a database.

Database design is a set of activities aimed at improving the development, implementation, and maintenance of an enterprise data management system. Its main goal is to reflect the outline of the database system into real models: physical and logical. Database designers determine what data is to be stored inside the database, and how objects are to be related to each other.

Database diagram / Reverse Engineering tools

List of database ER diagram (ERD) / schema visualization / reverse engineering tools for popular databases.

Database documentation generators

List of tools that automatically generate documentation for your database. For more advanced capabilities see Database Documentation Tools

Database documentation tools

List of tools that allow the documentation (e.g. with descriptions) of schema of your databases.

Database documentation is a set of descriptions of the data, that facilitate management, maintenance, and retrieval of resources. Documentation can take many forms, however, it usually acts as a data catalog that holds detailed information about the table schema, all relationships, data types, etc. Description of datasets as well as single data objects also is referred to as metadata (data about the data).

GDPR compliance software

List of GDPR compliance tools

Collecting, storing, and processing data of the EU citizens have changed on May 25, 2018, when European Parliament decided to replace Data Protection Directive 95/46/EC with General Data Protection Regulation (GDPR). This new law has placed more emphasis on consumer data protection and gave consumers greater control over their personal information. Complying with these rules gets complex as more customers’ data is collected and processed. Following GDPR Compliance software can help to simplify and speed up this tedious process.

GUI tools

Graphical user interface (GUI) tools consist of an interface that includes visual components such as buttons or icons, through which the user interacts with electronic devices. The biggest advantage of graphical user interface is an improvement in usability for the average person. It gives the ability to perform various tasks on an electronic device, without any programming knowledge.

Metadata Management tools

List of metadata management tools

Metadata management tool is a solution that supplements the data stored by the enterprise environment with useful information. Proper metadata management is a crucial factor to make information searchable, easy to locate and understand. Such tools add meaningful context to raw data, making it convenient to discover even by non-IT members of an organization.

Reference Data Management tools

Reference data management tools help manage and bring consistency to your data. They enable business users to aggregate data quickly and understand the overall condition of the business. Reference data tends to be overlooked in many enterprises but although its amount in most organizations is relatively small, it can cause a major difference in data management.

Sensitive data discovery tools

List of sensitive data discovery tools

In recent years many international organizations, countries, states etc. have introduced strict regulations regarding sensitive data storing and processing (GDPR in EU, CCPA in California state or PDBP in India just to name a few) to ensure that companies and organizations handle personal information correctly. This should motivate organizations to revise their data protection policies and identify all private data they collect and process. Invaluable help with this tedious task are sensitive data discovery tools.

SSIS documentation tools

List of SQL Server Integration Services (ETL) documentation tools.

SSIS stands from SQL Server Integration Services and is a platform that focuses on creating enterprise-level data integration and data transformation solutions. As a component of Microsoft SQL Server database software, it is commonly used for data integration and data transformation tasks. Proper operation requires multiple teams to work together. Without proper documentation development, maintenance, and retrieval tasks become demanding and time-consuming. This is where SSIS documentation tools come into play.