Big Data Glossary

high performance computing and storage needs that are too complex to be handled by a standard desktop workstation.

a mathematical formula placed in software to perform a mathematical analysis of a dataset.

an area of computer science concerned with the development of computers that are able to engage in human-like thought processes.

software/hardware that provides the tools and computational power needed to build and perform various analytical queries.

a language-neutral data serialization system. It is helpful quite helpful, as it deals with data formats that can be processed by multiple languages. It is also the preferred tool to serialize data in Hadoop.

using data about people’s behavior to understand how and why people behave they do.

a term often used to describe the massive amounts of generated structured and unstructured data that is too large, complex and varied for traditional processing software.

is an umbrella term that includes the applications, infrastructure and tools, and best practices that enable access to and analysis of information to improve and optimize decisions and performance.

users’ web activity analysis based on the items the clicked on a page.

data and computing resources available and accessed on-demand over the internet instead of local computers and devices.

a multivariate method which aims to classify a sample of subjects (or objects) on the basis of a set of measured variables into a number of different groups such that similar subjects are placed in the same group.

a database which stores data in columns, rather than rows. The big advantage of this type of database is faster hard disk access.

a form of analysis ensuring a step-by-step approach to comparing and calculating designed to notice patterns within giant data sets.

analysis attributing sales, showing average order value and the lifetime value.

the process of compiling information from different data sources to prepare for data processing and analysis.

the main responsibilities of a data analyst are to collect, manipulate and analyze data. Based on this data, they prepare reports, which may include charts, graphs, dashboards and other visualizations.

defines how data is stored, managed, and used in a system, establishing common guidelines for data operations enabling organizations to model, gauge, predict, and control the flow of data in the system.

conducting a review and revision of data to correct misspellings, remove duplicate entries, provide more consistency and add missing data.

the process of the organization of storing, processing and consuming of data.

a continuous stream of structured data providing users with updates of current information from multiple sources.

a set of processes or rules that ensure the integrity of the data and that data management best practices are met.

the process of combining data from different sources and presenting it in a single view.

a subdivision of the data warehouse, usually geared towards a specific business team or line.

an analytical process where large data sets are explored or “mined” to search for meaningful patterns, relationships and insights.

structuring the data for communication between technical and functional people to show the data needed for business processes.

the process of collecting statistics and information about data in an existing source.

measuring data to determine its worthiness for planning, operations and decision making.

a discipline that incorporates statistics, data visualization, data mining, machine learning and database engineering to solve complex problems.

a visual conceptualization of data designed to derive meaning or facilitating the communication of information.

a place to store data for the purpose of reporting and analysis.

using data to support the decision-making process.

a digital collection of data and the structure around which the data is organized. The data is usually stored into and accessed a database management system (DBMS).

a collection of data usually in tubular form.

is a computer program designed for retrieving, managing and storing document-oriented information, sometimes referred to as semi-structured data.

finding patterns within data without standard procedures or methods. It is a means of discovering the data and to find the data sets main characteristics.

a process used in data warehousing to prepare data for use in reporting or analytics.

Employing graph structures (a concrete set of certain entities or ordered pairs), with properties, nodes and edges for data storage, thus providing index-free adjacency i.e. every element is directly linked to its neighbour element.

a framework that allows distributed processing of large datasets across clusters of computers using a simple programming model.

was created to reliably store enormous amounts of data sets and stream those data sets to user applications at high bandwidth to user. Thousands of servers in a large cluster, both directly host the attached storage and execute user application tasks.

any database system that relies on memory for data storage.

an open-standard file format. It uses human-readable texts to transmit data objects which consist of array data types and attribute–value pairs (or any other serializable value).

created to securely store enormous data sets and stream those data sets to user applications at high bandwidth. If the cluster is large, thousands of servers both execute user application tasks and host directly attached storage.

any delay in a response or delivery of data from one point to another.

the use of algorithms to allow a computer to analyze data for the purpose of “learning” which action to perform when specific events or patterns occur.

a general term referring to the process of breaking down a problem into pieces that are disseminated across many computers within the same network or cluster.

data about data; gives information about what the data is about.

a database optimized to be processed simultaneously allowing multiple operations to be performed by multiple processing units at a time.

a branch of AI, which deals with making human language (in both spoken and written forms) understandable to computers.

a type of database management system which does not employ the relational model. It was  designed to handle large volumes that do not follow a fixed scheme.

databases which store data in the form of objects, as used by object-oriented programming. They differ from graph or relational databases and most of them offer a query language that allows object to be found with a declarative programming approach.

multidimensional data analysis using three operations: drill-down (the ability for users to see the underlying details) consolidation (the aggregation of available), and slice and dice (users are able to select subsets and view them from different perspectives).

the process of providing users with access to large amounts of transactional data in a way that they can derive meaning from it.

a location to gather and store data from multiple sources so that more operations can be performed on it before sending to the data warehouse for reporting.

a columnar storage format open to all projects in the Hadoop ecosystem, regardless of data model, the choice of data processing framework or programming language.

the use of statistical functions on a single or multiple dataset to predict future trends and events.

asking for information to answer a certain question.

data that is created, stored, analyzed, processed and visualized within milliseconds.

an algorithm on an ecommerce site which analyzes that analyzes customers’ purchases and actions and then uses that data to make recommendations for complementary products.

Data that describes an object and its properties. The object may be physical or virtual.

the structure that defines the organization of data in a database system.

data that is not structured by a formal data model, but provides other means of describing the data and hierarchies.

the application of statistical functions on comments people make on the web and through social networks to determine how they feel about a product or company.

is a method for distributing data across multiple machines.

a programming language for retrieving data from a relational database.

any data organized into structured fields such as a spreadsheet or database.

the application of statistical, linguistic, and machine learning techniques on text-based sources to derive meaning or insight.

Big data – and the business challenges and opportunities associated with it – are often discussed or described in the context of multiple V’s.

information that either does not have a pre-defined data model or is not organized into a pre-defined manner and is therefore not stored in a structured field.

this is the most important “V” from the business perspective. Big Data’s value usually comes from pattern recognition and insight discovery which leads to stronger customer relations, more effective operations and other clear and quantifiable business benefits.

data, by it’s nature, is always changing and companies are constantly on to capture, analyze and manage it. This can be in text or sentiment analytics, changes in the definition of phrases and keywords.

the range and diversity of various data types, including semi-structured data, unstructured data, and raw data.

the speed at which companies receive, store and manage data – e.g., the specific number of social media posts or search queries received within a day, hour or other unit of time.

the true accuracy of information and data assets, which usually determines confidence at the executive-level.

the size and amounts of big data that companies manage and analyze.