Relationship Between smartlife Data Hub and smartlife BI Data Sources

The smartlife Data Hub is an ETL component within the smartlife BI platform that enables data movement from various sources into a centralized data store. It supports data transformation, allowing only processed and aggregated data to be used in smartlife BI for efficient reporting and visualization.

smartlife Data Hub Pipelines

smartlife Data Hub Pipelines are primarily responsible for the data processing and data transformation workflows, enabling users to manage, transform, and load data into a format optimized for visualization in smartlife BI.

The core functionalities of smartlife Data Hub Pipelines include,

  1. Data Connectivity: smartlife Data Hub supports a variety of file types and web, SQL, and NoSQL data sources. please refer to Working With Data Hub Connectors
  2. Data Transformation: Data can be cleaned, filtered, and transformed using pipelines to prepare it for analysis.
  3. Data Integration: Different datasets from multiple sources can be integrated into one consolidated dataset.
  4. Automation: Pipelines can be automated to run at specific intervals, ensuring that data in smartlife BI is kept up to date.
  5. Data Aggregation: Data from multiple sources can be aggregated into a unified format, optimizing it for reporting and visualization.
  6. Destination Support: Data Hub Supports wide range of Destinations. please refer to Working With Data Store

You can configure multiple datastore destinations of the same server type and load data into them for each pipeline. This is useful for scenarios where multiple databases of the same type (e.g., multiple MySQL or PostgreSQL databases) are used for different environments like development, testing, staging, or production, or for different segments of business operations.

smartlife BI Data Sources

smartlife BI Data Sources, on the other hand, are the origin points of the data that is visualized within the smartlife BI platform. These data sources provide the raw data or aggregated data (when query mode is used) that users need for creating dashboards, and visualizations. The functionalities handled by smartlife BI Data Sources include:

  1. Data Connectivity: They enable connection to various data stores such as SQL databases, files, or APIs. please refer to Working With smartlife BI Connectors
  2. Querying: smartlife BI Data Sources can query the data directly from the connected source to retrieve data for visulization.
  3. Data Modeling: Some modeling and basic transformations may be handled within the data source level before being used in dashboards.
  4. Data Refresh: Data sources in smartlife BI are also responsible for periodic refreshes, pulling in new data for analysis when configured.
  5. Direct Query Mode: smartlife Data Hub relies on extracted data; smartlife BI supports querying SQL based live connectors.

Key Differences in Functionality

  • Data Hub Pipelines focus on data processing, transforming, cleaning, and integrating data from various sources into the data store. The transformed data can be stored in the datastore along with or without the original dataset. This extracted data will be created as a data source in smartlife BI for further operations. Also, smartlife Data Hub supports Open API and the Python connector, allowing users to write scripts and move data into smartlife Data Hub.

  • smartlife BI Data Sources supports querying connectors using live mode as well as extracted mode and display the data in the dashboards. smartlife Data Hub does not support modeling via UI as like smartlife BI’s calculated fields and relationship. However these options can be utilized using smartlife BI with the help of smartlife Data Store data sources once after extracting in Data Hub.

Can Multiple smartlife BI Data Sources Be Associated with a Pipeline in smartlife Data Hub?

No, multiple smartlife BI Data Sources cannot be associated with a pipeline in smartlife Data Hub. But the data from multiple pipelines can be combined in data source created from Data Hub as it is a live connection.