FileSystem:

This data source can easily stream files from the local filesystem using the reader source.

*The Reader source supports three types of file extension:

  • Csv files
  • Excel files
  • Json files
  • yaml files

Example Configuration

version: 1
encrypt_credentials: false
plugins:
  extractors:
    - name: FileSystem
      connectorname: FileSystem
      config:
      properties:
        type: <csv/excel/jsonl>
        filepath: <Paste the copied file path>
        # sheetname is required for excel type
        sheetname:

Configure the Data Hub to connect FileSystem/csv/excel/json/yaml Files

  1. To start, Click the smartlife Data Hub icon on the Navigation Pane.

FileSystem - smartlife BI

  1. Click Add Pipeline and provide the name for the new pipeline.

FileSystem - smartlife BI

  1. Select the newly created pipeline and choose the Filesystem / CSV / Excel / JSON / YAML connector. Double click or Click on Add Template option to add template.

FileSystem - smartlife BI

  1. Click the “Upload File” button to upload your Excel or CSV file

FileSystem - smartlife BI

  1. Copy the filepath and replace in filePath property. If it’s an Excel file, add the sheet name, otherwise remove it, if using Filesystem connector.

FileSystem - smartlife BI

FileSystem - smartlife BI

  1. Click Save and choose the desired destination to save the pipeline.

FileSystem - smartlife BI

  1. Now the pipeline will be saved and started automatically. Click on Logs to see if the run is completed and data source is created in smartlife BI.

FileSystem - smartlife BI

  1. Creating a Pipeline in smartlife Data Hub automatically creates a Data Source in smartlife BI. The smartlife BI Data Source is a live data source to the destination database used in smartlife Data Hub. For more information on the relationship between smartlife Data Hub Pipeline and the associated Data Sources in smartlife BI , please refer to Relationship between smartlife Data Hub Pipeline and Associated Data Sources in smartlife BI.

FileSystem - smartlife BI

Schedule Data Hub Job

  1. To configure interval-based scheduling, click on the schedules tab and select the created pipeline and click on the schedule icon and configure it.

FileSystem - smartlife BI

FileSystem - smartlife BI

  1. For on-demand refresh, click Run Now button.

FileSystem - smartlife BI.

  1. The Schedule history can be checked using the history option as well as logs.

FileSystem - smartlife BI

  1. Click on Logs to see if the run is completed and data source is created in smartlife BI.

FileSystem - smartlife BI

  1. Click Edit DataSource Option to view the created tables.

FileSystem - smartlife BI