Adding a File System Data Source to Data Cube

You can add file system data sources in Data Cube. After the data source is added, you can restructure the data. For example, you can combine data from two or more data sources into a single data source. For information about the available data restructuring options, see Restructuring Data in Data Cube.

Before You Begin

Procedure

  1. From the Data Cube dashboard, next to File System, click Add New.

    Tip

    Alternatively, click File System to open the Data Sources (File System) page, and then click Add File System in the upper-right corner.

    The New Data Source (File System) page appears.

  2. In the Data Source Name section, enter the following information:

    Field Name

    Description

    Steps

    Index Server

    The Index Server that you want to use for the data source.

    • Click the Index Server list, and then select an Index Server.

    Data Source Name

    The name of the data source as you would like for it to appear in Data Cube.

    • Enter a name for the data source.

    Data Source Description

    Enter a short description for the data source.

    • Optional: Enter a description for the data source.
  3. Click Next, and then in the Source Details section, enter the following information:

  4. Field Name

    Description

    Steps

    Access Node

    The access node is a MediaAgent in your CommCell environment that can connect to the file system data source.

    • Click the Access Node list and select a MediaAgent.

    Source Details

    Paths to the directories and files that you want to include in the data source.

    Examples

    • Local directory on the access node:

      C:\dir1\content

    • Shared UNC directories:

      \\server\myshare\dir1

      \\server2\c$\share2\dir2

    • Enter the path to the file system directories or files that you want to crawl.

      Tip

      To add multiple paths, enter each path on a separate line.

    User name

    Specify the username for the credentials with read access for all of the locations in the source details.

    Note

    If you are using paths on the access node, you can leave the username and password blank.

    • Enter your user name to access the data source locations.

      Note

      If you entered multiple locations in Source Details, enter a user name that has access to all of the locations.

    Password

    The password for the user name to access the data source.

    • Enter the password for the user name to access the data source.

    Incremental Crawl

    Incremental crawls only collect new data and data that changed since the previous crawl. Otherwise, data from the previous crawl is completely overwritten by each subsequent crawl. Incremental crawling is enabled by default.

    • Optional: Click the slider to disable incremental crawling.

    Exclude Filter

    Use this option to filter the data from the source that you want to exclude from Data Cube. You can exclude data in a specific directory or based on matching wildcard expressions in the file name or directory path.

    Examples:

    The following are some examples of filter patterns:

    • To exclude text files, enter *.txt.

    • To exclude different types of multimedia files, enter *avi, *mpg, *mp3, *mp4, *mov.

    • To exclude all of the files in a temporary directory, enter */temp.

    • Optional: Enter a path or pattern to exclude data from the data source.

      Tip

      To add multiple filters, enter the filters as a comma separated list.

  5. Click Next.

  6. To include the file content in the file system data source, do the following in the Content Search section:

    1. To enable content search, click the slider.

    2. To specify the types of files and the sizes of files that you want to be searchable, configure the Include Filter and File Size options.

      Note

      By default, the file system connector only collects metadata, such as file name and modified date, from source files. When content search is enabled, you can search the content of your data and enable features like entity extraction.

  7. Click Submit.

    The data source name Configuration page appears.

What to Do Next

Crawling a Data Source in Data Cube

Loading...