File Input Transporter Module Parameters

The File Input transporter reads data from local files, wraps the data with string, and sends it to the next module specified in the adapter configuration file. Set values for this transporter in the adapter configuration file.

The File Input transporter supports schema discovery.
Parameter Description
Dir

Type: string

(Required) Specify the absolute path to the data files which you want the adapter to read. For example, <username>/<foldername>. No default value.

Use a forward slash for both UNIX and Windows paths.

File

Type: string

(Required) Specify the file which you want the adapter to read or the regex pattern to filter the files on a given directory. See the DynamicMode parameter. No default value.

AccessMode

Type: string

(Required) Specify an access mode. The adapter supports two modes:
  • rowBased – the adapter reads one text line at a time
  • Streaming – the adapter reads a pre-configured size of bytes into a buffer
No default value.
DynamicMode

Type: string

(Advanced) Specify a dynamic mode. The adapter supports three modes:
  • Static – the adapter reads the file specified in the Dir and File parameters
  • dynamicFile – the adapter reads the file specified in the Dir and File parameters and keeps polling the new appended content. The polling period is specified in the PollingPeriod parameter.
  • dynamicPath – the adapter polls all the new files under the Dir parameter. Also, the File parameter acts as a regex pattern and filters out the necessary files.
The default value is Static. If the DynamicMode has been set to dynamicPath and you leave the File parameter empty, the adapter reads all the files under the specified directory.

An example regex pattern is ".*\.txt". This filters to only files that end with ".txt". In regex patterns, an escape character, "\", is necessary prior to the meta chars if you wish to include them in the pattern string.

PollingPeriod

Type: integer

(Advanced) Define the period, in seconds, to poll the specified file or directory. Set this parameter only if the value of the DynamicMode parameter is set to dynamicFile or dynamicPath.

A value of <=0 turns off polling. The default value is 0.

RemoveAfterProcess

Type: boolean

(Optional) If this property is set to true, the file is removed from the directory after the adapter processes it. If the value of the DynamicMode parameter is set to dynamicFile or dynamicPath, the file is not removed after the adapter processes it.

The default value is false.

ScanDepth

Type: integer

(Optional) Specify the depth of the schema discovery. The adapter reads the number of rows specified by this parameter value when discovering the input data schema.

The default value is three.