Configure the File JSON Input adapter by specifying values for the ESP connector, formatter, and transporter modules in the adapter configuration file.
Parameter | Description |
---|---|
Log4jProperty |
Type: string (Optional) Specify the path to the log4j.properties logging file you wish to use. The default value is $ESP_HOME/adapters/framework/config/log4j.properties. |
Parameter | Description |
---|---|
Module |
(Required) Section containing all information for this module. It contains a type attribute for specifying the module type. For example, transporter. |
InstanceName |
Type: string (Required) Specify the instance name of the specific module you wish to use. For example, MyInputTransporter. |
Name |
Type: string (Required) The name of the module as defined in the modulesdefine.xml file. For example, <TransporterType>InputTransporter. |
Next |
Type: string (Required) Specify the instance name of the module that follows this one. |
BufferMaxSize |
Type: integer (Advanced) Specify the capacity of the buffer queue between the transporter module and the next module. The default value is 10240. |
Parameters |
(Required) Section containing the FileInputTransporterParameters element. |
FileInputTransporterParameters |
(Required) Section containng parameters for the File Input transporter. |
Dir |
Type: string (Required) Specify the absolute path to the data files which you want the adapter to read. For example, <username>/<foldername>. No default value. Use a forward slash for both UNIX and Windows paths. |
File |
Type: string (Required) Specify the file which you want the adapter to read or the regex pattern to filter the files on a given directory. See the DynamicMode parameter. No default value. |
AccessMode |
Type: string (Required) Specify an access mode. The adapter supports two modes:
|
DynamicMode |
Type: string (Advanced) Specify a dynamic mode. The adapter supports three modes:
An example regex pattern is ".*\.txt". This filters to only files that end with ".txt". In regex patterns, an escape character, "\", is necessary prior to the meta chars if you wish to include them in the pattern string. |
PollingPeriod |
Type: integer (Advanced) Define the period, in seconds, to poll the specified file or directory. Set this parameter only if the value of the DynamicMode parameter is set to dynamicFile or dynamicPath. A value of <=0 turns off polling. The default value is 0. |
RemoveAfterProcess |
Type: boolean (Optional) If this property is set to true, the file is removed from the directory after the adapter processes it. If the value of the DynamicMode parameter is set to dynamicFile or dynamicPath, the file is not removed after the adapter processes it. The default value is false. |
ScanDepth |
Type: integer (Optional) Specify the depth of the schema discovery. The adapter reads the number of rows specified by this parameter value when discovering the input data schema. The default value is three. |
Parameter | Description |
---|---|
Module |
(Required) Section containing all information for this module. It contains a type attribute for specifying the module type. For example, formatter. |
InstanceName |
Type: string (Required) Specify the instance name of the specific module you wish to use. For example, MyInputTransporter. |
Name |
Type: string (Required) The name of the module as defined in the modulesdefine.xml file. For example, <TransporterType>InputTransporter. |
Next |
Type: string (Required) Specify the instance name of the module that follows this one. |
Parameters |
(Required) Section containing the JsonStreamToJsonStringFormatterParameters parameter. |
JsonStreamToJsonStringFormatterParameters |
(Required) Section containing parameters for the Streaming JSON to JSON String formatter. |
CharsetName |
Type: string (Optional) Specify the name of a supported charset. The default value is US-ASCII. |
Parameter | Description |
---|---|
Module |
(Required) Section containing all information for this module. It contains a type attribute for specifying the module type. For example, formatter. |
InstanceName |
Type: string (Required) Specify the instance name of the specific module you wish to use. For example, MyInputTransporter. |
Name |
Type: string (Required) The name of the module as defined in the modulesdefine.xml file. For example, <TransporterType>InputTransporter. |
Next |
Type: string (Required) Specify the instance name of the module that follows this one. |
Parameters |
(Required) Section containing the JsonStringToEspFormatterParameters parameter. |
JsonStringToEspFormatterParameters |
(Required) Section containing the JSON String to ESP formatter parameters. |
ColumnMappings |
(Required) Section containing the ColsMapping parameter. |
ColsMapping |
(Required) Section containing the Column parameter. |
Column |
Type: string (Required) Specify a value for the JSON data that you wish to map to ESP columns. This value is matched by a pattern path expression. For example, [<Column>JSONPath expression</Column>]. The first <Column/> is mapped to the first column of an ESP row, the second <Column/> is mapped to the second column of an ESP row, and so on. This parameter has two attributes:
|
DateFormat |
Type: string (Advanced) The format string for parsing date values. For example, yyyy-MM-dd'T'HH:mm:ss. |
TimestampFormat |
Type: string (Advanced) Format string for parsing timestamp values. For example, yyyy-MM-dd'T'HH:mm:ss.SSS. |
Parameter | Description |
---|---|
Module |
(Required) Section containing all information for this module. It contains a type attribute for specifying the module type. For example, formatter. |
InstanceName |
Type: string (Required) Specify the instance name of the specific module you wish to use. For example, MyInputTransporter. |
Name |
Type: string (Required) The name of the module as defined in the modulesdefine.xml file. For example, <TransporterType>InputTransporter. |
Parameters |
(Required) Section containing the EspPublisherParameters parameter. |
EspPublisherParameters |
(Required) Section containing parameters for the ESP publisher. |
ProjectName |
Type: string (Required if running adapter in standalone mode; optional if running in managed mode) Name of the ESP project to which the adapter is connected. For example, EspProject2. This is the same project tag that you specify later in the adapter configuration file in the Name parameter of the Event Stream Processor (EspProjects) parameters section. If you are starting the adapter with the ESP project to which it is attached (running the adapter in managed mode), you do not need to set this property as the adapter automatically detects the project name. |
StreamName |
Type: string (Required if running adapter in standalone mode; optional if running in managed mode) Name of the ESP stream to which the adapter publishes data. If you are starting the adapter with the ESP project to which it is attached (running the adapter in managed mode), you do not need to set this property as the adapter automatically detects the stream name. |
MaxPubPoolSize |
Type: positive integer (Optional) Specifies the maximum size of the record pool. Record pooling, also referred to as block or batch publishing, allows for faster publication since there is less overall resource cost in publishing multiple records together compared to publishing records individually. Block publishing (record pooling or batch publishing) is disabled if this value is set to 1. The default value is 256. |
UseTransactions |
Type: boolean (Optional) If set to true, pooled messages are published to Event Stream Processor in transactions. If set to false, they are published in envelopes. Default value is false. |
Event Stream Processor parameters configure communication between Event Stream Processor and the File JSON Input adapter.
Parameter | Description |
---|---|
EspProjects |
Section containing parameters for connecting to Event Stream Processor. |
EspProject |
Section containing the Name and Uri parameters. Specifies information for the ESP project to which the adapter is connected. |
Name |
Type: string (Required) Specifies the unique project tag of the ESP project which the espconnector (publisher/subscriber) module references. |
Uri |
Type: string (Required) Specifies the total project URI to connect to the Event Stream Processor cluster. For example, esp://localhost:19011/ws1/p1. |
Security |
Section containing all the authentication parameters below. Specifies details for the authentication method used for Event Stream Processor. |
User |
Type: string (Required) Specifies the user name required to log in to Event Stream Processor (see AuthType). No default value. |
Password |
Type: string (Required) Specifies the password required to log in to Event Stream Processor (see espAuthType). Includes an "encrypted" attribute indicating whether the Password value is encrypted. Default value is false. If set to true, the password value is decrypted using RSAKeyStore and RSAKeyStorePassword. |
AuthType |
Type: string (Required) Specifies method used to authenticate to the
Event Stream Processor. Valid values are:
If the adapter is operated as a Studio plug-in, AuthType is overridden by the Authentication Mode Studio start-up parameter. |
RSAKeyStore |
Type: string (Dependent required) Specifies the location of the RSA keystore, and decrypts the password value. Required if AuthType is set to server_rsa, or the encrypted attribute for Password is set to true, or both. |
RSAKeyStorePassword |
Type:string (Dependent required) Specifies the keystore password, and decrypts the password value. Required if AuthType is set to server_rsa, or the encrypted attribute for Password is set to true, or both. |
KerberosKDC |
Type: string (Dependent required) Specifies host name of Kerberos key distribution center. Required if AuthType is set to kerberos. |
KerberosRealm |
Type: string (Dependent required) Specifies the Kerberos realm setting. Required if AuthType is set to kerberos. |
KerberosService |
Type: string (Dependent required) Specifies the Kerberos principal name that identifies an Event Stream Processor cluster. Required if AuthType is set to kerberos. |
KerberosTicketCache |
Type: string (Dependent required) Specifies the location of the Kerberos ticket cache file. Required if AuthType is set to kerberos. |
EncryptionAlgorithm |
Type: string (Optional) Used when the encrypted attribute for Password is set to true. If left blank, RSA is used as default. |