File/Hadoop XML Input Adapter Configuration

Type: string

(Optional) Specify a full path to the log4j.properties logging file you wish to use. The default value is $ESP_HOME/adapters/framework/config/log4j.properties.

(Required) Element containing all information for this module. It contains a type attribute for specifying the module type.

For example, transporter.

Type: string

(Required) Instance name of the specific module you want to use. For example, MyInputTransporter.

Type: string

(Required) Name of the module as defined in the modulesdefine.xml file. For example, <TransporterType>InputTransporter.

Type: string

(Required) Instance name of the module that follows this one.

Type: integer

(Advanced) Capacity of the buffer queue between this module and the next. The default value is 10240.

(Required) Element containing the FileInputTransporterParameters element.

(Required) Element containing elements for the File Input transporter.

Type: string

(Required) Specify the absolute path to the data files which you want the adapter to read. For example, <username>/<foldername>. No default value.

To use Hadoop system files, use an HDFS folder uri instead of a local file system folder. For example, hdfs://<hdfsserver>:9000/<foldername>/<subfoldername>/<leaffoldername>.

To use Hadoop, download the binaries for Hadoop version 1.2.1 from http://hadoop.apache.org. Copy the hadoop-core.jar file (for example, for version 1.2.1 hadoop-core-1.2.1.jar) to %ESP_HOME%\adapters\framework\libj. Ensure you use a stable version rather than a beta.

Use a forward slash for both UNIX and Windows paths.

Type: string

(Required) Specify the file you want the adapter to read or the regex pattern to filter the files on a given directory. See the DynamicMode element. No default value.

Type: string

Type: string

An example regex pattern is ".*\.txt", which selects only files that end with ".txt". In regex patterns, you must include an escape character, "\", before meta chars to include them in the pattern string.

Type: integer

(Advanced) Define the period, in seconds, to poll the specified file or directory. Set this element only if the value of the DynamicMode element is set to dynamicFile or dynamicPath.

The default value is 0, which, along with all other values less than 0, turns off polling.

Type: boolean

(Optional) If this property is set to true, the file is removed from the directory after the adapter processes it. This element takes effect if the value of the DynamicMode element is set to dynamicPath and ignored if it is set to dynamicFile instead.

The default value is false.

Type: integer

(Optional) Specify the depth of the schema discovery. The adapter reads the number of rows specified by this element value when discovering the input data schema.

The default value is three.

(Required) Element containing all information for this module. It contains a type attribute for specifying the module type.

For example, formatter.

Type: string

(Required) Instance name of the specific module you want to use. For example, MyInputTransporter.

Type: string

(Required) Name of the module as defined in the modulesdefine.xml file. For example, <TransporterType>InputTransporter.

Type: string

(Required) Instance name of the module that follows this one.

Type: integer

(Advanced) Capacity of the buffer queue between this module and the next. The default value is 10240.

Type: boolean

(Optional) If set to true, the module runs as a separated thread. The default value is true.

(Required) Element containing the XmlDocStreamToEspFormatterParameters element.

(Required) Element containing the XMLDoc Stream to ESP formatter elements.

Type: string

(Required) Specify a pattern to indicate which XML elements in the XML doc are processed by the formatter. The matched elements are mapped to ESP rows whose attributes and child elements are mapped as columns of an ESP row. The adapter ignores any XML elements that do not match this pattern.

This pattern is a subset of XPath expressions. The [/]?NCName[/NCName]* path expression is the only supported expression, where NCName (Non-Colonized Name) is the local name element without a prefix or namespace.

If the elements in the path expression include a namespace URI (prefix), all of these elements belong to the same namespace. Provide the namespace in the XmlElemNamespace element.

Here are some examples of valid path expressions:

• /RootElement
• ParentElement
• ParentElement/ChildElement
• /RootElement/ParentElement

Type: string

(Required) Specify the namespace URI for elements that appear in the pattern path expression.

(Required) Element containing the Column element.

Type: string

(Required) Specify which attributes or child elements of the XML elements, which are matched by pattern path expression, to map to columns of the ESP row. For example, [<Column>XPath expression</Column>]+.

The XPath expression is any valid XPath expression specified by an XPath specification. The XPath expression can begin only from the last XML element that appears in the path pattern expression or its decedent elements.

The first <Column/> element value is mapped to the first column of an ESP row, the second <Column/> element value is mapped to the second column of an ESP row, and so on.

Type: string

(Optional) Format string for parsing date values. For example, yyyy-MM-dd'T'HH:mm:ss.

Type: string

(Optional) Format string for parsing timestamp values. For example, yyyy-MM-dd'T'HH:mm:ss.SSS.

(Required) Element containing all information for this module. It contains a type attribute for specifying the module type.

For example, formatter.

Type: string

(Required) Instance name of the specific module you want to use. For example, MyInputTransporter.

Type: string

(Required) Name of the module as defined in the modulesdefine.xml file. For example, <TransporterType>InputTransporter.

Type: integer

(Advanced) Capacity of the buffer queue between this module and the next. The default value is 10240.

(Required) Element containing the EspPublisherParameters element.

(Required) Element containing elements for the ESP publisher.

Type: string

(Required if adapter is running in standalone mode; optional if it is running in managed mode) Name of the ESP project to which the adapter is connected. For example, EspProject2.

This is the same project tag that you specify later in the adapter configuration file in the Name element within the Event Stream Processor (EspProjects) element.

If you are starting the adapter with the ESP project to which it is attached (that is, running the adapter in managed mode), you need not set this element as the adapter automatically detects the project name.

Type: string

(Required if adapter is running in standalone mode; optional if it is running in managed mode) Name of the ESP stream to which the adapter publishes data.

If you are starting the adapter with the ESP project to which it is attached (that is, running the adapter in managed mode), you need not set this element as the adapter automatically detects the stream name.

Type: positive integer

(Optional) Maximum size of the record pool. Record pooling, also referred to as block or batch publishing, allows for faster publication since there is less overall resource cost in publishing multiple records together, compared to publishing records individually.

Record pooling is disabled if this value is 1. The default value is 256.

Type: positive integer

(Optional) Maximum period of time, in milliseconds, for which records are pooled before being published. If not set, pooling time is unlimited and the pooling strategy is governed by maxPubPoolSize. No default value.

Type: boolean

(Optional) If set to true, pooled messages are published to Event Stream Processor in transactions. If set to false, they are published in envelopes. The default value is false.

Type: boolean

(Advanced) Converts the opcodes INSERT and UPDATE to UPSERT, and converts DELETE to SAFEDELETE. The default value is false.

Type: boolean

(Advanced) Skips the rows with opcodes DELETE or SAFEDELETE. The default value is false.