Configuring a New Adapter

Configure a new adapter by creating a configuration file for that adapter. The configuration file defines the adapter component chain through which data is processed, as well as the connection to Event Stream Processor.

Prerequisites

Create any custom transporters and formatters that you wish to use in this adapter instance.

Task

Create an <Adapter> element and include all the elements from the steps below within this element.
Add a <Name> element and specify a name for the adapter instance.
Add a <Description> element and specify the purpose of the adapter.
Add a <Modules> element that will contain all of the modules for your adapter instance.

For each module, specify:

Parameter	Description
InstanceName	Type: `string` (Required) Specify the instance name of the specific module you wish to use. For example, MyInputTransporter.
Name	Type: `string` (Required) The name of the module as defined in the modulesdefine.xml file. This should be a unique name. For example, MyCustomInputTransporter.
Next	Type: `string` (Required if another module follows this one) Specify the instance name of the module that follows this one.
BufferMaxSize	Type: `integer` (Advanced) Specify the capacity of the buffer queue between this module and the next. The default value is 10240.
Parallel	Type: `boolean` (Optional; applies to row-based formatters only) If set to true, the module runs as a separated thread. If set to false, the module shares thread with other modules. The default value is true.
Parameters	(Required) Specify parameters for the current module. For a custom module, the sub-element can reflect the name or type of the module, for example <MyCustomInputTransporterParameters>. The EspPublisher, EspMultiStreamPublisher, EspSubscriber, and EspMultiStreamSubscriber all have set parameters that need to be configured specified.

Configure one of these modules for your adapter:

For the EspPublisher, add a <EspPublisherParameters> sub-element and specify:

Parameter	Description
ProjectName	Type: `string` (Required if running adapter in standalone mode; optional if running in managed mode) Name of the ESP project to which the adapter is connected. For example, EspProject2. This is the same project tag that you specify later in the adapter configuration file in the Name parameter of the Event Stream Processor (EspProjects) parameters section. If you are starting the adapter with the ESP project to which it is attached (running the adapter in managed mode), you do not need to set this property as the adapter automatically detects the project name.
EspPublisherParameters	(Required) Section containing parameters for the ESP publisher.
StreamName	Type: `string` (Required if running adapter in standalone mode; optional if running in managed mode) Name of the ESP stream to which the adapter publishes data. If you are starting the adapter with the ESP project to which it is attached (running the adapter in managed mode), you do not need to set this property as the adapter automatically detects the stream name.
MaxPubPoolSize	Type: `positive integer` (Optional) Specifies the maximum size of the record pool. Record pooling, also referred to as block or batch publishing, allows for faster publication since there is less overall resource cost in publishing multiple records together compared to publishing records individually. Block publishing (record pooling or batch publishing) is disabled if this value is set to 1. The default value is 256.
MaxPubPoolTime	Type: `positive integer` (Optional) Specifies the maximum period of time, in milliseconds, for which records are pooled before being published. If not set, pooling time is unlimited and the pooling strategy is governed by maxPubPoolSize. No default value.
UseTransactions	Type: `boolean` (Optional) If set to true, pooled messages are published to Event Stream Processor in transactions. If set to false, they are published in envelopes. Default value is false.
SafeOps	Type: `boolean` (Advanced) Converts the opcodes INSERT and UPDATE to UPSERT, and converts DELETE to SAFEDELETE. The default value is false.
SkipDels	Type: `boolean` (Advanced) Skips the rows with opcodes DELETE or SAFEDELETE. The default value is false.

For the EspMultiStreamPublisher, add a <EspMultiStreamPublisherParameters> sub-element and specify:

Parameter	Description
Streams	(Required) Section containing the Stream, ProjectName, StreamName, and ColumnMapping parameters.
Stream	(Required) Section containing details for the target project and streams to which the adapter is connected. Contains the ProjectName , StreamName, and ColumnMapping parameters. You can specify multiple Stream sections.
Filter	(Optional) Section containing the MatchString parameter.
MatchString	(Optional) Filters records with one or more column values. Contains a value attribute for specifying the value you wish to filter by. No default value.
ProjectName	Type: `string` (Required) The name of the ESP project to which the adapter belongs. The same project as specified in the Name parameter of the ESP projects section of the adapter configuration file.
StreamName	Type: `string` (Required) The name of the target stream to which the adapter connects.
ColumnMapping	Type: `string` (Optional) The column index list in the source row to be published to the target stream. Separate the columns using space char. If you do not set this parameter, all columns are published to the target stream.

For the EspSubscriber, add a <EspSubscriberParameters> sub-element and specify:

Parameter	Description
ProjectName	Type: `string` (Required if running adapter in standalone mode; optional if running in managed mode) Specifies the unique project tag of the ESP project to which the adapter is connected. For example, EspProject2. This is the same project tag that you specify later in the adapter configuration file in the Name parameter of the Event Stream Processor (EspProjects) parameters section. If you are starting the adapter with the ESP project to which it is attached (running the adapter in managed mode), you do not need to set this property as the adapter automatically detects the project name.
StreamName	Type: `string` (Required if running adapter in standalone mode; optional if running in managed mode) Name of the ESP stream from which the adapter subscribes to data. If you are starting the adapter with the ESP project to which it is attached (running the adapter in managed mode), you do not need to set this property as the adapter automatically detects the stream name.

Parameter

Description

ProjectName

Type: string

(Required if running adapter in standalone mode; optional if running in managed mode) Specifies the unique project tag of the ESP project to which the adapter is connected. For example, EspProject2.

This is the same project tag that you specify later in the adapter configuration file in the Name parameter of the Event Stream Processor (EspProjects) parameters section.

If you are starting the adapter with the ESP project to which it is attached (running the adapter in managed mode), you do not need to set this property as the adapter automatically detects the project name.

StreamName

Type: string

(Required if running adapter in standalone mode; optional if running in managed mode) Name of the ESP stream from which the adapter subscribes to data.

For the EspMultiStreamSubscriber, add a <EspMultiStreamSubscriberParameters> sub-element and specify:

Parameter	Description
Streams	(Required) Section containing the Stream, ProjectName, StreamName, and ColumnMapping parameters.
Stream	(Required) Section containing details for the target project and streams to which the adapter is connected. Contains the ProjectName , StreamName, and ColumnMapping parameters. You can specify multiple Stream elements.
ProjectName	Type: `string` (Required) The name of the ESP project to which the adapter belongs. The same project as specified in the Name parameter of the ESP projects section of the adapter configuration file.
StreamName	Type: `string` (Required) The name of the source stream to which the adapter connects.
ColumnMapping	Type: `string` (Optional) The column list of the source stream to which the adapter connects. Separate the columns with a space char. This parameter contains the "enumtype" attribute which has two valid values: index – the index of the column name – the column name The default value is name.

Specify a connection to Event Stream Processor:

Parameter	Description
EspProjects	(Required) Section containing parameters for connecting to Event Stream Processor.
EspProject	(Required) Section containing the Name and Uri parameters. Specifies information for the ESP project to which the adapter is connected.
Name	Type: `string` (Required) Specifies the unique project tag of the ESP project which the EspConnector (publisher/subscriber) module references.
Uri	Type: `string` (Required) Specifies the total project URI to connect to the ESP project. For example, esp://localhost:19011/ws1/p1.
Security	(Required) Section containing all the authentication parameters below. Specifies details for the authentication method used for Event Stream Processor.
User	Type: `string` (Required) Specifies the user name required to log in to Event Stream Processor (see AuthType). No default value.
Password	Type: `string` (Required) Specifies the password required to log in to Event Stream Processor (see espAuthType). Includes an "encrypted" attribute indicating whether the Password value is encrypted. Default value is false. If set to true, the password value is decrypted using RSAKeyStore and RSAKeyStorePassword.
AuthType	Type: `string` (Required) Specifies method used to authenticate to the Event Stream Processor. Valid values are: server_rsa – RSA authentication using keystore kerberos – Kerberos authentication using ticket-based authentication user_password – LDAP, SAP BI, and Native OS (user name/password) authentication If the adapter is operated as a Studio plug-in, AuthType is overridden by the Authentication Mode Studio start-up parameter.
RSAKeyStore	Type: `string` (Dependent required) Specifies the location of the RSA keystore, and decrypts the password value. Required if AuthType is set to server_rsa, or the encrypted attribute for Password is set to true, or both.
RSAKeyStorePassword	Type:`string` (Dependent required) Specifies the keystore password, and decrypts the password value. Required if AuthType is set to server_rsa, or the encrypted attribute for Password is set to true, or both.
KerberosKDC	Type: `string` (Dependent required) Specifies host name of Kerberos key distribution center. Required if AuthType is set to kerberos.
KerberosRealm	Type: `string` (Dependent required) Specifies the Kerberos realm setting. Required if AuthType is set to kerberos.
KerberosService	Type: `string` (Dependent required) Specifies the Kerberos principal name that identifies an Event Stream Processor cluster. Required if AuthType is set to kerberos.
KerberosTicketCache	Type: `string` (Dependent required) Specifies the location of the Kerberos ticket cache file. Required if AuthType is set to kerberos.
EncryptionAlgorithm	Type: `string` (Optional) Used when the encrypted attribute for Password is set to true. If left blank, RSA is used as default.

(Optional) Add a GlobalParameters element. This node is visible to all modules configured within the adapter configuration file.
1. Define XML schema for the GlobalParameter in the $ESP_HOME/adapters/framework/parametersdefine.xsd file.
2. Call the Utility.getGlobalParameterNode() or utility.getGlobalParameters() function to get the XML object for this node.

Here is an example of a configuration file for the Socket JSON Input adapter:

<?xml version="1.0" encoding="utf-8"?>
<Adapter>
	<Name>socket_json_input</Name>
	<Description>An adapter which receives JSON message from socket server,transforms to ESP data format, and publishes to ESP stream.</Description>
	<Log4jProperty>./log4j.properties</Log4jProperty>
	<Modules>
		<Module type="transporter">
			<InstanceName>MyExampleSocketInTransporter</InstanceName>
			<Name>SocketInputTransporter</Name>
			<Next>MyJsonStreamToJsonStringFormatter</Next>
			<Parameters>
				<SocketInputTransporterParameters>
					<Host>localhost</Host>
					<Port>9998</Port>
					<EpFile></EpFile>
					<Retryperiod>60</Retryperiod>
					<Blocksize>512</Blocksize>
					<KeepAlive>true</KeepAlive>
				</SocketInputTransporterParameters>
			</Parameters>
		</Module>

		<Module type="formatter">
			<InstanceName>MyJsonStreamToJsonStringFormatter</InstanceName>
			<Name>JsonStreamToJsonStringFormatter</Name>
			<Next>MyJsonInFormatter</Next>
			<Parameters />
		</Module>

		<Module type="formatter">
			<InstanceName>MyJsonInFormatter</InstanceName>
			<Name>JsonStringToEspFormatter</Name>
			<Next>MyInStream_Publisher</Next>
			<Parameters>
				<JsonStringToEspFormatterParameters>
					<DateFormat>yyyy-MM-dd HH:mm:ss.SSS</DateFormat>
					<TimestampFormat>yyyy/MM/dd HH:mm:ss</TimestampFormat>
					<ColumnMappings>
						<ColsMapping streamname="EntityStream" rootpath="entities">
							<Column>display_text</Column>
							<Column>domain_role</Column>
							<Column>offset</Column>
							<Column>length</Column>
						</ColsMapping>
					</ColumnMappings>
				</JsonStringToEspFormatterParameters>
			</Parameters>
		</Module>

		<Module type="espconnector">
			<InstanceName>MyInStream_Publisher</InstanceName>
			<Name>EspPublisher</Name>
			<Parameters>
				<EspPublisherParameters>
					<ProjectName>EspProject1</ProjectName>
					<StreamName>EntityStream</StreamName>
					<MaxPubPoolSize>1</MaxPubPoolSize>
					<UseTransactions>false</UseTransactions>
					<SafeOps>true</SafeOps>
					<SkipDels>true</SkipDels>
				</EspPublisherParameters>
			</Parameters>
		</Module>

	</Modules>

	<EspProjects>
		<EspProject>
			<Name>EspProject1</Name>
			<Uri>esp://localhost:19011/sample_workspace/socket_json_input</Uri>
			<Security>
				<User></User>
				<Password encrypted="false"></Password>
				<AuthType>user_password</AuthType>
				<!-- <RSAKeyStore>/keystore/keystore.jks</RSAKeyStore> <RSAKeyStorePassword>Sybase123</RSAKeyStorePassword> -->
				<!-- <KerberosKDC>KDC</KerberosKDC> <KerberosRealm>REALM</KerberosRealm> 
					<KerberosService>service/instance</KerberosService> <KerberosTicketCache>/tmp/krb5cc_user</KerberosTicketCache> -->
				<EncryptionAlgorithm>RSA</EncryptionAlgorithm>
			</Security>
		</EspProject>
	</EspProjects>
	<GlobalParameters></GlobalParameters>
</Adapter>