Best Practices for Working with ETL Project Components

Review the best practices for working with ETL project components.

• Migrating Wide Tables
  Migrating tables with hundreds or thousands of columns consumes a lot of memory. To prevent errors while migrating wide tables with varying numbers of columns and rows from a source Sybase IQ database to a target Sybase IQ database, follow the recommendations described in this section.
• Importing an XML File with More than 32 Sibling Elements
  To import more than 32 sibling elements using the XML via SQL Data Provider component, set the Create Flat View to 0 in the XML Options property of the component. You must manually set up sub queries using the Content Explorer.
• Loading Last Row of Source Text File to Sybase IQ
  When you are using the IQ Loader File via Load Table component, Sybase IQ does not accept the last row of a source text file from Sybase IQ InfoPrimer if the last row does not end with a trailing row delimiter.
• Configuring Adaptive Server Enterprise for Bulk Copying
  If you are using Adaptive Server Enterprise for DB Staging, you must first configure the Adaptive Server database for bulk copying. If the Adaptive Server is not configured, you may encounter errors, although the project executes successfully.
• Data Calculator JavaScript and DB Staging Components Limitation
  Do not add more than 35 Data Calculator JavaScript and DB Staging components to a single project.
• Increasing the Text Size for the Adaptive Server ODBC Driver
  The Adaptive Server ODBC driver truncates text or image data values that are larger than the value set in the ODBC configuration in Microsoft Windows for the driver. You must increase the text size value in the ODBC Control Panel, or set the value in the database options parameter for your database connection to avoid this issue.
• Do Not Change Delimiters in the Source Text File When Project Is Executed on Different Platforms
  When you create a project using the Text Data Provider component on Windows and then execute it on UNIX or Linux, make sure the source text file is not converted to the UNIX or Linux format. The delimiters used in the source text file should always be same as what was designed in the Text Data Provider component.
• Setting Named Pipe Permission on Windows
  For the DB Bulk Load Sybase IQ component, if you want to provide a pipe name in the Load Stage property field, you must first make some settings to avoid any errors.
• Migrating Tables to IQ Containing LOB Columns
  Before migrating tables to IQ that contain character large object (CLOB), binary large object (BLOB), image or text columns with the IQ Loader DB via Insert Location component, review your IQ configuration.
• Simulation Does Not Roll Back Data When a PostProcessing SQL Statement Error Occurs
  Do not select the “Execute post-processing as for successful execution” option, in case a postprocessing SQL statement error occurs during execution.