Home

Chapter 1 Welcome to RTI Database Integration Service

1. PRIMARY KEY table owner table name domain id topic name Users should use the same SQL statement in their own applications if they want to create and populate this table before the Database Integration Service Daemon is started Table 4 18 describes how each column is used by the daemon in creating DataReaders and storing received data into tables Table 4 18 RTIDDS SUBSCRIPTIONS Table Schema Column Name SQL Type Nr Default if NULL Described in table_owner oe 128 No N A Section 4 5 2 1 23 table name Lae 128 No N A Section 4 5 2 1 23 1 See Starting the MySQL Server in ANSI QUOTES mode Section 4 1 2 4 4 42 Meta Tables Table 4 18 RTIDDS_SUBSCRIPTIONS Table Schema Column Name SQL Type Nuil Default if NULL Described in domain id INTEGER No N A Section 4 5 2 1 24 topic name E YES table owner table name Section 4 5 2 1 25 200 type name boe 200 YES topic name Section 4 5 2 1 26 table history depth INTEGER YES 0 Section 4 5 2 1 27 process_batch INTEGER YES 10 Section 4 5 2 1 28 process_period sec INTEGER YES 0 Section 4 5 2 1 28 process_period nsec INTEGER YES 100000000 Section 4 5 2 1 28 commit_type VARCHAR 17 YES COMMIT ON PROCESS Section 4 5 2 1 28 cache maximum size INTEGER YES 0 Section 4 5 2 1 29 cache ini
2. Usage rtirtc_mysql Options cfgFile lt file gt cfgName name appName name logFile file noDaemon use42eAlignment queueLibPath path queueDomainId int dbTransport lt 1 2 gt typeMode verbosity version help lt 0 1 gt 0 6 options Configuration file This parameter is optional since the configuration can be loaded from other locations Configuration name This parameter is required and it is used to find a lt real_time_connect gt matching tag in the configuration files Application name Used to name the domain participants Default cfgName Log file Run as a regular process Messages are sent to stdout and stderr Enables compatibility with RTI Data Distribution Service 4 2e This option should be used when compatibility with 4 2e is required and the topic data types contain double long long unsigned long long or long double members Location of the library rtirtc oracleq Default SORACLE HOME bin Domain ID of the channel connecting the MySQL server with RTI Database Integration Service Default 1 Transport used to communicate the MySQL server with RTI Database Integration Service 1 UDPv4 2 Shared Memory Default 2 Shared memory Type mode Specifies whether the names and semantics of the data types follow Oracle or TimesTen type rules Default 0 Oracle type mode RTI Database Integration Service verbosity silent exceptions Core Libraries and Serv
3. domain id 0 domain id distributed logger gt lt enabledstrue lt enableds lt distributed_logger gt lt administration gt lt general_options gt lt real_time_connect gt Replace the value of lt domain_id gt with the domain ID that Database Integration Service will use to send log messages when Distributed Logger is enabled There are more configuration tags that you can use to control Distributed Logger s behavior For example you can specify a filter so that only certain types of log messages are published For details see the Distributed Logger section of the RTI Connext DDS Core Libraries User s Manual 49 Enabling RTI Monitoring Library in Database Integration Service To enable monitoring of the Entities that are created by Database Integration Service you must specify the property rti monitor library in the QoS of the participants that you want to monitor For example participant qos property values element lt name gt rti monitor library lt name gt lt value gt rtimonitoring lt value gt lt propagate gt false lt propagate gt lt element gt lt value gt lt property gt participant qos 2 Connext DDS installation directory ndds version doc pdt RTI ConnextDDS CoreLibraries GettingStarted ExtensibleTypesAddendum pdf 1 RTI Monitor is a separate GUI application that can run on the same host as your application or on a different host 4 60 Ena
4. 1 Brian W Kernighan Dennis M Ritchie The C programming Language 2nd edition Pren tice Hall Software Series 1988 1 3 Chapter 2 Introduction 2 1 This chapter presents a conceptual view of Database Integration Service s architecture and high lights its unique features It includes the following sections L The Edge to Enterprise Integration Solution Section 2 1 1 Database Integration Service s Unique Features Section 2 2 The Edge to Enterprise Integration Solution Database Integration Service is a solution for integrating existing applications including service oriented architectures SOAs with high performance real time applications data and edge devices Database Integration Service provides a run time bridge between RTI s high throughput embeddable messaging infrastructure Connext DDS and integration and data management standards such as SQL XML Web services and JMS This allows developers to benefit from the performance scalability Quality of Service QoS control and broad platform support provided by RITs messaging technology while retaining interoperability with existing enterprise applica tions see Figure 2 1 Database Integration Service is a fully standards based solution for the integration of enterprise applications and high performance real time applications Enterprise applications typically use Structured Query Language SOL and Extensible Markup Language XML for data access and Jav
5. Oracle TimesTen with command line option typeMode 0 Create Table RTIDDS PUBLICATIONS table owner TT VARCHAR 128 NOT NULL table name TT VARCHAR 128 NOT NULL domain id TT INTEGER NOT NULL topic name TT VARCHAR 200 type name TT VARCHAR 200 table history depth TT INTEGER resolution column TT VARCHAR 255 idl member prefix max length TT INTEGER idl member suffix max length TT INTEGER profile name TT VARCHAR 255 pub present access scope TT VARCHAR 25 pub present ordered access TT TINYINT pub partition name TT VARCHAR 256 dw durability kind TT VARCHAR 30 dw liveliness lease dur sec TT INTEGER dw liveliness lease dur nsec TT INTEGER dw deadline period sec TT INTEGER dw deadline period nsec TT INTEGER dw history kind TT VARCHAR 21 dw history depth TT INTEGER dw ownership kind TT VARCHAR 23 dw ownership strength value TT INTEGER dw publish mode kind TT VARCHAR 29 dw res limits max samples TT INTEGER dw res limits max instances TT INTEGER PRIMARY KEY table owner table name domain id topic name Oracle Database 11g Create Table RTIDDS PUBLICATIONS table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL domain id NUMBER 10 NOT NULL topic name VARCHAR 200 type name VARCHAR 200 table history depth NUMBER 10 resolution column VARCHAR 255 idl member prefix max length NUMBER 10 idl member suffix max length NUMBER 10 profile
6. The Database Integration Service Daemon uses the Connext DDS API as well as SOL through the ODBC API In addition there is a custom interface for each supported database management system DBMS The three currently supported DBMSs are Oracle 11g Oracle TimesTen In 3 1 Database Integration Service Architecture Figure 3 1 Database Integration Service Architecture RTI Database Integration Service BPELProcess Daemon Manager Connext DDS 0 D 210 Cache Connect to Oracle Oracle 10g Database In Memory Database High performance Permanent storage Memory Database 11 2 1 and MySQL 5 1 There is a separate daemon executable for each of the specific DBMSs As illustrated in Figure 3 1 Oracle TimesTen In Memory Database and Oracle Database 11g can be used in combination In this case TimesTen acts as a front end cache to Oracle Database pro viding high performance access to real time data Database Integration Service s Unique Features Database Integration Service offers a unique set of features that enable seamless integration of real time embedded Connext DDS applications and enterprise services LJ Storage of Connext DDS Data in a DBMS Database Integration Service automatically stores received values of specified Topics in a database Once the data is propagated to the database it can be accessed by a user appli cation via regular SQL queries LJ Publication of DBMS Data via Connext
7. 5 13 Data Representation Mapping 5 2 10 NULL Values Null values exist in SQL databases but do not have an equivalent in IDL The Database Integra tion Service daemon converts NULL values into 0 values when publishing from a SQL table in the following way 1 numerical types 0 1 fixed length string types CHAR NCHAR LI variable length types VARCHAR NVARCHAR VARBINARY length 0 1 binary every byte is set to 0 timestamp 0 5 2 11 Sparse Data Types Sparse Data Types follow the same mapping as structures see Section 5 2 6 and Section 5 2 7 The fields that are not required or primary keys are created with the nullable attribute Table 5 16 Simple Sparse Type Type in Pseudo Language SQL Table Schema sparse MySparse Create Table MySparseContainer long my_key field key my key field INTEGER NOT NULL Short my short field my short field SMALLINT long my long field required my long field INTEGER NOT NULL y PRIMARY _KEY my key field T a Sparse types must be built dynamically There is no IDL construct sparse Appendix A Error Codes Table A 1 lists the native error and warning messages that may be logged by the Database Inte gration Service Daemon While some of these messages may actually provide enough informa tion by themselves to help users fix the problem many have to be used along with other data to help with debugging the issue Often several
8. C054 27 7 468 C054 J Smith Embedded Ll sl Enterprise Bridging between Domains Figure 3 5 shows how Database Integration Service can be used as a bridge between two domains by configuring the Database Integration Service Daemon to subscribe to data in one domain and publishing the same data in a different domain Data sent by Connext DDS applications in the first domain are stored by the daemon in a local in memory table Since changes in the table are sent by the daemon into a second domain the data is ultimately received by Connext DDS appli cations in the second domain There is no feedback cancellation needed since the data is being bridged across domains Usually domain bridges have to be written by users and modified whenever data types or Topics change Using Database Integration Service no programming is required to create a high performance bridge for any topic of any data type between any domains High Rate Data Streams Cached before Storage While disk based persistent databases can store terabytes of data the performance of such DBMSs is usually too low to capture data of real time applications streaming at ultra high rates of tens of thousands to over a million samples per second Figure 3 6 shows how Database Inte gration Service can use the Oracle TimesTen In Memory database as a front end cache to the per sistent Oracle database with Cache Connect to Oracle transferring table data from memory to disk in the bac
9. ENVS parameter in the file listener ora Refer to the Oracle Net manual and this link for more information on the listener ora file http download oracle com docs cd E11882 01 network 112 e10835 listener htm Refer to this link for more information on extproc ora http download oracle com docs cd E11882 01 appdev 112 e10471 adfns externproc htm 4 3 Introduction to the Database Integration Service Daemon 4 1 2 1 4 1 2 2 Windows Systems Using the dialog opened with Start Settings Control Panel System Advanced tab Environment Variables button add the directory with backslash V and semicolon sep arators containing the Connext DDS libraries to the System variable Path You will need to reboot the computer for this change to take effect A better method for setting this environment variable it only requires restarting the Ora cle database service and the Oracle listener service is using the extproc ora file or the ENVS parameter in the file listener ora Refer to the Oracle Net manual and this link for more information on the listener ora file http download oracle com docs cd E11882 01 network 112 e10835 listener htm Refer to this link for more information on extproc ora http download oracle com docs cd E11882 01 appdev 112 e10471 adfns externproc htm How to Run the Database Integration Service Daemon with MySQL Before Database Integration Service will run correctly with a MySQL data
10. INTEGER dw publish mode kind VARCHAR 29 dw res limits max samples INTEGER dw res limits max instances INTEGER changes queue maximum size INTEGER RTIRTC SCN BIGINT DEFAULT 0 PRIMARY KEY table owner table name domain id topic name Users should use the same SQL statement in their own applications if they want to create and populate this table before the Database Integration Service Daemon is started Table 4 17 describes how each column is used by the daemon in creating and using DataWriters that publish table changes Table 4 17 RTIDDS PUBLICATIONS Table Schema Column Name SQL Type Nav Default if NULL Described in table owner VARCHAR 128 No N A Section 4 5 1 1 4 table name VARCHAR 128 No N A Section 4 5 1 1 4 1 See Starting the MySQL Server in ANSI QUOTES mode Section 4 1 2 4 4 30 Meta Tables Table 4 17 RTIDDS PUBLICATIONS Table Schema Null Column Name SOL Type able Default if NULL Described in domain id INTEGER No N A Section 4 5 1 1 5 topic name VARCHAR 200 Yes table owner table name Section 4 5 1 1 6 type name VARCHAR 200 Yes topic name Section 4 5 1 1 7 table history depth INTEGER Yes JO Section 4 5 1 1 8 resolution column VARCHAR 255 Yes None Section 4 5 1 1 9 idl member prefix max length INTEGER Yes Value specified
11. 5 3 1 See the Connext DDS documentation for more details on how this QoS policy may be used dr multicast receive_port This column configures the multicast port on which the DataReader will receive data When the default value NULL or 0 is used the actual port number is determined by a formula as a func tion of the domain ID Note that the value of this field is ignored when dr multicast receive_address is NULL RTIRTC_SCN The System Change Number SCN column is available only for connections to a MySQL data base The value of this column is automatically maintained by Database Integration Service and is usually of no interest to the application For more information about the RTIRTC_SCN column see Section 4 6 Table Info The meta table RTIRTC_TBL_INFO stores meta information associated with the user tables When a table is automatically created by the Database Integration Service Daemon see Section 4 6 its TypeCode is stored in RTIRTC_TBL_INFO as a sequence of octets When the Database Integration Service Daemon is restarted the persisted TypeCodes corresponding to exist ing publications and subscriptions will be made available to other Connext DDS applications Table Info Schema The RTIRTC_TBL_INFO table is created with the following SQL statement Oracle TimesTen with command line option typeMode 1 Create Table RTIRTC_TBL INFO table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL type code VARBINAR
12. DDS Database Integration Service automatically publishes changes in specified database tables Changes made via the SQL API with the INSERT UPDATE and DELETE statements will be published into the network via Connext DDS so real time embedded applica tions and devices can respond to time critical changes with near zero latency Mapping Between IDL to SQL Data Types Database Integration Service provides automatic mapping between an IDL data type repre sentation and a SQL table schema representation This mapping is used to directly trans late a table record to a Connext DDS data structure and vice versa Previously this translation had to be done by custom developed code 3 2 Capturing Real Time Data in a DBMS 3 2 J History Database Integration Service can store a history of received values of a data instance Nor mally an instance of a topic is mapped to a single row in the associated database with the IDL key used as the primary key for the table But when Database Integration Service s data history feature is enabled multiple samples of a topic instance can be stored across multiple rows in the same table of the database supporting both real time and off line analysis based on historical data Configurable QoS Database Integration Service exposes many of the QoS attributes defined by the DDS stan dard This gives the user full control over the quality of service when capturing real time data or subscribing to cha
13. Data Representation Mapping In Connext DDS data is stored in data structures or classes defined using the Interface Definition Language IDL In relational databases data is stored in tables defined using SOL table sche mas While there is a good correspondence of IDL primitive data types to SOL data types this mapping is not one to one Both IDL and SQL have data types that the other does not define nor has an unambiguous mapping In addition many complex data structures in IDL such as unions and data structures that contain embedded data structures do not have equivalents in SOL This section describes the mapping used by the Database Integration Service Daemon when taking data received with DDS and storing it in tables or taking data from tables and publishing it with Connext DDS 1 IDL to SOL Mapping Section 5 2 1 on Page 5 3 Primitive Types Mapping Section 5 2 2 on Page 5 6 I Oracle In Memory Database Cache Mapping Section 5 2 3 on Page 5 9 Bit Field Mapping Section 5 2 4 on Page 5 9 I Enum Types Mapping Section 5 2 5 on Page 5 11 L Simple IDL Structures Section 5 2 6 on Page 5 11 L Complex IDL Structures Section 5 2 7 on Page 5 11 l Array Fields Section 5 2 8 on Page 5 12 1 Sequence Fields Section 5 2 9 on Page 5 13 LI NULL Values Section 5 2 10 on Page 5 14 1 Sparse Data Types Section 5 2 11 on Page 5 14 IDL to SQL Mapping Identifiers are used for the names of table columns in SOL and names of
14. SUBSCRIPTIONS 4 41 Meta Tables table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL domain id INTEGER NOT NULL topic name VARCHAR 200 type name VARCHAR 200 table history depth INTEGER process batch INTEGER process period sec INTEGER process period nsec INTEGER commit type VARCHAR 17 cache maximum size INTEGER cache initial size INTEGER delete on dispose INTEGER idl idl member prefix max length INTEGER member suffix max length INTEGER profile name VARCHAR 255 filter duplicates TINYINT ordered store TINYINT persist state TINYINT sub present access scope VARCHAR 25 sub present ordered access TINYINT sub partition name VARCHAR 256 dr dr n dr n dr dr dr durability kind VARCHAR 30 dr destination order kind VARCHAR 43 liveliness lease dur sec INTEGER dr dr reliability kind VARCHAR 27 liveliness lease dur nsec INTEGER deadline period sec INTEGER deadline period nsec INTEGER dr dr history kind VARCHAR 21 history depth INTEGER ownership kind VARCHAR 23 dr dr dr dr time_filter min_sep sec INTEGER time_filter min_sep nsec INTEGER res limits max samples INTEGER res limits max instances INTEGER unicast receive port INTEGER dr ar multicast receive_address VARCHAR 39 multicast receive_port INTEGER RTIRTC SCN BIGINT DEFAULT 0
15. Table Creation Section 4 6 This column is useful in the case that the user wants the Database Integration Service Daemon to both publish and subscribe to a Topic for the same user table The value set in lt table_history_depth gt will enable the daemon to create the user table correctly if the user wants to store more than a single value for an instance of the Topic in the table The possible values for lt table_history_depth gt column are NULL or 0 These values should be used if the user does not want to store more than a single value for an instance of a Topic in the table If the Database Integration Service Daemon creates the table it will not add any meta col umns for table history to the table schema I Any other value For any non zero value in this column the Database Integration Service Daemon will add meta columns for table history to the table schema when it creates the user table auto matically Meta Tables 4 5 1 1 9 4 5 1 1 10 A table s schema or definition cannot be changed to accommodate the table history meta columns after a table has been created So a non zero value for this column is useful if the user wants the table to be created with the ability to store historic values in support of entries in the RTIDDS_SUBSCRIPTIONS table that may be made later resolution_column This column is used to designate one of the columns of the user table for use as the timestamp when data changes are publ
16. VARCHAR 1024 NO N A Section 4 5 4 1 57 4 5 4 1 53 id This column stores a strictly incrementing integer for each log message that is generated by the daemon The largest value in the id column is the last message that was produced 4 56 User Table Creation 4 5 4 1 54 ts This column stores the system timestamp of when the log message was generated 4 5 4 1 55 type This column stores the kind of log message Possible values are ERROR WARNING STA TUS and SPECIAL SPECIAL messages are ones that are always printed independently of the log level 4 5 4 1 56 function line 4 5 4 1 57 4 6 These two columns contain the function name and line number of the Database Integration Service Daemon code where the message was generated It is useful only to support engineers at RTI code native_code message The code column contains the Database Integration Service error code that correspond to the mes sage This column will have NULL entries for messages of type STATUS The native_code column will contain the error code of any external APIs e g ODBC OS Con next DDS that the daemon has called and returned an error This column may have NULL entries Finally the message column will contain a statement with details on why the message was gen erated For a complete list of possible error codes and messages that can be generated by the Database Integration Service Daemon please s
17. Windows documents folderNrti workspaceNversionNexamples Note your Windows documents folder is described above You can specify that you do not want the examples copied to the workspace See the RTI Connext DDS Core Libraries Getting Started Guide for instructions Background Reading For information on distributed systems and databases L George Coulouris Jean Dollimore Tim Kindberg Distributed Systems Concepts and Design 3rd edition Addison Wesley 2000 J M Tamer Ozsu Patrick Valduriez Principles of Distributed Database Systems 2nd Edition Prentice Hall 1999 J Andrew S Tanenbaum Maarten van Steen Distributed Systems Principles and Paradigms 1st edition Prentice Hall 2002 For information on real time systems LI Qing Li Caroline Yao Real Time Concepts for Embedded Systems CMP Books 2003 I Doug Abbott Linux for Embedded and Real Time Applications Butterworth Heinemann 2002 1 David E Simon An Embedded Software Primer Addison Wesley 1999 For information on SQL I Joe Celko Joe Celko s SQL for Smarties Advanced SQL Programming expanded 2nd Edition Morgan Kaufmann 1999 1 Rick van der Lans Introduction to SQL Mastering the Relational Database Language 3rd edi tion Addison Wesley 1999 For information on ODBC I Microsoft Corporation Microsoft ODBC 2 0 Software Development Kit and Programmer s Reference Microsoft Press 1997 For information on the C programming language
18. a transaction commits and before it is sent on the network Figure 3 7 Replicating Tables Across Databases RTI Database Integration Service Daemon Table Tracks Table Tracks Fit Lat Long Fit Lat Long C129 345 102 C129 34 5 102 8 C129 A Johnson C054 27 7 46 8 C054 27 7 46 8 C054 J Smith Enterprise ponnnonnnnnnonnnnnnnnnnnnnnnannnnnnnnnnnniBe Enterprise 3 7 Real Time Database Replication If you need remote table initialization and application timestamp based conflict resolution you can enable this either by using the Database Integration Service configuration properties or by set ting individual QoS values This is described further in Configuration File Section 4 4 Even different DBMSs can be synchronized by the Database Integration Service Daemon with table changes in Oracle TimesTen In Memory databases propagated to corresponding tables in Oracle 11g or MySQL 5 1 databases and vice versa 3 8 Chapter 4 Using Database Integration Service This chapter provides detailed information on using the Database Integration Service Daemon to subscribe to and store data received as Topics into relational databases as well as to publish as Topic changes in relational database tables The contents of this chapter assume you have a working knowledge of Connext DDS and rela tional databases especially the MySQL Oracle and or Oracle TimesTen In Memory databases The chapter also assumes familiarity with
19. access column are J 0 default value if the column is NULL 1 For the best performance of the Database Integration Service Daemon you should set lt pub pres ent access_scope gt to TOPIC_PRESENTATION_QOS and lt pub present ordered_access gt to 1 This will require that the corresponding values in the DDS_PresentationQosPolicy of the Sub scriber in the receiving applications to be changed to those values as well See the Connext DDS documentation for more details on how this QoS policy may be used See also sub present access_scope sub present ordered_access Section 4 5 2 1 36 pub partition name For publishing table changes Database Integration Service creates a DataWriter per table The pub partitionname column maps directly to the DDS_PartitionQosPolicy of the DDS_PublisherQos used by the Publisher that is created with the DataWriter The DDS_PartitionQosPolicy introduces a logical partition concept inside the physical partition concept introduced by the domain ID A Publisher can communicate with a Subscriber only if they have some partition in common The value of the pub partition name column specifies a list of partitions separated by commas to which the Publisher belongs See the Connext DDS documentation for more details on how this QoS policy may be used See also sub partition name Section 4 5 2 1 37 Meta Tables 4 5 1 1 14 4 5 1 1 15 4 5 1 1 16 dw durability kind This column maps directly
20. actually stored for Oracle databases only e The str suffix is used to distinguish between w string lt x gt and sequence lt w char x gt 5 7 Data Representation Mapping Table 5 4 Table 5 5 f The 1d suffix is used to distinguish between octet x and long double Basic Types in IDL and SQL MySQL IDL Type IDL Field Name SOL Type Table Column Name char my field CHAR 1 my field char x my field CHAR x my field sequence lt char x gt my_field VARCHAR x my_field wchar my field NCHAR 1 my field wchar x P my field NCHAR x my field sequence wchar x my field NVARCHAR x my field octet my_field BINARY 1 my field octet x my field BINARY x my field sequence lt octet x gt my_field VARBINARY x my_field boolean my_field TINYINT my_field short my_field SMALLINT my field unsigned short my field SMALLINT my field unsigned my field INTEGER my field unsigned long my field INTEGER my field double my field DOUBLE my field float my field FLOAT my field string lt x gt my_field VARCHAR x my_field str d wstring lt x gt my_field NVARCHAR x my_field str d long long my_field BIGINT my_field unsigned long long my_field BIGINT my field long double my field BINARY 16 my field 1d unsigned long long my field DATETIME my
21. and DataReaders as well as corresponding user tables in the database The tables are just ordinary tables that users can create themselves before starting the Database Integration Service Daemon if so desired However if the user chooses to do so it is important that the tables be created with the exact tables schemas presented below otherwise the daemon may not work correctly If the daemon finds existing meta tables upon startup it will process every row in the tables as if they were newly inserted The meta tables RTIDDS_PUBLICATIONS and RTIDDS_SUBSCRIPTIONS can be populated using the lt publica tion gt and lt subscription gt tags in the configuration file see Section 4 4 4 4 or running Insert Update SQL statements There are two more meta tables created by the Database Integration Service Daemon L The meta table RTIRTC TBL INFO will store the typecode associated with the user tables created automatically by the RTC daemon see Section 4 5 3 L The meta table RTIRTC LOG will be created to store log messages generated by the Database Integration Service Daemon Use of this table is controlled by command line parameters and the connection discussed in Section 4 2 andSection 4 4 4 3 The following sections discuss the usage of these tables and describe the actions taken by the daemon when these tables are modified J Publications Table Section 4 5 1 1 Subscriptions Table Section 4 5 2 J Table Info Section 4 5 3 I Log Ta
22. and the columns idl member prefix max length and idl member suffix max length in the meta tables described in Section 4 5 1 1 10 tell the daemon the values to use The values defined in the meta table have precedence over the values defined in the configuration file Suffixes Suffixes are also needed for column names when multiple IDL primitive types map into the same SQL type Because there are more IDL primitive types than SQL primitive types a full mapping will result in the use of the same SQL type to hold more than one IDL type For exam ple an IDL long double has no equivalent in SQL Thus a SOL BINARY 16 does double duty and is used to store both an IDL long double as well as an IDL octet 16 If a long double could be treated the same as an octet 16 by the Database Integration Service Daemon then there would be no issue and no special name mapping would be needed How ever because the representation of a long double is Endianess dependent while an octet 16 is not the Database Integration Service Daemon must use the column name to decide whether or not a SOL BINARY 16 value needs to be byte swapped or not when converting to an IDL data type Since long double has no equivalent SQL type a ld must be appended to the name of a SOL BINARY 16 column that is used to store one Similarly a suffix of str is used to indicate that a SOL VARCHAR x stores IDL string which is a NULL terminated seq
23. available characters If a positive value n is provided for idl member suffix max length Database Integration Service will use the last n characters from the IDL member identifier to compose the associated column Meta Tables 4 5 2 1 32 4 5 2 1 33 4 5 2 1 34 4 5 2 1 35 name A value of 0 tells Database Integration Service to compose the column name using only the first characters of the identifiers as defined by the idl member prefix max length column A value of 1 instructs Database Integration Service to use all the available characters profile name This column specifies the name of the QoS Profile that Database Integration Service will use to cre ate the subscription The name must have the following format QoS profile library name gt lt QoS profile name gt See the RTI Connext DDS Core Libraries User s Manual for a complete description of QoS Profiles QoS values specified in the subscription table if they are not NULL take precedence over the same values specified in the QoS profile filler duplicates There are multiple scenarios in which Database Integration Service may receive duplicate samples see Chapter 11 Mechanisms for Achieving Information Durability and Persistence in the RTI Con next DDS Core Libraries User s Manual For example if RTI Persistence Service is used in the sys tem Database Integration Service could receive the same sample from the original writer and from RTI Persistence
24. creates a table from an IDL type that contains an array it will create the column names using a naming convention that prevents name collisions By default the daemon simply adds the suffix i where i is the array index of that element beginning at 0 for the first index The open bracket and close bracket characters can be configured using the tags in the configuration file lt open_bracket_char gt and lt close_bracket_char gt see page 4 19 Note this naming convention is not required for the Database Integration Service Daemon to serialize dese rialize IDL array fields Hou Note that array fields of type octet char and wchar are mapped into a single column ele ment of the corresponding SOL types BINARY x CHAR x and WCHAR x respectively Table 5 14 shows a mapping of an array field between IDL and SQL Table 5 14 Array Fields in IDL and SQL IDL Type SOL Table Schema struct MyArrayContainer CREATE TABLE MyArrayContainer long my key field Gkey my key field INTEGER NOT NULL short my arr field 2 my arr field 0 SMALLINT NOT NULL my arr field 1 SMALLINT NOT NULL PRIMARY KEY my key field Hg 5 2 2 Sequence Fields Sequences are basically variable sized arrays that have a maximum length and carry an addi tional integer that indicates the current size The mapping of IDL sequences to a table schema is similar to the array mapping with the following di
25. database will use 0 or 1 optional in the username of the UID who owns the Database Inte timesten connection gration Service Daemon process Table 4 10 Tags for MySQL Connections Additional Tags Number Allowed within Description of Tags mysql connection Allowed Provides an estimation of the maximum duration of a database trans action If a table change is not committed in the interval specified by this attribute it will not be published to Connext DDS Uncommitted table changes are stored in a per table queue The maxi mum size of that queue can be configured setting the value of the changes queue maximum size column in the transaction max RTIDDS PUBLICATIONS table see Section 4 5 1 1 21 oe duration gt If a change in the uncommitted changes queue has not been committed after transaction_max_duration milliseconds it will be discarded by the Database Integration Service Daemon With a value of 1 the Database Integration Service Daemon will not dis card changes into the uncommitted queue until they are committed Default 5000 Table 4 11 Tags for Oracle Connections Additional Tags Number Allowed within Description of Tags oracle connection Allowed Limits the size of BLOB columns for tables that do not have an associ ated TypeCode see Section 4 1 4 in the RTIRTC TBL INFO meta table see Section 4 5 3 blob default size If a user table has an associ
26. fields within an IDL structure SQL identifiers are a superset of IDL identifiers Because of that an IDL identifier can always be used as a SQL identifier However there are some SQL identifiers that cannot be used as IDL identifiers For example SOL allows special characters like to be part of an identifier whereas IDL does not There are two kind of SQL identifiers quoted identifiers and basic identifiers The quoted identifi ers can use any combination of characters Those identifiers need to be surrounded by double quotes when referenced The definition of a basic identifier changes depending on the database vendor In Oracle TimesTen a basic identifier can consist of any of letters A to Z decimal digits 0 to 9 or underscore _ The first character must be a letter In Oracle a basic identifier can consist of any of letters A to Z decimal digits 0 to 9 or underscore _ The first character must be a letter In MySQL a basic identifier can consist of any letters A to Z decimal digits 0 to 9 or underscore _ In Microsoft SQL Server a basic identifier can consist of any letters A to Z decimal digits 0 to 9 t or The first character may be a letter or If the daemon creates the user table it will use quoted strings for the identifiers of the table and column names only if they cannot be considered as basic identifiers according to the previous 5 3 Data Representatio
27. if running Configures a connection to a MySQL data rtirtc mysql base See Section 4 4 4 3 0 or more ignored if running other DBMS version of the daemon mysql connection 1 or more required if running Configures a connection to an Oracle data rtirtc oracle base See Section 4 4 4 3 0 or more ignored if running other DBMS version of the daemon oracle connection 1 or more required if running Configures a connection to an Oracle rtirtc timesten TimesTen database See Section 4 4 4 3 0 or more ignored if running other DBMS version of the daemon timesten connection 4 4 4 1 General Options Table 4 4 describes the general options these attributes are independent of any particular data base connection made by the Database Integration Service Daemon Table 4 4 General Options Tags Tags within Number B Description of Tags general options gt Allowed administration See Table 4 5 Administration Tags Oor1 Database Integration Service can be configured to perform real time TEN lazy database replication see Section 3 7 by setting this attribute lt enable_table_replication gt t Oor1 o true Default false This parameter controls the maximum number of objects per thread that Connext DDS can store If you run into problems related to the creation of Entities increasing this number may be max objects per thread necessary See the RTI Conne
28. lt table_history_depth gt lt type_name gt Note that there are columns in the tables RIIDDS PUBLICATIONS and RTIDDS_SUBSCRIPTIONS that don t have a corresponding tag inside lt publications gt and lt subscriptions gt Those columns represent configuration of QoS However you can configure the quality of service by using lt profile_name gt where you can refer to a QoS profile in your own XML configuration file or in any of the other QoS profile files loaded by the daemon see How to Load the XML Configuration Section 4 4 1 4 24 Configuration File Table 4 15 Table 4 16 4 4 4 5 Publications Tags ora Number Tags Allowed within publications Description P boat T Configures a publication by inserting or updating a row in the table 1 Or publication RTIDDS PUBLICATIONS See Table 4 16 on page 4 25 more Publication Tags T Number Tags Allowed within Description of Tags publication Allowed domain id zB names Inserts the tag value into the column with the same name in the table RTIDDS_PUBLICATIONS red require stable gwnes See Section 4 5 1 7 topic name idl member prefix max length lt idl member suffix Inserts the tag value into the column with the same name in the max length table RTIDDS PUBLICATIONS resolution column gt If the value is not specified NULL is inserted Dont lt profile_name g
29. my_field string lt x gt my_field varchar x my_field str wstring lt x gt my_field nvarchar x my field str long long my field bigint my field unsigned long long my field bigint my field long double my field binary 16 my_field ld unsigned long long b my_field datetime my_field a The str suffix is used to distinguish between w string lt x gt and sequence lt w char x gt b The datetime type is converted to from a 64 bit value expressed as microseconds in the Gregorian epoch Oracle In Memory Database Cache Mapping When Database Integration Service to TimesTen is used with Oracle In Memory Database Cache the definition of cache groups requires mapping the Oracle SQL types to TimesTen SQL types Table 5 6 describes the mapping supported by Database Integration Service MappingBetween Oracle and TimesTen types Oracle Type TimesTen Type TypeMode 0 CHAR x CHAR x VARCHAR2 x VARCHAR2 x NCHAR x NCHAR x NVARCHAR2 x NVARCHAR2 x RAW x VARBINARY x NUMBER 3 TT_TINYINT NUMBER 5 TT SMALLINT NUMBER 10 TT INTEGER BINARY DOUBLE BINARY FLOAT BINARY DOUBLE BINARY FLOAT NUMBER 20 TIMESTAMP TT BIGINT This mapping is not supported Bit Field Mapping IDL bit field type is an RTI extension that maps directly to C C bit fields but are stored in primitive types in Java with only the specified number of bits being
30. name VARCHAR 255 pub present access scope VARCHAR 25 pub present ordered access NUMBER 3 pub partition name VARCHAR 256 dw durability kind VARCHAR 30 dw liveliness lease dur sec NUMBER 10 dw liveliness lease dur nsec NUMBER 10 4 29 Meta Tables dw deadline period sec NUMBER 10 dw deadline period nsec NUMBER 10 dw history kind VARCHAR 21 dw history depth NUMBER 10 dw ownership kind VARCHAR 23 dw ownership strength value NUMBER 10 dw publish mode kind VARCHAR 29 dw res limits max_samples NUMBER 10 dw res limits max_instances NUMBER 10 PRIMARY KEY table owner table name domain id topic name MySQL Create Table RTIDDS PUBLICATIONS table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL domain id INTEGER NOT NULL topic name VARCHAR 200 type name VARCHAR 200 table history depth INTEGER resolution column VARCHAR 255 idl member prefix max length INTEGER idl member suffix max length INTEGER profile name VARCHAR 255 pub present access scope VARCHAR 25 pub present ordered access TINYINT pub partition name VARCHAR 256 dw durability kind VARCHAR 30 dw liveliness lease dur sec INTEGER dw liveliness lease dur nsec INTEGER dw deadline period sec INTEGER dw deadline period nsec INTEGER dw history kind VARCHAR 21 dw history depth INTEGER dw ownership kind VARCHAR 23 dw ownership strength value
31. table without a primary key lt table gt without primary key Database Integration Service to Oracle does not support tables without primary keys Non primitive IDL keys are not supported When Database Integration Service tries to create a table with complex keys it will report this error message Example with supported keys struct SupportedKeysSt string id str key long id long key 43 kaeo short id short Gkey Example with unsupported keys struct KeySt long id long struct NonSupportedKeysSt KeySt id st Gkey 44 Error creating subscriber state queue 45 Error updating subscription state 46 Error creating lt object gt A7 Path too long The path to the configuration file is too long The Database Integration Service Daemon 48 Error creating database publication cache had a problem creating the publication database cache The Database Integration Service Daemon 49 Error adding record to publication cache had a problem adding a new record to the publication database cache The maximum length of a column name in Database Integration Service is 30 characters To control the length of a column name use 50 Column name length exceeds maximum length the configuration tags idl member prefix max length and idl member suffix max length under database mapping options See Section 4 4 4 2 for additional details A 4 Table A 1 Database Integration Service Errors and Warnin
32. the name of the embedded structure field itself This naming convention is not required for serialization to work 5 11 Data Representation Mapping properly Just as long as the column types map to the types of the embedded structure then the Database Integration Service Daemon will properly handle the data irrelevant of the actual column name Table 5 12 shows the mapping of a complex structure between IDL and SQL Table 5 12 Nested Structures in IDL and SQL Table 5 13 5 2 8 IDL Type SOL Table Schema struct MyStruct Short my short field long my long field struct MyStructContainer long my key field Gkey MyStruct my struct field CREATE TABLE MyStruc my key field INTEGE tContainer R NOT NULL my struct field my short field SMALLINT NOT NULL my struct field my long field INTEGER NOT NULL PRIMARY KEY my key field E Union fields IDL unions are mapped by adding an extra column with the name _d to represent the discrim inator that is used to indicate which type is actually stored by the union These unions are also known as switched unions All of the individual union fields are mapped to corresponding columns in a table However only one of these columns will contain valid data as indicated by the discriminator column d If the Database Integration Service Daemon creates the table from an IDL containing an union it will generate t
33. to new DataReaders with their DDS_DurabilityQosPolicy set to TRANSIENT LOCAL DURABILITY QOS See the Connext DDS documentation for more details on how this QoS policy may be used See also dr durability kind Section 4 5 2 1 38 and dr reliability kind Section 4 5 2 1 39 dw liveliness lease dur These columns specify the lease duration for the DDS LivelinessQosPolicy for the DataWriter created to publish table changes The user may need to change the lease duration if remote applications have modified their DataReaders corresponding DDS LivelinessQosPolicy to non default values The possible values of the lt dw liveliness lease_dur sec gt seconds and dw iveli ness lease dur nsec nanoseconds columns are An infinite lease duration is specified if both columns are NULL or contain the value 2147483647 2 1 This is the DDS default value IJ A non zero value representing the number of seconds and nanoseconds for the lease duration Note DDS LivelinessQosPolicy kind is always set to DD5 AUTOMATIC LIVELINESS QOS See the Connext DDS documentation for more details on how this QoS policy may be used See also dr liveliness lease dur Section 4 5 2 1 41 dw deadline period These columns specify the deadline period for the DDS DeadlineQosPolicy for the DataWriter created to publish table changes The user may need to change the deadline period if remote Meta Tables 4 5 1 1 17 applications have modified their D
34. to the DDS_DurabilityQosPolicy of the DataWriter created to publish table changes By changing this policy the Database Integration Service Daemon can be configured to resend past changes to the database table to remote applications as soon as their subscriptions are discovered Only changes made by local applications to the table will be sent That is if the daemon is con figured to subscribe to and store changes into the table from remote DataWriters those changes are not sent In addition the changes will only be sent if the DataReader is created with a reliable setting for its DDS_ReliabilityQosPolicy The number of past changes that will be sent is limited by the values of the lt dw history kind gt lt dw history depth gt and lt dw res_limits max_samples gt columns The possible values for the lt dw durability kind gt column are 11 VOLATILE DURABILITY QOS This value prevents the daemon from sending past changes to the table to newly discov ered DataReaders This also the default value if the column is NULL LI TRANSIENT LOCAL DURABILITY QOS This value will enable the daemon to send past changes to the table to newly discovered DataReaders The DataReaders must be created with reliable DDS_ReliabilityQosPolicy NOTE If a table exists when the Database Integration Service Daemon creates a DataWriter the daemon will initialize the DataWriter with the current contents of the table such that those val ues will be sent
35. user consoles must have concurrent access to both raw and fused data Database Integration Service is used to store the raw data at high rates into an Oracle TimesTen In Memory database where Oracle Cache Connect is then used to prop agate the data into an Oracle 11g database for permanent storage Enterprise applications used for analysis or applying other business logic can access the data stored in Oracle using SOL or other standards such as JMS XML and HTTP via a BPEL process manager 22 Database Integration Service s Unique Features Figure 2 2 Example System Using Database Integration Service 2 2 1 2 2 2 Sensor1 Sensor2 Business System 1 tracks tracks Data Fusion om L5 Database Summary and Statistics L Connext DDSApp Co RDBMSApp Console2 RTI Database Integration HH service Daemon Oracle TimesTen 5 In Memory database Oracle BPEL Process Manager Database Integration Service is the bridge that connects real time high performance to complex analysis edge devices to business systems and embedded to enterprise Interconnecting Standards Until recently distributed real time systems were built using custom developed data structures and algorithms to store and manipulate data in combination with a commercial or even propri etary data distribution middleware layer This was necessary to meet real time performance requirements However in recent years DDS a standard
36. 0 type name VARCHAR 200 table history depth INTEGER process batch INTEGER process period sec INTEGER process period nsec INTEGER commit type VARCHAR 17 cache maximum size INTEGER cache initial size INTEGER delete on dispose INTEGER idl member prefix max length INTEGER 4 39 Meta Tables idl member suffix max length INTEGER profile name VARCHAR 255 filter duplicates TINYINT ordered store TINYINT persist state TINYINT sub present access scope VARCHAR 25 sub present ordered access TINYINT sub partition name VARCHAR 256 dr durability kind VARCHAR 30 dr reliability kind VARCHAR 27 dr destination_order kind VARCHAR 43 dr liveliness lease_dur sec INTEGER dr liveliness lease_dur nsec INTEGER dr deadline period sec INTEGER dr deadline period nsec INTEGER dr history kind VARCHAR 21 dr history depth INTEGER dr ownership kind VARCHAR 23 dr time filter min sep sec INTEGER dr time filter min sep nsec INTEGER dr res limits max samples INTEGER dr res limits max instances INTEGER dr unicast receive port INTEGER dr multicast receive address VARCHAR 39 dr multicast receive port INTEGER PRIMARY KEY table owner table name domain id topic name Oracle TimesTen with command line option typeMode 0 Create Table RTIDDS SUBSCRIPTIONS table owner TT VARCHAR 128 NOT NULL table name TT VARCHAR 128 NOT NULL domain id TT INTEGER NO
37. 2 3 4 Welcome to RTI Database Integration Service 1 1 1 1 Intended Readers ie etin fee i 1 1 1 2 Paths Mentioned in Documentation sess eene eene ether nnne tinet nnns 1 1 1 3 Background Readitig desint apre te nee een us 1 3 Introduction petSeMe NE UE 2 1 2 1 The Edge to Enterprise Integration Solution sse 2 1 22 Database Integration Service s Unique Features sss 2 2 22 1 Interconnecting Standards sess eene nene 2 3 2 2 2 Connectivity To Edge Devices essri n eee eene nnn 2 3 2 23 Flexibility and Scalability sssrinin seias 2 4 2 24 Matching Real Time Performance sss eee eee e nnns 2 4 225 High Availability ette aee tente be eet ette 2 4 2 2 6 Additional Benefits of Database Integration Service sess 2 4 rege EE 3 1 3 1 Database Integration Service Architecture sse 3 1 3 1 1 Database Integration Service Daemon sese eee enne 3 1 3 1 2 Database Integration Service s Unique Features sese 3 2 3 2 Capturing Real Time Data in a DBMS ssssssssseeeeeee ee eene eene eene 3 3 3 3 Remote Real Time Notification of Table Changes sse 3 4 3 4 Bidirectional Integration used etie ee E Re RAE a ai E 3 4 3 5 Bridging between DomaiNS eieo nioan aeea nene enne nennen nne 3 5 3 6 High Rate Data Streams Cached before Stor
38. 50 Error giving semaphore 1051 Error creating worker factory 1052 Error creating worker 1053 Error creating clock 1054 Error creating event manager 1055 Error creating timer 1056 Error posting event 1057 Error getting time 1058 Error creating semaphore 1059 Error loading DDS XML Profile 1060 Error getting TypeCode A 5 Table A 1 Database Integration Service Errors and Warnings Code Message Details 1061 Error cloning TypeCode 1062 Error parsing TypeCode 1063 Error creating TypeCode 2048 4095 ODBC related errors These message are produced through the interaction of the Database Integration Service Daemon with the database through the ODBC driver More information on each error can be found by examining the native ODBC errors codes that will be logged with these messages lt message gt 2048 General ODBC error message lt message gt lt ODBC driver error message gt 4096 8191 DBMS Log Connection related errors These messages are produced through the interaction of the Database Integration Service Daemon with the Connext DDS More information on each error can be found by examining the native Connext DDS errors codes that will be logged with these messages General DBMS log connection error mes 4096 lt message gt sage 8192 16383 OS related errors These messages are produced through the interaction of the Database Integration Servic
39. IDL Interface Definition Language the DDS and SQL specifications and APIs Finally you should be able to create and run applications using Connext DDS to publish and subscribe to data as well as applications that can access Oracle databases using SQL through either ODBC or JDBC interfaces Users can configure the Database Integration Service Daemon to subscribe to Topics and store received values in a table or to publish database changes as Topics using a combination of meth ods L Command line parameters L Environment variables I Configuration file L Configuration tables in the database This chapter includes the following sections 1 Introduction to the Database Integration Service Daemon Section 4 1 J Command Line Parameters Section 4 2 J Environment Variables Section 4 3 1 Configuration File Section 4 4 J Meta Tables Section 4 5 J User Table Creation Section 4 6 1 Support for Extensible Types Section 4 7 L Enabling RTI Distributed Logger in Database Integration Service Section 4 8 L Enabling RTI Monitoring Library in Database Integration Service Section 4 9 4 1 Introduction to the Database Integration Service Daemon 4 1 Introduction to the Database Integration Service Daemon Database Integration Service bridges the world of Connext DDS and the world of relational data bases The main element of the bridge is an executable that must run on the same host as the database management s
40. INTEGER native code INTEGER message VARCHAR 2048 NOT NULL Oracle TimesTen with command line option typeMode 0 Create Table RTIRTC LOG id TT INTEGER NOT NULL ts TT TIMESTAMP NOT NULL type TT VARCHAR 7 NOT NULL function TT VARCHAR 64 NOT NULL line TT INTEGER code TT INTEGER native code TT INTEGER message TT VARCHAR 2048 NOT NULL Oracle Create Table RTIRTC LOG id NUMBER 10 NOT NULL ts TIMESTAMP NOT NULL type VARCHAR 7 NOT NULL function VARCHAR 64 NOT NULL line NUMBER 10 code NUMBER 10 native code NUMBER 10 message VARCHAR 2048 NOT NULL MySQL Create Table RTIRTC LOG id INTEGER NOT NULL ts TIMESTAMP NOT NULL type VARCHAR 7 NOT NULL function VARCHAR 64 NOT NULL line INTEGER code INTEGER native code INTEGER message VARCHAR 2048 NOT NULL Each column of the Log meta table stores a different portion of a log message generated by the Database Integration Service Daemon Table 4 20 describes these columns Table 4 20 RTIRTC LOG Table Schema Column Name SOL Type Nullable Default if NULL Described in id INTEGER NO N A Section 4 5 4 1 53 ts TIMESTAMP NO N A Section 4 5 4 1 54 type VARCHAR 7 NO N A Section 4 5 4 1 55 function VARCHAR 64 NO N A Section 4 5 4 1 56 line INTEGER NO None Section 4 5 4 1 56 code INTEGER YES None Section 4 5 4 1 57 native code INTEGER YES None Section 4 5 4 1 57 message
41. L Type Table Column Name char my_field x CHAR 1 my field x wchar my field x NCHAR 1 my_field x octet my_field x BINARY 1 my field x short my field x NUMBER 5 my field x unsigned short my field x NUMBER 5 my field x long my field x NUMBER 10 my field x unsigned long my field x NUMBER 10 my field x a The column storing the last bit field in a set of bits will use name of my_field x Table 5 9 Bit Fields in IDL and SQL MySQL IDL Type IDL Field Name SOL Type Table Column Name char my field x CHAR 1 my field x wchar my field x NCHAR 1 my field x octet my field x BINARY 1 my field x short my field x SMALLINT my field x unsigned short my field x SMALLINT my field x Data Representation Mapping Table 5 9 Table 5 10 2 5 5 2 6 Table 5 11 5 2 7 Bit Fields in IDL and SQL MySQL Continued IDL Type IDL Field Name SQL Type Table Column Name long my_field x INTEGER my_field x unsigned long my_field x INTEGER my_field x a The column storing the last bit field in a set of bits will use name of my_field x Bit Fields in IDL and SQL SQL Server IDL Type IDL Field Name SQL Type Table Column Name char my field x char 1 my field x wchar my field x nchar 1 my field x octet my field x binary 1 my field x short my field x smallint my fie
42. RIT Database Integration Service User s Manual Version 5 2 0 f t Your systems Working as one 2015 Real Time Innovations Inc All rights reserved Printed in U S A First printing June 2015 Trademarks Real Time Innovations RTI NDDS RTI Data Distribution Service DataBus Connext Micro DDS the RTI logo 1RTI and the phrase Your Systems Working as one are E trademarks trademarks or service marks of Real Time Innovations Inc All other trademarks belong to their respective owners Third Party Copyright Notices The Oracle TimesTen In Memory Database and the Oracle Database are products of Oracle Copy and Use Restrictions No part of this publication may be reproduced stored in a retrieval system or transmitted in any form including electronic mechanical photocopy and facsimile without the prior written permission of Real Time Innovations Inc All software and documentation whether in hard copy or electronic form enclosed are subject to the license agreement The software and documentation may be used or copied only under the terms of the license agreement The programs in this book have been included for their instructional value RTI does not offer any warranties or representations in respect of their fitness for a particular purpose nor does RTI accept any liability for any loss or damage arising from their use Technical Support Real Time Innovations Inc 232 E Java Drive Sunnyval
43. SENTATION QOS The possible values for the sub present ordered access column are I 0 default value if the column is NULL L1 For the best performance of the Database Integration Service Daemon you should set sub pres ent access scope to TOPIC PRESENTATION QOOS and sub present ordered access to 1 This will require that the corresponding values in the DDS_PresentationQosPolicy of the Pub lisher in the sending applications to be changed to those values as well See the Connext DDS documentation for more details on how this QoS policy may be used See also pub present access scope pub present ordered access Section 4 5 1 1 12 sub partition name For capturing data in a table Database Integration Service creates a DataReader per Topic The sub partition name column maps directly to the DDS_PartitionQosPolicy of the DDS SubscriberQos used by the Subscriber that is created with the DataReader The DDS PartitionQosPolicy introduces a logical partition concept inside the physical partition concept introduced by the domain ID A Subscriber can communicate with a Publisher only if they have some partition in common The value of the sub partition name column specifies a list of partitions separated by commas to which the Subscriber belongs See the Connext DDS documentation for more details on how this QoS policy may be used See also pub partition name Section 4 5 1 1 13 dr durability kind This column maps directl
44. Service The filter duplicates column specifies whether or not duplicates should be filtered by the Data base Integration Service Daemon If duplicates are not filtered the subscription data table may end up containing duplicates rows Note Durable Reader State configurations see Section 11 4 in the RTI Connext DDS Core Libraries User s Manual are ignored by Database Integration Service Duplicate filtering and subscription state persistence are implemented by the Database Integration Service Daemon ordered store This column specifies whether or not the samples associated with a DataWriter identified by a virtual GUID x see Chapter 11 Mechanisms for Achieving Information Durability and Persistence in the RTI Connext DDS Core Libraries User s Manual must be stored in the database in order The field only applies when filter duplicates Section 4 5 2 1 33 is set to 1 Ordered storage means that given a DataWriter with virtual GUID x a sample with virtual sequence number sn 1 will be stored after a virtual sample with virtual sequence number sn If there is only one DataWriter with virtual GUID x in the system for example if there are no RTI Persistence Services the value of this column does not affect behavior Note Database Integration Service stores samples in the database as soon as they are received by the Database Integration Service subscriptions Connext DDS DataReaders If ordered store is set to 1 and there are
45. T NULL topic name TT VARCHAR 200 type name TT VARCHAR 200 table history depth TT INTEGER process batch TT INTEGER process period sec TT INTEGER process period nsec TT INTEGER commit type TT VARCHAR 17 cache maximum size TT INTEGER cache initial size TT INTEGER delete on dispose TT INTEGER idl member prefix max length TT INTEGER idl member suffix max length TT INTEGER profile name TT VARCHAR 255 filter duplicates TT TINYINT ordered store TT TINYINT persist state TT TINYINT sub present access scope TT VARCHAR 25 sub present ordered access TT TINYINT Sub partition name TT VARCHAR 256 dr durability kind TT VARCHAR 30 dr reliability kind TT VARCHAR 27 dr destination order kind TT VARCHAR 43 dr liveliness lease dur sec TT INTEGER dr liveliness lease dur nsec TT INTEGER dr deadline period sec TT INTEGER dr deadline period nsec TT INTEGER 4 40 Meta Tables dr history kind TT VARCHAR 21 dr history depth TT INTEGER dr ownership kind TT VARCHAR 23 dr time filter min sep sec TT INTEGER dr time filter min sep nsec TT INTEGER dr res limits max samples TT INTEGER dr res limits max instances TT INTEGER dr unicast receive port TT INTEGER dr multicast receive address TT VARCHAR 39 dr multicast receive port TT INTEGER PRIMARY KEY table owner table name domain id topic name Oracle Database 11g Create Table RTIDDS SUBSCRIPTIONS
46. TC HISTORY ORDER These SOL INTEGER columns are used to implement the ability of the Database Integra tion Service Daemon to store multiple values historical of the same data instance into a table Usually a single data instance maps to a single row of a table As new values for the instance is received by the daemon the value of the same row is changed However users may use the table history depth columns see Sections 4 5 1 1 8 and 4 5 2 1 27 of the RTIDDS PUBLICATIONS and RIIDDS SUBSCRIPTIONS meta tables to direct the daemon to store multiple past values of a data instance These values Support for Extensible Types 4 7 are be stored in multiple rows of a table To support table history the daemon must add the meta columns lt RTIRTC_HISTORY_SLOT gt and lt RTIRTC_HISTORY_ORDER gt to a table They will only be added if the lt table_history_depth gt column for an entry is non NULL and has a non 0 value The RTIRTC HISTORY SLOT is an auto increment column that will also be added as a primary key column of the table The RTIRTC HISTORY ORDER is a column that will contain a number that is incre mented as data is stored into the table The oldest row of an instance will have the lowest value for this column whereas the most recent row of an instance will have the highest value RTIRTC SCN The System Change Number SCN meta column SOL BIGINT is only required for connections to a MySQL database The SCN m
47. UBSCRIPTIONS and or RTIDDS PUBLICATIONS tables By pre creating the database table used to store publish the data with the right schema Since the DataReaders created by the Database Integration Service Daemon have their TypeCon sistencyEnforcementQosPolicy s kind set to DISALLOW TYPE COERCION they will not match with DataWriters whose types are not equal to the DataReader s type You can learn more in the RTI Connext DDS Core Libraries Getting Started Guide Addendum for Extensible Types see the section on Rules for Type Consistency Enforcement 1 The OMG specification is available here http www omg org spec DDS XTypes Enabling RTI Distributed Logger in Database Integration Service 4 8 Enabling RTI Distributed Logger in Database Integration Service Database Integration Service provides integrated support for RTI Distributed Logger When you enable Distributed Logger Database Integration Service will publish its log messages to Connext DDS Then you can use RTI Monitor to visualize the log message data Since the data is provided in a Connext DDS topic you can also use rtiddsspy or even write your own visualiza tion tool To enable Distributed Logger modify the Database Integration Service XML configuration file In the general options administration section add the distributed logger tag as shown in the example below real time connect name default gt general options administration
48. Y 65000 PRIMARY KEY table owner table name Oracle TimesTen with command line option typeMode 0 Create Table RTIRTC_TBL INFO table owner TT VARCHAR 128 NOT NULL table name TT VARCHAR 128 NOT NULL type code VARBINARY 65000 PRIMARY KEY table owner table name Oracle Create Table RTIRTC TBL INFO table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL type code BLOB PRIMARY KEY table owner table name MySQL Create Table RTIRTC TBL INFO table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL type code VARBINARY 65000 4 54 Meta Tables Table 4 19 4 5 3 1 51 4 5 3 1 52 4 5 4 4 5 4 1 RTIRTC SCN BIGINT DEFAULT 0 PRIMARY KEY table owner table name Table 4 19 describes the meta table columns RTIRTC TBL INFO Table Schema Column Name SOL Type Nullable Default if NULL Described In table owner VARCHAR 128 No N A Section 4 5 3 1 51 table name VARCHAR 128 No N A type code VARCHAR 65000 Yes NULL Section 4 5 3 1 52 table owner table name These columns specify the user table associated with the meta information described in the other columns Because a DBMS uses a combination of table owner table name to identify a table both of these columns must have valid values Note In MySQL the value of the table owner column corresponds to the table schema or data base name type code This co
49. a Message Service JMS Service Oriented Architectures SOAs and Web services for inte gration The most commonly adopted standard in high performance real time systems for both integra tion and data access is Object Management Group s OMG Data Distribution Service for Real Time Systems DDS Database Integration Service enables interoperability by bridging between enterprise and embedded standards at the data and communications levels allowing existing applications to be integrated with few or no changes Database Integration Service also mitigates the performance mismatch between enterprise and embedded applications Real time applications using Connext DDS can deliver messages and data at rates between 10 and 100 times faster than can be accepted by enterprise applications Database Integration Service decouples applications at the data level preventing high throughput real time messages and data from overwhelming applications managing business processes by buffering and storing all data in an in memory DBMS the Oracle TimesTen In Memory database which is bundled with Database Integration Service Oracle TimesTen is a memory optimized database delivering exceptionally high performance enabling it to easily support the high data throughput of real time and embedded applications 2 1 Database Integration Service s Unique Features Figure 2 1 2 2 Database Integration Service Bridges Embedded and Enterprise Applica
50. ability kind gt is TRANSIENT_LOCAL_DURABILITY_QOS then the value for this column must be set to RELIABLE RELIABILITY QOS The possible values for the lt dr reliability kind gt column are L BEST EFFORT RELIABILITY QOS default value if the column is NULL This value means that the DataWriters will send their data to the DataReader using best efforts Data may be lost if the system is too busy RELIABLE RELIABILITY QOS This value means that the DataWriters will send their data to the DataReader using a reli able protocol The exact semantics of the reliable connection is controlled by the DDS HistoryQosPolicy of both the DataWriter and DataReader See the Connext DDS documentation for more details on how this QoS policy may be used See also dw durability kind Section 4 5 1 1 14 dw historykind dw history depth Section 4 5 1 1 17 and dr history kind dr history depth Section 4 5 2 1 43 dr destination order kind This column sets the DestinationOrderQosPolicy for the DataReader created to subscribe to Topic data that is stored in the table The value in this column determines how the DataReader treats data received for the same instance of the Topic from different DataWriters When a data instance is received a timestamp associated with the data is compared to the time stamp of the last value of the data instance If the time of the new data is older than the time of the last data received for that instance the
51. ables Table 4 1 4 3 Command line Options Option Description Database Integration Service verbosity level 0 No verbosity 1 Exceptions Connext DDS and Database Integration Ser vice default 2 Warnings Database Integration Service 3 Information Database Integration Service 4 Warnings Connext DDS and Database Integration Service 5 Tracing Database Integration Service 6 Tracing Connext DDS and Database Integration Service Each verbosity level n includes all the verbosity levels smaller than n As the Database Integration Service Daemon runs it may gen verbosity lt verbosity level gt erate log messages reflecting error conditions warning messages or general execution status The messages may be produced by the daemon or by Connext DDS The messages produced by the daemon can be redirected to three possible destinations stdout stderr a file and log tables in the databases to which it is connected Each of these destinations may be enabled independently of each other The first two stdout stderr and file are controlled by command line parameters discussed above and the last log table is controlled in the configuration of a connection as discussed in Database Connection Options Section 4 4 4 3 In this Database Integration Service version the messages produced by Connext DDS can be redirected only to stdout stderr version Prints the Database Integration Ser
52. age eee 3 5 3 7 Real Time Database Replication esee eene nennen nennen 3 6 Using Database Integration Service 4 1 4 1 Introduction to the Database Integration Service Daemon sss 4 2 41 1 How to Run the Database Integration Service Daemon with Oracle 4 2 41 2 How to Run the Database Integration Service Daemon with MySQL 4 4 41 53 How to Run the Database Integration Service Daemons as Windows Services 4 6 4 T4 Iypecodes aient ect rhe ene HER eu e Eu ee n E evo Peg ed leen vae 4 6 4 2 Coinmiand Line Paramieters eate dette ee iiie ete inaia Da ana a aii 4 7 4 3 Environment Variables 2 ecu e ice die dee geredet dene ded ge Poen 4 10 4 4 Configuration File eene tedesco eere ier tiit terae eed 4 11 44 31 How to Load the XML Configuration sse nennen 4 11 442 XML Syntax and Validation iieri nii tried 4 12 AAS Top Level XML Tags ei uote titi tereti eere roter erre tee ee 4 13 44 4 Database Configuration Using the Database Integration Service XML Tag 4 15 4 5 Meta Tables iere price blo tete ste ko ovem t a Ce HU HOA 4 26 4 5 1 Publications Table ccccccccccccsccccssssssscesccessevscccesecossssencecsesessncoteccecsesvesescacessboncecsaceoees 4 26 452 Subscriptions Iablez oleae dai eame wrist teme tn 4 38 45 3 Table Trif Oo n re oo t ert vet Redeem e et Ene o
53. ailable characters Meta Tables 4 5 1 1 11 4 5 1 1 12 4 5 1 1 13 profile_name This column specifies the name of the QoS Profile that Database Integration Service will use to cre ate the publication The name must have the following format QoS profile library name gt lt QoS profile name gt See the Connext DDS documentation for a complete description of QoS Profiles The QoS values specified in the publication table if they are not NULL take precedence over the same values specified in the QoS profile pub present access_scope pub present ordered_access These two columns map directly to the DDS_PresentationQosPolicy of the DDS_PublisherQos used by the Publisher that is created with the DataWriter for publishing changes to the table The DDS_PresentationQosPolicy specifies how the samples representing changes to data instances are presented to a subscribing application The specific columns affect the relative order of changes seen by subscribers to the table The values of these columns must be coordinated with the values of the DDS_PresentationQosPolicy used by the Subscriber in the receiving application or else published data may not be received by the subscriber The possible values for the lt pub present access_scope gt column are L INSTANCE PRESENTATION QOS default value if the column is NULL LI TOPIC PRESENTATION QOS L GROUP PRESENTATION QOS The possible values for the pub present ordered
54. and line parameter cfgFile The command line option cfgFile see Section 4 2 can be used to specify a configuration file You may use a combination of the above approaches Configuration File 4 4 2 XML Syntax and Validation The XML configuration file must follow these syntax rules L The syntax is XML the character encoding is UTF 8 L Opening tags are enclosed in lt gt closing tags are enclosed in lt gt L A tag value is a UTF 8 encoded string Legal values are alphanumeric characters The routing service s parser will remove all leading and trailing spaces from the string before it is processed For example tag value lt tag gt is the same as lt tag gt value lt tag gt J All values are case sensitive unless otherwise stated L Comments are enclosed as follows lt comment gt 1 The root tag of the configuration file must be lt dds gt and end with lt dds gt Database Integration Service provides DTD and XSD files that describe the format of the XML con tent We recommend including a reference to one of these documents in the XML file that con tains the Database Integration Service s configuration this provides helpful features in code editors such as Visual Studio and Eclipse including validation and auto completion while you are editing the XML file The DTD and XSD definitions of the XML elements are in lt Database Integration Service instal lation directory gt resource s
55. and the rtirtc_mysql exe for MySQL The Database Integration Service Daemon uses Connext DDS to communicate with the MySQL and Oracle 11g servers This command line option sets the domain ID used for the connection between the daemon and the servers Default 1 queueLibPath lt directory lib rtirtc oracleq so dll containing This parameter is only available for the rtirtc oracle exe for Oracle Database 11g The Oracle server must find and load lib rtirti oracle so dll in order to connect to the Database Integration Service Daemon See How to Run the Database Integration Service Daemon with Oracle Section 4 1 1 for more information By default the daemon will ask the Ora cle server to look in the directory DRACLE HOME bin If lib rtirti oracle so dll is installed on the server in a different directory then the Database Integration Service Dae mon must be started with this option set to that directory typeMode This parameter is only available for rtirtc timesten exe Specifies whether the names and semantics of the data types in the TimesTen database follow Oracle or TimesTen type rules There are two possible values 0 Oracle type mode default 1 TimesTen type mode For additional information about TypeMode refer to Ora cle amp TimesTen In Memory Database Reference Note To work with Oracle In Memory Database Cache this option must be set to 0 4 9 Environment Vari
56. ase dur Section 4 5 2 1 41 dr deadline period Section 4 5 2 1 42 and dw ownership kind dw ownership strength value Section 4 5 1 1 18 dr time filter min sep This column specifies the minimum separation duration between subsequent samples for the DDS_TimeBasedFilterQosPolicy for the DataReader created to subscribe to Topic data that is stored in the table By setting the values in these columns the user configures the DataReader to see at most one change every the minimum separation period The possible values of the dr time filter min sep sec seconds and lt dr time_filter min_sep nsec gt nanoseconds columns are I A 0 minimum separation duration is specified if both columns are NULL or contain the value 0 This is the DDS default value With this value the DataReader is potentially interested in all the samples LJ A non zero value representing the number of seconds and nanoseconds for the mini mum separation duration That value must be smaller than the deadline period and con tained in the interval 0 1 year See the Connext DDS documentation for more details on how this QoS policy may be used dr res limits max samples dr res limits max instances These columns set some parameters for the DDS ResourceLimits QosPolicy for the DataReader created to subscribe to Topic data that is stored in the table In particular they control the amount of memory that the system is allowed to allocate for storing published data
57. ata Types nepie iecore ies er get BU d 5 14 A JEHOI COGOS 5 c 3D eec ie vacent icta cete uae tide aeaa A 1 B Database LIMITS eeu I B 1 B 1 Maximum Columns for Oracle 11g sse eee enne nnns B 2 B 2 Maximum Columns for MySQL sees ee eee nene n nennen nnne nn nenen B 2 Chapter 1 Welcome to RTI Database Integration Service 1 1 1 2 Welcome to RTI Database Integration Service a high performance solution for integrating applications and data across real time and enterprise systems from RTI Database Integration Service is the integration of two complementary technologies data centric publish subscribe middleware and relational database management systems RDBMS This powerful integration allows your applications to uniformly access data from real time embed ded and enterprise data sources via RTI Connext DDS or via database interfaces Since both these technologies are data centric and complementary they can be combined to enable a new class of applications In particular Connext can be used to produce a truly decentralized distrib uted RDBMS while RDBMS technology can be used to provide persistence for real time data Intended Readers This document presents the general concepts behind Database Integration Service s architecture and provides basic information on how to develop applications using Database Integration Ser vice The chapters assume general knowledge of relational database
58. ataReaders corresponding DDS_DeadlineQosPolicy to non default values The possible values of the lt dw deadline period sec gt seconds and lt dw deadline period nsec gt nanoseconds columns are IJ An infinite deadline period is specified if both columns are NULL or contain the value 2147483647 2 1 This is the DDS default value 1 A non zero value representing the number of seconds and nanoseconds for the deadline period The DDS DeadlineQosPolicy sets a commitment by the DataWriter to publish a value for every data instance to DataReaders every deadline period If this value is set to a non infinite value user applications must update the value of every instance of the Topic stored in the table within each deadline period or the contract with DataReaders that subscribe to the changes to the table will be violated See the Connext DDS documentation for more details on how this QoS policy may be used See also dr deadline period Section 4 5 2 1 42 dw history kind dw history depth These columns directly map to the DDS_HistoryQosPolicy for the DataWriter created to publish table changes The values set for this QoS policy affect the DDS ReliabilityOosPolicy and the DDS DurabilityQosPolicy Using a KEEP ALL HISTORY QOS will ensure that reliable DataReaders will receive every change to the table reliably With a KEEP LAST HISTORY QOS the Database Integration Ser vice Daemon will only guarantee that the last lt dw
59. ated TypeCode the maximum size of the 0 or 1 BLOB columns is determined on a per column basis using the Type Code information Default 65536 Limits the size of CLOB and NCLOB columns for tables that do not have an associated TypeCode see Section 4 1 4 in the RTIRTC TBL INFO meta table see Section 4 5 3 clob default size If a user table has an associated TypeCode the maximum size of the 0 or 1 CLOB and NCLOB columns is determined on a per column basis using the TypeCode information Default 65536 4 22 Configuration File Table 4 12 Tags for TimesTen Connections id Number Additional Tags Allowed isi SE gs Description of Tags within timesten connection Allowed Sets the transaction log acknowledgement period in seconds The transaction log will be acknowledged at the period speci log acknowledgement period fied by the tag value This allows TimesTen to purge old 0 or 1 d transaction log files when no longer needed Default 30 seconds This attribute is only used if the Oracle TimesTen DSN speci fies a diskless connection xla buffer size See Setting the XLA Staging Buffer Size for Diskless O0 or1 Connnections Section 4 4 4 5 Default 409600 4 4 4 4 Initial Subscriptions and Publications As explained in Meta Tables Section 4 5 the daemon is configured to publish and subscribe to data in the database by inserting entries in two meta ta
60. ations on the net work that are using Connext DDS See Typecodes Section 4 1 4 for more details on how the dae mon uses typecodes If the table is created by the Database Integration Service Daemon the daemon may add up to 5 additional columns 6 in MySQL that store meta data used by the daemon when storing data received via Connext DDS or sending table changes via Connext DDS Although optional there are specific operating scenarios where these meta columns are required for the proper operation of the daemon We suggest that the user understands the purpose of the meta columns and if the user applications create the tables used by the Database Integration Service Daemon the user code itself should add the meta columns to the table schema when appropriate There is no spe cific order required for the new columns The meta columns that may be created are J RTIDDS DOMAIN ID and RTIRTC REMOTE These two SOL INTEGER columns are always added to the tables created by the dae mon These additional columns are used by the daemon when user has created entries in both the RTIDDS PUBLICATIONS and RTIDDS SUBSCRIPTIONS meta tables for the same user table In that situation changes to the table made by local user applications will be published via Connext DDS at the same time that the daemon itself may store data into the table received via Connext DDS Database Integration Service Daemon uses these meta columns in order to prevent the republishi
61. base the procedures described in this section must be completed Installing MySQL ODBC 5 1 6 driver The Database Integration Service Daemon requires the installation of the MySQL ODBC 5 1 6 driver or higher The driver is not bundled with the MySQL server and must be installed sepa rately The ODBC connector can be downloaded from http dev mysql com downloads connector odbc 5 1 html The installation guide can be found at http dev mysql com doc refman 5 1 en connector odbc installation html The MySQL ODBC driver requires an ODBC driver manager In Windows the ODBC driver manager is automatically installed with the OS For Linux systems we recommend the use of UnixODBC 2 2 12 or higher a complete free open ODBC solution for Unix and Linux sys tems The driver manager can be downloaded from http www unixodbc org Installing and Configuring the MySQL Server to Access lib rtirti_mysqlq so dll To work with a MySQL database there is a shared library distributed with Database Integration Service that must be installed correctly on the host of the MySOL database server Communica tion by the Database Integration Service Daemon with the MySQL server is accomplished through user defined functions UDF executed by the MySQL server when triggers installed by the Database Integration Service Daemon are fired These functions are provided in a shared library on UNIX based systems or DLL on Windows systems called lib rti
62. bases Connext DDS provides an API to send and receive data between networked applications follow ing a publish subscribe paradigm Oracle provides both file based and in memory products that allow applications to store and retrieve data following a relational database paradigm The corresponding standard API in the database world is SQL Both the Connext DDS and SQL APIs have various language bindings in C C and Java How Database Integration Service maps actions semantics and data types data representation from Connext DDS to SQL and vice versa is described in the following sections 1 Semantic Mapping Section 5 1 Data Representation Mapping Section 5 2 Semantic Mapping Connext DDS applications publish and subscribe to topics which are named data structures using functions like DDSDataWriter write and DDSDataReader read Relational databases contain tables that applications access data using SQL operations such as INSERT UPDATE DELETE and SELECT Table 5 1 describes the mapping between Connext DDS and relational database semantic models Connext DDS DBMS Semantic Models Connext DDS Relational Database Details Accessed via Connext DDS Accessed via SQL API Various language bind Various language bindings ings e g C C and e g C C Java ODBC Java and JDBC Fields in data structures are mapped to columns of a table Each row of a table represents a differ Tables ent value for a da
63. be stored in the table If the storing of each data sample is committed sep arately then an error committing any one sample will only result in the loss of that sample The possible values of the lt process_batch gt column are Ll n 0 The daemon will process data samples in batches of n A value less than or equal to 0 will result in an error that is logged by the daemon A value of n 1 means that the daemon will store each data sample as it arrives The default value is 10 if this column is NULL The possible values of the lt process_period sec gt seconds and lt process_period nsec gt nanosec onds columns are Ll 0 If both columns are 0 then the daemon will not commit received samples periodically Ll n gt 0 A background thread will process received but un stored data at the period specified by these columns It is essential that a non zero period be used if lt process_batch gt is greater than 1 to insure that all received data is eventually stored The default value for process period is 0 1 seconds 0 sec 100000000 nanosec if both col umns are NULL Meta Tables The possible values of the lt commit_type gt column are LJ COMMIT ON PROCESS default value if the columns are NULL This value will direct the Database Integration Service Daemon to commit the storage of a batch of data as a single transaction This will result in higher performance at the risk of losing more data than necessary when t
64. be to or publish those topics unless the user manually creates the corresponding tables in the database or defines the topic types in the configuration file see Table 4 2 The only way to determine whether or not this situation exists is to examine the log messages printed by the daemon A status message will indicate when there is no typecode found for a Topic This message may have been generated because the typecode associated with the topic is too large for the daemon By increasing the DomainParticipantQos resource_limits type_code_max_serialized_length QoS policy the daemon can be configured to handle larger typecodes for complex IDL types The Database Integration Service Daemon will store all the typecodes that it receives with discov ered Topics These typecodes may be used by the daemon to create user accessible tables in the database from which changes are published or data received via Connext DDS is stored See Pub lications Table Section 4 5 1 and Subscriptions Table Section 4 5 2 for more information of how typecodes are used by the daemon 4 6 Command Line Parameters 4 2 Command Line Parameters Any user can start a Database Integration Service Daemon The user name ID and password with which it connects to a database is specified in the configuration file see Table 4 9 When starting a Database Integration Service Daemon the following command line parameters are supported the cfgName parameter is required
65. ble Section 4 5 4 Publications Table When entries rows are added to the meta table RTIDDS_PUBLICATIONS the Database Inte gration Service Daemon will try to create a DataWriter and Publisher along with a DomainPar ticipant if required and use it to send changes to the designated user table via the Connext DDS If the RTIDDS PUBLICATIONS table does not exist at startup the Database Integration Service Daemon will create it with the table owner set to the user name of the database connection as specified in the daemon s configuration file see Section 4 4 The schema and meaning of the col umns of this table are described in the next section Users may insert new rows or modify the column values of existing rows in this table at any time For a new row the daemon will first check to see if the designated user table exists If so it Meta Tables will immediately create the DataWriter with the QoS values as specified by the entry The name of the Topic to publish may be specified by the topic_name column or be automatically con structed as lt table_owner gt lt table_name gt if the topic_name entry is NULL If the user table does not exist the Database Integration Service Daemon will look for the typecode associated with the topic defined in the topic_name column If it finds the typecode the daemon will create the user table with a SQL table schema derived from the typecode following the IDL type to SQL type mapping described
66. bles RTIDDS PUBLICATIONS and RTIDDS SUBSCRIPTIONS In your XML configuration you can specify initial values to insert in these tables For example mysql connection subscriptions delete true gt subscription table owner user table owner table name gt mytablel lt table name gt lt domain_id gt 54 lt domain_id gt topic name mytopicl topic name gt type name mytypel type name gt subscription subscription subscription subscriptions publications publication overwrite true gt table owner user table owner table name gt mytable2 lt table name gt lt domain_id gt 54 lt domain_id gt topic name mytopic2 topic name gt type name mytype2 type name gt lt publication gt lt publication gt lt publication gt lt publications gt lt mysql_connection gt 4 23 Configuration File Table 4 13 Table 4 14 Within lt subscriptions gt and lt publications gt tags you can specify as many lt subscription gt and lt publication gt tags as you want The content of each tag inside lt subscription gt lt publication gt represents the value for a column with the same name in the table RTIDDS_SUBSCRIPTIONS RTIDDS_PUBLICATIONS Each of these lt subscription gt lt publication gt tags may result in the insertion or update of a row in the corresponding meta table All the rows in the tables can be deleted before inserting new rows if the at
67. bling RTI Monitoring Library in Database Integration Service The QoS associated with the DomainParticipants that are created by Database Integration Service can be configured in three different ways LJ By setting the attribute is default qos in the tag qos profile containing the participant qos to true In this case that profile is the default configuration for all the Entities created by the Database Integration Service Daemon For a list of XML files where you can declare the QoS Profile see Section 4 4 1 1 By referring to a profile using the XML tag profile name within publication and subscription see Section 4 4 4 4 LI By referring to a profile in the profile name column of the tables RTIDDS PUBLICATIONS or RTIDDS SUBSCRIPTIONS see Section 45 1 and Section 4 5 2 Notice that since Database Integration Service is statically linked with RTI Monitoring Library you do not need to have it in your Path on Windows systems or LD LIBRARY PATH on UNIX based systems For details on how to configure the monitoring process see the Monitoring Library section of the RTI Connext DDS Core Libraries User s Manual Chapter 5 IDL SQL Semantic and Data Mapping 5 1 Table 5 1 This chapter describes the semantic and data representation mapping that RTI Database Integra tion Service uses to connect DDS based applications such as Connext DDS to MySQL Microsoft SQL Server Oracle and Oracle TimesTen In Memory data
68. chema rti_real_time_connect dtd and lt Database Integration Ser vice installation directory gt resource schema rti_real_time_connect xsd respectively To include a reference to the XSD document in your XML file use the attribute xsi znoN amespaceSchemaLocation in the lt dds gt tag For example lt xml version 1 0 encoding UTF 8 gt lt dds xmlns xsi http www w3 org 2001 XMLSchema instance xsi noNamespaceSchemaLocation lt NDDSHOME gt resource schema rti real time connect xsd d s lt NDDSHOMES is described in Paths Mentioned in Documentation Section 1 2 To include a reference to the DTD document in your XML file use the lt DOCTYPE gt tag For example lt xml version 1 0 encoding UTF 8 gt lt DOCTYPE dds SYSTEM lt NDDSHOME gt resource schema rti_ routing service dtd gt lt dds gt lt dds gt We recommend including a reference to the XSD file in the XML documents this provides stricter validation and better auto completion than the corresponding DTD file 1 Leading and trailing spaces in enumeration fields will not be considered valid if you use the distributed XSD doc ument to do validation at run time with a code editor 4 12 Configuration File 4 4 3 Top Level XML Tags Let s look at an example configuration file You will learn the meaning of each line as you read the rest of the sections lt xml version 1 0 gt lt dds gt real time connect na
69. d with user customizable QoSs to publish Topics whenever the specified table changes in the database Mapping the SQL table schema to the equivalent Topic described by IDL is done automatically by the daemon no user configuration necessary Notifying Remote Applications of Changes in a Database Connext DDS App SQL UPDATE INSERT 4 RTI Database Integration 4 Service Daemon Database write SQL SELECT fd di X Table Tracks Table Passengers Fit Lat E Fit Name Addrs C129 345 5 402 3 C129 A Johnson Connext DDS C054 27 7 46 8 C054 J Smith App Embedded lt Q enn Enterprise Bidirectional Integration Figure 3 4 shows a system that integrates the capabilities described in the last two sections Data base Integration Service provides bidirectional dataflow between embedded Connext DDS appli cations and enterprise database systems This approach can typically be used to create a closed loop system where sensory data is collected processed and analyzed in an in memory data base and the resulting analysis creates state changes that are fed back to remote sensors and devices to control their behavior and mode of operation 3 4 Bridging between Domains Figure 3 4 Bidirectional Integration 3 5 3 6 Connext DDS RTI Database Integration Service Daemon Connext SQL Table Tracks Table Passengers Fit Lat Long Fit Name Addrs C129 345 1023 C129 A Johnson
70. e CA 94089 Phone 408 990 7444 Email support rti com Website https support rti com Available Documentation The following documentation is available for RTI Database Integration Service IJ The Release Notes RTI Database Integration Service ReleaseNotes pdf This document pro vides an overview of the current release s features and lists changes since the previous release sys tem requirements supported architectures and compatibility with other products L The Getting Started Guide RTI Database Integration Service GettingStarted pdf This docu ment provides installation instructions a short Hello World tutorial and troubleshooting tips 1 This User s Manual starts with an overview of RTI Database Integration Service s basic concepts ter minology and unique features It then describes how to develop and implement applications that use RTI Database Integration Service Additional Resources LJ The ODBC API Reference from Microsoft is available from http msdn microsoft com en us library ms714562 VS 85 aspx I The documentation for the Oracle TimesTen In Memory Database can be found in the doc direc tory of the Oracle TimesTen installation J The documentation for Oracle databases can be found here http www oracle com technology documentation index html I The documentation for MySQL databases can be found here http dev mysql com doc refman 5 1 en index html iii Contents
71. e communication transport There are two possible values 1 UDPv4 2 Shared memory default Note Shared memory communication between the Data base Integration Service Daemon and the database servers does not work on Windows 2003 Windows Vista or Win dows 7 systems when the Database Integration Service dae mon runs with the option nodaemon and the database server runs as a service For this use case communication can be enabled by using UDPv4 as the transport help Prints out a usage message listing the command line parameters logFile lt log file gt Pathname of the file to be used for log messages If specified log messages will automatically be stored in the file noDaemon Start as a normal process Without this option running the Database Integration Service Daemon executable will start a daemon process on Linux systems or start a service on Windows systems As a dae mon no log messages of any kind are printed to stdout or stderr However by specifying this option the daemon will start as a regular process which can be run as a background process using the standard OS with the command line option amp and log messages will be printed to stdout and stderr 4 8 Command Line Parameters Table 4 1 Command line Options Option Description queueDomainId domain ID This parameter is only available for the rtirtc oracle exe for Oracle Database 11g
72. e A in RTIDDS_PUBLICATIONS meta table UPDATE delete existing pub INSERT Goes user table exist Yes create new No does type code exist Yes create user table in DB No defer creation until typecode is discovered 4 5 1 1 Publications Table Schema The RTIDDS PUBLICATIONS table is created with the following SOL statement Oracle TimesTen with command line option typeMode 1 Create Table RTIDDS PUBLICATIONS table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL domain id INTEGER NOT NULL topic name VARCHAR 200 type name VARCHAR 200 table history depth INTEGER resolution column VARCHAR 255 idl member prefix max length INTEGER idl member suffix max length INTEGER profile name VARCHAR 255 pub present access scope VARCHAR 25 pub present ordered access TINYINT pub partition name VARCHAR 256 dw durability kind VARCHAR 30 dw liveliness lease dur sec INTEGER dw liveliness lease dur nsec INTEGER dw deadline period sec INTEGER dw deadline period nsec INTEGER if necessary create DomainParticipant 4 28 Meta Tables dw history kind VARCHAR 21 dw history depth INTEGER dw ownership kind VARCHAR 23 dw ownership strength value INTEGER dw publish mode kind VARCHAR 29 dw res limits max samples INTEGER dw res limits max instances INTEGER PRIMARY KEY table owner table name domain id topic name
73. e Daemon with the operating system More information on each error can be found by examining the native OS errors codes that will be logged with these messages 8192 message General OS error message 8193 Error handling OS signals 8194 Unable to set signal handler for signal 8195 Error getting the host name 8196 Error allocating memory for lt object gt From 16384 Warning messages These are warning messages that may be logged by the Database Integration Service Daemon Timestamps prior to 1970 01 01 00 00 00 00 cannot be used for conflict resolution The RTC daemon will always use 1970 01 01 00 00 00 00 as the timestamp for those cases 16384 The Timestamp value of the resolution column is NULL The RTC daemon will use the value 1970 01 01 00 00 00 00 16385 This warning appears only with Oracle 16386 Diskless log buffer overflow ies Ten databases The prefix suffix based name associated with member A in IDL type T collides with 16387 IDL member identifier collision the name of another member inside the same type Database Integration Service will resolve the conflict using an index 16388 Invalid configuration parameter A QoS value has been ignored by Database 16900 Ignored Qoo value Integration Service A 6 Table A 1 Database Integration Service Errors and Warnings Code Message Details 16392 Dyna
74. e Database Integration Service XML Tag Section 4 4 4 How to Load the XML Configuration Database Integration Service loads its XML configuration from multiple locations This section presents the various approaches listed in load order The first three locations only contain QoS Profiles and are inherited from Connext DDS see Chapter 15 in the RTI Connext DDS Core Libraries User s Manual lt NDDSHOMES is described in Paths Mentioned in Documentation Section 1 2 NDDSHOME resource xml NDDS QOS PROFILES xml This file contains the Connext DDS default QoS values it is loaded automatically if it exists First to be loaded I File in NDDS QOS PROFILES The files or XML strings separated by semicolons referenced in this environment vari able are loaded automatically M working directory USER QOS PROFILES xml This file is loaded automatically if it exists The next locations are specific to Database Integration Service L NDDSHOME resource xml RTI REAL TIME CONNECT xml This file contains the default Database Integration Service configuration and QoS Profiles it is loaded if it exists The default configuration does not work out of the box because it requires setting the parameters that configure the database connections such as dsn username and password see Section 4 4 4 LI working directory USER REAL TIME CONNECT xml This file is loaded automatically if it exists LL File specified using the comm
75. e Integration Service installs in the Oracle 11g server The size of the PL SOL programs depends on the column data types For tables in which all the columns are numbers RTI has been able to publish with approximately 625 col umns For tables in which all the columns are of type VARCHAR2 RTI has been able to publish with approximately 650 columns If the maximum number of columns is exceeded for Database Integration Service publications you will see an error message similar to DDSQLDaemonCore onUpdateMetaTableEntry line 3126 ERROR 4096 5009 CNA CNAMonitorTable ODBC call failed ORA 24344 success with compilation error B 2 Maximum Columns for MySQL The exact limit for the number of columns is driven by two factors L The maximum row size for a table not counting BLOBs is 65535 LJ The maximum size of the meta information schema associated with a table is 64 KB The meta information includes the column names The longer the column names the smaller the maximum number of columns Table B 2 provides information about the maximum number of columns that RTI was able to use for different column name lengths and column types For more information about MySQL restrictions on the number of columns see http dev mysql com doc refman 5 1 en column count limit html Table B 2 Max Number of Columns Used by RTI for Different Column Types and Name Lengths Column Type Column Name Length charac
76. e database 1 required in mysql connection and oracle connection O0 or 1 optional in timesten connection publications This tags allows inserting publications in the table RTIRTC PUBLICATIONS when the daemon starts up See Initial Subscriptions and Publications Sec tion 4 4 4 4 Oor1 send period The send period value specifies the milliseconds interval at which the Database Integration Service Dae mon publishes database changes The value must be greater than or equal to 0 With a value of 0 the dae mon publishes database changes as soon as they are available A shorter time interval reduces latency Default 100 ms Oor1 subscriptions This tags allows inserting subscriptions in the table RTIRTC SUBSCRIPTIONS when the daemon starts up See Initial Subscriptions and Publications Sec tion 4 4 4 4 Oor1 4 21 Configuration File Table 4 9 Common Tags for all Database Connections Common Tags for mysql connection oracle connection timesten connection Description Number of Tags Allowed Specifies the user name to connect to the database This attribute is mandatory for Oracle and MySQL 1 required in databases since there is no default user for database mysql connection and connections oracle connection user name It is optional for Oracle TimesTen if this attribute is missing then the connection to the
77. e installed on the Oracle server host 4 2 Introduction to the Database Integration Service Daemon 4 1 1 1 4 1 1 2 Notes The exact platforms that are supported on Windows and Linux systems may be different for Oracle versus Oracle TimesTen In Memory databases Please consult the Release Notes for specific details of the supported platforms for your release of Database Integra tion Service Not only do the libraries have to be present on the Oracle server host the Oracle server must also be configured to find the libraries There are separate procedures for the librtirtc oracleq and libnddsxxx libraries These procedures are detailed below Installing and Configuring the Oracle Server to Access lib rtirti oracleq so dll There are two options for installing lib rtirti oracleq so dll 1 Copy the appropriate version of the library into either ORACLE HOMBE bin or SORACLE HOME lib on the server host SORACLE HOME is the installation direc tory of the Oracle DBMS Or 2 Copy the appropriate version of the library into any directory on the server host It can even be used directly from the Database Integration Service installation directory if that directory can be accessed by the Oracle server With the second option the location of the lib rtirti oracleq so dll library must be defined in the file extproc ora located at 0RACLE HOME hs admin on UNIX operat ing systems and at ORACLE HOME hs Nadmin on Wind
78. e same type in a single column element To map complex IDL types to SQL table schemas embedded data structures are unfolded so that ele ments of an embedded structure are stored individually in separate columns When the Database Integration Service Daemon creates a table schema from a Topic it will auto matically flatten hierarchical data structures into tables In doing so the names of columns that store the fields of embedded structures will have hierarchical names For example given this IDL definition struct bar struct foo long one bar element long two Fi The table constructed from a Topic which uses the foo type would have the following schema by default CREATE table foo INTEGER element one INTEGER element two The Database Integration Service Daemon allows the configuration of the separator character using the attribute IdentifierSeparatorChar defined in the general options of the configuration file described in Section 4 4 4 1 While for most embedded structures the hierarchical naming of columns is not needed for the Database Integration Service Daemon to handle type translation correctly the proper hierarchical naming of columns is essential for the daemon to serialize and deserialize IDL unions and sequences These types are variable in length however the table must have enough columns to hold the maximum size of the IDL data type Hierarchical naming allows the Database Integration Service Da
79. ecuting SQL UPDATE or INSERT every time data is received the daemon only stores the data after it receives a certain number of samples set by lt process_batch gt If the value lt process_batch gt is greater than 1 then it is essential that the lt process_period sec nsec gt is set to be non zero Thus the daemon will process stored data periodically even if the total number of data samples received is less than lt process_batch gt lt process_period sec nsec gt is an upper limit on the amount of delay that will be incurred before received data is stored in the database The period can be set to 0 only if lt process_batch gt is set to 1 This means that the daemon will store each data sample as it is received so there is no need for periodic processing of the received samples Use these values to have the daemon store the data with minimal latency at the cost of lower overall throughput Finally using the column lt commit_type gt you can choose whether or not the SQL UPDATE INSERT statements are committed when each data sample is stored or after all of the data being processed have been stored There is significant performance enhancement if the storing of mul tiple data samples is committed as a single transaction However if there is a problem during an SQL commit for example the transaction log of the database is full then the entire transaction is rolled back which means that none of the received data in that batch will
80. ed on the MySQL server host Note Please consult the Release Notes for specific details of the supported platforms for your release of Database Integration Service to MySQL The libraries nddsc and nddscore must be located in a directory that is searched by the system dynamic linker UNIX based Systems L Copy the shared libraries to a directory such as usr lib E Alternatively add the libraries to the environment variable LD_LIBRARY_PATH that must be set for the user who starts the MySQL server This method requires restarting the MySQL server Windows Systems Copy the dll files to the system directory WINDOWS System32 or WINDOWS Sys tem Alternatively you can add the directories containing the libraries to the System variable Path as follows Using the dialog opened with Start Settings Control Panel System Advanced tab Environment Variables button add the directories with backslash and semicolon separators containing the libraries to the System variable Path If the MySQL server is running as a service you will need to reboot the computer for this change to take effect Starting the MySQL Server in ANSI QUOTES mode The MySQL server can operate in different sql modes The Database Integration Service Daemon requires the MySQL server to be configured in ANSI QUOTES mode Under that configuration the MySQL server treats as an identifier quote character instead of a string quote charact
81. ee Appendix A User Table Creation The Database Integration Service Daemon may create tables automatically for user applications in the database when entries are made in the RTIDDS PUBLICATIONS or RTIDDS_SUBSCRIPTIONS meta tables see Sections 4 5 1 and 4 5 2 The daemon will create the table with the table owner and table name specified in an entry in one of those tables if 1 There is no existing table in the database with the same lt table_owner gt lt table_name gt identifier and 2 A type corresponding to the lt type_name gt column for the entry has been defined in the XML configuration file see Section 4 4 3 or A typecode corresponding to the lt topic_name gt column for the entry has been discov ered If either condition above is not satisfied then the daemon will not create the user table If the user table already exists then the daemon will attempt to use that table when publishing or sub scribing to Topics It is up to the user to create the table with a schema that maps to the Topic IDL type See Data Representation Mapping Section 5 2 for details on how SQL table schemas and IDL types are mapped to each other If the table does not exist and there is no XML definition for the type and the typecode for the IDL type specified by the entry is unknown the Database Integration Service Daemon will defer User Table Creation creation of the table until the typecode has been discovered from other applic
82. emon to identify columns that form a embedded complex element Hou For variable length types other than sequences of char wchar or octet an extra column with the suffix length is also added to the table to hold the current length of the type Also each column that represents a field in an element of the variable length type must have a suffix x in its name that identifies the index of the element where x 0 to max length 1 During the serialization and deserialization process the daemon will usually be working with less than the maximum length of data and thus will need to use the hierarchical naming along with the suffix to determine which columns belong to unused elements that should be skipped 5 4 Data Representation Mapping This hierarchical flattening operation of member names may lead to very long column names in the generated table and can easily exceed the maximum number of characters supported by the database some databases limit the column names to 30 characters To reduce the length of the generated names you can instruct Database Integration Service to con sider only the first n and the last m characters of the flattened name and eventually resolve any conflict by using a progressive number between the prefix and the suffix The two tags lt idl_member_prefix_max_length gt and idl member suffix max length see page 4 19 defined in the configuration file described in Section 4 4
83. er To verify if the MySQL server is already configured in ANSI QUOTES mode run the following SOL statement SELECT global sql mode If the string ANSI QUOTES is not part of the result the MySQL server needs to be configured in ANSI QUOTES mode using the option sgl modez ANSI QUOTES to start the server That same effect can be achieved at runtime by executing the following SOL statement SET GLOBAL sql mode ANSI QUOTES Note The specific configuration of the MySQL server may require the use of additional SOL mode strings when starting the server 4 5 Introduction to the Database Integration Service Daemon How to Run the Database Integration Service Daemons as Windows Services On Windows the Database Integration Service Daemons rtirtc_oracle exe rtirtc_timesten exe and rtirtc_mysql exe can be run as system services During the installation process you may choose to install these daemons as Windows services which can then be controlled through the Start Programs Administrative Tools Services application The Database Integration Service services will be installed in manual mode Use the Services application to change this automatic to have the services start when the Windows machine boots up The configuration file used by the Windows services is the default file lt Database Integration Service installation directory gt resource xml RTI_LREAL_TIME_CONNECT xmll You can change the location of the config
84. ere are no global network wide locks on records when a transaction is being exe cuted conflicts can occur The best way to avoid conflicts is to have only one host modify a spe cific row instance or table topic but that is not always possible The second best way is to design the application in such a way that conflicts can never occur for instance due to data flow dependencies But that also is often hard to achieve By default conflict resolution is not enabled when you set lt enable_table_replication gt to true You can enable conflict resolution by setting the column dr destination_order kind in RTIDDS_SUBSCRIPTIONS to BY_SOURCE_TIMESTAMP_DESTINATIONORDER_QOS see Section 4 5 2 1 40 With this setting eventual consistency can be guaranteed Conflicts can cause a temporary inconsistency between the databases but eventually these are resolved by the Database Integration Service conflict resolution mechanism By default conflicts are resolved using a timestamp corresponding to the system time when the update occurred You can over write this behavior by providing your own timestamp in a separate database column see Section 4 5 1 1 9 If you do not need conflict resolution you can disable it by setting the column dr destination order kind in RTIDDS SUBSCRIPTIONS to BY RECEPTION TIMESTAMP DESTINATIONORDER OOS 4 17 Configuration File 4 4 4 1 3 Table Initialization 4 4 4 2 Table 4 8 When a host starts using a
85. este eto eive den 4 54 4545 Log Tables eren eter reet ree he tette ee etie e ne 4 55 4 6 User Table Crea th ori 35 c erret ates tee eerta nade cgsts suid eeigee tet dive eb diee epe euer ad 4 57 4 7 Support for Extensible Types tssis tine tette tite dint etait iri ir eite aiaa 4 59 4 8 Enabling RTI Distributed Logger in Database Integration Service s 4 60 4 9 Enabling RTI Monitoring Library in Database Integration Service sss 4 60 5 IDL SQL Semantic and Data Mapping 5 1 5 1 Semantic Mapping rsi anr a bite db irse n EUER p e EE EY RR EDI Ga EP LEON ve pue 5 1 52 Data Representation Mapping ccccececcsssssesesenenesenesesesneneesecessnsesesessseseseeseseseseseseseseneneness 5 3 521 IDEto SOL Mapptng 5u iae oie ge neri pire ni Pe er P deel ia teats 5 3 5 22 Primitive Types Mapping nennen enne n nnne nenne nnns 5 6 5 23 Oracle In Memory Database Cache Mapping sse 5 9 52 4 BitField Mappitig iude ete te nU ee ut teer teda 5 9 5 25 En m Types Mapping ote ani pee Feier E Pete din 5 11 52 6 imple DL Structutes eee mee etie seule deus ree ee iet etie 5 11 S27 Complex IDE Structures 5n eroe ree ei A Rt AE Lebe ted 5 11 52 8 Array Fields siete ee eite aa ce ba Hei pene iim etta 5 12 5229 Sequence Fields it eta De ee DP EH p rdi Pret eg 5 13 5 210 NULE Values 5t TERR E e RE ERE TERES 5 14 52 11 Sparse D
86. eta column is used to detect committed changes in a table Its value is automatically assigned by the MySQL server Each time there is a change in a table row or a new row is inserted the MySQL server assigns a new SCN value to the column RTIRTC SCN The assignment is done during the execution of the BEFORE UPDATE INSERT trigger installed by the Database Integra tion Service Daemon Support for Extensible Types Database Integration Service includes partial support for the Extensible and Dynamic Topic Types for DDS specification from the Object Management Group OMG This section assumes that you are familiar with Extensible Types and you have read the RTI Connext DDS Core Libraries Getting Started Guide Addendum for Extensible Types 1 The Database Integration Service Daemon can publish and subscribe to topics associated with final and extensible types L The Database Integration Service Daemon can publish and or subscribe to only one ver sion of the types associated with a topic within a domain L The DataReaders created when a new entry is inserted in the RTIDDS SUBSCRIPTIONS table are configured with the TypeConsistencyEnforcementOosPolicy s kind set to DISALLOW TYPE COERCION You cannot overwrite this setting l You can select the type version for a given topic within a domain in two different ways By providing the type description via XML and then referring to that description using the column type name in the RTIDDS S
87. fferences LJ An extra column is added with the suffix length used to store the current length of the sequence 1 The total number of columns created is equal to the maximum number of elements that the sequence can hold although the number of columns containing valid data at a given time is stored in the length column IJ The naming convention of adding the suffix i to each column is required for the Database Integration Service Daemon to handle the mapping between IDL and SQL cor rectly The open bracket and close bracket characters can be configured using the tags lt open_bracket_char gt and lt close_bracket_char gt see page 4 19 L Sequence elements can contain the NULL value since not all elements may be used at a given time Hou Note Sequences of the IDL types char wchar or octet map directly into the variable length SOL types VARCHAR VARWCHAR and VARBINARY respectively Table 5 15 shows the mapping of a sequence field between IDL and SQL Table 5 15 Sequence Fields in IDL and SQL IDL Type SOL Table Schema CREATE TABLE MySequenceContainer my key field INTEGER NOT NULL my seq field length INTEGER NOT NULL my seq field 0 SMALLINT long my key field Gkey y_seq_ my seq field 1 SMALLINT sequence lt short 4 gt my_seq_field my seq field 2 MALIN hr my seq field 3 SMALLINT PRIMARY KEY my key field struct MySequenceContainer
88. field a The format on the wire of char and char x is the same b The format on the wire of wchar and wchar x is the same c The format on the wire of octet and octet x is the same d The str suffix is used to distinguish between w string lt x gt and sequence lt w char x gt e The 1d suffix is used to distinguish between octet x and long double Basic Types in IDL and SQL SQL Server IDL Type IDL Field Name SOL Type Table Column Name char my field char 1 my field char x my field char x my field sequence lt char x gt my_field varchar x my_field wchar my_field nchar 1 my_field wchar x my_field nchar x my_field sequence lt wchar x gt my_field nvarchar x my_field octet my_field binary 1 my_field octet x my_field binary x my_field sequence lt octet x gt my_field varbinary x my_field 5 8 Data Representation Mapping Table 5 5 5 2 3 Table 5 6 5 2 4 Basic Types in IDL and SQL SQL Server IDL Type IDL Field Name SOL Type Table Column Name boolean my field tinyint my field short my field smallint my field unsigned short my field smallint my field long my field int my field unsigned long my field int my field double my field float 53 my_field float my_field real
89. for data distribution has emerged as the premier method to integrate and build distributed real time systems For decades in enterprise systems standards for communications data representation and data storage has enabled the tremendous growth of software applications for business processes worldwide The standards such as SOL ODBC JMS HTML XML and WDSL have greatly increased the interoperability of those business systems Database Integration Service is the first commercial product that interconnects the DDS standard newly established in the embedded world to the common standards of the enterprise world With Database Integration Service enterprise applications have direct access to real time data and real time applications have access to the plethora of processes and logic that has been developed to configure and direct actions based on business decisions Connectivity To Edge Devices For edge devices such as sensors and hand helds Database Integration Service integrates Connext DDS applications with databases Applications can publish data into relational databases and subscribe to changes in relational databases using the standard Connext DDS application pro gramming interface Integration between Connext DDS and relational database applications is 2 3 Database Integration Service s Unique Features 2 2 3 2 2 4 2 2 5 2 2 6 supported by an IDL to SQL mapping that allows both types of applications to access a unif
90. gs Code Message Details 1024 2047 Connext DDS related errors These messages are produced through the interaction of the Database Integration Service Daemon with Con next DDS More information on each error can be found by examining the native Connext DDS errors codes that will be logged with these messages 1024 lt message gt General Connext DDS error message 1025 Error getting lt entity gt default QoS 1026 Error getting lt entity gt QoS 1027 Error setting lt entity gt QoS 1028 Error creating lt entity gt 1029 Error getting lt entity gt 1030 Error enabling lt entity gt 1031 Error cloning type code 1032 Error reading RTI DDS samples 1033 Error setting entity user data 1034 Error disposing RTI DDS instance 1035 Error unregistering RTI DDS instance 1036 Error writing RTI DDS sample 1037 Error ignoring entity 1038 Error creating waitset type waitset 1039 Error waiting in waitset type waitset 1040 Error getting builtin transport property 1041 Error setting builtin transport property 1042 Error creating waitset type guard condition 1043 Error attaching condition 1044 Error registering type lt type gt 1045 Error taking REDA buffer 1046 Error creating mutex 1047 Error creating thread thread 1048 Error creating REDA fast buffer 1049 Error taking semaphore 10
91. gure the separator character S for the columns in the meta tables identifier separator char 00r1 The default value of in Oracle and Oracle TimesTen will generate columns names that must be referenced using double quotes See IDL to SOL Mapping Section 5 2 1 for more infor mation about double quoted identifiers Default For Oracle TimesTen for MySQL 4 18 Configuration File Table 4 8 Database Mapping Options Tags within Number lt database_mapping_ Description of Tags options gt Allowed Controls the prefix length of the IDL member identifiers that will be used to truncate column names when a table is auto matically created If the default value 1 is used Database Integration Service will not truncate IDL member identifiers when these are used to create column names If a positive value n is provided Database Integration Service lt idl_member_prefix_ will use the first n characters from the IDL member identifier max length to compose the associated column name 0or1 A value of 0 tells Database Integration Service to compose the column name using only the last characters of the identifiers as defined by idl member suffix max length This value can be overridden per table by assigning a value to the idl member prefix max length column in the meta tables Default 1 unlimited Controls the suffix length of the IDL member identifiers that
92. he data columns with hierarchical names from the name of the union field and the name of the union itself In addition the values of the switch case statement in the IDL union are encoded into the names of the data columns as well e g c 0 1 c 2 def This naming convention is required for the proper serialization and deserialization of unions The Database Integration Service Daemon uses the name of the fields when processing an IDL union to know which column s correspond to the value of the discriminator Table 5 13 shows the mapping of an union between IDL and SQL Union Fields in IDL and SQL IDL Type SQL Table Schema union MyUnion switch long CREATE TABLE MyUnionContainer case 0 my key field INTEGER NOT NULL case 1 my union field d INTEGER NOT NULL long my long field my union field c 0 1 my long field INTEGER case 2 my union field c 2 my double field DOUBLE double my double field my union field c def my short field SMALLINT default PRIMARY KEY my key field Short my short field struct MyUnionContainer long my_key field key MyUnion my union field Array Fields For array fields where the array type is different from octet H char and wchar an IDL array type is stored as consecutive columns of the same type in a SQL table If the Database Integration 5 12 Data Representation Mapping Service Daemon
93. he profile name column of the tables RIIDDS PUBLICATIONS or RTIDDS SUBSCRIPTIONS see Section 4 5 1 and Section 4 5 2 4 13 Configuration File Table 4 2 Top Level Tags Number Tags within lt dds gt Description of Tags Allowed Specifies a Database Integration Service configuration This tag is used to specify the set of databases to which the daemon will connect Note There is no way to dynamically configure the Database Integration 1 or more lt real_time_connect gt Service Daemon Daemon to connect to a database after it has started required All database connections must be specified within this tag before the daemon starts See Table 4 3 for a description of the elements contained inside lt real_time_connect gt Defines types that can be used to create database tables The type description is done using the Connext DDS XML format for type definitions For more information see Section 3 4 in the RTI Con next DDS Core Libraries User s Manual For example lt types gt lt struct name Point gt lt types gt lt member name x type long gt Oor1 member name y type long gt lt struct gt lt types gt Database Integration Service supports automatic table creation by using the types defined within this tag or the typecode sent by Connext DDS applications See Section 4 6 for additional information on user table creation Because a configuration file may contain
94. he transaction is rolled back because an error with the database LI COMMIT ON SAMPLE This value will direct the daemon to commit the storage of each data sample as a sepa rate transaction Although the daemon will use more resources if an error occurs when a transaction is committed only that data sample is lost 4 5 2 1 29 cache maximum size cache initial size These columns control the size of a cache used to store keys that exist in the table that the Data base Integration Service Daemon maintains for each DataReader When a data instance is received the daemon first checks the cache to see if a row corresponding to the data already exists in the table If the key is in the cache then the daemon executes an SOL UPDATE to store the data in the table If the key does not exist in the cache then the Database Integration Service Daemon will INSERT a row with the key instead The key cache can greatly enhance the performance of the daemon in storing data into the database by saving an SQL operation each time data is received Without a cache the daemon would need to execute 2 SOL statements to store data with the cache only 1 The trade off is the memory used to store keys versus the performance gain The default values of cache maximum size and cache initial size are 0 if the columns are NULL The sizes are specified as the number of keys For small tables the cache could be sized to hold all of the keys Thus the si
95. history depth 1 Keeps one record per primary history kind KEEP_LAST_HISTORY_QOS key value writer_resource_limits instance_replacement DDS_DISPOSED_INSTANCE_REPLACEMENT Enables replacement of writer_resource_limits replace_empty_instances deleted rows DDS_BOOLEAN_FALSE DataReader QoS Changes when lt enable_table_replication gt is True QoS Change Purpose reliability kind RELIABLE_RELIABILITY_QOS Enables reliability destination_order source_timestamp_tolerance sec Performs conflict resolution DURATION_INFINITE_SEC Note lt enable_table_replication gt sets destination_order source_timestamp_tolerance nanosec some QoS related to conflict resolution DURATION_INFINITE_ NSEC but it does not enable the feature See ownership kind SHARED_OWNERSHIP_QOS Section 4 4 4 1 2 for additional details Enables propagation of delete opera protocol propagate_dispose_of_unregistered_instances true tons Sends table contents to late joiners durability kind TRANSIENT_LOCAL_DURABILITY_QOS table initialization Keeps one sample per primary key history kind KEEP_LAST_HISTORY_QOS value These QoS changes have priority over the values set using QoS Profiles However they can be overwritten per publication and per subscription by setting the corresponding fields in the RTIDDS_PUBLICATIONS and RTIDDS_SUBSCRIPTIONS meta tables see Section 4 5 1 and Section 4 5 2 Conflict Resolution Because th
96. history depth gt changes for each data instance are sent reliably If the lt dw durability kind gt column of the row is set to TRANSIENT LOCAL DURA BILITY QOS then these columns determine how many past data changes are sent to new sub scribers to table changes The possible values of the lt dw history kind gt and lt dw history depth gt columns are I KEEP LAST HISTORY QOS For this setting the column lt dw history depth gt determines how many published changes for each data instance in the table are stored in the DataWriter to support reli ability or durability lt dw history depth gt should be set to an integer greater than 0 The default value for his tory depth is 1 if this column is NULL J KEEP ALL HISTORY QOS default value if the column is NULL This setting implies that the DataWriter created to publish table changes will store all of the changes to the table that it has sent The total number of changes that can be stored is limited by the value in the dw res limits max samples column For this setting the value in lt dw history depth gt is ignored See the Connext DDS documentation for more details on how this QoS policy may be used See also dw durability kind Section 4 5 1 1 14 and dw res limits max samples dw res limits max instances Section 4 5 1 1 20 Meta Tables 4 5 1 1 18 4 5 1 1 19 4 5 1 1 20 dw ownership kind dw ownership_strength value These columns directly map t
97. ice warnings Service information Service warnings Core Libraries and Service tracing Service 6 tracing Core Libraries and Service Default 1 exceptions Prints the RTI Database Integration Service version Displays this information ox OF WN FH O 4 7 Command Line Parameters Table 4 1 Command line Options Option Description appName lt application name gt Assigns a name to the Database Integration Service execution The application name is used to set the Entity NameQosPol icy of the DomainParticipants created by Database Integration Service cfgFile configuration file gt Specifies an XML configuration file for Database Integration Service The parameter is optional since the Database Integration Ser vice configuration can be loaded from other locations See Section 4 4 1 for further details cfgName configuration name Required Specifies the name of the configuration to load The Data base Integration Service Daemon will look for the first tag real time connect with that name See Configuration File Section 4 4 dbTransport 112 This parameter is only available for the rtirtc oracle exe for Oracle Database 11g and the rtirtc_mysql exe for MySQL By default Database Integration Service uses shared memory to communicate with the MySQL and Oracle database serv ers The dbTransport parameter can be used to change th
98. ies that were pending on the dis covery of the typecode The daemon will also create the DataReader if there is an entry in the RTIDDS SUBSCRIPTIONS table without an associated typecode but the user subsequently creates the corresponding table If user applications modify an existing row in the RTIDDS SUBSCRIPTIONS table the Data base Integration Service Daemon will first delete the DataReader that was created for that entry if Meta Tables it exists and then go through the same process of trying to create the user table and DataReader as if the row was newly inserted If user applications delete an existing row in the RTIDDS_SUBSCRIPTIONS table the Database Integration Service Daemon will delete the associated DataReader if it exists A flow chart describing this logic is provided below wait for change in RTIDDS SUBSCRIPTIONS meta table UPDATE delete existing sub INSERT Goes user table exist Yes create new sub if necessary create DomainParticipant No Yes does type code exist create user table in DB No defer creation until typecode is discovered 4 5 2 1 Subscriptions Table Schema The RTIDDS SUBSCRIPTIONS table is created with the following SOL statement Oracle TimesTen with command line option typeMode 1 Create Table RTIDDS SUBSCRIPTIONS table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL domain id INTEGER NOT NULL topic name VARCHAR 20
99. if both columns are NULL or contain the value 2147483647 2 1 This is the DDS default value I A non zero value representing the number of seconds and nanoseconds for the lease duration Note The DDS_LivelinessQosPolicy kind is always set to DDS_AUTOMATIC_LIVELINESS_QOS See the Connext DDS documentation for more details on how this QoS policy may be used See also dw liveliness lease_dur Section 4 5 1 1 15 droownership kind Section 4 5 2 1 44 and dw ownership kind dw ownership_strength value Section 4 5 1 1 18 dr deadline period These columns specify the deadline period for the DDS_DeadlineQosPolicy for the DataReader created to subscribe to Topic data that is stored in the table By setting the values in this column the user is setting an expectation that DataWriters will publish new values for data instances at least as fast as the deadline period The possible values of the lt dr deadline period sec gt seconds and lt dr deadline period nsec gt nanoseconds columns are An infinite deadline period is specified if both columns are NULL or contain the value 2147483647 2 1 This is the DDS default value Meta Tables 4 5 2 1 43 4 5 2 1 44 1 A non zero value representing the number of seconds and nanoseconds for the deadline period See the Connext DDS documentation for more details on how this QoS policy may be used See also dw deadline period Section 4 5 1 1 16 dr history kind dr his
100. ility Availability is an essential requirement for most distributed real time applications Systems built in the Defense and Aerospace industries are typically safety critical and are required to operate in crisis situations In telecommunications a minute of system downtime may mean many mil lions of dollars in lost revenue With Database Integration Service automatic data caching and rep lication can serve as the foundation technology for high availability Applications can use Database Integration Service to maintain copies of SQL database tables on two or more hosts in the network In the event of a host failure copies of the tables are available from other hosts to con tinue operation Database Integration Service s automated replication management and no single point of failure guarantees the availability of critical information With Database Integration Service tables can be stored on multiple hosts allowing applications and services to concurrently read and write in multiple tables Conflict resolution can be based on application defined timestamps Additional Benefits of Database Integration Service IJ Achieve quick time to market Start application development immediately using well known interfaces Minimize time consuming custom programming Easily integrate into existing solutions using industry standard interfaces 1 Reduce development costs Use widely available modeling and database tools Eliminate expensi
101. in Chapter 5 IDL SQL Semantic and Data Mapping Then the daemon will proceed to create the associated DataWriter More about the creation of user tables by the daemon can be found in User Table Creation Section 4 6 How the daemon discovers and stores typecodes is described in Typecodes Section 4 1 4 If the Database Integration Service Daemon has not yet have a typecode associated with the topic_name it will defer the creation of the DataWriter until the typecode is discovered When a new type code is discovered the daemon will scan all rows in the RTIDDS_PUBLICATIONS meta table and create the user tables and DataWriters for entries that were pending on the discovery of the typecode The daemon will also create the DataWriter if there is an entry in the RTIDDS_PUBLICATIONS table without an associated typecode but the user subsequently creates the corresponding table If user applications modify an existing row in the RTIDDS_PUBLICATIONS table the Database Integration Service Daemon will first delete the DataWriter that was created for that entry if it exists and then go through the same process of trying to create the user table and DataWriter as if the row was newly inserted If user applications delete an existing row in the RTIDDS_PUBLICATIONS table the Database Integration Service Daemon will delete the associ ated DataWriter if it exists Meta Tables A flow chart describing this logic is provided below wait for chang
102. in the configuration file Section 4 5 1 1 10 idl member suffix max length INTEGER Yes Value specified in the configuration file Section 4 5 1 1 10 profile name VARCHAR 255 Yes boues a Rae tera Section 4 5 1 1 11 pub present access_scope VARCHAR 25 Yes INSTANCE PRESENTATION QOS Section 4 5 1 1 12 pub present ordered_access TINYINT Yes 0 false Section 4 5 1 1 12 pub partition name VARCHAR 256 Yes Empty string partition Section 4 5 1 1 13 dw durability kind VARCHAR 30 Yes VOLATILE_DURABILITY_QOS Section 4 5 1 1 14 dw liveliness lease_dur sec INTEGER Yes Infinite Section 4 5 1 1 15 dw liveliness lease dur nsec INTEGER Yes _ Infinite Section 4 5 1 1 15 dw deadline period sec INTEGER Yes _ Infinite Section 4 5 1 1 16 dw deadline period nsec INTEGER Yes Infinite Section 4 5 1 1 16 dw history kind VARCHAR 21 Yes KEEP LAST HISTORY QOS Section 4 5 1 1 17 dw history depth INTEGER Yes 1 Section 4 5 1 1 17 dw ownership kind VARCHAR 23 Yes SHARED OWNERSHIP QOS Section 4 5 1 1 18 dw ownership strength value INTEGER Yes 0 Section 4 5 1 1 18 dw publish_mode kind VARCHAR 29 Yes MODE D a Section 4 5 1 1 19 dw res limits max samples INTEGER Yes Infinite Section 4 5 1 1 20 dw res limits max instances INTEGER Yes Infinite Section 4 5 1 1 20 changes queue maximum size INTEGER Yes Infinite Section 4 5 1 1 21 RTIRTC SCN BIGINT Yes Next SCN number Section 4 5 1 1 22 a Primary key column 4 5 1 1 4 table owner tab
103. ion Service Daemon had a problem in generating the string for preparing or executing a SQL statement 21 Error skipping parameter for the field lt column gt 22 Error binding parameter for the column lt column gt Error binding parameters A 2 Table A 1 Database Integration Service Errors and Warnings Code Message Details Supported SQL types are SOL CHAR SOL WCHAR SOL VARCHAR SOL WVARCHAR SOL BINARY SOL VARBINARY 23 oo lt column gt has an unexpected SQL SOL_INTEGER yP SQL_SMALLINT SQL_TINYINT SQL_BIGINT SQL_REAL SQL_FLOAT SQL_DOUBLE SQL_TIMESTAMP 24 Error opening configuration file filename 25 Error reading configuration file filename 26 Error parsing configuration file filename 27 The maximum length for a lt type gt field is lt length gt eer A problem occurred when serializing a 28 Error serializing record table row for publishing as a Topic A problem occurred when deserializing 29 Error deserializing RTI DDS sample data received via Connext DDS for storing into a table Could not create cache of known instance 30 Error creating key cache keys see cache_maximum_size cache_initial_size Section 4 5 2 1 29 Problem occurred while storing instance 31 Error inserting key in cache key in cache see cache_maximum_size cache_initial_size Section 4 5 2 1 29 A precondition failed in which a NULL 32 Null p
104. ion for more details on how this QoS policy may be used See also dr res limits max samples dr res limits max instances Section 4 5 2 1 46 dr ownership kind These columns directly map to the DDS_OwnershipQosPolicy and DDS Ownership Strength QosPolicy for the DataReader created to subscribe to Topic data that is stored in the table These policies control whether or not the DataReader is allowed to receive changes to an instance of a Topic from multiple DataWriters simultaneously The possible values of the lt dr ownership kind gt column are L SHARED OWNERSHIP QOS default value if the column is NULL This setting allows the DataReader to receive updates for an instance of a Topic from multiple DataWriters at the same time Ll EXCLUSIVE OWNERSHIP QOS This setting prevents the DataReader from receiving changes from more than a single DataWriter for an instance of a Topic at the same time The DataReader will receive changes for a topic instance from the DataWriter with the greatest value of ownership strength If the liveliness of the DataWriter fails or if the DataWriter fails to write within a deadline period then the DataReader will receive pub lished changes to the topic instance from the DataWriter with the next highest ownership strength Meta Tables 4 5 2 1 45 4 5 2 1 46 4 5 2 1 47 4 5 2 1 48 See the Connext DDS documentation for more details on how this QoS policy may be used See also drliveliness e
105. ional Integration Section 3 4 LI Bridging between Domains Section 3 5 L High Rate Data Streams Cached before Storage Section 3 6 1 Real Time Database Replication Section 3 7 Database Integration Service Architecture The Database Integration Service architecture is designed to integrate existing systems that use the Connext DDS API or relational databases with minimal modification to working applications In many situations existing applications do not have to change at all As seen in Figure 3 1 Database Integration Service consists of a daemon that acts like bridge between two software development domains One uses the OMG Data Distribution Service API to publish and subscribe to data that may be generated at high rates with real time constraints The other applies algorithms and logic representing business processes to megabytes gigabytes or terabytes of data stored in relational databases Database Integration Service Daemon The Database Integration Service Daemon oversees the incoming subscribed data and outgoing published data It enables automatic storage of data published by Connext DDS applications in a database by mapping a Topic to a table in the database and storing an instance of a Topic as a row in that table Also the daemon can automatically publish changes in a database table as a Topic Users have total control of the Quality of Service that the daemon uses for publishing and subscribing to Connext DDS data
106. ished with the DataWriter Instead of using the system time when a row in the user table changes the Database Integration Service Daemon will take the current value of the designated column and use it in the DataWriter write_w_timestamp method when publishing the value of the row The possible values for the lt resolution_column gt column are LJ NULL If this column is NULL then the Database Integration Service Daemon will just call DDS DataWriter write to publish the table changes This implies that the source timestamp used by Connext DDS will be the system time when the write occurred LJ column_name The column name of any column in the user table that has a valid type The column must be one of the following SQL types INTEGER SMALLINT BIGINT or TIMESTAMP If the user directs the daemon to use a column from the user table as the timestamp then it is imperative to the proper operation of the publication that the value in the timestamp column is monotonically increasing with every table change So when a change is made to a row of the table the value in the column lt resolution_column gt must be larger than the last value of this column that was published The lt resolution_column gt can be used with the lt dr destination_order kind gt column of the RTIDDS_SUBSCRIPTIONS table to implement a conflict resolution policy in a system where Database Integration Service is used to implement database table replication acro
107. kground 3 5 Real Time Database Replication Figure 3 5 Bridging Domains 3 7 RTI Real Time Connect Daemon Embedded Embedded This solution enables the archival of high throughput Connext DDS data streams that would oth erwise be uncaptureable by standard database technologies Real Time Database Replication By running multiple Database Integration Service daemons on different nodes connected to differ ent databases and configuring all of the daemons to publish and subscribe to the same table changes made by applications to a table on one node can be automatically replicated to tables on all of the other nodes Figure 3 7 shows how Database Integration Service can be used to perform lazy table replication between distributed databases 3 6 Real Time Database Replication Figure 3 6 High rate Data Streams are Cached Before Storing onto Disk TimesTen Cache Connectto Oracle RTI Database Integration gt Service Daemon High Performance In Memory Database Table Tracks Table Passengers Fit Lat Long Fit Name Addrs C129 34 5 1023 C129 A Johnson C054 27 7 46 8 C054 J Smith Permanent disk High data rates storage With lazy replication an update is sent to the subscribers after the transaction is committed into the local database The advantages of lazy replication are short response time and high concur rency since locks in the data cache are immediately released after
108. l You should see these errors in your own applications as well as in the Database Integration Service Daemon if the daemon is subscrib ing to Topics and trying to store the received data in the database These ODBC errors will appear in the daemon log If you encounter this situation you should increase the value of XLABUFFERSIZE appropriately The Oracle TimesTen native error codes that you may see when the XLABUFFERSIZE is too small are l TT8009 Transaction Log API Buffer size too small or too large TT986 Log buffer overflow transaction must rollback L TT987 Log record larger than log buffer transaction must rollback 4 25 Meta Tables 4 5 4 5 1 More details about Oracle TimesTen error codes can be found in the Oracle TimesTen In Memory Database Error Messages and SNMP Traps Release 11 2 1 Meta Tables After the Database Integration Service Daemon has started and successfully made a connection to a database the user will still need to configure the daemon to publish table changes as Topics as well as subscribe to Topics for storing received data into a table This configuration is done by inserting entries into two tables RTIDDS_PUBLICATIONS and RTIDDS_SUBSCRIPTIONS These tables will be created by the Database Integration Service Daemon if they do not already exist in the database The two tables are referred to as meta tables since their data is not user data but information used by the daemon to create DataWriters
109. ld octet my field RAW 1 my_field bin octet x my field RAW x my_field bin sequence lt octet x gt my field RAW x if x lt 2000 otherwise BLOB my field boolean my field NUMBER 3 my field short my field NUMBER 5 my field unsigned short my field NUMBER 5 my field unsigned my field NUMBER 10 my field unsigned long my field NUMBER 10 my field double my field BINARY DOUBLE my field float my field BINARY FLOAT my field string lt x gt my_field VARCHAR2 x if x lt 4000 otherwise CLOB my field str wstring lt x gt my_field NVARCHAR2 x if x lt 4000 otherwise NCLOB my field str d long long my_field NUMBER 20 my_field unsigned long long my_field NUMBER 20 my_field long double my_field RAW 16 my field 1d unsigned long long my_field TIMESTAMP my field a The format on the wire of char and char x is the same b The format on the wire of wchar and wchar x is the same c The format on the wire of octet and octet x is the same d The bin suffix is only used for Oracle databases not Oracle TimesTen The reason is that Oracle doesn t support SOL BINARY x or VARBINARY x Binary data must be always stored as variable length raw data using the Oracle type RAW Thus Oracle RAW can be used to store IDL octet octet x and sequence lt octet x gt The bin suffix allows the daemon to determine which is
110. ld x unsigned short my field x smallint my field x long my field x int my field x unsigned long my field x int my field x a The column storing the last bit field in a set of bits will use name of my_field x Enum Types Mapping IDL enumeration fields are mapped to columns of type SOL INTEGER No special naming is required Simple IDL Structures Simple IDL structures containing only basic or primitive types directly map to SOL schemas with fields in the structure becoming columns in the table Table 5 11 shows the mapping of a simple structure between IDL and SOL Simple Structures in IDL and SQL IDL Types SOL Table Schema struct MyStruct CREATE TABLE MyStructContainer long my key field key my key field INTEGER NOT NULL Short my short field my short field SMALLINT NOT NULL PRIMARY KEY my key field yg a IDL fields marked as keys are mapped to the primary keys of SOL tables Complex IDL Structures IDL structures that contain more complex fields fields that are structures unions or sequences and arrays of types other than octet char or wchar are mapped to SQL tables by flatten ing the embedded structures so that their fields are all at the top and only level Structure fields Elements of embedded structures map into individual table columns with names that are hierar chically composed from the name of the field in the embedded structure and
111. le name These columns specify the user table for which changes will be published using a DataWriter Because a DBMS uses a combination of table owner table name to identify a table both of these columns must have valid values should the user want these entries to refer to an existing table If no table exists in the database with the identifier lt table_owner gt lt table_name gt at the time that the daemon sees this entry in the RTIDDS PUBLICATIONS meta table it will create a user table with this name automatically see User Table Creation Section 4 6 Note In MySQL the value of the table owner column corresponds to the table schema or data base name 4 31 Meta Tables 4 5 1 1 5 4 5 1 1 6 4 5 1 1 7 4 5 1 1 8 domain_id This column specifies the domain ID that will be used to publish changes in the table Before cre ating a DataWriter if no DomainParticipant has previously been created with the domain ID the Database Integration Service Daemon will create a DomainParticipant with the specified ID If the publications entry has associated a QoS profile Database Integration Service will use the val ues in this profile to create the participant The participant will also be configured using the QoS values of a profile when the attribute is_default_qos is set to 1 in that profile see the RTI Con next DDS Core Libraries User s Manual for additional details topic_name These column defi
112. lish mode kind This column controls the type of DataWriter that Database Integration Service will create for pub lishing data This column can take the following values 14 ASYNCHRONOUS PUBLISH MODE QOS I SYNCHRONOUS PUBLISH MODE QOS default value Asynchronous DataWriters were introduced in Connext DDS to support large data packets greater than 64Kb If IDL data types exceed the 64Kb limit and reliable communication is used dw publish mode kind must be set to ASYNCHRONOUS PUBLISH MODE QOS See the Connext DDS documentation for more details on the differences between synchronous and asynchronous DataWriters dw res limits max samples dw res limits max instances These columns set some parameters for the DDS ResourceLimitsQosPolicy for the DataWriter created to publish table changes In particular they control the amount of memory that the sys tem is allowed to allocate for storing published data values as well as the total number of data instances different primary keys that can be handled by the DataWriter A value of 1 for either of these columns means infinite An infinite setting means that the DataWriter is allowed to allocate memory as needed to store published table changes and man age new keys I The default value for dw res limits max samples if set to NULL is 32 LJ The default value for dw res limits max instances if set to NULL is 1 Meta Tables 4 5 1 1 21 4 5 1 1 22 4 5 2 The number of key
113. ll over write the oldest values stored Key IDL data types may con tain one or more fields that are used to distinguish dif ferent instances of the Topic Primary key Most relational data bases require table schemes to identify one or more columns to act as the primary key for the table Keys are mapped to the primary keys of a table When a table is created by the Database Integration Service Daemon the columns corresponding to the IDL key fields will be created as primary key columns For tables created by user code the correspon dence of IDL key fields to table primary key col umns must be set correctly Values published for Topics will be stored into a database table by the Database Integration Service Daemon DDSDataWriter write tau i Table rows modified by SQL INSERT or UPDATE commands will be published by the Database Integration Service Daemon as values of Topics DDSDataReader take DDSDataReader read AQL SELECT When SQL DELETE is used to delete a row from a table the Database Integration Service Daemon will call DDSDataWriter dispose to dispose the instance corresponding to the row DDSDataWriter dispose SOL DELETE If a user application calls DDSDataWriter dis pose to dispose an instance the Database Integra tion Service Daemon may be configured to delete or keep the corresponding rows 5 2 Data Representation Mapping 5 2 5 2 1
114. ls 10 The type of the column lt column gt is not valid The meta columns RTIDDS_DOMAIN_ID and RTIRTC_REMOTE must be added to tables that the user creates and wished to connect to via Database Integration Service They will be automatically if the Database Integration Service Daemon creates the table If the user creates the table and adds the two columns they must be of type INTE GER This message is produced if these columns exist and are of the wrong type 11 12 Error publishing record instance Error disposing record instance The Database Integration Service Daemon had a problem publishing a table change as a Topic The Database Integration Service Daemon had a problem disposing of an instance of Topic when the user deleted a row in a table 13 Error initializing module module The Database Integration Service Daemon had problems initializing an internal code module 14 The definition of environment variable s is required 15 Error opening the database connection associated to the DSN lt DSN gt 16 Error enabling database log 17 lt string gt too long maximum length lt length gt 18 Error creating connection to database log 19 The value of the column column in the table lt meta table gt is not valid 20 Error generating type SQL statement string Error generating SOL statement string The Database Integrat
115. lumn contains the TypeCode information associated to the user table identified by table owner table name The TypeCode information stored in this table is used when publications and subscriptions are created after the Database Integration Service Daemon is restarted Log Table A meta table named RTIRTC LOG is used to store log messages generated by the daemon Whether or not this table is created and used depends on the loglevel option see Section 4 2 and the LOGTODB and LOGHISTORY Database Integration Service Daemon connection attri butes see Section 4 4 4 2 Users should treat the contents of this table as read only There is no reason for users to modify this table The number of rows in the Log table is controlled by the LOGHISTORY connection attribute If set to 1 the table will hold as many log messages as generated by the Database Inte gration Service Daemon Otherwise the daemon will only store the last n log messages as speci fied by LOGHISTORY using the table as a circular buffer Users may use the id column to determine the last log message that was generated by the dae mon see Section 4 5 4 1 53 Log Table Schema The RTIRTC LOG table is created with the following SOL statement Oracle TimesTen with command line option typeMode 1 Create Table RTIRTC LOG id INTEGER NOT NULL ts TIMESTAMP NOT NULL type VARCHAR 7 NOT NULL function VARCHAR 64 NOT NULL line INTEGER Meta Tables code
116. me Example gt database mapping options identifier separator char D ou e database mapping options mysql connection lt dsn gt Example MySQL lt dsn gt lt user_name gt Student lt user_name gt lt password gt mypsswrd lt password gt lt mysql_connection gt oracle connection gt dsn Example Oracle lt dsn gt user name Student user name lt password gt mypsswrd lt password gt oracle connection timesten connection dsn Example TT dsn timesten connection real time connect dds Table 4 2 describe the top level tags allowed within the root dds tag Notice that the real time connect tag is required Table 4 2 Top Level Tags Number Tags within lt dds gt Description of Tags Allowed Specifies a QoS library and profiles The contents of this tag are specified in the same manner as for an Con next DDS QoS profile file see Chapter 15 in the RTI Connext DDS Core Libraries User s Manual The profiles you specify here can be used in three ways l By setting the attribute is default qos in the tag qos profile to true In this case that profile is the default configuration for qos library 0 or more all the Entities created by the Database Integration Service dae mon 1 By referring to a profile using the XML tag profile name within publication and subscription see Section 4 4 4 4 I By referring to a profile in t
117. mic loading of monitoring library is not sup ported RTI Monitoring Library is statically linked In Database Integration Service there is no need to load this library dynamically 32769 Requested incompatible QoS The QoS of a Database Integration Service subscription is incompatible with the QoS of a Connext DDS publication The name of the policy that is incompatible is shown in this warning message 32770 Offered incompatible QoS The QoS of a Database Integration Service publication is incompatible with the QoS of a Connext DDS subscription The name of the policy that is incompatible is shown in this warning message 32771 Sample lost message A Database Integration Service subscription lost a sample 32772 Sample rejected message A Database Integration Service subscription rejected a sample A 7 Appendix B Database Limits Table B 1 The maximum number of columns is limited by the underlying database product The maxi mum length of a column is independent of the database and it is limited to 30 characters Table B 1 notes the database limits of Database Integration Service Real Time Connect Database Limits Oracle Microsoft SOL Oracle 11g TimesTen 11 2 1 MySQL 5 1 Server 2012 For subscriptions Maximum 1000 PA ei m number of For publications 1 000 y 1 024 considerably smaller see columns 625 650 Section B 2 See Secti
118. multiple lt real_time_connect gt tags one file can be used to configure multiple daemon executions When you start RTI Real Time Connect you have to use the cfgName option to specify which lt real_time_connect gt tag to use For example lt dds gt real time connect name rtcA gt real time connect real time connect name rtcB gt Jresi t ins connect gt lt dds gt Starting Database Integration Service with the following command will use the lt real_time_connect gt tag with the name rtcA rtirtc_mysql cfgFile example xml cfgName rtcA If there is no lt real_time_connect gt tag matching the name provided with the command line option cfgName the daemon will report an error and it will list the available configurations 4 14 Configuration File 4 4 4 Database Configuration Using the Database Integration Service XML Tag Table 4 3 describes the tags allowed with the lt real_time_connect gt section of the XML file Table 4 3 Database Integration Service Tags Tags within zxeal tune connedis Description Number of Tags Allowed Configures how the IDL identifier names are mapped to the database column names 0 or 1 See Section 4 4 4 2 database mapping options Contains attributes that are independent of any particular database connection made by the Database Integration Service Daemon See Section 4 4 4 1 general options Oor1 1 or more required
119. multiple DataWriters with the same virtual GUID in the system old samples will not be stored in the database A sample with sequence number sn will be ignored if a sam ple with sequence number sn 1 for the same virtual writer has been already stored in the data base persist state This column specifies whether or not the state of a Database Integration Service subscription must be persisted into the database The field only applies when filter duplicates Section 4 5 2 1 33 is set to 1 The subscription state is used on restore by Database Integration Service in order to avoid receiving duplicate samples Meta Tables 4 5 2 1 36 4 5 2 1 37 4 5 2 1 38 sub present access_scope sub present ordered_access These two columns map directly to the DDS_PresentationQosPolicy of the DDS_SubscriberQos used by the Subscriber that is created with the DataReader for storing received data in the table The DDS_PresentationQosPolicy specifies how the data instances sent by a publishing applica tion are ordered before they are received The values of these columns must be coordinated with the values of the DDS_PresentationQosPolicy used by the Publisher in the sending application or else published data may not be received by the DataReader The possible values for the lt sub present access_scope gt column are INSTANCE PRESENTATION QOS This also the default value if the column is NULL TOPIC PRESENTATION QOS LJ GROUP PRE
120. n 4 5 2 1 45 dr time filter min sep nsec INTEGER YES 0 Section 4 5 2 1 45 dr res limits max samples INTEGER YES Infinite Section 4 5 2 1 46 dr res_limits max_instances INTEGER YES Infinite Section 4 5 2 1 46 dr unicast receive_port INTEGER YES 0 Section 4 5 2 1 47 dr multicast_receive_address VARCHAR 15 YES None Section 4 5 2 1 48 4 43 Meta Tables Table 4 18 RTIDDS SUBSCRIPTIONS Table Schema Column Name SQL Type Nuil Default if NULL Described in dr multicast receive_port INTEGER YES l0 Section 4 5 2 1 49 RTIRTC_SCN BIGINT YES Next SCN number Section 4 5 2 1 50 a Primary key column 4 5 2 1 23 4 5 2 1 24 4 5 2 1 25 4 5 2 1 26 4 5 2 1 27 table_owner table_name These columns specify the user table into which data received by a DataReader will be stored Because a DBMS uses a combination of lt table_owner gt lt table_name gt to identify a table both of these columns must have valid values should the user want these entries to refer to an existing table If no table exists in the database with the identifier lt table_owner gt lt table_name gt at the time that the daemon sees this entry in the RTIDDS SUBSCRIPTIONS meta table it will create a user table with this name automatically see User Table Creation Section 4 6 Note In MySQL the value of the table owner column corresponds to the table schema or data base name domain id This column specifies
121. n Mapping definitions Thus user applications should also use quoted strings when referring to those col umn and table names in their SOL statements Ordinarily the name of a field in an IDL data structure can just be used as the name of a column in a table In fact for those data types with clear and obvious mappings the column name can be independent of the field name used in the IDL type However because there is no one to one mapping of all IDL data types to all SQL data types for certain types the column names used in SQL table schemas must follow certain conventions that tie them to the names of the fields of IDL types from which they are mapped This is true for only the small subset of primitive IDL data types and for the complex IDL data types that would otherwise have ambiguous map pings i e multiple ways to map IDL to SQL or vice versa The Database Integration Service Daemon scans for identifies and uses special mappings of col umn names when serializing and deserializing IDL data to and from a table in a database There are two types of special mappings hierarchical naming and suffixes Hierarchical Naming Complex IDL types may have fields that are actually embedded structures so a field may actu ally contain multiple values In SQL columns usually contain a single value for each column element although there are a few types like BINARY x CHAR x VARBINARY x and VAR CHAR x that can store multiple values of th
122. n the XML configuration file See User Table Creation Section 4 6 for more details table history depth This column determines the number of values of each instance received by the DataReader that can be stored in the table by the Database Integration Service Daemon For non keyed Topics there is only a single instance thus the table history depth would correspond to the maximum size of the table in rows Meta Tables For keyed Topics the Database Integration Service Daemon may store up to lt table_history_depth gt values of each instance of the Topic that the DataReader receives When the history depth is reached the rows are reused as a circular buffer with the newest values replacing the oldest To support this capability the associated user table may be created with additional columns meta columns to help the Database Integration Service Daemon manage history for a table Whether or not meta columns need to be added to support table history is based on the value of the entry in lt table_history_depth gt The two meta columns for supporting table history are J RTIRTC HISTORY SLOT INTEGER This column is also added to the Primary Key of the table There is usually no need for users to access this column it is only used by the daemon It is only needed since many DBMS systems do not allow you to alter the value of a Primary Key column J RTIRTC HISTORY ORDER INTEGER This value of this column is maintained b
123. n the new data is dropped What this column does is to set which timestamp the one associated with the source of the data when it was sent or the one associated with the data when it was received the DataReader will use This column has no practical effect unless the value of the dr ownership kind column is SHARED OWNERSHIP QOS The possible values for the dr destinaton order kind column are LJ BY RECEPTION TIMESTAMP DESTINATIONORDER QOS default value if the column is NULL This configures the DataReader to use the timestamp of when the data was received to determine whether or not to drop the data In practice this setting means all data received from all DataWriters will be accepted since the timestamp will always be newer for the new data L BY SOURCE TIMESTAMP DESTINATIONORDER QOS Meta Tables 4 5 2 1 41 4 5 2 1 42 This value means that the DataReader will use the timestamp that was sent with the data in determining whether or not to accept the data This timestamp was added by the DataWriter when the data was published Because different DataWriters may run in applications on different machines it is likely that the clocks on the different machines are only synchronized to a certain resolution or not synchronized at all Thus the DataReader may receive data with timestamps older than the last data that received and thus drop those data However if all DataReaders of the same Topic used the source times
124. name and configure the connec tion The database connection tags are DBMS specific L mysql connection oracle connection l timesten connection A real time connect tag may have multiple database connec tion tags The DBMS specific Database Integration Service Daemon will only parse the tags that apply to it As explained earlier the Database Integration Service Daemon will make a connection to a database using the DSN attribute for every connection tag that it parses This is the only way to direct the daemon to connect to a database No other connections will be made after startup Example real time connect name MyRtc gt mysql connection lt dsn gt Example MySQL lt dsn gt user name gt Student lt user name gt lt password gt mypsswrd lt password gt lt send_period gt 100 lt send_period gt database logging lt enabledstrue lt enabled gt history depth gt 100 lt history depth database logging publications lt publication gt lt publication gt lt publications gt lt subscriptions gt lt subscription gt lt subscription gt lt subscriptions gt lt mysql_connection gt lt real_time_connect gt lt database_connection gt lt dsn gt user name lt password gt Table 4 9 describes tags allowed within all three types of lt database_connection gt tags Tables 4 10 through 4 12 describe additional tags for each connection ty
125. nes the Topic that will be used to publish the changes in the associated table The lt topic_name gt need to match the Topic used by subscriptions in user applications that expect to received data changes from the table If the Database Integration Service Daemon has discovered the typecode associated with the lt topic_name gt and the user table does not exist in the database the daemon will use the typecode to create the table using entries in the lt table_owner gt and lt table_name gt column See User Table Creation Section 4 6 for more details type_name This column defines the registered name of the type associated with the Topic defined using the column lt topic_name gt If the user table does not exist in the database the daemon will use the type name to find a typecode in the XML configuration file See User Table Creation Section 4 6 for more details table_history_depth The lt table_history_depth gt column in the RTIDDS_PUBLICATIONS determines whether or not the Database Integration Service Daemon will create the user table with additional meta col umns that support the storing of historic or past values of instances of Topics by DataReaders created with the RTIDDS_SUBSCRIPTIONS table It is only used if the daemon creates the table because it does not exist More about the ability to store historic data in the table as well as the added meta columns can be found in table_history_depth Section 4 5 2 1 27 and in User
126. ng of tables values that were changed because they were received via Connext DDS User applications that create the table do not need to add these columns if the dae mon is configured only to publish data from the table or to store data into the table However it is essential that these columns do exist for the situation where both publica tions and subscriptions are tied to the same table If the meta columns are omitted then when Database Integration Service Daemon receives data via Connext DDS it will be echoed republished as a change to the table RTIRTC KEY This SOL INTEGER column is added by the daemon if the IDL type that is used to create the table does not contain any fields marked as a topic key i e non keyed IDL types In such cases the lt RTIRTC_KEY gt column will be added to the table as the primary key column The value in that column will always be 0 Thus there is only a single instance of the Topic which means the table will only ever have a single row subject to whether or not the user wants the table to store historical value of data instances see the details for the lt RTIRTC_ HISTORY_SLOT gt and RTIRTC HISTORY ORDER meta columns below If the IDL type does have key fields then the fields will be mapped into columns that are marked as primary keys This meta column is not added and the table can contain as many rows as there are different instance keys primary keys LJ RTIRTC HISTORY SLOT and RTIR
127. nges in the database Supported QoS attributes include reliabil ity durability multicasting delivery ordering and many others Capturing Real Time Data in a DBMS Figure 3 2 shows how Database Integration Service can be used to capture real time data streams generated by embedded Connext DDS applications into one or more tables in a in memory DBMS In this scenario the Database Integration Service Daemon has been configured with user customizable QoSs to subscribe to Topics When new values arrive the daemon stores the data in the appropriate table in the database Mapping the Topic described by IDL to the equivalent SOL table schema is done automatically by the daemon with no user configuration necessary Figure 3 2 Storing Published Connext DDS Data in a SQL Database RDBMS App RDBMS App SQL SELECT write RTI Database Integration Service Daemon SQL N UPDATE INSERT Table Tracks Table Passengers Fit Lat Long Fit Name Addrs write C129 34 5 102 3 C129 A Johnson C054 277 468 C054 J Smith Embedded I Mc Enterprise 3 3 Remote Real Time Notification of Table Changes 3 3 Figure 3 3 3 4 Remote Real Time Notification of Table Changes Figure 3 3 shows how Database Integration Service can be used to notify remote Connext DDS applications running in embedded devices of time critical changes in the database In this sce nario the Database Integration Service Daemon has been configure
128. o the DDS_OwnershipQosPolicy and DDS_Ownership Strength QosPolicy for the DataWriter created to publish table changes These policies control whether or not DataReaders are allowed to receive changes to an instance of a Topic from multiple DataW riters simultaneously The possible values of the lt dw ownership kind gt and lt dw ownership_strength value gt columns are L SHARED_OWNERSHIP_QOS default value if the column is NULL This setting allows DataReaders to receive updates for an instance of a Topic from multi ple DataWriters at the same time Ll EXCLUSIVE OWNERSHIP QOS This setting prevents a DataReader from receiving changes from more than a single DataWriter for an instance of a Topic at the same time The DataReader will receive changes for a topic instance from the DataWriter with the greatest value of ownership strength If the liveliness of the DataWriter fails or if the DataWriter fails to write within a deadline period then the DataReader will receive pub lished changes to the topic instance from the DataWriter with the next highest ownership strength The ownership strength set is set in the dw ownerhip strength value column The default value is 0 if the column is NULL See the Connext DDS documentation for more details on how this QoS policy may be used See also dw liveliness lease dur Section 4 5 1 1 15 dw deadline period Section 4 5 1 1 16 and dr ownership kind Section 4 5 2 1 44 dw pub
129. of these messages will be logged for a single problem A failure at a lower layer will cause log messages to be printed at various levels of the Database Integration Service Daemon logic These messages will be valuable to you and to RTI support engineers in debugging issues with Database Integration Service Table A 1 Database Integration Service Errors and Warnings Code Message Details 0 1023 Database Integration Service Daemon errors These messages are produced by the logic of the Database Integration Service Daemon itself Should never occur 0 Unexpected error Contact support rti com if seen 1 message General error There was an error when storing value 2 Error storing RTI DDS sample in table lt table gt received with Connext DDS into the data base 3 Error creating lt entity gt 4 Error creating lt entity gt associated to the table lt table gt 5 Error getting lt entity gt There was an entry in a meta table 6 lt meta table gt entry not valid RTIDDS_PUBLICATIONS or RTIRTCSUB SCRIPTIONS that was not valid There was an error when creating or pre 7 Error creating lt type gt SQL statement paring a SQL statement 8 Error creating table lt table gt 9 Error opening RTI DDS connection There was a problem initializing Connext DDS A 1 Table A 1 Database Integration Service Errors and Warnings Code Message Detai
130. ointer argument pointer was passed to an internal daemon function 33 Error reading table lt table gt The RTIDDS PUBLICATIONS table con tained an entry in the resolution column The resolution column lt column gt does not exist in l 34 h which does not match the name of an exist the table lt table gt ing column in the corresponding table See resolution_column Section 4 5 1 1 9 The type of the resolution column lt column gt in the A column specified in the column table lt table gt is not valid The type of the resolu resolution_column in the 35 tion column can be SQL_INTEGER RTIDDS_PUBLICATIONS table is not of an SOL SMALLINT SOL BIGINT and acceptable type See resolution column SOL TIMESTAMP Section 4 5 1 1 9 Error gathering Connext DDS instance 36 Error gathering instance information information through the execution of the associated SELECT statement The schema of the metatables is not valid 37 Invalid metatable schema It is possible that those tables were created with a previous version of RTI RTC A 3 Table A 1 Database Integration Service Errors and Warnings Code Message Details 38 Error deleting key from cache 39 Error deleting a row from lt table gt There woes problem deleting row froma user data table The user tried to create a publication sub A Error creating publication subscription for the scription for a
131. on B 1 Maximum column name 30 30 30 30 gt length characters CLOB BLOB Not supported by Dor supported by RIC Maximum record i YES databa maximum record size is ize is 8 000 bytes suppor atabase 65535 bytes size is 8 y CHAR maximum 2000 8300 2554 8 000 size bytes VARCHAR maximum 4000 4194304 65535 d 8 000 size bytes BINARY maximum 2000 8300 2554 8 000 size bytes VARBINARY maximum 2000 4194304 65535 d 8 000 size bytes a Limit is imposed by Real Time Connect not MySQL which allows column names of up to 64 characters b Limit is imposed by Real Time Connect not SQL Server which allows column names of up to 128 characters c varbinary max and varchar max are not supported by Real Time Connect d The maximum size of a row in MySQL 5 1 is limited to 65535 For example you cannot have two fields of type VARCHAR 40000 because the total width of the columns would exceed 65535 bytes For additional information on this restriction see http dev mysql com doc refman 5 1 en column count limit html B 1 Maximum Columns for Oracle 11g B 1 Maximum Columns for Oracle 11g For Database Integration Service subscriptions the maximum number of columns is 1000 This limit is imposed by the maximum number of columns in an Oracle 11g table For Database Integration Service publications the maximum number of columns is limited by the maximum size of the PL SQL programs that Databas
132. orm data model Flexibility and Scalability By leveraging Connext DDS Quality of Service QoS settings Database Integration Service sup ports an unprecedented variety of deployment configurations to accommodate a wide range of scenarios from reliable point to point delivery to best effort multicasting that enables real time transaction streaming to large numbers of subscribers By setting QoS policies system through put response time reliability footprint and network bandwidth consumption can be tuned to meet application requirements Previously a system was hard coded with parameters set for a specific operation profile during integration In contrast Database Integration Service provides run time configurable policy settings which greatly enhances system deployment flexibility Matching Real Time Performance Database Integration Service integration with the Oracle TimesTen In Memory Database allows the user to capture data into standard relational databases at rates far exceeding most applica tion requirements Data in Oracle TimesTen can be synchronized in the background to a perma nent Oracle database using the TimesTen option Cache Connect to Oracle Thus the in memory database can act as a fast data cache in front of terabytes of data storage and tremendous analysis capability of a disk based database Database Integration Service solves the impedance mismatch between real time and enterprise applications High Availab
133. ows or in the file listener ora located at DRACLE HOME network admin using the ENVS parameter Additional information on how to load external procedures can be found in the Oracle manual by following this URL http download oracle com docs cd E11882 01 app dev 112 e10471 adfns externproc htm Important With either option if lib rtirti oracleq so dll is not located in SORACLE HOMBE bin the rtirtc oracle daemon executable must be started with the additional command line option queuelibpath directory containing lib rtirti oracleq so d11 see Command Line Parameters Section 4 2 Installing lib nddsc so dll and lib nddscore so dll on the Oracle Server The shared library lib rtirti oracleq so dll installed in the previous section will need access to additional shared libraries provided by Connext DDS The libraries libInddscl so dll and lib nddscore so dll should be copied to the server host from the appropriate lib lt platform gt directory in the installation of Connext DDS Then follow the procedure below to add these files to the library search path for the Oracle server UNIX based Systems The directory containing the Connext DDS libraries should be added to the environment variable LD LIBRARY PATH This environment variable must be set in the environ ment of the user who started the Oracle server A better method for setting this environment variable is in the extproc ora file or with the
134. pe 4 20 Configuration File Table 4 9 Common Tags for all Database Connections Common Tags for mysql connection oracle connection timesten connection Description Number of Tags Allowed database logging enabled history depth If enabled the Database Integration Service Daemon s log messages will be stored in a table named RTIRTC LOG in the database specified by the DSN Optionally you can specify the history depth of the log This value limits the size of the table RTIRTC LOG in the database that the daemon uses for logging messages The default is 1000 rows and a value of 1 implies no limit When the table is filled new log messages will replace the oldest messages effectively using the table as a circular buffer Default disabled Oor1 dsn You must specify a valid DSN that is found in a ODBCINI file or the Windows registry see Environ ment Variables Section 4 3 The Database Integration Service Daemon will make a connection to this DSN Database Integration Service detects changes in an Ora cle TimesTen data store by reading from the transac tion log Consequently Database Integration Service will not work for DSN configurations where logging is turned off Logging 0 This limitation applies only to Oracle TimesTen In Memory Database not to Oracle Database 11g or MySQL 1 required password Specifies the password to connect to th
135. rtc mysglq so dll This library is distributed with Database Integration Service and can be found in the lib plat form directory of the installation directory The correct version of the library to use depends on the platform on which MySQL server is running For example platform can be I x64Linux2 6cc4 1 1 for Red Hat Enterprise Linux 5 systems on 64 bit x86 processors LI i86Linux2 6cc4 1 1 for Red Hat Enterprise Linux 5 systems on 32 bit x86 processors LI i86Win32 for Windows systems on 32 bit x86 processors To install lib rtirtc mysqlq so dll copy the appropriate version of lib rtirtc mysqlq so dll into the MySQL server s plugin directory the directory named by the plugin dir system vari 4 4 Introduction to the Database Integration Service Daemon 4 1 2 3 4 1 2 4 able The plugin directory can be changed by setting the value of plugin_dir when the MySQL server is started For example you can set its value in the my cnf configuration file mysqld plugin _dir path to plugin directory For additional information about the plugin directory see the following link http dev mysql com doc refman 5 1 en install plugin html Installing libnddsc so dll and libnddscore so dll on the MySQL Server Since the library librtirtc mysqlq so dll internally uses Connext DDS the corresponding shared libraries libInddsc so dll and lib nddscore so dll distributed with Connext DDS also need to be install
136. rvice correctly with an Oracle database you must complete the pro cedures described in this section These procedures are not needed when using Database Integration Service with the Oracle TimesTen In Memory database To work with an Oracle database there is a shared library distributed with Database Integration Service that must be installed correctly on the host of the Oracle database server Communication by the Database Integration Service Daemon with the Oracle server is accomplished through exter nal procedures executed by the server when triggers installed by the daemon are fired These external procedures are provided in the shared library on UNIX based systems or DLL on Windows systems called lib rtirtc oracleq so dll This library is distributed with Database Integration Service and can be found in the lib architec ture directory of the installation directory The correct version of the library to use depends on the platform on which Oracle server is running For example architecture can be LI x64Linux2 6cc4 1 1 for Red Hat Enterprise Linux 5 systems on 64 bit x86 processors 1 i86Linux2 6cc4 1 1 for Red Hat Enterprise Linux 5 systems on 32 bit x86 processors L i86Win32 for Windows systems on 32 bit x86 processors Since the library lib rtirtc oracleq so dll internally uses Connext DDS the corresponding shared libraries lib nddsc so dll and lib nddscore so dll distributed with Connext DDS must also b
137. s and SQL familiarity with the ODBC API IDL and the Connext API and a working knowledge of the C C programming languages Paths Mentioned in Documentation The documentation refers to L lt NDDSHOME gt This refers to the installation directory for Connext DDS The default installation paths are Mac OS X systems Applications rti connext dds version e UNIX based systems non root user home your user namelrti connext dds version 1 1 Paths Mentioned in Documentation e UNIX based systems root user lopt rti connext dds version Windows systems user without Administrator privileges your home directory Nrti connext dds version Windows systems user with Administrator privileges C Program Files rti_connext_dds version for 64 bits machines or C Program Files x86 Nrti connext dds version for 32 bit machines You may also see NDDSHOME or NDDSHOME which refers to an environment variable set to the installation path Wherever you see lt NDDSHOME gt used in a path replace it with your installation path Note for Windows Users When using a command prompt to enter a command that includes the path C Program Files or any directory name that has a space enclose the path in quotation marks For example C Program Files rti_connext_dds version bin rtiddsgen or if you have defined the NDDSHOME environment variable NDDSHOME bin rtiddsgen RTI Workspace directory rti_work
138. s of this table are described in the next section You may insert new rows or modify the column values of existing rows in this table at any time For a new row the daemon will first check to see if the designated user table exists If so it will immediately create the DataReader with the QoS values specified by the entry The name of the Topic to subscribe to may be specified by the topic name column or automatically constructed as table owner table name if the topic name entry is NULL If the user table does not exist the Database Integration Service Daemon will look for the typecode associated with the type defined in the topic name column If it finds the typecode the daemon will create the user table with a SOL table schema derived from the typecode following the IDL type to SOL type mapping described in Chapter 5 IDL SOL Semantic and Data Mapping Then the daemon will proceed to create the associated DataReader More about the creation of user tables by the daemon can be found in User Table Creation Section 4 6 How the daemon discovers and stores typecodes is described in Typecodes Section 4 1 4 If the Database Integration Service Daemon does not yet have a typecode associated with the topic name it will defer the creation of the DataReader until the typecode is discovered When a new typecode is discovered the daemon will scan all rows in the RTIDDS SUBSCRIPTIONS meta table and create the user tables and DataReaders for entr
139. s that the DataWriter is allowed to manage places an upper limit on the num ber of rows that the related table in the database can have See the Connext DDS documentation for more details on how this QoS policy may be used changes_queue_maximum_size This column is available only for connections to a MySQL database The value of the column configures the maximum size of the queue that maintains the list of uncommitted changes Note that there is a separate queue per table A value of 1 is used to indicate unlimited size RTIRTC_SCN The System Change Number SCN column is available only for connections to a MySQL data base The value of this column is automatically maintained by Database Integration Service and is usually of no interest to the application For more information about the RTIRTC_SCN column see Section 4 6 Subscriptions Table When entries rows are added to the meta table RTIDDS_SUBSCRIPTIONS the Database Inte gration Service Daemon will try to create a DataReader and Subscriber along with a DomainPar ticipant if required and use it to receive data via the Connext DDS for a Topic and store values into the designated user table If the RTIDDS_SUBSCRIPTIONS table does not exist at startup the Database Integration Service Daemon will create it with the table owner set to the user name of the database connection as specified in the daemon s configuration file see Section 4 4 The schema and meaning of the col umn
140. s to store at the time the entry is made LIn gt 1 Meta columns will be added to accommodate the storing of historic values for instances The last n values received for an instance will be stored by the table LI 1 Meta columns will be added to accommodate the storing of historic values for instances All values received by the DataReader will be stored by the table See User Table Creation Section 4 6 for more information on meta columns Meta Tables 4 5 2 1 28 process_batch process_period commit_type These columns allow users to tune the Database Integration Service Daemon for optimal through put performance When the daemon receives data from a DataReader it may be configured to delay storing the data into a table and or committing the transaction until more data arrives For a data streams with high throughput thousands of samples per seconds the ability for the daemon to process incoming data in batches greatly improves the efficiency and ultimately the maximum sustainable throughput rate for a given Topic The trade off is latency The more data that is processed in a single batch the more efficiently the processing can occur However A greater delay between the receiving of the data by the dae mon and the time that it can be accessed by user applications in the database The column lt process_batch gt controls how many data samples are processed at a time by the Database Integration Service Daemon Instead of ex
141. sic Types in IDL and SQL Oracle on page 5 7 1 Table 5 4 Basic Types in IDL and SQL MySQL on page 5 8 1 Table 5 5 Basic Types in IDL and SQL SQL Server on page 5 8 Table 5 2 Basic Types in IDL and SQL TimesTen IDL SQL Type SOL Type Table IDL Type Field Name TypeMode 0 TypeMode 1 Column Name char my field TT CHAR 1 CHAR 1 my field or CHAR 1 char x my_field TT_CHAR x CHAR x my field or CHAR x sequence lt char x gt my_field TT_VARCHAR x VARCHAR x my_field or VARCHAR2 x wchar P my field TT NCHAR x NCHAR 1 my field or NCHAR 1 wcharl x b my_field TT_NCHAR x or NCHAR x my_field NCHAR x sequence lt wchar x gt my_field TT_NVARCHAR x NVARCHAR x my_field or NVARCHAR2 x octet my_field BINARY 1 BINARY 1 my field octet x my field BINARY x BINARY x my field sequence lt octet x gt my field VARBINARY x VARBINARY x my field boolean my field TT TINYINT TINYINT my field short my field TT SMALLINT SMALLINT my field unsigned short my field TT SMALLINT SMALLINT my field unsigned my field TT INTEGER INTEGER my field unsigned long my field TT INTEGER INTEGER my field double my field BINARY DOUBLE DOUBLE my field float my field BINARY FLOAT REAL my field string lt x gt my_field TT_VARCHAR x or VARCHAR x my_field str d VARCHAR2 x wstring l
142. significant When mapped to SQL a full primitive SOL type is used to store the value but only a subset of the bits are sig 5 9 Data Representation Mapping nificant A suffix must be added to the column name to indicate to the Database Integration Ser vice Daemon which bits to serialize when translating the table data into an IDL structure The following tables show the mapping of bit fields between IDL and SQL LJ Table 5 7 Bit Fields in IDL and SQL TimesTen LJ Table 5 8 Bit Fields in IDL and SQL Oracle LJ Table 5 9 Bit Fields in IDL and SOL MySQL LJ Table 5 10 Bit Fields in IDL and SQL SQL Server Table5 7 Bit Fields in IDL and SQL TimesTen IDL SOL Type SOL Type Table Column IDL Type Field Name TypeMode 0 TypeMode 1 Name TT CHAR 1 char my field x Or CHAR 1 my field x CHAR 1 TT_NCHAR 1 wchar my_field x or NCHAR 1 my_field x NCHAR 1 octet my field x BINARY 1 BINARY 1 my field x short my field x TT SMALLINT SMALLINT my field x unsigned short my field x TT SMALLINT SMALLINT my field x long my field x TT INTEGER INTEGER my field x unsigned long my field x TT INTEGER INTEGER my field x a The column storing the last bit field in a set of bits will use name of my_field x Table 5 8 Bit Fields in IDL and SQL Oracle IDL Type IDL Field Name SQ
143. space The RTI Workspace is where all configuration files for the applications and example files are located All configuration files and examples are copied here the first time you run RTI Launcher or any script in NDDSHOME bin The default path to the RTI Work space directory is Mac OS X systems Users your user namelrti workspace e UNIX based systems home your user namelrti workspace Windows systems your Windows documents folderNrti workspace Note your Windows documents folder depends on your version of Windows For example on Windows 7 the folder is C Users your user name Documents on Windows Server 2003 the folder is C Documents and Settings your user name Documents You can specify a different location for the rti_workspace directory See the RTI Connext DDS Core Libraries Getting Started Guide for instructions lt path to examples gt Examples are copied into your home directory the first time you run RTI Launcher or any script in lt NDDSHOME gt bin This document refers to the location of these examples as path to examples Wherever you see path to examples replace it with the appropri ate path 1 2 Background Reading 1 3 By default the examples are copied to rti_workspace version examples So the paths are Mac OS X systems Users your user namelrti workspace versionlexamples e UNIX based systems home your user name rti_workspace version examples Windows systems your
144. ss a network See dr destination_order kind Section 4 5 2 1 40 and TableReplicationMode on page 4 16 for more information idl member prefix max length idl member suffix max length These columns define how Database Integration Service maps IDL member identifiers into col umn names In particular they control how the column names are formed by using as a prefix n characters from the identifier s prefix and m characters from the identifier s suffix They can assume any value greater than or equal to 1 They cannot both be set to zero If a positive value n is provided for idl member prefix max length Database Integration Service will use the first n characters from the IDL member identifier to compose the associated column name A value of 0 tells Database Integration Service to compose the column name using only the last characters of the identifiers as defined by the idl member suffix max length column A value of 1 instructs Database Integration Service to use all the available characters If a positive value n is provided for idl member suffix max length Database Integration Service will use the last n characters from the IDL member identifier to compose the associated column name A value of 0 tells Database Integration Service to compose the column name using only the first characters of the identifiers as defined by the idl member prefix max length column A value of 1 instructs Database Integration Service to use all the av
145. t See Section 4 5 1 lt table_history_depth gt lt type_name gt Setting the XLA Staging Buffer Size for Diskless Connnections For diskless connections the minimum size of the Oracle TimesTen XLA staging buffer must be at least as big as the largest single SQL transaction executed through the database connection The size of a transaction is related to the aggregated size of data values that have changed as a result of the transaction For example if a transaction modifies every column in a row of a table then the transaction is at least as large as the size in bytes of a row Note that a single transaction may modify multiple rows of a table It is up to the user to deter mine what the largest transaction size in a database may be and set the XLABUFFERSIZE attri bute for the Database Integration Service Daemon appropriately However the maximum transaction size is only the minimum value that should be set for XLABUFFERSIZE You may need to set XLABUFFERSIZE to be much larger since the XLA staging buffer in Oracle TimesTen must hold more than a single non committed transaction simultaneously if there are multiple threads or processes accessing the same database at the same time Recall that this dis cussion only pertains to Oracle TimesTen DSNs that specify a diskless connection If the XLABUFFERSIZE is too small then SQLExecute or SOLExecDirect statements that were working will return an error indicating that a buffer is ful
146. t x gt my_field TT_NVARCHAR x NVARCHAR x my_field str or NVARCHAR2 x long long my_field TT_BIGINT BIGINT my_field 5 6 Data Representation Mapping Table 5 2 Table 5 3 Basic Types in IDL and SQL TimesTen Continued IDL SQL Type SOL Type Table IDL Type Field Name TypeMode 0 TypeMode 1 Column Name unsigned long long my_field TT_BIGINT BIGINT my_field long double my_field BINARY 16 BINARY 16 my field 1d unsigned long long my field TT TIMESTAMP TIMESTAMP my field a The format on the wire of char and char x is the same b The format on the wire of wchar and wchar x is the same c The format on the wire of octet and octet x is the same d The str suffix is used to distinguish between w string lt x gt and sequence lt w char x gt e The 1d suffix is used to distinguish between octet x and long double Basic Types in IDL and SQL Oracle IDL Field Table Column IDL Type Name SOL Type Name char my field CHAR 1 my_field char x my_field CHAR x my field sequence lt char x gt my_field VARCHAR2 x if x lt 4000 otherwise CLOB my_field wchar my field NCHAR 1 my field wchar x b my_field NCHAR x my field sequence lt wchar x gt my field NVARCHAR2 x if x lt 4000 otherwise NCLOB my fie
147. ta structure The exact mapping Defined by table schema of IDL data structures to table schemas is described in Data Representation Mapping Sec tion 5 2 Data structures Defined by IDL Interface Description Language 5 1 Semantic Mapping Table 5 1 Connext DDS DBMS Semantic Models Connext DDS Relational Database Details Topic Table Topic names and table names do not have to be Identified by a name Identified by a name the same when making a correspondence string string between a Topic and a database table DataWriter can publish Applications can write values for Topics and DataReaders can subscribe values or read values from tables using SQL Data values Rows in table No history A single row in a table History Multiple rows in a table When the Database Integration Service Daemon table history option is turned OFF see Sections 4 5 1 1 8 and 4 5 2 1 27 only the last value of a topic instance is stored in the table So a non keyed topic will be stored in a single row whereas for keyed topics there will be as many rows as there are topic instances When the Database Integration Service Daemon table history option is turned ON each instance will occupy up to a user settable maximum num ber of rows so that the last N values received for the Topic are stored in the table When N values have been stored the N rows are used as a circu lar buffer so that new values received wi
148. table in the distributed shared database it is essential that the local table is up to date Database Integration Service supports two approaches to filling the local table s contents 1 If all the rows in the table are updated frequently it is sufficient to apply these updates to the data cache 2 The table can be synchronized by explicitly requesting the table s contents from the other hosts This is called table initialization If table initialization is not needed you can disable it by setting the columns dw durability kind in RIIDDS PUBLICATIONS and dw durability kind in RIIDDS SUBSCRIPTIONS to VOLATILE DURABILITY QOS Database Mapping Options Table 4 8 describes the options that are allowed with the database mapping options tag Database Mapping Options Tags within Number database mapping Description of Tags options Allowed Sets the closing bracket character that is used in the index component of the arrays and sequences members names See Array Fields Section 5 2 8 and Sequence Fields Section 5 2 9 for more information about the mapping of IDL arrays and sequences into SOL columns 0or1 close bracket char The default value of will generate columns names that must be referenced using double quotes Default Sets the character that is used as a separator in the hierarchical names generated when mapping IDL fields into SOL table col umns The attribute is also used to confi
149. table owner VARCHAR 128 NOT NULL table name VARCHAR 128 NOT NULL domain id NUMBER 10 NOT NULL topic name VARCHAR 200 type name VARCHAR 200 table history depth NUMBER 10 process batch NUMBER 10 process period sec NUMBER 10 process period nsec NUMBER 10 commit type VARCHAR 17 cache maximum size NUMBER 10 cache initial size NUMBER 10 delete on dispose NUMBER 10 idl member prefix max length NUMBER 10 idl member suffix max length NUMBER 10 profile name VARCHAR 255 filter duplicates NUMBER 3 ordered store NUMBER 3 persist state NUMBER 3 sub present access scope VARCHAR 25 sub present ordered access NUMBER 3 sub partition name VARCHAR 256 dr durability kind VARCHAR 30 dr reliability kind VARCHAR 27 dr destination_order kind VARCHAR 43 dr liveliness lease_dur sec NUMBER 10 dr liveliness lease_dur nsec NUMBER 10 dr deadline period sec NUMBER 10 dr deadline period nsec NUMBER 10 dr history kind VARCHAR 21 dr history depth NUMBER 10 dr ownership kind VARCHAR 23 dr time filter min sep sec NUMBER 10 dr time filter min sep nsec NUMBER 10 dr res limits max samples NUMBER 10 dr res limits max instances NUMBER 10 dr unicast receive port NUMBER 10 dr multicast receive address VARCHAR 39 dr multicast receive port NUMBER 10 PRIMARY KEY table owner table name domain id topic name MySQL Create Table RTIDDS
150. tamp to filter the data then all DataReaders will end up with the same final value for a data instance If DataReaders used the reception timestamp the DataReaders may end up with differ ent final values because data from different DataWriters may be received in a different order by different DataReaders See the Connext DDS documentation for more details on how this QoS policy may be used See also resolution column Section 4 5 1 1 9 dr liveliness lease dur These columns specify the lease duration for the DDS LivelinessQosPolicy for the DataReader created to subscribe to Topic data that is stored in the table This value is useful when there are redundant DataWriters that publish values for the same data instance for the Topic and the value set for the lt dr ownership kind gt column is EXCLUSIVE OWNERSHIP QOS The liveliness of a DataWriter is monitored by the DataReader These columns control how quickly the DataReader can determine that the DataWriter with the highest ownership strength has lost liveliness because heartbeat packets or data were not received within the liveliness lease duration When liveliness is lost the DataReader will then receive the data instance from the DataWriter with the next highest ownership strength that is still alive The possible values of the lt dr liveliness lease_dur sec gt seconds and drliveli ness lease_dur nsec gt nanoseconds columns are LJ An infinite lease duration is specified
151. ters Maximum Number of Columns INTEGER DOUBLE 30 1283 CHAR 50 VARCHAR 50 INTEGER 15 1823 DOUBLE CHAR 50 15 1308 VARCHAR 50 15 1283 a Limited by schema size b Limited by row size B2
152. the domain ID that will be used to subscribe to Topics whose values will be stored in the table Before creating a DataReader if no DomainParticipant has previously been created with the domain ID the Database Integration Service Daemon will create a Domain Participant with the specified ID If the subscriptions entry has an associated QoS profile Database Integration Service will use the values in this profile to create the participant The participant will also be configured using the QoS values of a profile when the attribute is default qos is set to 1 in that profile see the RTT Connext DDS Core Libraries User s Manual for additional details topic name This column defines the Topic that will be subscribed to and whose received values will be stored in the associated table The topic name entry needs to match the Topic used by the DataWriters that are sending data changes If the Database Integration Service Daemon has discovered the typecode associated with the topic name and the user table does not exist in the database the daemon will use the type code to create the table using entries in the table owner and table name column See User Table Creation Section 4 6 for more details type name This column defines the registered name of the type associated with the Topic defined using the column topic name If the user table does not exist in the database the daemon will use the type name to find a typecode i
153. tial size INTEGER YES 0 Section 4 5 2 1 29 delete on dipose INTEGER YES 0 Section 4 5 2 1 30 idl member prefix max length INTEGER YES Value specified in the configuration file Section 4 5 2 1 31 idl member suffix max length INTEGER YES Value specified in the configuration file Section 4 5 2 1 31 pul rame ums jore e ee filter duplicates TINYINT YES 0 Section 4 5 2 1 33 ordered_store TINYINT YES 1 Section 4 5 2 1 34 persist_state TINYINT YES 0 Section 4 5 2 1 35 sub present access_scope VARCHAR 25 YES INSTANCE PRESENTATION QOS Section 4 5 2 1 36 sub present ordered access TINYINT YES 0 false Section 4 5 2 1 36 sub partition name Vct 256 YES Empty partition string Section 4 5 2 1 37 dr durability kind VARCHAR 30 YES VOLATILE DURABILITY QOS Section 4 5 2 1 38 dr reliability kind VARCHAR 27 YES BEST_EFFORT_RELIABILITY_QOS Section 4 5 2 1 39 dr destination_order kind VARCHAR 43 YES DERI Section 4 5 2 1 40 drliveliness lease dur sec INTEGER YES Infinite Section 4 5 2 1 41 dr liveliness lease_dur nsec INTEGER YES Infinite Section 4 5 2 1 41 dr deadline period sec INTEGER YES Infinite Section 4 5 2 1 42 dr deadline period nsec INTEGER YES Infinite Section 4 5 2 1 42 dr history kind VARCHAR 21 YES KEEP_LAST_HISTORY_QOS Section 4 5 2 1 43 dr history depth INTEGER YES 1 Section 4 5 2 1 43 dr ownership kind VARCHAR 23 YES SHARED OWNERSHIP QOS Section 4 5 2 1 44 dr time filter min sep sec INTEGER YES 0 Sectio
154. tions RTI Database Integration Service BPELProcess Daemon Manager Connext DDS ODBC Cache Connect to Oracle Oracle TimesTen In Memory Database Oracle 10g Database High performance Permanent storage Using the Oracle TimesTen Cache Connect to Oracle the data and messages buffered by the in memory database can be synchronized and stored in a disk based Oracle database for access by enterprise applications directly or through enterprise level integration options such as a BPEL Business Process Execution Language process manager Multiple copies of Database Integration Service can also run in parallel on separate systems Dif ferent instances can be responsible for a discrete set of messages and data maximizing through put and providing load balancing Or each instance can bridge the same messages and data providing high availability and fault tolerance Database Integration Service s Unique Features In this section a few of the unique qualities and features of Database Integration Service are dis cussed in greater detail Figure 2 2 shows an example system where Database Integration Service serves as the central integration technology to interconnect the real time embedded world with the analysis and high level decision making processes of the enterprise world In Figure 2 2 sensors of physical processes produce data that must be filtered fused and stored for use in business processes In addition multiple
155. tory depth These columns directly map to the DDS_HistoryQosPolicy for the DataReader created to sub scribe to Topic data that is stored in the table The values set for this QosPolicy affect the DDS_ReliabilityQosPolicy Using a KEEP_ALL_HISTORY_QOS will ensure that reliable DataReaders will receive every change to the table reliably With a KEEP_LAST_HISTORY_QOS the Database Integration Ser vice Daemon will only guarantee that the last lt dr history depth gt changes for each data instance are received reliably The possible values of the lt dr history kind gt and lt dr history depth gt columns are I KEEP LAST HISTORY QOS For this setting the column lt dr history depth gt determines the maximum number of val ues for each data instance that be buffered in the DataReader before the Database Integra tion Service Daemon stores the received values into the table dr history depth should be set to an integer greater than 0 The default value for his tory depth is 1 if this column is NULL LJ KEEP ALL HISTORY QOS default value if the column is NULL This setting implies that the DataReader created to subscribe to Topic data has an unlim ited queue in which to save received data before the data is stored in the table The actual size of the queue is limited by the value in dr res limits max samples column For this setting the value in lt dr history depth gt is ignored See the Connext DDS documentat
156. tribute delete in lt publications gt lt subscriptions gt is set to true If a lt publication gt or lt subscription gt already exists in its table the primary key is the same then the insertion won t succeed However you can set the attribute overwrite to true In that case if the insertion fails an update is performed on that row Subscriptions Tags Tags Allowed Number within Description of Tags lt subscriptions gt Allowed vac M Configures a subscription by inserting or updating a row in the table 1 or supscripiion RTIDDS_SUBSCRIPTIONS See Table 4 14 on page 4 24 more Subscription Tags Number Eins e Head Description of Tags P Allowed domain id Inserts the tag value into the column with the same name in the table Stable nantes RTIDDS SUBSCRIPTIONS required table owner See Section 4 5 2 req lt topic_name gt lt cache_initial_size gt lt cache_maximum_size gt lt commit_type gt lt delete_on_dispose gt lt filter_duplicates gt idl member prefix max length Inserts the tag value into the column with the same name in the table idl member suffix RTIDDS SUBSCRIPTIONS max length If the value is not specified NULL is inserted Cori lt ordered_store gt See Section 4 5 2 lt persist_state gt lt process_batch gt lt process_period gt lt profile_name gt
157. uence of the primitive type char Without the suffix in the col umn name a SOL VARCHAR x naturally stores a sequence of chars the IDL type sequence lt char x gt For the Oracle database but not the Oracle TimesTen In Memory database the IDL octet octet x and sequence lt octet x gt are all stored in the Oracle type RAW A suffix of bin is used to distinguish between using RAW to store octet and octet x which can be treated the same and sequence lt octet x gt which must be treated differently by the Database Integration Ser vice Daemon NOTE Because of the use of suffixes in the mapping of identifiers of certain IDL datatypes the identifiers str 1d and bin are reserved keywords that should not be used as the name of fields in IDL structures For example the following IDL definitions have the same SOL mapping which would in result in the incorrect treatment of the type Foo2 by the daemon Each would result in a table schema that would have the ten columns named my field 0 str my field 1 str my field 2 str struct Fool struct Bar string lt 10 gt my field and sequence lt char 10 gt str struct Foo2 struct Bar my_field 5 5 Data Representation Mapping 5 2 2 Primitive Types Mapping The following tables show the mapping between basic types in IDL and SQL I Table 5 2 Basic Types in IDL and SQL TimesTen on page 5 6 I Table 5 3 Ba
158. uration file by running the Windows service with the command line option cfgFile see Section 4 2 Typecodes Typecodes are runtime parsible descriptions of data generated for user data types from an IDL file by the Connext DDS utility rtiddsgen Typecodes are automatically propagated during the discovery process of Connext DDS applications Unless the user has specifically disabled rtidds gen from generating typecodes applications built with types generated by rtiddsgen should be propagating typecodes for all of the Topics that they use and thus are compatible with Database Integration Service Please consult Connext DDS documentation for more information about type codes and their generation An important note is that typecodes can become quite large as the corresponding IDL type becomes more complex By default Connext DDS applications allocate 2048 bytes to store a type code The default size for the Database Integration Service Daemon is 2048 bytes as well The type code size is controlled by the QoS parameter DomainParticipantQos resource_limits type_code_max_serialized_length in the Connext DDS API In Database Integration Service you can change the typecode limit using XML QoS Pro files see Table 4 2 If the Database Integration Service Daemon discovers Topics that have typecodes that a are larger than what it has been configured to handle or b have no associated typecodes at all the dae mon will not be able to subscri
159. used for Distributed Logger when enabled The value profile name is the fully qualified name of the QoS Profile represented as a string 0 or 1 with this form QoS profile library name QoS profile name gt Enabling Table Replication Enabling database replication will automatically configure the QoS values of publications and subscriptions to provide conflict resolution and table initialization see Table 4 6 and Table 4 7 The attribute also enables automatic table creation see typecode from table schema in Table 4 4 and propagation of NULL values DataWriter QoS Changes when enable table replication is True QoS Change Purpose reliability kind RELIABLE RELIABILITY QOS Enables reliability destination order kind BY SOURCE TIMESTAMP DESTINATIONORDER QOS destination order source timestamp tolerance sec 0 Performs conflict re destination order source timestamp tolerance nanosec 0 ownership kind SHARED OWNERSHIP QOS solution protocol serialize key with dispose true writer data lifecycle autodispose unregistered instances false durability kind TRANSIENT LOCAL DURABILITY QOS Propagates delete o perations Sends table contents to late joiners table initialization 4 16 Configuration File Table 4 6 Table 4 7 4 4 4 1 2 DataWriter QoS Changes when lt enable_table_replication gt is True QoS Change Purpose
160. values as well as the total number of data instances different primary keys that can be handled by the DataReader A value of 1 for either of these columns means infinite This is also the default value for these columns if they are NULL An infinite setting means that the DataReader is allowed to allocate memory as needed to store received table changes and manage new keys The number of keys that the DataReader is allowed to manage places an upper limit on the number of rows that the related table in the database can have See the Connext DDS documentation for more details on how this QoS policy may be used dr unicast receive port This column is used to configure the unicast port on which the DataReader will receive data When the default value NULL or 0 is used the actual port number is determined by a formula as a function of the domain ID dr multicast receive address This column is used to set a multicast address for the DataReader to receive values for the Topic The column maps to the DDS TransportMulticastQosPolicy of the DataReader The possible values for the dr multicast receive address column are LJ NULL A NULL column means that the DataReader will receive Topic data using unicast I A string that contains a valid multicast address in the form xxx xxx xxx xxx The DataReader for the table will subscribe to the Topic on the multicast address pro vided Meta Tables 4 5 2 1 49 4 5 2 1 50 4 5 3 4
161. ve complex coding for real time data management and commu nication Integrate edge devices distributed real time data management and enterprise databases using a single set of standard Application Programming Interfaces 2 4 Database Integration Service s Unique Features 1 Deliver cutting edge solutions Process massive amounts of information across networks in real time Turn near instantaneous responses to remote critical events into a business advantage Seamlessly integrate networked applications services and devices l Minimize operational costs Maintain complex networked applications with near zero administration Dynamically add or change system components Run on common hardware platforms and networks LJ Reduce risks Guarantee continuous system availability through dynamic replication manage ment Rely on continuous high quality technical support Build on years of experience in the world s most demanding real time application domains 2 5 Chapter 3 Architecture 3 1 This chapter presents a more detailed view of RTI Database Integration Service s architecture and highlights the different ways that RTI Database Integration Service can be used to integrate sys tems It includes the following sections L Database Integration Service Architecture Section 3 1 1 Capturing Real Time Data in a DBMS Section 3 2 1 Remote Real Time Notification of Table Changes Section 3 3 1 Bidirect
162. vice version Environment Variables Since the Database Integration Service Daemon will be making connections to databases using ODBC on UNIX based systems the following environment variables may be used to find DSNs data source names via ODBCINI files J ODBCINI location of INI file for database connections If not set ODBCINI will be set to HOME odbc ini where HOME is the home directory of the user who started the daemon L SYSODBCINI location of system INI file used if the DSN is not found in the file speci fied by ODBCINI If the Database Integration Service Daemon cannot find a valid DSN in any ODBC INI file then no connections to any databases can be made On a Windows system the equivalent functionality of the ODBCINI file is found in the Win dows registry You create and modify DSNs using the application found in Start Programs Administrative Tools Data Sources ODBC 4 10 Configuration File 4 4 4 4 1 Configuration File When you start Database Integration Service you can provide a configuration file in XML format it is not required Among other things this file can be used to specify the set of databases to which the daemon will connect and the properties of the database connections This section describes I How to Load the XML Configuration Section 4 4 1 1 XML Syntax and Validation Section 4 4 2 L Top Level XML Tags Section 4 4 3 1 Database Configuration Using th
163. will be used to truncate column names when a table is auto matically created If a positive value n is provided Database Integration Service will use the last n characters from the IDL member identifier to compose the associated column name A value of 0 tells Database Integration Service to compose the idl member suffix column name using only the first characters of the identifiers max length as defined by idl member prefix max length Dor This value can be overridden per table by assigning a value to the idl member suffix max length column in the meta tables Note that although idl member prefix max length and idl member suffix max length can be individually set to zero they cannot be both zero at the same time Default 1 unlimited Sets the opening bracket character that is used in the index component of the arrays and sequences members names See Array Fields Section 5 2 8 and Sequence Fields Section 5 2 9 for more information about the mapping of IDL arrays and sequences into SOL columns 0or1 open bracket char The default value of will generate columns names that must be referenced using double quotes Default 4 19 Configuration File 4 4 4 3 Database Connection Options The database connection tags in the XML file direct the Database Integration Service Daemon to connect to a particular database as specified by a DSN data source
164. xt DDS Core Libraries User s Manual 0or1 for more information Default Connext DDS default 1024 4 15 Configuration File Table 4 4 General Options Tags Table 4 5 4 4 4 1 1 Table 4 6 er Number Tags within eee lt general_options gt Description Pee This tag can be used to enable typecode generation from table schemas If this parameter is set to true and a publication or subscription is created for a database table without an associated typecode Data base Integration Service will create the typecode from the table typecode from table schema ort schema gt The new typecode will be made available to other Connext DDS applications or RTC daemons via discovery traffic When Database Integration Service is used for table replication the default value for this parameter is true allowing automatic table creation in the replicas Default false except when enable_table_replication is set to true Administration Tags ee Number Tags within er 2 Description of Tags administration Allowed Configures RTI Distributed Logger lt distributed_logger gt See Enabling RTI Distributed Logger in Database Integration Service Oorl Section 4 8 Eros Specifies which domain ID Database Integration Service will use to send 1 domain id T log messages when Distributed Logger is enabled required Determines which QoS profile to use when creating the DomainPartic ipant that will be
165. y the Database Integration Service Daemon when it stores data received via Connext DDS into the table The column stores a strictly incre menting counter that represents the received sequence number starting at 0 of the data that is stored in that row User should use a combination of the instance key and the value of RTIRTC HISTORY ORDER to find the latest data received for an instance in the table The possible values for the table history depth column are LI NULL or 0 Only the current value of an instance of the Topic is stored For non keyed topics this implies the table will only have a single row For keyed topics each instance will corre spond to a single row in the table This is the most common value for tables that are pub lished with Connext DDS No meta columns are added to help manage history 1 1 Exactly the same behavior as NULL or 0 a single value is stored in the table per instance of the Topic However table history meta columns are added to the table schema if the Database Integration Service Daemon creates the user table automatically This value is useful for preparing the table to store more than a single value per instance after the table is created Because table schema cannot be changed to accommodate the table history meta columns after a table has been created using a value of 1 for this col umn is useful if the user wants to store historic values of instances but does not know how many instance
166. y to the DDS DurabilityOosPolicy of the DataReader created to sub scribe to Topic data that is stored in the table By changing this policy the DataReader can be configured to request for past values published for the Topic to be sent by existing applications soon as their matching DataWriters are discovered The DataWriter s DDS DurabilityOosPolicy most also be set appropriately to permit the send ing of historic or past published data In addition the column dr realiability kind for the entry must be set to RELIABLE RELIABILITY QOS for historic data to be received The possible values for the dr durability kind column are LI VOLATILE DURABILITY QOS default value if the column is NULL Meta Tables 4 5 2 1 39 4 5 2 1 40 This value means that the DataReader does not request past data to be sent 2 TRANSIENT_LOCAL_DURABILITY_QOS This value requests that existing DataWriters of the Topic send past data that they are storing to the DataReader See the Connext DDS documentation for more details on how this QoS policy may be used See also dw durability kind Section 4 5 1 1 14 and dr reliability kind Section 4 5 2 1 39 dr reliability kind This column sets the DDSReliabilityQosPolicy for the DataReader created to subscribe to Topic data that is stored in the table The value in this column determines whether or not DataWriters will send their data reliably to the DataReader If the value for lt dr dur
167. ystem DBMS This executable is called the Database Integration Service Daemon Database Integration Service uses Connext DDS and supports three databases Oracle Database 11g Oracle TimesTen In Memory Database 11 2 1 and MySQL 5 1 There is a separate execut able that you must run depending on which database you are using 1 Oracle Database rtirtc_oracle exe I Oracle TimesTen In Memory Database rtirtc timesten exe LI MySQL rtirtc_mysqll exe These executables can be executed as foreground processes during development or as back ground processes or as a service on Windows systems You can configure the general behavior of the Database Integration Service Daemon by using command line parameters environment variables and configuration files Meta tables in the database are used to configure the specific topics and tables that are bridged by the daemon Besides using compatible versions of Connext DDS and Oracle Oracle TimesTen MySQL data bases see the Release Notes for a list of compatible versions the Database Integration Service Daemon expects that typecodes for the IDL types used by Connext DDS applications have been generated and are being propagated If typecodes for IDL types were not generated you must create the tables used by the daemon for storing or publishing data yourself or declare the types in the configuration files How to Run the Database Integration Service Daemon with Oracle To run Database Integration Se
168. ze of the cache would be the maximum number of rows in the table However this is not practical for large tables and thus the cache will be smaller 4 5 2 1 30 delete on dispose This column configures the behavior of the Database Integration Service Daemon when a DataW riter disposes an instance stored into the database When delete on dispose is initialized to 0 the default value the rows corresponding to the instance will not be deleted from the database If delete on dispose is initialized to 1 all the rows associated with the instance will be deleted from the database 4 5 2 1 31 idl member prefix max length id member suffix max length These columns define how Database Integration Service maps IDL member identifiers into col umn names In particular they control how the column names are formed by using as a prefix n characters from the identifier s prefix and m characters from the identifier s suffix They can assume any value greater than or equal to 1 They cannot both be set to zero If a positive value n is provided for id member prefix max length Database Integration Service will use the first n characters from the IDL member identifier to compose the associated column name A value of 0 tells Database Integration Service to compose the column name using only the last characters of the identifiers as defined by the idl member suffix max length column A value of 1 instructs Database Integration Service to use all the

Chapter 1 Welcome to RTI Database Integration Service

Contents

Download Pdf Manuals

Related Search

Related Contents