Differences between ODI 10g and 11g
Runtime Agent
Oracle Data Integrator runtime agent has been enhanced with the features listed in this section.
Java EE Agent
The Runtime Agent can now be deployed as a Java EE component within an application server. It benefits in this configuration from the application server layer features such as clustering and connection pooling for large configurations. This Java EE Agent exposes an MBeans interface enabling lifecycle operations (start/stop) from the application server console and metrics that can be used by the application server console to monitor the agent activity and health.
Standalone Agent
In addition to the Java EE Agent, a Standalone Agent, similar to the one available in previous Oracle Data Integrator releases, is still available. It runs in a simple Java Virtual Machine and can be deployed where needed to perform the integration flows.
Connected Scheduler
Both agent flavors are now always connected to a master repository, and are started with the built-in scheduler service. This scheduler service takes its schedules from all the Work Repositories attached to the connected Master.
HTTP Protocol for Component Communication
Communications with the run-time agents (for example, when sending an execution request to a remote agent) now use standard HTTP protocol. This feature facilitates network management and security for Oracle Data Integrator components in distributed environments.
Oracle WebLogic Server Integration
Oracle Data Integrator components integrate seamlessly with Oracle's Java EE application server.
Java EE Agent Template Generation
Oracle Data Integrator provides a wizard to automatically generate templates for deploying Java EE agents in Oracle WebLogic Server. Such a template includes the Java EE Agent and its configuration, and can optionally include the JDBC datasources definitions required for this agent as well as the drivers and libraries files for these datasources to work.
By using the Oracle WebLogic Configuration Wizard, domain administrators can extend their domains or create a new domain for the Oracle Data Integrator Java EE runtime agents.
Automatic Datasource Creation for WebLogic Server
Java EE Components use JDBC datasources to connect to the repositories as well as to the source and target data servers, and benefit, when deployed in an application server, from the connection pooling feature of their container.
To facilitate the creation of these datasources in the application server, Oracle Data Integrator Studio provides an option to deploy a datasource into a remote Oracle WebLogic application server.
Pre-Packaged WebLogic Server Templates for Java EE Components
Oracle Data Integrator Java EE components that can be deployed in an application server are provided now with pre-packaged templates for Oracle WebLogic Server. Oracle Data Integrator provides templates for:
Java EE Runtime Agent
Oracle Data Integrator Console
Public Web Service
These templates are used to create a WLS domain for Oracle Data Integrator or extend an existing domain with Oracle Data Integrator components.
Web Services
Oracle Data Integrator web services support has been enhanced with the features listed in this section.
JAX-WS Support for Web Services
Oracle Data Integrator Web Services - including the Public Web Service as well as the generated Data Services - now support the market standard Java API for XML Web Services (JAX-WS 2.0). As a consequence, they can be deployed into any web service container that implements this API. The use of the Axis2 stack for these web services is deprecated.
Web Services Changes and Reorganization
The web services have been reorganized and the following run-time web operations are now part of the run-time agent application:
getVersion - Retrieve agent version. This operation is new in this version.
getSessionStatus - Retrieve the status of a session.
invokeRestartSess - Restart a session.
invokeStartScen - Start a scenario.
The Public Web Service application retains the following operations:
listScenario - List the scenarios.
listContext- List the contexts.
Advanced Security Capabilities
Security in Oracle Data Integrator can be hardened with the enterprise features listed in this section.
External Password Storage
Source and target data server passwords, as well as the context passwords, can optionally be stored in an external credential store instead of storing them in an encrypted form in the master repository. This credential store is accessed via the Java Platform Security (JPS) Credential Store Framework (CSF). The password storage method (internal or external with JPS) is defined at repository creation, and can be switched for existing repositories.
With this password storage approach, administrators can choose to rely on a corporate credential store for securing their data server passwords.
External Authentication and SSO
Oracle Data Integrator users can be authenticated using an external authentication service. Using Oracle Platform Security Services (OPSS), Oracle Data Integrator users authenticate against an external Enterprise Identity Store (LDAP, Oracle Internet Directory, Active Directory), which contains in a central place enterprise user and passwords.
With this feature, the master repository retains the Oracle Data Integrator-specific privileges and the user names, but passwords rely in a centralized identity store, and authentication always takes place against this external store. The authentication mode (internal or external) is defined at repository creation, and can be switched for existing repositories.
This feature enables Single Sign-On (SSO) for Oracle Data Integrator Console, and seamless authentication integration between Enterprise Manager and Oracle Data Integrator Console.
Default Password Policy
Oracle Data Integrator is now installed with a default password policy that prevents from setting passwords with a low security level.
Java EE Components Passwords in Credential Store
When deploying in Oracle WebLogic Server a Java EE component that requires a bootstrap connection to a repository (Java EE Agent, Oracle Data Integrator Console), the configuration of this component contains a Supervisor user login. To enforce a strong security policy this user's password is not stored within the application configuration, but centralized in the WLS Credential Store. The configuration will refer to this centralized store.
Production and Monitoring
Oracle Data Integrator provides new features for an enhanced experience in production.
Enhanced Error Messages
Error messages raised by Oracle Data Integrator Components and Sessions have been enhanced to provide administrators and production operators with precise information for troubleshooting and fixing the status of the architecture, and debugging the sessions. Enhanced messages cover:
Component lifecycle (startup, shutdown, schedule refresh, etc.)
Session lifecycle (incorrect scenario version, load balancing issue, agent not available, etc.)
Session Tasks/Steps (source/target not available, interface error). Database errors are enriched with information allowing developers or production operators to quickly identify the location and reason for an error.
These error messages are standardized with Oracle Data Integrator error codes.
Enhanced Notifications and Logging
Oracle Data Integrator components are now using the Oracle Logging Framework. Logging in any component can be configured to meet the requirements of development, test and production environments.
In addition to this logging capability, agent components can now raise status and session information in the form of Java Management Extension (JMX) notifications that propagate to any administration console.
Error Tables
Error tables can now be managed via Oracle Data Integrator Console. Production operators can review the content of the error tables and purge their content selectively.
Purge Log on Session Count
The OdiPurgeLog tool has been enhanced to support a purge of the log while retaining only a number of sessions in the log. Purged sessions can be automatically archived by the tool before performing the purge.
New Oracle Data Integrator Console
The Metadata Navigator UI has been replaced with the Oracle Data Integrator Console. This web interface for production operations has been rewritten using the ADF-Faces Ajax Framework for a rich user experience. Using this console, production users can set up an environment, export/import the repositories, manage run-time operations, monitor the sessions, diagnose the errors, browse through design-time artifacts, and generate lineage reports.
This web interface integrates seamlessly with Oracle Fusion Middleware Control Console and allows Fusion Middleware administrators to drill down into the details of Oracle Data Integrator components and sessions.
Oracle Fusion Middleware Control Console Integration
Oracle Data Integrator provides an extension integrated into the Oracle Fusion Middleware Control Console. The Oracle Data Integrator components can be monitored as a domain via this console and administrators can have a global view of these components along with other Fusion Middleware components from a single administration console.
This extension discovers Oracle Data Integrator components and allows administrators to:
Monitor the status and view the metrics of the master and work repositories, Java EE and Standalone Agents components, and the Oracle Data Integrator Console
Review from a central location the notifications raised by any of these components
Transparently drill down into Oracle Data Integrator console to browse detailed information stored in the repositories
Start and stop Oracle Data Integrator Console and Java EE Agent applications
Monitor session executions and review session statistics attached to any of those components
Search for specific sessions, view a session status, and drill down into the session details in Oracle Data Integrator Console.
Kill Sessions Immediate
Sessions can now be stopped in an immediate mode. This new mode attempts to abort the current operation (for example, SQL statements launched against a database engine) instead of waiting for its completion.
High Availability
For an enterprise scale deployment, the features enable high available of the Oracle Data Integrator components.
Stale Session Detection and Management
Oracle Data Integrator is now able to detect sessions pending due to an unexpected shutdown of the agent or repository. Such stale session are now managed and pushed to an error state.
Repository Connection Retry
The Agent, when connected to a repository based on Oracle RAC technology, can be configured with connection retry logic. If the one of the Oracle RAC nodes supporting sessions for an agent becomes unavailable, the agent is able to retry and continue its session on another node of the Oracle RAC infrastructure.
Support for WLS Clustering
Clustering is supported for the Java EE agents deployed on a WebLogic Server. Clustering includes schedule porting on a different cluster node. Unrecoverable running sessions are automatically moved to an error state.
OPMN Integration
Standalone agent can be now made highly available using Oracle Process Manager and Notification Server (OPMN). Scripts are provided to configure OPMN to protect standalone agents against failure.
Improved Integration Design
Integration interface design and performance are enhanced with the following features.
Partitioning
Oracle Data Integrator now supports partitioning features of the data servers. Partitions can be reverse-engineered using RKMs or manually created in models. When designing an interface, it is possible to define the partition to address on the sources and target datastores. Oracle Data Integrator code generation handles the partition usage syntax for each technology that supports this feature.
Lookups
A wizard is available in the interface editor to create lookups using a source as the driving table and a model or target datastore as the driving table. These lookups now appear as a compact graphical object in the Sources diagram of the interface. The user can choose how the lookup is generated: as a Left Outer Join in the FROM clause or as an expression in the SELECT clause (in-memory lookup with nested loop). This second syntax is sometimes more efficient on small lookup tables.
This feature simplifies the design and readability of interfaces using lookups, and allows for optimized code for executing lookups.
Datasets and Set-Based Operators
This major enhancement introduces the notion of dataset in interfaces. A dataset represents the data flow coming from a group of joined and filtered source datastores. Each dataset includes the target mappings for this group of sources. Several datasets can be merged into the interface target datastore using set-based operators such as Union and Intersect.
This feature accelerates the interface design and reduces the number of interfaces needed to merge several data flows into the same target datastore.
Derived Select for Temporary Interfaces
When using a temporary interface as a source or a lookup table in another interface, you can choose not to persist the target of the temporary interface, and generate instead a Derived Select (sub-select) statement corresponding to the loading of the temporary datastore. Consequently, the temporary interface no longer needs to be executed to load the temporary datastore. The code generated for the sub-select is either a default generated code, or a customized syntax defined in an IKM.
This feature eliminates the need for complex packages handling temporary interfaces and simplifies the execution of cascades of temporary interfaces.
Support for Native Sequences
Oracle Data Integrator now provides support for a new type of sequence that directly maps to database-defined sequences. When created, these can be picked from a list retrieved from the database. Native Sequences are used as regular Oracle Data Integrator sequences, and the code generation automatically handles technology-specific syntax for sequences.
This feature simplifies the use of native sequences in all expressions, and enables cross references when using such sequences.
Support for Natural Joins
Oracle Data Integrator now provides support for the Natural join, defined at technology level. This join does not require any join expression to be specified, and is handled by the engine that processes it. This engine matches automatically columns with the same name.
Automatic Temporary Index Management
When creating joins or filters on source tables, it is possible to have Oracle Data Integrator automatically generate temporary indexes for optimizing the execution of these joins or filters. The user selects the type of index that needs to be created in the list of index types for the technology. Knowledge modules automatically generate the code for handling indexes creation before executing the join and filters as well as deletion after usage.
This feature provides automated optimization of the joins and filters execution, and enables better performances for integration interfaces.
New Interface Editor
The interface editor, used to create the integration interfaces, has been entirely redesigned to use the JDeveloper diagramming framework.
The advantages of this new diagram include:
Improved look and feel and better user experience
Support for graphical options on diagram objects. For example, compact and expanded view can be used for better readability.
Thumbnail and zoom in/out is supported on the sources and flow diagram to navigate large diagrams.
Multiple source columns can be dropped directly onto the target datastore for faster mapping.
Target mapping table is improved. Mapping properties (Position, Indicator, Name and Mapping expression) can be displayed selectively and sorted.
Sources, targets, filters, joins can be selected and edited directly in the flow diagram.
Quick-Edit
The new interface editor includes a new Quick-Edit tab to edit the interface diagram faster. Quick-Edit displays these components in a tabular form, supports mass-updates and intuitive keyboard navigation.
Auto fixing
When saving an interface or clicking the error button from the interface editor toolbar, a list of all the design errors in the interface is displayed with meaningful messages and tips. Automated fixes are suggested and can be applied with a single click.
Code Simulation
When performing an execution of design-time objects from the Oracle Data Integrator Studio (for example, when running an interface, a procedure, a package or a customized reverse-engineering process), it is possible to make a code simulation instead of a full execution.
Code simulation displays a session simulation report. This report includes complete session, step, and task information and contains the full generated code. The session simulation report can be reviewed and saved in XML or HTML format.
With this features Oracle Data Integrator developers can easily review the generated code for troubleshooting, debugging, optimization purposes, and save this generated code for documentation or archive purposes.
Reverse-Engineering Improvements
When a model is created the reverse-engineering context is automatically set to the default context, instead of having to select it manually. In addition, when performing a selective reverse-engineering, the system tables are now hidden from the display.
Scenario Naming Convention
When generating a scenario or a group of scenarios from the Studio or using a tool, the naming convention that is used for naming the scenario can be defined in a pattern (using the object name, folder path or project name) using the Scenario Naming Convention user parameter.
Object Name Length Extension
Object names have been extended to support long database object names (128 characters) and repository object labels (400 characters).
Oracle Data Integrator Java API
Oracle Data Integrator provides a Java API for managing run-time and design time artifacts. Using this API, Java developers can embed Oracle Data Integrator in their product and can drive integration process creation from their own user interface.
Oracle Data Integrator Studio
Oracle Data Integrator provides a new IDE called the Studio, based on JDeveloper. This component includes the following features:
New Navigator Organization
The new Oracle Data Integrator studio is used as a replacement for all Oracle Data Integrator modules (Designer, Topology, Operator and Security Manager). All the features of these modules now appear as Navigators within the Oracle Data Integrator Studio window.
This new Navigator organization provides the following features:
Navigators can be docked/undocked and displayed/hidden using the View menu. These Navigators allow access to the former module-specific actions from their Navigator toolbar menu (for example, the export/import master repository operations in the Topology Navigator)
Accordions group the tree views that appear in the Navigators (for example the Project and Models accordions in the Designer Navigator). Accordions that are not frequently used can be minimized into the lower section of the Navigator to allow more room for the other tree views. Accordions allow access to the tree view-specific actions from their toolbar menu (for example, import project from the Project Accordion in the Designer Navigator).
Tree Views objects are provided with context menus and markers the same way as in Oracle Data Integrator 10g. Tree view objects can be dragged and dropped within a tree view or across tree views for defining the security policies. Double clicking an object opens by default the corresponding Object Editor.
Context Menus have been reorganized into groups with separators and normalized across the interface.
This feature provides a single user interface from which the user can perform all the tasks in a project lifecycle. It also provides a better productivity for the user.
New Look and Feel
The look and feel of Oracle Data Integrator has been enhanced with the use of the JDeveloper base IDE. This new look and feel is customizable with the Preferences menu option. Icons are being redesigned in a new, trendy style to enhance the overall visual appeal of Oracle Data Integrator
Redesigned Editors
All object editors in Oracle Data Integrator have been redesigned for better usability.
Main changes include:
Tabs are organized as finger tabs on the left hand-side of the editor. Complex editors (as for example Interface or Package Editors) have also tabs appearing in the bottom of the editor.
Fields have been grouped under headers. These field groups implement an expand/collapse behavior.
Fields and labels have been organized in a standard way for all editors for a better readability of the editors.
Text Buttons in the editors are transformed into hyperlinks, and all buttons appearing in editors have been redesigned.
Knowledge Modules, Actions and Procedure editors have been redesigned in order to edit the Lines directly from the main editor instead of opening a separate editor.
Window Management
The windows, editors and navigators in the Oracle Data Integrator Studio benefit from the following JDeveloper IDE features:
Full Docking Support: All windows, editors and navigators can now be docked and undocked intuitively. The visual feedback provided when repositioning editor windows and dockable windows has been improved. You now see an outline shape of where the window will be placed when the mouse is released. You can also now reorder the document tabs using drag and drop.
Fast maximize and restore: To quickly maximize a dockable window or the editor area, double-click on the title bar of the window you want to maximize. To restore the window to its previous dimensions, double-click again on the title bar.
Title bars as tabs: The tab for a dockable window (when tabbed with another dockable window) is now also the title bar. This makes more effective use of the space on the screen. Reposition a window by dragging its tab. Some additional related enhancements include a new context menu from the gray background area behind the tab, change in terminology from "auto-hide" and "show" to "minimize" and "restore", ability to minimize a set of tabbed windows with a single click, and toggling the display of a minimized window by clicking on its button.
Document Management and Navigation
Object edition has been enhanced in the Oracle Data Integrator Studio with improved document management. This includes:
Save and close multiple editors: You can easily save all your work with a single click using the File > Save All option and close all opened editors similarly. You can also close all the editors but the current one.
Forward and back buttons: Now you can easily return to a previously visited document with the convenient browser-style forward and back buttons on the main toolbar. These buttons maintain a history, so you can drop down the back or forward button to get a list of the documents and edit locations you have visited. Alt+Left and Alt+Right activate the back and forward buttons.
Quick document switching: Switching between editors and navigators is also possible. Now when you press Ctrl+Tab or Ctrl+F6, you can choose which document you want to switch from a list ordered by the most recently used. You can use the same technique to switch between open dockable windows by first placing focus in a dockable window, then pressing Ctrl+Tab or Ctrl+F6.
Improved User Assistance
Oracle Data Integrator introduces intuitive new features that improve usability:
Help Center/Welcome Page: The Welcome page has been transformed into the Help Center, redesigned to provide the user with quick access to help topics and common tasks, as well as links to useful Oracle resources.
New On-Line Help: The online help has been entirely re-written for supporting the new user interface.
Help bookmarks: The Help window has a tab labeled Favorites. While browsing the help, you can click on the Add to Favorites button to add the document to this tab.
Export/Import
Export/import is enhanced in this new release with the following features:
Import Report
After objects have been imported, an import report displays the objects that have been imported or deleted in the target repository. In addition, missing objects referenced by the imported objects are indicated as missing references, and missing references fixed by the import are also indicated. Import reports can be saved in XML or HTML format
With this feature, importing objects becomes a very transparent operation as all changes can be identified and archived.
Repository Corruption Prevention
When importing objects across repositories, the following cases have been taken into account to avoid the risks of import errors and repository corruption:
The import in Synonym mode that may result in overwriting a text (for example, a mapping expression) with a text from a different origin (for example, a filter expression) is now verified and not allowed.
It is not allowed to import objects from two repositories with the same repository identifier into a target repository. This avoids object collision and corruption.
When attaching a work repository that contains objects imported from another repository, a warning is raised to the user.
In addition, import of objects that reference non-existing objects now create missing references, identified in the import report. Such references can be resolved by importing the missing object.
Repository Renumbering
It is now possible to change the identifier of a master or work repository after its creation. This operation automatically updates the internal identifier of the objects created in this repository to match the new identifier.
This feature facilitates configuration management and fixing import/export situations when multiple repositories have been created with the same identifier.
KM Enhancements in Release 10.1.3 Patch sets
The following improved and new knowledge modules have been delivered in 10gR3 patch sets and are available in this release.
Oracle GoldenGate Knowledge Modules
Oracle Data Integrator uses Oracle GoldenGate to replicate online data from a source to a staging database. A Journalizing KM manages the Oracle Data Integrator CDC infrastructure and automatically generates the configuration for Oracle GoldenGate.
Oracle E-Business Suite Knowledge Modules
Oracle Data Integrator Knowledge Modules for E-Business Suite provide comprehensive, bidirectional connectivity between Oracle Data Integrator and E-Business Suite, which enables you to extract and load data. The Knowledge Modules support all modules of E-Business Suite and provide bidirectional connectivity through EBS objects tables/views and interface tables.
Oracle OLAP Knowledge Modules
The Oracle Data Integrator Knowledge Modules for Oracle OLAP provide integration and connectivity between Oracle Data Integrator and Oracle OLAP cubes. The Oracle Data Integrator KMs for Oracle OLAP support reverse-engineering of Oracle OLAP data structures (all tables used by a ROLAP or a MOLAP cube) and data integration in an Oracle Analytical Workspace target in incremental update mode.
Oracle PeopleSoft Knowledge Modules
The Oracle Data Integrator Knowledge Modules for PeopleSoft provide integration and connectivity between Oracle Data Integrator and the PeopleSoft platform. These KMs enable Data-level integration for PeopleSoft and support reverse-engineering of PeopleSoft data structures (Business Objects, tables, views, columns, keys, and foreign keys) and data extraction from PeopleSoft.
Oracle Siebel Knowledge Modules
The Oracle Data Integrator Siebel Knowledge Modules support reverse-engineering Siebel data structures (Business Components and Business Objects) and Enterprise Integration Manager (EIM) tables, data extraction from Siebel using data-level integration and data extraction and integration with Siebel using the EIM tables
JDE EnterpriseOne Knowledge Modules
The Oracle Data Integrator Knowledge Modules for JDE EnterpriseOne provide connectivity and integration of the JDE EnterpriseOne platform with any database application through Oracle Data Integrator. These KM support reverse-engineering of JDE EnterpriseOne data structures, data extraction from JDE EnterpriseOne (Direct Database Integration) and integration through the Z-tables to an JDE Application (Interface Table Integration)
Oracle Changed Data Capture Adapters/Attunity Streams Knowledge Modules
The Oracle Data Integrator CDC Knowledge Module provides integration from Oracle Changed Data Capture Adapters/Attunity Streams Staging Areas via a JDBC interface. This KM reads changed data, loads this data into a staging area and handles the Oracle Changed Data Capture Adapters/Attunity Streams context to ensure consistent consumption of the changes read.
Hyperion Adapters
Knowledge Modules and technologies have been added for integrating the Hyperion technologies using Oracle Data Integrator.
These KMs support the following Hyperion products:
Hyperion Financial Management, to load and extract metadata and data.
Hyperion Planning, to load metadata and data into Hyperion Planning.
Hyperion Essbase, to load and extract Essbase metadata and data.
Row-By-Row KMs for Debugging
Knowledge modules supporting row-by-row loading (LKM SQL to SQL (row by row)) and integration (IKM SQL Incremental Update (row by row)) have been introduced for debugging purposes. These KMs allow logging of each row operation performed by the KM.
Teradata Optimizations
Teradata knowledge modules have been enhanced for Teradata to enable best performances.
This includes the following features:
Support for Teradata Utilities (TTU).
Support for customized Primary Indexes (PI) for temporary tables.
Support for named pipes when using TTU.
Optimized Management of Temporary tables.
SAP ERP Adapter
The SAP ERP Adapter allows extraction of data from SAP ERP systems. The Oracle Data Integrator SAP ABAP Knowledge Modules included in this adapter provide integration from SAP ERP systems using SAP JCo libraries and generated ABAP programs.
SAP BW Adapter
The SAP BW Adapter allows extraction of data from SAP BW systems. The Oracle Data Integrator SAP BW Knowledge Modules included in this adapter provide integration from SAP BW using SAP JCo libraries and generated ABAP programs. This adapter supports ODS, Info Objects, Info Cubes, Open Hub and Delta Extraction.
KM Enhancements in Release 11.1.1
The following knowledge modules enhancements are new to this release.
KM Enhancements for New Core Features
Knowledge modules have been enhanced to support the core features added in this version of Oracle Data Integrator. The following KMs have been updated to support these features:
Support for Partitioning: Oracle RKM reverse-engineers partitions.
Datasets and Set-Based Operators: All IKMs have been updated to support this feature.
Automatic Temporary Index Management: Oracle and Teradata IKMs and LKMs have been updated to support this feature.
Oracle Business Intelligence Enterprise Edition - Physical
Oracle Data Integrator provides the ability to reverse-engineer View Objects that are exposed in Oracle Business Intelligence Enterprise Edition (OBI-EE) physical layer. These objects can be used as sources of integration interfaces.
Oracle Multi-Table Inserts
A new Integration KM for Oracle allows populating several target tables from a single source, reading the data only once. It uses the INSERT ALL statement.
Teradata Multi-Statements
A new Teradata Integration KM provides support for Teradata Multi-Statements, allowing integration of several flows in parallel.
No comments:
Post a Comment