ENRICHING THE ‘INTEGRATION AS A SERVICE’ PARADIGM FOR THE CLOUD ERA

ENRICHING THE ‘INTEGRATION AS A SERVICE’ PARADIGM FOR THE CLOUD ERA

INTRODUCTION TO CLOUD INTEGRATION:

• Cloud Integration is about simplifying technology so that it could be adopted by a much broader audience. 

• Cloud integration is the process of configuring multiple application programs to share data in the cloud. In a network that incorporates cloud integration; various applications communicate either directly or through third-party software.

• Cloud integration is the linking of information flows between cloud-hosted applications and other applications hosted in different clouds or in the datacenter.

• Without integration, cloud applications can’t share data with the rest of the Company and that limits or even eliminates the utility of the cloud.

ADVANTAGES OF CLOUD INTEGRATION: Cloud integration offers the following advantages over older, compartmentalized organizational methods:

We propose the following definition of cloud computing:

• Each user can access personal data in real time from any device and from any location with Internet access.
• Each user can integrate personal data such as calendars and contact lists served by diverse application programs.
• Each user can employ the same logon information (username and password) for all personal applications.
• The system efficiently passes control messages among application programs.
• By avoiding the use of data silos, data integrity is maintained.
• Cloud integration offers scalability to allow for future expansion in terms of the number of users, the number of applications, or both.

INTEGRATION as a service (IaaS):

• In B2B systems are capable of driving this new on-demand integration model because they are traditionally employed to automate business processes between manufacturers and their trading partners.
• That means they provide application-to-application connectivity along with the functionality that is very crucial for linking internal and external software securely.
• The use of hub & spoke (H&S) architecture further simplifies the implementation and avoids placing an excessive processing burden on the customer sides.
• The hub is installed at the SaaS provider’s cloud center to do the heavy lifting such as reformatting files.
• A spoke unit at each user site typically acts as basic data transfer utility.

THE EVOLUTION OF SaaS:

• There are several ways clouds can be leveraged inspiringly and incredibly for diverse IT problems.

• Today there is a small list of services being delivered via the clouds and in future, many more critical applications will be deployed and consumed.

• IT as a Service (ITaaS) is the most recent and efficient delivery method in the important IT landscape.
• Integration as a service (IaaS) is the budding and distinctive capability of clouds in fulfilling the business integration requirements.

1. The Web is the largest digital information superhighway

2. The Web is the largest repository of all kinds of resources such as web pages, applications comprising enterprise components, business services, POJOs, blogs, corporate data, etc.

3. The Web is turning out to be the open, cost-effective and generic business execution platform (E-commerce, business, auction, etc. happen in the web for global users) comprising a wider variety of containers, adaptors, drivers, connectors, etc.

4. The Web is the global-scale communication infrastructure (VoIP, Video conferencing, IP TV etc,)

5. The Web is the next-generation discovery, Connectivity, and integration middleware.

THE CHALLENGES OF SaaS PARADIGM:

1. Controllability

2. Visibility & flexibility

3. Security and Privacy

4. High Performance and Availability

5. Integration and Composition

6. Standards

• A number of approaches are being investigated for resolving the identified issues and flaws. Private cloud, hybrid and the latest community cloud are being prescribed as the solution for most of these inefficiencies and deficiencies.
• There are several companies focusing on this issue. Boomi (http://www.dell.com) is one among them. This company has published several well-written white papers elaborating the issues confronting those enterprises thinking and trying to embrace the third-party public clouds for hosting their services and applications.

• Integration Challenges: Challenges specific to integration

• The first issue is that the majority of SaaS applications are point

solutions and service one line of business.

• As a result, companies without a method of synchronizing data between multiple lines of businesses are at a serious disadvantage in terms of maintaining accurate data, forecasting, and automating key business processes.
• Real-time data and functionality sharing is an essential ingredient for clouds.

• Application Programming Interfaces (API) are Insufficient :

• Many SaaS providers have responded to the integration challenge by developing APIs.
• Unfortunately, accessing and managing data via an API requires a significant amount of coding as well as maintenance due to frequent API modifications and updates.
• No standardization or consensus on the structure or format of SaaS APIs.
• The IT department expends an excess amount of time and resources developing and maintaining a unique method of communication for the API of each SaaS application deployed within the organization.

• Data Transmission Security:
  • Transfer data from on-premise systems or applications behind the firewall with SaaS applications hosted outside of the client’s data center poses new challenges
  • Best-of-breed integration providers can offer the ability to do so by utilizing the same security as when a user is manually typing data into a web browser behind the firewall.
  • As SaaS applications were not initially designed keeping the interoperability requirement in mind, the integration process has become a little tougher assignment.
  • Data integrity, confidentiality, quality and value have to be preserved as services and applications are interlinked and saddled to work together.

The Impacts of Clouds:

• Integration is being stretched further to the level of the expanding Internet and this is really a litmus test for system architects and integrators.
• Interoperability between SaaS and non-SaaS solutions remains the lead demand as integration leads to business aware and people-centric composite systems and services.

APPROACHING THE SaaS INTEGRATION ENIGMA:

• Integration as a Service (IaaS) is all about the migration of the functionality of a typical enterprise application integration into the cloud for providing for smooth data transport between any enterprise and SaaS applications.
• Cloud middleware will be made available as a service.
• Due to varying integration requirements and scenarios, there are a number of middleware technologies and products such as JMScompliant message queues and integration backbones such as EAI, ESB,EDB, CEP, etc.
• Enterprise Application Integration(EAI)
• Enterprise Service Bus (ESB)
• Enterprise Data Bus(EDB)
• Complex Event Processing(CEP)

• For service integration, it is enterprise service bus (ESB)
• For data integration, it is enterprise data bus (EDB).
• Complex event processing (CEP) connects decoupled systems where Events are coming up fast.
• Cloud infrastructure is not very useful without SaaS applications that run on top of them, and SaaS applications are not very valuable without access to the critical corporate data that is typically locked away in various corporate systems.
• That brings out the importance of SaaS integration subject
• Integration is not easier either to implement as successful untangling from the knotty situation is a big issue.

The constraining attributes of SaaS applications are:

• Dynamic nature of the SaaS interfaces that constantly change
• Dynamic nature of the metadata native to a SaaS provider such as Salesforce.com
• Managing assets that exist outside of the firewall.
• Massive amounts of information that need to move between SaaS
• On-premise systems daily and the need to maintain data quality and integrity.

WHY SAAS INTEGRATION IS HARD?

Integration is more complicated: The most probable reasons are

• New integration scenarios
• Access to the cloud may be limited
  • Access to cloud resources (SaaS, PaaS, and the infrastructures) is more limited than local applications.
  • Once applications move to the cloud, custom applications must be designed to support integration because there is no longer that low level of access.
  • Enterprises putting their applications in the cloud or those subscribers of cloud-based business services are dependent on the vendor to provide the integration hooks and APIs.
  • But service providers fail to provide support.
• Dynamic Resources: Due to the dynamism factor that is sweeping the whole could ecosystem, application versioning and infrastructural changes are liable for dynamic changes. These would clearly impact the integration model.
• Performance: Clouds support application scalability and resource elasticity the network distances between elements in the cloud are no longer under our control. The round trip latency is an issue that slows down the cloud integration.

NEW INTEGRATION SCENARIOS:

1. within a Public Cloud:

• Two different applications are hosted in a cloud. The role of the cloud integration middleware (say cloud-based ESB or internet service bus (ISB)) is to seamlessly enable these applications to talk to each other.
• The possible sub-scenarios include these applications can be owned by two different companies. They may live in a single physical server but run on different virtual machines.

2. Homogeneous Clouds:

• The applications to be integrated are posited in two geographically Separated cloud infrastructures.
• The integration middleware can be in cloud 1 or 2 or in a separate cloud. There is a need for data and protocol transformation and they get done by the ISB.

3. Heterogeneous Clouds:

• One application is in public cloud and the other Application is private cloud.

• This is the currently dominating scene for cloud integration

THE INTEGRATION METHODOLOGIES:

1. Traditional enterprise integration tools deployed on-premise with special connectors to access cloud based applications: 
   • This is a most likely approach for IT organizations that have already standardized on an integration tool for integrating their enterprise applications, and now looking to extend that capability to include cloud applications.
   • Many integration vendors are also providing subscription based licensing options to help customers license their technologies for term-based projects. 
   • Examples include data integration technologies like Informatica’s PowerCenterproduct, message based EAI/ESB technologies, and appliance based integration tools.
2. Traditional enterprise integration tools hosted in the cloud:

• This approach is similar to the first option, except that the equipment for installing the integration software is hosted by a third party provider so that the customer does not have to worry about procuring and managing the hardware, or installing the integration software. 
• This is a good fit for IT organizations or systems integrators who have the skills and resources to build integration jobs and manage them on behalf of their end users or customers.
• It is a good fit for cloud-to-cloud integrations, but requires a secure VPN tunnel to access on-premise data.
• An example of a hosted integration technology is Cloud Edition on Amazon EC2. 

3. Integration-as-a-Service or On Demand Integration offerings :

• These are SaaS applications that are designed to deliver integration securely over the internet, and are able to integrate cloud applications to on-premise systems, cloud-to-cloud applications, or on-premise to on-premise applications without requiring the users to setup a VPN connection or provide any special access through their firewall. 
• This approach is a good fit for companies who care the most about ease of use, ease of maintenance, time to deployment, and are on a tight budget. 
• It is appealing to small and mid-sized companies, as well as large enterprises with a departmental application deployment. 
• good example is Informatica’s On Demand Integration Services.

No single approach is a perfect fit for all scenarios, and the right approach for any company will depend on some of the following factors:

• Who will build and maintain integration jobs – In-house IT resources, the application administrator/business analyst or both?
• The scope and complexity of the project – Is it a pilot project, a small departmental project or a strategic enterprise deployment?
• The size of the company – what is their budget for an integration tool and do they prefers a buy vs. lease approach?

CHARACTERISTICS OF INTEGRATION SOLUTIONS AND PRODUCTS:

• Connectivity refers to the ability of the integration engine to engage with both the source and target systems using available native interfaces.
• Semantic Mediation refers to the ability to account for the differences between application semantics between two or more systems. Semantics means how information gets understood, interpreted and represented within information systems.
• Data Mediation converts data from a source data format into destination data format.
• Data Migration means that the data in the old system is mapped.
• Data Security means the ability to ensure that information.
• Data Integrity means data is complete and consistent.
• Governance refers to the processes and technologies that surround a system or systems, which control how those systems are accessed and leveraged. Within the integration perspective, governance is about managing changes to core information resources, including data semantics, structure, and interfaces.

DATA INTEGRATION ENGINEERING LIFE CYCLE :

The pivotal phases of Data Integration Engineering Life Cycle are:

1. Understanding the existing problem domain means defining the metadata that is native within the source system.

2. Definition refers to the process of taking the information called during the previous step and defining it at a high level including what the information represents, ownership, and physical attributes.

3. Design the integration solution around the movement of data from one point to another accounting for the differences in the semantics.

4. Implementation refers to actually implementing the data integration solution within the selected technology.

5. Testing refers to assuring that the integration is properly designed and implemented and that the data synchronizes properly between the involved systems.

• We need to ensure that the data mediation mechanisms function correctly as well as review the overall performance, durability, security, modifiability and sustainability of the integrated systems.

SAAS INTEGRATION PRODUCTS AND PLATFORMS:

Jitterbit:

• Jitterbit is a fully graphical integration solution that provides users a versatile platform and a suite of productivity tools to reduce the integration efforts sharply.

• Apart from a scalable and secure server, Jitterbit provides a powerful graphical environment to help quickly design, implement, test, deploy, and manage the integration projects.

Jitterbit Integration Environment: Point-and-click GUI that enables to quickly configure, test, deploy and manage integration projects on the Jitterbit server.

Jitterbit Integration Server: A powerful and scalable run-time engine that processes all the integration operations, fully configurable and manageable from the Jitterbit application.

• Jitterbit is making integration easier, faster, and more affordable than ever before.

• Using Jitterbit, one can connect force.com with a wide variety of on-premise systems including ERP, databases, flat files and custom applications

The figure illustrates how Jitterbit links a number of functional and vertical enterprise systems with on-demand applications

BOOMI SOFTWARE:

Bungee Connect:

• For professional developers, Bungee Connect offers an application development and deployment platform that enables highly interactive applications integrating multiple data sources and facilitating instant deployment
• Bungee automates the development of rich UI and eases the difficulty of deployment to multiple web browsers.

OpSource Connect: OpSource services provide both enterprise-class cloud and managed server hosting that delivers the security, control, high-performance and ease of integration that enterprise customer’s demand.

OpSource Services Portfolio

OpSource Cloud Hosting: pay-as-you-go compute, networks and storage services

OpSource Managed Hosting: managed dedicated and virtual server hosting for SaaS and enterprise

OpSource Application Operations: application management services to scale operations.

http://www.opsource.net

SnapLogic:

• SnapLogic is a capable, clean solution for data integration that can be deployed in enterprise as well as in cloud landscapes.
• SnapLogic offers a solution that provides flexibility for today’s data integration challenges.
  • Changing data sources. SaaS and on-premise applications,Web APIs, and RSS feeds

• Changing deployment options. On-premise, hosted, private and public cloud platforms

• Changing delivery needs. Databases, files, and data services

http://www.snaplogic.com

Bluewolf:

• Bluewolf proactively alerts its customers of any issues with integration and helps to solves them quickly.
• The Bluewolf Integrator integrates with Salesforce, BigMachines, Oracle, SAP, Microsoft SQL server, MySQL, and supports flat files, such as CSV, XHTML and many more.

http://www.bluewolf.com/