Showing posts with label Architecture. Show all posts
Showing posts with label Architecture. Show all posts

Friday, October 22, 2010

Calling Web Services Dynamically

In normal SOA architecture projects, we have static web services which get consumed by clients but in some cases ( especially for Orchestration) we may need to call web services dynamically through client.

There are several ways available to consume services dynamically and you may get lots of material on Google for these methods. I am collecting some of the ways I have tried out, for reference purpose and will let you know pros and cons of each of them.

Following are the ways to call web services dynamically
1) Call through a Java client
2) Call through a PL/SQL procedure
3) Call through a BPEL process.

1) Call through a Java Client: As I am from Java back ground, I used it as my first choice.
With use of javax.xml and javax.jws package, one can invoke web services.
Advantage of calling web services through Java client is, it is very easy to maintain code. Exception handling is also very rich. Formatting and altering xml input and output data is also quite easy.
Limitations/issues with this approach are, though it supports all types of protocols but it require to write code separately for each format. Second issue with this approach is, it require XML to object and object to XML conversion, which make it a bit slow.
Following is a code snippet for your help. I am calling a service support SOAP1.2 RPC.

public String callService(String namespace, String Servicename,
String porttype, URL WSDLURL, String dataload) {
String targetNS = namespace;
QName serviceName = new QName(targetNS, Servicename);
QName portName = new QName(targetNS, porttype);
URL wsdlURL = WSDLURL;

Service service = Service.create(wsdlURL, serviceName);
service.addPort(portName, SOAPBinding.SOAP11HTTP_BINDING,
wsdlURL.toExternalForm());
Dispatch

dispatch = null;
StreamSource xmlSource = null;
ByteArrayOutputStream xmlByteArray = new ByteArrayOutputStream();
TransformerFactory fac = TransformerFactory.newInstance();
dataload = addSoapEnvelope(dataload);

try {
Transformer x = fac.newTransformer();
x.transform(new StreamSource(new StringReader(dataload)),
new StreamResult(xmlByteArray));
xmlSource =
new StreamSource(new StringReader(xmlByteArray.toString()));

dispatch =
service.createDispatch(portName, Source.class, Mode.MESSAGE);
} catch (TransformerConfigurationException e) {
e.printStackTrace();
} catch (TransformerException e) {
e.printStackTrace();
}
//calling WS with help of Dispatcher and getting Result as Source
Source result = dispatch.invoke(xmlSource);

StringWriter outputxml = new StringWriter();
Transformer outx;
try {
outx = fac.newTransformer();
outx.transform(result, new StreamResult(outputxml));
} catch (TransformerConfigurationException e) {
e.printStackTrace();
} catch (TransformerException e) {
e.printStackTrace();
}
String resultstr = outputxml.toString();
if (resultstr.indexOf("?>") > 0) {
resultstr = resultstr.substring(resultstr.indexOf("?>") + 2);
}
MessageFactory factory;
StringWriter output = null;
//Converting Source result to SOAP Message
try {
factory = MessageFactory.newInstance();
SOAPMessage message = factory.createMessage();
message.getSOAPPart().setContent(new StreamSource(new StringReader(resultstr)));
message.saveChanges();
NodeList list = message.getSOAPBody().getChildNodes();
Node node = list.item(0);
output = new StringWriter();
Transformer out;
out = fac.newTransformer();
out.transform(new DOMSource(node), new StreamResult(output));

} catch (TransformerConfigurationException e) {
e.printStackTrace();
} catch (TransformerException e) {
e.printStackTrace();
} catch (SOAPException e) {
e.printStackTrace();
}
//Converting SOAP Return to String for processing
String outputstr = output.toString();
//removing from returned output
if (outputstr.indexOf("?>") > 0) {
outputstr = outputstr.substring(outputstr.indexOf("?>") + 2);
}
return (outputstr);
}
Please note that it is not the best way to do it. You may find better way to do the same like using WS-I APIs


2) Call through a PL/SQL procedure: If you are using event based program where events are getting stored in DB, you may need to call services from DB side. I am aware of Oracle Database, so I am using it only but I think similar option must be available with other databases as well.
Advantage is, you need not have additional program call to invoke WS and you can secure some n/w bandwidth and processing time. It is hard to believe but database too have some very good API's to handle XML data.
Limitation is, it doesn't support security and oracle standards/recommendations do not suggest for this approach. But in some places it may work as life saver.
FUNCTION new_request(method IN VARCHAR2,
namespace IN VARCHAR2)
RETURN request AS
req request;
BEGIN
req.method := method;
req.namespace := namespace;
RETURN req;
END;
Following is the code snippet from soap_rpc package for raising new request:

PROCEDURE add_parameter(req IN OUT NOCOPY request,
name IN VARCHAR2,
type IN VARCHAR2,
value IN VARCHAR2) AS
BEGIN
req.body := req.body ||
'<' xsi:type="'||type||'">'||value||'';
END;

PROCEDURE generate_envelope(req IN OUT NOCOPY request,
env IN OUT NOCOPY VARCHAR2) AS
BEGIN

env := '

<' '||req.namespace|| ' SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">'||
req.body||'';
END;

For more details you may refer to http://www.oracle.com/technology/tech/webservices/htdocs/samples/dbwebservice/DBWebServices_PLSQL.html page.


3) Call through a BPEL process This is the most declarative approach where the services of similar input and output type can be called dynamically. Here I have used Oracle SOA Suite 11g and again I am pretty much sure that other SOA Suite must have similar feature.
Here I have created a dummy WSDL with request and response and associated with BPEL Process. Now at Run time, we can assign endpoint reference to dummy partner link to invoke the service.
Advantage is, it is purely declarative in nature; it is very useful in case of load balancing where we can use multiple end points for similar feature and redirect them to other endpoint if one service end point is down.
Limitation is, it will work only in case of similar messages as the message types are bind with BPEL.

Following is the code of Dynamic BPEL. bpel file snippet for your reference:

Thursday, October 21, 2010

Essential Features for an Orchestration Project for inbound and outbound data

For an orchestration project it is required that the defined system have capability to communicate with other external systems. There are multiple ways defied by which two (homogeneous or heterogeneous) systems can communicate with each other like event driven communication on Push –pull model or peer to peer architecture on tightly coupled model .

Major challenge for any Orchestration systems is
• Multiple model and architecture support.
• Dynamic addition of models and events in system.

In summary, an orchestration system should capable to listen external system(s) and perform predefine or registered action(s). These external system triggers or events which may be pre seeded or user may register them after implementing orchestration system.
All these communication can be broadly divided in to two categories.
A. Outbound communication
B. Inbound communication
A) Outbound and related inbound communication
In outbound communication, it is possible that the external system will pass some data back to original system. This related inbound may be in form of confirmation note, some payload (in case of synchronous service call) or exception or error message. We will discuss features related to outbound communication and related inbound communication in following section.
For an outbound communication system, system should provide, but not limited to, following features.
Routing
Routing is an essential part of outbound communication. It will provide flexibility of calling different external system or different services in same external system on the basis of provided conditions.

Transformation of Data
While communicating to different system we need to transform the data load before passing to other system as per their understandable format. To achieve this orchestration system should have provision to transform data as per user setup.
Business rule connectivity
As the business needs are changing day to day basis, a system should be scalable enough to accommodate new and change business requirements without stopping the current business operations. To achieve it system should support business rule connectivity so that they can add new routing rules or change pre existing rules without downtime.
Filtration criteria
Audit Trails
An Orchestration system should have a mechanism to check the records of communication done by the system and the status of these communications.
To achieve it, there should be a Audit trail mechanism to record all these communications for future references.
Exception Handling
In case of any abrupt behavior in system, a system should be fail safe and capable to record the incident for auditing purpose.
Component as service
Orchestration system has its own data which may be required by external system for their references. This data may be in form of rules, audit trails or setups got created by user setup or actions.
There should be some open endpoints available for other systems to request for this information and orchestration system should provide the response with proper authentication and authorization.
Data Security
An orchestration system access and pass data to external system(s). This data may be confidential and may have viewing restrictions. In this case, an Orchestration system should have proper authentication and authorization mechanism for data security.
Ability to call both synchronous and a synchronous process
An orchestration system may call services from heterogeneous system and these services may be defined as synchronous (will provide reply at the same time) or a synchronous (will process the data and will reply later).
An Orchestration system should facilitate both types of services without any loss of data.
Ability to place payload in persistent model
An external system may not be available for an instance of time. An Orchestration system should have a provision to store request in persistent model like message queues or database in such case so that it can be passed to external system once it came back from down time.
Related inbound data load
An inbound related to outbound of orchestration system should have following features
No Duplicity
In case of duplication of input data, system should intelligent enough to filter these inputs and accept only one of them.
Response handling
For a synchronous communication from out bound, related response will be handled by inbound. This inbound will be specific to the instance of outbound call and inbound should have mechanism to relate it and associate it with correct outbound instance.
Exception handling for response
In case of any exception thrown by external system for outbound call, inbound system should capture it and record it for audit trail purpose.

B) Inbound communication and its data load
In inbound communication, an Orchestration system has to listen from heterogeneous systems and perform on the basis of these events. As all the inbound data may not be useful for orchestration system so there is a challenge of filtering out only those events which are useful for business and registered with orchestration system.
For an inbound communication system, system should provide, but not limited to, following features.

Ability to push event definition and related payload to persistent mode (such as DB)
As the data coming to system is valuable and any business cannot afford to lose this data, it is essential to keep inbound data in any persistent model so that it can be retrieved back for reference and audit trail.
An Orchestration system should have a provision to persist inbound data in any persistent model like Database.
Ability to identify the incoming events and registration of new events
As the business needs and requirements are changing very frequently with business growth and time line, an orchestration system should have provision to listen new business events that may come from any new system or any preregistered system.
There should be a provision to make change in already registered events so that they can change with change in external system.

Filtration of events and see them as inbound
As all incoming data to inbound system may not be useful for orchestration system, an orchestration system should have provision to filter the required events and ignore other events.
Audit trails
In case of any error or failure in system at inbound side, it is essential to record the incident and keep the system up for other request. An orchestration system should be robust enough to keep the system up in case of any failure and recode the incident in audit trail to reference.
Ability to define and use relationship between events
As business events may interrelated and/or dependent on each other, it is essential for an orchestration system to keep a track of relationship between business events so that the system can behave properly and maintain its data integrity.
Transformation capability of inbound payload
Like outbound, inbound communication may involve heterogeneous system and it is essential to transform the data for further processing. In orchestration system, there should be a provision by which user can define the transformation process of inbound data so that system can transform and understand it.

Exception handling with notification feature
If there is any discrepancy in inbound data because of which orchestration system cannot process the event further, it should be tracked as an error in system and orchestration system should have provision to send notification to defined user(s) so that the error can be rectified

Tuesday, October 19, 2010

Protocol and Network

While writing my first code for client server environment, I was quite amazed to see how it works. It was a thick client chat server and client code, written on Swings with Socket program in early 2000.
First thing I learned by this program was the term "Protocol". For me protocol is like a language grammar. While communicating, we need a mutually understandable language and a medium by which we can pass the expression.In same way protocol help two different entities to fix their grammar and communicate on network medium.
It is like a universal truth, any types of machines can communicate with each other if they have set protocol and communication medium.
Now a days, most popular medium is 'Internet' as many of us must be aware of that internet is also a network so any machine connected with a network can use network as medium.
Other used mediums are , Intranet, LAN(Local area Network) , WAN( Wide Area Network) etc.
Now in terms of Protocol, most popular and simpler is HTTP (Hyper Text Transfer Protocol).
HTTP is a grammar by which two machine can understand each other's messages.
Think about a letter you have received filled with 1's and 0's..you will definitely feel puzzled..isn't it?
but if you know binary number format, you will try to convert it to some meaningful number's..
but what after it.. will those number give you any clue for the message.. you may further try to decode it with some pattern..but what if you know the pattern?..life will be easy for you!!..
you can read the message now..
Protocols help machines to write data on network and recover it from network so that machine can understand it.
Some other popular protocols used now a days are.. FTP (for File Transfer), WAP( for wireless Access) SOAP(Simple Object Access Protocol for web services communication) .
One thing I would like to mention over here that, it is not necessary to use Standard protocol only for communication. If you know socket programming then you can create a socket and Server socket where socket can pass any message to your server socket in your defined protocol.
I have used it in my chat program, where multiple messages were sent via a delimiter(||).
I will post a code snippet of writing such program later.