Friday, August 15, 2014

MTOM/XOP example using JAX-WS / SOA Suite & OSB - 2

Oracle SOA Suite

Below we show how to consume the service that is MTOM enabled in Oracle SOA Suite. We implement it by attached the MTOM client WS-Policy to the reference binding as shown below :




Oracle Service Bus

Oracle Service Bus supports parsing inbound messages & sending outbound messages in MTOM/XOP format. This capability is available for any XML based (or) WSDL based services.

The below image shows how to enable MTOM support as part of the message handling tab for the proxy services.

Include Binary data by Reference - can be used for pass through scenarios where in we don't really need to parse the attachment (or) to pass the attachment directly to MFL or Java callout. The attachment will be available as part of $body/ctx:binary-content variable.

Include Binary data by Value - can be used when you want to validate the xml schema (or) scenarios where the outbound service doesn't support MTOM. In this case, the xop:include will be replaced with the actual base64 binary content.

One important point to note is that, if MTOM is enabled on a OSB Service we cannot attach any other OWSM policy to it( not mentioned in any documents as far as i know)


MTOM/XOP example using JAX-WS / SOA Suite & OSB - 1

Binary data sent using SOAP across the wire needs to be encoded using base64Encoding to convert it as text. This process will increase the size of data by 33%. Thus to optimize the transmission of xml data of type xs:base64Binary and xs:hexBinary in SOAP messages, W3C recommends usage of MTOM/XOP (Message Transmission Optimization Mechanism / XML Optimized Packaging). When the transport protocol is HTTP, MIME attachments are used to carry that data while at the same time allowing both the sender and the receiver direct access to the XML data in the SOAP message without having to be aware that any MIME artefacts were used to marshal the base64Binary or hexBinary data. MTOM primarily provides the following:
  • Encoding of the binary data
  • Removal the binary data from the SOAP Envelope
  • Compression of the Binary data
  • Attach the binary data to the MIME package
  • Pointer to the MIME package in the SOAP Envelope
JAX-WS provides annotations support for MTOM/XOP through annotations.

JAVA - XML Type conversion for MTOM

The java types/classes javax.activation.DataHandler, java.awt.Image, java.xml.transform.Source are by default converted to xs:base64Binary or xs:hexBinary XML types. While the XML types xs:base64Binary or xs:hexBinary are converted to byte[] by default.

One of the highlights of the MTOM specification is that, it doesn't mandate that always the data will be sent as attachments by default. It allows us to specify the threshold limit after which the data needs to sent as attachment. This way for lesser data need not be passed as attachment and thus can avoid unnecessary overhead.

@MTOM(threshold=<Threshold Value>)

Steps to Enable MTOM
  • Annotate the datatype that needs to be sent as attachment
  • Enable MTOM on both the Web Service & its client
Example (Start with Java)

Lets create the class 'DocumentManager' and define a method to archive the incoming document into the disc... The document sent across the wire will use MTOM providing an optimized transmission.


public class DocumentManager {
    public DocumentManager() {
        super();
    }
    
    public void archiveDocument(javax.activation.DataHandler _pDocumentHandler) {
        try
        {
            //Hardcoding stuff here...you can get the content type to set the extension dynamically....
            File fileToArchive = new File("/tmp/"+"test.pdf");             
            FileOutputStream outputStream = new FileOutputStream(fileToArchive);
            _pDocumentHandler.writeTo(outputStream); //Writes the stream into the disk.
            outputStream.flush();
            outputStream.close();
            
        }
        catch(Exception e) {
            e.printStackTrace();
        }
        
    }

Now lets expose the class as a JAX-WS web service and enable MTOM for the same:

 









The generated service would look something like below:
package fmw.techsamples.java.ws.mtom;

import java.io.File;
import java.io.FileOutputStream;

import javax.jws.WebService;

import javax.xml.ws.soap.MTOM;

@WebService(endpointInterface = "fmw.techsamples.java.ws.mtom.DocumentManagerPortType")
@MTOM(enabled=true,threshold=1024)
public class DocumentManager {
    public DocumentManager() {
        super();
    }
    
    public void archiveDocument(javax.activation.DataHandler _pDocumentHandler) {
        try
        {
            //you can get the content type to set the extension dynamically....
            File fileToArchive = new File("/tmp/"+"test.pdf"); 
            
         FileOutputStream outputStream = new FileOutputStream(fileToArchive);
         _pDocumentHandler.writeTo(outputStream);
         outputStream.flush();
         outputStream.close();
            
        }
        catch(Exception e) {
            e.printStackTrace();
        }
        
    }
}

I have set the threshold  to 1024 bytes, which means till that threshold is reached MTOM will not be used. The below image shows the WSDL & XSD definition generated after deploying the web service.
The WSDL would contain the WS policy definition for MTOM/XOP. Also, note that the XSD definition shows the input as 'base64Binary'



Ok now to testing. We will use SOAP UI to do the testing, but to trap the http request that is sent through the wire to the server(to show that the request has been optimized through MTOM), we will make use of HttpAnalyzer in the jdeveloper. First we start  the HttpAnalyzer to listen in port 8099 and then set the same as the proxy in SOAP UI (via preferences as shown below).

This would mean that all the request flows through HttpAnalyzer and we can trap the request sent across the wire. In the SOAPUI, we can use the Insert file as Base64 option to convert a pdf to base64binary (or you could use openssl or base64 command to do that easily in a mac). Also, we need to set the 'Enable MTOM' to true (equivalent to setting new MTOMFeature() in a java client)



After executing the request, lets see the request sent via HttpAnalyzer. The below SOAP request shows the XOP-include element having the reference to binary data sent separately as MIME attachments.
Also, the end pdf is shown written to the /tmp directory in the server. 


In the next instalment I will show a quick sample on how to access the MTOM enabled service through Oracle SOA Suite / OSB.

Wednesday, August 13, 2014

Implementing Asynchronous communication with Oracle Service Bus

Its a more prevalent requirement to design asynchronous web services when dealing with long running business process and its well known fact that Oracle SOA suite has got a very robust implementation support using WS-Addressing under the hoods.

But there aren't many examples available which depicts how the same can be implemented through technologies like Oracle Service Bus. There are couple of well-defined patterns available like

Using JMS for example to decouple the request and callback


Using SOA-Direct / SB Transport binding - Using WS-Addressing internally.




But there may be situations like:

  • The architecture doesn't allow using proprietary transport bindings 
  • Using JMS is a overkill. 
  • Require better fault management requirement.
For these situation fortunately, we can still use WS-Addressing in OSB to decouple request/response albeit with additional extra work.

The suggested pattern is shown below :



The WS-Addressing has been forwarded to the external system using a customer header to decouple the pattern logic from the external system. I will try to explain the implementation with a much simpler example.
WS-Addressing purpose is to provide a standard way to specify where a message is going and where to send the response (or) the fault back. The below wsdl specifies the contract for Hello World Asynchronous service with the request sent using 'sayHello' operation and the response/fault is retrieved through 'sayHelloResponse' (or) 'sayHelloFaultResponse' respectively.

Schema

<?xml version="1.0" encoding="UTF-8" ?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://techsamples.fmw/v1/HelloWorldAsync"
    targetNamespace="http://techsamples.fmw/v1/HelloWorldAsync" elementFormDefault="qualified" xmlns:flt="http://techsamples.fmw/v1/FaultObject">
    <xsd:import schemaLocation="../xsd/techsamples.fmw.cmn.FaultObject.xsd" namespace="http://techsamples.fmw/v1/FaultObject"></xsd:import>
    <xsd:element name="sayHello" type="tns:SayHelloType"/>
    <xsd:complexType name="SayHelloType">
        <xsd:sequence>
            <xsd:element name="name" type="xsd:string"/>
        </xsd:sequence>
    </xsd:complexType>
    
    <xsd:element name="sayHelloResponse" type="tns:SayHelloResponseType"/>
    <xsd:complexType name="SayHelloResponseType">
        <xsd:sequence>
            <xsd:element name="result" type="xsd:string"/>
        </xsd:sequence>
    </xsd:complexType>
    
    <xsd:element name="sayHelloFaultResponse" type="tns:SayHelloFaultResponseType"/>
    <xsd:complexType name="SayHelloFaultResponseType">
        <xsd:sequence>
            <xsd:element ref="flt:faultObject"/>
        </xsd:sequence>
    </xsd:complexType></xsd:schema>

WSDL Contract
<?xml version="1.0" encoding="UTF-8" ?><wsdl:definitions targetNamespace="http://techsamples.fmw/v1/HelloWorldAsync" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
    xmlns:tns="http://techsamples.fmw/v1/HelloWorldAsync" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/"
    xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
    xmlns:flt="http://techsamples.fmw/v1/FaultObject">
    <wsdl:types>
        <xsd:schema>
            <xsd:import schemaLocation="techsamples.fmw.ebm.HelloWorldAsync.xsd"
                namespace="http://techsamples.fmw/v1/HelloWorldAsync"/>
            <xsd:import schemaLocation="../xsd/techsamples.fmw.cmn.FaultObject.xsd" namespace="http://techsamples.fmw/v1/FaultObject"></xsd:import>            
        </xsd:schema>
    </wsdl:types>
    <wsdl:message name="serviceFault">
        <wsdl:part name="serviceFault" element="flt:faultObject"/>
    </wsdl:message>
    <wsdl:message name="sayHello">
        <wsdl:part name="sayHello" element="tns:sayHello"/>
    </wsdl:message>
    <wsdl:message name="sayHelloResponse">
        <wsdl:part name="sayHelloResponse" element="tns:sayHelloResponse"/>
    </wsdl:message>
    <wsdl:message name="sayHelloFaultResponse">
        <wsdl:part name="sayHelloFaultResponse" element="tns:sayHelloFaultResponse"/>
    </wsdl:message>
    
    <wsdl:portType name="HelloWorldAsync.PortType">
        <wsdl:operation name="sayHello">
            <wsdl:input message="tns:sayHello"/>
        </wsdl:operation>
    </wsdl:portType>
    <wsdl:portType name="HelloWorldAsyncCallback.PortType">
        <wsdl:operation name="sayHelloResponse">
            <wsdl:input message="tns:sayHelloResponse"/>
        </wsdl:operation>
        <wsdl:operation name="sayHelloFaultResponse">
            <wsdl:input message="tns:sayHelloFaultResponse"/>
        </wsdl:operation>
    </wsdl:portType>
    
    <wsdl:binding name="HelloWorldAsync.Binding" type="tns:HelloWorldAsync.PortType">
        <soap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
        <wsdl:operation name="sayHello">
            <soap:operation soapAction="sayHello" style="document"/>
            <wsdl:input name="sayHello">
                <soap:body use="literal"/>
            </wsdl:input>
        </wsdl:operation>
    </wsdl:binding>
    
    <wsdl:binding name="HelloWorldAsyncCallback.Binding" type="tns:HelloWorldAsyncCallback.PortType">
        <soap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
        <wsdl:operation name="sayHelloResponse">
            <soap:operation soapAction="sayHelloResponse" style="document"/>
            <wsdl:input name="sayHelloResponse">
                <soap:body use="literal"/>
            </wsdl:input>
        </wsdl:operation>
        <wsdl:operation name="sayHelloFaultResponse">
            <soap:operation soapAction="sayHelloFaultResponse" style="document"/>
            <wsdl:input name="sayHelloFaultResponse">
                <soap:body use="literal"/>
            </wsdl:input>
        </wsdl:operation>
    </wsdl:binding>
    <wsdl:service name="HelloWorldAsync.Service">
        <wsdl:port binding="tns:HelloWorldAsync.Binding" name="HelloWorldAsync.Port">
            <soap:address location="http://@@osb.target.address@@/techsamples.fmw/HelloWorldAsync"/>
        </wsdl:port>
    </wsdl:service>
    <wsdl:service name="HelloWorldAsyncCallback.Service">
        <wsdl:port binding="tns:HelloWorldAsyncCallback.Binding" name="HelloWorldAsyncCallback.Port">
            <soap:address location="http://@@osb.target.address@@/techsamples.fmw/HelloWorldAsyncCallback"/>
        </wsdl:port>
    </wsdl:service></wsdl:definitions>

Artefacts

  • HelloWorldAsync.Proxy - Proxy to handle the HelloWorldAsync.Binding(sayHello operation)
  • HelloWorldAsyncCallback.Proxy - Proxy for the callback wsdl binding(sayHelloResponse & sayHelloFaultResponse operations). The expectation is that the response will always flow through this proxy. In this example, there is no external service,so HelloWorldAsync.proxy will directly send the callback response via this proxy. 
  • HelloWorldAsyncClientCallback.biz - Abstract business service representing the actual callback client whose endpoint & the operation(action) are set at runtime using WS-Addressing.
  • HelloWorldAsync Client BPEL Process - Client BPEL Process - The client process which call the HelloWorldAsync.sayHello operation and waiting for the callback.
HelloWorldAsync.Proxy


The SetCallbackURI assign activity in the CommonInitializationPipelinePair determines the recipient for the actual route(ideally this would be that of the external service) but for the simplicity we are setting it to the callback proxy directly. The xquery would look like something below:


if(fn:exists($inbound/ctx:transport/ctx:request/tp:headers/http:Host)) then
(
 <Callback>{fn:concat("http://",$inbound/ctx:transport/ctx:request/tp:headers/http:Host/text()[normalize-space(.)],"/techsamples.fmw/HelloWorldAsyncCallback")}</Callback>
)
else
(
 if($header/*:To/text()[normalize-space(.)]!='') then
 ( 
  <Callback>{fn:concat("http://",fn:substring-before(fn:substring-after($header/*:To/text()[normalize-space(.)],"http://"),"/"),"/techsamples.fmw/HelloWorldAsyncCallback")}</Callback> 
 )
 else
 (
  <Callback/> 
 )
)

the below image shows the routing part which sets the WS-Addressing for the callback to forward the response to the actual client.


Similar to the response callback, if there are any faults in the processing, it will managed by 'MainErrorHandler' and the fault is dispatched to the callback proxy as a fault response:


Also, note that on the call back proxy, you can decide the operational branch selection based on the WS-Addressing as well as shown below: HelloWorldAsyncCallback.Proxy Now the callback's responsibility is to forward the response back to the client that is waiting for the response. Firstly we capture the final destination from the WS-Addressing header:-


The routing below shows the response is forwarded to the HelloWorldAsyncClientCallback business service, which in turn sends the response to the Callback Client using WS-Addressing.


HelloWorldAsyncClientProcess The below client bpel process invokes the 'sayHello' operation in the HelloWorldAsync proxy and waits for the response or fault via the callback ports:



Lets run the client process and see the outcome ...



Lets throw a dummy error from the proxy mimicking a fault from the external service and see how the fault is propagated to the client.

That' all.. In the next instalment I will try to put a sample to implement the same scenario using jaxws web services.

Friday, August 8, 2014

Simple Script to switch jdk in mac

With the Oracle FMW 12c version release, one of the normal requirement is to switch between multiple jdk versions for 12c & 11g build environments in mac. Below is a simple script which can help you to do that:

function switchjdk() {  
  if [ -z $1 ]; then
 echo "please provide with the right jdk version to switch"
  else
 echo "switching to jdk version --- $1"
 export PATH=$(echo $PATH | sed -E -e "s;:$JAVA_HOME/bin?;;" -e "s;$JAVA_HOME/bin:?;;")
 export JAVA_HOME=$(/usr/libexec/java_home -v $1)
 export PATH=$JAVA_HOME/bin:$PATH
 echo $(java -version)
  fi  
 }  
switchjdk $1

Oracle Service Bus Deployment - Offline export of Config jar

With PS6, Oracle Service Bus provides the utility jar (com.bea.alsb.tools.configjar.ant.ConfigJarTask) to export the OSB configuration jar without requiring OEPE.

A sample ant task as provided by the Oracle documentation is given below:

<project name="ConfigExport" default="usage" basedir=".">

    <property environment="env" /> 
    <property name="mw.home" location="${env.MW_HOME}" /> 
    <property name="wl.home" location="${env.WL_HOME}" /> 
    <property name="osb.home" location="${env.OSB_HOME}" /> 

    <taskdef name="configjar" 
             classname="com.bea.alsb.tools.configjar.ant.ConfigJarTask" /> 

    <target name="init">
       <property name="task.debug" value="false" /> 
       <property name="task.failonerror" value="true" /> 
       <property name="task.errorproperty" value="" /> 
    </target>

    <target name="run" depends="init">
       <fail unless="settingsFile"/>
       <configjar debug="${task.debug}" 
                  failonerror="${task.failonerror}" 
                  errorProperty="${task.errorproperty}" 
                  settingsFile="${settingsFile}" />
    </target>
</project>

The task expects a mandatory settingsFile parameter, which dictates the rules (files inclusion/exclusion) with regards to OSB project that needs to be exported.

A sample settingsFile as provided by the Oracle documentation is given below:
<configjarSettings xmlns="http://www.bea.com/alsb/tools/configjar/config">
    <source>
        <project dir="/scratch/workspaces/myworkspace/projectX"/>
        <project dir="/scratch/workspaces/myworkspace/projectY"/>
        <extensionMapping>
            <mapping type="Xquery" extensions="xquery,xq,xqy"/>
            <mapping type="XML" extensions="toplink"/>
        </extensionMapping>
    </source>
    <configjar jar="/scratch/workspaces/myworkspace/sbconfig.jar">
        <resourceLevel/>
    </configjar>
</configjarSettings>

The source element defines the source artefacts that needs to be picked up during the load phase and the configjar element defines the details about the configuration jar and what projects/resources it should include.

In this post we will take a look at how this task can be mavenized and the settingsFile can be externalized, so that we don't need to explicitly create different settingsFile for each deployment unit.

The first step is to create the custom settingsFile and load the same into the maven repository.

osb-project-settings.xml

<?xml version="1.0" encoding="UTF-8" ?>
<p:configjarSettings xmlns:p="http://www.bea.com/alsb/tools/configjar/config">
    <p:source>
        <p:project dir="@@osb.project.dir.path@@"/>
        <p:fileset>
            <p:exclude name="*/.settings/**" />
            <p:exclude name="*/import.*" />
            <p:exclude name="*/pom.xml" />
            <p:exclude name="*/target/**" />
            <p:exclude name="*/security/**" />
            <p:exclude name="*/import.*" />
            <p:exclude name="*/alsbdebug.xml" />
            <p:exclude name="*/configfwkdebug.xml" />
            <p:exclude name="*/*settings.xml" />
            <p:exclude name="*/@@osb.project.export.jar@@" />
        </p:fileset>
    </p:source>
    <p:configjar jar="@@osb.project.export.jar@@">
        <p:projectLevel includeSystem="@@osb.include.system@@">
            <p:project>@@osb.project.name@@</p:project>
        </p:projectLevel>
    </p:configjar>
</p:configjarSettings>

pom.xml


<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
    http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion> 
    <parent>
        <groupId>techsamples.fmw</groupId>
        <artifactId>parent</artifactId>
        <version>1.0</version>
    </parent>
    <groupId>techsamples.fmw</groupId>
    <artifactId>osb-build-config</artifactId>
    <packaging>jar</packaging>
    <version>1.0</version>
    <name>osb-build-config</name>
    <!-- copy osb-project-settings.xml  & osbImport.py files -->
    <build>
        <resources>
            <resource>
                <directory>${project.basedir}</directory>
                <includes>
                    <include>osb-project-settings.xml</include>
                    <include>osbImport.py</include>
                </includes>
            </resource>
        </resources>
    </build>
</project>

Now lets load the osb-project-settings-file into the maven repository

The osb-project-settings.xml file will be unpacked from the repository as part of the deployment process and all the tokens are substituted during runtime with the osb project specific details. This way we don't need to have different settings file for each project.

Below are the plugin details for

  • Unpacking the settings file
  • Replacing the tokens in the settings file
  • Export/Deploy the osb configuration jar 



<plugin>
    <artifactId>maven-dependency-plugin</artifactId>
    <dependencies>
        <dependency>
            <groupId>techsamples.fmw</groupId>
            <artifactId>osb-build-config</artifactId>
            <version>1.0</version>
        </dependency>
    </dependencies>
    <executions>
        <execution>
            <id>unpack-osb-build-config-files</id>
            <phase>process-resources</phase>
            <goals>
                <goal>unpack</goal>
            </goals>
            <configuration>
                <artifactItems>
                    <artifactItem>
                        <groupId>techsamples.fmw</groupId>
                        <artifactId>osb-build-config</artifactId>
                        <version>1.0</version>
                        <overWrite>true</overWrite>
                        <outputDirectory>${project.build.directory}</outputDirectory>
                        <includes>
                            *
                        </includes>
                    </artifactItem>
                </artifactItems>
                <overWriteReleases>true</overWriteReleases>
                <overWriteSnapshots>true</overWriteSnapshots>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>com.google.code.maven-replacer-plugin</groupId>
    <artifactId>replacer</artifactId>
    <version>1.5.3</version>
    <executions>
        <execution>
            <id>replace-tokens</id>
            <phase>process-resources</phase>
            <goals>
                <goal>replace</goal>
            </goals>
        </execution>
    </executions>
    <configuration>  
        <includes>  
            <include>${project.build.directory}/osb-project-settings.xml</include>    
        </includes>  
        <replacements>
            <replacement>
                <token>@@osb.project.export.jar@@</token>
                <value>${project.artifactId}-${project.version}.jar</value>
            </replacement>
            <replacement>
                <token>@@osb.project.dir.path@@</token>
                <value>${basedir}/${project.artifactId}</value>
            </replacement>
            <replacement>
                <token>@@osb.project.name@@</token>
                <value>${project.artifactId}</value>
            </replacement>
            <replacement>
                <token>@@osb.include.system@@</token>
                <value>${osb.include.system}</value>
            </replacement>
        </replacements>
    </configuration>
</plugin>
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.2.1</version>
    <executions>
        <execution>
            <id>export-sbconfigjar</id> 
            <phase>package</phase>
            <goals>
                <goal>exec</goal>
            </goals>
            <configuration>
                <executable>java</executable>
                <commandlineArgs>-Dosb.home=${osb.product.home} -Dweblogic.home=${wls.product.home} -classpath ${osb.classpath} com.bea.alsb.tools.configjar.ConfigJar -settingsfile ${project.build.directory}/osb-project-settings.xml</commandlineArgs>
                <removeAll>true</removeAll>
            </configuration>
        </execution>
        <execution>
            <id>deploy</id>
            <phase>deploy</phase>
            <configuration>
                <executable>java</executable>
                <commandlineArgs>
                    -Dosb.home=${osb.product.home} -Dweblogic.home=${wls.product.home} -classpath ${osb.classpath}
                    -Dwls.deploy.userid=${wls.deploy.userid}
                    -Dwls.deploy.password=${wls.deploy.password}
                    -Dadmin.target.serverURL=t3://${admin.target.address}
                    -Dosb.project.name=${project.artifactId}
                    -Dosb.project.export.jar=${project.build.directory}/${project.artifactId}-${project.version}.${project.packaging}
                    -Dosb.project.customizationFile=${project.basedir}/${project.artifactId}/${project.artifactId}-customizationFile.xml
                    weblogic.WLST ${project.build.directory}/osbImport.py</commandlineArgs>
                
            </configuration>
            <goals>
                <goal>exec</goal>
            </goals>
        </execution>
    </executions>
</plugin>