Export Audit Trail from Database SOA 10g and 11g

For a while I thought that was impossible to query the SOA composite instance audit trail directly from database. Several references on internet say that only SOA Management APIs can understand the format in which this information is stored into SOA Dehydration Store. Based on this, I posted about a java tool which does the audit export based on these APIs: https://gibaholms.wordpress.com/2013/07/18/soa-11g-audit-trail-exporter-tool/

The “SOA 11g Audit Trail Exporter Tool” still very useful due to simple features like the easy-of-use, remote export, no need database access to export (only a valid EM user) and file output. However, I finally found a way to query the audit trail XML directly from Database. The audit trail extraction is very fast and simple, and can be done by anyone with access to the SOAINFRA tables.

Edit 03-12-2015: New package version 1.1 available, with a bug fix to work properly on SOA 10g (thanks FRIEDRICH OBER-HONGSERMEIER for the contribution).

To perform this extraction directly from database, I wrote a simple PL-SQL function that does the Audit Trail tricks. I encapsulated it into a package called SOA_UTIL which can be compiled into a controlled area with some governance:


CREATE OR REPLACE PACKAGE SOA_UTIL AS
  
   -------------------------------------------------------------------------------------------
   -- Written by   : Gilberto Holms (https://gibaholms.wordpress.com/)
   -- Last update  : 07/11/2014
   -- Version      : 1.1
   -- Description  : Utility code to handle Oracle SOA Suite tables
   -- Release notes:
   -- 	1.0: Initial release
   --	1.1: Bug fix to work properly on SOA 10g
   -------------------------------------------------------------------------------------------
  
  FUNCTION GET_AUDIT_TRAIL(P_SCHEMA_NAME VARCHAR2, P_CIKEY NUMBER) RETURN CLOB;
  
END SOA_UTIL;

/

CREATE OR REPLACE PACKAGE BODY SOA_UTIL AS
  
   -------------------------------------------------------------------------------------------
   -- Written by   : Gilberto Holms (https://gibaholms.wordpress.com/)
   -- Last update  : 07/11/2014
   -- Version      : 1.1
   -- Description  : Utility code to handle Oracle SOA Suite tables
   -- Release notes:
   -- 	1.0: Initial release
   --	1.1: Bug fix to work properly on SOA 10g
   -------------------------------------------------------------------------------------------
  
  FUNCTION GET_AUDIT_TRAIL(P_SCHEMA_NAME VARCHAR2, P_CIKEY NUMBER) RETURN CLOB
  AS
    V_AUDIT_BLOB BLOB;
    V_AUDIT_CLOB CLOB;

	V_CUR_AUDIT SYS_REFCURSOR;

	TYPE TP_AUDIT_RECORD IS RECORD(
      BLOCK NUMBER(38,0),
      LOG BLOB
	);
    TYPE TP_AUDIT_ARRAY IS TABLE OF TP_AUDIT_RECORD;
    V_AUDIT_ARRAY TP_AUDIT_ARRAY;
    V_AUDIT_COMPLETE BLOB;

	V_BUFFER_LENGTH PLS_INTEGER := 32767;
    V_BUFFER VARCHAR2(32767);
    V_READ_START PLS_INTEGER := 1;
BEGIN

	DBMS_LOB.CREATETEMPORARY(V_AUDIT_BLOB, TRUE);
	DBMS_LOB.CREATETEMPORARY(V_AUDIT_CLOB, TRUE);
	DBMS_LOB.CREATETEMPORARY(V_AUDIT_COMPLETE, TRUE);
	
	OPEN V_CUR_AUDIT FOR 'SELECT BLOCK, LOG FROM ' || P_SCHEMA_NAME || '.AUDIT_TRAIL WHERE CIKEY = :cikey ORDER BY COUNT_ID' USING P_CIKEY;
	FETCH V_CUR_AUDIT BULK COLLECT INTO V_AUDIT_ARRAY;
	CLOSE V_CUR_AUDIT;

	FOR j IN 1..V_AUDIT_ARRAY.COUNT LOOP
		IF j = 1 THEN
			DBMS_LOB.APPEND (V_AUDIT_BLOB, V_AUDIT_ARRAY(j).LOG);
		ELSE
			IF ( V_AUDIT_ARRAY(j).BLOCK = V_AUDIT_ARRAY(j-1).BLOCK ) THEN
				DBMS_LOB.APPEND (V_AUDIT_BLOB, V_AUDIT_ARRAY(j).LOG);
			ELSE
				DBMS_LOB.APPEND (V_AUDIT_COMPLETE, UTL_COMPRESS.LZ_UNCOMPRESS(V_AUDIT_BLOB));
				DBMS_LOB.CREATETEMPORARY(V_AUDIT_BLOB, TRUE);
				DBMS_LOB.APPEND (V_AUDIT_BLOB, V_AUDIT_ARRAY(j).LOG);
			END IF;
		END IF;
	END LOOP;

	DBMS_LOB.APPEND (V_AUDIT_COMPLETE, UTL_COMPRESS.LZ_UNCOMPRESS(V_AUDIT_BLOB));
	V_AUDIT_ARRAY.DELETE;

	FOR i IN 1..CEIL(DBMS_LOB.GETLENGTH(V_AUDIT_COMPLETE) / V_BUFFER_LENGTH) LOOP
		V_BUFFER := UTL_RAW.CAST_TO_VARCHAR2(DBMS_LOB.SUBSTR(V_AUDIT_COMPLETE, V_BUFFER_LENGTH, V_READ_START));
		DBMS_LOB.WRITEAPPEND(V_AUDIT_CLOB, LENGTH(V_BUFFER), V_BUFFER);
		V_READ_START := V_READ_START + V_BUFFER_LENGTH;
	END LOOP;
	
	RETURN V_AUDIT_CLOB;
	
END;
  
END SOA_UTIL;

/

Then you can use this function inside a main query which filters the desired instances. This function works for both SOA 10g and 11g.

Usage example for Oracle SOA Suite 11g:


SELECT 
  CI.COMPOSITE_NAME AS COMPOSITE_NAME,
  CI.CMPST_ID AS INSTANCE_ID,
  CI.CREATION_DATE AS CREATION_DATE,
  CI.STATUS AS STEP,
  SOA_UTIL.GET_AUDIT_TRAIL('FMW11116_SOAINFRA', CI.CIKEY) AS AUDIT_TRAIL_XML
FROM
  SOAINFRA.CUBE_INSTANCE CI
WHERE 
  CI.COMPOSITE_NAME = 'MyComposite'
  AND CI.CREATION_DATE BETWEEN (sysdate - 1) AND (sysdate)
  ORDER BY CI.CREATION_DATE DESC;

Usage example for Oracle SOA Suite 10g:


SELECT 
  CI.PROCESS_ID AS PROCESS_ID,
  CI.CIKEY AS INSTANCE_ID,
  CI.CREATION_DATE AS CREATION_DATE,
  CI.STATUS AS STEP,
  SOA_UTIL.GET_AUDIT_TRAIL('ORABPEL', CI.CIKEY) AS AUDIT_TRAIL_XML
FROM
  ORABPEL.CUBE_INSTANCE CI
WHERE 
  CI.PROCESS_ID = 'MyBPEL'
  AND CI.CREATION_DATE BETWEEN (sysdate - 1) AND (sysdate)
  ORDER BY CI.CREATION_DATE DESC;

Event Details

Edit 03-12-2015: Added Event Details topic.

The audit trail is a large XML containing an array of events. The event XML usually contains all the message and details, like show below:


<event ...>
	<message>...</message>
	<details><![CDATA[ ... ]]></details>
</event>

However, for some very large XML payload, these event details are stored outside this structure, causing the event XML looks like this:


<event ...>
	<message>...</message>
	<details id="0" />
</event>

As you can see above, there is no payload inside the details tag, but there is a detail “id”.

So, if you are parsing the audit trail and you are interested on the details payload, every time you face this situation for some specific instance, you must make a separate call to a table named “AUDIT_DETAILS”, like shown below:


SELECT UTL_COMPRESS.LZ_UNCOMPRESS(BIN) AS AUDIT_DETAILS_XML 
FROM SOAINFRA.AUDIT_DETAILS 
WHERE CIKEY = '162524' -- put the cikey here 
AND DETAIL_ID = 0; -- put the details id here 

I used this technique in the company I work for, and by using the concepts shown in this article, I was able to do a full extract and parsing from SOA 10g and 11g databases getting all the information I needed to my use case. I hope you find it useful too.

Advertisements

OSB Throw Exception in XQuery

In this post I will show how to throw exceptions from an Oracle Service Bus 11g XQuery and then handle it into the pipeline flow. This technique is very useful to identify business exceptions into transformations and do business validations against the message payload.

The function that throws the exception is very simple, and can be called from any part of an XQuery code:

error(xs:QName('MyBusinessFault'), 'My message for the fault')

To handle the exception in the pipeline flow, you must add a “Service Error Handler” and then add a “Stage” with the following pattern:

OSB Flow Pattern to handle the exception thrown from XQuery

1. Checks if the exception handled was thrown from a XQuery error function:

$fault/ctx:errorCode = 'BEA-382513'

2. Identify the specific business exception by its name:

contains(fn-bea:serialize($fault/ctx:reason), 'MyBusinessFault')

3. Create a SOAP Fault Body and assign to the “body” variable. Here comes a little trick to extract the message from the exception, you must do exactly as shown:

<soap-env:Body>
  <soap-env:Fault>
	 <faultcode>Server</faultcode>
	 <faultstring>{substring-after($fault/ctx:reason/text(), 'MyBusinessFault: ')}</faultstring>
	 <detail/>
  </soap-env:Fault>
</soap-env:Body>

4. Reply with failure.

The use case presented here is only an example, but you can adapt the code to do whatever you want.

SOA 11g Audit Trail Exporter Tool

This project is a simple audit trail exporter tool, made for Oracle SOA Suite 11g (tested on version 11.1.1.6). It works for both BPEL and BPM based SOA composites.

Edit 03-13-2015: Text reviewed.

The SOA “Dehydration Store” stores all instance info and audit trail on some SOAINFRA tables (like AUDIT_TRAIL, AUDIT_DETAILS, XML_DOCUMENT and so on). If you have read privileges to these tables, you can do the export directly from database, as I show in this article: Export Audit Trail from Database SOA 10g and 11g

However, if you don’t have direct access to database or want a simple and controlled way to get the audit information for a specific set of instances, you can do the export using the tool presented here.

In a production environment, as the number of instances increases, the Enterprise Manager interface stuck many times, and can be very slow to access composite instance audit trail. Furthermore, is impossible to find a single instance based on some request payload info.

To get programmatic access to SOA audit information and do much more cool things, Oracle provides the Oracle SOA Suite Infrastructure Management Java API:

http://docs.oracle.com/cd/E28280_01/admin.1111/e10226/soaadmin_apimanage.htm

This project embeds this API, and provides a command-line interface to export instances audit trail, which can be filtered by a date range or the proper instance id. The result is the full audit trail, in form of human readable XML, for every single auditable component of the instances that matches the search filters. Then you can use system tools like “grep” to find a specific instance based on payload content, and see its full execution audit trail.

You can download the SOA Audit Exporter Tool on the link below:

SOAAuditTrailExporter.zip

Usage example:


Full Export:

  java -jar SOAAuditTrailExporter.jar -host=10.2.2.35 -port=8000 -username=weblogic -password=welcome01 -outBaseFolder=/temp/SOAAuditTrailExporter -compositePartition=default -compositeName=MySOAComposite -compositeRevision=1.0

Export By Date:

  java -jar SOAAuditTrailExporter.jar -host=10.2.2.35 -port=8000 -username=weblogic -password=welcome01 -outBaseFolder=/temp/SOAAuditTrailExporter -compositePartition=default -compositeName=MySOAComposite -compositeRevision=1.0 -minCreationDate=07/01/2013 -maxCreationDate=07/03/2013

Export By Instance ID:

  java -jar SOAAuditTrailExporter.jar -host=10.2.2.35 -port=8000 -username=weblogic -password=welcome01 -outBaseFolder=/temp/SOAAuditTrailExporter -compositePartition=default -compositeName=MySOAComposite -compositeRevision=1.0 -instanceId=20158

Enjoy!

XQuery TitleCase Function in Oracle Service Bus 11g

One of my clients made a requirement of transforming some customer data, returned by the legacy systems wrapped by OSB services, to the “Title Case” format, also known as “Pascal Case” and “Upper Camel Case”. That means that the text must be tokenized at the blank spaces and the first letter of every word must be in capitalized.

For example:

Input: hello world

Output: Hello World

The easiest and more convenient way to implement this requirement is by creating a function in pure Java and generating a Custom XPath or making a Java Call in OSB.

To my client, I made a pure Java Custom XPath, delivered with all the JUnit tests to guarantee the maintenance and evolution of the function. But, in parallel, I spent a little time making the “Title Case” function using only XQuery, just to exercise the language, which can be very complex at a first look.

Saying again, the only purpose of this article is to show the power of XQuery and demonstrate the kind of things that can be done using this powerful language, which supports variable creation, decision and flow control structures, like any other programming language.

Title Case XQuery test project structure:

Below is the “xq/TitleCase.xq” file, implemented only using XQuery:

(:: pragma type="xs:string" ::)

declare namespace xf = "http://tempuri.org/xq/TitleCase/";

declare function xf:TitleCase($str as xs:string) as xs:string {
	let $words := tokenize($str, '\s')
	return 
		let $result := 
			for $word in $words
			return concat(upper-case(substring($word, 1, 1)), substring($word, 2))
		return string-join($result, ' ')
};

declare variable $str as xs:string external;

xf:TitleCase($str)

To test the function, just create the XQuery above in the sbconsole and launch the test console:

The “Title Case” XQuery has only one string parameter, so input some text and execute it:

Then you must see the text “Title Cased”, like shown below:

To functions that will be widely and heavily used, making them using Java is the best approach of course. However, to do simplest and smaller things, or to avoid an xpath deploy and a server restart, you always can use the good XQuery language.

Attachments

Source Code:
https://github.com/gibaholms/articles/tree/master/XQuery_TitleCase_Function_in_OSB

Read XML Resource in Oracle Service Bus 11g

Some time ago I was thinking in how to read XML Resources available in OSB projects at runtime. It would be very useful to hold some configuration data to be read by the proxy services through XPath or Java Call. After many searches in Google, I did not find any documentation or example in how to read OSB resources content programmatically at runtime. The popular OSB APIs and MBeans do not provide any method to get the resources content, but only their references. The question was how to get the resources content through their references.

With the help of JShrink I spent some time decompiling the OSB native jar functions and making a reverse engineering to discover how to do this. Fortunately, after some good tries I found what I was looking for. Using some internal OSB built-in libraries we can access any project resource at runtime.

In this article I will show how to read a sample XML Resource programmatically at runtime. In this sample I will use a Custom XPath to read a XML resource that holds application configuration parameters. Obs.: the same API shown here can also be used into a Java Call.

The sample project structure will look like this:

File “xsd/Parameters.xsd”: a simple schema that we’ll use in this scenario to describe the XML structure that holds the parameters:

<xsd:complexType name="Parameters">
	<xsd:sequence>
		<xsd:element name="ParameterList" type="tns:ParameterList"/>
	</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="ParameterList">
	<xsd:sequence>
		<xsd:element name="Parameter" type="tns:Parameter" minOccurs="0" maxOccurs="unbounded"/>
	</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="Parameter">
	<xsd:sequence>
		<xsd:element name="Key" type="xsd:string"/>
		<xsd:element name="Value" type="xsd:string"/>
	</xsd:sequence>
</xsd:complexType>

File “xml/Parameters.xml”: the sample XML that will be read by our XPath function at runtime:

<ns0:Parameters xmlns:ns0="https://gibaholms.wordpress.com/samples/xsd/2012/02/parameters">
	<ns0:ParameterList>
		<ns0:Parameter>
			<ns0:Key>Color</ns0:Key>
			<ns0:Value>Blue</ns0:Value>
		</ns0:Parameter>
		<ns0:Parameter>
			<ns0:Key>Size</ns0:Key>
			<ns0:Value>500</ns0:Value>
		</ns0:Parameter>
	</ns0:ParameterList>
</ns0:Parameters>

File “xq/ReadXmlParameters.xq”: a sample test XQuery that calls our XPath function passing as argument the full reference path to the XML parameters file:

xquery version "1.0" encoding "Cp1252";
(:: pragma bea:schema-type-return type="ns0:Parameters" location="../xsd/Parameters.xsd" ::)

declare namespace xf = "http://tempuri.org/ReadXmlResource/xq/ReadXmlParameters/";
declare namespace ns0 = "https://gibaholms.wordpress.com/samples/xsd/2012/02/parameters";
declare namespace param = "https://gibaholms.wordpress.com/xpath/ReadXmlResource";

declare function xf:ReadXmlParameters() as element() {
    param:readXml("ReadXmlResource/xml/Parameters")
};

xf:ReadXmlParameters()

Now, let’s create the custom XPath that does the magic. Create a simple Java project and add the following compile dependencies:

  • <MIDDLEWARE_HOME>\Oracle_OSB1\modules\com.bea.common.configfwk_1.5.0.0.jar
  • <MIDDLEWARE_HOME>\Oracle_OSB1\modules\com.bea.core.xml.xmlbeans_2.1.0.0_2-5-1.jar
  • <MIDDLEWARE_HOME>\Oracle_OSB1\lib\modules\com.bea.alsb.resources.core.jar
  • <MIDDLEWARE_HOME>\Oracle_OSB1\lib\modules\com.bea.alsb.resources.xml.jar

The XPath project structure is shown below:

In the file “osb-readxmlresourcefunction.xml” we describe the XPath functions contract. This file is a requirement of OSB and must be copied to the functions directory with the generated jar file:

<xpf:xpathFunctions xmlns:xpf="http://www.bea.com/wli/sb/xpath/config">
	<xpf:category id="Custom Functions">
		<xpf:function>
			<xpf:name>readXml</xpf:name>
			<xpf:comment>This function reads a XML Resource file from OSB</xpf:comment>
			<xpf:namespaceURI>https://gibaholms.wordpress.com/xpath/ReadXmlResource</xpf:namespaceURI>
			<xpf:className>com.wordpress.gibaholms.xpath.ReadXmlResource</xpf:className>
			<xpf:method>org.apache.xmlbeans.XmlObject readXml(java.lang.String)</xpf:method>
			<xpf:isDeterministic>true</xpf:isDeterministic>
			<xpf:scope>Pipeline</xpf:scope>
			<xpf:scope>SplitJoin</xpf:scope>
		</xpf:function>
	</xpf:category>
</xpf:xpathFunctions>

The secret to access OSB resources at runtime is the various “*Repository” classes. The code of our “readXml” XPath function is shown below:

package com.wordpress.gibaholms.xpath;

import org.apache.xmlbeans.XmlException;
import org.apache.xmlbeans.XmlObject;

import com.bea.wli.config.Ref;
import com.bea.wli.config.component.NotFoundException;
import com.bea.wli.sb.resources.config.XmlEntryDocument;
import com.bea.wli.sb.resources.xml.XmlRepository;

public class ReadXmlResource {

	public static XmlObject readXml(String xmlRefPath) {
		Ref ref = new com.bea.wli.config.Ref("XML", Ref.getNames(xmlRefPath));
		XmlObject xmlObject = null;
		try {
			XmlEntryDocument xmlEntryDocument = XmlRepository.get().getEntry(ref);
			String xmlContent = xmlEntryDocument.getXmlEntry().getXmlContent();
			xmlObject = XmlObject.Factory.parse(xmlContent);
		} catch (NotFoundException e) {
			e.printStackTrace();
			throw new RuntimeException("XML Resource not found.");
		} catch (XmlException e) {
			e.printStackTrace();
			throw new RuntimeException("Error parsing XML content.", e);
		}
		return xmlObject;
	}
	
}

As we can see in the code, the class “com.bea.wli.sb.resources.xml.XmlRepository” does the magic of retrieve the XML resource content through it “com.bea.wli.config.Ref” reference. Following the same principle, we can use the other various “*Repository” classes to access other types of resources in projects at runtime, like Schemas, XSLTs, WSDLs and so on.

To test the function, just run the XQuery “xq/ReadXmlParameters.xq” in the sbconsole:

The test XQuery has no parameters, so just execute it:

Then you must see the parameters XML loaded at runtime, like shown below:

I’m glad to share this discovery with other ALSB / OSB developers and I hope help you to do more useful things with this knowledge.

If someone already has another solution to read OSB project resources at runtime or already use this API for some other purpose, please share the experience with us through comments. Thanks.

Attachments

Source Code:
https://github.com/gibaholms/articles/tree/master/Read_XML_Resource_in_OSB

Advanced Validation in Oracle Service Bus 11g

When implementing Proxy Services in Oracle Service Bus (old BEA AquaLogic Service Bus), it’s important to think about validation of request data.

The OSB provides a Validate action that is a good mechanism to validate when the incoming request is conformant to the XML Schema defined in the service contract. However, using only this action is a very poor mechanism to effectively validate a request because of the following reasons:

  • The Validate action accepts validation only against XML Schema (xsd), that has a limited syntax and cannot support more complex validations or rules;
  • XML Schema cannot support business oriented validations, only structural validations (and very limited);
  • A good choice for better support in validation would be Schematron, but Oracle Service Bus 11g does not provide Schematron support yet (SOA Suite Mediator already supports, but OSB not);
  • If your company uses Canonical Model, is very common to use on all your xsd entities elements with minOccurs=”0” to get full reusability on the model, however is hard to validate when a specific service operation needs that an element be mandatory, and it weakens the service contract.

In this article I will show a good pattern that I use to do more advanced, business oriented validation in Oracle Service Bus using XQuery. Note that deep XQuery knowledge is not in the scope of this article, for more in-depth knowledge about XQuery you need to google (a good start can be found at http://www.w3schools.com/xquery/default.asp). I will predict that the reader has basic XPath concepts to find some nodes in a XML instance (more at http://www.w3schools.com/xpath/default.asp).

In this sample I will focus on the Proxy Service, so I will ignore any business-proxy-business transformations and the sample will show a simple Business Service virtualization through data by-pass, but implementing some complex validation on the request input that is the main objective of this article. You can download the example artifacts in the links bellow this article.

Let’s take a look into the sample request input message that we’ll use in this scenario:

<xsd:element name="OrderRequest">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Customer">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="FirstName" type="xsd:string" minOccurs="0" />
<xsd:element name="LastName" type="xsd:string" minOccurs="0" />
<xsd:element name="DocumentNumber" type="xsd:string" minOccurs="0" />
<xsd:element name="Email" type="xsd:string" minOccurs="0" />
<xsd:element name="Age" type="xsd:int" minOccurs="0"/>
<xsd:element name="Password" type="xsd:string" minOccurs="0" />
<xsd:element name="PasswordConfirmation" type="xsd:string" minOccurs="0" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="Product">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="ProductID" type="xsd:string" minOccurs="0" />
<xsd:element name="ProductRestriction" minOccurs="0">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:enumeration value="EVERYONE"/>
<xsd:enumeration value="OVER18"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="Quantity" type="xsd:int" minOccurs="0" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>

Observations:

  • Obviously this dummy contract is not real world and has no business relevance in an real order scenario;
  • The sample simulates the use of Canonical Model entities which sets minOccurs=”0” all over the place, but we know that some information is mandatory to our execute order operation.

The starting point of this sample is the structure shown below, that consists of:

  • One OSB Project named “AdvancedValidation” in the default configuration
  • The “OrderService-v1.wsdl” that represents the contract used in the sample
  • One Business Service named “OrderBusiness” created from the OrderService WSDL (I recommend you to generate a Mock Service in SoapUI tool to point this Business Service to and run your tests)
  • One Proxy Service named “OrderProxy” created from the OrderService WSDL, with a basic Message Flow that only routes the original request to the OrderBusiness in the execute operation.

By now the focus will be the Proxy Service message flow.

To start our robust validation let’s add the main pipeline pair and put the simple “Validate” action into the request branch, because the minimum expected is that the request message does comply to the XML Schema, and if don’t, the flow does not need to work anymore. In the XPath select the OrderRequest input element in the wizard, in variable body against its XML Schema definition present into the OrderService WSDL.

– XPath: ./ord:OrderRequest
– In Variable: body
– Against Resource: OrderService-v1.wsdl/OrderRequest
– Raise Error

Now comes the great part, lets prepare our advanced XQuery validation. Create a XSD named “validation.xsd” into xsd folder. This schema will hold the validation data though the flow. Use the following structure:

<xsd:complexType name="Validation">
<xsd:sequence>
<xsd:element name="Payload">
<xsd:complexType mixed="true">
<xsd:sequence>
<xsd:any/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="ValidationErrorList" type="tns:ValidationErrorList"/>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="ValidationErrorList">
<xsd:sequence>
<xsd:element name="ValidationError" type="tns:ValidationError" minOccurs="0" maxOccurs="unbounded"/>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="ValidationError">
<xsd:sequence>
<xsd:element name="code" type="xsd:int"/>
<xsd:element name="message" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>

The next step is create the XQuery transformation file to do the validation and fill the previous structure. The input of transformation is the OrderRequest schema and the output is the Validation schema.

– Source Types: OrderService-v1.wsdl/OrderRequest
– Target Types: validation.xsd/Validation

Now I suggest you to forget the visual editor. Drag-and-drop fields will not help you in more complex transformations (it’s not .NET rs) and will make a mess in the source code, so the better to do is reading some concepts of XQuery and then make your code by the hands. In this sample we’ll validate the following:
– The mandatory fields to the operation, that are: FirstName, DocumentNumber, Email, Age, ProductID and Quantity
– DocumentNumber: must contain only numbers
– Email: must comply to the regex (^[a-z0-9]+([\._-][a-z0-9]+)*@[a-z0-9_-]+(\.[a-z0-9]+){0,4}\.[a-z0-9]{1,4}$)
– Password and PasswordConfirmation: if filled, must be the same value
– ProductRestriction: if filled and value is “OVER18”, the customer must be 18+ years old
– Echo the full request message in the payload tag to use in the flow if necessary

(:: pragma bea:global-element-parameter parameter="$orderRequest" element="ns1:OrderRequest" location="../wsdl/OrderService-v1.wsdl" ::)
(:: pragma bea:schema-type-return type="ns0:Validation" location="../xsd/validation.xsd" ::)

declare namespace xf = "http://tempuri.org/AdvancedValidation/xq/RequestValidation/";
declare namespace ns1 = "https://gibaholms.wordpress.com/samples/wsdl/Order-v1.0";
declare namespace ns0 = "https://gibaholms.wordpress.com/samples/xsd/2011/11/validation";

declare function xf:RequestValidation($orderRequest as element(ns1:OrderRequest)) as element() {
<ns0:Validation>
<ns0:Payload>{$orderRequest/.}</ns0:Payload>
<ns0:ValidationErrorList>{
(: BEGIN - Required Field Validations 🙂
if (empty($orderRequest/ns1:Customer/ns1:FirstName/text())) then
<ns0:ValidationError>
<ns0:code>1</ns0:code>
<ns0:message>FirstName: Required Field</ns0:message>
</ns0:ValidationError>
else if (empty($orderRequest/ns1:Customer/ns1:DocumentNumber/text())) then
<ns0:ValidationError>
<ns0:code>2</ns0:code>
<ns0:message>DocumentNumber: Required Field</ns0:message>
</ns0:ValidationError>
else if (empty($orderRequest/ns1:Customer/ns1:Email/text())) then
<ns0:ValidationError>
<ns0:code>3</ns0:code>
<ns0:message>Email: Required Field</ns0:message>
</ns0:ValidationError>
else if (empty($orderRequest/ns1:Customer/ns1:Age/text())) then
<ns0:ValidationError>
<ns0:code>4</ns0:code>
<ns0:message>Age: Required Field</ns0:message>
</ns0:ValidationError>
else if (empty($orderRequest/ns1:Product/ns1:ProductID/text())) then
<ns0:ValidationError>
<ns0:code>5</ns0:code>
<ns0:message>ProductID: Required Field</ns0:message>
</ns0:ValidationError>
else if (empty($orderRequest/ns1:Product/ns1:Quantity/text())) then
<ns0:ValidationError>
<ns0:code>6</ns0:code>
<ns0:message>Quantity: Required Field</ns0:message>
</ns0:ValidationError>
(: END - Required Field Validations 🙂

else

(: BEGIN - Field Specific Validations 🙂
if (not(matches($orderRequest/ns1:Customer/ns1:DocumentNumber/text(), '^[0-9]+$'))) then
<ns0:ValidationError>
<ns0:code>7</ns0:code>
<ns0:message>DocumentNumber: Must Contain Only Numbers</ns0:message>
</ns0:ValidationError>
else if (not(matches($orderRequest/ns1:Customer/ns1:Email/text(), '^[a-z0-9]+([\._-][a-z0-9]+)*@[a-z0-9_-]+(\.[a-z0-9]+){0,4}\.[a-z0-9]{1,4}$'))) then
<ns0:ValidationError>
<ns0:code>8</ns0:code>
<ns0:message>Email: Not a Valid Email Format</ns0:message>
</ns0:ValidationError>
else if (not(empty($orderRequest/ns1:Customer/ns1:Password/text()))
and $orderRequest/ns1:Customer/ns1:Password/text() != $orderRequest/ns1:Customer/ns1:PasswordConfirmation/text()) then
<ns0:ValidationError>
<ns0:code>9</ns0:code>
<ns0:message>Password: Must Match PasswordConfirmation</ns0:message>
</ns0:ValidationError>
else if (not(empty($orderRequest/ns1:Product/ns1:ProductRestriction/text()))
and $orderRequest/ns1:Product/ns1:ProductRestriction/text() = 'OVER18'
and xs:int($orderRequest/ns1:Customer/ns1:Age/text()) < 18) then
<ns0:ValidationError>
<ns0:code>10</ns0:code>
<ns0:message>Customer Must Be OVER 18 Years</ns0:message>
</ns0:ValidationError>
(: END - Field Specific Validations 🙂

else ''
}</ns0:ValidationErrorList>
</ns0:Validation>
};

declare variable $orderRequest as element(ns1:OrderRequest) external;

xf:RequestValidation($orderRequest)

Now add an “Assign” activity to evaluate the validation XQuery against a newly created validation variable. We’ll use this variable in the error handler to gain access to the validation codes and messages.

– Expression: $body/ord:OrderRequest
– Variable: validation

We put now an “IF-ELSE” control to check if exists any validation errors. Don’t forget to add the validation xsd target namespace into “val” prefix.

– Add Namespace Definition: val – https://gibaholms.wordpress.com/samples/xsd/2011/11/validation
– Condition: count($validation/val:ValidationErrorList/val:ValidationError)

Then we raise a custom error with the code of our choice “BUS-1”. Note that this code will be used to capture the error in the handler, just remember that code.

– Code: BUS-1
– Message: Request Validation Error

Now the pipeline is finished. We now concentrate the fault treatment in a single error handler associated to the entire flow, adding a “IF-ELSE” action to act as a switch, verifying which of the code is present in the “fault” variable and manually throwing the typed faults declared in our service contract (OrderBusinessFault and OrderTechnicalFault). Obs.: the code “BEA-382505” refers to the internal OSB code that is used for “Validate” action errors.

– Condition: $fault/ctx:errorCode = ‘BEA-382505’


– Condition: $fault/ctx:errorCode = ‘BUS-1’

Now just add an Assign activity in the “if” branches that apply the typed soap faults. Just remember that the built-in $fault variable refers to the OSB infrastructure fault, and not to soap fault. So if you try to assign a soap fault directly to this variable, it just doesn’t work. To throw a soap fault you must assign the entire soap body. Don’t forget to add the “ord” prefix namespace pointing to “https://gibaholms.wordpress.com/samples/wsdl/Order-v1.0”. First let’s add the technical faults, that apply in case of schema validation failure (BEA-382505) and case else (any infrastructure error that we are not treating):

– Add Namespace Definition: ord – https://gibaholms.wordpress.com/samples/wsdl/Order-v1.0
– Variable: body
– Expression:

<soap-env:Body>
<soap-env:Fault>
<faultcode>soap-env:Server</faultcode>
<faultstring/>
<detail>
<ord:OrderTechnicalFault>
<ord:message>Schema Validation Failure</ord:message>
</ord:OrderTechnicalFault>
</detail>
</soap-env:Fault>
</soap-env:Body>

– Add Namespace Definition: ord – https://gibaholms.wordpress.com/samples/wsdl/Order-v1.0
– Variable: body
– Expression:

<soap-env:Body>
<soap-env:Fault>
<faultcode>soap-env:Server</faultcode>
<faultstring/>
<detail>
<ord:OrderTechnicalFault>
<ord:message>SOA Infrastructure Error</ord:message>
</ord:OrderTechnicalFault>
</detail>
</soap-env:Fault>
</soap-env:Body>

Now to add the treatment to our “BUS-1” fault, we can use the message applied by the validation XQuery (notice that any information that might be of interest can be propagated through the $validation variable). Don’t forget to declare the “val” namespace prefix to “https://gibaholms.wordpress.com/samples/xsd/2011/11/validation”:

– Add Namespace Definition: ord – https://gibaholms.wordpress.com/samples/wsdl/Order-v1.0
– Variable: body
– Expression:

<soap-env:Body>
<soap-env:Fault>
<faultcode>soap-env:Server</faultcode>
<faultstring/>
<detail>
<ord:OrderBusinessFault>
<ord:message>{$validation/val:ValidationErrorList/val:ValidationError[1]/val:message/text()}</ord:message>
</ord:OrderBusinessFault>
</detail>
</soap-env:Fault>
</soap-env:Body>

To finish our sample, just add the “Reply” activities to each “if” branch indicating that invocation returned “With Failure” (causes http 500 return code):

– Reply: With Failure

Well done! You can open the OSB console and do the tests in the Proxy Service. Input some data and observe the results. Don’t forget to start the Mock Service into SoapUI and point the Business Service endpoint to the mock address. An easy way to simulate a generic technical fault is stopping the mock service, crashing the business service like it was “unavailable”.Now I hope this article has helped you to doing more complex validations beyond the limits of xsd into Oracle Service Bus 11g.

Attachments

Source Code: https://github.com/gibaholms/articles/tree/master/Advanced_Validation_in_OSB