Siebel Connection Pool in Oracle Service Bus 11g

When a middleware is used to make a lot of Web Service calls to Siebel CRM, the usual login mechanism is too slow and expensive. In the usual login, the Siebel application makes one login / logoff to every Siebel layer involved (application, database, etc) to every single call. When making a lot of concurrent calls, contention may occur in any of the underlying layers and some requests can be delayed or rejected, returning an error to the caller in some cases.

To ease this problem and improve performance, the most commonly solution is to use a Stateless Session Management of Authorization, which means that username and password are provided only in the first call, and subsequent calls are made using a SessionToken that is returned every time for the next call.

Obviously it would be amazing to do some kind of Connection Pool, or more specifically, a Token Pool/Cache to open some stateless sessions on demand, and reuse them using the next tokens.

I did a search in Google to find some references and cases of this approach, and I found some articles about using JAX-WS SOAP Handlers. But I was disappointed because I can’t use SOAP Handlers in my Middleware, and I can’t propagate this pool solution along all my applications making a Point-To-Point integration to Siebel.

The SOA architecture that we use here takes advantage of the Oracle Service Bus 11g to encapsulate all Siebel calls acting as a Proxy, doing input validation, security centralization and enforcement, endpoint management, policy attachment, statistics monitoring, parameterized data enrichment, service lifecycle management, and so on. Basically, all clients take access to Siebel through OSB:

In this article I will show a solution to make the Siebel Connection Pool using SessionToken at the OSB level, in a SOA approach. The best thing about this solution is the fact that none of the Siebel service clients are impacted by this change because the connection pool is centralized at the mediation layer, in this case, OSB, and no interfaces are changed.

The Connection Pool Classes

To control the pool, I made some simple Java classes, the first one is a simple DTO do pack the Siebel username and password into a single object to act as the key to store the tokens (implementing equals and hashcode default generated by Eclipse):

package com.wordpress.gibaholms;

public class SiebelCredential {

	private String username;
	private String password;

	public SiebelCredential(String username, String password) {
		this.username = username;
		this.password = password;
	}

	@Override
	public String toString() {
		return "SiebelCredential [username=" + username + ", password=" + password + "]";
	}

	@Override
	public int hashCode() {
		final int prime = 31;
		int result = 1;
		result = prime * result + ((password == null) ? 0 : password.hashCode());
		result = prime * result + ((username == null) ? 0 : username.hashCode());
		return result;
	}

	@Override
	public boolean equals(Object obj) {
		if (this == obj)
			return true;
		if (obj == null)
			return false;
		if (getClass() != obj.getClass())
			return false;
		SiebelCredential other = (SiebelCredential) obj;
		if (password == null) {
			if (other.password != null)
				return false;
		} else if (!password.equals(other.password))
			return false;
		if (username == null) {
			if (other.username != null)
				return false;
		} else if (!username.equals(other.username))
			return false;
		return true;
	}

	public String getUsername() {
		return username;
	}

	public String getPassword() {
		return password;
	}

}

I created another simple class to hold a Siebel token with his creation date. This date will be used to avoid returning certainly expired tokens:

package com.wordpress.gibaholms;

import java.util.Date;

public class SiebelToken {

	private String token;
	private Date creationDate;

	public SiebelToken(String token, Date creationDate) {
		this.token = token;
		this.creationDate = creationDate;
	}
	
	public SiebelToken(String token) {
		this(token, new Date());
	}
	
	@Override
	public String toString() {
		return "SiebelToken [token=" + token + ", creationDate=" + creationDate + "]";
	}

	public String getToken() {
		return token;
	}

	public Date getCreationDate() {
		return creationDate;
	}

}

The most important class to control the pool is very simple, just making a static cache of a Queue of tokens mapped by credential (username and password pair). I also added some counters to be retrieved by one statistics service if necessary. As you can see, the methods to get/add tokens are synchronized to support concurrent requests from OSB. Obviously some short locks may occur, but the performance loss is insignificant compared to the login/logoff Siebel problem, because the methods are very simple and cheap:

package com.wordpress.gibaholms;


import java.util.HashMap;
import java.util.LinkedList;
import java.util.Map;
import java.util.Queue;

public class TokenPool {
	
	private static final int SIEBEL_TOKEN_EXPIRATION_IN_MINUTES = 15;
	
	private static final Map<SiebelCredential, Queue<SiebelToken>> tokensByCredential;
	private static long addCount;
	private static long pollCount;
	private static long hitCount;
	private static long missCount;
	
	static {
		tokensByCredential = new HashMap<SiebelCredential, Queue<SiebelToken>>();
	}
	
	public synchronized static String pollToken(String username, String password) {
		pollCount++;
		SiebelCredential credential = new SiebelCredential(username, password);
		if (tokensByCredential.containsKey(credential)) {
			Queue<SiebelToken> tokens = tokensByCredential.get(credential);
			while (tokens.size() > 0) {
				SiebelToken token = tokens.poll();
				long diffInMinutes = (System.currentTimeMillis() - token.getCreationDate().getTime()) / (60 * 1000);
				if (diffInMinutes < SIEBEL_TOKEN_EXPIRATION_IN_MINUTES) {
					hitCount++;
					return token.getToken();
				}
			}
		}
		missCount++;
		return null;
	}
	
	public synchronized static void addToken(String username, String password, String token) {
		addCount++;
		SiebelCredential credential = new SiebelCredential(username, password);
		if (!tokensByCredential.containsKey(credential)) {
			tokensByCredential.put(credential, new LinkedList<SiebelToken>());
		}
		tokensByCredential.get(credential).add(new SiebelToken(token));
	}

	public static void eraseCounters() {
		addCount = 0;
		pollCount = 0;
		hitCount = 0;
		missCount = 0;
	}
	
	public static long getAddCount() {
		return addCount;
	}

	public static long getPollCount() {
		return pollCount;
	}

	public static long getHitCount() {
		return hitCount;
	}

	public static long getMissCount() {
		return missCount;
	}
	
}

Notice also that we have a constant SIEBEL_TOKEN_EXPIRATION_IN_MINUTES to give us a hint to indicate if the token would be expired on Siebel. The token expiration time on Siebel is a system parameter named SessionTokenTimeout that can be adjusted according to your needs. The default value indicated in Siebel documentation is 15 minutes. More information can be found on the official docs here: http://docs.oracle.com/cd/B40099_02/books/EAI2/EAI2_WebServices32.html#wp178856

Using this simple date control we avoid having old tokens into pool and minimize the chances of waste Siebel calls to discover that the token is expired to try again with full header unnecessarily, improving the pool efficiency. However some expired tokens may still pass because sometimes may occur a milliseconds window between the token creation date on Siebel and the token creation date into our pool, so the expired token error still need to be handled.

Now just assemble these classes into a JAR file and we are ready to define the best Message Flow to get/put tokens into this cache in the appropriate moments.

The Service Bus Project

Our OSB project looks like this:

  • SiebelSWEBusiness.biz
    Is a generic business service for all Siebel Web Services. All SWE services use the same endpoint and are only differed by WSDL, so we can use a single BS of type “Any SOAP Service – SOAP 1.1”.

  • siebel-connection-pool-1.0.jar
    Is a simple JAR file containing the classes shown above in this article. We will use the “Java Callout” activity to get/add tokens from the pool using the static methods of the following class: “com.wordpress.gibaholms.TokenPool”.
  • SiebelAccount.sa
    Is a Service Account of type “Static” that holds the Siebel username and password. The OSB clients authenticate themselves through the Weblogic security realm using WS-Security, which centralizes the access policies to corporative resources.

  • AssignSiebelHeaderFull.xq
    Is a reusable XQuery which mounts the full Siebel header, with username and password:

  • AssignSiebelHeaderToken.xq
    Is a reusable XQuery which mounts the Siebel header with SessionToken tag only:

  • SiebelStatelessSession.proxy
    Is the main Proxy Service which contains all the logic to handle the requests and manage the Siebel tokens. It would be a bad idea to replicate all this logic along all Siebel Proxy Services, so I decided to encapsulate this logic into a single PS of type “Any SOAP Service” and protocol “local”. Then any Siebel service can use the connection pool, just routing the flow to this PS:

    The message flow shown above is the secret for the solution. To understand more deeply I suggest download the code and inspect the boxes, but I will try to explain the flow in a more high level:

    Stage

    Description

    AssignSiebelAccount Get the username and password from the SiebelAccount.sa for use later when mounting the headers
    GetTokenFromPool Try to get a token from the pool and assign to a variable. If the pool is empty, the variable will be null
    AssignSiebelHeader If exists token, the token is used in the header, otherwise, the full header is mounted
    BackupRequestBody This stage saves the original request body into a variable for the case of getting token expired error on the first attempt, because, in this case, it will be used for a second attempt
    DefineErrorType According to the return of the first attempt, we need to take some actions. The OSB Error Handler is a very bad place to put complex logic because of the product architecture, then the route handler only detects the type of error occurred and let the flow resume to the response pipeline, which takes the correct actions.
    HandleSiebelReturn In this place the flow detects four possibilities:

    TokenExpired: makes a second attempt using the full header, with username and password

    ErrorWithToken: put the response token into the pool and return the error

    ErrorWithoutToken: just return the error

    Success: put the response token into the pool and return the response

    For security reasons, the response header is removed from the message in all cases.

    HandleSecondAttemptError The flow will fall into this block only if the second attempt results in error too, so it just verify if the error response have a token, which is added into the pool, otherwise, the error is returned to the caller.
  • MySiebelService.wsdl
    This WSDL represents our Siebel regular SWE Web Service, generated by Siebel Tools. No secrets here.
  • MySiebelService.proxy
    This Proxy Service represents the access point to our Siebel service, which will be called by end consumers. Note that is very simple to reuse the “SiebelStatelessSession.proxy” by just creating an ordinary Proxy Service from Siebel service WSDL and adding a route, replying the original fault:

Supported Scenarios

  • Success on the first call – OK;
  • Generic application error on the first call, without returning token – OK;
  • Generic application error on the first call, returning token – OK;
  • Token expired error in the first call, and success on the retry call – OK;
  • Token expired error in the first call, and generic application error on the retry call, without retuning token – OK;
  • Token expired error in the first call, and generic application error on the retry call, retuning token – OK.

Benefits of the Solution

  • No changes to the service interfaces;
  • No changes to the service consumers;
  • No code replication;
  • No infrastructure logic leak to the service consumers;
  • Centralized solution;
  • Easy to manage and maintain;
  • Pluggable solution, easy to enable/disable to any service;
  • Incredible performance gain to the entire corporation, because all Siebel services can be pool enabled as fast as add a route in OSB.

Now we can easy enable the Connection Pool to any Siebel SWE service into the SOA mediation layer, with no impacts on actual service consumers and “zero” code replication, in a corporative way.

Attachments

(Updated 08/02/2012 – 08:47)

Source Code:
https://github.com/gibaholms/articles/tree/master/Siebel_Connection_Pool_in_OSB

Advertisements

16 thoughts on “Siebel Connection Pool in Oracle Service Bus 11g

  1. Steffen 07/05/2012 / 9:07 AM

    Great work. I used the same approach for salesforce.com, but not so sophisticated.

  2. Bruce Daley 07/17/2012 / 8:15 PM

    This is great work Gilberto! Very helpful and useful.

  3. Jussi 02/06/2013 / 11:26 AM

    Hi!

    Thanks for this post. It seems very useful to us. can you tell us how your solution behaves when there is a OSB domain wiht multiple Managed servers? Do all nodes have their own connection/token pools?

    • gibaholms 02/06/2013 / 1:14 PM

      Hi Jussi,

      Using exactly the presented code, each node will have its own pool… but this pool class is only to prove the concept, a most robust implementation might use the Coherence distributed cache to store the tokens, making a coherence cluster between all managed servers… then all nodes will be able to share the tokens. The change is only in the java callout implementation, all OSB code remain the same. When I have free time, I will post this coherence approach… or if you do this, I will be glad if you share your code is this thread.

      Regards,
      Gilberto

  4. Bram Philo 03/21/2013 / 8:28 AM

    Hi,

    I’m try to implement this in my OSB project, but I’m getting security authentication error.

    Here are the error :
    Security exception while calling to java method “public static synchronized java.lang.String com.wordpress.gibaholms.TokenPool.pollToken(java.lang.String,java.lang.String)”. Service account: OSB Siebel Connection Pool/sa/SiebelAccount. com.bea.wli.sb.security.CredentialNotFoundException: Authentication failed.
    com.bea.wli.sb.security.CredentialNotFoundException: Authentication failed.
    at com.bea.wli.sb.svcacct.StaticServiceAccountRuntime._getSubject(StaticServiceAccountRuntime.java:71)
    at com.bea.wli.sb.svcacct.ServiceAccountRuntime.getSubject(ServiceAccountRuntime.java:61)
    at com.bea.wli.sb.svcacct.ServiceAccountRuntimeManagerImpl.getSubject(ServiceAccountRuntimeManagerImpl.java:286)
    at com.bea.wli.sb.pipeline.SubjectHelperImpl.getSubject(SubjectHelperImpl.java:56)

    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
    Caused by: javax.security.auth.login.FailedLoginException: [Security:090304]Authentication Failed: User SADMIN javax.security.auth.login.FailedLoginException: [Security:090302]Authentication Failed: User SADMIN denied
    at weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:261)
    at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
    at java.security.AccessController.doPrivileged(Native Method)

    at com.bea.wli.sb.security.authentication.BoundedTTLAuthenticationCache.authenticate(BoundedTTLAuthenticationCache.java:129)
    at com.bea.wli.sb.security.CachedWLSAuthenticationService.authenticate(CachedWLSAuthenticationService.java:72)
    at com.bea.wli.sb.svcacct.StaticServiceAccountRuntime._getSubject(StaticServiceAccountRuntime.java:69)
    … 169 more

    Can u help me know what wrong with this ?

    Thanks,

    Bram Philo

    • gibaholms 03/21/2013 / 9:36 AM

      Hi Bram,

      Seems that your Weblogic is trying to Authenticate the user “sadmin” against it security realm… check if your service account is “Static” and are not attached to any service.

      • Bram Philo 03/21/2013 / 12:43 PM

        Hi,

        Thanks, for your quick response.

        You r correct, I’m getting error due to of I’m attached the SA when I call the java callout which causing weblogic try to do authenticate the user.

        Thanks a lot.

        Bram

  5. Harish Pareek 06/27/2013 / 4:23 PM

    Proxy Service Testing – SCSiebelContactService Help

    Request Document

    1-9OQ-8588

    Response Document

    The invocation resulted in an error: .

    SOAP-ENV:Server

    There is no active Web Service with operation named ‘http://siebel.com/asi/:EAIQueryById_Input’.(SBL-EAI-04313)

    EAIObjMgr_enu_0034_35651597.log

    SBL-EAI-04313
    IDS_EAI_WS_OP_NOT_FOUND

    There is no active Web Service with operation named ‘http://siebel.com/asi/:EAIQueryById_Input’.(SBL-EAI-04313)

    Response Metadata

    text/xml; charset=utf-8

    1
    utf-8

    Invocation Trace

    (receiving request)

    Initial Message Context
    added $body

    1-9OQ-8588

    added $header

    added $inbound

    EAIQueryById

    /SiebelConnectionPool/proxy/SCSiebelContactService

    request-response
    best-effort

    text/xml; charset=utf-8

    “document/http://siebel.com/asi/:EAIQueryById”

    utf-8

    text/xml

    0

    added $messageID
    8084799192811168582–7cfe3cb2.13f676cf608.-7d56

    RouteTo_SiebelStatelessSession

    Routed Service
    Route to: “SiebelStatelessSession”
    $outbound:

    request-response
    best-effort

    false

    $body (request):

    1-9OQ-8588

    $header (request):

    $attachments (request):

    Message Context Changes
    added $outbound

    request-response
    best-effort

    utf-8

    1
    utf-8

    false

    changed $body

    SOAP-ENV:Server

    There is no active Web Service with operation named ‘http://siebel.com/asi/:EAIQueryById_Input’.(SBL-EAI-04313)

    EAIObjMgr_enu_0034_35651597.log

    SBL-EAI-04313
    IDS_EAI_WS_OP_NOT_FOUND

    There is no active Web Service with operation named ‘http://siebel.com/asi/:EAIQueryById_Input’.(SBL-EAI-04313)

    changed $attachments

    changed $inbound

    EAIQueryById

    /SiebelConnectionPool/proxy/SCSiebelContactService

    request-response
    best-effort

    text/xml; charset=utf-8

    “document/http://siebel.com/asi/:EAIQueryById”

    utf-8

    text/xml

    1

    changed $header

    Route Error Handler

    $fault:
    BEA-380000

    RouteTo_SiebelStatelessSession
    response-pipeline

    • gibaholms 06/28/2013 / 8:24 AM

      Hi,

      Seems that your Siebel EAI did not find the WebService “There is no active Web Service with operation named ‘http://siebel.com/asi/:EAIQueryById_Input’”

      Regards,
      Giba

      • Harish Pareek 06/28/2013 / 11:03 AM

        Hi,

        The Siebel webservice operation name is EAIQueryById. The Siebel webservice has multiple operations and when I invoke the EAIQueryById from the proxy service the SiebelSWEBusiness.biz automatically picks the operation. I do not see where can you map the operation.

        Thanks,

        Harish

  6. Anthony 01/08/2014 / 8:30 PM

    Very well-thought out and robust example, thanks! I had a question regarding trying to run parallel transactions:

    It seems Siebel cannot truly perform operations in parallel if only using one user ID. If I have two OSB threads call Siebel with the same SessionToken, they both pick up the same SessionToken from the java callout, and then the second transaction will be queued behind the first in Siebel, in order to use the same EAI handler. In the worst case, the second transaction arrives just after the first one completes, causing an invalid token error and a retry.

    From Siebel guidance, they recommend “a data structure of at least size 10” to handle up to 10 concurrent sessions.

    Does this mean that I must have at least 10 user IDs and therefore 10 Service Accounts, if I want to be able to process 10 transactions in parallel from OSB->Siebel?

  7. Ruban 09/01/2015 / 1:02 PM

    We are getting this error message:-

    Failed to invoke SblGetTokenConsumer.sblGetToken due to application error: SOAP11Fault [faultCode=SOAP-ENV:Server, faultString=Error Code: 7668282 Error Message: Error: Login is not allowed at certain times or from certain machines due to the security policy. Alternatively, your account has been disabled due to excessive failed login attempts. Please review your company’s security policy or contact your system administrator for assistance.(SBL-DAT-00570), faultActor=null, detail={http://schemas.xmlsoap.org/soap/envelope/}Fault]\n\tat com.ebay.soaframework.sif.impl.internal.service.BaseServiceProxy.wrapInvocationException(BaseServiceProxy.java:51)\n\tat

    “message”:”Error while getting request builder”,”errorName”:”INTERNAL_ERROR”,”cause”:”java.lang.NullPointerException\n\tat

    • gibaholms 09/01/2015 / 1:39 PM

      Hi Ruban,

      I think this error “SBL-DAT-00570” is caused by many login attempts using wrong credentials. Did you check if the username and password is being sent correctly ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s