TeamCity, maven and encoding problems

I's been a while since my last post (haven't been doing smth interesting – just coding, building and surfing) so it is the high time to add smth. Too late, on the production environment, we have noticed that there is a problem with encoding in a name, but all the other text on an HTML page had no encoding problem. Strange, so we were looking for the source of this problem. My fellow noticed that the source of the names are included in a XML file under the /WEB-INF/conf/ of our war file. He noticed that the XML prolog has windows-1250 encoding set, but the content was UTF-8 encoded. Looking to the source XML file he noticed, that it is encoded correctly (windows-1250). So we identified the problem to be in a build system. We are using TeamCity with maven and the agent (building the war file) runs on a Linux system with UTF-8 file encoding. Before I continue, here comes the maven configuration part that was used to copy the war resources:

<plugin>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
        <warName>web_isc</warName>
        <webResources>
            <resource>
                <directory>etc</directory>
                <targetPath>WEB-INF</targetPath>
                <filtering>true</filtering>
                <includes>
                    <include>conf/*.*</include>
                </includes>
            </resource>
        </webResources>
    </configuration>
</plugin>

The problem is in the filtering feature set to true, because the file is loaded using the platform encoding, that is unfortunately UTF-8 for our Linux system, thus wrong. Though, I was looking for an option to specify encoding for the maven war plugin, but there is no such option available so far. The workaround for this missing feature is to include only specific files that need to be filtered and exclude that doesn't need to be. Luckily, there was no need to included the incorrectly converted file in the filtering feature, so we ended up with the following setting that works :)

<plugin>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
        <warName>web_isc</warName>
        <webResources>
            <resource>
                <directory>etc</directory>
                <targetPath>WEB-INF</targetPath>
                <filtering>true</filtering>
                <includes>
                    <include>conf/spring.xml</include>
                    <include>conf/webapp.properties</include>
                </includes>
            </resource>
            <resource>
                <directory>etc</directory>
                <targetPath>WEB-INF</targetPath>
                <filtering>false</filtering>
                <includes>
                    <include>conf/*.*</include>
                </includes>
                <excludes>
                    <exclude>conf/spring.xml</exclude>
                    <exclude>conf/webapp.properties</exclude>
                </excludes>
            </resource>
        </webResources>
    </configuration>
</plugin>

Happy mavenizing...

Web Service with Oracle SSO

Today, I was facing a new challenge: using Axis2 Web Service client, we need to access a User Web Service that uses Single sign-on (SSO) authentication. With a little help of google I finally found the solution. My first problem was SSL. The Oracle SSO service is (of course) using HTTPS protocol and the server's certificate was expired. An easy fix was to download the certificate (using my Mozilla Firefox and export it), create a new truststore and use it in my test case: Create a new truststore with the server's certificate imported:

keytool -import -alias sso.cez -file sso.cez.cert -keystore cez.truststore

Specify the truststore for JSSE and its password in JVM options:

-Djavax.net.ssl.trustStore=cez.truststore
-Djavax.net.ssl.trustStorePassword=changeit

My second problem was the Axis2 Web Service client and the Oracle SSO service. Even if I supplied correct credentials to Axis2 Service client stub for the SSO service, I got the following exception:

11.6.2009 14:21:27 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (org.apache.commons.httpclient.NoHttpResponseException) caught when processing request: The server portaltest.fg.cz failed to respond
11.6.2009 14:21:27 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
11.6.2009 14:21:27 org.apache.axis2.transport.http.HTTPSender sendViaPost
INFO: Unable to sendViaPost to url[http://portaltest.fg.cz:8080/cezdochtest/services/UserService]
org.apache.commons.httpclient.NoHttpResponseException: The server portaltest.fg.cz failed to respond
 at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1976)
 at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
 at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
 at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
 at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
 at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
 at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
 at org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:542)
 at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:189)
 at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
 at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:371)
 at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:209)
 at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:448)
 at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:401)
 at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:228)
 at org.apache.axis2.client.OperationClient.execute(OperationClient.java:163)
 at cz.elanor.egdoch.webservices.userservice.client.UserServiceStub.jds_prit_osoba(UserServiceStub.java:178)
 at cz.elanor.egdoch.webservices.userservice.client.UserService.getAttendanceStatus(UserService.java:48)
 at cz.elanor.egdoch.webservices.userservice.client.UserServiceTest.testIt(UserServiceTest.java:28)

After a while (looking at the stacktrace and using some HTTP monitors) I figured out that the Oracle SSO service (or something in the way) closes TCP connection forcibly when it detects a POST request with no SSO token header. So I tried to create an easy test to prove my thoughts. Using the HttpClient from apache with GET method and correct credentials I could finally get through to the Web Service:

public void testGet() {
    HttpClient client = new HttpClient();
    HttpMethod method = new GetMethod("http://portaltest.fg.cz:8080/cezdochtest/services/UserService");
    final AuthScope authScope = new AuthScope(AuthScope.ANY);
    client.getState().setCredentials(authScope, new UsernamePasswordCredentials("username", "password"));
    client.getParams().setParameter("http.protocol.allow-circular-redirects", Boolean.TRUE);
    try {
        int result = client.executeMethod(method);
        System.out.println(result);
        System.out.println(new String(method.getResponseBody()));
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        method.releaseConnection();
    }
}

The HTTP client communicates with the Web Service as follows:

• It issues a GET request, the SSO service detects no SSO token and redirects to Oracle SSO service (HTTP status 302).
• The client starts HTTPS communication (trusting the server's certificate) and forwards to the SSO, the server responds 401 Unauthorized with BASIC challenge (this challenge can be skiped when client uses preemptive authentication in the previous bullet).
• The client sends configured credentials, SSO server authenticates the client request and redirects (using HTTP status 302 again – that's why "http.protocol.allow-circular-redirects" must be allowed). To be honest, there is another redirect as part of the Oracel SSO authentication challenge, but it is not so important to our use case.
• Now, the client has an SSO token in its cookies (authenticated token) and is able to pass through the SSO service to the web service.

Super duper! So, the only thing that we need to do is to obtain the Oracle SSO token for the Axis2 Web Service client - but how? An easy answer is to do a separate HTTP GET request, obtain a token and use it in the Axis2 client's stub. Here comes the final code for my Axis2 Web Service client:

public class UserService {
    private static final Log log = LogFactory.getLog(UserService.class);

    // the service stub
    private UserServiceStub stub;
    private String username;
    private String password;

    public UserService(String endpointAddress) {
        try {
            stub = new UserServiceStub(endpointAddress);
            // reuse the client (due to SSO)
            stub._getServiceClient().getOptions().setProperty(HTTPConstants.REUSE_HTTP_CLIENT, "true");
        } catch (Exception e) {
            log.error("Unable to instantiate a user service due to the following error!", e);
        }
    }

    public UserServiceStub.PritOsobaWrapper getAttendanceStatus(String kpjm, String osc) {
        UserServiceStub.Jds_prit_osobaE osobaE = new UserServiceStub.Jds_prit_osobaE();
        UserServiceStub.Jds_prit_osoba param = new UserServiceStub.Jds_prit_osoba();
        param.setArg0(kpjm);
        param.setArg1(osc);
        osobaE.setJds_prit_osoba(param);
        for (; ;) {
            try {
                final UserServiceStub.Jds_prit_osobaResponseE response = stub.jds_prit_osoba(osobaE);
                return response.getJds_prit_osobaResponse().get_return();
            } catch (RemoteException e) {
                // try to re-authenticate (SSO token might have been expired or missing)
                try {
                    authenticate();
                } catch (AuthenticationException ae) {
                    log.error(e.getMessage(), e);
                    log.error("Unable to authenticate, the HTTP response status is: " + ae.getHttpStatus(), ae);
                    return null;
                }
            }
        }
    }

    protected void authenticate() throws AuthenticationException {
        // a TCP connection is closed (probably by SSO) when a POST with no SSO token is issued. That's why we use
        // an HTTP client with user name credentials supplited to issue a GET request. This request is accepted by the
        // SSO server and it issues an authentication challenge that is handled by the HTTP client
        HttpClient client = new HttpClient();
        HttpMethod method = new GetMethod(stub._getServiceClient().getOptions().getTo().getAddress());
        try {
            final AuthScope authScope = new AuthScope(AuthScope.ANY);
            client.getState().setCredentials(authScope, new UsernamePasswordCredentials(username, password));
            client.getParams().setParameter("http.protocol.allow-circular-redirects", Boolean.TRUE);
            int status = client.executeMethod(method);
            if (status < 200 || status >= 300) {
                throw new AuthenticationException("Unable to authenticate!", status);
            }
            List headers = new ArrayList();
            headers.add(method.getRequestHeader("cookie"));
            stub._getServiceClient().getOptions().setProperty(HTTPConstants.HTTP_HEADERS, headers);
        } catch (IOException e) {
            throw new AuthenticationException(e, -1);
        } finally {
            method.releaseConnection();
        }
    }

    public void setUsernameAndPassword(String username, String password) {
        if (username == null || password == null) {
            log.info("Unable to set username and password for \"osoba\" service, either or both are not specified.");
            return;
        }
        this.username = username;
        this.password = password;
        // try to authenticate
        try {
            authenticate();
        } catch (AuthenticationException ae) {
            log.error("Unable to authenticate, the HTTP response status is: " + ae.getHttpStatus(), ae);
        }
    }

    // use the authentication exception only for clarity
    private class AuthenticationException extends RuntimeException {
        private final int httpStatus;

        private AuthenticationException(String message, int status) {
            super(message);
            this.httpStatus = status;
        }

        private AuthenticationException(Throwable cause, int status) {
            super(cause);
            this.httpStatus = status;
        }

        public int getHttpStatus() {
            return httpStatus;
        }
    }

}

Hope this post is clear (or at least) helpful ;)

iBATIS cache is not flushed on transaction rollback

Today, I got several reports of failing test cases from our Teamcity. As usually, I run all the tests locally, but (with no surprise) all passed. So I looked in the build log to figure out what can be wrong and after an hour of digging in and out I found the problem.

In FG, we use AbstractTransactionalDataSourceSpringContextTests to speed-up database tests by running each test case in a separate transaction that is always rolled-back on a test-end. This feature is nice, it speeds database tests greatly and all the test methods are guaranteed to have test data in a consistent state. But when an iBATIS memory cache is in its way, it causes the following problem:

I have a test method that lists entities from database and it removes some of these entities (causing the cache to be cleared). So far, so good, but in the same test method I list entities again. This puts the result to the cache, the deleted entities are not in the result (which is correct for the test method). At the end of the test execution a rollback is issued by the spring. Now, the next test method starts and it also lists entities with the same statement. And here comes the problem - the iBATIS cache is simple (hopefully due to performance problems;) - it doesn't support cache flush on a transaction rollback. So the cache was not flushed, holding less entities than expected and the current test method failed on this assertion.

Because I'm lazy, I used the following workaround in my database abstract test case to force all the caches to be flushed on each test-end.

protected void onTearDownInTransaction() throws Exception {
    // force the iBATIS caches to be cleared (topicCache and postCache)
    this.forumStorage.deleteTopic(-1);
}

Well, it's time to pay more attention to this issue: iBATIS cache transaction rollback