Showing posts with label Java. Show all posts
Showing posts with label Java. Show all posts

Saturday, May 22, 2010

Gateway Software Symposium 2010 (NFJS St. Louis)

I'm attending (and speaking at) the NFJS conference here in St. Louis this weekend. The quality of the talks is excellent, and the speakers clearly know their stuff.

The two talks I'll be delivering on Sunday are:


I hope if you're attending the conference you come to the talks to learn and participate.

Some of my fellow OCI-ers are also presenting:


Sessions I've attended include:

All good stuff.

Wednesday, February 10, 2010

Guice 2.0 - tasty

I'm finally getting a chance to do some work with Guice 2.0. I don't know if I just couldn't wrap my head around Guice in the past and I've finally "gotten" it, or if Guice 2.0 provides a more approachable API. Either way, I'm finding it great to use.

I've been mildly whiny about Guice in the past, stating that it wasn't completely statically typed (which is true, since it's possible you'll ask for a resource at runtime that isn't available because it wasn't configured). But even with that small limitation, I'm finding Guice 2.0 to be far better than Spring for dependency injection. The code is much smaller, much more type-safe, not XML (a big plus), and much more strongly type-checked.

If you haven't checked out Guice, or if you tried Guice 1.0 but haven't tried out 2.0, you should give 2.0 a serious look.

Now I need to integrate Guice with Jersey (the JAX-RS reference implementation)...

Monday, February 08, 2010

JAX-WS tarpit

I'm currently doing some work with an open source framework (to be left unnamed) built upon JAX-WS (using SOAP, SAML, and HTTPS). I'm making headway working with it, but talk about a tar pit. You go into the code and it's almost impossible to get out. Every time you gain a bit of understanding, some other anti-pattern slaps you until you see starts and you're back to global searches with google and looking through the code to make further progress.

Violations of the DRY principle are rampant. The generated code has magic constants sprinkled liberally throughout. Generated classes have default constructors and public getters and setters for all fields despite the fact that some fields are actually required; no hashCode; no equals; no ability to determine if one of these data objects is valid or not. Turning the WSDL into code generates vast numbers of classes (reminding me of the terrible mapping from CORBA IDL to Java). The generated code has almost no helpful comments (despite the fact that generating at least some reasonable back-references in generated code is easy precisely because you are generating code). And we've only delved a bit into the SAML aspects of things; I expect that to be another can of worms entirely.

I've heard that the SOAP/WS-* specifications were co-opted by certain large companies and made so complex it's nearly impossible to work with them except with the very big IDEs-with-god-complexes those vendors sell. Based on these experiences and previous ones working with some SOA frameworks, I can believe it. If that isn't the reason for their incredible opaqueness and anti-patterns, I'm afraid to find out who thought all of this was a good idea.

So while I'm making progress and getting things done, the entire experience makes me want to go wash my hands and update my resume.

If you know of good ways to work with this stuff short of plunking down vast quantities of money for some IDE that will sort-of/mostly/kind-of hide all the complexity (until the moment things break and you really need to understand what is going on), I'd love to hear about them...

Tuesday, February 02, 2010

Java Annotations have Become Pixie Dust

I was giving a talk about RESTful services using JAX-RS and Jersey recently and was asked why I had used Mark Volkmann's WAX for generating HTML and XML. The person asking the question pointed out that Jersey has integration with JAXB.

There were two answers to that question. One answer is that I am leery of anything which automatically converts my Java objects into a serialized format (bitten by Java's object serialization in the past). Incompatible object changes can be difficult or impossible to reconcile in a backward-compatible manner.

But the main answer I gave got some chuckles and further questions. I explained I was trying to avoid too much "pixie dust". In the example code, I was already using the Java Persistence API (JPA) and JAX-RS and their associated annotations. If I had not been careful, there would have been annotations for Spring and JAXB as well. All of these annotations are small in the code but have very large effects. Those innocent looking annotations subject my poor unsuspecting code to some very big (and some would argue bad) frameworks. Understanding how these frameworks interact is not only hard, but those interactions change as the frameworks change (possibly resulting in the system breaking with no code changes).

I have real misgivings about the number of annotation-based technologies that should be applied to any one project. Each annotation you use represents some amount of code you don't have to write. And that is, of course, a good thing from a development perspective. But every annotation you use represents 'pixie dust', behavior which is hidden from you and generally performed at runtime. That means that when things go wrong, there isn't any code to look at. It also means that small differences between configurations can produce large changes in behavior. That's a very bad thing for testing, production deployment, production maintenance, etc.

I've been thinking about this issue for some time*, so I was pleasantly surprised to find Stephen Schmidt's post admonishing us to Be careful with magical code. His post is not specific to annotations (he calls out aspect-oriented programming, for example - I agree that AOP is another kind of pixie dust). And he points out some examples of the "pixie dust" phenomenon. While I don't agree with his 'magic containment' example, it's a good post. You should read it.

As a rule of thumb, I think two kinds of pixie dust is the maximum to sprinkle on a project. So think hard and choose wisely when picking which ones to use: the more kinds of pixie dust you sprinkle, the harder it will be for you and others to understand and troubleshoot things, now and especially in the future.



*Thanks to Mike Easter for planting the idea of talking about the state of annotations in Java

Tuesday, July 21, 2009

RESTful clients with the Jersey Client API

[Note: This is one in a series of posts about JAX-RS. You may wish to start at the beginning]


Jersey, the reference implementation of the JAX-RS specification, is a framework that makes it very easy to implement RESTful web services. But there is more to Jersey than just the server side. In this post I will spend a little time exploring the Jersey client library.


Depending upon how you design your RESTful service, you may or may not want to have separate URLs for separate representations of the same resource. This presents a problem when trying to test with a browser. There is no way to tell popular browsers that you want a text/html representation or an image/jpeg representation. The browser has a list of preferred media types, but none that I'm aware of allow you to customize this (either in general or for a particular request). Even more importantly, we need to be able to build solid unit tests for our services. The Jersey client framework provides a good solution to this problem. It is designed to allow developers of RESTful web services to write good unit tests, but is more general purpose than that. It can also be used to write RESTful client applications.


There is an excellent tutorial on the Jersey client API, which you should download and read if you plan to use it. But I will give you a taste of the API here.


The Jersey client API is very clean, and requires a minimum of fuss to use. As an example, let's write a unit test which tests our web service serving up information about fighter planes. First, we write the code to set up and tear down our service implementation. This code is the same as that in our Main class before, just split up between the setup and tear-down methods.

public class Test3b {

private SelectorThread selector;

@org.junit.Before
public void createService() throws IOException {
Map initParams = new HashMap();
initParams.put("com.sun.jersey.config.property.packages", "net.gilstraps.server");
selector = GrizzlyWebContainerFactory.create("http://localhost:9998/", initParams);
}

@org.junit.After
public void DestroyService() {
selector.stopEndpoint();
selector = null;
}

The only difference in this case is we don't read from standard in to shut down the service, since we always want to shut it down at the end of the unit test.


Now, we can test that the text/html representation returned by invoking the service is what we expect. First, we do a bit of hoop jumping to read in a copy of the HTML we expect to receive:


@org.junit.Test
public void testF22Html() {
try {
File expected = new File("f22.html");
long fileSize = expected.length();
if (fileSize > Integer.MAX_VALUE) {
throw new IllegalArgumentException("File it larger than a StringWriter can hold");
}
int size = (int) fileSize;
BufferedReader r = new BufferedReader(new FileReader(expected), size);
char[] chars = new char[size];
int readChars = r.read(chars);
if (readChars != size) {
throw new RuntimeException("Failed to read all chars of the expected result html file");
}
final String expectedText = new String(chars);

At this point, the variable expectedText contains what we should receive back from our request. Making the request is quite straightforward. First, we create a JAX-RS client:

Client client = new Client();

These clients are 'heavyweight' objects (taking time to create and using significant resources). As such, in a production client we would want to create this client once and use it many times. For the sake of independent unit tests, we will go ahead and create a Client object for each test.


Next, we specify the resource we want to retrieve using a WebResource object:

WebResource f22 = client.resource("http://localhost:9998/planes/f22/f22.html");

And then we ask the client to retrieve the resource for us, specifying that we want a text/html representation (MediaType.TEXT_HTML_TYPE) and specifying that we want to get back a ClientResponse object:


ClientResponse response =
f22.accept(MediaType.TEXT_HTML_TYPE).get(ClientResponse.class);

This code is using the builder pattern, where we build up our request through a chain of method calls. In this case, the chain is only two calls long. First, we call the accept method to specify the media types we will accept in the response (text/html), then we call the get method to actually retrieve the resource. It is possible to chain together more calls to specify other characteristics of either the request or the expected response (for more information, see the whitepaper on the Jersey client API).


Now that we have gotten a representation of the resource in the form of an HTTP response, we can then retrieve the HTML entity contained within that response as a string:


String returnedHTML = response.getEntity(String.class);

And finally, since this is a unit test, we make sure that what we got back matches what we expected:

assertEquals("Expected and actual HTML don't match"
expectedText, returnedHTML);

The ClientResponse object is useful if you want to look at other characteristics of the response, such as the returned headers. In this case, we could just as well have asked for the string from the WebResource directly. To do so, we would replace these two lines:

ClientResponse response =
f22.accept(MediaType.TEXT_HTML_TYPE).get(ClientResponse.class);
String returnedHTML = response.getEntity(String.class);

With this one:

String returnedHTML = asHTML.accept(MediaType.TEXT_HTML_TYPE).get(String.class);

Testing for retrieval of the JPEG representation of an image is almost identical. The only differences are that we read in the image file as an array of bytes, ask for the response entity as an array of bytes, and compare the two as arrays of bytes. Here is the entire test method:


@org.junit.Test
public void testF22JPEG() {
try {
// Read in our expected result
File imageFile = new File("f22.jpg");
long fileSize = imageFile.length();
if (fileSize > Integer.MAX_VALUE) {
throw new IllegalArgumentException("File it larger than a StringWriter can hold");
}
int size = (int) fileSize;
byte[] expectedBytes = new byte[size];
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(imageFile), size);
int bytesRead = bis.read(expectedBytes);
assertEquals(size, bytesRead);

// Request the representation
Client client = new Client();
WebResource f22 = client.resource("http://localhost:9998/planes/f22/f22.jpg");
ClientResponse response = f22.accept(MediaType.WILDCARD).get(ClientResponse.class);
MultivaluedMap headers = response.getHeaders();
for ( String key : headers.keySet() ) {
System.out.println( key + "=" + headers.get(key) );
}
byte[] returnedBytes = new byte[0];
returnedBytes = response.getEntity(returnedBytes.getClass());

// Compare the two sets of bytes to make sure they match
assertEquals(size, returnedBytes.length);
assertTrue(Arrays.equals(expectedBytes,returnedBytes));
}
catch (FileNotFoundException e) {
AssertionError ae = new AssertionError("File containing expected HTML not found!");
ae.initCause(e);
throw ae;
}
catch (IOException e) {
AssertionError ae = new AssertionError("Problems reading expected text!");
ae.initCause(e);
throw ae;
}
}

Just like the core of Jersey, the client library enables clear, short code that clearly expresses what the developer is trying to do. This is a marked departure from many other web frameworks and APIs. It is refreshing to be able to write web services clients in such a terse, yet clear style.

[Note: I plan to continue working with Jersey and continue posting about it. But I'm going to take a few weeks to look into some other topics first. I'll make sure to update the list of JAX-RS and Jersey posts as I add more]

Monday, June 08, 2009

JAX-RS Hello World

[NOTE: All of my examples will use standalone services (not housed inside a container). The integration between Jersey and the Grizzly HTTP server makes this a snap]

So, what is the simplest, smallest hello world we can implement with JAX-RS? The Jersey site provides an example and it's surprisingly small. We will start with an example almost identical to the hello world example from the Jersey documentation and grow from there.

If we ignore the package statements and imports, there are two files of about ten lines each. The first one is our main class that starts up the service and generally gets things going. Later examples will show why this is it's own class separate from the rest of a service's implementation. Here's the class in its entirety:

package net.gilstraps.server;

import com.sun.grizzly.http.SelectorThread;
import com.sun.jersey.api.container.grizzly.GrizzlyWebContainerFactory;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

public class Main {

public static void main(String[] args) throws IOException {
Map<String,String> initParams = new HashMap<String, String>();
initParams.put( "com.sun.jersey.config.property.packages", "net.gilstraps.server" );
SelectorThread selector = GrizzlyWebContainerFactory.create( "http://localhost:9998/", initParams );
//noinspection ResultOfMethodCallIgnored
System.in.read();
threadSelector.stopEndpoint();
System.exit(0);
}

}

The code here is pretty easy to understand, so let's look at it. These two lines are a crucial part of configuring Jersey:

Map<String,String> initParams = new HashMap<String, String>();
initParams.put( "com.sun.jersey.config.property.packages", "net.gilstraps.server" );

With these lines, we tell Jersey which java packages should be examined at runtime to find classes which implement RESTful resources. We've specified that Jersey should look at the "net.gilstraps.server" package and examine the classes in that package to find ones which have JAX-RS annotations on them. Those annotations tell a JAX-RS implementation what resources each class serves up to clients and what URI's those resources have. There are also annotations for specifying which HTTP methods match up to which Java methods. We'll look at these shortly.

The next line actually instantiates the Grizzly HTTP server, telling it the base URI for all the URL's it should support and providing it with initialization parameters:

SelectorThread selector = GrizzlyWebContainerFactory.create( "http://localhost:9998/", initParams );

The call returns us a SelectorThread which represents the control object for the HTTP server. With the string "http://localhost:9998/", we are telling the service to use port 9998 of the host named 'localhost'. Grizzly has built-in knowledge of Jersey and our inclusion of the "com.sun.jersey.config.property.packages" parameter causes Jersey to used to provide the web service.

The remaining lines provide a simple means to shut down the service gracefully. The program reads input from System.in and then shuts things down cleanly. So if you run the program on a command line you can get it to shut down without killing it:

System.in.read();
threadSelector.stopEndpoint();
System.exit(0);

So, where is our cliched "hello world" resource? There is one more class used to implement our service, also quite small. Here it is:

package net.gilstraps.server;

import javax.ws.rs.Produces;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.swing.text.html.HTML;

@Path("/helloworld")
public class HelloWorldResource {
@GET @Produces("text/plain")
public String getClichedMessage() {
return "Hello Stage 1";
}
}

This class is almost a POJO (plain old Java object). It doesn't implement a particular interface, or extend a class. It has a very simple method ('getClichedMessage') that returns a regular Java string. How can this implement a hello world web service? The key to making it a JAX-RS resource class is the annotations. Let's examine each one:

@Path

The '@Path' annotation attached to the class itself tells a JAX-RS what URI or set of URIs this class serves. In this case, we are specifying just the URI of "/helloworld". Thus, with the base URI of "http://localhost:9998" we would access this resource via the URI "http://localhost:9998/helloworld".

@GET

The @GET annotation on a method tells JAX-RS that the Java method implements HTTP's GET verb (the verb your browser uses every time you visit a web page). So JAX-RS now knows that a client doing a GET of "http://localhost:9998/helloworld" should result in the 'getClichedMessage' method being invoked.

@Produces

The @Produces annotation tells JAX-RS that the MIME type of the resource returned by 'getClichedMessage' is "text/plan", which is the MIME type for a plain text document. We've aimed for a minimal service here, so we're going for the simplest type of resource we can. It's also possible to return other kinds of entities as the result of a request, and it's even possible to use separate conversion classes to convert different kinds of Java objects into particular kinds of response entities (like a JPEG image, for example). But this service just returns a text document.

At this point, JAX-RS knows everything it needs to know to support a client doing a GET on the resource identified by the URI "http://localhost:9998/helloworld". When we compile and run our service, it doesn't output anything at first. But if we open a browser and go to the URL, we see some output from Grizzly/Jersey:

Jun 12, 2009 9:02:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
net.gilstraps.server
Jun 12, 2009 9:02:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Root resource classes found:
class net.gilstraps.server.HelloWorldResource
Jun 12, 2009 9:02:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Provider classes found:

From this we can see that Grizzly brings Jersey in to handle the request we made with the browser. Jersey then scans for classes which serve up REST resources. It tells us that it found our class:

Jun 12, 2009 9:02:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Root resource classes found:
class net.gilstraps.server.HelloWorldResource

Even more interesting, the browser where we made the GET request gets the string that was returned from 'getClichedMessage' as the result of it's GET request:



And that's a simple 'hello world' implemented using JAX-RS with Jersey and Grizzly.

Tuesday, June 02, 2009

Setting up Jersey for use

If you use Maven you can use it to download Jersey and keep it up to date. There are links to the POM files on the getting started page. In particular, you need the jersey-server module, and the grizzly-servlet-webserver module. If you want to use WADL with JDK 1.5 you need the jaxb-impl module. If you are not using Maven, you need only a few JARs:

grizzly-servlet-webserver.jar
jersey-server.jar
jersey-core.jar
jsr311-api.jar
asm.jar

[NOTE: these projects are under active development, so you may find newer versions of these jars available]

As with using Jersey via Maven, if you want to use WADL with JDK 1.5, you need some additional jars:

jaxb-impl.jar
jaxb-api.jar
activation.jar
stax-api.jar

We will ignore WADL for now, which limits the required jars to just the first five. But in addition to the binaries, I prefer to have the sources and javadoc available, which makes the total number of jars 15 (a binary, source, and javadoc jar for each).

Once you have the JARs, configure your classpath or your IDE to include them in your project. You're then ready to write the obligatory 'hello world' program.

Wednesday, May 27, 2009

Exploring RESTful web services with JAX-RS and Jersey.

I've been reading about RESTful web services for a while now. The best book on the subject I've found is RESTful Web Services by Leonard Richardson and Sam Ruby. I've been reading this book for what seems like forever (a function of my life and available time, not the book). I'm getting close to done and have begun using Jersey, the reference implementation of the JAX-RS specification to experiment with building RESTful web services.

I really like Jersey. A big reason for that is because Jersey makes it straightforward to stand up a standalone RESTful service. And once you've done it once, standing up additional services is easy. Deploying services without a heavyweight container makes me happy. Containers were created in the early days of Java when Java's limitations made them necessary. Since then, they have taken on a life of their own. Yet, in today's world I think the container has outlived almost all of its purpose. It's disadvantages now outweigh the few remaining benefits.

In the coming months, I plan to explore how to build, deploy, replicate, secure, and evolve industrial-class services without a container. I think I will succeed, but who knows? Maybe I'll learn a bunch due to abject failures. Either way, it should be fun.

My next post will describe how you download all the various pieces needed to stand up a Jersey-based REST service. There aren't very many, and it doesn't take long. Stay tuned.

Thursday, September 11, 2008

Interface Chaining

I was recently talking with my friend Mark Volkmann about an application framework he was developing. The framework, named WAX, is a very nice Java API for writing out XML. Despite it's small size, it's much nicer than any other XML writing API I've ever seen and you should go check it out.

As part of the API, Mark implementing method chaining. This allows for very compact code that is still quite easy to read. For example, you could produce this XML:

<?xml version="1.0" encoding="UTF-8"?>
<car year="2008">
<model>Prius</model>
</car>

with just this code::

WAX wax = new WAX(WAX.Version.V1_0);
wax.start("car").attr("year", 2008)
.start("model" ).text( "Prius").close();

Much of the compactness comes from the fact that each method called returns the WAX object, allowing the developer to chain method calls together.

As we were discussing the code, the topic of incorrect ordering of calls came up. For example, what happens if you try to call the attr method to create an attribute after already creating some text inside the element with the text method?

Because WAX is a streaming API, you can't 'go back' and add an attribute in the start element tag after having already 'moved on' to putting text in the body of the element. For example, what if I tried to create an attribute to the model of the car after putting text in it, like this:

WAX wax = new WAX(WAX.Version.V1_0);
wax.start("car").attr("year", 2008)
.start("model" ).text( "Prius").attr( "trim", "GT" ).close();

WAX detects this situation and throws an exception at runtime, which it should do. But I'm a fan of static type checking. I love having tools do as much checking for me as possible. So I suggested to Mark that he create a set of interfaces. Each interface would contain only the methods that make sense at the current point in the output stream. This would allow the compiler to catch invalid method invocations. Even better, modern IDEs like IDEA would flag the method invocation visually even before compilation.

For example, if we arrange for the text method to return an interface which does not have an attr method, then the compiler will catch it and the IDE will flag the error visually. For example, in IDEA, the error is flagged like this:



Now we get the compactness of runtime exceptions (which Mark really likes) and static checking (which I really like). I did some looking around and couldn't find a design pattern that really fits. A number of people suggested that this was an example of a domain-specific language (DSL). But to my thinking, something that deserves the moniker of 'language' should be more involved than simply constructing some interfaces so that static checking is applied.

But whatever you call it, I think it's a useful design pattern than can be applied in other situations to make for clean, compact, yet statically checked APIs.

Thursday, April 24, 2008

Insidious Coupling

This will get to the topic of code coupling soon enough, but bear with me for a bit of background.

I like to use the 'fail fast' approach to my code. This means that invalid inputs cause a failure in the code as quickly as possible. Perhaps the simplest example of this in the JDK is the collections classes, whose iterators fail as soon as they detect that the underlying collection object has been modified. This allows them to avoid synchronization costs while also detecting unsafe usage.

In my own code, the most common situation where I find fast-fail code is in constructors. If an argument the constructor cannot be null (either because it will be used in the constructor, saved to a field, or both), I check that the object reference is not null. For example:

public class Foo {
// ...
public Foo( Bar b ) {
// Check that b is not null
// rest of constructor...
}
}

I started out performing this check with an if statement:

public class Foo {
// ...
public Foo( final Bar b ) {
if ( b == null ) {
throw new IllegalArgumentException( "Bar b cannot be null" );
}
// rest of constructor...
}
}

This gets us a pretty understandable stack trace:

Exception in thread "main" java.lang.IllegalArgumentException: Bar b cannot be null
at org.sandbox.Foo.(Foo.java:10)
at org.sandbox.SimpleDoesStuff.doIt(SimpleDoesStuff.java:11)
at org.sandbox.Baz.main(Baz.java:10)

But this Java code is pretty verbose, especially if you have multiple arguments to check. After that I tried creating a static method named assureNotNull which would perform the check:

public class Foo {
// ...
public Foo( final Bar b ) {
assureNotNull( b );
// rest of constructor...
}
//...
private static void assureNotNull( final Object o ) throws IllegalArgumentException {
if ( o == null ) {
throw new IllegalArgumentException( "Object cannot be null" );
}
}
}

This has the unfortunate consequence that it becomes harder to track down the nature of the problem.

Exception in thread "main" java.lang.IllegalArgumentException: Object cannot be null
at org.sandbox.Foo.assureNotNull(Foo.java:14)
at org.sandbox.Foo.(Foo.java:9)
at org.sandbox.SimpleDoesStuff.doIt(SimpleDoesStuff.java:11)
at org.sandbox.Baz.main(Baz.java:10)

The stack trace on the exception shows assureNotNull as the source of the exception, which while strictly true doesn't clearly show that the real problem was with someone passing a null Bar to the Foo constructor. This can be mitigated to some degree by taking an description as argument:

public class Foo {
// ...
public Foo( final Bar b ) {
assureNotNull( b, "Bar b cannot be null" );
// rest of constructor...
}
//...
private static void assureNotNull( final Object o, final String msg ) throws IllegalArgumentException {
if ( o == null ) {
throw new IllegalArgumentException( msg );
}
}
}

This provides a somewhat more helpful exception message but the method generating the exception is still 'off by one':

Exception in thread "main" java.lang.IllegalArgumentException: Bar b cannot be null
at org.sandbox.Foo.assureNotNull(Foo.java:14)
at org.sandbox.Foo.(Foo.java:9)
at org.sandbox.SimpleDoesStuff.doIt(SimpleDoesStuff.java:11)
at org.sandbox.Baz.main(Baz.java:10)

And even with this semi-fix to help find the cause of the problem, we're still left with recreating this method in every class. From here the logical next step is to centralize the method. We can do this with an Assertions class:

public class Assertions {
// ...
public static void assureNotNull( final Object o, final String msg ) throws IllegalArgumentException {
if ( o == null ) {
throw new IllegalArgumentException( msg );
}
}
// ...
}

Now, our Foo constructor becomes simpler:

import org.acme.Assertions;

public class Foo {
// ...
public Foo( final Bar b ) {
Assertions.assureNotNull( b, "Bar b cannot be null" );
// rest of constructor...
}
//...
}

And with Java 5's static imports, we can even simplify it further:

import static org.acme.Assertions.assureNotNull;

public class Foo {
// ...
public Foo( final Bar b ) {
assureNotNull( b, "Bar b cannot be null" );
// rest of constructor...
}
//...
}

But we still have the wrong method as the first listed in the stack trace:

Exception in thread "main" java.lang.IllegalArgumentException: Bar b cannot be null
at org.acme.Assertions.assureNotNull(Assertions.java:9)
at org.sandbox.Foo.(Foo.java:7)
at org.sandbox.SimpleDoesStuff.doIt(SimpleDoesStuff.java:11)
at org.sandbox.Baz.main(Baz.java:10)

Since all of us Java developers are big on frameworks, we might even take it to the 'next level'. Why not centralize the Assertions in some common framework? If the Assertions class were in a common framework, the thinking goes, then we'd all be able to use it everywhere. That consistency would really help us read unfamiliar code.

Ignoring the issue of getting any set of developers of size greater than one to agree on anything, especially a basic assertions framework used everywhere, we've just introduced insidious code coupling where it really doesn't belong. In our desire to achieve clarity, brevity, and avoid redundancy, we've coupled code together. If we have a common class in our framework, all the classes in the framework are now coupled to it and indirectly to each other. If we move the class to a generic framework, we externalize the coupling, making it even broader. This coupling introduces many hazards for versioning and incompatibility problems that come with the independent evolution of code.

So, what choice are we left with? Well, we could always depend upon the runtime itself to generate an exception for us. We can do this quite briefly, with less redundancy of code than the other approaches, and with zero coupling:

public class Foo {
// ...
public Foo(Bar b) {
b.getClass(); // Assure not null
}
}

This code is quite short, and generates a NullPointerException with the top element pointing to exactly the line of code where the problem occurred:

Exception in thread "main" java.lang.NullPointerException
at org.sandbox.Foo.(Foo.java:7)
at org.sandbox.SimpleDoesStuff.doIt(SimpleDoesStuff.java:11)
at org.sandbox.Baz.main(Baz.java:10)

This code also avoids nearly all redundancy, and I think after seeing it once it becomes quite clear. As a bonus, it's easy to create a macro in any modern IDE to help in generating the code.

Sometimes, the right solution to a problem is not abstracting code into a separate method/class/framework.

Wednesday, February 13, 2008

An alternate syntax for Java closures

I'm no expert on the details of the closures proposals. I do think if we're going to put closures into the language, we need to make them complete, full-fledged closures. Generics are a mess precisely because the semantics and features were compromised for the goal of getting it into the language.

But one thing I do know about the BGGA closures proposal is that I don't like the syntax. Take the example in Weiqi Gao's recent Java Quiz:

public class Fib {
private static {int=>int} fib = { int n =>
n==0 ? 0 :
n==1 ? 1 :
fib.invoke(n-1) + fib.invoke(n-2)
};

public static void main(String[] args) {
System.out.println(fib.invoke(6));
}
}

To me, this is just hard to read. The whole "=>" thing seems like a weak attempt to create a new lexical character and is too visually similar to ==, >>, etc. I also don't like the 'invoke' syntax. I was discussing this with Weiqi the other day and came up with a syntax that I like better.

public class Fib {
private static {int # int} fib = { # int n #
n==0 ? 0 :
n==1 ? 1 :
#fib(n-1) + #fib(n-2)
};

public static void main(String[] args) {
System.out.println(#fib(6));
}
}

Note that you could also invoke fib like this:

System.out.println(#Fib.fib(6));

I wonder what other people think of this alternative?

Sunday, November 04, 2007

Java 5+ Iterator Adapters

I've been working with Java Generics more lately, and have run smack into the "Wall of Erasure". I'm referring to the fact that generics disappear in the bytecode. As you probably know, generics were a targeted feature at a time when Sun was attempting to avoid changing the Java Virtual Machine (JVM). They really wanted to maintain backward compatibility. So, this compromise was put forth to maintain backward compatibility. In a classic case of irony, other changes forced a change the JVM in the same release that gave us generices, but the compromised version of generics stayed. So we are stuck with half an implementation of generics.

Recently, I've been doing work with interfaces and extending interfaces, as in:

public interface Superclass {
// ...
}

public interface Subclass extends Superclass {
// ...
}

I have classes which implement a third interface:

public interface DoesStuff {
Iterator<Superclass> iterator();
// ...
}


You would think that an implementation of DoesStuff could return an Iterator for the iterator method:

public class MyClass implements DoesStuff {
// ...
private Set<Subclass> myThings = new HashSet<Subclass>();
// ...
public Iterator<Superclass> iterator() {
return myThings.iterator();
}
// ...
}

Unfortunately, the Wall of Erasure means we can't do this. Try it for yourself if you don't believe me.


So, what to do? Happily, there is a simple answer. We create an adapter that adapts the subclass iterator to the superclass iterator:


/**
* Adapts a derived class iterator to a base class.
*/
public class IteratorAdapter<B, D extends B> implements Iterator<B> {
private Iterator<D> d;

public IteratorAdapter(Iterator<D> aD) {
d = aD;
}

public boolean hasNext() {
return d.hasNext();
}

public B next() {
return d.next();
}

public void remove() {
d.remove();
}
}

The sad thing is that this shouldn't be necessary. It's trivial. It shouldn't have to be written by developers. The simplicity of this is a clear sign that we need to tear down the Wall of Erasure.

Will it happen any time soon? I hope so. Though I'm hearing talk that "erasing erasure" is probably off the table for Java 7. That would be a truly sad thing.

Friday, October 26, 2007

Making BufferedReader Iterable

I was reading recently about a issues around reading an entire file into a string. I pretty much never do that, since I like my code to be robust even in the face of very large files. But it did get me thinking about reading in lines of a file one at a time. I do have reason to do that on occasion.

Originally, I might have done it something like this:

File file = ...
BufferedReader br = new BufferedReader(new FileReader(file));
String line = br.readLine();
while ( line != null ) {
// Do something with line
line = br.readLine();
}

That's not bad, but I've been working with Ruby lately and really like the terse yet easy-to-read nature of the code. We can improve the above code with a for loop:

File file = ...
BufferedReader br = new BufferedReader(new FileReader(file));
for (String line = br.readLine(); line != null; line = br.readLine()) {
// Do something with line
}

Better, but still not great. Now, what if the BufferedReader were Iterable? It isn't, but we can wrap an Iterable around it. First, let's look at usage:

File file = ...
BufferedReaderIterable bri = new BufferedReaderIterable(new BufferedReader(new FileReader(file)));
for (String read : bri) {
// Do something with the line
}

Very nice. So, all we need is to implement BufferedReaderIterable. Happily, it's quite simple:

public class BufferedReaderIterable implements Iterable<String> {

private Iterator<String> i;

public BufferedReaderIterable( BufferedReader br ) {
i = new BufferedReaderIterator( br );
}
public Iterator iterator() {
return i;
}

private class BufferedReaderIterator implements Iterator<String> {
private BufferedReader br;
private java.lang.String line;

public BufferedReaderIterator( BufferedReader aBR ) {
(br = aBR).getClass();
advance();
}

public boolean hasNext() {
return line != null;
}

public String next() {
String retval = line;
advance();
return retval;
}

public void remove() {
throw new UnsupportedOperationException("Remove not supported on BufferedReader iteration.");
}

private void advance() {
try {
line = br.readLine();
}
catch (IOException e) { /* TODO */}
}
}
}

It would be nice if we could make the construction cleaner. What if BufferedReaderIterable had a constructor that took a File?


public class BufferedReaderIterable implements Iterable<String> {

private BufferedReader mine;
private Iterator<String> i;

public BufferedReaderIterable( BufferedReader br ) {
i = new BufferedReaderIterator( br );
}

public BufferedReaderIterable( File f ) throws FileNotFoundException {
mine = new BufferedReader( new FileReader( f ) );
i = new BufferedReaderIterator( mine );
}
public Iterator<String> iterator() {
return i;
}

private class BufferedReaderIterator implements Iterator<String> {
private BufferedReader br;
private String line;

public BufferedReaderIterator( BufferedReader aBR ) {
(br = aBR).getClass();
advance();
}

public boolean hasNext() {
return line != null;
}

public String next() {
String retval = line;
advance();
return retval;
}

public void remove() {
throw new UnsupportedOperationException("Remove not supported on BufferedReader iteration.");
}

private void advance() {
try {
line = br.readLine();
}
catch (IOException e) { /* TODO */}
if ( line == null && mine != null ) {
try {
mine.close();
}
catch (IOException e) { /* Ignore - probably should log an error */ }
mine = null;
}
}
}
}

Now, usage is even cleaner:

File file = ...
BufferedReaderIterable bri = new BufferedReaderIterable(file);
for (String read : bri) {
// Do something with the line
}

[Editorial Note: Updated to put the template angle brackets back in after Eric Burke noticed Blogger was eating them]

Tuesday, July 31, 2007

RemoteIterator, episode 2

Last time, I lamented the lack of a standard RemoteIterator interface in Java. I went on to show a version that showed up not long ago in Jackrabbit (an implementation of JSR 170) and the version I've recreated several times in the past. Let's take a quick look at them again.

Jackrabbit's:

public interface RemoteIterator extends Remote {

long getSize() throws RemoteException;

void skip(long items) throws NoSuchElementException, RemoteException;

Object[] nextObjects() throws IllegalArgumentException, RemoteException;

}

The RemoteIterator interface I've recreated a number of times over the years:

public interface RemoteIterator<T> extends Remote {

public boolean hasMore() throws RemoteException;

public List<T> next( int preferredBatchSize ) throws RemoteException;

}

Let's go through the methods individually:

  • long getSize() throws RemoteException

    This seems problematic to me. Perhaps in Jackrabbit, it is reasonable to assume that you will always know the size of the collection to be iterated. But in many situations, this is not the case. For example, if the iterator is ultimately coming from a database cursor and the collection is large, we want to avoid having to run two SQL queries or store the entire collection in memory (which may be prohibitively expensive). I can see a derived interface, perhaps named BoundedRemoteIterator or something, but not in this interface.

  • void skip(long items) throws NoSuchElementException, RemoteException

    I love this idea. Wish I'd thought of it myself. A trivial implementation would do whatever next( items ), would do and simply not return the items. A smart implementation can save time and space by not actually generating the objects (since they won't be returned). The only change I would make is to have a return value of the number of items skipped and get rid of the NoSuchElementException. If I ask to skip 50 items and only 29 more exist, I would get a return of 29 and no exception. Admittedly, if the interface included a getSize operation, throwing NoSuchElementException is less strange. But even in that case, it seems too rigid to consider skipping more items than are left an 'exception'.

  • boolean hasMore() throws RemoteException

    At first, you might think this is a wasteful operation. Why make a call to find out if there are more items? This results in 'extra' remote calls instead of simply asking for the next item. However, if we imagine the items themselves as quite large (e.g. multi-megapixel images), we start to see a use for this API. If we were keeping the getSize operation, this might not be necessary, but again, getSize() requires state tracking on the client side that can substantially complicate the client code.

  • Object[] nextObjects() throws IllegalArgumentException, RemoteException

    This is similar to my next operation discussed below, except (1) it throws IllegalArgumentException, (2) returns an array of objects instead of a List of a specific type, and (3) does not take a preferred size.

    (1) I'm guessing that the getSize operation is intended to be called and then the caller is expected to know when all items in the iterator are consumed. This creates a problem if the client consumes different parts of the iterator in different parts of the code. Now, they have to pass around 2 or 3 items (remote iterator, number of items consumed so far, and possibly the total size) to do this. This is clunky to say the least.

    (2) Jackrabbit may have been specified before 1.5 was in wide use, or may feel the need for backward compatibility, hence the use of an array of Object. The downside to this is the loss of type-safety and loss of an API that can be optimized. See the discussion of next below.

    (3) See the discussion of next below for the rationale for a preferred batch size.

  • List<T> next( int preferredBatchSize ) throws RemoteException

    By returning a List<T>, we not only introduce type information/safety, we also allow for further optimization. The List returned could be a simple implementation such as LinkedList or ArrayList. Or, it could be a smart List that collaborates with the RemoteIterator to delay retrieving its contents or retrieve the contents in separate threads. There are lots of possibilities introduced by avoiding raw Java arrays.

    Allowing the client to pass a preferred batch size provides many optimization opportunities. For example, if displaying the items in a GUI, they can ask for 'page-sized' numbers of items. By making the preferred size a suggestion, it allows an implementation leeway to ignore requests that are deemed unmanageably small or large for the kind of information being returned.


So, where does this leave us? Our RemoteIterator now looks like this:

import java.rmi.Remote;
import java.rmi.RemoteException;
import java.util.List;

/**
* Interface for a remote iterator that returns entries of a particular type.
* @see java.util.Iterator
* @see java.rmi.Remote
*/
public interface RemoteIterator<T> extends Remote {

/**
* Determine if there are more items to be iterated across
* @return true if there are more items to be
* iterated, false otherwise.
* @throws RemoteException if a problem occurs with
* communication or on the remote server.
*/
boolean hasMore() throws RemoteException;

/**
* Skip some number of items.
* @param items the number of items to skip at maximum. If there are fewer
* items than this left, all remaining items are skipped.
* @return the number of itmes actually skipped. This will equal <code>items</code>
* unless there were not that many items left in the iteration.
* @throws RemoteException if a problem occurs with
* communication or on the remote server.
*/
int skip(long items) throws RemoteException;

/**
* Get some number of items
* @param preferredBatchSize a suggested number of items to return. The implementation
* is not required to honor this request if it will prove too difficult.
* @return a List of the next items in the iteration, in iteration order.
* @throws RemoteException if a problem occurs with
* communication or on the remote server.
*/
List<T> next( int preferredBatchSize ) throws RemoteException;

}


I'm still wondering why this isn't part of the JDK.

Thursday, July 19, 2007

RemoteIterator, where art thou?

I've written my fair share of remote iterators over the years. The concept is simple enough: provide an interface for iterating over a set of items that come from a remote source. The harder part is making a good abstraction that can still perform well.

Even so, it surprises me that, as far as I can tell, there hasn't been a standardized form of this. There's been discussion for a long time, with my earliest memories stemming from the JINI mailing lists back in 2001. Yet there's nothing in the core as of Java 6. There are random postings on various groups on the subject through the years, but no actual API that I can find.

Recently, I was teaching my course in Java performance tuning and discussing remote iterators, when a student mentioned that Jackrabbit (an implementation of JSR 170) has an RMI remoting API called Jackrabbit JCR-RMI which includes a RemoteIterator. As of version 1.0.1, it looks like this:

public interface RemoteIterator extends Remote {

long getSize() throws RemoteException;

void skip(long items) throws NoSuchElementException, RemoteException;

Object[] nextObjects() throws IllegalArgumentException, RemoteException;

}

Here is the RemoteIterator interface I've recreated a number of times over the years:

public interface RemoteIterator extends Remote {

public boolean hasMore() throws RemoteException;

public List next( int preferredBatchSize ) throws RemoteException;

}

These two APIs are strikingly similar. They both extend Remote, they both provide a means to get some number of the next items to be retrieved (mine with generics and theirs without), and they are both quite small. Yet they are also strikingly different: JackRabbit's API has both getSize and skip while mine doesn't.

Next time, I'll describe what I see as the major differences and what I like about each. I'll also compare the client-side front-end for remote iterators from Jackrabbit and the one I've created. In the mean time, consider it a challenge to comment about everything I was going to bring up and more.

Thursday, February 01, 2007

Adding a JNI Library template to Xcode

I've been doing some work with Xcode, Apple's IDE, recently. I wanted to use it for doing Java JNI work, since I'm doing some of that, but found the normal tutorial assumes you are building an entire application with Xcode. Well, Xcode pales in comparison to IDEA for working with Java.

So I wondered if I could use IDEA for working with the Java code and Xcode for managing the JNI bindings and the C++ code that goes with it. After a bit of playing around (primarily pulling bits and pieces from Apple's mini-tutorial on JNI with Xcode), I figured out how to build a JNI library that I can use easily from IDEA.

That got me to thinking about the four or five steps I would have to do each time I created a new one of these projects (which I expect to be doing for a while). I vaguely remembered a friend talking about customizing Xcode by creating new templates. So I set out to create one for building JNI libraries.

I googled around a bit and found out that Xcode keeps its templates in:

/Library/Application Support/Apple/Developer Tools

Looking there, I see a folder called "Project Templates", and sure enough inside are all the templates for Xcode projects.

There are several that look promising, but what's the difference between a "BSD Dynamic Library", a "C++ Dynamic Library", and a "C++ Standard Dynamic Library"? Something based on the "C++ Standard Dynamic Library" seems right.

So, I create a test project and make a few changes to produce a JNI library:
  • Change "Executable Extension" to be "jnilib"
  • Change "Header Search Paths" to include "$(SDK)/System/Library/Frameworks/JavaVM.framework/Headers"
  • Delete the Carbon framework (why was it there in the first place?)

Now, save the project, make sure it builds (which it does) and create a simple Java HelloWorld with a native method that calls our Test class.

Having verified that this sort of project can be used to produce a JNI Library, it's time to create the custom project template. First I copy the "C++ Standard Dynamic Library template" to "JNI C++ Standard Dynamic Library". Now I need to find the right places to insert the additional items that were changed (and how to remove the Carbon framework). Looking at the template directory, I see a number of files. But most of these are just boilerplate for source files in the project ('.pch', '.h', '.cp'). I think I can safely ignore these files at least for now (though I made note of them; if there prove to be certain common files in my JNI libraries moving forward, I can come back and try to tweak the project template to have different boilerplates).

The key 'file' appears to be CppShared.xcodeproj. Right-clicking on it shows it is really a 'package' and I take a look inside.



When I do so, I see two files: "TemplateInfo.plist" and "project.pbxproj". The '.plist' file has some interesting items about template files and lists of files upon which to perform macro expansion. Interestingly, the list of files matches the list of boilerplate files almost exactly. For now, I'm leaving these alone (again making note in case I want to change the set of boilerplate files in the project). There is also a Description item, which shows up in the "New Project..." window in Xcode. After tweaking that to make it better describe this new kind of project I'm creating, I move on to the "project.pbxproj" file.

The "project.pbxproj" file is semi-text and semi-binary. So it's time to be careful and make a backup before experimenting. The first section of interest is this:

/* Begin PBXFileReference section */
08FB77AAFE841565C02AAC07 /* Carbon.framework */ = {isa = ...
32BAE0B70371A74B00C91783 /* «PROJECTNAME»_Prefix.pch */ = {isa = ...
50149BD909E781A5002DEE6A /* «PROJECTNAME».h */ = {isa = ...
5073E0C409E734A800EC74B6 /* «PROJECTNAME».cp */ = {isa = ...
5073E0C609E734A800EC74B6 /* «PROJECTNAME»Proj.xcconfig */ = {isa = ...
5073E0C709E734A800EC74B6 /* «PROJECTNAME»Target.xcconfig */ = {isa = ...
50B2938909F016FC00694E55 /* «PROJECTNAME»Priv.h */ = {isa = ...
D2AAC09D05546B4700DB518D /* lib«PROJECTNAME».dylib */ = {isa =
PBXFileReference; explicitFileType = "compiled.mach-o.dylib";
includeInIndex = 0; path = "lib«PROJECTNAME».dylib"; sourceTree = BUILT_PRODUCTS_DIR; };
/* End PBXFileReference section */

I ditch the Carbon framework reference (since I'm doing JNI) by deleting the line that references Carbon. If I want to use Carbon in a JNI library, I can always add it later. It turns out there are a bunch of other references to Carbon in the template, and I have to remove those as well.

Now, I see the line that makes reference to "lib«PROJECTNAME».dylib". I want the library to end with '.jnilib' since that is required by Mac OS X. So I patch this:

D2AAC09D05546B4700DB518D /* lib«PROJECTNAME».jnilib */ = {isa =
PBXFileReference; explicitFileType = "compiled.mach-o.dylib";
includeInIndex = 0; path = "lib«PROJECTNAME».jnilib"; sourceTree = BUILT_PRODUCTS_DIR; };
/* End PBXFileReference section */

Now, I want the header search paths to include the location of the Java headers, and the executable extension to be '.jnilib'. Searching the test project's '.pbxproj' file, I find them in the Debug and Release build sections. However, there are two pairs of these, one pair for the library target and one pair for the overall project. I need to change the ones for the overall project:

1DEB916508733D950010E9CD /* Debug */ = {
isa = XCBuildConfiguration;
baseConfigurationReference = 5073E0C609E734A800EC74B6 /* «PROJECTNAME»Proj.xcconfig */;
buildSettings = {
COPY_PHASE_STRIP = NO;
DEBUG_INFORMATION_FORMAT = dwarf;
EXECUTABLE_EXTENSION = jnilib;
GCC_ENABLE_FIX_AND_CONTINUE = YES;
GCC_OPTIMIZATION_LEVEL = 0;
HEADER_SEARCH_PATHS = "$(SDK)/System/Library/Frameworks/JavaVM.framework/Headers";
ZERO_LINK = YES;
};
name = Debug;
};
1DEB916608733D950010E9CD /* Release */ = {
isa = XCBuildConfiguration;
baseConfigurationReference = 5073E0C609E734A800EC74B6 /* «PROJECTNAME»Proj.xcconfig */;
buildSettings = {
ARCHS = (
ppc,
i386,
);
DEAD_CODE_STRIPPING = YES;
DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
EXECUTABLE_EXTENSION = jnilib;
HEADER_SEARCH_PATHS = "$(SDK)/System/Library/Frameworks/JavaVM.framework/Headers";

PRESERVE_DEAD_CODE_INITS_AND_TERMS = YES;
SEPARATE_STRIP = YES;
};
name = Release;
};

I was encouraged when creating a new project didn't fail. Looking at the default setup, everything seems to be in order (no Carbon framework, all the boilerplate files are there, etc.). A few minutes to add in the JNI binding function and change the Java program to use the new JNI library and I have a successful test.

I'm sure there are many other customizations I could make if I only knew about them. I also wish there were a better tool than a text editor for editing these files (hopefully there is and someone will point it out to me).