Calling C From Java Is Easy

Sometimes we need to access operating system functions that the standard Java API doesn’t expose, or use non-Java libraries. Although it’s well known that you can call this “native code” from Java using JNI, there is not so much entry-level material on how it’s actually done. It is often left out of introductory material–including the official Java Tutorial. Here I hope to give a short introduction to get you started.

Programming in JNI starts with defining a class with methods declared as native. Next you generate a C header file that declares functions that implement. Then you define these functions in a separate file and compile it into a shared library. In this tutorial we’ll use Linux and the GCC compiler.

Suppose we need to know the name of the terminal device from which the JVM was launched. (A slightly silly example since you could just use /dev/tty.) From C you would do this by calling the POSIX functions isatty and ttyname. Let’s make a class that gives us access to them:

package ex;
public class TTYUtil {
    static { System.loadLibrary("ttyutil"); }
    public static native boolean isTTY();
    public static native String getTTYName();
}

The call to System.loadLibrary in the class initializer looks for a shared library and links it to the JVM. The file name depends on the operating system: on Windows this code would look for ttyutil.dll, on Linux and Solaris libttyutil.so.

The next step is compiling the Java code and generating the C header file:

$ javac ex/TTYUtil.java
$ javah ex.TTYUtil

The javah tool will create a C header file called ex_TTYUtil.h. This file contains the declarations for the C functions we have to define:

JNIEXPORT jboolean JNICALL Java_ex_TTYUtil_isTTY
  (JNIEnv *, jclass);

JNIEXPORT jstring JNICALL Java_ex_TTYUtil_getTTYName
  (JNIEnv *, jclass);

The idea is that the header file should be generated automatically during the build process so you should not modify it manually. If you need other declarations or #include statements you should use a different file.

Create ex_TTYUtil.c to define the C functions:

#include "ex_TTYUtil.h"
#include <unistd.h>

JNIEXPORT jstring JNICALL Java_ex_TTYUtil_getTTYName
  (JNIEnv *env, jclass cls)
{
    char *name = ttyname(STDOUT_FILENO);
    return (*env)->NewStringUTF(env, name);
}

JNIEXPORT jboolean JNICALL Java_ex_TTYUtil_isTTY
  (JNIEnv *env, jclass cls)
{
    return isatty(STDOUT_FILENO)? JNI_TRUE: JNI_FALSE;
}

These functions receive the JNI environment object as the first argument. The second argument is the class for static native methods and the object for non-static methods. The rest of the arguments, if any, are the method arguments from Java. The JNI environment is used for interacting with the virtual machine, like here the NewStringUTF function is used to create a new Java String object from a C string.

To compile and link the C code you can use:

$ gcc -fPIC -c ex_TTYUtil.c -I $JAVA_HOME/include
$ gcc ex_TTYUtil.o -shared -o libttyutil.so -Wl,-soname,ttyutil

Now you should have libttyutil.so in your working directory. Let’s try using the library.

import ex.TTYUtil;
public class Test {
    public static void main(String[] args) {
        if (TTYUtil.isTTY()) {
            System.out.println("TTY: "+TTYUtil.getTTYName());
        } else {
            System.out.println("Not a TTY");
        }
    }
}

Compile this class like you normally would and then run it:

$ export LD_LIBRARY_PATH=.
$ java Test
TTY: /dev/pts/3

And what if the output is not connected to a terminal?

$ java Test | cat
Not a TTY

The JVM looks for native libraries in the paths specified in the system property java.library.path in addition to what’s normal for the operating system. Here we made the shared library available to the JVM temporarily by adding the current directory to LD_LIBRARY_PATH. To permanently install a shared library on Linux you would copy the .so to /usr/lib (or any other directory mentioned in /etc/ld.so.conf) and then run ldconfig.

The JVM ate my variable!

Consider this code:

Object obj = new Object();
WeakReference<Object> ref = new WeakReference<Object>(obj);

List<byte[]> filler = new LinkedList<byte[]>();
while (ref.get() != null) {
    filler.add(new byte[1000]);
}
System.out.println("Filler size " + filler.size());

When executed it should always run out of memory: ref.get() will not return null because the referenced object cannot be garbage collected. The local variable obj still holds a reference to it. The filler increases in size indefinitely, and the program will ultimately crash with an OutOfMemoryError.

What really happens is this:

$ java test
Filler size 28186

Wait, what?!

Continue reading “The JVM ate my variable!”

MySQL Collations in practice

Collations are important for how text is ordered by indices and ORDER BY clauses; in fact a collation defines the order. Sadly developers seem to often ignore collations in data models, and the database default settings are used. If you are using MySQL and haven’t configured any particular collation, your database is probably sorted according to Swedish rules.

Let’s check what MySQL collations mean in practice. First create a table with three fields, each with the same data but a different collations:

CREATE TABLE test (
  en varchar(3) CHARACTER SET utf8 COLLATE utf8_general_ci,
  se varchar(3) CHARACTER SET utf8 COLLATE utf8_swedish_ci,
  es varchar(3) CHARACTER SET utf8 COLLATE utf8_spanish_ci,
);

Then let’s fill the columns with some data:

INSERT INTO test (en, se, es) VALUES ('A','A','A');
INSERT INTO test (en, se, es) VALUES ('Ä','Ä','Ä');
INSERT INTO test (en, se, es) VALUES ('N','N','N');
INSERT INTO test (en, se, es) VALUES ('Ñ','Ñ','Ñ');
INSERT INTO test (en, se, es) VALUES ('Z','Z','Z');
INSERT INTO test (en, se, es) VALUES ('Ö','Ö','Ö');
INSERT INTO test (en, se, es) VALUES ('Nz','Nz','Nz');
INSERT INTO test (en, se, es) VALUES ('Az','Az','Az');

Now comes the fun part. The generic collation is loosely based on English, where Ä=A, N=Ñ, and O=Ö. We get this ordering:

SELECT en FROM test ORDER BY en;
-- Result: A Ä Az N Ñ Nz Ö Z

In Spanish N and Ñ are different letters so we get this ordering:

SELECT es FROM test ORDER BY es:
-- Result: A Ä Az N Nz Ñ Ö Z

In Swedish Ä and Ö are not only separate from A and O; they are the last letters of the alphabet:

SELECT se FROM test ORDER BY se:
-- Result: A Az N Ñ Nz Z Ä Ö

Ok, that’s it for ORDER BY. What about indices?

mysql> ALTER TABLE test ADD UNIQUE INDEX idx_en (en);
ERROR 1062 (23000): Duplicate entry 'Ñ' for key 'idx_en'

Makes sense, N and Ñ are the same according to English! With Swedish you would get the same error. What about Spanish?

mysql> ALTER TABLE test ADD UNIQUE INDEX idx_es (es);
ERROR 1062 (23000): Duplicate entry 'A' for key 'idx_es'

Here N and Ñ are different, but A and Ä are considered still the same. There are more examples of the effects of collation in the MySQL documentation.

The default collation on many installations is Swedish because MySQL AB, the company that created the database, is Swedish. So, next time when you create a table don’t forget to set the collation to something that makes sense for your users.