Monthly Archives: May 2013

Check for JNI exceptions!

Recently, I had a problem on a production system with an ALGOL to Java JNI (Java Native Interface) connection. Since this problem is still fresh in my head, I will post about it now. In a later blog I’ll go through the steps I took to set up the JNI connection in more detail.

At this customer site, a separate database is run for each brand they are marketing. Each brand has a connection to Websphere-MQ. A Java program on the JProcessor connects to MQ, the ALGOL program calls methods in the Java program. The whole Java environment (JVM) is like a library to the ALGOL program.

The ALGOL program contains the controlling loop. It calls a Java method to fetch a message from the Message Queue. The ALGOL program processeses the message and calls Java to put a reply message on the queue. It then fetches the next message etc. If are no messages in the queue, the Java method blocks until a message arrives.

But, when we setting up a new brand, and consequently another copy of the ALGOL-Java program, we could not get it to work. The new copy of the ALGOL program kept failing with a “REQUESTED MEMORY SIZE GREATER THAN 65535 WORDS” message. I usually put an ON ANYFAULT, PROGRAMDUMP(ALL) statement in my ALGOL programs, so the fault produced a program dump. The fault appeared to come from the SYSTEM/JAVAJNILIB. This librray tried to resize an array beyond the 65535 words limit as soon as the new environment tried to call a Java method. To test if we had hit a limit somewhere, we stopped one of the running environments and started the new one. Now it worked and restarting the stopped, existing environment failed.

At this point I contacted Unisys. After all, the dump showed that it was the SYSTEM/JAVAJNILIB that had all of it’s 65535 data buffers filled up with parameters from ALGOL to Java. The Unisys people jumped onto the problem right away and after some mails back and forth they came up with three problem areas: The JAVAJNILIB could do with some better failure detection/recovery, the generated include file should call JNI_CANCEL_PARAMETER_LIST if creating a parameter list to Java fails and, most importantly, my ALGOL program should check for a JVM exception after each call to a Java method. Because the ALGOL program did not check for Java exceptions, it kept firing off method calls to a JVM that was no longer functional, thus, at high speed, eating up all the data buffers in the JAVAJNILIB.
Here is the code snippet that does a method call from ALGOL to Java:

00105900    JNILIB_MCPJNICONNECTION_COMMAND(command,                            
00106000        0, offset(pc), result, rsltOffset, rsltSz);                     
00106010    if JNI_Check_Exception(JNIEnv) then                                 
00106020    begin                                                               
00106030      errorCode := JNI_GET_LAST_ERROR(JNIEnv);                          
00106052      replace axmsg by "JNI Exception: Error code = ",                  
00106054         errorCode for * digits, NULL;                                  
00106056      display(axmsg);                                                   
00106058      myself.status := value(TERMINATED);                               
00106060    end;

After the call to the Java method in line 105900, JNI_Check_Exception is called in line 106010. If that call returns a true, the JNI error code is determined. The exception is displayed and the program is terminated. In my case, I could leave it to COMS to restart the program.
A Java utility “JifGen” is provided to generate wrapper procedures that do the low level JNI calls. The current JifGen does not generate the JNI_CANCEL_PARAMETER_LIST in, but you can add that bit of code yourself. This the procedure JNILIB_MCPJNICONNECTION_COMMAND that is called from the user program:

00016500%                                                                       
00016600% Class:     nl_acme_mcp_mq_MCPJNIConnection                          
00016700% Method:    command                                                    
00016800% Signature: (Ljava/lang/String;)Ljava/lang/String;                     
00016900% ID: 2                                                                 
00017000%                                                                       
00017100PROCEDURE JNILIB_MCPJNICONNECTION_COMMAND                               
00017200    (ARG0, ARG0_OFFSET, ARG0_SIZE, RESULT, RESULT_OFFSET, RESULT_SIZE); 
00017300  VALUE         ARG0_OFFSET, ARG0_SIZE, RESULT_OFFSET ;                 
00017400  EBCDIC ARRAY  ARG0[0];                % Input                         
00017500  INTEGER       ARG0_OFFSET;                                            
00017600  INTEGER       ARG0_SIZE;              % Input                         
00017700  EBCDIC ARRAY  RESULT[0];              % Output                        
00017800  INTEGER       RESULT_OFFSET;                                          
00017900  INTEGER       RESULT_SIZE;            % Output                        
00018000BEGIN                                                                   
00018100  LABEL xit;                                                            
00018200  OWN DOUBLE methodID;                                                  
00018300  REAL hParams;                                                         
00018400                                                                        
00018500  IF JNI_IsNull(methodID) THEN                                          
00018600  BEGIN                                                                 
00018700    methodID := JNI_GET_STATIC_METHOD_ID_E(                             
00018800            JNIEnv,                                                     
00018900            CLASS_MCPJNIConnection_REF,                                 
00019000            JNILIB_StringPool, 50,       % command                      
00019100            JNILIB_StringPool, 58 );     %                              
00019200    %  (Ljava/lang/String;)Ljava/lang/String;                           
00019300    IF JNI_Check_Exception( JNIenv ) THEN GO TO xit;                    
00019400  END IsNull;                                                           
00019500                                                                        
00019600  hParams := JNI_CREATE_PARAMETER_LIST( JNIEnv, methodID,               
00019700   JNILIB_StringPool, 58, 600 );           %                            
00019800  %  (Ljava/lang/String;)Ljava/lang/String;                             
00019900  IF hParams EQL 0 THEN GO TO xit;                                      
00020000  JNI_SET_STRING_EBCDIC_PARAMS_LEN(JNIEnv, hParams, ARG0, ARG0_OFFSET,  
00020100   ARG0_SIZE);                                                          
00020200  IF JNI_Check_Exception( JNIenv ) THEN                                 
00020220  begin                                                                 
00020240   JNI_CANCEL_PARAMETER_LIST(JNIEnv, hParams);                          
00020260   GO TO xit;                                                           
00020280  end;                                                                  
00020300  JNI_CALL_STATIC_STRING_EBCDIC_METHOD_P(                               
00020400    JNIEnv, CLASS_MCPJNIConnection_REF, methodID, hParams, RESULT,      
00020500     RESULT_OFFSET, RESULT_SIZE);                                       
00020600                                                                        
00020700  xit:                                                                  
00020800END JNILIB_MCPJNICONNECTION_COMMAND;

The code generated by GifGen just does the GO TO xit. I added the JNI_CANCEL_PARAMETER_LIST in line 20240.

Allthough a UCF was submitted for the JAVAJNILIB and the JifGen, these two points are not really an issue, as long as you make sure you check for exceptions after each call to the JVM.

References:

  • The Java Native Interface Programmer’s Guide and Specification, Sheng Liang, 1999
  • The Unisys Javadoc for JNI on the MCP system in *JRE/jni

Tomcat on the JProcessor is a walk in the park

However, if you want to go off the paved paths, you’d better bring your boots.

My roots are in the MCP environment, so when I install new software, I like to install and run it under some usercode. Preferably non-privileged so I can’t accidently break anything else. But the Java environment relies on the POSIX file system with it’s concept of permanent directories. On the MCP system, permanent directories can only exist in the *DIR/= directory of a pack family. Although the Tomcat installation and running works fine under a usercode, deploying or redeploying web applications under usercode can be surprising, if not impossible. In a later blog I’ll cover my experiments with Tomcat under a usercode.

Tomcat comes from the Apache web site (http://tomcat.apache.org) as a zip file. The easiest way to install it, is by unzipping the file on a Windows PC and use a Windows share to an MCP pack to copy the unzipped Tomcat directory to the MCP system as something like *DIR/tomcat/=. The files will have the OWNER attribute set to the user of the Windows share. If this is not the Tomcat usercode, you’ll have to change the file OWNER attribute, e.g.:

WFL ALTER *DIR/TOMCAT/= (OWNER=”TOMCAT”)

Better not try to copy the files around using LIBRARY MAINTENANCE, this may result in things like “Missing Permanent Directory”.

I checked the Linux startup file (catalina.sh) to see what parameters and settings Tomcat needs to run and copied these arguments to a WFL. Here is the WFL I’m using to start Tomcat (my Tomcat is in *DIR/TOMCAT7 ON USER):

NEXT+ ....*....1....*....2....*....3....*....4....*....5....*....6....*....7..
00000100BEGIN JOB WFL/START/TOMCAT;
00000200CLASS = 20;
00000300RUN *DIR/JRE6/BIN/JAVA (
00000400 " -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,"
00000500&"server=y,suspend=n"
00000600&" -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
00000700&" -Djava.util.logging.config.file=./conf/logging.properties"
00000800&" -Djava.endorsed.dirs=./endorsed"
00000900&" -classpath .:bin/bootstrap.jar:bin/tomcat-juli.jar"
00001000&" -Dcatalina.home=/-/USER/DIR/TOMCAT7"
00001100&" org.apache.catalina.startup.Bootstrap"
00001200&" start");
00001300 CURRENTDIRECTORY="/-/USER/DIR/TOMCAT7";
00001500
00001600END JOB

In this WFL:

  • Line 300 starts the Java Virtual Machine (JVM).
  • Lines 400 and 500 make the JVM run in debug mode. This way you can connect the IDE to the running Tomcat and debug the application while it runs on the JProcessor. (How is that for debugging; the power of a Java IDE and the MCP SUMLOG to figure out what is going on!). Spaces are the argument separators in Java, so don’t put extra spaces in.
  • Lines 600, 700, 800 and 1000 set JVM system properties. These properties can be accessed from the Java code and are used by Tomcat.
  • Line 900 sets the classpath, the Bootstrap class searches the directories given in the classpath to load classes.
  • Line 1100 and 1200 actually execute Tomcat’s Bootstrap class with a “start” parameter.
  • Line 1300 tells the system that all relative paths are relative to this directory.

And this is the WFL I used to stop Tomcat:

NEXT+ ....*....1....*....2....*....3....*....4....*....5....*....6....*....7..
00000100BEGIN JOB WFL/STOP/TOMCAT;
00000400
00000500RUN *DIR/JRE6/BIN/JAVA (
00000800 " -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
00000900&" -Djava.util.logging.config.file=./conf/logging.properties"
00001000&" -Djava.endorsed.dirs=./endorsed"
00001100&" -classpath .:bin/bootstrap.jar:bin/tomcat-juli.jar"
00001300&" -Dcatalina.home=/-/USER/DIR/TOMCAT7"
00001500&" org.apache.catalina.startup.Bootstrap"
00001600&" stop");
00001700 CURRENTDIRECTORY="/-/USER/DIR/TOMCAT7";
00001800
00001900END JOB

Basically it is the same WFL, only this time the argument to the Boostrap class is “stop”.

The only configuration change you really need to make, is in the tomcat-users.xml file. As the name suggests, it is an XML file and you at least need to have in there :

<?xml version=”1.0”?>
<tomcat-users>
<role rolename=”manager-gui”/>
<user username=”manager” password=”secret” roles=”manager-gui”/>
</tomcat-users>

This gives you a logon to the Tomcat Manager pages (with user = “manager” and password = “secret” in this example), from where you can deploy, start, stop and undeploy web applications. If web application’s pages need logons, these roles, users and passwords are specified in here as well.

Now, Tomcat can be started. Since nothing was changed in the server.xml file, Tomcat will use port 8080 to serve pages.

I like to use the “WAR file deploy” option of the Manager. It allows me to select a Web application ARchive (WAR file) on my local PC, upload it to the server, unpack it into the webapps directory and start the application.

I mostly use the Netbeans IDE for Java programming. In Netbeans I can program the HTML pages, JSPs, sevlets and supporting classes. I can also maintain the web.xml and context.xml files there. By doing a “Clean and Build” Netbeans creates a ready to deploy WAR file. Of course any Java IDE, like Eclipse, can do this.