Tag Archives: ALGOL

Java as a library, part 2

In the previuos blog I’ve made a Java class Grep that exports a method named doGrep(). I also supplied a Java test driver to test the Grep class. In this blog I will write the ALGOL program that takes the search text and directory as a parameter and calls the Java doGrep() method to do the actual work.

The utility JifGen has been run to generate the ALGOL include file that holds the definitions of the Java methods (GREP_INCL.ALG_M). I copied this ALGOL symbol to my MCP system as
GREP/SRC/INCLUDE/GREP_INCL.

My ALGOL program looks like this:

NEXT+   ....*....1....*....2....*....3....*....4....*....5....*....6....*....7..
00001000%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
00002000% grep                                                                  
00003000% Parameters:                                                           
00004000%   text:    String parameter of the text to search for.                
00005000%   path:    String parameter of file or directory to search.           
00006000%   recurse: Boolean, if TRUE all subdirectories will also be           
00007000%            searched.                                                  
00008000%                                                                       
00009000% Note: The Apache Software Licence applies to this code.               
00010000% (http://www.apache.org/licenses/LICENSE-2.0.html)                     
00011000%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
00012000$ tads                                                                  
00013000$$ set level 2                                                          
00014000procedure grep(arguments);                                              
00015000array arguments[*];                                                     
00016000begin                                                                   
00017000$$  INCLUDE JNI="*DIR/JRE6/JDK/INCLUDE/JNILIB/ALGOL ON DISK"            
00018000                                                                        
00019000    REAL ARRAY envEtc[0:JNI_MIN_ENV_ARRAY_SIZE+JNI_MIN_JVM_ARRAY_SIZE ];
00020000    ARRAY JNIEnv[0] = envEtc;                                           
00021000    ARRAY JavaVM[-JNI_MIN_ENV_ARRAY_SIZE] = envEtc;                     
00022000    ebcdic value array jvmOptions ("-cp ./grep.jar " 48"00");           
00023000    ebcdic value array mcpOptions (48"00");                             
00024000    integer jvmRslt;                                                    
00025000    double returnArray;                                                 
00026000    ebcdic array param[0:299];                                          
00027000    ebcdic array ax[0:119];                                             
00028000    integer errorCode;                                                  
00029000    pointer pparam;                                                     
00029200    file lp(kind = printer);                                            
00029400    ebcdic array line[0:132];                                           
00030000                                                                        
00031000$$  INCLUDE GREPJNI="GREP/SRC/INCLUDE/GREP_INCL"                        
00032000                                                                        
00033000    procedure listHits(returnArray);                                    
00034000    value returnArray;                                                  
00035000    double returnArray;                                                 
00036000    begin                                                               
00038000        ebcdic array found[0:300];                                      
00039000        integer foundLen;                                               
00040000        double arrayElement;                                            
00041000        integer returnLength;                                           
00042000        integer i;                                                      
00043000                                                                        
00044000        returnLength := JNI_GET_ARRAY_LENGTH(JNIEnv, returnArray);      
00045000                                                                        
00046000        for i := 0 step 1 until (returnLength - 1) do                   
00047000        begin                                                           
00048000            replace found by " " for 300;                               
00049000            foundLen := 300;                                            
00050000            arrayElement := JNI_GET_OBJECT_ARRAY_ELEMENT                
00051000                (JNIEnv, returnArray, i);                               
00052000            JNI_GET_STRING_EBCDIC_CHARS                                 
00053000                (JNIEnv, arrayElement, found, 0, foundLen);             
00054000            if foundLen > 0 then                                        
00055000            begin                                                       
00056000                write(lp, *, found);                                    
00057000            end;                                                        
00058000        end;                                                            
00059000    end of listHits;                                                    
00060000                                                                        
00061000    if (size(arguments) * 6) > 300 then                                 
00062000    begin                                                               
00063000        resize(param, size(arguments) * 6, DISCARD);                    
00064000    end;                                                                
00065000    replace pparam:param[0] by pointer(arguments)                       
00066000        for size(arguments) * 6 until = 48"00";                         
00067000                                                                        
00068000    % Initialize the Java Virtual Machine                               
00069000    jvmRslt := JNI_CREATE_JAVA_VM_STAR(JavaVM, JNIEnv,                  
00070000                                       JNI_VERSION_1_6,                 
00071000                                       jvmOptions,                      
00072000                                       mcpOptions);                     
00072100    if jvmRslt < 0 then                                                 
00072200    begin                                                               
00072300        replace line by " " for 132;                                    
00072400        replace line by "Result from JNI_CREATE_JAVA_VM_STAR: ",        
00072500            jvmRslt for * digits;                                       
00072600        write(lp, 132, line);                                           
00072700    end                                                                 
00072800    else                                                                
00072900    begin                                                               
00073000        INITIALIZE_GREP_;                                               
00074000        returnArray := GREP__GREP_DO_GREP(param,                        
00075000            0, offset(pparam));                                         
00076000        if JNI_Check_Exception(JNIEnv) then                             
00077000        begin                                                           
00078000            errorCode := JNI_GET_LAST_ERROR(JNIEnv);                    
00079000            replace ax by "JNI Exception: Error code = ",               
00080000                errorCode for * digits, 48"00";                         
00081000            display(ax);                                                
00082000        end;                                                            
00083000        listHits(returnArray);                                          
00083500        JNI_DESTROY_JAVA_VM(JavaVM);                                    
00084000    end;                                                                
00085000end of grep.                                                            

Line 17000 includes the variables and procedure declarations that are necessary to communicate via JNI. It needs to be included before the include of the generated GREP/SRC/INCLUDE/GREP_INCL. The lines 19000 – 21000 declare some arrays that the generated ALGOL file needs. The comments at the top of the generated include file remind you of this.

Lines 22000 and 23000 define parameter strings that need to be supplied to the JNI_CREATE_JAVA_VM_STAR procedure (line 69000). This procedure will start a Java Virtual Machine on the JProcessor. The array jvmOptions sets the CLASSPATH of the JVM. In this program I have hardcoded the CLASSPATH relative to the CURRENTDIRECTORY attribute of the program. I a production system you would probably get these parameters from a configuration file.

The INIALIZE_GREP_ in line 73000 results in a call (via GREP/SRC/INCLUDE/GREP_INCL) of the JAVAJNILIB library procdeure JNI_FIND_CLASS_E. This procedure returns a handle to the Grep class.

Since I've made the doGrep() method a static method, I don't need to create an instance of the Grep class. I find this makes the ALGOL code less complicated. In a production system that I built, I defined a single class to interface between Java and ALGOL. A call to a static method in this class creates a single instance of a Manager class, the Manager class in turn, instantiates other objects as needed. This scheme can work because a JVM is dedicated to one program stack.

In line 74000, the actual doGrep method is called. The parameter param is an EBCDIC array that contains the arguments for doGrep. The next parameter, 0, is the offset into the param array where the actual arguments start. The last parameter gives the length, in characters, of the arguments string. The GREP__GREP_DO_GREP procedure in the generated include file wraps the actual, more complex, JNI call.

I started a test using the WFL below:

NEXT+   ....*....1....*....2....*....3....*....4....*....5....*....6....*....7..
00000100BEGIN JOB WFL/GREP;                                                     
00000200                                                                        
00000300RUN GREP/GREPDRIVER(                                                    
00000350        "-r grep  /-/user/usercode/ruud/grep/src");                     
00000400    LIBRARY JAVAVM(LIBACCESS = BYTITLE,                                 
00000500        TITLE=*DIR/JRE6/BIN/JAVAPROXYLIB ON DISK);                      
00000600    CURRENTDIRECTORY="/-/USER/USERCODE/RUUD/GREP";                      
00000700                                                                        
00000800END JOB 

The CURRENTDIRECTORY equation is, apparently, just for the GREP/GREPDRIVER program, it is not propagated to the Java Virtual Machine. That means that the path to directory to be searched by Grep must be an absolute path, it cannot be relative to the CURRENTDIRECTORY.

I also got a message “JAVAHOMEPATH not provided; JRE location extracted from proxy name (/-/DISK/DIR/JRE6).”. This message comes from the program *SYSTEM/JAVAFILE, which is used by the JVM to access the MCP file system. Adding a file equation FILE JAVAHOMEPATH(PATHNAME="/-/DISK/DIR/JRE6"); did not get rid of the message. But using the path from the PROXYLIB is just fine.

When I ran my test WFL, much to my surprise, I didn't get any results back. I figured the JNI_CREATE_JAVA_VM_STAR failed, that's why I added the code in lines 72300 – 72600. But there was no problem there, nor was there a problem in the GREP__GREP_DO_GREP call. Wouldn't it be nice if I could debug my Java code now. Well, I could, the way to do that was obvious: Just insert the debug arguments
-Xdebug -Xrunjdwp:transport=dt_socket,address=9000,server=y,suspend=y
at the beginning of the jvmOptions array and I could connect my Netbeans to the JVM under test. Stepping through the doGrep() Java code, the problem jumped right at me. The param.split(“ “) method splits the parameter string using a single space as a token separator, but my test job has two spaces between “grep” and “/-/user/....”. I have now changed the regular expression in the param.split to “ +”, telling the split to use one or more spaces to separate the string.

Java as a library Part 1

In the Unix/Linux world there is this utility called ‘grep’. It is a very handy tool that allows you to search for a string of text in one or more files of a directory. The original grep can handle wildcards in file names and regular expressions in the target text. My goal is to experiment with the Java Native Interface (JNI) between ALGOL and Java and to do so, I will implement a simplified version of grep. The simple version of grep still shows the power of MCP Java. Java is used to search for file names in directories and to scan the files for the target text. Using Java to find a target text in a string is all standard stuff, but I’m especially pleased with the Java ‘listFiles’ method, which is so much easier to use than setting up GETSTATUS calls from ALGOL.

I have set up a Netbeans project named ‘grep’. The class I am going to call from ALGOL is Grep. This class contains one public method, doGrep. The sourec code for the Grep class is here.

Only the method doGrep has been declared public. JifGen will generate an ALGOL call for this public method.

I am catching all exceptions in the Java program, because I have not yet quite figured out how to recover from all sorts of Java exceptions on the ALGOL end. It seems that simply re-initializing the JVM is not good enough. I suppose this can be the subject of another blog post.

To test the Grep class in my MCP environment, I included a test driver class, Greptest, in the project. GrepTest just contains a main method that calls Grep.doGrep(). I can now test and debug the Grep class while it runs on the MCP and actually reads MCP files and directories. To easily start the tests in debug mode, I made a small WFL with all the necessary parameters:

NEXT+   ....*....1....*....2....*....3....*....4....*....5....*....6....*....7..
00000100BEGIN JOB DEBUG;                                                        
00000200                                                                        
00000300RUN *DIR/JRE6/BIN/JAVA                                                  
00000420    ("  -Xdebug -Xrunjdwp:transport=dt_socket,address=9000,"            
00000440    &"server=y,suspend=y"                                               
00000490    &" -jar grep.jar -r  grep /-/user/usercode/ruud/grep/src");         
00000500    CURRENTDIRECTORY = "/-/USER/USERCODE/RUUD/GREP";                    
00000600                                                                        
00000700END JOB                                                                 

The arguments “-r grep /-/user/usercode/ruud/grep/src” tell grep to recursively look through the files in directory (RUUD)GREP/SRC/= ON USER searching for the text “grep”. The output of this run is below:

/-/user/usercode/ruud/grep/src/java/Grep.java:package com.mcpjava.mcp.grep;
/-/user/usercode/ruud/grep/src/java/Grep.java:                hits = grep(args[0].trim(), args[1].trim(), false);
/-/user/usercode/ruud/grep/src/java/Grep.java:                    hits = grep(args[1].trim(), args[2].trim(), true);
/-/user/usercode/ruud/grep/src/java/Grep.java:                    hits = grep(args[1].trim(), args[2].trim(), false);
/-/user/usercode/ruud/grep/src/java/Grep.java:    private static ArrayList grep(String text, String path, boolean recurse) {
/-/user/usercode/ruud/grep/src/java/Grep.java:                        ArrayList rslt = grep(text, f.getAbsolutePath(), recurse);
/-/user/usercode/ruud/grep/src/java/GrepTest.java:package com.mcpjava.mcp.grep;
/-/USER/USERCODE/RUUD/GREP/SRC/INCLUDE/GREP_INCL:11700:% com.mcpjava.mcp.grep.Grep
/-/USER/USERCODE/RUUD/GREP/SRC/INCLUDE/GREP_INCL:12200:  80"com/mcpjava/mcp/grep/Grep" 48"00"
/-/USER/USERCODE/RUUD/GREP/SRC/INCLUDE/GREP_INCL:13100:  % com/mcpjava/mcp/grep/Grep
/-/USER/USERCODE/RUUD/GREP/SRC/INCLUDE/GREP_INCL:13500:% ALGOL interfaces for class:com.mcpjava.mcp.grep.Grep
/-/USER/USERCODE/RUUD/GREP/SRC/INCLUDE/GREP_INCL:13900:% Class:       com_mcpjava_mcp_grep_Grep
/-/USER/USERCODE/RUUD/GREP/SRC/INCLUDE/GREP_INCL:18600:% Class:       com_mcpjava_mcp_grep_Grep
/-/USER/USERCODE/RUUD/GREP/SRC/GREPDRIVER:200:% grep
/-/USER/USERCODE/RUUD/GREP/SRC/GREPDRIVER:11000:procedure grep(arguments);
/-/USER/USERCODE/RUUD/GREP/SRC/GREPDRIVER:16520:    ebcdic value array jvmOptions ("-cp ./grep.jar" 48"00");
/-/USER/USERCODE/RUUD/GREP/SRC/GREPDRIVER:18000:end of grep.

Once I’m satisfied with the Grep class, I can generate the ALGOL interface. On my development PC I have a folder named \libs\mcp in which I put all the jar files that are part of the MCP Java release. The JifGen utility is included in the mcptools.jar file. A Netbeans “Clean and Build” creates a jar file in the “dist” folder of the project. The jar file, by default, has the name of the Netbeans project, in my case “grep.jar”. I directed the Command Prompt window to the Netbeans directory for the grep project and laboriously typed in the following command to generate the ALGOL include file:

java -cp /libs/mcp/mcp.jar;/libs/mcp/mcptools.jar;/"program files"/java/jdk1.6.0_24/lib/tools.jar com.unisys.mcp.tools.JifGen -algol -classpath dist/grep.jar -d incldir -output GREP_ -verbose com.mcpjava.mcp.grep.Grep

JifGen has now generated an ALGOL include file named GREP_INCL.ALG_M in directory incldir. An ALGOL procedure has been genereated for each public method in Java class Grep. It generated a procedure DOUBLE PROCEDURE GREP__GREP_DO_GREP which reflects the method Grep.doGrep on the Java side. The default class constructor procedure, DOUBLE PROCEDURE GREP__GREP_GREP, is also generated, but since my callable method is declared static, I don’t need to construct a Grep object.
On my first attempt I had left the “”program files”/java/jdk1.6.0_24/lib/tools.jar” out. This resulted in a Java Exception:
Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/javadoc/Doclet
I reread the Unisys JifGen Javadoc and yes, it’s documented there; apart from mcp.jar and mcptools.jar you also need to include the Java SDK (Oracle) provided tools.jar (that is where the com.sun.javadoc.Doclet is located).
Initially, I also had some trouble getting my classpaths right, but you have to realize that the
-cp /libs/mcp/mcp.jar;/libs/mcp/mcptools.jar;/"program files"/java/jdk1.6.0_24/lib/tools.jar
is to get JifGen going and the
-classpath dist/grep.jar
near the end is to tell JifGen where it can find the classes it must generate ALGOL for. JifGen does not need to see the Java sources because it uses Java Reflection to get the info from the compiled classes. This is the generated GREP_INCL.ALG_M.

In the previous blog I mentioned that the generated file needed an addition to call JNI_CANCEL_PARAMETER_LIST in case of an exception. No change needed anymore, in the latest MCPJava release JifGen has already been fixed!
In the next post I will write an ALGOL program that takes the string parameter, calls the Java’s Grep.doGrep() method with the parameter and unravels the result.