1 /**************************************************************
2 JLex: A Lexical Analyzer Generator for Java(TM)
3 Written by Elliot Berk. Copyright 1996.
4 Contact at ejberk@princeton.edu.
6 Version 1.2.4, 7/24/99, [C. Scott Ananian]
7 Correct the parsing of '-' in character classes, closing active
8 bug #1. Behaviour follows egrep: leading and trailing dashes in
9 a character class lose their special meaning, so [-+] and [+-] do
10 what you would expect them to.
11 New %ignorecase directive for generating case-insensitive lexers by
12 expanding matched character classes in a unicode-friendly way.
13 Handle unmatched braces in quoted strings or comments within
15 Fixed input lexer to allow whitespace in character classes, closing
16 active bug #9. Whitespace in quotes had been previously fixed.
17 Made Yylex.YYEOF and %yyeof work like the manual says they should.
19 Version 1.2.3, 6/26/97, [Raimondas Lencevicius]
20 Fixed the yy_nxt[][] assignment that has generated huge code
21 exceeding 64K method size limit. Now the assignment
22 is handled by unpacking a string encoding of integer array.
23 To achieve that, added
24 "private int [][] unpackFromString(int size1, int size2, String st)"
25 function and coded the yy_nxt[][] values into a string
26 by printing integers into a string and representing
27 integer sequences as "value:length" pairs.
28 Improvement: generated .java file reduced 2 times, .class file
29 reduced 6 times for sample grammar. No 64K errors.
30 Possible negatives: Some editors and OSs may not be able to handle
31 the huge one-line generated string. String unpacking may be slower
32 than direct array initialization.
34 Version 1.2.2, 10/24/97, [Martin Dirichs]
36 Changed yy_instream to yy_reader of type BufferedReader. This reflects
37 the improvements in the JDK 1.1 concerning InputStreams. As a
38 consequence, changed yy_buffer from byte[] to char[].
39 The lexer can now be initialized with either an InputStream
40 or a Reader. A third, private constructor is called by the other
41 two to execute user specified constructor code.
43 Version 1.2.1, 9/15/97 [A. Appel]
44 Fixed bugs 6 (character codes > 127) and 10 (deprecated String constructor).
46 Version 1.2, 5/5/97, [Elliot Berk]
48 Simply changed the name from JavaLex to JLex. No other changes.
50 Version 1.1.5, 2/25/97, [Elliot Berk]
52 Simple optimization to the creation of the source files.
53 Added a BufferedOutputStream in the creation of the DataOutputStream
54 field m_outstream of the class CLexGen. This helps performance by
55 doing some buffering, and was suggested by Max Hailperin,
56 Associate Professor of Computer Science, Gustavus Adolphus College.
58 Version 1.1.4, 12/12/96, [Elliot Berk]
60 Added %public directive to make generated class public.
62 Version 1.1.3, 12/11/96, [Elliot Berk]
64 Converted assertion failure on invalid character class
65 when a dash '-' is not preceded with a start-of-range character.
66 Converted this into parse error E_DASH.
68 Version 1.1.2, October 30, 1996 [Elliot Berk]
69 Fixed BitSet bugs by installing a BitSet class of my own,
70 called JavaLexBitSet. Fixed support for '\r', non-UNIX
71 sequences. Added try/catch block around lexer generation
72 in main routine to moderate error information presented
73 to user. Fixed macro expansion, so that macros following
74 quotes are expanded correctly in regular expressions.
75 Fixed dynamic reallocation of accept action buffers.
77 Version 1.1.1, September 3, 1996 [Andrew Appel]
78 Made the class "Main" instead of "JavaLex",
79 improved the installation instructions to reflect this.
81 Version 1.1, August 15, 1996 [Andrew Appel]
82 Made yychar, yyline, yytext global to the lexer so that
83 auxiliary functions can access them.
85 /***************************************************************
87 **************************************************************/
90 /***************************************************************
92 **************************************************************/
93 import java.io.InputStream;
94 import java.io.OutputStream;
95 import java.io.FileInputStream;
96 import java.io.FileOutputStream;
97 import java.io.DataOutputStream;
98 import java.io.BufferedOutputStream;
100 import java.lang.System;
101 import java.lang.Integer;
102 import java.lang.Character;
104 import java.util.Enumeration;
105 import java.util.Stack;
106 import java.util.Hashtable;
107 import java.util.Vector;
109 /******************************
111 2) How should I use the Java package system
112 to make my tool more modularized and
116 !) Fix BitSet issues -- expand only when necessary.
117 2) Repeated accept rules.
118 6) Clean up the CAlloc class and use buffered
120 7) Reading and writing in Unicode.
121 Use DataInputStream and DataOutputStream,
122 along with writeUTF???
123 8) Fix or at least annotate ^x bug.
124 9) Add to spec about extending character set.
125 10) making sure newlines are handled correctly
126 S and consistly with the m_unix flag
127 11) m_verbose -- what should be done with it?
128 12) turn lexical analyzer into a coherent
130 13) turn lexical analyzer generator into a
131 coherent Java package
132 16) pretty up generated code
133 17) make it possible to have white space in
135 18) clean up all of the class files the lexer
136 generator produces when it is compiled,
137 and reduce this number in some way.
138 24) character format to and from file: writeup
140 25) Debug by testing all arcane regular expression cases.
141 26) Look for and fix all UNDONE comments below.
142 27) Fix package system.
143 28) Clean up unnecessary classes.
144 *****************************/
146 /***************************************************************
148 **************************************************************/
151 /***************************************************************
153 **************************************************************/
155 /* Lexical States. */
156 Hashtable m_states; /* Hashtable taking state indices (Integer)
157 to state name (String). */
159 /* Regular Expression Macros. */
160 Hashtable m_macros; /* Hashtable taking macro name (String)
161 to corresponding char buffer that
162 holds macro definition. */
165 CNfa m_nfa_start; /* Start state of NFA machine. */
166 Vector m_nfa_states; /* Vector of states, with index
167 corresponding to label. */
169 Vector m_state_rules[]; /* An array of Vectors of Integers.
170 The ith Vector represents the lexical state
171 with index i. The contents of the ith
172 Vector are the indices of the NFA start
173 states that can be matched while in
174 the ith lexical state. */
177 int m_state_dtrans[];
180 Vector m_dfa_states; /* Vector of states, with index
181 corresponding to label. */
182 Hashtable m_dfa_sets; /* Hashtable taking set of NFA states
183 to corresponding DFA state,
184 if the latter exists. */
186 /* Accept States and Corresponding Anchors. */
187 Vector m_accept_vector;
188 int m_anchor_array[];
190 /* Transition Table. */
191 Vector m_dtrans_vector;
196 /* Regular expression token variables. */
202 /* Verbose execution flag. */
205 /* JLex directives flags. */
206 boolean m_integer_type;
207 boolean m_intwrap_type;
209 boolean m_count_chars;
210 boolean m_count_lines;
211 boolean m_cup_compatible;
214 boolean m_ignorecase;
219 char m_init_throw_code[];
220 int m_init_throw_read;
228 char m_eof_value_code[];
229 int m_eof_value_read;
231 char m_eof_throw_code[];
232 int m_eof_throw_read;
234 char m_yylex_throw_code[];
235 int m_yylex_throw_read;
237 /* Class, function, type names. */
238 char m_class_name[] = {
242 char m_implements_name[] = {};
243 char m_function_name[] = {
247 char m_type_name[] = {
253 /* Lexical Generator. */
254 private CLexGen m_lexGen;
256 /***************************************************************
258 ***********************************************************/
259 static final int NONE = 0;
260 static final int START = 1;
261 static final int END = 2;
263 /***************************************************************
265 Description: Constructor.
266 **************************************************************/
274 /* Initialize regular expression token variables. */
275 m_current_token = m_lexGen.EOS;
280 /* Initialize hashtable for lexer states. */
281 m_states = new Hashtable();
282 m_states.put(new String("YYINITIAL"),new Integer(m_states.size()));
284 /* Initialize hashtable for lexical macros. */
285 m_macros = new Hashtable();
287 /* Initialize variables for lexer options. */
288 m_integer_type = false;
289 m_intwrap_type = false;
290 m_count_lines = false;
291 m_count_chars = false;
292 m_cup_compatible = false;
296 m_ignorecase = false;
298 /* Initialize variables for JLex runtime options. */
302 m_nfa_states = new Vector();
304 m_dfa_states = new Vector();
305 m_dfa_sets = new Hashtable();
307 m_dtrans_vector = new Vector();
308 m_dtrans_ncols = CUtility.MAX_SEVEN_BIT + 1;
312 m_accept_vector = null;
313 m_anchor_array = null;
318 m_init_throw_code = null;
319 m_init_throw_read = 0;
321 m_yylex_throw_code = null;
322 m_yylex_throw_read = 0;
330 m_eof_value_code = null;
331 m_eof_value_read = 0;
333 m_eof_throw_code = null;
334 m_eof_throw_read = 0;
336 m_state_dtrans = null;
338 m_state_rules = null;
342 /***************************************************************
344 **************************************************************/
347 /***************************************************************
349 **************************************************************/
350 private CSpec m_spec;
351 private DataOutputStream m_outstream;
353 /***************************************************************
354 Constants: Anchor Types
355 **************************************************************/
356 private final int START = 1;
357 private final int END = 2;
358 private final int NONE = 4;
360 /***************************************************************
362 **************************************************************/
363 private final boolean EDBG = true;
364 private final boolean NOT_EDBG = false;
366 /***************************************************************
368 Description: Constructor.
369 **************************************************************/
377 /***************************************************************
379 Description: Clears member variables.
380 **************************************************************/
389 /***************************************************************
391 Description: Initializes member variables.
392 **************************************************************/
396 OutputStream outstream
401 CUtility.assert(null != spec);
402 CUtility.assert(null != outstream);
406 m_outstream = new DataOutputStream(outstream);
409 /***************************************************************
410 Function: emit_imports
411 Description: Emits import packages at top of
412 generated source file.
413 **************************************************************/
417 OutputStream outstream
419 throws java.io.IOException
425 CUtility.assert(null != m_spec);
426 CUtility.assert(null != m_outstream);
429 /*m_outstream.writeBytes("import java.lang.String;\n");
430 m_outstream.writeBytes("import java.lang.System;\n");
431 m_outstream.writeBytes("import java.io.BufferedReader;\n");
432 m_outstream.writeBytes("import java.io.InputStream;\n");*/
437 /***************************************************************
438 Function: print_details
439 Description: Debugging output.
440 **************************************************************/
441 private void print_details
453 System.out.println("---------------------- Transition Table "
454 + "----------------------");
456 for (i = 0; i < m_spec.m_row_map.length; ++i)
458 System.out.print("State " + i);
460 accept = (CAccept) m_spec.m_accept_vector.elementAt(i);
463 System.out.println(" [nonaccepting]");
467 System.out.println(" [accepting, line "
468 + accept.m_line_number
470 + (new java.lang.String(accept.m_action,0,
471 accept.m_action_read))
474 dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(m_spec.m_row_map[i]);
477 state = dtrans.m_dtrans[m_spec.m_col_map[0]];
478 if (CDTrans.F != state)
481 System.out.print("\tgoto " + state + " on [" + ((char) 0));
483 for (j = 1; j < m_spec.m_dtrans_ncols; ++j)
485 next = dtrans.m_dtrans[m_spec.m_col_map[j]];
488 if (CDTrans.F != state)
490 System.out.print((char) j);
498 System.out.println("]");
501 if (CDTrans.F != state)
504 System.out.print("\tgoto " + state + " on [" + ((char) j));
510 System.out.println("]");
514 System.out.println("---------------------- Transition Table "
515 + "----------------------");
518 /***************************************************************
520 Description: High-level access function to module.
521 **************************************************************/
525 OutputStream outstream
527 throws java.io.IOException
533 CUtility.assert(null != m_spec);
534 CUtility.assert(null != m_outstream);
537 if (CUtility.OLD_DEBUG) {
550 /***************************************************************
551 Function: emit_construct
552 Description: Emits constructor, member variables,
554 **************************************************************/
555 private void emit_construct
558 throws java.io.IOException
562 CUtility.assert(null != m_spec);
563 CUtility.assert(null != m_outstream);
567 m_outstream.writeBytes("\tprivate final int YY_BUFFER_SIZE = 512;\n");
569 m_outstream.writeBytes("\tprivate final int YY_F = -1;\n");
570 m_outstream.writeBytes("\tprivate final int YY_NO_STATE = -1;\n");
572 m_outstream.writeBytes("\tprivate final int YY_NOT_ACCEPT = 0;\n");
573 m_outstream.writeBytes("\tprivate final int YY_START = 1;\n");
574 m_outstream.writeBytes("\tprivate final int YY_END = 2;\n");
575 m_outstream.writeBytes("\tprivate final int YY_NO_ANCHOR = 4;\n");
577 m_outstream.writeBytes("\tprivate final char YY_EOF = \'\\uFFFF\';\n");
578 if (m_spec.m_integer_type || true == m_spec.m_yyeof)
579 m_outstream.writeBytes("\tpublic final int YYEOF = -1;\n");
581 /* User specified class code. */
582 if (null != m_spec.m_class_code)
584 m_outstream.writeBytes(new String(m_spec.m_class_code,0,
585 m_spec.m_class_read));
588 /* Member Variables */
589 m_outstream.writeBytes("\tprivate java.io.BufferedReader yy_reader;\n");
590 m_outstream.writeBytes("\tprivate int yy_buffer_index;\n");
591 m_outstream.writeBytes("\tprivate int yy_buffer_read;\n");
592 m_outstream.writeBytes("\tprivate int yy_buffer_start;\n");
593 m_outstream.writeBytes("\tprivate int yy_buffer_end;\n");
594 m_outstream.writeBytes("\tprivate char yy_buffer[];\n");
595 if (m_spec.m_count_chars)
597 m_outstream.writeBytes("\tprivate int yychar;\n");
599 if (m_spec.m_count_lines)
601 m_outstream.writeBytes("\tprivate int yyline;\n");
603 m_outstream.writeBytes("\tprivate int yy_lexical_state;\n");
604 /*if (m_spec.m_count_lines || true == m_spec.m_count_chars)
606 m_outstream.writeBytes("\tprivate int yy_buffer_prev_start;\n");
608 m_outstream.writeBytes("\n");
611 /* Function: first constructor (Reader) */
612 m_outstream.writeBytes("\t");
613 if (true == m_spec.m_public) {
614 m_outstream.writeBytes("public ");
616 m_outstream.writeBytes(new String(m_spec.m_class_name));
617 m_outstream.writeBytes(" (java.io.Reader reader)");
619 if (null != m_spec.m_init_throw_code)
621 m_outstream.writeBytes("\n");
622 m_outstream.writeBytes("\t\tthrows ");
623 m_outstream.writeBytes(new String(m_spec.m_init_throw_code,0,
624 m_spec.m_init_throw_read));
625 m_outstream.writeBytes("\n\t\t{\n");
629 m_outstream.writeBytes(" {\n");
632 m_outstream.writeBytes("\t\tthis ();\n");
633 m_outstream.writeBytes("\t\tif (null == reader) {\n");
634 m_outstream.writeBytes("\t\t\tthrow (new Error(\"Error: Bad input "
635 + "stream initializer.\"));\n");
636 m_outstream.writeBytes("\t\t}\n");
637 m_outstream.writeBytes("\t\tyy_reader = new java.io.BufferedReader(reader);\n");
638 m_outstream.writeBytes("\t}\n\n");
641 /* Function: second constructor (InputStream) */
642 m_outstream.writeBytes("\t");
643 if (true == m_spec.m_public) {
644 m_outstream.writeBytes("public ");
646 m_outstream.writeBytes(new String(m_spec.m_class_name));
647 m_outstream.writeBytes(" (java.io.InputStream instream)");
649 if (null != m_spec.m_init_throw_code)
651 m_outstream.writeBytes("\n");
652 m_outstream.writeBytes("\t\tthrows ");
653 m_outstream.writeBytes(new String(m_spec.m_init_throw_code,0,
654 m_spec.m_init_throw_read));
655 m_outstream.writeBytes("\n\t\t{\n");
659 m_outstream.writeBytes(" {\n");
662 m_outstream.writeBytes("\t\tthis ();\n");
663 m_outstream.writeBytes("\t\tif (null == instream) {\n");
664 m_outstream.writeBytes("\t\t\tthrow (new Error(\"Error: Bad input "
665 + "stream initializer.\"));\n");
666 m_outstream.writeBytes("\t\t}\n");
667 m_outstream.writeBytes("\t\tyy_reader = new java.io.BufferedReader(new java.io.InputStreamReader(instream));\n");
668 m_outstream.writeBytes("\t}\n\n");
671 /* Function: third, private constructor - only for internal use */
672 m_outstream.writeBytes("\tprivate ");
673 m_outstream.writeBytes(new String(m_spec.m_class_name));
674 m_outstream.writeBytes(" ()");
676 if (null != m_spec.m_init_throw_code)
678 m_outstream.writeBytes("\n");
679 m_outstream.writeBytes("\t\tthrows ");
680 m_outstream.writeBytes(new String(m_spec.m_init_throw_code,0,
681 m_spec.m_init_throw_read));
682 m_outstream.writeBytes("\n\t\t{\n");
686 m_outstream.writeBytes(" {\n");
689 m_outstream.writeBytes("\t\tyy_buffer = new char[YY_BUFFER_SIZE];\n");
690 m_outstream.writeBytes("\t\tyy_buffer_read = 0;\n");
691 m_outstream.writeBytes("\t\tyy_buffer_index = 0;\n");
692 m_outstream.writeBytes("\t\tyy_buffer_start = 0;\n");
693 m_outstream.writeBytes("\t\tyy_buffer_end = 0;\n");
694 if (m_spec.m_count_chars)
696 m_outstream.writeBytes("\t\tyychar = 0;\n");
698 if (m_spec.m_count_lines)
700 m_outstream.writeBytes("\t\tyyline = 0;\n");
702 m_outstream.writeBytes("\t\tyy_lexical_state = YYINITIAL;\n");
703 /*if (m_spec.m_count_lines || true == m_spec.m_count_chars)
705 m_outstream.writeBytes("\t\tyy_buffer_prev_start = 0;\n");
708 /* User specified constructor code. */
709 if (null != m_spec.m_init_code)
711 m_outstream.writeBytes(new String(m_spec.m_init_code,0,
712 m_spec.m_init_read));
715 m_outstream.writeBytes("\t}\n\n");
719 /***************************************************************
720 Function: emit_states
721 Description: Emits constants that serve as lexical states,
723 **************************************************************/
724 private void emit_states
727 throws java.io.IOException
733 states = m_spec.m_states.keys();
735 while (states.hasMoreElements())
737 state = (String) states.nextElement();
741 CUtility.assert(null != state);
744 m_outstream.writeBytes("\tprivate final int "
747 + (m_spec.m_states.get(state)).toString()
752 m_outstream.writeBytes("\tprivate final int yy_state_dtrans[] = {\n");
753 for (index = 0; index < m_spec.m_state_dtrans.length; ++index)
755 m_outstream.writeBytes("\t\t" + m_spec.m_state_dtrans[index]);
756 if (index < m_spec.m_state_dtrans.length - 1)
758 m_outstream.writeBytes(",\n");
762 m_outstream.writeBytes("\n");
765 m_outstream.writeBytes("\t};\n");
768 /***************************************************************
769 Function: emit_helpers
770 Description: Emits helper functions, particularly
771 error handling and input buffering.
772 **************************************************************/
773 private void emit_helpers
776 throws java.io.IOException
780 CUtility.assert(null != m_spec);
781 CUtility.assert(null != m_outstream);
784 /* Function: yy_do_eof */
785 m_outstream.writeBytes("\tprivate boolean yy_eof_done = false;\n");
786 if (null != m_spec.m_eof_code)
788 m_outstream.writeBytes("\tprivate void yy_do_eof ()");
790 if (null != m_spec.m_eof_throw_code)
792 m_outstream.writeBytes("\n");
793 m_outstream.writeBytes("\t\tthrows ");
794 m_outstream.writeBytes(new String(m_spec.m_eof_throw_code,0,
795 m_spec.m_eof_throw_read));
796 m_outstream.writeBytes("\n\t\t{\n");
800 m_outstream.writeBytes(" {\n");
803 m_outstream.writeBytes("\t\tif (false == yy_eof_done) {\n");
804 m_outstream.writeBytes(new String(m_spec.m_eof_code,0,
806 m_outstream.writeBytes("\t\t}\n");
807 m_outstream.writeBytes("\t\tyy_eof_done = true;\n");
808 m_outstream.writeBytes("\t}\n");
813 /* Function: yybegin */
814 m_outstream.writeBytes("\tprivate void yybegin (int state) {\n");
815 m_outstream.writeBytes("\t\tyy_lexical_state = state;\n");
816 m_outstream.writeBytes("\t}\n");
818 /* Function: yy_initial_dtrans */
819 /*m_outstream.writeBytes("\tprivate int yy_initial_dtrans (int state) {\n");
820 m_outstream.writeBytes("\t\treturn yy_state_dtrans[state];\n");
821 m_outstream.writeBytes("\t}\n");*/
823 /* Function: yy_advance */
824 m_outstream.writeBytes("\tprivate char yy_advance ()\n");
825 m_outstream.writeBytes("\t\tthrows java.io.IOException {\n");
826 /*m_outstream.writeBytes("\t\t{\n");*/
827 m_outstream.writeBytes("\t\tint next_read;\n");
828 m_outstream.writeBytes("\t\tint i;\n");
829 m_outstream.writeBytes("\t\tint j;\n");
830 m_outstream.writeBytes("\n");
832 m_outstream.writeBytes("\t\tif (yy_buffer_index < yy_buffer_read) {\n");
833 m_outstream.writeBytes("\t\t\treturn yy_buffer[yy_buffer_index++];\n");
834 /*m_outstream.writeBytes("\t\t\t++yy_buffer_index;\n");*/
835 m_outstream.writeBytes("\t\t}\n");
836 m_outstream.writeBytes("\n");
838 m_outstream.writeBytes("\t\tif (0 != yy_buffer_start) {\n");
839 m_outstream.writeBytes("\t\t\ti = yy_buffer_start;\n");
840 m_outstream.writeBytes("\t\t\tj = 0;\n");
841 m_outstream.writeBytes("\t\t\twhile (i < yy_buffer_read) {\n");
842 m_outstream.writeBytes("\t\t\t\tyy_buffer[j] = yy_buffer[i];\n");
843 m_outstream.writeBytes("\t\t\t\t++i;\n");
844 m_outstream.writeBytes("\t\t\t\t++j;\n");
845 m_outstream.writeBytes("\t\t\t}\n");
846 m_outstream.writeBytes("\t\t\tyy_buffer_end = yy_buffer_end - yy_buffer_start;\n");
847 m_outstream.writeBytes("\t\t\tyy_buffer_start = 0;\n");
848 m_outstream.writeBytes("\t\t\tyy_buffer_read = j;\n");
849 m_outstream.writeBytes("\t\t\tyy_buffer_index = j;\n");
850 m_outstream.writeBytes("\t\t\tnext_read = yy_reader.read(yy_buffer,\n");
851 m_outstream.writeBytes("\t\t\t\t\tyy_buffer_read,\n");
852 m_outstream.writeBytes("\t\t\t\t\tyy_buffer.length - yy_buffer_read);\n");
853 m_outstream.writeBytes("\t\t\tif (-1 == next_read) {\n");
854 m_outstream.writeBytes("\t\t\t\treturn YY_EOF;\n");
855 m_outstream.writeBytes("\t\t\t}\n");
856 m_outstream.writeBytes("\t\t\tyy_buffer_read = yy_buffer_read + next_read;\n");
857 m_outstream.writeBytes("\t\t}\n");
858 m_outstream.writeBytes("\n");
860 m_outstream.writeBytes("\t\twhile (yy_buffer_index >= yy_buffer_read) {\n");
861 m_outstream.writeBytes("\t\t\tif (yy_buffer_index >= yy_buffer.length) {\n");
862 m_outstream.writeBytes("\t\t\t\tyy_buffer = yy_double(yy_buffer);\n");
863 m_outstream.writeBytes("\t\t\t}\n");
864 m_outstream.writeBytes("\t\t\tnext_read = yy_reader.read(yy_buffer,\n");
865 m_outstream.writeBytes("\t\t\t\t\tyy_buffer_read,\n");
866 m_outstream.writeBytes("\t\t\t\t\tyy_buffer.length - yy_buffer_read);\n");
867 m_outstream.writeBytes("\t\t\tif (-1 == next_read) {\n");
868 m_outstream.writeBytes("\t\t\t\treturn YY_EOF;\n");
869 m_outstream.writeBytes("\t\t\t}\n");
870 m_outstream.writeBytes("\t\t\tyy_buffer_read = yy_buffer_read + next_read;\n");
871 m_outstream.writeBytes("\t\t}\n");
873 m_outstream.writeBytes("\t\treturn yy_buffer[yy_buffer_index++];\n");
874 m_outstream.writeBytes("\t}\n");
876 /* Function: yy_move_start */
877 m_outstream.writeBytes("\tprivate void yy_move_start () {\n");
878 if (m_spec.m_count_lines)
882 m_outstream.writeBytes("\t\tif ((byte) '\\n' "
883 + "== yy_buffer[yy_buffer_start]) {\n");
887 m_outstream.writeBytes("\t\tif ((byte) '\\n' "
888 + "== yy_buffer[yy_buffer_start]\n");
889 m_outstream.writeBytes("\t\t\t|| (byte) '\\r' "
890 + "== yy_buffer[yy_buffer_start]) {\n");
892 m_outstream.writeBytes("\t\t\t++yyline;\n");
893 m_outstream.writeBytes("\t\t}\n");
895 if (m_spec.m_count_chars)
897 m_outstream.writeBytes("\t\t++yychar;\n");
899 m_outstream.writeBytes("\t\t++yy_buffer_start;\n");
900 m_outstream.writeBytes("\t}\n");
902 /* Function: yy_pushback */
903 m_outstream.writeBytes("\tprivate void yy_pushback () {\n");
904 m_outstream.writeBytes("\t\t--yy_buffer_end;\n");
905 m_outstream.writeBytes("\t}\n");
907 /* Function: yy_mark_start */
908 m_outstream.writeBytes("\tprivate void yy_mark_start () {\n");
909 if (m_spec.m_count_lines || true == m_spec.m_count_chars)
911 if (m_spec.m_count_lines)
913 m_outstream.writeBytes("\t\tint i;\n");
914 m_outstream.writeBytes("\t\tfor (i = yy_buffer_start; "
915 + "i < yy_buffer_index; ++i) {\n");
918 m_outstream.writeBytes("\t\t\tif ((byte) '\\n' == yy_buffer[i]) {\n");
922 m_outstream.writeBytes("\t\t\tif ((byte) '\\n' == yy_buffer[i] || "
923 + "(byte) '\\r' == yy_buffer[i]) {\n");
925 m_outstream.writeBytes("\t\t\t\t++yyline;\n");
926 m_outstream.writeBytes("\t\t\t}\n");
927 m_outstream.writeBytes("\t\t}\n");
929 if (m_spec.m_count_chars)
931 m_outstream.writeBytes("\t\tyychar = yychar\n");
932 m_outstream.writeBytes("\t\t\t+ yy_buffer_index - yy_buffer_start;\n");
935 m_outstream.writeBytes("\t\tyy_buffer_start = yy_buffer_index;\n");
936 m_outstream.writeBytes("\t}\n");
938 /* Function: yy_mark_end */
939 m_outstream.writeBytes("\tprivate void yy_mark_end () {\n");
940 m_outstream.writeBytes("\t\tyy_buffer_end = yy_buffer_index;\n");
941 m_outstream.writeBytes("\t}\n");
943 /* Function: yy_to_mark */
944 m_outstream.writeBytes("\tprivate void yy_to_mark () {\n");
945 m_outstream.writeBytes("\t\tyy_buffer_index = yy_buffer_end;\n");
946 m_outstream.writeBytes("\t}\n");
948 /* Function: yytext */
949 m_outstream.writeBytes("\tprivate java.lang.String yytext () {\n");
950 m_outstream.writeBytes("\t\treturn (new java.lang.String(yy_buffer,\n");
951 m_outstream.writeBytes("\t\t\tyy_buffer_start,\n");
952 m_outstream.writeBytes("\t\t\tyy_buffer_end - yy_buffer_start));\n");
953 m_outstream.writeBytes("\t}\n");
955 /* Function: yylength */
956 m_outstream.writeBytes("\tprivate int yylength () {\n");
957 m_outstream.writeBytes("\t\treturn yy_buffer_end - yy_buffer_start;\n");
958 m_outstream.writeBytes("\t}\n");
960 /* Function: yy_double */
961 m_outstream.writeBytes("\tprivate char[] yy_double (char buf[]) {\n");
962 m_outstream.writeBytes("\t\tint i;\n\t\tchar newbuf[];\n");
963 m_outstream.writeBytes("\t\tnewbuf = new char[2*buf.length];\n");
964 m_outstream.writeBytes("\t\tfor (i = 0; i < buf.length; ++i) {\n");
965 m_outstream.writeBytes("\t\t\tnewbuf[i] = buf[i];\n");
966 m_outstream.writeBytes("\t\t}\n");
967 m_outstream.writeBytes("\t\treturn newbuf;\n");
968 m_outstream.writeBytes("\t}\n");
970 /* Function: yy_error */
971 m_outstream.writeBytes("\tprivate final int YY_E_INTERNAL = 0;\n");
972 m_outstream.writeBytes("\tprivate final int YY_E_MATCH = 1;\n");
973 m_outstream.writeBytes("\tprivate java.lang.String yy_error_string[] = {\n");
974 m_outstream.writeBytes("\t\t\"Error: Internal error.\\n\",\n");
975 m_outstream.writeBytes("\t\t\"Error: Unmatched input.\\n\"\n");
976 m_outstream.writeBytes("\t};\n");
977 m_outstream.writeBytes("\tprivate void yy_error (int code,boolean fatal) {\n");
978 m_outstream.writeBytes("\t\tjava.lang.System.out.print(yy_error_string[code]);\n");
979 m_outstream.writeBytes("\t\tjava.lang.System.out.flush();\n");
980 m_outstream.writeBytes("\t\tif (fatal) {\n");
981 m_outstream.writeBytes("\t\t\tthrow new Error(\"Fatal Error.\\n\");\n");
982 m_outstream.writeBytes("\t\t}\n");
983 m_outstream.writeBytes("\t}\n");
985 /* Function: yy_next */
986 /*m_outstream.writeBytes("\tprivate int yy_next (int current,char lookahead) {\n");
987 m_outstream.writeBytes("\t\treturn yy_nxt[yy_rmap[current]][yy_cmap[lookahead]];\n");
988 m_outstream.writeBytes("\t}\n");*/
990 /* Function: yy_accept */
991 /*m_outstream.writeBytes("\tprivate int yy_accept (int current) {\n");
992 m_outstream.writeBytes("\t\treturn yy_acpt[current];\n");
993 m_outstream.writeBytes("\t}\n");*/
996 // Function: private int [][] unpackFromString(int size1, int size2, String st)
997 // Added 6/24/98 Raimondas Lencevicius
998 // May be made more efficient by replacing String operations
999 // Assumes correctly formed input String. Performs no error checking
1000 m_outstream.writeBytes("private int [][] unpackFromString(int size1, int size2, String st)\n");
1001 m_outstream.writeBytes(" {\n");
1002 m_outstream.writeBytes(" int colonIndex = -1;\n");
1003 m_outstream.writeBytes(" String lengthString;\n");
1004 m_outstream.writeBytes(" int sequenceLength = 0;\n");
1005 m_outstream.writeBytes(" int sequenceInteger = 0;\n");
1007 m_outstream.writeBytes(" int commaIndex;\n");
1008 m_outstream.writeBytes(" String workString;\n");
1010 m_outstream.writeBytes(" int res[][] = new int[size1][size2];\n");
1011 m_outstream.writeBytes(" for (int i= 0; i < size1; i++)\n");
1012 m_outstream.writeBytes(" for (int j= 0; j < size2; j++)\n");
1013 m_outstream.writeBytes(" {\n");
1014 m_outstream.writeBytes(" if (sequenceLength == 0) \n");
1015 m_outstream.writeBytes(" { \n");
1016 m_outstream.writeBytes(" commaIndex = st.indexOf(',');\n");
1017 m_outstream.writeBytes(" if (commaIndex == -1)\n");
1018 m_outstream.writeBytes(" workString = st;\n");
1019 m_outstream.writeBytes(" else\n");
1020 m_outstream.writeBytes(" workString = st.substring(0, commaIndex);\n");
1021 m_outstream.writeBytes(" st = st.substring(commaIndex+1);\n");
1022 m_outstream.writeBytes(" colonIndex = workString.indexOf(':');\n");
1023 m_outstream.writeBytes(" if (colonIndex == -1)\n");
1024 m_outstream.writeBytes(" {\n");
1025 m_outstream.writeBytes(" res[i][j] = Integer.parseInt(workString);\n");
1026 m_outstream.writeBytes(" }\n");
1027 m_outstream.writeBytes(" else \n");
1028 m_outstream.writeBytes(" {\n");
1029 m_outstream.writeBytes(" lengthString = workString.substring(colonIndex+1); \n");
1030 m_outstream.writeBytes(" sequenceLength = Integer.parseInt(lengthString);\n");
1031 m_outstream.writeBytes(" workString = workString.substring(0,colonIndex);\n");
1032 m_outstream.writeBytes(" sequenceInteger = Integer.parseInt(workString);\n");
1033 m_outstream.writeBytes(" res[i][j] = sequenceInteger;\n");
1034 m_outstream.writeBytes(" sequenceLength--;\n");
1035 m_outstream.writeBytes(" }\n");
1036 m_outstream.writeBytes(" }\n");
1037 m_outstream.writeBytes(" else \n");
1038 m_outstream.writeBytes(" {\n");
1039 m_outstream.writeBytes(" res[i][j] = sequenceInteger;\n");
1040 m_outstream.writeBytes(" sequenceLength--;\n");
1041 m_outstream.writeBytes(" }\n");
1042 m_outstream.writeBytes(" }\n");
1043 m_outstream.writeBytes(" return res;\n");
1044 m_outstream.writeBytes(" }\n");
1050 /***************************************************************
1051 Function: emit_header
1052 Description: Emits class header.
1053 **************************************************************/
1054 private void emit_header
1057 throws java.io.IOException
1061 CUtility.assert(null != m_spec);
1062 CUtility.assert(null != m_outstream);
1065 m_outstream.writeBytes("\n\n");
1066 if (true == m_spec.m_public) {
1067 m_outstream.writeBytes("public ");
1069 m_outstream.writeBytes("class ");
1070 m_outstream.writeBytes(new String(m_spec.m_class_name,0,
1071 m_spec.m_class_name.length));
1072 if (m_spec.m_implements_name.length > 0) {
1073 m_outstream.writeBytes(" implements ");
1074 m_outstream.writeBytes(new String(m_spec.m_implements_name,0,
1075 m_spec.m_implements_name.length));
1077 m_outstream.writeBytes(" {\n");
1080 /***************************************************************
1081 Function: emit_table
1082 Description: Emits transition table.
1083 **************************************************************/
1084 private void emit_table
1087 throws java.io.IOException
1099 CUtility.assert(null != m_spec);
1100 CUtility.assert(null != m_outstream);
1103 m_outstream.writeBytes("\tprivate int yy_acpt[] = {\n");
1104 size = m_spec.m_accept_vector.size();
1105 for (elem = 0; elem < size; ++elem)
1107 accept = (CAccept) m_spec.m_accept_vector.elementAt(elem);
1111 is_start = (0 != (m_spec.m_anchor_array[elem] & CSpec.START));
1112 is_end = (0 != (m_spec.m_anchor_array[elem] & CSpec.END));
1114 if (is_start && true == is_end)
1116 m_outstream.writeBytes("\t\tYY_START | YY_END");
1120 m_outstream.writeBytes("\t\tYY_START");
1124 m_outstream.writeBytes("\t\tYY_END");
1128 m_outstream.writeBytes("\t\tYY_NO_ANCHOR");
1133 m_outstream.writeBytes("\t\tYY_NOT_ACCEPT");
1136 if (elem < size - 1)
1138 m_outstream.writeBytes(",");
1141 m_outstream.writeBytes("\n");
1143 m_outstream.writeBytes("\t};\n");
1145 m_outstream.writeBytes("\tprivate int yy_cmap[] = {\n\t\t");
1146 for (i = 0; i < m_spec.m_col_map.length; ++i)
1148 m_outstream.writeBytes((new Integer(m_spec.m_col_map[i])).toString());
1150 if (i < m_spec.m_col_map.length - 1)
1152 m_outstream.writeBytes(",");
1155 if (0 == ((i + 1) % 8))
1157 m_outstream.writeBytes("\n\t\t");
1161 m_outstream.writeBytes(" ");
1164 m_outstream.writeBytes("\n\t};\n");
1166 m_outstream.writeBytes("\tprivate int yy_rmap[] = {\n\t\t");
1167 for (i = 0; i < m_spec.m_row_map.length; ++i)
1169 m_outstream.writeBytes((new Integer(m_spec.m_row_map[i])).toString());
1171 if (i < m_spec.m_row_map.length - 1)
1173 m_outstream.writeBytes(",");
1176 if (0 == ((i + 1) % 8))
1178 m_outstream.writeBytes("\n\t\t");
1182 m_outstream.writeBytes(" ");
1185 m_outstream.writeBytes("\n\t};\n");
1187 // 6/24/98 Raimondas Lencevicius
1188 // Code to the method end modified
1189 /* yy_nxt[][] is assigned result of
1190 "int [][] unpackFromString(int size1, int size2, String st)".
1191 yy_nxt[][] values are coded into a string
1192 by printing integers and representing
1193 integer sequences as "value:length" pairs. */
1195 m_outstream.writeBytes
1196 ("\tprivate int yy_nxt[][] = unpackFromString(");
1197 size = m_spec.m_dtrans_vector.size();
1198 int sequenceLength = 0; // RL - length of the number sequence
1199 boolean sequenceStarted = false; // RL - has number sequence started?
1200 int previousInt = -20; // RL - Bogus -20 state.
1202 // RL - Output matrix size
1203 m_outstream.writeBytes((new Integer(size)).toString());
1204 m_outstream.writeBytes(",");
1205 m_outstream.writeBytes((new Integer(m_spec.m_dtrans_ncols)).toString());
1206 m_outstream.writeBytes(",");
1207 m_outstream.writeBytes("\n\"");
1209 // RL - Output matrix
1210 for (elem = 0; elem < size; ++elem)
1213 dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(elem);
1215 for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
1217 int writeInt = dtrans.m_dtrans[i];
1218 if (writeInt == previousInt) // RL - sequence?
1220 if (sequenceStarted)
1226 m_outstream.writeBytes((new Integer(writeInt)).toString());
1227 m_outstream.writeBytes(":");
1229 sequenceStarted = true;
1232 else // RL - no sequence or end sequence
1234 if (sequenceStarted)
1236 m_outstream.writeBytes((new Integer(sequenceLength)).toString());
1237 m_outstream.writeBytes(",");
1239 sequenceStarted = false;
1243 if (previousInt != -20)
1245 m_outstream.writeBytes((new Integer(previousInt)).toString());
1246 m_outstream.writeBytes(",");
1250 previousInt = writeInt;
1253 if (sequenceStarted)
1255 m_outstream.writeBytes((new Integer(sequenceLength)).toString());
1259 m_outstream.writeBytes((new Integer(previousInt)).toString());
1261 m_outstream.writeBytes("\");\n");
1264 /***************************************************************
1265 Function: emit_driver
1267 **************************************************************/
1268 private void emit_driver
1271 throws java.io.IOException
1275 CUtility.assert(null != m_spec);
1276 CUtility.assert(null != m_outstream);
1281 if (m_spec.m_integer_type)
1283 m_outstream.writeBytes("\tpublic int ");
1284 m_outstream.writeBytes(new String(m_spec.m_function_name));
1285 m_outstream.writeBytes(" ()\n");
1287 else if (m_spec.m_intwrap_type)
1289 m_outstream.writeBytes("\tpublic java.lang.Integer ");
1290 m_outstream.writeBytes(new String(m_spec.m_function_name));
1291 m_outstream.writeBytes(" ()\n");
1295 m_outstream.writeBytes("\tpublic ");
1296 m_outstream.writeBytes(new String(m_spec.m_type_name));
1297 m_outstream.writeBytes(" ");
1298 m_outstream.writeBytes(new String(m_spec.m_function_name));
1299 m_outstream.writeBytes(" ()\n");
1302 /*m_outstream.writeBytes("\t\tthrows java.io.IOException {\n");*/
1303 m_outstream.writeBytes("\t\tthrows java.io.IOException");
1304 if (null != m_spec.m_yylex_throw_code)
1306 m_outstream.writeBytes(", ");
1307 m_outstream.writeBytes(new String(m_spec.m_yylex_throw_code,0,
1308 m_spec.m_yylex_throw_read));
1309 m_outstream.writeBytes("\n\t\t{\n");
1313 m_outstream.writeBytes(" {\n");
1316 m_outstream.writeBytes("\t\tchar yy_lookahead;\n");
1317 m_outstream.writeBytes("\t\tint yy_anchor = YY_NO_ANCHOR;\n");
1318 /*m_outstream.writeBytes("\t\tint yy_state "
1319 + "= yy_initial_dtrans(yy_lexical_state);\n");*/
1320 m_outstream.writeBytes("\t\tint yy_state "
1321 + "= yy_state_dtrans[yy_lexical_state];\n");
1322 m_outstream.writeBytes("\t\tint yy_next_state = YY_NO_STATE;\n");
1323 /*m_outstream.writeBytes("\t\tint yy_prev_stave = YY_NO_STATE;\n");*/
1324 m_outstream.writeBytes("\t\tint yy_last_accept_state = YY_NO_STATE;\n");
1325 m_outstream.writeBytes("\t\tboolean yy_initial = true;\n");
1326 m_outstream.writeBytes("\t\tint yy_this_accept;\n");
1327 m_outstream.writeBytes("\n");
1329 m_outstream.writeBytes("\t\tyy_mark_start();\n");
1330 /*m_outstream.writeBytes("\t\tyy_this_accept = yy_accept(yy_state);\n");*/
1331 m_outstream.writeBytes("\t\tyy_this_accept = yy_acpt[yy_state];\n");
1332 m_outstream.writeBytes("\t\tif (YY_NOT_ACCEPT != yy_this_accept) {\n");
1333 m_outstream.writeBytes("\t\t\tyy_last_accept_state = yy_state;\n");
1334 m_outstream.writeBytes("\t\t\tyy_mark_end();\n");
1335 m_outstream.writeBytes("\t\t}\n");
1339 m_outstream.writeBytes("\t\tjava.lang.System.out.println(\"Begin\");\n");
1342 m_outstream.writeBytes("\t\twhile (true) {\n");
1344 m_outstream.writeBytes("\t\t\tyy_lookahead = yy_advance();\n");
1345 m_outstream.writeBytes("\t\t\tyy_next_state = YY_F;\n");
1346 m_outstream.writeBytes("\t\t\tif (YY_EOF != yy_lookahead) {\n");
1347 /*m_outstream.writeBytes("\t\t\t\tyy_next_state = "
1348 + "yy_next(yy_state,yy_lookahead);\n");*/
1349 m_outstream.writeBytes("\t\t\t\tyy_next_state = "
1350 + "yy_nxt[yy_rmap[yy_state]][yy_cmap[yy_lookahead]];\n");
1352 m_outstream.writeBytes("\t\t\t}\n");
1356 m_outstream.writeBytes("java.lang.System.out.println(\"Current state: \""
1358 m_outstream.writeBytes("+ \"\tCurrent input: \"\n");
1359 m_outstream.writeBytes(" + ((char) yy_lookahead));\n");
1363 m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"State = \""
1364 + "+ yy_state);\n");
1365 m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"Accepting status = \""
1366 + "+ yy_this_accept);\n");
1367 m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"Last accepting state = \""
1368 + "+ yy_last_accept_state);\n");
1369 m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"Next state = \""
1370 + "+ yy_next_state);\n");
1371 m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"Lookahead input = \""
1372 + "+ ((char) yy_lookahead));\n");
1375 m_outstream.writeBytes("\t\t\tif (YY_F != yy_next_state) {\n");
1376 m_outstream.writeBytes("\t\t\t\tyy_state = yy_next_state;\n");
1377 m_outstream.writeBytes("\t\t\t\tyy_initial = false;\n");
1378 /*m_outstream.writeBytes("\t\t\t\tyy_this_accept = yy_accept(yy_state);\n");*/
1379 m_outstream.writeBytes("\t\t\t\tyy_this_accept = yy_acpt[yy_state];\n");
1380 m_outstream.writeBytes("\t\t\t\tif (YY_NOT_ACCEPT != yy_this_accept) {\n");
1381 m_outstream.writeBytes("\t\t\t\t\tyy_last_accept_state = yy_state;\n");
1382 m_outstream.writeBytes("\t\t\t\t\tyy_mark_end();\n");
1383 m_outstream.writeBytes("\t\t\t\t}\n");
1384 /*m_outstream.writeBytes("\t\t\t\tyy_prev_state = yy_state;\n");*/
1385 /*m_outstream.writeBytes("\t\t\t\tyy_state = yy_next_state;\n");*/
1386 m_outstream.writeBytes("\t\t\t}\n");
1388 m_outstream.writeBytes("\t\t\telse {\n");
1389 m_outstream.writeBytes("\t\t\t\tif (YY_EOF == yy_lookahead "
1390 + "&& true == yy_initial) {\n");
1391 if (null != m_spec.m_eof_code)
1393 m_outstream.writeBytes("\t\t\t\t\tyy_do_eof();\n");
1396 if (m_spec.m_integer_type || true == m_spec.m_yyeof)
1398 m_outstream.writeBytes("\t\t\t\t\treturn YYEOF;\n");
1400 else if (null != m_spec.m_eof_value_code)
1402 m_outstream.writeBytes(new String(m_spec.m_eof_value_code,0,
1403 m_spec.m_eof_value_read));
1407 m_outstream.writeBytes("\t\t\t\t\treturn null;\n");
1410 m_outstream.writeBytes("\t\t\t\t}\n");
1412 m_outstream.writeBytes("\t\t\t\telse if (YY_NO_STATE == yy_last_accept_state) {\n");
1415 /*m_outstream.writeBytes("\t\t\t\t\tyy_error(YY_E_MATCH,false);\n");
1416 m_outstream.writeBytes("\t\t\t\t\tyy_initial = true;\n");
1417 m_outstream.writeBytes("\t\t\t\t\tyy_state "
1418 + "= yy_state_dtrans[yy_lexical_state];\n");
1419 m_outstream.writeBytes("\t\t\t\t\tyy_next_state = YY_NO_STATE;\n");*/
1420 /*m_outstream.writeBytes("\t\t\t\t\tyy_prev_state = YY_NO_STATE;\n");*/
1421 /*m_outstream.writeBytes("\t\t\t\t\tyy_last_accept_state = YY_NO_STATE;\n");
1422 m_outstream.writeBytes("\t\t\t\t\tyy_mark_start();\n");*/
1423 /*m_outstream.writeBytes("\t\t\t\t\tyy_this_accept = yy_accept(yy_state);\n");*/
1424 /*m_outstream.writeBytes("\t\t\t\t\tyy_this_accept = yy_acpt[yy_state];\n");
1425 m_outstream.writeBytes("\t\t\t\t\tif (YY_NOT_ACCEPT != yy_this_accept) {\n");
1426 m_outstream.writeBytes("\t\t\t\t\t\tyy_last_accept_state = yy_state;\n");
1427 m_outstream.writeBytes("\t\t\t\t\t}\n");*/
1429 m_outstream.writeBytes("\t\t\t\t\tthrow (new Error(\"Lexical Error: Unmatched Input.\"));\n");
1430 m_outstream.writeBytes("\t\t\t\t}\n");
1432 m_outstream.writeBytes("\t\t\t\telse {\n");
1433 m_outstream.writeBytes("\t\t\t\t\tyy_to_mark();\n");
1435 m_outstream.writeBytes("\t\t\t\t\tyy_anchor = yy_acpt[yy_last_accept_state];\n");
1436 /*m_outstream.writeBytes("\t\t\t\t\tyy_anchor "
1437 + "= yy_accept(yy_last_accept_state);\n");*/
1438 m_outstream.writeBytes("\t\t\t\t\tif (0 != (YY_END & yy_anchor)) {\n");
1439 m_outstream.writeBytes("\t\t\t\t\t\tyy_pushback();\n");
1440 m_outstream.writeBytes("\t\t\t\t\t}\n");
1441 m_outstream.writeBytes("\t\t\t\t\tif (0 != (YY_START & yy_anchor)) {\n");
1442 m_outstream.writeBytes("\t\t\t\t\t\tyy_move_start();\n");
1443 m_outstream.writeBytes("\t\t\t\t\t}\n");
1446 m_outstream.writeBytes("\t\t\t\t\tswitch (yy_last_accept_state) {\n");
1448 emit_actions("\t\t\t\t\t");
1450 m_outstream.writeBytes("\t\t\t\t\tdefault:\n");
1451 m_outstream.writeBytes("\t\t\t\t\t\tyy_error(YY_E_INTERNAL,false);\n");
1452 /*m_outstream.writeBytes("\t\t\t\t\t\treturn null;\n");*/
1453 m_outstream.writeBytes("\t\t\t\t\tcase -1:\n");
1454 m_outstream.writeBytes("\t\t\t\t\t}\n");
1456 m_outstream.writeBytes("\t\t\t\t\tyy_initial = true;\n");
1457 m_outstream.writeBytes("\t\t\t\t\tyy_state "
1458 + "= yy_state_dtrans[yy_lexical_state];\n");
1459 m_outstream.writeBytes("\t\t\t\t\tyy_next_state = YY_NO_STATE;\n");
1460 /*m_outstream.writeBytes("\t\t\t\t\tyy_prev_state = YY_NO_STATE;\n");*/
1461 m_outstream.writeBytes("\t\t\t\t\tyy_last_accept_state = YY_NO_STATE;\n");
1463 m_outstream.writeBytes("\t\t\t\t\tyy_mark_start();\n");
1465 /*m_outstream.writeBytes("\t\t\t\t\tyy_this_accept = yy_accept(yy_state);\n");*/
1466 m_outstream.writeBytes("\t\t\t\t\tyy_this_accept = yy_acpt[yy_state];\n");
1467 m_outstream.writeBytes("\t\t\t\t\tif (YY_NOT_ACCEPT != yy_this_accept) {\n");
1468 m_outstream.writeBytes("\t\t\t\t\t\tyy_last_accept_state = yy_state;\n");
1469 m_outstream.writeBytes("\t\t\t\t\t}\n");
1471 m_outstream.writeBytes("\t\t\t\t}\n");
1472 m_outstream.writeBytes("\t\t\t}\n");
1473 m_outstream.writeBytes("\t\t}\n");
1474 m_outstream.writeBytes("\t}\n");
1476 /*m_outstream.writeBytes("\t\t\t\t\n");
1477 m_outstream.writeBytes("\t\t\t\n");
1478 m_outstream.writeBytes("\t\t\t\n");
1479 m_outstream.writeBytes("\t\t\t\n");
1480 m_outstream.writeBytes("\t\t\t\n");
1481 m_outstream.writeBytes("\t\t}\n");*/
1484 /***************************************************************
1485 Function: emit_actions
1487 **************************************************************/
1488 private void emit_actions
1492 throws java.io.IOException
1501 CUtility.assert(m_spec.m_accept_vector.size()
1502 == m_spec.m_anchor_array.length);
1506 size = m_spec.m_accept_vector.size();
1507 for (elem = 0; elem < size; ++elem)
1509 accept = (CAccept) m_spec.m_accept_vector.elementAt(elem);
1512 m_outstream.writeBytes(tabs + "case " + elem
1513 /*+ (new Integer(elem)).toString()*/
1515 m_outstream.writeBytes(tabs + "\t");
1516 m_outstream.writeBytes(new String(accept.m_action,0,
1517 accept.m_action_read));
1518 m_outstream.writeBytes("\n");
1519 m_outstream.writeBytes(tabs + "case " + bogus_index + ":\n");
1520 m_outstream.writeBytes(tabs + "\tbreak;\n");
1526 /***************************************************************
1527 Function: emit_footer
1529 **************************************************************/
1530 private void emit_footer
1533 throws java.io.IOException
1537 CUtility.assert(null != m_spec);
1538 CUtility.assert(null != m_outstream);
1541 m_outstream.writeBytes("}\n");
1545 /***************************************************************
1547 **************************************************************/
1550 /***************************************************************
1552 **************************************************************/
1553 Vector m_nfa_set; /* Vector of CNfa states in dfa state. */
1554 JavaLexBitSet m_nfa_bit; /* BitSet representation of CNfa labels. */
1555 CAccept m_accept; /* Accepting actions, or null if nonaccepting state. */
1556 int m_anchor; /* Anchors on regular expression. */
1557 int m_accept_index; /* CNfa index corresponding to accepting actions. */
1559 /***************************************************************
1561 Description: Constructor.
1562 **************************************************************/
1570 m_anchor = CSpec.NONE;
1571 m_accept_index = -1;
1575 /***************************************************************
1577 **************************************************************/
1580 /***************************************************************
1582 **************************************************************/
1583 private CSpec m_spec;
1584 private CLexGen m_lexGen;
1585 private CInput m_input;
1587 /***************************************************************
1589 Description: Constructor.
1590 **************************************************************/
1598 /***************************************************************
1600 Description: Resets CMakeNfa member variables.
1601 **************************************************************/
1611 /***************************************************************
1613 Description: Sets CMakeNfa member variables.
1614 **************************************************************/
1624 CUtility.assert(null != input);
1625 CUtility.assert(null != lexGen);
1626 CUtility.assert(null != spec);
1634 /***************************************************************
1636 Description: High level access function to module.
1637 Deposits result in input CSpec.
1638 **************************************************************/
1645 throws java.io.IOException
1651 /* Set member variables. */
1653 set(lexGen,spec,input);
1655 size = m_spec.m_states.size();
1656 m_spec.m_state_rules = new Vector[size];
1657 for (i = 0; i < size; ++i)
1659 m_spec.m_state_rules[i] = new Vector();
1662 /* Initialize current token variable
1664 /*m_spec.m_current_token = m_lexGen.EOS;
1665 m_lexGen.advance();*/
1667 m_spec.m_nfa_start = machine();
1669 /* Set labels in created nfa machine. */
1670 size = m_spec.m_nfa_states.size();
1671 for (i = 0; i < size; ++i)
1673 elem = (CNfa) m_spec.m_nfa_states.elementAt(i);
1677 /* Debugging output. */
1678 if (CUtility.DO_DEBUG)
1680 m_lexGen.print_nfa();
1683 if (m_spec.m_verbose)
1685 System.out.println("NFA comprised of "
1686 + (m_spec.m_nfa_states.size() + 1)
1693 /***************************************************************
1694 Function: discardCNfa
1696 **************************************************************/
1697 private void discardCNfa
1702 m_spec.m_nfa_states.removeElement(nfa);
1705 /***************************************************************
1706 Function: processStates
1708 **************************************************************/
1709 private void processStates
1711 JavaLexBitSet states,
1718 size = m_spec.m_states.size();
1719 for (i = 0; i < size; ++i)
1723 m_spec.m_state_rules[i].addElement(current);
1728 /***************************************************************
1730 Description: Recursive descent regular expression parser.
1731 **************************************************************/
1732 private CNfa machine
1735 throws java.io.IOException
1739 JavaLexBitSet states;
1741 if (CUtility.DESCENT_DEBUG)
1743 CUtility.enter("machine",m_spec.m_lexeme,m_spec.m_current_token);
1746 states = m_lexGen.getStates();
1748 /* Begin: Added for states. */
1749 m_spec.m_current_token = m_lexGen.EOS;
1751 /* End: Added for states. */
1753 start = CAlloc.newCNfa(m_spec);
1757 processStates(states,p.m_next);
1759 while (m_lexGen.END_OF_INPUT != m_spec.m_current_token)
1761 /* Make state changes HERE. */
1762 states = m_lexGen.getStates();
1764 /* Begin: Added for states. */
1766 if (m_lexGen.END_OF_INPUT == m_spec.m_current_token)
1770 /* End: Added for states. */
1772 p.m_next2 = CAlloc.newCNfa(m_spec);
1776 processStates(states,p.m_next);
1779 if (CUtility.DESCENT_DEBUG)
1781 CUtility.leave("machine",m_spec.m_lexeme,m_spec.m_current_token);
1787 /***************************************************************
1789 Description: Recursive descent regular expression parser.
1790 **************************************************************/
1794 throws java.io.IOException
1800 int anchor = CSpec.NONE;
1802 if (CUtility.DESCENT_DEBUG)
1804 CUtility.enter("rule",m_spec.m_lexeme,m_spec.m_current_token);
1807 pair = CAlloc.newCNfaPair();
1809 if (m_lexGen.AT_BOL == m_spec.m_current_token)
1811 start = CAlloc.newCNfa(m_spec);
1812 start.m_edge = '\n';
1813 anchor = anchor | CSpec.START;
1817 start.m_next = pair.m_start;
1823 start = pair.m_start;
1827 if (m_lexGen.AT_EOL == m_spec.m_current_token)
1830 end.m_next = CAlloc.newCNfa(m_spec);
1832 /* This is the way he does it. */
1833 end.m_edge = CNfa.CCL;
1834 end.m_set = new CSet();
1836 end.m_set.add('\n');
1838 if (false == m_spec.m_unix)
1840 end.m_set.add('\r');
1844 anchor = anchor | CSpec.END;
1847 /* Handle end of regular expression. See page 103. */
1848 end.m_accept = m_lexGen.packAccept();
1849 end.m_anchor = anchor;
1851 /* Begin: Removed for states. */
1852 /*m_lexGen.advance();*/
1853 /* End: Removed for states. */
1855 if (CUtility.DESCENT_DEBUG)
1857 CUtility.leave("rule",m_spec.m_lexeme,m_spec.m_current_token);
1863 /***************************************************************
1865 Description: Recursive descent regular expression parser.
1866 **************************************************************/
1871 throws java.io.IOException
1876 if (CUtility.DESCENT_DEBUG)
1878 CUtility.enter("expr",m_spec.m_lexeme,m_spec.m_current_token);
1883 CUtility.assert(null != pair);
1886 e2_pair = CAlloc.newCNfaPair();
1890 while (m_lexGen.OR == m_spec.m_current_token)
1895 p = CAlloc.newCNfa(m_spec);
1896 p.m_next2 = e2_pair.m_start;
1897 p.m_next = pair.m_start;
1900 p = CAlloc.newCNfa(m_spec);
1901 pair.m_end.m_next = p;
1902 e2_pair.m_end.m_next = p;
1906 if (CUtility.DESCENT_DEBUG)
1908 CUtility.leave("expr",m_spec.m_lexeme,m_spec.m_current_token);
1912 /***************************************************************
1914 Description: Recursive descent regular expression parser.
1915 **************************************************************/
1916 private void cat_expr
1920 throws java.io.IOException
1924 if (CUtility.DESCENT_DEBUG)
1926 CUtility.enter("cat_expr",m_spec.m_lexeme,m_spec.m_current_token);
1931 CUtility.assert(null != pair);
1934 e2_pair = CAlloc.newCNfaPair();
1936 if (first_in_cat(m_spec.m_current_token))
1941 while (first_in_cat(m_spec.m_current_token))
1946 pair.m_end.mimic(e2_pair.m_start);
1947 discardCNfa(e2_pair.m_start);
1949 pair.m_end = e2_pair.m_end;
1952 if (CUtility.DESCENT_DEBUG)
1954 CUtility.leave("cat_expr",m_spec.m_lexeme,m_spec.m_current_token);
1958 /***************************************************************
1959 Function: first_in_cat
1960 Description: Recursive descent regular expression parser.
1961 **************************************************************/
1962 private boolean first_in_cat
1969 case m_lexGen.CLOSE_PAREN:
1970 case m_lexGen.AT_EOL:
1975 case m_lexGen.CLOSURE:
1976 case m_lexGen.PLUS_CLOSE:
1977 case m_lexGen.OPTIONAL:
1978 CError.parse_error(CError.E_CLOSE,m_input.m_line_number);
1981 case m_lexGen.CCL_END:
1982 CError.parse_error(CError.E_BRACKET,m_input.m_line_number);
1985 case m_lexGen.AT_BOL:
1986 CError.parse_error(CError.E_BOL,m_input.m_line_number);
1996 /***************************************************************
1998 Description: Recursive descent regular expression parser.
1999 **************************************************************/
2004 throws java.io.IOException
2009 if (CUtility.DESCENT_DEBUG)
2011 CUtility.enter("factor",m_spec.m_lexeme,m_spec.m_current_token);
2016 if (m_lexGen.CLOSURE == m_spec.m_current_token
2017 || m_lexGen.PLUS_CLOSE == m_spec.m_current_token
2018 || m_lexGen.OPTIONAL == m_spec.m_current_token)
2020 start = CAlloc.newCNfa(m_spec);
2021 end = CAlloc.newCNfa(m_spec);
2023 start.m_next = pair.m_start;
2024 pair.m_end.m_next = end;
2026 if (m_lexGen.CLOSURE == m_spec.m_current_token
2027 || m_lexGen.OPTIONAL == m_spec.m_current_token)
2029 start.m_next2 = end;
2032 if (m_lexGen.CLOSURE == m_spec.m_current_token
2033 || m_lexGen.PLUS_CLOSE == m_spec.m_current_token)
2035 pair.m_end.m_next2 = pair.m_start;
2038 pair.m_start = start;
2043 if (CUtility.DESCENT_DEBUG)
2045 CUtility.leave("factor",m_spec.m_lexeme,m_spec.m_current_token);
2049 /***************************************************************
2051 Description: Recursive descent regular expression parser.
2052 **************************************************************/
2057 throws java.io.IOException
2063 if (CUtility.DESCENT_DEBUG)
2065 CUtility.enter("term",m_spec.m_lexeme,m_spec.m_current_token);
2068 if (m_lexGen.OPEN_PAREN == m_spec.m_current_token)
2073 if (m_lexGen.CLOSE_PAREN == m_spec.m_current_token)
2079 CError.parse_error(CError.E_SYNTAX,m_input.m_line_number);
2084 start = CAlloc.newCNfa(m_spec);
2085 pair.m_start = start;
2087 start.m_next = CAlloc.newCNfa(m_spec);
2088 pair.m_end = start.m_next;
2090 if (m_lexGen.L == m_spec.m_current_token &&
2091 Character.isLetter(m_spec.m_lexeme))
2099 if (false == (m_lexGen.ANY == m_spec.m_current_token
2100 || m_lexGen.CCL_START == m_spec.m_current_token
2101 || (m_spec.m_ignorecase && isAlphaL)))
2103 start.m_edge = m_spec.m_lexeme;
2108 start.m_edge = CNfa.CCL;
2110 start.m_set = new CSet();
2112 /* Match case-insensitive letters using character class. */
2113 if (m_spec.m_ignorecase && isAlphaL)
2115 start.m_set.addncase(m_spec.m_lexeme);
2117 /* Match dot (.) using character class. */
2118 else if (m_lexGen.ANY == m_spec.m_current_token)
2120 start.m_set.add((byte) '\n');
2121 if (false == m_spec.m_unix)
2123 start.m_set.add((byte) '\r');
2125 start.m_set.complement();
2130 if (m_lexGen.AT_BOL == m_spec.m_current_token)
2134 /*start.m_set.add((byte) '\n');
2135 if (false == m_spec.m_unix)
2137 start.m_set.add((byte) '\r');
2139 start.m_set.complement();
2141 if (false == (m_lexGen.CCL_END == m_spec.m_current_token))
2143 dodash(start.m_set);
2147 for (c = 0; c <= ' '; ++c)
2149 start.m_set.add((byte) c);
2157 if (CUtility.DESCENT_DEBUG)
2159 CUtility.leave("term",m_spec.m_lexeme,m_spec.m_current_token);
2163 /***************************************************************
2165 Description: Recursive descent regular expression parser.
2166 **************************************************************/
2171 throws java.io.IOException
2175 if (CUtility.DESCENT_DEBUG)
2177 CUtility.enter("dodash",m_spec.m_lexeme,m_spec.m_current_token);
2180 while (m_lexGen.EOS != m_spec.m_current_token
2181 && m_lexGen.CCL_END != m_spec.m_current_token)
2183 // DASH loses its special meaning if it is first in class.
2184 if (m_lexGen.DASH == m_spec.m_current_token && -1 != first)
2187 // DASH loses its special meaning if it is last in class.
2188 if (m_spec.m_current_token == m_lexGen.CCL_END)
2190 // 'first' already in set.
2194 for ( ; first <= m_spec.m_lexeme; ++first)
2196 if (m_spec.m_ignorecase)
2197 set.addncase((char)first);
2204 first = m_spec.m_lexeme;
2205 if (m_spec.m_ignorecase)
2206 set.addncase(m_spec.m_lexeme);
2208 set.add(m_spec.m_lexeme);
2214 if (CUtility.DESCENT_DEBUG)
2216 CUtility.leave("dodash",m_spec.m_lexeme,m_spec.m_current_token);
2221 /***************************************************************
2223 **************************************************************/
2226 /***************************************************************
2228 **************************************************************/
2233 /***************************************************************
2235 Description: Constructor.
2236 **************************************************************/
2244 /***************************************************************
2246 Description: Resets member variables.
2247 **************************************************************/
2257 /***************************************************************
2259 Description: Sets member variables.
2260 **************************************************************/
2268 CUtility.assert(null != spec);
2276 /***************************************************************
2278 Description: High-level access function to module.
2279 **************************************************************/
2287 /* Remove redundant states. */
2290 /* Column and row compression.
2291 Save accept states in auxilary vector. */
2297 /***************************************************************
2299 Description: Copies source column into destination column.
2300 **************************************************************/
2301 private void col_copy
2311 n = m_spec.m_dtrans_vector.size();
2312 for (i = 0; i < n; ++i)
2314 dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
2315 dtrans.m_dtrans[dest] = dtrans.m_dtrans[src];
2319 /***************************************************************
2321 Description: Copies source row into destination row.
2322 **************************************************************/
2323 private void row_copy
2331 dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(src);
2332 m_spec.m_dtrans_vector.setElementAt(dtrans,dest);
2335 /***************************************************************
2338 **************************************************************/
2339 private boolean col_equiv
2349 n = m_spec.m_dtrans_vector.size();
2350 for (i = 0; i < n; ++i)
2352 dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
2353 if (dtrans.m_dtrans[col1] != dtrans.m_dtrans[col2])
2362 /***************************************************************
2365 **************************************************************/
2366 private boolean row_equiv
2376 dtrans1 = (CDTrans) m_spec.m_dtrans_vector.elementAt(row1);
2377 dtrans2 = (CDTrans) m_spec.m_dtrans_vector.elementAt(row2);
2379 for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
2381 if (dtrans1.m_dtrans[i] != dtrans2.m_dtrans[i])
2390 /***************************************************************
2393 **************************************************************/
2408 set = new JavaLexBitSet();
2410 /* Save accept nodes and anchor entries. */
2411 size = m_spec.m_dtrans_vector.size();
2412 m_spec.m_anchor_array = new int[size];
2413 m_spec.m_accept_vector = new Vector();
2414 for (i = 0; i < size; ++i)
2416 dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
2417 m_spec.m_accept_vector.addElement(dtrans.m_accept);
2418 m_spec.m_anchor_array[i] = dtrans.m_anchor;
2419 dtrans.m_accept = null;
2422 /* Allocate column map. */
2423 m_spec.m_col_map = new int[m_spec.m_dtrans_ncols];
2424 for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
2426 m_spec.m_col_map[i] = -1;
2429 /* Process columns for reduction. */
2430 for (reduced_ncols = 0; ; ++reduced_ncols)
2434 for (i = 0; i < reduced_ncols; ++i)
2436 CUtility.assert(-1 != m_spec.m_col_map[i]);
2440 for (i = reduced_ncols; i < m_spec.m_dtrans_ncols; ++i)
2442 if (-1 == m_spec.m_col_map[i])
2448 if (i >= m_spec.m_dtrans_ncols)
2455 CUtility.assert(false == set.get(i));
2456 CUtility.assert(-1 == m_spec.m_col_map[i]);
2461 m_spec.m_col_map[i] = reduced_ncols;
2463 /* UNDONE: Optimize by doing all comparisons in one batch. */
2464 for (j = i + 1; j < m_spec.m_dtrans_ncols; ++j)
2466 if (-1 == m_spec.m_col_map[j] && true == col_equiv(i,j))
2468 m_spec.m_col_map[j] = reduced_ncols;
2473 /* Reduce columns. */
2475 for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
2483 j = m_spec.m_col_map[i];
2487 CUtility.assert(j <= i);
2498 m_spec.m_dtrans_ncols = reduced_ncols;
2502 CUtility.assert(k == reduced_ncols);
2505 /* Allocate row map. */
2506 nrows = m_spec.m_dtrans_vector.size();
2507 m_spec.m_row_map = new int[nrows];
2508 for (i = 0; i < nrows; ++i)
2510 m_spec.m_row_map[i] = -1;
2513 /* Process rows to reduce. */
2514 for (reduced_nrows = 0; ; ++reduced_nrows)
2518 for (i = 0; i < reduced_nrows; ++i)
2520 CUtility.assert(-1 != m_spec.m_row_map[i]);
2524 for (i = reduced_nrows; i < nrows; ++i)
2526 if (-1 == m_spec.m_row_map[i])
2539 CUtility.assert(false == set.get(i));
2540 CUtility.assert(-1 == m_spec.m_row_map[i]);
2545 m_spec.m_row_map[i] = reduced_nrows;
2547 /* UNDONE: Optimize by doing all comparisons in one batch. */
2548 for (j = i + 1; j < nrows; ++j)
2550 if (-1 == m_spec.m_row_map[j] && true == row_equiv(i,j))
2552 m_spec.m_row_map[j] = reduced_nrows;
2559 for (i = 0; i < nrows; ++i)
2567 j = m_spec.m_row_map[i];
2571 CUtility.assert(j <= i);
2582 m_spec.m_dtrans_vector.setSize(reduced_nrows);
2586 /*System.out.print("k = " + k + "\nreduced_nrows = " + reduced_nrows + "\n");*/
2587 CUtility.assert(k == reduced_nrows);
2591 /***************************************************************
2592 Function: fix_dtrans
2593 Description: Updates CDTrans table after minimization
2594 using groups, removing redundant transition table states.
2595 **************************************************************/
2596 private void fix_dtrans
2603 Vector dtrans_group;
2607 new_vector = new Vector();
2609 size = m_spec.m_state_dtrans.length;
2610 for (i = 0; i < size; ++i)
2612 if (CDTrans.F != m_spec.m_state_dtrans[i])
2614 m_spec.m_state_dtrans[i] = m_ingroup[m_spec.m_state_dtrans[i]];
2618 size = m_group.size();
2619 for (i = 0; i < size; ++i)
2621 dtrans_group = (Vector) m_group.elementAt(i);
2622 first = (CDTrans) dtrans_group.elementAt(0);
2623 new_vector.addElement(first);
2625 for (c = 0; c < m_spec.m_dtrans_ncols; ++c)
2627 if (CDTrans.F != first.m_dtrans[c])
2629 first.m_dtrans[c] = m_ingroup[first.m_dtrans[c]];
2635 m_spec.m_dtrans_vector = new_vector;
2638 /***************************************************************
2640 Description: Removes redundant transition table states.
2641 **************************************************************/
2642 private void minimize
2646 Vector dtrans_group;
2650 int old_group_count;
2662 group_count = m_group.size();
2663 old_group_count = group_count - 1;
2665 while (old_group_count != group_count)
2667 old_group_count = group_count;
2671 CUtility.assert(m_group.size() == group_count);
2674 for (i = 0; i < group_count; ++i)
2676 dtrans_group = (Vector) m_group.elementAt(i);
2678 group_size = dtrans_group.size();
2679 if (group_size <= 1)
2684 new_group = new Vector();
2687 first = (CDTrans) dtrans_group.elementAt(0);
2688 for (j = 1; j < group_size; ++j)
2690 next = (CDTrans) dtrans_group.elementAt(j);
2692 for (c = 0; c < m_spec.m_dtrans_ncols; ++c)
2694 goto_first = first.m_dtrans[c];
2695 goto_next = next.m_dtrans[c];
2697 if (goto_first != goto_next
2698 && (goto_first == CDTrans.F
2699 || goto_next == CDTrans.F
2700 || m_ingroup[goto_next] != m_ingroup[goto_first]))
2704 CUtility.assert(dtrans_group.elementAt(j) == next);
2707 dtrans_group.removeElementAt(j);
2710 new_group.addElement(next);
2715 m_group.addElement(new_group);
2717 m_ingroup[next.m_label] = m_group.size() - 1;
2721 CUtility.assert(m_group.contains(new_group)
2723 CUtility.assert(m_group.contains(dtrans_group)
2725 CUtility.assert(dtrans_group.contains(first)
2727 CUtility.assert(dtrans_group.contains(next)
2729 CUtility.assert(new_group.contains(first)
2731 CUtility.assert(new_group.contains(next)
2733 CUtility.assert(dtrans_group.size() == group_size);
2734 CUtility.assert(i == m_ingroup[first.m_label]);
2735 CUtility.assert((m_group.size() - 1)
2736 == m_ingroup[next.m_label]);
2746 System.out.println(m_group.size() + " states after removal of redundant states.");
2748 if (m_spec.m_verbose
2749 && true == CUtility.OLD_DUMP_DEBUG)
2751 System.out.println("\nStates grouped as follows after minimization");
2758 /***************************************************************
2759 Function: init_groups
2761 **************************************************************/
2762 private void init_groups
2772 Vector dtrans_group;
2774 boolean group_found;
2776 m_group = new Vector();
2779 size = m_spec.m_dtrans_vector.size();
2780 m_ingroup = new int[size];
2782 for (i = 0; i < size; ++i)
2784 group_found = false;
2785 dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
2789 CUtility.assert(i == dtrans.m_label);
2790 CUtility.assert(false == group_found);
2791 CUtility.assert(group_count == m_group.size());
2794 for (j = 0; j < group_count; ++j)
2796 dtrans_group = (Vector) m_group.elementAt(j);
2800 CUtility.assert(false == group_found);
2801 CUtility.assert(0 < dtrans_group.size());
2804 first = (CDTrans) dtrans_group.elementAt(0);
2806 if (CUtility.SLOW_DEBUG)
2812 s = dtrans_group.size();
2813 CUtility.assert(0 < s);
2815 for (k = 1; k < s; ++k)
2817 check = (CDTrans) dtrans_group.elementAt(k);
2818 CUtility.assert(check.m_accept == first.m_accept);
2822 if (first.m_accept == dtrans.m_accept)
2824 dtrans_group.addElement(dtrans);
2830 CUtility.assert(j == m_ingroup[dtrans.m_label]);
2837 if (false == group_found)
2839 dtrans_group = new Vector();
2840 dtrans_group.addElement(dtrans);
2841 m_ingroup[i] = m_group.size();
2842 m_group.addElement(dtrans_group);
2847 if (m_spec.m_verbose
2848 && true == CUtility.OLD_DUMP_DEBUG)
2850 System.out.println("Initial grouping:");
2852 System.out.println("");
2856 /***************************************************************
2858 **************************************************************/
2868 size = dtrans_group.size();
2869 for (i = 0; i < size; ++i)
2871 dtrans = (CDTrans) dtrans_group.elementAt(i);
2872 System.out.print(dtrans.m_label + " ");
2876 /***************************************************************
2878 **************************************************************/
2879 private void pgroups
2887 group_size = m_group.size();
2888 for (i = 0; i < group_size; ++i)
2890 System.out.print("\tGroup " + i + " {");
2891 pset((Vector) m_group.elementAt(i));
2892 System.out.println("}\n");
2895 System.out.println("");
2896 dtrans_size = m_spec.m_dtrans_vector.size();
2897 for (i = 0; i < dtrans_size; ++i)
2899 System.out.println("\tstate " + i
2906 /***************************************************************
2908 **************************************************************/
2911 /***************************************************************
2913 **************************************************************/
2914 private CSpec m_spec;
2915 private int m_unmarked_dfa;
2916 private CLexGen m_lexGen;
2918 /***************************************************************
2920 **************************************************************/
2921 private static final int NOT_IN_DSTATES = -1;
2923 /***************************************************************
2925 **************************************************************/
2933 /***************************************************************
2936 **************************************************************/
2948 /***************************************************************
2951 **************************************************************/
2961 /***************************************************************
2963 Description: High-level access function to module.
2964 **************************************************************/
2979 if (m_spec.m_verbose && true == CUtility.OLD_DUMP_DEBUG)
2981 System.out.println(m_spec.m_dfa_states.size()
2982 + " DFA states in original machine.");
2988 /***************************************************************
2989 Function: make_dtrans
2990 Description: Creates uncompressed CDTrans transition table.
2991 **************************************************************/
2992 private void make_dtrans
2995 /* throws java.lang.CloneNotSupportedException*/
3008 System.out.print("Working on DFA states.");
3010 /* Reference passing type and initializations. */
3011 bunch = new CBunch();
3014 /* Allocate mapping array. */
3015 nstates = m_spec.m_state_rules.length;
3016 m_spec.m_state_dtrans = new int[nstates];
3018 for (istate = 0; nstates > istate; ++istate)
3020 if (0 == m_spec.m_state_rules[istate].size())
3022 m_spec.m_state_dtrans[istate] = CDTrans.F;
3026 /* Create start state and initialize fields. */
3027 bunch.m_nfa_set = (Vector) m_spec.m_state_rules[istate].clone();
3028 sortStates(bunch.m_nfa_set);
3030 bunch.m_nfa_bit = new JavaLexBitSet();
3032 /* Initialize bit set. */
3033 size = bunch.m_nfa_set.size();
3034 for (i = 0; size > i; ++i)
3036 nfa = (CNfa) bunch.m_nfa_set.elementAt(i);
3037 bunch.m_nfa_bit.set(nfa.m_label);
3040 bunch.m_accept = null;
3041 bunch.m_anchor = CSpec.NONE;
3042 bunch.m_accept_index = CUtility.INT_MAX;
3045 add_to_dstates(bunch);
3047 m_spec.m_state_dtrans[istate] = m_spec.m_dtrans_vector.size();
3049 /* Main loop of CDTrans creation. */
3050 while (null != (dfa = get_unmarked()))
3052 System.out.print(".");
3057 CUtility.assert(false == dfa.m_mark);
3060 /* Get first unmarked node, then mark it. */
3063 /* Allocate new CDTrans, then initialize fields. */
3064 dtrans = new CDTrans(m_spec.m_dtrans_vector.size(),m_spec);
3065 dtrans.m_accept = dfa.m_accept;
3066 dtrans.m_anchor = dfa.m_anchor;
3068 /* Set CDTrans array for each character transition. */
3069 for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
3073 CUtility.assert(0 <= i);
3074 CUtility.assert(m_spec.m_dtrans_ncols > i);
3077 /* Create new dfa set by attempting character transition. */
3078 move(dfa.m_nfa_set,dfa.m_nfa_bit,i,bunch);
3079 if (null != bunch.m_nfa_set)
3086 CUtility.assert((null == bunch.m_nfa_set
3087 && null == bunch.m_nfa_bit)
3088 || (null != bunch.m_nfa_set
3089 && null != bunch.m_nfa_bit));
3092 /* Create new state or set state to empty. */
3093 if (null == bunch.m_nfa_set)
3095 nextstate = CDTrans.F;
3099 nextstate = in_dstates(bunch);
3101 if (NOT_IN_DSTATES == nextstate)
3103 nextstate = add_to_dstates(bunch);
3109 CUtility.assert(nextstate < m_spec.m_dfa_states.size());
3112 dtrans.m_dtrans[i] = nextstate;
3117 CUtility.assert(m_spec.m_dtrans_vector.size() == dfa.m_label);
3120 m_spec.m_dtrans_vector.addElement(dtrans);
3124 System.out.println("");
3127 /***************************************************************
3128 Function: free_dfa_states
3129 **************************************************************/
3130 private void free_dfa_states
3134 m_spec.m_dfa_states = null;
3135 m_spec.m_dfa_sets = null;
3138 /***************************************************************
3139 Function: free_nfa_states
3140 **************************************************************/
3141 private void free_nfa_states
3145 /* UNDONE: Remove references to nfas from within dfas. */
3146 /* UNDONE: Don't free CAccepts. */
3148 m_spec.m_nfa_states = null;
3149 m_spec.m_nfa_start = null;
3150 m_spec.m_state_rules = null;
3153 /***************************************************************
3155 Description: Alters and returns input set.
3156 **************************************************************/
3157 private void e_closure
3170 CUtility.assert(null != bunch);
3171 CUtility.assert(null != bunch.m_nfa_set);
3172 CUtility.assert(null != bunch.m_nfa_bit);
3175 bunch.m_accept = null;
3176 bunch.m_anchor = CSpec.NONE;
3177 bunch.m_accept_index = CUtility.INT_MAX;
3179 /* Create initial stack. */
3180 nfa_stack = new Stack();
3181 size = bunch.m_nfa_set.size();
3182 for (i = 0; i < size; ++i)
3184 state = (CNfa) bunch.m_nfa_set.elementAt(i);
3188 CUtility.assert(bunch.m_nfa_bit.get(state.m_label));
3191 nfa_stack.push(state);
3195 while (false == nfa_stack.empty())
3197 state = (CNfa) nfa_stack.pop();
3199 if (CUtility.OLD_DUMP_DEBUG)
3201 if (null != state.m_accept)
3203 System.out.println("Looking at accepting state " + state.m_label
3205 + (new String(state.m_accept.m_action,0,
3206 state.m_accept.m_action_read))
3211 if (null != state.m_accept
3212 && state.m_label < bunch.m_accept_index)
3214 bunch.m_accept_index = state.m_label;
3215 bunch.m_accept = state.m_accept;
3216 bunch.m_anchor = state.m_anchor;
3218 if (CUtility.OLD_DUMP_DEBUG)
3220 System.out.println("Found accepting state " + state.m_label
3222 + (new String(state.m_accept.m_action,0,
3223 state.m_accept.m_action_read))
3229 CUtility.assert(null != bunch.m_accept);
3230 CUtility.assert(CSpec.NONE == bunch.m_anchor
3231 || 0 != (bunch.m_anchor & CSpec.END)
3232 || 0 != (bunch.m_anchor & CSpec.START));
3236 if (CNfa.EPSILON == state.m_edge)
3238 if (null != state.m_next)
3240 if (false == bunch.m_nfa_set.contains(state.m_next))
3244 CUtility.assert(false == bunch.m_nfa_bit.get(state.m_next.m_label));
3247 bunch.m_nfa_bit.set(state.m_next.m_label);
3248 bunch.m_nfa_set.addElement(state.m_next);
3249 nfa_stack.push(state.m_next);
3253 if (null != state.m_next2)
3255 if (false == bunch.m_nfa_set.contains(state.m_next2))
3259 CUtility.assert(false == bunch.m_nfa_bit.get(state.m_next2.m_label));
3262 bunch.m_nfa_bit.set(state.m_next2.m_label);
3263 bunch.m_nfa_set.addElement(state.m_next2);
3264 nfa_stack.push(state.m_next2);
3270 if (null != bunch.m_nfa_set)
3272 sortStates(bunch.m_nfa_set);
3278 /***************************************************************
3280 Description: Returns null if resulting NFA set is empty.
3281 **************************************************************/
3285 JavaLexBitSet nfa_bit,
3294 bunch.m_nfa_set = null;
3295 bunch.m_nfa_bit = null;
3297 size = nfa_set.size();
3298 for (index = 0; index < size; ++index)
3300 state = (CNfa) nfa_set.elementAt(index);
3302 if (b == state.m_edge
3303 || (CNfa.CCL == state.m_edge
3304 && true == state.m_set.contains(b)))
3306 if (null == bunch.m_nfa_set)
3310 CUtility.assert(null == bunch.m_nfa_bit);
3313 bunch.m_nfa_set = new Vector();
3315 = new JavaLexBitSet(m_spec.m_nfa_states.size());*/
3316 bunch.m_nfa_bit = new JavaLexBitSet();
3319 bunch.m_nfa_set.addElement(state.m_next);
3320 /*System.out.println("Size of bitset: " + bunch.m_nfa_bit.size());
3321 System.out.println("Reference index: " + state.m_next.m_label);
3322 System.out.flush();*/
3323 bunch.m_nfa_bit.set(state.m_next.m_label);
3327 if (null != bunch.m_nfa_set)
3331 CUtility.assert(null != bunch.m_nfa_bit);
3334 sortStates(bunch.m_nfa_set);
3340 /***************************************************************
3341 Function: sortStates
3342 **************************************************************/
3343 private void sortStates
3357 size = nfa_set.size();
3358 for (begin = 0; begin < size; ++begin)
3360 elem = (CNfa) nfa_set.elementAt(begin);
3361 smallest_value = elem.m_label;
3362 smallest_index = begin;
3364 for (index = begin + 1; index < size; ++index)
3366 elem = (CNfa) nfa_set.elementAt(index);
3367 value = elem.m_label;
3369 if (value < smallest_value)
3371 smallest_index = index;
3372 smallest_value = value;
3376 begin_elem = (CNfa) nfa_set.elementAt(begin);
3377 elem = (CNfa) nfa_set.elementAt(smallest_index);
3378 nfa_set.setElementAt(elem,begin);
3379 nfa_set.setElementAt(begin_elem,smallest_index);
3382 if (CUtility.OLD_DEBUG)
3384 System.out.print("NFA vector indices: ");
3386 for (index = 0; index < size; ++index)
3388 elem = (CNfa) nfa_set.elementAt(index);
3389 System.out.print(elem.m_label + " ");
3391 System.out.print("\n");
3397 /***************************************************************
3398 Function: get_unmarked
3399 Description: Returns next unmarked DFA state.
3400 **************************************************************/
3401 private CDfa get_unmarked
3408 size = m_spec.m_dfa_states.size();
3409 while (m_unmarked_dfa < size)
3411 dfa = (CDfa) m_spec.m_dfa_states.elementAt(m_unmarked_dfa);
3413 if (false == dfa.m_mark)
3415 if (CUtility.OLD_DUMP_DEBUG)
3417 System.out.print("*");
3421 if (m_spec.m_verbose && true == CUtility.OLD_DUMP_DEBUG)
3423 System.out.println("---------------");
3424 System.out.print("working on DFA state "
3426 + " = NFA states: ");
3427 m_lexGen.print_set(dfa.m_nfa_set);
3428 System.out.print("\n");
3440 /***************************************************************
3441 function: add_to_dstates
3442 Description: Takes as input a CBunch with details of
3443 a dfa state that needs to be created.
3444 1) Allocates a new dfa state and saves it in
3445 the appropriate CSpec vector.
3446 2) Initializes the fields of the dfa state
3447 with the information in the CBunch.
3448 3) Returns index of new dfa.
3449 **************************************************************/
3450 private int add_to_dstates
3459 CUtility.assert(null != bunch.m_nfa_set);
3460 CUtility.assert(null != bunch.m_nfa_bit);
3461 CUtility.assert(null != bunch.m_accept
3462 || CSpec.NONE == bunch.m_anchor);
3465 /* Allocate, passing CSpec so dfa label can be set. */
3466 dfa = CAlloc.newCDfa(m_spec);
3468 /* Initialize fields, including the mark field. */
3469 dfa.m_nfa_set = (Vector) bunch.m_nfa_set.clone();
3470 dfa.m_nfa_bit = (JavaLexBitSet) bunch.m_nfa_bit.clone();
3471 dfa.m_accept = bunch.m_accept;
3472 dfa.m_anchor = bunch.m_anchor;
3475 /* Register dfa state using BitSet in CSpec Hashtable. */
3476 m_spec.m_dfa_sets.put(dfa.m_nfa_bit,dfa);
3477 /*registerCDfa(dfa);*/
3479 if (CUtility.OLD_DUMP_DEBUG)
3481 System.out.print("Registering set : ");
3482 m_lexGen.print_set(dfa.m_nfa_set);
3483 System.out.println("");
3489 /***************************************************************
3490 Function: in_dstates
3491 **************************************************************/
3492 private int in_dstates
3499 if (CUtility.OLD_DEBUG)
3501 System.out.print("Looking for set : ");
3502 m_lexGen.print_set(bunch.m_nfa_set);
3505 dfa = (CDfa) m_spec.m_dfa_sets.get(bunch.m_nfa_bit);
3509 if (CUtility.OLD_DUMP_DEBUG)
3511 System.out.println(" FOUND!");
3517 if (CUtility.OLD_DUMP_DEBUG)
3519 System.out.println(" NOT FOUND!");
3521 return NOT_IN_DSTATES;
3526 /***************************************************************
3528 **************************************************************/
3531 /***************************************************************
3533 **************************************************************/
3541 dfa = new CDfa(spec.m_dfa_states.size());
3542 spec.m_dfa_states.addElement(dfa);
3547 /***************************************************************
3548 Function: newCNfaPair
3550 **************************************************************/
3551 static CNfaPair newCNfaPair
3555 CNfaPair pair = new CNfaPair();
3560 /***************************************************************
3563 **************************************************************/
3571 /* UNDONE: Buffer this? */
3575 /*p.m_label = spec.m_nfa_states.size();*/
3576 spec.m_nfa_states.addElement(p);
3577 p.m_edge = CNfa.EPSILON;
3583 /***************************************************************
3585 Description: Top-level lexical analyzer generator function.
3586 **************************************************************/
3589 /***************************************************************
3591 **************************************************************/
3592 public static void main
3596 throws java.io.IOException
3602 System.out.println("Usage: JLex.Main <filename>");
3606 /* Note: For debuging, it may be helpful to remove the try/catch
3607 block and permit the Exception to propagate to the top level.
3608 This gives more information. */
3611 lg = new CLexGen(arg[0]);
3616 System.out.println(e.getMessage());
3621 /***************************************************************
3623 **************************************************************/
3626 /*************************************************************
3628 ***********************************************************/
3634 /*************************************************************
3636 ***********************************************************/
3637 static final int F = -1;
3639 /*************************************************************
3641 ***********************************************************/
3648 m_dtrans = new int[spec.m_dtrans_ncols];
3650 m_anchor = CSpec.NONE;
3655 /***************************************************************
3657 **************************************************************/
3660 /***************************************************************
3662 ***********************************************************/
3668 JavaLexBitSet m_nfa_bit;
3671 /***************************************************************
3673 **************************************************************/
3683 m_anchor = CSpec.NONE;
3692 /***************************************************************
3694 **************************************************************/
3697 /***************************************************************
3699 **************************************************************/
3704 /***************************************************************
3706 **************************************************************/
3716 m_action_read = action_read;
3718 m_action = new char[m_action_read];
3719 for (elem = 0; elem < m_action_read; ++elem)
3721 m_action[elem] = action[elem];
3724 m_line_number = line_number;
3727 /***************************************************************
3729 **************************************************************/
3737 m_action_read = accept.m_action_read;
3739 m_action = new char[m_action_read];
3740 for (elem = 0; elem < m_action_read; ++elem)
3742 m_action[elem] = accept.m_action[elem];
3745 m_line_number = accept.m_line_number;
3748 /***************************************************************
3750 **************************************************************/
3758 m_action_read = accept.m_action_read;
3760 m_action = new char[m_action_read];
3761 for (elem = 0; elem < m_action_read; ++elem)
3763 m_action[elem] = accept.m_action[elem];
3768 /***************************************************************
3769 Class: CAcceptAnchor
3770 **************************************************************/
3773 /***************************************************************
3775 **************************************************************/
3779 /***************************************************************
3780 Function: CAcceptAnchor
3781 **************************************************************/
3787 m_anchor = CSpec.NONE;
3791 /***************************************************************
3793 **************************************************************/
3796 /***************************************************************
3798 **************************************************************/
3802 /***************************************************************
3804 **************************************************************/
3814 /***************************************************************
3817 **************************************************************/
3820 /***************************************************************
3822 **************************************************************/
3823 private InputStream m_input; /* JLex specification file. */
3825 private byte m_buffer[]; /* Input buffer. */
3826 private int m_buffer_read; /* Number of bytes read into input buffer. */
3827 private int m_buffer_index; /* Current index into input buffer. */
3829 boolean m_eof_reached; /* Whether EOF has been encountered. */
3830 boolean m_pushback_line;
3832 char m_line[]; /* Line buffer. */
3833 int m_line_read; /* Number of bytes read into line buffer. */
3834 int m_line_index; /* Current index into line buffer. */
3836 int m_line_number; /* Current line number. */
3838 /***************************************************************
3840 **************************************************************/
3841 private static final int BUFFER_SIZE = 1024;
3842 static final boolean EOF = true;
3843 static final boolean NOT_EOF = false;
3845 /***************************************************************
3848 **************************************************************/
3856 CUtility.assert(null != input);
3859 /* Initialize input stream. */
3862 /* Initialize buffers and index counters. */
3863 m_buffer = new byte[BUFFER_SIZE];
3867 m_line = new char[BUFFER_SIZE];
3871 /* Initialize state variables. */
3872 m_eof_reached = false;
3874 m_pushback_line = false;
3877 /***************************************************************
3879 Description: Returns true on EOF, false otherwise.
3880 Guarantees not to return a blank line, or a line
3882 **************************************************************/
3886 throws java.io.IOException
3890 /* Has EOF already been reached? */
3893 if (CUtility.OLD_DEBUG)
3895 System.out.print("Line 1: ");
3896 System.out.print(new String(m_line,0,
3903 /* Pushback current line? */
3904 if (m_pushback_line)
3906 m_pushback_line = false;
3908 /* Check for empty line. */
3909 for (elem = 0; elem < m_line_read; ++elem)
3911 if (false == CUtility.isspace(m_line[elem]))
3918 if (elem < m_line_read)
3924 if (CUtility.OLD_DEBUG)
3926 System.out.print("Line 2: ");
3927 System.out.print(new String(m_line,0,
3934 /* Refill buffer? */
3935 if (m_buffer_index >= m_buffer_read)
3937 m_buffer_read = m_input.read(m_buffer);
3938 if (-1 == m_buffer_read)
3940 m_eof_reached = true;
3942 if (CUtility.OLD_DEBUG)
3944 System.out.print("Line 3: ");
3945 System.out.print(new String(m_line,0,
3956 while ((byte) '\n' != m_buffer[m_buffer_index]
3957 && (byte) '\r' != m_buffer[m_buffer_index])
3959 m_line[m_line_read] = (char) m_buffer[m_buffer_index];
3963 /* Refill buffer? */
3964 if (m_buffer_index >= m_buffer_read)
3966 m_buffer_read = m_input.read(m_buffer);
3967 if (-1 == m_buffer_read)
3969 m_eof_reached = true;
3971 /* Check for empty lines and discard them. */
3973 while (CUtility.isspace(m_line[elem]))
3976 if (elem == m_line_read)
3982 if (elem < m_line_read)
3984 if (CUtility.OLD_DEBUG)
3986 System.out.print("Line 4:");
3987 System.out.print(new String(m_line,
3997 if (CUtility.OLD_DEBUG)
3999 System.out.print("Line <e>:");
4000 System.out.print(new String(m_line,0,
4013 /* Resize line buffer? */
4014 if (m_line_read >= m_line.length)
4016 m_line = CUtility.doubleSize(m_line);
4019 /* Save newline in buffer. */
4021 m_line[m_line_read] = (char) m_buffer[m_buffer_index];
4025 if ((byte) '\r' == m_buffer[m_buffer_index])
4027 /* This is a hack and may propagate
4028 some (pre-existing) line counting bugs on non-UNIX,
4029 but otherwise it will work fine.
4030 The only effect will be some bad error messages.
4031 The other option is to do some massive
4032 hacking on the input routines
4033 that may destabilize the code. */
4034 m_line[m_line_read] = '\n';
4040 m_line[m_line_read] = (char) m_buffer[m_buffer_index];
4044 /* End of New Source. */
4047 /* Check for empty lines and discard them. */
4049 while (CUtility.isspace(m_line[elem]))
4052 if (elem == m_line_read)
4058 if (elem < m_line_read)
4064 if (CUtility.OLD_DEBUG)
4066 System.out.print("Line <f>:");
4067 System.out.print(new String(m_line,0,m_line_read));
4075 /********************************************************
4077 *******************************************************/
4080 /********************************************************
4082 *******************************************************/
4083 static final boolean DEBUG = true;
4084 static final boolean SLOW_DEBUG = true;
4085 static final boolean DUMP_DEBUG = true;
4086 /*static final boolean DEBUG = false;
4087 static final boolean SLOW_DEBUG = false;
4088 static final boolean DUMP_DEBUG = false;*/
4089 static final boolean DESCENT_DEBUG = false;
4090 static final boolean OLD_DEBUG = false;
4091 static final boolean OLD_DUMP_DEBUG = false;
4092 static final boolean FOODEBUG = false;
4093 static final boolean DO_DEBUG = false;
4095 /********************************************************
4096 Constants: Integer Bounds
4097 *******************************************************/
4098 static final int INT_MAX = 2147483647;
4100 /* UNDONE: What about other character values??? */
4101 static final int MAX_SEVEN_BIT = 127;
4102 static final int MAX_EIGHT_BIT = 256;
4104 /********************************************************
4106 Description: Debugging routine.
4107 *******************************************************/
4115 System.out.println("Entering " + descent
4116 + " [lexeme: " + lexeme
4117 + "] [token: " + token + "]");
4120 /********************************************************
4122 Description: Debugging routine.
4123 *******************************************************/
4131 System.out.println("Leaving " + descent
4132 + " [lexeme:" + lexeme
4133 + "] [token:" + token + "]");
4136 /********************************************************
4138 Description: Debugging routine.
4139 *******************************************************/
4145 if (DEBUG && false == expr)
4147 System.out.println("Assertion Failed");
4148 throw new Error("Assertion Failed.");
4152 /***************************************************************
4153 Function: doubleSize
4154 **************************************************************/
4155 static char[] doubleSize
4160 char newBuffer[] = new char[2 * oldBuffer.length];
4163 for (elem = 0; elem < oldBuffer.length; ++elem)
4165 newBuffer[elem] = oldBuffer[elem];
4171 /***************************************************************
4172 Function: doubleSize
4173 **************************************************************/
4174 static byte[] doubleSize
4179 byte newBuffer[] = new byte[2 * oldBuffer.length];
4182 for (elem = 0; elem < oldBuffer.length; ++elem)
4184 newBuffer[elem] = oldBuffer[elem];
4190 /********************************************************
4192 *******************************************************/
4198 if ('0' <= c && '9' >= c)
4200 return (char) (c - '0');
4202 else if ('a' <= c && 'f' >= c)
4204 return (char) (c - 'a' + 10);
4206 else if ('A' <= c && 'F' >= c)
4208 return (char) (c - 'A' + 10);
4211 CError.impos("Bad hexidecimal digit" + c);
4215 /********************************************************
4216 Function: ishexdigit
4217 *******************************************************/
4218 static boolean ishexdigit
4223 if (('0' <= c && '9' >= c)
4224 || ('a' <= c && 'f' >= c)
4225 || ('A' <= c && 'F' >= c))
4233 /********************************************************
4235 *******************************************************/
4241 if ('0' <= c && '7' >= c)
4243 return (char) (c - '0');
4246 CError.impos("Bad octal digit " + c);
4250 /********************************************************
4251 Function: isoctdigit
4252 *******************************************************/
4253 static boolean isoctdigit
4258 if ('0' <= c && '7' >= c)
4266 /********************************************************
4268 *******************************************************/
4269 static boolean isspace
4287 /********************************************************
4289 *******************************************************/
4290 static boolean isnewline
4304 /********************************************************
4306 *******************************************************/
4307 static boolean isalpha
4312 if (('a' <= c && 'z' >= c)
4313 || ('A' <= c && 'Z' >= c))
4321 /********************************************************
4323 *******************************************************/
4329 if (('a' <= c && 'z' >= c))
4331 return (char) (c - 'a' + 'A');
4337 /********************************************************
4339 Description: Compares up to n elements of
4340 byte array a[] against byte array b[].
4341 The first byte comparison is made between
4342 a[a_first] and b[b_first]. Comparisons continue
4343 until the null terminating byte '\0' is reached
4344 or until n bytes are compared.
4345 Return Value: Returns 0 if arrays are the
4346 same up to and including the null terminating byte
4347 or up to and including the first n bytes,
4348 whichever comes first.
4349 *******************************************************/
4361 for (elem = 0; elem < n; ++elem)
4363 /*System.out.print((char) a[a_first + elem]);
4364 System.out.print((char) b[b_first + elem]);*/
4366 if ('\0' == a[a_first + elem] && '\0' == b[b_first + elem])
4368 /*System.out.println("return 0");*/
4371 if (a[a_first + elem] < b[b_first + elem])
4373 /*System.out.println("return 1");*/
4376 else if (a[a_first + elem] > b[b_first + elem])
4378 /*System.out.println("return -1");*/
4383 /*System.out.println("return 0");*/
4387 /********************************************************
4389 *******************************************************/
4401 for (elem = 0; elem < n; ++elem)
4403 if ('\0' == a[a_first + elem] && '\0' == b[b_first + elem])
4407 if (a[a_first + elem] < b[b_first + elem])
4411 else if (a[a_first + elem] > b[b_first + elem])
4421 /********************************************************
4423 *******************************************************/
4426 /********************************************************
4429 *******************************************************/
4435 System.out.println("JLex Error: " + message);
4438 /********************************************************
4440 Description: Error codes for parse_error().
4441 *******************************************************/
4442 static final int E_BADEXPR = 0;
4443 static final int E_PAREN = 1;
4444 static final int E_LENGTH = 2;
4445 static final int E_BRACKET = 3;
4446 static final int E_BOL = 4;
4447 static final int E_CLOSE = 5;
4448 static final int E_NEWLINE = 6;
4449 static final int E_BADMAC = 7;
4450 static final int E_NOMAC = 8;
4451 static final int E_MACDEPTH = 9;
4452 static final int E_INIT = 10;
4453 static final int E_EOF = 11;
4454 static final int E_DIRECT = 12;
4455 static final int E_INTERNAL = 13;
4456 static final int E_STATE = 14;
4457 static final int E_MACDEF = 15;
4458 static final int E_SYNTAX = 16;
4459 static final int E_BRACE = 17;
4460 static final int E_DASH = 18;
4462 /********************************************************
4464 Description: String messages for parse_error();
4465 *******************************************************/
4466 static final String errmsg[] =
4468 "Malformed regular expression.",
4469 "Missing close parenthesis.",
4470 "Too many regular expressions or expression too long.",
4471 "Missing [ in character class.",
4472 "^ must be at start of expression or after [.",
4473 "+ ? or * must follow an expression or subexpression.",
4474 "Newline in quoted string.",
4475 "Missing } in macro expansion.",
4476 "Macro does not exist.",
4477 "Macro expansions nested too deeply.",
4478 "JLex has not been successfully initialized.",
4479 "Unexpected end-of-file found.",
4480 "Undefined or badly-formed JLex directive.",
4481 "Internal JLex error.",
4482 "Unitialized state name.",
4483 "Badly formed macro definition.",
4485 "Missing brace at start of lexical action.",
4486 "Special character dash - in character class [...] must\n"
4487 + "\tbe preceded by start-of-range character."
4490 /********************************************************
4491 Function: parse_error
4493 *******************************************************/
4494 static void parse_error
4500 System.out.println("Error: Parse error at line "
4501 + line_number + ".");
4502 System.out.println("Description: " + errmsg[error_code]);
4503 throw new Error("Parse error.");
4507 /********************************************************
4509 *******************************************************/
4512 /********************************************************
4514 *******************************************************/
4515 private JavaLexBitSet m_set;
4516 private boolean m_complement;
4518 /********************************************************
4520 *******************************************************/
4525 m_set = new JavaLexBitSet();
4526 m_complement = false;
4529 /********************************************************
4530 Function: complement
4531 *******************************************************/
4536 m_complement = true;
4539 /********************************************************
4541 *******************************************************/
4550 /********************************************************
4552 *******************************************************/
4553 void addncase // add, ignoring case.
4558 /* Do this in a Unicode-friendly way. */
4559 /* (note that duplicate adds have no effect) */
4561 add(Character.toLowerCase(c));
4562 add(Character.toTitleCase(c));
4563 add(Character.toUpperCase(c));
4566 /********************************************************
4568 *******************************************************/
4576 result = m_set.get(i);
4580 return (false == result);
4586 /********************************************************
4588 *******************************************************/
4594 m_complement = set.m_complement;
4595 m_set = (JavaLexBitSet) set.m_set.clone();
4599 /********************************************************
4601 *******************************************************/
4604 /********************************************************
4606 *******************************************************/
4607 int m_edge; /* Label for edge type:
4609 CCL (character class),
4615 CSet m_set; /* Set to store character classes. */
4616 CNfa m_next; /* Next state (or null if none). */
4618 CNfa m_next2; /* Another state with type == EPSILON
4619 and null if not used.
4620 The NFA construction should result in two
4621 outgoing edges only if both are EPSILON edges. */
4623 CAccept m_accept; /* Set to null if nonaccepting state. */
4624 int m_anchor; /* Says if and where pattern is anchored. */
4628 JavaLexBitSet m_states;
4630 /********************************************************
4632 *******************************************************/
4633 static final int NO_LABEL = -1;
4635 /********************************************************
4636 Constants: Edge Types
4637 Note: Edge transitions on one specific character
4638 are labelled with the character Ascii (Unicode)
4639 codes. So none of the constants below should
4640 overlap with the natural character codes.
4641 *******************************************************/
4642 static final int CCL = -1;
4643 static final int EMPTY = -2;
4644 static final int EPSILON = -3;
4646 /********************************************************
4648 *******************************************************/
4658 m_anchor = CSpec.NONE;
4663 /********************************************************
4665 Description: Converts this NFA state into a copy of
4667 *******************************************************/
4673 m_edge = nfa.m_edge;
4675 if (null != nfa.m_set)
4681 m_set.mimic(nfa.m_set);
4688 m_next = nfa.m_next;
4689 m_next2 = nfa.m_next2;
4690 m_accept = nfa.m_accept;
4691 m_anchor = nfa.m_anchor;
4693 if (null != nfa.m_states)
4695 m_states = (JavaLexBitSet) nfa.m_states.clone();
4704 /***************************************************************
4706 **************************************************************/
4709 /***************************************************************
4711 **************************************************************/
4712 private InputStream m_instream; /* JLex specification file. */
4713 private DataOutputStream m_outstream; /* Lexical analyzer source file. */
4715 private CInput m_input; /* Input buffer class. */
4717 private Hashtable m_tokens; /* Hashtable that maps characters to their
4718 corresponding lexical code for
4719 the internal lexical analyzer. */
4720 private CSpec m_spec; /* Spec class holds information
4721 about the generated lexer. */
4722 private boolean m_init_flag; /* Flag set to true only upon
4723 successful initialization. */
4725 private CMakeNfa m_makeNfa; /* NFA machine generator module. */
4726 private CNfa2Dfa m_nfa2dfa; /* NFA to DFA machine (transition table)
4727 conversion module. */
4728 private CMinimize m_minimize; /* Transition table compressor. */
4729 private CEmit m_emit; /* Output module that emits source code
4730 into the generated lexer file. */
4733 /********************************************************
4735 *******************************************************/
4736 private static final boolean ERROR = false;
4737 private static final boolean NOT_ERROR = true;
4738 private static final int BUFFER_SIZE = 1024;
4740 /********************************************************
4741 Constants: Token Types
4742 *******************************************************/
4743 static final int EOS = 1;
4744 static final int ANY = 2;
4745 static final int AT_BOL = 3;
4746 static final int AT_EOL = 4;
4747 static final int CCL_END = 5;
4748 static final int CCL_START = 6;
4749 static final int CLOSE_CURLY = 7;
4750 static final int CLOSE_PAREN = 8;
4751 static final int CLOSURE = 9;
4752 static final int DASH = 10;
4753 static final int END_OF_INPUT = 11;
4754 static final int L = 12;
4755 static final int OPEN_CURLY = 13;
4756 static final int OPEN_PAREN = 14;
4757 static final int OPTIONAL = 15;
4758 static final int OR = 16;
4759 static final int PLUS_CLOSE = 17;
4761 /***************************************************************
4763 **************************************************************/
4768 throws java.io.FileNotFoundException, java.io.IOException
4770 /* Successful initialization flag. */
4771 m_init_flag = false;
4773 /* Open input stream. */
4774 m_instream = new FileInputStream(filename);
4775 if (null == m_instream)
4777 System.out.println("Error: Unable to open input file "
4782 /* Open output stream. */
4784 = new DataOutputStream(new BufferedOutputStream(
4785 new FileOutputStream(filename + ".java")));
4786 if (null == m_outstream)
4788 System.out.println("Error: Unable to open output file "
4789 + filename + ".java.");
4793 /* Create input buffer class. */
4794 m_input = new CInput(m_instream);
4796 /* Initialize character hash table. */
4797 m_tokens = new Hashtable();
4798 m_tokens.put(new Character('$'),new Integer(AT_EOL));
4799 m_tokens.put(new Character('('),new Integer(OPEN_PAREN));
4800 m_tokens.put(new Character(')'),new Integer(CLOSE_PAREN));
4801 m_tokens.put(new Character('*'),new Integer(CLOSURE));
4802 m_tokens.put(new Character('+'),new Integer(PLUS_CLOSE));
4803 m_tokens.put(new Character('-'),new Integer(DASH));
4804 m_tokens.put(new Character('.'),new Integer(ANY));
4805 m_tokens.put(new Character('?'),new Integer(OPTIONAL));
4806 m_tokens.put(new Character('['),new Integer(CCL_START));
4807 m_tokens.put(new Character(']'),new Integer(CCL_END));
4808 m_tokens.put(new Character('^'),new Integer(AT_BOL));
4809 m_tokens.put(new Character('{'),new Integer(OPEN_CURLY));
4810 m_tokens.put(new Character('|'),new Integer(OR));
4811 m_tokens.put(new Character('}'),new Integer(CLOSE_CURLY));
4813 /* Initialize spec structure. */
4814 m_spec = new CSpec(this);
4816 /* Nfa to dfa converter. */
4817 m_nfa2dfa = new CNfa2Dfa();
4818 m_minimize = new CMinimize();
4819 m_makeNfa = new CMakeNfa();
4821 m_emit = new CEmit();
4823 /* Successful initialization flag. */
4827 /***************************************************************
4830 **************************************************************/
4834 throws java.io.IOException, java.io.FileNotFoundException
4836 if (false == m_init_flag)
4838 CError.parse_error(CError.E_INIT,0);
4843 CUtility.assert(null != this);
4844 CUtility.assert(null != m_outstream);
4845 CUtility.assert(null != m_input);
4846 CUtility.assert(null != m_tokens);
4847 CUtility.assert(null != m_spec);
4848 CUtility.assert(m_init_flag);
4851 /*m_emit.emit_imports(m_spec,m_outstream);*/
4853 if (m_spec.m_verbose)
4855 System.out.println("Processing first section -- user code.");
4858 if (m_input.m_eof_reached)
4860 CError.parse_error(CError.E_EOF,m_input.m_line_number);
4863 if (m_spec.m_verbose)
4865 System.out.println("Processing second section -- "
4866 + "JLex declarations.");
4869 if (m_input.m_eof_reached)
4871 CError.parse_error(CError.E_EOF,m_input.m_line_number);
4874 if (m_spec.m_verbose)
4876 System.out.println("Processing third section -- lexical rules.");
4879 if (CUtility.DO_DEBUG)
4884 if (m_spec.m_verbose)
4886 System.out.println("Outputting lexical analyzer code.");
4888 m_emit.emit(m_spec,m_outstream);
4890 if (m_spec.m_verbose && true == CUtility.OLD_DUMP_DEBUG)
4895 m_outstream.close();
4898 /***************************************************************
4900 Description: Process first section of specification,
4901 echoing it into output file.
4902 **************************************************************/
4903 private void userCode
4906 throws java.io.IOException
4910 if (false == m_init_flag)
4912 CError.parse_error(CError.E_INIT,0);
4917 CUtility.assert(null != this);
4918 CUtility.assert(null != m_outstream);
4919 CUtility.assert(null != m_input);
4920 CUtility.assert(null != m_tokens);
4921 CUtility.assert(null != m_spec);
4924 if (m_input.m_eof_reached)
4926 CError.parse_error(CError.E_EOF,0);
4931 if (m_input.getLine())
4934 CError.parse_error(CError.E_EOF,0);
4937 if (2 <= m_input.m_line_read
4938 && '%' == m_input.m_line[0]
4939 && '%' == m_input.m_line[1])
4941 /* Discard remainder of line. */
4942 m_input.m_line_index = m_input.m_line_read;
4946 m_outstream.writeBytes(new String(m_input.m_line,0,
4947 m_input.m_line_read));
4951 /***************************************************************
4953 **************************************************************/
4954 private char[] getName
4961 /* Skip white space. */
4962 while (m_input.m_line_index < m_input.m_line_read
4963 && true == CUtility.isspace(m_input.m_line[m_input.m_line_index]))
4965 ++m_input.m_line_index;
4969 if (m_input.m_line_index >= m_input.m_line_read)
4971 CError.parse_error(CError.E_DIRECT,0);
4974 /* Determine length. */
4975 elem = m_input.m_line_index;
4976 while (elem < m_input.m_line_read
4977 && false == CUtility.isnewline(m_input.m_line[elem]))
4982 /* Allocate non-terminated buffer of exact length. */
4983 buffer = new char[elem - m_input.m_line_index];
4987 while (m_input.m_line_index < m_input.m_line_read
4988 && false == CUtility.isnewline(m_input.m_line[m_input.m_line_index]))
4990 buffer[elem] = m_input.m_line[m_input.m_line_index];
4992 ++m_input.m_line_index;
4998 private final int CLASS_CODE = 0;
4999 private final int INIT_CODE = 1;
5000 private final int EOF_CODE = 2;
5001 private final int INIT_THROW_CODE = 3;
5002 private final int YYLEX_THROW_CODE = 4;
5003 private final int EOF_THROW_CODE = 5;
5004 private final int EOF_VALUE_CODE = 6;
5006 /***************************************************************
5009 **************************************************************/
5010 private char[] packCode
5018 throws java.io.IOException
5022 CUtility.assert(INIT_CODE == specified
5023 || CLASS_CODE == specified
5024 || EOF_CODE == specified
5025 || EOF_VALUE_CODE == specified
5026 || INIT_THROW_CODE == specified
5027 || YYLEX_THROW_CODE == specified
5028 || EOF_THROW_CODE == specified);
5031 if (0 != CUtility.charncmp(m_input.m_line,
5035 start_dir.length - 1))
5037 CError.parse_error(CError.E_INTERNAL,0);
5040 if (null == prev_code)
5042 prev_code = new char[BUFFER_SIZE];
5046 if (prev_read >= prev_code.length)
5048 prev_code = CUtility.doubleSize(prev_code);
5051 m_input.m_line_index = start_dir.length - 1;
5054 while (m_input.m_line_index >= m_input.m_line_read)
5056 if (m_input.getLine())
5058 CError.parse_error(CError.E_EOF,m_input.m_line_number);
5061 if (0 == CUtility.charncmp(m_input.m_line,
5065 end_dir.length - 1))
5067 m_input.m_line_index = end_dir.length - 1;
5072 m_spec.m_class_read = prev_read;
5076 m_spec.m_init_read = prev_read;
5080 m_spec.m_eof_read = prev_read;
5083 case EOF_VALUE_CODE:
5084 m_spec.m_eof_value_read = prev_read;
5087 case INIT_THROW_CODE:
5088 m_spec.m_init_throw_read = prev_read;
5091 case YYLEX_THROW_CODE:
5092 m_spec.m_yylex_throw_read = prev_read;
5095 case EOF_THROW_CODE:
5096 m_spec.m_eof_throw_read = prev_read;
5100 CError.parse_error(CError.E_INTERNAL,m_input.m_line_number);
5108 while (m_input.m_line_index < m_input.m_line_read)
5110 prev_code[prev_read] = m_input.m_line[m_input.m_line_index];
5112 ++m_input.m_line_index;
5114 if (prev_read >= prev_code.length)
5116 prev_code = CUtility.doubleSize(prev_code);
5122 /***************************************************************
5123 Member Variables: JLex directives.
5124 **************************************************************/
5125 private char m_state_dir[] = {
5131 private char m_char_dir[] = {
5137 private char m_line_dir[] = {
5143 private char m_cup_dir[] = {
5149 private char m_class_dir[] = {
5155 private char m_implements_dir[] = {
5156 '%', 'i', 'm', 'p', 'l', 'e', 'm', 'e', 'n', 't', 's',
5160 private char m_function_dir[] = {
5167 private char m_type_dir[] = {
5173 private char m_integer_dir[] = {
5180 private char m_intwrap_dir[] = {
5187 private char m_full_dir[] = {
5193 private char m_unicode_dir[] = {
5200 private char m_ignorecase_dir[] = {
5208 private char m_notunix_dir[] = {
5215 private char m_init_code_dir[] = {
5221 private char m_init_code_end_dir[] = {
5227 private char m_init_throw_code_dir[] = {
5235 private char m_init_throw_code_end_dir[] = {
5243 private char m_yylex_throw_code_dir[] = {
5251 private char m_yylex_throw_code_end_dir[] = {
5259 private char m_eof_code_dir[] = {
5265 private char m_eof_code_end_dir[] = {
5271 private char m_eof_value_code_dir[] = {
5278 private char m_eof_value_code_end_dir[] = {
5285 private char m_eof_throw_code_dir[] = {
5293 private char m_eof_throw_code_end_dir[] = {
5301 private char m_class_code_dir[] = {
5306 private char m_class_code_end_dir[] = {
5311 private char m_yyeof_dir[] = {
5317 private char m_public_dir[] = {
5323 /***************************************************************
5324 Function: userDeclare
5326 **************************************************************/
5327 private void userDeclare
5330 throws java.io.IOException
5336 CUtility.assert(null != this);
5337 CUtility.assert(null != m_outstream);
5338 CUtility.assert(null != m_input);
5339 CUtility.assert(null != m_tokens);
5340 CUtility.assert(null != m_spec);
5343 if (m_input.m_eof_reached)
5346 CError.parse_error(CError.E_EOF,
5347 m_input.m_line_number);
5350 while (false == m_input.getLine())
5352 /* Look for double percent. */
5353 if (2 <= m_input.m_line_read
5354 && '%' == m_input.m_line[0]
5355 && '%' == m_input.m_line[1])
5357 /* Mess around with line. */
5358 for (elem = 0; elem < m_input.m_line.length - 2; ++elem)
5360 m_input.m_line[elem] = m_input.m_line[elem + 2];
5362 m_input.m_line_read = m_input.m_line_read - 2;
5364 m_input.m_pushback_line = true;
5365 /* Check for and discard empty line. */
5366 if (0 == m_input.m_line_read
5367 || '\r' == m_input.m_line[0]
5368 || '\n' == m_input.m_line[0])
5370 m_input.m_pushback_line = false;
5376 if (0 == m_input.m_line_read)
5381 if ('%' == m_input.m_line[0])
5383 /* Special lex declarations. */
5384 if (1 >= m_input.m_line_read)
5386 CError.parse_error(CError.E_DIRECT,
5387 m_input.m_line_number);
5391 switch (m_input.m_line[1])
5394 if (0 == CUtility.charncmp(m_input.m_line,
5398 m_class_code_dir.length - 1))
5400 m_spec.m_class_code = packCode(m_class_code_dir,
5401 m_class_code_end_dir,
5402 m_spec.m_class_code,
5403 m_spec.m_class_read,
5408 /* Bad directive. */
5409 CError.parse_error(CError.E_DIRECT,
5410 m_input.m_line_number);
5414 if (0 == CUtility.charncmp(m_input.m_line,
5418 m_char_dir.length - 1))
5420 /* Set line counting to ON. */
5421 m_input.m_line_index = m_char_dir.length;
5422 m_spec.m_count_chars = true;
5425 else if (0 == CUtility.charncmp(m_input.m_line,
5429 m_class_dir.length - 1))
5431 m_input.m_line_index = m_class_dir.length;
5432 m_spec.m_class_name = getName();
5435 else if (0 == CUtility.charncmp(m_input.m_line,
5439 m_cup_dir.length - 1))
5441 /* Set Java CUP compatibility to ON. */
5442 m_input.m_line_index = m_cup_dir.length;
5443 m_spec.m_cup_compatible = true;
5447 /* Bad directive. */
5448 CError.parse_error(CError.E_DIRECT,
5449 m_input.m_line_number);
5453 if (0 == CUtility.charncmp(m_input.m_line,
5457 m_eof_code_dir.length - 1))
5459 m_spec.m_eof_code = packCode(m_eof_code_dir,
5466 else if (0 == CUtility.charncmp(m_input.m_line,
5468 m_eof_value_code_dir,
5470 m_eof_value_code_dir.length - 1))
5472 m_spec.m_eof_value_code = packCode(m_eof_value_code_dir,
5473 m_eof_value_code_end_dir,
5474 m_spec.m_eof_value_code,
5475 m_spec.m_eof_value_read,
5479 else if (0 == CUtility.charncmp(m_input.m_line,
5481 m_eof_throw_code_dir,
5483 m_eof_throw_code_dir.length - 1))
5485 m_spec.m_eof_throw_code = packCode(m_eof_throw_code_dir,
5486 m_eof_throw_code_end_dir,
5487 m_spec.m_eof_throw_code,
5488 m_spec.m_eof_throw_read,
5493 /* Bad directive. */
5494 CError.parse_error(CError.E_DIRECT,
5495 m_input.m_line_number);
5499 if (0 == CUtility.charncmp(m_input.m_line,
5503 m_function_dir.length - 1))
5505 /* Set line counting to ON. */
5506 m_input.m_line_index = m_function_dir.length;
5507 m_spec.m_function_name = getName();
5510 else if (0 == CUtility.charncmp(m_input.m_line,
5514 m_full_dir.length - 1))
5516 m_input.m_line_index = m_full_dir.length;
5517 m_spec.m_dtrans_ncols = CUtility.MAX_EIGHT_BIT + 1;
5521 /* Bad directive. */
5522 CError.parse_error(CError.E_DIRECT,
5523 m_input.m_line_number);
5527 if (0 == CUtility.charncmp(m_input.m_line,
5531 m_integer_dir.length - 1))
5533 /* Set line counting to ON. */
5534 m_input.m_line_index = m_integer_dir.length;
5535 m_spec.m_integer_type = true;
5538 else if (0 == CUtility.charncmp(m_input.m_line,
5542 m_intwrap_dir.length - 1))
5544 /* Set line counting to ON. */
5545 m_input.m_line_index = m_integer_dir.length;
5546 m_spec.m_intwrap_type = true;
5549 else if (0 == CUtility.charncmp(m_input.m_line,
5553 m_init_code_dir.length - 1))
5555 m_spec.m_init_code = packCode(m_init_code_dir,
5556 m_init_code_end_dir,
5562 else if (0 == CUtility.charncmp(m_input.m_line,
5564 m_init_throw_code_dir,
5566 m_init_throw_code_dir.length - 1))
5568 m_spec.m_init_throw_code = packCode(m_init_throw_code_dir,
5569 m_init_throw_code_end_dir,
5570 m_spec.m_init_throw_code,
5571 m_spec.m_init_throw_read,
5575 else if (0 == CUtility.charncmp(m_input.m_line,
5579 m_implements_dir.length - 1))
5581 m_input.m_line_index = m_implements_dir.length;
5582 m_spec.m_implements_name = getName();
5585 else if (0 == CUtility.charncmp(m_input.m_line,
5589 m_ignorecase_dir.length-1))
5591 /* Set m_ignorecase to ON. */
5592 m_input.m_line_index = m_ignorecase_dir.length;
5593 m_spec.m_ignorecase = true;
5597 /* Bad directive. */
5598 CError.parse_error(CError.E_DIRECT,
5599 m_input.m_line_number);
5603 if (0 == CUtility.charncmp(m_input.m_line,
5607 m_line_dir.length - 1))
5609 /* Set line counting to ON. */
5610 m_input.m_line_index = m_line_dir.length;
5611 m_spec.m_count_lines = true;
5615 /* Bad directive. */
5616 CError.parse_error(CError.E_DIRECT,
5617 m_input.m_line_number);
5621 if (0 == CUtility.charncmp(m_input.m_line,
5625 m_notunix_dir.length - 1))
5627 /* Set line counting to ON. */
5628 m_input.m_line_index = m_notunix_dir.length;
5629 m_spec.m_unix = false;
5633 /* Bad directive. */
5634 CError.parse_error(CError.E_DIRECT,
5635 m_input.m_line_number);
5639 if (0 == CUtility.charncmp(m_input.m_line,
5643 m_public_dir.length - 1))
5645 /* Set public flag. */
5646 m_input.m_line_index = m_public_dir.length;
5647 m_spec.m_public = true;
5651 /* Bad directive. */
5652 CError.parse_error(CError.E_DIRECT,
5653 m_input.m_line_number);
5657 if (0 == CUtility.charncmp(m_input.m_line,
5661 m_state_dir.length - 1))
5663 /* Recognize state list. */
5664 m_input.m_line_index = m_state_dir.length;
5669 /* Undefined directive. */
5670 CError.parse_error(CError.E_DIRECT,
5671 m_input.m_line_number);
5675 if (0 == CUtility.charncmp(m_input.m_line,
5679 m_type_dir.length - 1))
5681 /* Set Java CUP compatibility to ON. */
5682 m_input.m_line_index = m_type_dir.length;
5683 m_spec.m_type_name = getName();
5687 /* Undefined directive. */
5688 CError.parse_error(CError.E_DIRECT,
5689 m_input.m_line_number);
5693 if (0 == CUtility.charncmp(m_input.m_line,
5697 m_unicode_dir.length - 1))
5699 m_input.m_line_index = m_unicode_dir.length;
5700 /* UNDONE: What to do here? */
5704 /* Bad directive. */
5705 CError.parse_error(CError.E_DIRECT,
5706 m_input.m_line_number);
5710 if (0 == CUtility.charncmp(m_input.m_line,
5714 m_yyeof_dir.length - 1))
5716 m_input.m_line_index = m_yyeof_dir.length;
5717 m_spec.m_yyeof = true;
5719 } else if (0 == CUtility.charncmp(m_input.m_line,
5721 m_yylex_throw_code_dir,
5723 m_yylex_throw_code_dir.length - 1))
5725 m_spec.m_yylex_throw_code = packCode(m_yylex_throw_code_dir,
5726 m_yylex_throw_code_end_dir,
5727 m_spec.m_yylex_throw_code,
5728 m_spec.m_yylex_throw_read,
5734 /* Bad directive. */
5735 CError.parse_error(CError.E_DIRECT,
5736 m_input.m_line_number);
5740 /* Undefined directive. */
5741 CError.parse_error(CError.E_DIRECT,
5742 m_input.m_line_number);
5748 /* Regular expression macro. */
5749 m_input.m_line_index = 0;
5753 if (CUtility.OLD_DEBUG)
5755 System.out.println("Line number "
5756 + m_input.m_line_number + ":");
5757 System.out.print(new String(m_input.m_line,
5758 0,m_input.m_line_read));
5763 /***************************************************************
5765 Description: Processes third section of JLex
5766 specification and creates minimized transition table.
5767 **************************************************************/
5768 private void userRules
5771 throws java.io.IOException
5775 if (false == m_init_flag)
5777 CError.parse_error(CError.E_INIT,0);
5782 CUtility.assert(null != this);
5783 CUtility.assert(null != m_outstream);
5784 CUtility.assert(null != m_input);
5785 CUtility.assert(null != m_tokens);
5786 CUtility.assert(null != m_spec);
5789 /* UNDONE: Need to handle states preceding rules. */
5791 if (m_spec.m_verbose)
5793 System.out.println("Creating NFA machine representation.");
5795 m_makeNfa.thompson(this,m_spec,m_input);
5801 CUtility.assert(END_OF_INPUT == m_spec.m_current_token);
5804 if (m_spec.m_verbose)
5806 System.out.println("Creating DFA transition table.");
5808 m_nfa2dfa.make_dfa(this,m_spec);
5810 if (CUtility.FOODEBUG) {
5814 if (m_spec.m_verbose)
5816 System.out.println("Minimizing DFA transition table.");
5818 m_minimize.min_dfa(m_spec);
5821 /***************************************************************
5823 Description: Debuggng routine that outputs readable form
5825 **************************************************************/
5826 private void printccl
5833 System.out.print(" [");
5834 for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
5836 if (set.contains(i))
5838 System.out.print(interp_int(i));
5841 System.out.print(']');
5844 /***************************************************************
5847 **************************************************************/
5857 return (new String("--"));
5860 index = m_spec.m_nfa_states.indexOf(state);
5862 return ((new Integer(index)).toString());
5865 /***************************************************************
5866 Function: interp_int
5868 **************************************************************/
5869 private String interp_int
5877 return (new String("\\b"));
5880 return (new String("\\t"));
5883 return (new String("\\n"));
5886 return (new String("\\f"));
5889 return (new String("\\r"));
5892 return (new String("\\ "));
5895 return ((new Character((char) i)).toString());
5899 /***************************************************************
5902 **************************************************************/
5917 System.out.println("--------------------- NFA -----------------------");
5919 size = m_spec.m_nfa_states.size();
5920 for (elem = 0; elem < size; ++elem)
5922 nfa = (CNfa) m_spec.m_nfa_states.elementAt(elem);
5924 System.out.print("Nfa state " + plab(nfa) + ": ");
5926 if (null == nfa.m_next)
5928 System.out.print("(TERMINAL)");
5932 System.out.print("--> " + plab(nfa.m_next));
5933 System.out.print("--> " + plab(nfa.m_next2));
5938 printccl(nfa.m_set);
5942 System.out.print(" EPSILON ");
5946 System.out.print(" " + interp_int(nfa.m_edge));
5953 System.out.print(" (START STATE)");
5956 if (null != nfa.m_accept)
5958 System.out.print(" accepting "
5959 + ((0 != (nfa.m_anchor & CSpec.START)) ? "^" : "")
5961 + (new String(nfa.m_accept.m_action,0,
5962 nfa.m_accept.m_action_read))
5964 + ((0 != (nfa.m_anchor & CSpec.END)) ? "$" : ""));
5967 System.out.println("");
5970 states = m_spec.m_states.keys();
5971 while (states.hasMoreElements())
5973 state = (String) states.nextElement();
5974 index = (Integer) m_spec.m_states.get(state);
5978 CUtility.assert(null != state);
5979 CUtility.assert(null != index);
5982 System.out.println("State \"" + state
5983 + "\" has identifying index "
5984 + index.toString() + ".");
5985 System.out.print("\tStart states of matching rules: ");
5987 i = index.intValue();
5988 vsize = m_spec.m_state_rules[i].size();
5990 for (j = 0; j < vsize; ++j)
5992 nfa = (CNfa) m_spec.m_state_rules[i].elementAt(j);
5994 System.out.print(m_spec.m_nfa_states.indexOf(nfa) + " ");
5997 System.out.println("");
6000 System.out.println("-------------------- NFA ----------------------");
6003 /***************************************************************
6005 Description: Parses the state area of a rule,
6006 from the beginning of a line.
6007 < state1, state2 ... > regular_expression { action }
6008 Returns null on only EOF. Returns all_states,
6009 initialied properly to correspond to all states,
6010 if no states are found.
6011 Special Notes: This function treats commas as optional
6012 and permits states to be spread over multiple lines.
6013 **************************************************************/
6014 private JavaLexBitSet all_states = null;
6015 JavaLexBitSet getStates
6018 throws java.io.IOException
6022 JavaLexBitSet states;
6030 CUtility.assert(null != this);
6031 CUtility.assert(null != m_outstream);
6032 CUtility.assert(null != m_input);
6033 CUtility.assert(null != m_tokens);
6034 CUtility.assert(null != m_spec);
6039 /* Skip white space. */
6040 while (CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6042 ++m_input.m_line_index;
6044 while (m_input.m_line_index >= m_input.m_line_read)
6046 /* Must just be an empty line. */
6047 if (m_input.getLine())
6055 /* Look for states. */
6056 if ('<' == m_input.m_line[m_input.m_line_index])
6058 ++m_input.m_line_index;
6060 states = new JavaLexBitSet();
6065 /* We may have reached the end of the line. */
6066 while (m_input.m_line_index >= m_input.m_line_read)
6068 if (m_input.getLine())
6071 CError.parse_error(CError.E_EOF,m_input.m_line_number);
6078 /* Skip white space. */
6079 while (CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6081 ++m_input.m_line_index;
6083 while (m_input.m_line_index >= m_input.m_line_read)
6085 if (m_input.getLine())
6088 CError.parse_error(CError.E_EOF,m_input.m_line_number);
6094 if (',' != m_input.m_line[m_input.m_line_index])
6099 ++m_input.m_line_index;
6102 if ('>' == m_input.m_line[m_input.m_line_index])
6104 ++m_input.m_line_index;
6105 if (m_input.m_line_index < m_input.m_line_read)
6107 m_advance_stop = true;
6112 /* Read in state name. */
6113 start_state = m_input.m_line_index;
6114 while (false == CUtility.isspace(m_input.m_line[m_input.m_line_index])
6115 && ',' != m_input.m_line[m_input.m_line_index]
6116 && '>' != m_input.m_line[m_input.m_line_index])
6118 ++m_input.m_line_index;
6120 if (m_input.m_line_index >= m_input.m_line_read)
6122 /* End of line means end of state name. */
6126 count_state = m_input.m_line_index - start_state;
6128 /* Save name after checking definition. */
6129 name = new String(m_input.m_line,
6132 index = (Integer) m_spec.m_states.get(name);
6135 /* Uninitialized state. */
6136 System.out.println("Uninitialized State Name: " + name);
6137 CError.parse_error(CError.E_STATE,m_input.m_line_number);
6139 states.set(index.intValue());
6143 if (null == all_states)
6145 all_states = new JavaLexBitSet();
6147 size = m_spec.m_states.size();
6148 for (i = 0; i < size; ++i)
6154 if (m_input.m_line_index < m_input.m_line_read)
6156 m_advance_stop = true;
6161 /********************************************************
6162 Function: expandMacro
6163 Description: Returns false on error, true otherwise.
6164 *******************************************************/
6165 private boolean expandMacro
6182 CUtility.assert(null != this);
6183 CUtility.assert(null != m_outstream);
6184 CUtility.assert(null != m_input);
6185 CUtility.assert(null != m_tokens);
6186 CUtility.assert(null != m_spec);
6189 /* Check for macro. */
6190 if ('{' != m_input.m_line[m_input.m_line_index])
6192 CError.parse_error(CError.E_INTERNAL,m_input.m_line_number);
6196 start_macro = m_input.m_line_index;
6197 elem = m_input.m_line_index + 1;
6198 if (elem >= m_input.m_line_read)
6200 CError.impos("Unfinished macro name");
6204 /* Get macro name. */
6206 while ('}' != m_input.m_line[elem])
6209 if (elem >= m_input.m_line_read)
6211 CError.impos("Unfinished macro name at line "
6212 + m_input.m_line_number);
6216 count_name = elem - start_name;
6219 /* Check macro name. */
6220 if (0 == count_name)
6222 CError.impos("Nonexistent macro name");
6229 CUtility.assert(0 < count_name);
6232 /* Retrieve macro definition. */
6233 name = new String(m_input.m_line,start_name,count_name);
6234 def = (String) m_spec.m_macros.get(name);
6237 /*CError.impos("Undefined macro \"" + name + "\".");*/
6238 System.out.println("Error: Undefined macro \"" + name + "\".");
6239 CError.parse_error(CError.E_NOMAC, m_input.m_line_number);
6242 if (CUtility.OLD_DUMP_DEBUG)
6244 System.out.println("expanded escape: " + def);
6247 /* Replace macro in new buffer,
6248 beginning by copying first part of line buffer. */
6249 replace = new char[m_input.m_line.length];
6250 for (rep_elem = 0; rep_elem < start_macro; ++rep_elem)
6252 replace[rep_elem] = m_input.m_line[rep_elem];
6256 CUtility.assert(rep_elem < replace.length);
6260 /* Copy macro definition. */
6261 if (rep_elem >= replace.length)
6263 replace = CUtility.doubleSize(replace);
6265 for (def_elem = 0; def_elem < def.length(); ++def_elem)
6267 replace[rep_elem] = def.charAt(def_elem);
6270 if (rep_elem >= replace.length)
6272 replace = CUtility.doubleSize(replace);
6276 /* Copy last part of line. */
6277 if (rep_elem >= replace.length)
6279 replace = CUtility.doubleSize(replace);
6281 for (elem = end_macro + 1; elem < m_input.m_line_read; ++elem)
6283 replace[rep_elem] = m_input.m_line[elem];
6286 if (rep_elem >= replace.length)
6288 replace = CUtility.doubleSize(replace);
6292 /* Replace buffer. */
6293 m_input.m_line = replace;
6294 m_input.m_line_read = rep_elem;
6296 if (CUtility.OLD_DEBUG)
6298 System.out.println(new String(m_input.m_line,0,m_input.m_line_read));
6303 /***************************************************************
6305 Description: Saves macro definition of form:
6306 macro_name = macro_definition
6307 **************************************************************/
6308 private void saveMacro
6323 CUtility.assert(null != this);
6324 CUtility.assert(null != m_outstream);
6325 CUtility.assert(null != m_input);
6326 CUtility.assert(null != m_tokens);
6327 CUtility.assert(null != m_spec);
6330 /* Macro declarations are of the following form:
6331 macro_name macro_definition */
6335 /* Skip white space preceding macro name. */
6336 while (CUtility.isspace(m_input.m_line[elem]))
6339 if (elem >= m_input.m_line_read)
6341 /* End of line has been reached,
6342 and line was found to be empty. */
6347 /* Read macro name. */
6349 while (false == CUtility.isspace(m_input.m_line[elem])
6350 && '=' != m_input.m_line[elem])
6353 if (elem >= m_input.m_line_read)
6355 /* Macro name but no associated definition. */
6356 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6359 count_name = elem - start_name;
6361 /* Check macro name. */
6362 if (0 == count_name)
6364 /* Nonexistent macro name. */
6365 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6368 /* Skip white space between name and definition. */
6369 while (CUtility.isspace(m_input.m_line[elem]))
6372 if (elem >= m_input.m_line_read)
6374 /* Macro name but no associated definition. */
6375 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6379 if ('=' == m_input.m_line[elem])
6382 if (elem >= m_input.m_line_read)
6384 /* Macro name but no associated definition. */
6385 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6389 /* Skip white space between name and definition. */
6390 while (CUtility.isspace(m_input.m_line[elem]))
6393 if (elem >= m_input.m_line_read)
6395 /* Macro name but no associated definition. */
6396 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6400 /* Read macro definition. */
6405 while (false == CUtility.isspace(m_input.m_line[elem])
6408 || true == saw_escape)
6410 if ('\"' == m_input.m_line[elem] && false == saw_escape)
6412 in_quote = !in_quote;
6415 if ('\\' == m_input.m_line[elem] && false == saw_escape)
6423 if (false == saw_escape && false == in_quote) { // CSA, 24-jul-99
6424 if ('[' == m_input.m_line[elem] && false == in_ccl)
6426 if (']' == m_input.m_line[elem] && true == in_ccl)
6431 if (elem >= m_input.m_line_read)
6437 count_def = elem - start_def;
6439 /* Check macro definition. */
6442 /* Nonexistent macro name. */
6443 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6449 CUtility.assert(0 < count_def);
6450 CUtility.assert(0 < count_name);
6451 CUtility.assert(null != m_spec.m_macros);
6454 if (CUtility.OLD_DEBUG)
6456 System.out.println("macro name \""
6457 + new String(m_input.m_line,start_name,count_name)
6459 System.out.println("macro definition \""
6460 + new String(m_input.m_line,start_def,count_def)
6464 /* Add macro name and definition to table. */
6465 m_spec.m_macros.put(new String(m_input.m_line,start_name,count_name),
6466 new String(m_input.m_line,start_def,count_def));
6469 /***************************************************************
6470 Function: saveStates
6471 Description: Takes state declaration and makes entries
6472 for them in state hashtable in CSpec structure.
6473 State declaration should be of the form:
6474 %state name0[, name1, name2 ...]
6475 (But commas are actually optional as long as there is
6476 white space in between them.)
6477 **************************************************************/
6478 private void saveStates
6487 CUtility.assert(null != this);
6488 CUtility.assert(null != m_outstream);
6489 CUtility.assert(null != m_input);
6490 CUtility.assert(null != m_tokens);
6491 CUtility.assert(null != m_spec);
6495 if (m_input.m_eof_reached)
6503 CUtility.assert('%' == m_input.m_line[0]);
6504 CUtility.assert('s' == m_input.m_line[1]);
6505 CUtility.assert(m_input.m_line_index <= m_input.m_line_read);
6506 CUtility.assert(0 <= m_input.m_line_index);
6507 CUtility.assert(0 <= m_input.m_line_read);
6510 /* Blank line? No states? */
6511 if (m_input.m_line_index >= m_input.m_line_read)
6516 while (m_input.m_line_index < m_input.m_line_read)
6518 if (CUtility.OLD_DEBUG)
6520 System.out.println("line read " + m_input.m_line_read
6521 + "\tline index = " + m_input.m_line_index);
6524 /* Skip white space. */
6525 while (CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6527 ++m_input.m_line_index;
6528 if (m_input.m_line_index >= m_input.m_line_read)
6530 /* No more states to be found. */
6535 /* Look for state name. */
6536 start_state = m_input.m_line_index;
6537 while (false == CUtility.isspace(m_input.m_line[m_input.m_line_index])
6538 && ',' != m_input.m_line[m_input.m_line_index])
6540 ++m_input.m_line_index;
6541 if (m_input.m_line_index >= m_input.m_line_read)
6543 /* End of line and end of state name. */
6547 count_state = m_input.m_line_index - start_state;
6549 if (CUtility.OLD_DEBUG)
6551 System.out.println("State name \""
6552 + new String(m_input.m_line,start_state,count_state)
6554 System.out.println("Integer index \""
6555 + m_spec.m_states.size()
6559 /* Enter new state name, along with unique index. */
6560 m_spec.m_states.put(new String(m_input.m_line,start_state,count_state),
6561 new Integer(m_spec.m_states.size()));
6564 if (',' == m_input.m_line[m_input.m_line_index])
6566 ++m_input.m_line_index;
6567 if (m_input.m_line_index >= m_input.m_line_read)
6576 /********************************************************
6577 Function: expandEscape
6578 Description: Takes escape sequence and returns
6579 corresponding character code.
6580 *******************************************************/
6581 private char expandEscape
6590 CUtility.assert(m_input.m_line_index < m_input.m_line_read);
6591 CUtility.assert(0 < m_input.m_line_read);
6592 CUtility.assert(0 <= m_input.m_line_index);
6595 if ('\\' != m_input.m_line[m_input.m_line_index])
6597 ++m_input.m_line_index;
6598 return m_input.m_line[m_input.m_line_index - 1];
6602 ++m_input.m_line_index;
6603 switch (CUtility.toupper(m_input.m_line[m_input.m_line_index]))
6606 ++m_input.m_line_index;
6610 ++m_input.m_line_index;
6614 ++m_input.m_line_index;
6618 ++m_input.m_line_index;
6622 ++m_input.m_line_index;
6626 ++m_input.m_line_index;
6627 r = (char) (CUtility.toupper(m_input.m_line[m_input.m_line_index])
6629 ++m_input.m_line_index;
6633 ++m_input.m_line_index;
6635 if (CUtility.ishexdigit(m_input.m_line[m_input.m_line_index]))
6637 r = CUtility.hex2bin(m_input.m_line[m_input.m_line_index]);
6638 ++m_input.m_line_index;
6640 if (CUtility.ishexdigit(m_input.m_line[m_input.m_line_index]))
6642 r = (char) (r << 4);
6643 r = (char) (r | CUtility.hex2bin(m_input.m_line[m_input.m_line_index]));
6644 ++m_input.m_line_index;
6646 if (CUtility.ishexdigit(m_input.m_line[m_input.m_line_index]))
6648 r = (char) (r << 4);
6649 r = (char) (r | CUtility.hex2bin(m_input.m_line[m_input.m_line_index]));
6650 ++m_input.m_line_index;
6655 if (false == CUtility.isoctdigit(m_input.m_line[m_input.m_line_index]))
6657 r = m_input.m_line[m_input.m_line_index];
6658 ++m_input.m_line_index;
6662 r = CUtility.oct2bin(m_input.m_line[m_input.m_line_index]);
6663 ++m_input.m_line_index;
6665 if (CUtility.isoctdigit(m_input.m_line[m_input.m_line_index]))
6667 r = (char) (r << 3);
6668 r = (char) (r | CUtility.oct2bin(m_input.m_line[m_input.m_line_index]));
6669 ++m_input.m_line_index;
6672 if (CUtility.isoctdigit(m_input.m_line[m_input.m_line_index]))
6674 r = (char) (r << 3);
6675 r = (char) (r | CUtility.oct2bin(m_input.m_line[m_input.m_line_index]));
6676 ++m_input.m_line_index;
6684 /********************************************************
6685 Function: packAccept
6686 Description: Packages and returns CAccept
6687 for action next in input stream.
6688 *******************************************************/
6692 throws java.io.IOException
6699 boolean instarcomment;
6700 boolean inslashcomment;
6704 action = new char[BUFFER_SIZE];
6709 CUtility.assert(null != this);
6710 CUtility.assert(null != m_outstream);
6711 CUtility.assert(null != m_input);
6712 CUtility.assert(null != m_tokens);
6713 CUtility.assert(null != m_spec);
6716 /* Get a new line, if needed. */
6717 while (m_input.m_line_index >= m_input.m_line_read)
6719 if (m_input.getLine())
6721 CError.parse_error(CError.E_EOF,m_input.m_line_number);
6726 /* Look for beginning of action. */
6727 while (CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6729 ++m_input.m_line_index;
6731 /* Get a new line, if needed. */
6732 while (m_input.m_line_index >= m_input.m_line_read)
6734 if (m_input.getLine())
6736 CError.parse_error(CError.E_EOF,m_input.m_line_number);
6742 /* Look for brackets. */
6743 if ('{' != m_input.m_line[m_input.m_line_index])
6745 CError.parse_error(CError.E_BRACE,m_input.m_line_number);
6748 /* Copy new line into action buffer. */
6750 inquotes = inslashcomment = instarcomment = false;
6751 escaped = slashed = false;
6754 action[action_index] = m_input.m_line[m_input.m_line_index];
6756 /* Look for quotes. */
6757 if (inquotes && escaped)
6758 escaped=false; // only protects one char, but this is enough.
6759 else if (inquotes && '\\' == m_input.m_line[m_input.m_line_index])
6761 else if ('\"' == m_input.m_line[m_input.m_line_index])
6762 inquotes=!inquotes; // unescaped quote.
6763 /* Look for comments. */
6764 if (instarcomment) { // inside "/*" comment; look for "*/"
6765 if (slashed && '/' == m_input.m_line[m_input.m_line_index])
6766 instarcomment = slashed = false;
6767 else // note that inside a star comment, slashed means starred
6768 slashed = ('*' == m_input.m_line[m_input.m_line_index]);
6769 } else if (!inslashcomment) { // not in comment, look for /* or //
6771 (slashed && '/' == m_input.m_line[m_input.m_line_index]);
6773 (slashed && '*' == m_input.m_line[m_input.m_line_index]);
6774 slashed = ('/' == m_input.m_line[m_input.m_line_index]);
6777 /* Look for brackets. */
6778 if (!inquotes && !instarcomment && !inslashcomment) {
6779 if ('{' == m_input.m_line[m_input.m_line_index])
6783 else if ('}' == m_input.m_line[m_input.m_line_index])
6790 ++m_input.m_line_index;
6798 /* Double the buffer size, if needed. */
6799 if (action_index >= action.length)
6801 action = CUtility.doubleSize(action);
6804 ++m_input.m_line_index;
6805 /* Get a new line, if needed. */
6806 while (m_input.m_line_index >= m_input.m_line_read)
6808 inslashcomment = slashed = false;
6809 if (inquotes) { // non-fatal
6810 CError.parse_error(CError.E_NEWLINE,m_input.m_line_number);
6813 if (m_input.getLine())
6815 CError.parse_error(CError.E_SYNTAX,m_input.m_line_number);
6821 accept = new CAccept(action,action_index,m_input.m_line_number);
6825 CUtility.assert(null != accept);
6828 if (CUtility.DESCENT_DEBUG)
6830 System.out.print("Accepting action:");
6831 System.out.println(new String(accept.m_action,0,accept.m_action_read));
6837 /********************************************************
6839 Description: Returns code for next token.
6840 *******************************************************/
6841 private boolean m_advance_stop = false;
6845 throws java.io.IOException
6847 boolean saw_escape = false;
6850 /*if (m_input.m_line_index > m_input.m_line_read) {
6851 System.out.println("m_input.m_line_index = " + m_input.m_line_index);
6852 System.out.println("m_input.m_line_read = " + m_input.m_line_read);
6853 CUtility.assert(m_input.m_line_index <= m_input.m_line_read);
6856 if (m_input.m_eof_reached)
6858 /* EOF has already been reached,
6859 so return appropriate code. */
6861 m_spec.m_current_token = END_OF_INPUT;
6862 m_spec.m_lexeme = '\0';
6863 return m_spec.m_current_token;
6866 /* End of previous regular expression?
6867 Refill line buffer? */
6868 if (EOS == m_spec.m_current_token
6870 || m_input.m_line_index >= m_input.m_line_read)
6873 if (m_spec.m_in_quote)
6875 CError.parse_error(CError.E_SYNTAX,m_input.m_line_number);
6880 if (false == m_advance_stop
6881 || m_input.m_line_index >= m_input.m_line_read)
6883 if (m_input.getLine())
6885 /* EOF has already been reached,
6886 so return appropriate code. */
6888 m_spec.m_current_token = END_OF_INPUT;
6889 m_spec.m_lexeme = '\0';
6890 return m_spec.m_current_token;
6892 m_input.m_line_index = 0;
6896 m_advance_stop = false;
6899 while (m_input.m_line_index < m_input.m_line_read
6900 && true == CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6902 ++m_input.m_line_index;
6905 if (m_input.m_line_index < m_input.m_line_read)
6912 if (CUtility.DEBUG) {
6913 CUtility.assert(m_input.m_line_index <= m_input.m_line_read);
6918 if (false == m_spec.m_in_quote
6919 && '{' == m_input.m_line[m_input.m_line_index])
6921 if (false == expandMacro())
6926 if (m_input.m_line_index >= m_input.m_line_read)
6928 m_spec.m_current_token = EOS;
6929 m_spec.m_lexeme = '\0';
6930 return m_spec.m_current_token;
6933 else if ('\"' == m_input.m_line[m_input.m_line_index])
6935 m_spec.m_in_quote = !m_spec.m_in_quote;
6936 ++m_input.m_line_index;
6938 if (m_input.m_line_index >= m_input.m_line_read)
6940 m_spec.m_current_token = EOS;
6941 m_spec.m_lexeme = '\0';
6942 return m_spec.m_current_token;
6951 if (m_input.m_line_index > m_input.m_line_read) {
6952 System.out.println("m_input.m_line_index = " + m_input.m_line_index);
6953 System.out.println("m_input.m_line_read = " + m_input.m_line_read);
6954 CUtility.assert(m_input.m_line_index <= m_input.m_line_read);
6957 /* Look for backslash, and corresponding
6959 if ('\\' == m_input.m_line[m_input.m_line_index])
6968 if (false == m_spec.m_in_quote)
6970 if (false == m_spec.m_in_ccl &&
6971 CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6973 /* White space means the end of
6974 the current regular expression. */
6976 m_spec.m_current_token = EOS;
6977 m_spec.m_lexeme = '\0';
6978 return m_spec.m_current_token;
6981 /* Process escape sequence, if needed. */
6984 m_spec.m_lexeme = expandEscape();
6988 m_spec.m_lexeme = m_input.m_line[m_input.m_line_index];
6989 ++m_input.m_line_index;
6995 && (m_input.m_line_index + 1) < m_input.m_line_read
6996 && '\"' == m_input.m_line[m_input.m_line_index + 1])
6998 m_spec.m_lexeme = '\"';
6999 m_input.m_line_index = m_input.m_line_index + 2;
7003 m_spec.m_lexeme = m_input.m_line[m_input.m_line_index];
7004 ++m_input.m_line_index;
7008 code = (Integer) m_tokens.get(new Character(m_spec.m_lexeme));
7009 if (m_spec.m_in_quote || true == saw_escape)
7011 m_spec.m_current_token = L;
7017 m_spec.m_current_token = L;
7021 m_spec.m_current_token = code.intValue();
7025 if (CCL_START == m_spec.m_current_token) m_spec.m_in_ccl = true;
7026 if (CCL_END == m_spec.m_current_token) m_spec.m_in_ccl = false;
7028 if (CUtility.FOODEBUG)
7030 System.out.println("Lexeme: " + m_spec.m_lexeme
7031 + "\tToken: " + m_spec.m_current_token
7032 + "\tIndex: " + m_input.m_line_index);
7035 return m_spec.m_current_token;
7038 /***************************************************************
7040 Description: High level debugging routine.
7041 **************************************************************/
7042 private void details
7055 System.out.println("\n\t** Macros **");
7056 names = m_spec.m_macros.keys();
7057 while (names.hasMoreElements())
7059 name = (String) names.nextElement();
7060 def = (String) m_spec.m_macros.get(name);
7064 CUtility.assert(null != name);
7065 CUtility.assert(null != def);
7068 System.out.println("Macro name \"" + name
7069 + "\" has definition \""
7073 System.out.println("\n\t** States **");
7074 states = m_spec.m_states.keys();
7075 while (states.hasMoreElements())
7077 state = (String) states.nextElement();
7078 index = (Integer) m_spec.m_states.get(state);
7082 CUtility.assert(null != state);
7083 CUtility.assert(null != index);
7086 System.out.println("State \"" + state
7087 + "\" has identifying index "
7088 + index.toString() + ".");
7091 System.out.println("\n\t** Character Counting **");
7092 if (false == m_spec.m_count_chars)
7094 System.out.println("Character counting is off.");
7100 CUtility.assert(m_spec.m_count_lines);
7103 System.out.println("Character counting is on.");
7106 System.out.println("\n\t** Line Counting **");
7107 if (false == m_spec.m_count_lines)
7109 System.out.println("Line counting is off.");
7115 CUtility.assert(m_spec.m_count_lines);
7118 System.out.println("Line counting is on.");
7121 System.out.println("\n\t** Operating System Specificity **");
7122 if (false == m_spec.m_unix)
7124 System.out.println("Not generating UNIX-specific code.");
7125 System.out.println("(This means that \"\\r\\n\" is a "
7126 + "newline, rather than \"\\n\".)");
7130 System.out.println("Generating UNIX-specific code.");
7131 System.out.println("(This means that \"\\n\" is a "
7132 + "newline, rather than \"\\r\\n\".)");
7135 System.out.println("\n\t** Java CUP Compatibility **");
7136 if (false == m_spec.m_cup_compatible)
7138 System.out.println("Generating CUP compatible code.");
7139 System.out.println("(No current results.)");
7143 System.out.println("Not generating CUP compatible code.");
7144 System.out.println("(No current results.)");
7147 if (CUtility.FOODEBUG) {
7148 if (null != m_spec.m_nfa_states && null != m_spec.m_nfa_start)
7150 System.out.println("\n\t** NFA machine **");
7155 if (null != m_spec.m_dtrans_vector)
7157 System.out.println("\n\t** DFA transition table **");
7161 /*if (null != m_spec.m_accept_vector && null != m_spec.m_anchor_array)
7163 System.out.println("\n\t** Accept States and Anchor Vector **");
7168 /***************************************************************
7170 **************************************************************/
7180 size = nfa_set.size();
7184 System.out.print("empty ");
7187 for (elem = 0; elem < size; ++elem)
7189 nfa = (CNfa) nfa_set.elementAt(elem);
7190 /*System.out.print(m_spec.m_nfa_states.indexOf(nfa) + " ");*/
7191 System.out.print(nfa.m_label + " ");
7195 /***************************************************************
7196 Function: print_header
7197 **************************************************************/
7198 private void print_header
7205 int chars_printed=0;
7207 int last_transition;
7213 System.out.println("/*---------------------- DFA -----------------------");
7215 states = m_spec.m_states.keys();
7216 while (states.hasMoreElements())
7218 state = (String) states.nextElement();
7219 index = (Integer) m_spec.m_states.get(state);
7223 CUtility.assert(null != state);
7224 CUtility.assert(null != index);
7227 System.out.println("State \"" + state
7228 + "\" has identifying index "
7229 + index.toString() + ".");
7231 i = index.intValue();
7232 if (CDTrans.F != m_spec.m_state_dtrans[i])
7234 System.out.println("\tStart index in transition table: "
7235 + m_spec.m_state_dtrans[i]);
7239 System.out.println("\tNo associated transition states.");
7243 for (i = 0; i < m_spec.m_dtrans_vector.size(); ++i)
7245 dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
7247 if (null == m_spec.m_accept_vector && null == m_spec.m_anchor_array)
7249 if (null == dtrans.m_accept)
7251 System.out.print(" * State " + i + " [nonaccepting]");
7255 System.out.print(" * State " + i
7256 + " [accepting, line "
7257 + dtrans.m_accept.m_line_number
7259 + (new String(dtrans.m_accept.m_action,0,
7260 dtrans.m_accept.m_action_read))
7262 if (CSpec.NONE != dtrans.m_anchor)
7264 System.out.print(" Anchor: "
7265 + ((0 != (dtrans.m_anchor & CSpec.START))
7267 + ((0 != (dtrans.m_anchor & CSpec.END))
7274 accept = (CAccept) m_spec.m_accept_vector.elementAt(i);
7278 System.out.print(" * State " + i + " [nonaccepting]");
7282 System.out.print(" * State " + i
7283 + " [accepting, line "
7284 + accept.m_line_number
7286 + (new String(accept.m_action,0,
7287 accept.m_action_read))
7289 if (CSpec.NONE != m_spec.m_anchor_array[i])
7291 System.out.print(" Anchor: "
7292 + ((0 != (m_spec.m_anchor_array[i] & CSpec.START))
7294 + ((0 != (m_spec.m_anchor_array[i] & CSpec.END))
7300 last_transition = -1;
7301 for (j = 0; j < m_spec.m_dtrans_ncols; ++j)
7303 if (CDTrans.F != dtrans.m_dtrans[j])
7305 if (last_transition != dtrans.m_dtrans[j])
7307 System.out.print("\n * goto " + dtrans.m_dtrans[j]
7312 str = interp_int((int) j);
7313 System.out.print(str);
7315 chars_printed = chars_printed + str.length();
7316 if (56 < chars_printed)
7318 System.out.print("\n * ");
7322 last_transition = dtrans.m_dtrans[j];
7325 System.out.println("");
7327 System.out.println(" */\n");
7332 * @(#)BitSet.java 1.12 95/12/01
7334 * Revision: 10/21/96
7335 * Modified by Elliot Joel Berk (ejberk@princeton.edu), for the purposes
7336 * of fixing a bug that was causing unjustified exceptions and crashes.
7337 * This bug was the result of resizing the internal buffer not large enough
7338 * in some cases. The name was then changed to JavaLexBitSet.
7340 * Copyright (c) 1995 Sun Microsystems, Inc. All Rights Reserved.
7342 * Permission to use, copy, modify, and distribute this software
7343 * and its documentation for NON-COMMERCIAL purposes and without
7344 * fee is hereby granted provided that this copyright notice
7345 * appears in all copies. Please refer to the file "copyright.html"
7346 * for further important copyright and licensing information.
7348 * SUN MAKES NO REPRESENTATIONS OR WARRANTIES ABOUT THE SUITABILITY OF
7349 * THE SOFTWARE, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
7350 * TO THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
7351 * PARTICULAR PURPOSE, OR NON-INFRINGEMENT. SUN SHALL NOT BE LIABLE FOR
7352 * ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF USING, MODIFYING OR
7353 * DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES.
7357 * A set of bits. The set automatically grows as more bits are
7360 * @version 1.12, 01 Dec 1995
7361 * @author Arthur van Hoff, revised by Elliot Joel Berk
7363 final class JavaLexBitSet implements Cloneable {
7364 final static int BITS = 6;
7365 final static int MASK = (1<<BITS)-1;
7369 * Creates an empty set.
7371 public JavaLexBitSet() {
7376 * Creates an empty set with the specified size.
7377 * @param nbits the size of the set
7379 public JavaLexBitSet(int nbits) {
7380 bits = new long[nbits2size(nbits)];
7383 private int nbits2size (int nbits) {
7384 return ((nbits >> BITS) + 1);
7387 private void resize(int nbits) {
7388 int newsize = Math.max(bits.length, nbits2size(nbits));
7389 long newbits[] = new long[newsize];
7390 System.arraycopy(bits, 0, newbits, 0, bits.length);
7396 * @param bit the bit to be set
7398 public void set(int bit) {
7400 if (n >= bits.length) {
7403 bits[n] |= (1L << (bit & MASK));
7408 * @param bit the bit to be cleared
7410 public void clear(int bit) {
7412 if (n >= bits.length) {
7415 bits[n] &= ~(1L << (bit & MASK));
7420 * @param bit the bit to be gotten
7422 public boolean get(int bit) {
7424 return (n < bits.length) ?
7425 ((bits[n] & (1L << (bit & MASK))) != 0) :
7430 * Logically ANDs this bit set with the specified set of bits.
7431 * @param set the bit set to be ANDed with
7433 public void and(JavaLexBitSet set) {
7434 int n = Math.min(bits.length, set.bits.length);
7435 for (int i = n ; i-- > 0 ; ) {
7436 bits[i] &= set.bits[i];
7438 for (; n < bits.length ; n++) {
7444 * Logically ORs this bit set with the specified set of bits.
7445 * @param set the bit set to be ORed with
7447 public void or(JavaLexBitSet set) {
7448 for (int i = Math.min(bits.length, set.bits.length) ; i-- > 0 ;) {
7449 bits[i] |= set.bits[i];
7454 * Logically XORs this bit set with the specified set of bits.
7455 * @param set the bit set to be XORed with
7457 public void xor(JavaLexBitSet set) {
7458 for (int i = Math.min(bits.length, set.bits.length) ; i-- > 0 ;) {
7459 bits[i] ^= set.bits[i];
7464 * Gets the hashcode.
7466 public int hashCode() {
7468 for (int i = bits.length; --i >= 0; ) {
7471 return (int)((h >> 32) ^ h);
7475 * Calculates and returns the set's size
7478 return bits.length << BITS;
7482 * Compares this object against the specified object.
7483 * @param obj the object to commpare with
7484 * @return true if the objects are the same; false otherwise.
7486 public boolean equals(Object obj) {
7487 if ((obj != null) && (obj instanceof JavaLexBitSet)) {
7488 JavaLexBitSet set = (JavaLexBitSet)obj;
7490 int n = Math.min(bits.length, set.bits.length);
7491 for (int i = n ; i-- > 0 ;) {
7492 if (bits[i] != set.bits[i]) {
7496 if (bits.length > n) {
7497 for (int i = bits.length ; i-- > n ;) {
7502 } else if (set.bits.length > n) {
7503 for (int i = set.bits.length ; i-- > n ;) {
7504 if (set.bits[i] != 0) {
7515 * Clones the JavaLexBitSet.
7517 public Object clone() {
7519 JavaLexBitSet set = (JavaLexBitSet)super.clone();
7520 set.bits = new long[bits.length];
7521 System.arraycopy(bits, 0, set.bits, 0, bits.length);
7523 } catch (CloneNotSupportedException e) {
7524 // this shouldn't happen, since we are Cloneable
7525 throw new InternalError();
7530 * Converts the JavaLexBitSet to a String.
7532 public String toString() {
7534 for (int i = 0 ; i < (bits.length << BITS) ; i++) {
7536 if (str.length() > 0) {
7542 return "{" + str + "}";
7546 /************************************************************************
7547 JLEX COPYRIGHT NOTICE, LICENSE AND DISCLAIMER.
7549 Copyright 1996 by Elliot Joel Berk
7551 Permission to use, copy, modify, and distribute this software and its
7552 documentation for any purpose and without fee is hereby granted,
7553 provided that the above copyright notice appear in all copies and that
7554 both the copyright notice and this permission notice and warranty
7555 disclaimer appear in supporting documentation, and that the name of
7556 Elliot Joel Berk not be used in advertising or publicity pertaining
7557 to distribution of the software without specific, written prior permission.
7559 Elliot Joel Berk disclaims all warranties with regard to this software,
7560 including all implied warranties of merchantability and fitness. In no event
7561 shall Elliot Joel Berk be liable for any special, indirect or consequential
7562 damages or any damages whatsoever resulting from loss of use, data or
7563 profits, whether in an action of contract, negligence or other
7564 tortious action, arising out of or in connection with the use or
7565 performance of this software.
7566 ***********************************************************************/