Java Lexer Analyzer

Our program will have a Lexer class and a Token class. The Lexer provided in the starter code is incomplete. You can use the Token class as is.

  • The class Lexer should provide the following methods provided:
    • public constructor which takes a file name as a parameter
    • private getInput which reads the data from the file into a String.
  • You’ll need to add the following methods to Lexer:
    • public getNextToken which returns a Token object.
    • private functions for getIdentifier and getInteger. These helper functions should handle extracting the rest of an identifier/integer once one is identified.
    • A getAllTokens which calls getNextToken within a loop and returns a list of Tokens.
  • The class Token should have two data members, type and value, both of type String, along with a constructor which takes two parameters, and a toString method which prints a token’s type and value.

This is the starter code and also uses the Token Class as well below

import java.io.IOException;import java.nio.file.Files;import java.nio.file.Path;import java.nio.file.Paths;import java.util.ArrayList;import java.util.Scanner;/** * Class to build an array of Tokens from an input file * @author rena * @see Token * @see Parser */public class Lexer { //The class Token should have two data members, type and value, both of type String String type; //data member String value; //data member String buffer; int index = 0; public static final String INTTOKEN=”INT”; public static final String IDTOKEN=”ID”; public static final String ASSMTTOKEN=”ASSMT”; public static final String PLUSTOKEN=”PLUS”; public static final String EOFTOKEN=”EOF”; /** * call getInput to get the file data into our buffer * @param fileName the file we open */ public Lexer(String fileName) { getInput(fileName); } public getNextToken(){ # no parameters }/*private functions for getIdentifier and getInteger. These helper functionsshould handle extracting the rest of an identifier/integer once one is identified. */ private getIdentifier(){ # no parameters } private getInteger(){ # no parameter } //A getAllTokens which calls getNextToken within a loop and returns a list of Tokens. public getAllTokens(){ # no parameters // // call getNextToken }

/** * Reads given file into the data member buffer * @param fileName name of file to parse */ private void getInput(String fileName) { try { Path filePath = Paths.get(fileName); byte[] allBytes = Files.readAllBytes(filePath); buffer = new String (allBytes); } catch (IOException e) { System.out.println (“You did not enter a valid file name in the run arguments.”); System.out.println (“Please enter a string to be parsed:”); Scanner scanner = new Scanner(System.in); buffer=scanner.nextLine(); } } /** * Return all the token in the file * @return ArrayList of Token */ public ArrayList<Token> getAllTokens(){ //TODO: place your code here for lexing file return new ArrayList<Token>(); // don’t forget to change the return statement } /** * Before your run this starter code * Select Run | Edit Configurations from the main menu. * In Program arguments add the name of file you want to test (e.g., test.txt) * @param args args[0] */ public static void main(String[] args) { String fileName=””; if (args.length==0) { System.out.println(“You can test a different file by adding as an argument”); System.out.println(“See comment above main”); System.out.println(“For this run, test.txt used”); fileName=”test.txt”; } else { fileName=args[0]; } Lexer lexer = new Lexer(fileName); // just print out the text from the file System.out.println(lexer.buffer); // here is where you’ll call getAllTokens }}public class Token {
public String type;
public String value;

public Token(String type, String value) {
this.type=type;
this.value=value;
}

public String toString(){

return type+” “+value;
}

}

Place this order or similar order and get an amazing discount. USE Discount code “GET20” for 20% discount