Logo

LEX

LEX

Lex is a tool/computer program that generates a Lexical analyzer. Lex is developed by Vern Paxson in C around 1987. Lex works together with the YACC parser generator. It allows us to identify a Lexical analyzer by specifying regular expressions to describe patterns for tokens. The Lex language will be the input for the Lex tool, and the tool is termed as the Lex compiler. The role of the Lex compiler is to convert the input pattern into a transition diagram and produce code in a file called Lex.yy.c

Installing Lex on Ubuntu

The following commands are used to install the Lex on Ubuntu:

sudo apt-get update

sudo apt-get install flex   

Use of Lex

The below figure shows the working of Lex. The input file lex.1 is written in Lex language and describes the Lexical analyzer to be generated. Next, the Lex compiler executes the Lex.1 program and transform it into a C program, named lex.yy.c. Then, the C compiler compiles this file into a program a.out. The C compiler's output is working as a Lexical analyzer that takes a stream of input characters and produces a stream of tokens.

The output of the C compiled file, named a.out, is a subroutine of the parser. It is a C function that returns an integer value and will be a code for one of the possible token names. The global variable yy1val consists of the attribute value, symbol table's pointer or nothing. It will share it between the parser and the Lexical analyzer. Therefore making it simple to return both the token name and attribute value. 

Structure of Lex program

Any Lex program is separated by %% delimiter into three sections. The syntax of the Lex program is given below:

                          {Declarations}

                           %%

                          {Translation rules}

                           %%

                           {Auxiliary functions}

The first part contains a declaration of a variable, regular definition, and manifest constant. The text in the declaration section is enclosed in "%{%}" brackets. 

The syntax of translation rule is:

 pattern { Action }

Every pattern is a regular expression. The standard definitions declared in the declaration part may be used by a pattern of this part. The action is a C programming code. We can develop many types of Lex by using various languages. The rule section is enclosed in "%%%%."

The last section contains the C statement and some additional functions. We can compile these functions separately and loaded with a Lexical analyzer.

The Lexical analyzer created by Lex works with the parser in the following ways. When the parser invokes the Lex, the Lexical analyzer starts reading the remaining inputs by taking one input symbol at a time until it finds the longest starting symbol of the input that matches the patterns pi. Then, it performs the action Ai. Ai will return to the parser. If it does not match due to whitespace or comments in Pi, in that case, the Lexical analyzer will further proceed to find additional Lexeme until one of the corresponding actions cause a return to the parser. The Lexical analyzer produces the token name, which will be used by the parser but uses the shared integer variable yylval to forward other information regarding the Lexeme found if needed.

Related Topics

Run-Time Storage Management

Run-Time Storage Management Every executing program has its own logical address space. Logical address space is partitioned into: Code: It is responsible for storing the executable target code. Static: It is used to...

3 minutes read.

Data Flow Analysis in Compiler Design

Data Flow Analysis All the optimization techniques we have learned earlier depend on data flow analysis. DFA is a technique used to know about how the data is flowing in any...

3 minutes read.

Stack Allocation of Space

Stack Allocation of Space Almost all compilers for languages that use procedure, functions, or methods manage their run-time memory as a stack. Whenever a procedure is called, the local variable's space...

2 minutes read.

Phases of Compiler

Phases of Compiler The compilation process of a compiler goes through various phases. Each phase of the compiler takes the output from the previous as an input. Here we will see all...

2 minutes read.

Translation of Array References in Compiler Design

Translation of Array References We can access the elements of an array stored in consecutive blocks very easily and quickly. In a programming language like C and Java, the size of...

2 minutes read.

Machine-Independent Optimizations in Compiler Design

Machine-Independent Optimizations The main aim of machine-independent optimization is to improve the generated intermediate code so that compiler can get better target code. Eliminating unwanted code from the object code or replacing...

8 minutes read.

Target Machine

Target Machine A target machine is a byte-addressable machine. This machine has n general-purpose registers, R0, R1,.....Rn-1. A Simple Target Machine Model has three-address instruction. A full-edged assembly language would have...

3 minutes read.

Syntax-Directed Translation

Syntax-Directed Translation A context-free-grammar with some additional rules is known as a syntax-directed definition. In SDT, attributes are associated with grammar symbols and rules are associated with productions. The attributes can...

3 minutes read.

Syntax Analysis Compiler Design

Syntax Analysis This article will describe the parsing method used in the compiler. The grammatical rule of programming language can be constructed with the help of context-free grammars or BNF (Backus–Naur...

3 minutes read.

Shift Reduce Parsing in Compiler Design

Like the bottom-up parsing, the shift reduce parser also builds the parse tree from the leaves (bottom) to the root (up). The LR parser is a more versatile variation of...

4 minutes read.

Intermediate-Code Generator Compiler Design

Intermediate-Code Generator The process of translating a source language into machine code for a given target machine is done by intermediate-code. It lies between the high-level language and the machine language....

2 minutes read.

Lexical Analysis in Compiler Design

Lexical Analysis It is the first phase of the compiler. As we know, it is also known as a scanner. The input for lexical analysis is source code. After taking source...

2 minutes read.

S-attributed and L-attributed SDTs

S-attributed and L-attributed SDTs STD stands for Syntax Directed Translation. When we associate some informal notations called semantic rules and the grammar, they are known as STD. So we can say...

2 minutes read.

LR Parser in Compiler Design

LR Parser The most popular type of bottom-up parsing is LR(K) parsing. The LR() parser scans the input from left – to – right, which is the actual abbreviation of L...

2 minutes read.

LR Parser Compiler Design

LR Parser LR parsing is a type of bottom-up parsing that is used to parse the large class of grammars. Here "L" stands for left-to-right scanning of the input "R" stands...

6 minutes read.

Basic Blocks and Flow Graphs in Compiler Design

Basic Blocks and Flow Graphs In this section, we are going to learn how to work with basic block and flow graphs in compiler design. Basic Block The basic block is a set...

3 minutes read.

Three-Address Code

Three-Address Code If there is at most one operator on the right side of the instruction, then the instruction will be the three-address code so that no arithmetic expressions are permitted....

2 minutes read.

Code Generation

Code Generation The last phase of the compiler is code generation. It is the compiler's back-end that makes multiple passes over the IR before generating the target program. The code generator's...

4 minutes read.

Finite State Machine

Finite State Machine A finite state machine is a simple machine to recognize patterns. It takes a string of symbols as the input and changes its state to another state, but...

3 minutes read.

Errors in Compiler Design

Introduction Errors in compiler design refer to mistakes or issues that arise during the process of execution of the program. A compiler is a program that translates source code written in...

4 minutes read.