Which compiler component groups a stream of input characters into tokens?

Concept

Lexical analysis is the first phase of a compiler. It reads the source program as a stream of characters and groups them into tokens such as identifiers, keywords, literals, and operators.

The component that performs lexical analysis is called a scanner or lexical analyzer. Its token stream is then passed to the parser.

Application

The stem asks for the compiler component that groups input characters into tokens. That operation is lexical analysis, so the relevant component is the scanner.

Contrast

• A parser consumes tokens and organizes them according to the grammar, typically producing a parse tree or syntax tree.

• A code optimizer transforms an intermediate representation to improve properties such as execution time or code size while preserving program behavior.

• A code generator translates an intermediate representation into target assembly or machine code.

• A scanner reads the source character stream and emits lexical units such as identifiers, keywords, literals, and operators.

Result

Therefore, the required compiler component is the scanner.