Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 40 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,45 @@
# Plare
Language description language, Lexer, and Parser for Python 3

A lexer/parser framework for Python 3.12+.
Plare lets you define a tokeniser and an LALR(1) parser using plain Python
classes and dictionaries — no code generation, no external grammar files.

## Features

- **Stateful lexer** — regex-driven, named states let you switch tokenisation
modes mid-stream (e.g., to skip comments)
- **LALR(1) parser** — efficient shift/reduce parser with automatic conflict
detection
- **Operator precedence** — resolve shift/reduce conflicts by setting
`precedence` and `associative` class variables on token classes
- **No build step** — install and import

## Installation
You can install Plare with pip:

```bash
pip install plare
```
```

## Quick Example

```python
from plare.lexer import Lexer
from plare.parser import Parser
from plare.token import Token

class NUM(Token):
def __init__(self, value: str, *, lineno: int, offset: int) -> None:
super().__init__(value, lineno=lineno, offset=offset)
self.value = int(value)

class PLUS(Token):
pass

lexer = Lexer({"start": [(r"\d+", NUM), (r"\+", PLUS), (r" +", "start")]})
parser = Parser({"exp": [(["exp", PLUS, "exp"], Add, [0, 2]), ([NUM], Const, [0])]})
```

## Examples

- [`examples/calc/`](examples/calc/) — integer arithmetic with operator precedence
- [`examples/sum_of_list/`](examples/sum_of_list/) — list parsing with recursive grammar
12 changes: 6 additions & 6 deletions examples/calc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ To test the program first, run the following command.
```bash
$ python calc.py data/ex02.calc
```
The program will parse the `data/ex02.calc`, interprete, and show the answer as follows.
The program will parse the `data/ex02.calc`, interpret, and show the answer as follows.
```
== Source (data/ex02.calc) ==
(1 + (1 * 2))
== Result (data/ex02.calc) ==
3
```
Note that the program correctly parse the program considering the precedence of multiplication and addition.
Note that the program correctly parses the program considering the precedence of multiplication and addition.

## Defining Language
To make a parser, first thing to do is defining a language.
Expand Down Expand Up @@ -43,7 +43,7 @@ class Add:

## Defining Lexer
A lexer will tokenize the given source string.
To make a lexer you only have to do is make tokens that inherits`Token` class.
To make a lexer, all you need to do is define token classes that inherit from `Token`.
For example, `+` can be defined as follows:
```python
from plare.token import Token
Expand Down Expand Up @@ -74,7 +74,7 @@ For example, the following rule will create a `PLUS` token when it matches `+`:
}
```
You can switch to other lexing rules, by assigning a string instead of a `Token`.
The following code will switchin to `"comment"` mode when the string starts with `//`:
The following code will switch to `"comment"` mode when the string starts with `//`:
```python
{
"start": [
Expand Down Expand Up @@ -135,7 +135,7 @@ For example, you can pass the 2nd subtree like following:
}
```

After defining grannar, you can make a parser with `Parser`, like:
After defining grammar, you can make a parser with `Parser`, like:
```python
from plare.parser import Parser

Expand All @@ -151,7 +151,7 @@ parsed = parser.parse("exp", lexer.lex("start", "1 + 2"))
## Resolving conflicts
You can resolve conflicts between shift and reduce actions based on "precedence" and "associative".
The precedence and associative can be defined as class variable when defining tokens.
By default, all token have precedence of `0` and and `"left"` associative.
By default, all tokens have a precedence of `0` and `"left"` associativity.
In this example, `STAR` and `DIV` have higher precedence, which means `1 + 2 * 3` will be parsed into `(1 + (2 * 3))`, not `((1 + 2) * 3)`.

## Interprete
Expand Down
4 changes: 2 additions & 2 deletions examples/sum_of_list/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ list -> N::list
```

A node in AST must initialized with `Token` objects or other node objects.
That is, You need to define 3 different type of list, which are initialized with different arguments, to support the arbitrary length of list: `IntList`, `SingleIntList`, and `EmptyIntList`.
That is, you need to define 3 different types of list, which are initialized with different arguments, to support the arbitrary length of list: `IntList`, `SingleIntList`, and `EmptyIntList`.

## Defining Lexer
A lexer will tokenize the given source string.
To make a lexer you only have to do is make tokens that inherits`Token` class.
To make a lexer, all you need to do is define token classes that inherit from `Token`.
In this example, you only need to define 4 tokens: `COMMA`, `LBRACKET`, `RBRACKET`, and `NUM`.

## Defining Parser
Expand Down
Loading