Today I have merged the unknownsymbol branch to master. This branch introduces partially defined grammar support for Parser.

A Partially defined grammar contains at least one special symbol UnknownSymbol. This symbol tells the parser to consume anything, but it is always consumed after any other symbol (String, Word or NullSymbol)

Here is an example:


//List
#parser=weighted

S ::= start unk optional end
optional ::= null | separator unk optional
start := String,[
end := String,]
separator := String,,
unk := Unk
null := Null


this is a right recursive implementation of a list. It accepts anything as an unknown element:

echo "[1,1,4]" | python3 bin/check.py  -d 1 list
True
echo "[asdf,1,4]" | python3 bin/check.py  -d 1 list
True

Partial grammars can be used as a normal grammar, and it is the most important element in pydsl abstraction support. listlen is an example of abstraction:


bin/translate.py listlen -e "input:[a\,b\,c]"
{'output': '3'}

bin/translate.py listlen -e "input:[a\,b\,1]"
{'output': '3'}