Improve the speed of PPI

[szabgab]

In order to use PPI for syntax checking or syntax highlighting we have to make it much faster. The options were thought of so far:

• Implement more in PPI::XS
• Allow PPI to restart parsing at a certain point in the document

[pshangov]

I have started doing some work on speeding up the PPI-based syntax highlighter. The basic idea is that PPI should parse as little as possible on each EVT_STC_STYLENEEDED event. Here is a simplified overview of the algorithm I am trying to implement.

1. Figure out which portions (lines) of the code need styling. This can be done in two ways:
   1. with each EVT_STC_STYLENEEDED event we can access the stat and end positions in the editor that wxSTC thinks need styling.
   2. we can alternatively use information from the EVT_STC_MODIFIED event, and figure out which parts of the code have been changed. This may also reduce the number of calls to the PPI parser since EVT_STC_STYLENEEDED appears to be raised much more often than EVT_STC_MODIFIED.
2. After we have a bunch (normally only a few) of lines that need restyling we have to figure out if we can just feed them to PPI. First we have to take care of quote-like strings:
   1. if the last token before the text we need to restyle is an unfinished quote-like token, we include the text of this token in the string we feed to PPI.
   2. if the last token in the text we need to restyle is an unfinished quote-like token, we search forward for the end of this token and include that in the string we feed to PPI.
3. A similar logic is applied to pod sections:
   1. if the last token before the text we need to restyle is an unfinished pod token, we include the text of this token in the string we feed to PPI.
   2. if the last token in the text we need to restyle is an unfinished pod token, we search forward for the end of this token and include that in the string we feed to PPI.

Feedback welcome.

[szabgab]

Last known discussion ​http://mail.perlide.org/pipermail/padre-dev/2009-July/001151.html

and a related discussion ​http://mail.perlide.org/pipermail/padre-dev/2010-February/001539.html

[adamk]

This doesn't really belong in the Padre trac anymore