KonstantinSeurer
/
mesa
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

glsl/glcpp: Treat CR+LF pair as a single newline 

The GLSL specification says that either carriage-return, line-feed, or both
together can be used to terminate lines. Further, it says that when used
together, the pair of terminators shall be interpreted as a single line.

This final requirement has not been respected by glcpp up until now, (it has
been emitting two newlines for every CR+LF pair).

Here, we fix the lexer by using a regular expression for NEWLINE that eats
up both "\r\n" (or even "\n\r") if possible before also considering a single
'\n' or a single '\r' as a line terminator.

Before this commit, the test results are as follows:

	\r:	135/143 tests pass
	\r\n:	  4/143 tests pass
	\n\r:	  4/143 tests pass

After this commit, the test results are as follows:

	\r:	135/143 tests pass
	\r\n:	140/143 tests pass
	\n\r:	139/143 tests pass

So, obviously, a dramatic improvement.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
This commit is contained in:
Carl Worth 2014-07-01 16:31:07 -07:00 committed by Ian Romanick
parent f4ddd026c6
commit 04e40fd337
2 changed files with 9 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
src/glsl/glcpp/glcpp-lex.l
View File

@ -172,6 +172,7 @@ SPACE [[:space:]]
NONSPACE [^[:space:]]
HSPACE [ \t]
HASH #
NEWLINE (\r\n|\n\r|\r|\n)
IDENTIFIER [_a-zA-Z][_a-zA-Z0-9]*
PP_NUMBER [.]?[0-9]([._a-zA-Z0-9]|[eEpP][-+])*
PUNCTUATION [][(){}.&*~!/%<>^|;,=+-]
@ -256,9 +257,9 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]?
/* Multi-line comments */
<INITIAL,DEFINE,HASH>"/*" { yy_push_state(COMMENT, yyscanner); }
<COMMENT>[^*\r\n]*
<COMMENT>[^*\r\n]*[\r\n] { yylineno++; yycolumn = 0; parser->commented_newlines++; }
<COMMENT>[^*\r\n]*{NEWLINE} { yylineno++; yycolumn = 0; parser->commented_newlines++; }
<COMMENT>"*"+[^*/\r\n]*
<COMMENT>"*"+[^*/\r\n]*[\r\n] { yylineno++; yycolumn = 0; parser->commented_newlines++; }
<COMMENT>"*"+[^*/\r\n]*{NEWLINE} { yylineno++; yycolumn = 0; parser->commented_newlines++; }
<COMMENT>"*"+"/" {
yy_pop_state(yyscanner);
/* In the <HASH> start condition, we don't want any SPACE token. */
@ -289,7 +290,7 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]?
/* Swallow empty #pragma directives, (to avoid confusing the
* downstream compiler). */
<HASH>pragma{HSPACE}*/[\r\n] {
<HASH>pragma{HSPACE}*/{NEWLINE} {
BEGIN INITIAL;
}
@ -305,7 +306,7 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]?
RETURN_TOKEN (LINE);
}
<HASH>\n {
<HASH>{NEWLINE} {
BEGIN INITIAL;
RETURN_TOKEN_NEVER_SKIP (NEWLINE);
}
@ -521,7 +522,7 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]?
/* We preserve all newlines, even between #if 0..#endif, so no
skipping.. */
<*>[\r\n] {
<*>{NEWLINE} {
if (parser->commented_newlines) {
BEGIN NEWLINE_CATCHUP;
} else {

3
src/glsl/glcpp/tests/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
subtest-cr/
subtest-cr-lf/
subtest-lf-cr/