Process QSL grammar file - Revision history

Wikicatglobe at 04:43, 18 October 2013

2013-10-18T04:43:00Z

← Older revision		Revision as of 04:43, 18 October 2013
Line 1:		Line 1:
	<accesscontrol>~~Administrators,,Cem,,Maysunshine~~</accesscontrol>		<accesscontrol>Main:MyGroup</accesscontrol>
	== Introduction ==		== Introduction ==
	Since the first version of QSL grammar file, there have been many changes and the current grammar is completely different from the original. Unlike other projects, project related to language seems to be not easy to follow. In order to make it clear, i write an article dedicated to technique we use to solve QSL grammar.		Since the first version of QSL grammar file, there have been many changes and the current grammar is completely different from the original. Unlike other projects, project related to language seems to be not easy to follow. In order to make it clear, i write an article dedicated to technique we use to solve QSL grammar.

Cg_pham at 09:30, 4 October 2011

2011-10-04T09:30:26Z

← Older revision		Revision as of 09:30, 4 October 2011
Line 1:		Line 1:
			<accesscontrol>Administrators,,Cem,,Maysunshine</accesscontrol>
	== Introduction ==		== Introduction ==
	Since the first version of QSL grammar file, there have been many changes and the current grammar is completely different from the original. Unlike other projects, project related to language seems to be not easy to follow. In order to make it clear, i write an article dedicated to technique we use to solve QSL grammar.		Since the first version of QSL grammar file, there have been many changes and the current grammar is completely different from the original. Unlike other projects, project related to language seems to be not easy to follow. In order to make it clear, i write an article dedicated to technique we use to solve QSL grammar.

Catglobe: /* Future improvement for QSL */

2009-10-06T04:45:54Z

Future improvement for QSL

← Older revision		Revision as of 04:45, 6 October 2009
Line 49:		Line 49:

	== Future improvement for QSL ==		== Future improvement for QSL ==
	From now on, QSL cover all the aspect of a questionnaire template. For any further improvement, it is reasonable to consider to wrap question text in double quote symbol ("). It will reduce the semantic prediction and the grammar file looks nicer.		From now on, QSL cover all the aspect of a questionnaire template. For any further improvement, it is reasonable to consider to wrap question text in double quote symbol ("). It will reduce the semantic prediction and the grammar file looks nicer.<br>
	Logical validating properties should be more strictly, especially those properties we split to answer option or sub question properties set. This feature can lead to a new design for error/warning system.		Logical validating properties should be more strictly, especially those properties we split to answer option or sub question properties set. This feature can lead to a new design for error/warning system.<br>
	Format exported question text (including answer option text and sub question text ) in export QSL module. We know that it is HTML content and sometimes reading a bunch without any indent could be a pain. This feature may require an open library to format text.		Format exported question text (including answer option text and sub question text ) in export QSL module. We know that it is HTML content and sometimes reading a bunch without any indent could be a pain. This feature may require an open library to format text.<br>

	[[category:Technical guidelines]]		[[category:Technical guidelines]]

Catglobe: /* Future improvement for QSL */

2009-10-06T04:45:26Z

Future improvement for QSL

← Older revision		Revision as of 04:45, 6 October 2009
Line 49:		Line 49:

	== Future improvement for QSL ==		== Future improvement for QSL ==
	From now on, QSL cover all the aspect of a questionnaire template. For any further improvement, it is reasonable to consider to wrap question text in double quote symbol ("). It will reduce the semantic prediction and the grammar file looks nicer.		From now on, QSL cover all the aspect of a questionnaire template. For any further improvement, it is reasonable to consider to wrap question text in double quote symbol ("). It will reduce the semantic prediction and the grammar file looks nicer.
			Logical validating properties should be more strictly, especially those properties we split to answer option or sub question properties set. This feature can lead to a new design for error/warning system.
			Format exported question text (including answer option text and sub question text ) in export QSL module. We know that it is HTML content and sometimes reading a bunch without any indent could be a pain. This feature may require an open library to format text.

	[[category:Technical guidelines]]		[[category:Technical guidelines]]

Catglobe: /* Sequence of processing QSL script */

2009-09-30T08:53:39Z

Sequence of processing QSL script

← Older revision		Revision as of 08:53, 30 September 2009
Line 5:		Line 5:
	== Sequence of processing QSL script ==		== Sequence of processing QSL script ==
	[[Image:Process_QSL.png]]		[[Image:Process_QSL.png]]
			<br>
	The lexical analyzer, or lexer, breaks up the input stream into tokens. The parser feeds off this token stream and tries to recognize the sentence structure. Next, we translate the sentences get from parser in to abstract structure tree (AST). Finally, we parse AST into data object.		The lexical analyzer, or lexer, breaks up the input stream into tokens. The parser feeds off this token stream and tries to recognize the sentence structure. Next, we translate the sentences get from parser in to abstract structure tree (AST). Finally, we parse AST into data object.
	This article focuses on lexical level as it caused many problem need to solved (From my point of view, parser grammar of QSL is pretty clean and easy to understand)		This article focuses on lexical level as it caused many problem need to solved (From my point of view, parser grammar of QSL is pretty clean and easy to understand)

	== Structure of QSL grammar file ==		== Structure of QSL grammar file ==
	Normally, there are individual file for lexer grammar and parser grammar but in QSL, we use a mixed grammar file. It means that all lexer rules and parser rules are defined in the same file. This is the original structure and decide to keep it since from my own experience, it is easier to work in ANTLR Works.		Normally, there are individual file for lexer grammar and parser grammar but in QSL, we use a mixed grammar file. It means that all lexer rules and parser rules are defined in the same file. This is the original structure and decide to keep it since from my own experience, it is easier to work in ANTLR Works.

Catglobe: /* Future improvement for QSL */

2009-09-30T08:52:48Z

Future improvement for QSL

← Older revision		Revision as of 08:52, 30 September 2009
Line 47:		Line 47:

	== Future improvement for QSL ==		== Future improvement for QSL ==
	From now on, QSL cover all the aspect of a questionnaire template. For any further improvement, it is reasonable to consider to wrap question text in double quote symbol ("). It will reduce the semantic prediction and the grammar file ~~look~~ nicer.		From now on, QSL cover all the aspect of a questionnaire template. For any further improvement, it is reasonable to consider to wrap question text in double quote symbol ("). It will reduce the semantic prediction and the grammar file looks nicer.

	[[category:Technical guidelines]]		[[category:Technical guidelines]]

Catglobe at 08:52, 30 September 2009

2009-09-30T08:52:10Z

← Older revision		Revision as of 08:52, 30 September 2009
Line 46:		Line 46:
	This solution works pretty perfectly when we combine with semantic prediction in grammar file. If we can use our flag to direct the input cursor into a path that only can match a token, it is very effective way to solve the "Code too large" problem. When working project "VN2615QNR - Condition for QSL language", we use ANTLR API method to handle 3 tokens : EXPRESSION, ANY_CHARACTER and COMMENT. It could take much time but it also bring many useful experiences.		This solution works pretty perfectly when we combine with semantic prediction in grammar file. If we can use our flag to direct the input cursor into a path that only can match a token, it is very effective way to solve the "Code too large" problem. When working project "VN2615QNR - Condition for QSL language", we use ANTLR API method to handle 3 tokens : EXPRESSION, ANY_CHARACTER and COMMENT. It could take much time but it also bring many useful experiences.

			== Future improvement for QSL ==
			From now on, QSL cover all the aspect of a questionnaire template. For any further improvement, it is reasonable to consider to wrap question text in double quote symbol ("). It will reduce the semantic prediction and the grammar file look nicer.

	[[category:Technical guidelines]]		[[category:Technical guidelines]]

Catglobe: /* Negative effect */

2009-09-30T08:44:56Z

Negative effect

← Older revision		Revision as of 08:44, 30 September 2009
Line 32:		Line 32:
	[[image:DFADicision.jpg]]		[[image:DFADicision.jpg]]
	<br>		<br>
	Whenever we expand the grammar (add new keyword for example), the complexity is increased and leads to "Code too large" problem.		Whenever we expand the grammar (add new keyword for example), the complexity is increased and leads to "Code too large" problem.
			===== Heavy semantic prediction =====
			Back to the meaning of lexical level, it analyzes the input and return tokens for parser. In QSL Lexer, almost all the tokens have semantic prediction (we use condition and flag to direct the input cursor). Most of semantic prediction we use in lexer come from Parser (we decide tokens base on it's role in a complete sentence). From my own experience, that can make someone who is new to QSL grammar confused and the lexer grammar looks like hard to understand.

	=== Identified problems when working with QSL grammar in Java environment ===		=== Identified problems when working with QSL grammar in Java environment ===
	==== "Code too large" ====		==== "Code too large" ====

Catglobe at 08:31, 30 September 2009

2009-09-30T08:31:56Z

← Older revision		Revision as of 08:31, 30 September 2009
Line 33:		Line 33:
	<br>		<br>
	Whenever we expand the grammar (add new keyword for example), the complexity is increased and leads to "Code too large" problem.		Whenever we expand the grammar (add new keyword for example), the complexity is increased and leads to "Code too large" problem.
			=== Identified problems when working with QSL grammar in Java environment ===
			==== "Code too large" ====
			===== Cause =====
			Java compiler could not compile code if there is any function larger than 64 kB in size. Some time our rule is complex and the code behind it could up to more than 64kB. No file could be compiled and we can't use the generated file for our purposes.
			===== Solution =====
			====== Composite grammar ======
			In this way, we split the grammar file into some smaller file. It is the solution mentioned in ANTLR official page. Unluckily, we tried but not succeed with it.
			====== Write our own lexer for a particular tokens ======
			This solution works pretty perfectly when we combine with semantic prediction in grammar file. If we can use our flag to direct the input cursor into a path that only can match a token, it is very effective way to solve the "Code too large" problem. When working project "VN2615QNR - Condition for QSL language", we use ANTLR API method to handle 3 tokens : EXPRESSION, ANY_CHARACTER and COMMENT. It could take much time but it also bring many useful experiences.



	[[category:Technical guidelines]]		[[category:Technical guidelines]]

Catglobe: /* From the bottom of QSL lexer */

2009-09-30T07:57:32Z

From the bottom of QSL lexer

@@ Line 14: / Line 14: @@
 This flag is true in case we scan each line of the input, searching for keyword or just a text line (question text, answer option text and sub question text). lineMode is false when the input cursor is inside of block (properties or expression).
 === Question text is not wrapped ===
-Question text(include answer option text, sub question text) used to be wrapped with '##' when the the very first version of QSL came out. Later, in an update, we decided to remove '##'. This is a big change and effect the tactical of handling lexer.
+Question text(including answer option text, sub question text) used to be wrapped with '##' when the the very first version of QSL came out. Later, in an update, we decided to remove '##'. This is a big change and effect the tactical of handling lexer.
 [[category:Technical guidelines]]