(drak)
2014-11-19: merge for release: reader execution and SRFI. stable v0.8.1 merge for release: reader execution and SRFI. - wisp-reader: fix wisp-scheme code-tree evaluation - SRFI: lots of polish.
diff --git a/.bugs/bugs b/.bugs/bugs
--- a/.bugs/bugs
+++ b/.bugs/bugs
@@ -7,10 +7,11 @@ wisp-mode: export wisp to html fails in
an empty line with : should start with double parens | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:2e188ddf44d36e4605030d3c58607ebfa97d189e, time:1390328674.43
wisp-in-wisp: remove the leading . for continuation lines after inferring the brackets. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:2e42e5b64622f0cc383eb8acc3d510912e925bf0, time:1377476687.79
'() gives REPR-QUOTE-... | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:30c42de75c137f483245898e2a62af1e65cf19a6, time:1415060388.34
+multiple escaped initial underscores must be unescaped. | owner:, open:True, id:314e45488da4c7c8298c4c64ece03359918d057b, time:1415959749.14
wisp: handle general paren prefixes like quote, quasiquote, etc. | owner:, open:False, id:323ff94b5be635742619467e1cb44f4c0d96f63f, time:1379047798.47
throw an exception when reducing indentation to an unknown indentation level. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:424186bd85f186b7279c5c59e2bd42f847284719, time:1376003568.91
wisp-in-wisp: only parses the first 272 lines, for some reason. | owner:, open:False, id:4cb6c556d7136609e2da9ab3fc045a39847f1ef3, time:1377014682.98
-adjust runtests.sh to use testrunner.w | owner:, open:True, id:4d4e76343fe09f0ec72a3e5eb0077bd16e12f9d5, time:1415127234.43
+adjust runtests.sh to use testrunner.w | owner:, open:False, id:4d4e76343fe09f0ec72a3e5eb0077bd16e12f9d5, time:1415127234.43
implement wisp in wisp | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:6299306916706410702029289bf32edab1e7f17c, time:1367113341.49
support other types of brackets, like square brackets. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:6749f3abcb9455eac9271efd8265797bce114239, time:1389134151.98
linebreaks in parens still break | owner:, open:False, id:6797987c7834a53358fb4ebbd8b9b36c2c4a8f01, time:1379004764.14
@@ -32,3 +33,4 @@ comments containing a closing parenthesi
breaks on empty files | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:e40fa7a93eb2c497dca1af7eed22ad5ed5cfbe7f, time:1390325470.91
wisp-scheme: breaks on lines with only underscores. These should be treated as empty lines. | owner:, open:False, id:e464b5ce49deb14a80f67d50c6d70043ca9bde25, time:1415124488.16
quote as only char in a line gets parenthized instead of becoming a paren-prefix. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:eb7468387e90bb5d13f5a5d81c6f4a704f2ca0fb, time:1390326369.6
+in the REPL output can be delayed by one line: Appears only when submitting the next command. | owner:, open:True, id:f1e42bbd4c17a2dec886c26d9c14e770bcff66d2, time:1415972414.48
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1,7 +1,7 @@
dnl run `autoreconf -i` to generate a configure script.
dnl Then run ./configure to generate a Makefile.
dnl Finally run make to generate the project.
-AC_INIT([wisp], [0.8.0],
+AC_INIT([wisp], [0.8.1],
[arne_bab@web.de])
# Check for programs I need for my build
AC_CANONICAL_TARGET
diff --git a/docs/srfi.org b/docs/srfi.org
--- a/docs/srfi.org
+++ b/docs/srfi.org
@@ -1,10 +1,12 @@
-#+title: SRFI: wisp: whitespace-to-lisp preprocessing
-#+options: toc:nil num:t
+#+title: SRFI: wisp: simpler indentation-sensitive scheme
+#+options: toc:nil num:t ^:nil
+
+# wisp: indentation-based scheme project
#+BEGIN_ABSTRACT
This SRFI describes a simple syntax which allows making scheme easier to read for newcomers while keeping the simplicity, generality and elegance of s-expressions. Similar to SRFI-110, SRFI-49 and Python it uses indentation to group expressions. Like SRFI-110 wisp is general and homoiconic.
-Different from its precedessors, wisp only uses the absolute minimum of additional syntax-elements which are required for writing and exchanging arbitrary code-structures. As syntax elements it only uses a colon surrounded by whitespace, the period as first code-character on the line and underscores at the beginning of the line.
+Different from its precedessors, wisp only uses the absolute minimum of additional syntax-elements which are required for writing and exchanging arbitrary code-structures. As syntax elements it only uses a colon surrounded by whitespace, the period followed by whitespace as first code-character on the line and optional underscores followed by whitespace at the beginning of the line.
It resolves a limitation of SRFI-110 and SRFI-49, both of which force the programmer to use a single argument per line if the arguments to a function need to be continued after a function-call.
@@ -20,12 +22,14 @@ Wisp expressions can include any s-expre
#+BEGIN_SRC scheme
(define (hello who)
(format #t "~A ~A!\n"
- "Hello" who))
+ "Hello" who))
#+END_SRC
#+html: </td></tr></table>
#+END_ABSTRACT
+# to make indentation-based code safe to share in non-whitespace preserving environments
+
#+toc: headlines 2
* SRFI process :noexport:
@@ -80,25 +84,25 @@ Remember, even if a proposal becomes an
** Acknowledgments
-- Thanks for lots of constructive discussions goes to Alan Manuel K. Gloria and David A. Wheeler.
+- Thanks for many constructive discussions goes to Alan Manuel K. Gloria and David A. Wheeler.
- Also thanks to Mark Weaver for his help with the wisp parser and the guile integration - including a 20x speedup.
* Related SRFIs
- SRFI-49 (Indentation-sensitive syntax): superceded by this SRFI,
- SRFI-110 (Sweet-expressions (t-expressions)): alternative to this SRFI,
-- SRFI-105 (neoteric expressions and curly infix): supported in this SRFI by treating curly braces like brackets and parens.
+- SRFI-105 (neoteric expressions and curly infix): supported in this SRFI by treating curly braces like brackets and parentheses. Curly infix is required.
- SRFI-30 (Nested Multi-line comments): complex interaction. Should be avoided at the beginning of lines, because it can make the indentation hard to distinguish for humans. SRFI-110 includes them, so there might be value in adding them. The wisp reference implementation does not treat them specially, though, which might create arbitrary complications.
* Rationale
-A big strength of Scheme and other lisp-like languages is their minimalistic syntax. By using only the most common characters like the period, the comma, the quote and quasiquote, the hash, the semicolon and the parens for the syntax[fn:1], they are very close to natural language. Along with the minimal list-structure of the code, this gives these languages a timeless elegance.
+A big strength of Scheme and other lisp-like languages is their minimalistic syntax. By using only the most common characters like the period, the comma, the quote and quasiquote, the hash, the semicolon and the parens for the syntax (=.,"'`#;()=), they are very close to natural language[fn:1]. Along with the minimal list-structure of the code, this gives these languages a timeless elegance.
-But as SRFI-110 explains very thoroughly (which we need not repeat here), the parentheses at the beginning of lines hurt readability and scare away newcomers. Additionally using indentation to mark the structure of the code follows naturally from the observation that most programmers use indentation, with many programmers actually letting their editor indent code automatically to fit the structure. Indentation is an important way how programmers understand code and using it directly to define the structure avoids errors due to mismatches between indentation and actual meaning.
+But as SRFI-110 explains very thoroughly (which we need not repeat here), the parentheses at the beginning of lines hurt readability and scare away newcomers. Additionally using indentation to mark the structure of the code follows naturally from the observation that most programmers use indentation, with many programmers letting their editor indent code automatically to fit the structure. Indentation is an important way how programmers understand code and using it directly to define the structure avoids errors due to mismatches between indentation and actual meaning.
-As a solution to this, SRFI-49 and SRFI-110 provide a way to write whitespace sensitive scheme, but both have their share of problems.
+As a solution to this, SRFI-49 and SRFI-110 provide a way to write whitespace sensitive scheme, but both have their share of issues.
-As noted in SRFI-110, there are a number of implementation-problems in SRFI-49, as well as specification shortcomings like choosing the name “group” for the construct which is necessary to represent double parentheses. Additionally to the problems named in SRFI-110, SRFI-49 is not able to continue the arguments to a function on one line, if a prior argument was a function call. The example code in the abstract would have to be written in SRFI-49 as follows:
+As noted in SRFI-110, there are a number of implementation-problems in SRFI-49, as well as specification shortcomings like choosing the name “group” for the construct which is necessary to represent double parentheses. In addition to the problems named in SRFI-110, SRFI-49 is not able to continue the arguments to a function on one line, if a prior argument was a function call. The example code in the abstract would have to be written in SRFI-49 as follows:
#+BEGIN_SRC scheme
,* 5
@@ -107,7 +111,7 @@ As noted in SRFI-110, there are a number
1
#+END_SRC
-SRFI-110 improves a lot over the implementation of SRFI-49. It resolves the group-naming and tries to reduce the need to continue the argument-list by introducing 3 different grouping-syntaxes (=$=, =\\= and =<* *>=). These additional syntax-elements however hurt readability for newcomers a lot (obviously the authors of SRFI-110 disagree with this assertion. Their point is discussed in SRFI-110 in the section about wisp). The additional syntax elements lead to structures like the following:
+SRFI-110 improves a lot over the implementation of SRFI-49. It resolves the group-naming and reduces the need to continue the argument-list by introducing 3 different grouping syntaxes (=$=, =\\= and =<* *>=). These additional syntax-elements however hurt readability for newcomers (obviously the authors of SRFI-110 disagree with this assertion. Their view is discussed in SRFI-110 in the section about wisp). The additional syntax elements lead to structures like the following (taken from examples from the readable project):
#+BEGIN_SRC scheme
myfunction
x: \\ original-x
@@ -125,19 +129,21 @@ myfunction
This is not only hard to read, but also makes it harder to work with the code, because the programmer has to learn these additional syntax elements and keep them in mind before being able to understand the code.
-Like SRFI-49 SRFI-110 also cannot continue the argument-list without resorting to single-element lines, though it reduces this problem by advertising the use of neoteric expressions (SRFI-105).
+Like SRFI-49 SRFI-110 also cannot continue the argument-list without resorting to single-element lines, though it reduces this problem by the above grouping syntaxes and advertising the use of neoteric expressions from SRFI-105.
** Advantages of Wisp
-Wisp draws on the strength of SRFI-110 but avoids its complexities. It was actually conceived and improved in the discussions within the readable-project which preceded SRFI-110 and there is a comparison between readable in wisp in SRFI-110.
+Wisp draws on the strength of SRFI-110 but avoids its complexities. It was conceived and improved in the discussions within the readable-project which preceded SRFI-110 and there is a comparison between readable in wisp in SRFI-110.
-Like SRFI-110, wisp is general and homoiconic and interacts nicely with SRFI-105 (neoteric expressions and curly infix). Like SRFI-110, the expressions are the same in the REPL and in code-files. Like SRFI-110, wisp has been used for implementing multiple smaller programs, though the biggest program in wisp is still its implementation.
+Like SRFI-110, wisp is general and homoiconic and interacts nicely with SRFI-105 (neoteric expressions and curly infix). Like SRFI-110, the expressions are the same in the REPL and in code-files. Like SRFI-110, wisp has been used for implementing multiple smaller programs, though the biggest program in wisp is still its implementations (written in wisp and bootstrapped via a simpler wisp preprocessor).
But unlike SRFI-110, wisp only uses the minimum of additional syntax-elements which are necessary to support arbitrary code-structures with indentation-sensitive code which is intended to be shared over the internet. To realize these syntax-elements, it generalizes existing syntax and draws on the most common non-letter non-math characters in prose. This allows keeping the actual representation of the code elegant and inviting to newcomers.
** Disadvantages of Wisp
-Using the colon as syntax element keeps the code very close to written prose, but it can interfere with type definitions as for example used in Typed Racket[fn:6]. This can be mitigated in let- and lambda-forms by using the parenthesized form. When doing so, wisp avoid the double-paren for type-declarations and as such makes them easier to catch by eye. For function definitions (the only =define= call where type declarations are needed in typed-racket[fn:7]), a =declare= macro directly before the =define= should work well.
+Using the colon as syntax element keeps the code very close to written prose, but it can interfere with type definitions as for example used in Typed Racket[fn:6]. This can be mitigated in let- and lambda-forms by using the parenthesized form. When doing so, wisp avoids the double-paren for type-declarations and as such makes them easier to catch by eye. For function definitions (the only =define= call where type declarations are needed in typed-racket[fn:7]), a =declare= macro directly before the =define= should work well.
+
+Using the period to continue the argument list is unusual compared to other languages and as such can lead to errors when trying to return a variable from a procedure and forgetting the period.
* Specification
@@ -145,7 +151,7 @@ The specification is separated into four
** Overview
-The basic rules for wisp-code can be defined in 4 rules, each of which emerges directly from a requirement:
+The basics of wisp syntax can be defined in 4 rules, each of which emerges directly from a requirement:
*** Wisp syntax 1/4: function calls
@@ -165,6 +171,8 @@ becomes
(newline)
#+END_SRC
+/requirement: call functions without parenthesis./
+
*** Wisp syntax 2/4: Continue Argument list
The period:
@@ -185,6 +193,8 @@ becomes
This also works with just one argument after the period. To start a line without a function call, you have to prefix it with a period followed by whitespace.[fn:2]
+/requirement: continue the argument list of a function after an intermediate call to another function./
+
*** Wisp syntax 3/4: Double Parens
The colon:[fn:3]
@@ -207,6 +217,8 @@ becomes
(body))
#+END_SRC
+/requirement: represent code with two adjadent blocks in double-parentheses./
+
*** Wisp syntax 4/4: Resilient Indentation
The underscore (optional):
@@ -229,6 +241,7 @@ becomes
(body))
#+END_SRC
+/requirement: share code in environments which do not preserve whitespace./
*** Summary
@@ -242,14 +255,14 @@ The syntax shown here is the minimal syn
*** Unindented line
-*A line without indentation is a function call*, just as if it would start with a bracket.
+*A line without indentation is a function call*, just as if it would start with a parenthesis.
#+BEGIN_SRC wisp
display "Hello World!" ; (display "Hello World!")
#+END_SRC
*** Sibling line
-*A line which is more indented than the previous line is a sibling to that line*: It opens a new bracket.
+*A line which is more indented than the previous line is a sibling to that line*: It opens a new parenthesis.
#+BEGIN_SRC wisp
display ; (display
@@ -257,7 +270,7 @@ The syntax shown here is the minimal syn
#+END_SRC
*** Closing line
-*A line which is not more indented than previous line(s) closes the brackets of all previous lines which have higher or equal indentation*. You should only reduce the indentation to indentation levels which were already used by parent lines, else the behaviour is undefined.
+*A line which is not more indented than previous line(s) closes the parentheses of all previous lines which have higher or equal indentation*. You should only reduce the indentation to indentation levels which were already used by parent lines, else the behaviour is undefined.
#+BEGIN_SRC wisp
display ; (display
@@ -267,7 +280,7 @@ The syntax shown here is the minimal syn
*** Prefixed line
-*To add any of ' , ` #' #, #` or #@, to the first bracket on a line, just prefix the line with that symbol* followed by at least one space. Implementations are free to add more prefix symbols.
+*To add any of ' , ` #' #, #` or #@, to the first parenthesis on a line, just prefix the line with that symbol* followed by at least one space. Implementations are free to add more prefix symbols.
#+BEGIN_SRC wisp
' "Hello World!" ; '("Hello World!")
@@ -275,7 +288,7 @@ The syntax shown here is the minimal syn
*** Continuing line
-*A line whose first non-whitespace characters is a dot followed by a space (". ") does not open a new bracket: it is treated as simple continuation of the first less indented previous line*. In the first line this means that this line does not start with a bracket and does not end with a bracket, just as if you had directly written it in lisp without the leading ". ".
+*A line whose first non-whitespace characters is a dot followed by a space (". ") does not open a new parenthesis: it is treated as simple continuation of the first less indented previous line*. In the first line this means that this line does not start with a parenthesis and does not end with a parenthesis, just as if you had directly written it in lisp without the leading ". ".
#+BEGIN_SRC wisp
string-append "Hello" ; (string-append "Hello"
@@ -285,7 +298,7 @@ The syntax shown here is the minimal syn
*** Empty indentation level
-*A line which contains only whitespace and a colon (":") defines an indentation level at the indentation of the colon*. It opens a bracket which gets closed by the next less-indented line. If you need to use a colon by itself. you can escape it as "\:".
+*A line which contains only whitespace and a colon (":") defines an indentation level at the indentation of the colon*. It opens a parenthesis which gets closed by the next less-indented line. If you need to use a colon by itself. you can escape it as "\:".
#+BEGIN_SRC wisp
let ; (let
@@ -296,7 +309,7 @@ The syntax shown here is the minimal syn
*** Inline Colon
-*A colon sourrounded by whitespace (" : ") starts a bracket which gets closed at the end of the line*.
+*A colon sourrounded by whitespace (" : ") starts a parenthesis which gets closed at the end of the line*.
#+BEGIN_SRC wisp
define : hello who ; (define (hello who)
@@ -304,7 +317,7 @@ The syntax shown here is the minimal syn
string-append "Hello " who "!" ; (string-append "Hello " who "!")))
#+END_SRC
-If the colon starts a line, it starts a bracket which gets closed at the end of the line *and* defines an indentation level at the position of the colon.
+If the colon starts a line, it starts a parenthesis which gets closed at the end of the line *and* defines an indentation level at the position of the colon.
*** Initial Underscores
*You can replace any number of consecutive initial spaces by underscores*, as long as at least one whitespace is left between the underscores and any following character. You can escape initial underscores by prefixing the first one with \ ("\___ a" → "(___ a)"), if you have to use them as function names.
@@ -321,7 +334,7 @@ If the colon starts a line, it starts a
#+BEGIN_SRC wisp
define : stringy s
string-append s "can be varied as follows:
-"
+ "
string-capitalize s
string-reverse s
. (string-capitalize
@@ -333,9 +346,9 @@ define : stringy s
** Clarifications
-- Code-blocks end after 2 empty lines followed by a newline. Indented non-empty lines after 2 empty lines should be treated as error. A line is empty if it only contains whitespace.
+- Code-blocks end after 2 empty lines followed by a newline. Indented non-empty lines after 2 empty lines should be treated as error. A line is empty if it only contains whitespace. A line with a comment is never empty.
-- Inside parentheses wisp parsing is disabled. Consequently linebreaks inside parentheses are not considered linebreaks for wisp-parsing. For the parser everything which happens inside parentheses is considered as a black box.
+- Inside parentheses, wisp parsing is disabled. Consequently linebreaks inside parentheses are not considered linebreaks for wisp-parsing. For the parser everything which happens inside parentheses is treated as a black box.
- Square brackets and curly braces should be treated the same way as parentheses: They stop the indentation processing until they are closed.
@@ -343,6 +356,8 @@ define : stringy s
- A colon (:) at the beginning of a line adds an extra open parentheses that gets closed at end-of-line (rule 4.2.7) *and* defines an indentation level.
+- using a quote to escape a symbol separated from it by whitespace is forbidden. This would make the meaning of quoted lines ambigous.
+
* Syntax justification
/I do not like adding any unnecessary syntax element to lisp. So I want to show explicitely why the syntax elements are required./
@@ -354,23 +369,31 @@ See also http://draketo.de/light/english
** . (the dot)
+To represent general code trees, we have to be able to represent continuation of the arguments of a function with an intermediate call to another (or the same) function.
+
The dot at the beginning of the line as marker of the continuation of a variable list is a generalization of using the dot as identity function - which is an implementation detail in many lisps.
-`(. a)` is just `a`.
+#+BEGIN_QUOTE
+=(. a)= is just =a=
+#+END_QUOTE
-So for the single variable case, this would not even need additional parsing: wisp could just parse ". a" to "(. a)" and produce the correct result in most lisps. But forcing programmers to always use separate lines for each parameter would be very inconvenient, so the definition of the dot at the beginning of the line is extended to mean “take every element in this line as parameter to the parent function”.
+So for the single variable case, this would not even need additional parsing: wisp could just parse =. a= to =(. a)= and produce the correct result in most lisps. But forcing programmers to always use separate lines for each parameter would be very inconvenient, so the definition of the dot at the beginning of the line is extended to mean “take every element in this line as parameter to the parent function”.
-Essentially this dot-rule means that we mark variables in the code instead of function calls, since in Lisp variables at the beginning of a line are much rarer than in other programming languages. In lisp assigning a value to a variable is a function call while it is a syntax element in many other languages, so what would be a variable at the beginning of a line in other languages is a function call in lisp..
+#+BEGIN_QUOTE
+=(. a)= → =a= is generalized to =(. a b c)= → =a b c=.
+#+END_QUOTE
+
+At its core, this dot-rule means that we mark variables in the code instead of function calls. We do so, because variables at the beginning of a line are much rarer in Scheme than in other programming languages.
** : (the colon)
-For double brackets and for some other cases we must have a way to mark indentation levels without any code. I chose the colon, because it is the most common non-alpha-numeric character in normal prose which is not already reserved as syntax by lisp when it is surrounded by whitespace, and because it already gets used for marking keyword arguments to functions in Emacs Lisp, so it does not add completely alien characters.
+For double parentheses and for some other cases we must have a way to mark indentation levels which do not contain code. Wisp uses the colon, because it is the most common non-alpha-numeric character in normal prose which is not already reserved as syntax by Scheme when it is surrounded by whitespace, and because it already gets used without sourrounding whitespace for marking keyword arguments to functions in Emacs Lisp and Common Lisp, so it does not add completely alien concepts.
The inline function call via inline " : " is a limited generalization of using the colon to mark an indentation level: If we add a syntax-element, we should use it as widely as possible to justify adding syntax overhead.
-But if you need to use : as variable or function name, you can still do so by escaping it with a backslash, so this does not forbid using the character.
+But if you need to use =:= as variable or function name, you can still do so by escaping it with a backslash (=\:=), so this does not forbid using the character.
-For simple cases, the colon could be replaced by clever whitespace parsing, but there are complex cases which make this impossible. A simple example is a theoretical doublelet which does not require a body:[fn:4]
+For simple cases, the colon could be replaced by clever whitespace parsing, but there are complex cases which make this impossible. The minimal example is a theoretical doublelet which does not require a body:[fn:4]
#+BEGIN_SRC scheme
(doublelet
@@ -384,7 +407,7 @@ The wisp version of this is
doublelet
:
foo bar
- : ; <- this double backstep is the real issue
+ : ; <- this empty backstep is the real issue
bla foo
#+END_SRC
@@ -396,7 +419,7 @@ doublelet
: bla foo
#+END_SRC
-The need to be able to represent things like this is the real reason, why the colon exists. The inline and start-of-line use is only a generalization of that principle (we add a syntax-element, so we should see how far we can push it to reduce the effective cost of introducing the additional syntax).
+The need to be able to represent arbitrary syntax trees which can contain expressions like this is the real reason, why the colon exists. The inline and start-of-line use is only a generalization of that principle (we add a syntax-element, so we should see how far we can push it to reduce the effective cost of introducing the additional syntax).
*** Clever whitespace-parsing which would not work
@@ -405,59 +428,84 @@ There are two alternative ways to tackle
Defining intermediate indentation-levels by later elements (deferred definition) would be a problem, because it would create code which is really hard to understand. An example is the following:
#+BEGIN_SRC wisp
-defun flubb
-
+define (flubb)
nubb
+ hubb
+ subb
gam
#+END_SRC
would become
#+BEGIN_SRC scheme
-(defun flubb ()
+(define (flubb)
((nubb))
+ ((hubb))
+ ((subb))
(gam))
#+END_SRC
+while
+
+#+BEGIN_SRC wisp
+define (flubb)
+ nubb
+ hubb
+ subb
+#+END_SRC
+
+would become
+
+#+BEGIN_SRC scheme
+(define (flubb)
+ (nubb)
+ (hubb)
+ (subb))
+#+END_SRC
+
+Knowledge of later parts of the code would be necessary to understand the parts a programmer is working on at the moment. This would call for subtle errors which would be hard to track down, because the effect of a change in code would not be localized at the point where the change is done but could propagate backwards.
+
Fixed indentation width (alternative option to inferring it from later lines) would make it really hard to write readable code. Stuff like this would not be possible:
#+BEGIN_SRC wisp
-if
- equals wrong
+when
+ equal? wrong
isright? stuff
fixstuff
#+END_SRC
** _ (the underscore)
-In Python the whitespace hostile html already presents problems with sharing code - for example in email list archives and forums. But in Python the indentation can mostly be inferred by looking at the previous line: If that ends with a colon, the next line must be more indented (there is nothing to clearly mark reduced indentation, though). In wisp we do not have this help, so we need a way to survive in that hostile environment.
+In Python the whitespace hostile html already presents problems with sharing code - for example in email list archives and forums. But Python-programmers can mostly infer the indentation by looking at the previous line: If that ends with a colon, the next line must be more indented (there is nothing to clearly mark reduced indentation, though). In wisp we do not have this support, so we need a way to survive in the hostile environment of todays web.
-The underscore is commonly used to denote a space in URLs, where spaces are inconvenient, but it is rarely used in lisp (where the dash ("-") is mostly used instead), so it seems like a a natural choice.
+The underscore is commonly used to denote a space in URLs, where spaces are inconvenient, but it is rarely used in Scheme (where the dash ("-") is mostly used instead), so it seems like a a natural choice.
You can still use underscores anywhere but at the beginning of the line, and even at the beginning of the line you simply need to escape it by prefixing the first underscore with a backslash ("\____").
* Implementation
-This reference implementation realizes a general wisp-preprocessor which can be used for any lisp-like language. It contains special syntax-constructs for scheme, though. The reference-preprocessor uses GNU Guile and can also be used at the REPL. Due to being a simple preprocessor, wisp can also be implemented as an external program which gets called on reading. It does not actually have to understand the code itself.
+This reference implementation realizes a specialized parser for Scheme. It uses GNU Guile and can also be used at the REPL.
-A wisp-preprocessor which is specialiized for scheme should be much easier to realize, though, by using the parsing methods from an existing scheme implementation.
+The wisp code also contains a general wisp-preprocessor which can be used for any lisp-like language and can used as an external program which gets called on reading. It does not actually have to understand the code itself. This is not part of this SRFI, though.
-Since generality of the reference implementation makes it quite heavyweight, it would be great to have someone step up and create a more lightweight scheme-specific alternative. To allow for this, the test-suite in the next chapter only contains scheme-specific snippets.
+To allow for easy re-implementation, the chapter after the implementation itself contains a test-suite with commonly used wisp constructs and parenthesized counterparts.
-** The generic wisp preprocessor (code)
+The wisp preprocessor implementation can be found at http://draketo.de/proj/wisp. Both implementations are explicitly licensed to allow inclusion in an SRFI.
+
+** The generic wisp processor (code)
TODO: Include the code from http://draketo.de/proj/wisp
** Test Suite
-The wisp test-suite consists of a large number of wisp-snippets and the corresponding scheme-code. A wisp-implementation may call itself compliant to the wisp test-suite if it successfully converts each wisp-snippet into the corresponging scheme-snippet. Blank lines at the end of the file and non-functional white-space in the produced scheme-file do not matter for this purpose.
+The wisp test-suite consists of a large number of wisp-snippets and the corresponding scheme-code.
+
+A wisp-implementation may call itself compliant with the wisp test-suite if the code tree parsed from the wisp file is the same as a code tree parsed from the equivalent Scheme file.
+
+A wisp-implementation may call itself a compliant wisp pre-processor if it successfully converts each wisp-snippet into the corresponging scheme-snippet. Blank lines at the end of the file and non-functional white-space in the produced scheme-file do not matter for this purpose.
This test-suite is also available in the [[http://draketo.de/proj/wisp][wisp repository]] along with a script-runner (runtests.sh) which tests the reference wisp-implementation with GNU Guile against this testsuite.[fn:5]
-The test-suite included here only contains scheme-compatible code to allow for scheme-specific wisp-implementations which use existing parsing functions to simplify the code.
-
-/TODO: Some of the snippets were transformed from emacs lisp to scheme by hand and this might have introduced bugs. They still need to be tested again./
-
*** tests/syntax-underscore.w
#+begin_src wisp
define : a b c
@@ -1028,7 +1076,7 @@ do not break me!"
h (I am in brackets:
do not : change "me")
. i
- , ' j k
+ , 'j k
. l
@@ -1090,7 +1138,7 @@ do not break me!")
(h (I am in brackets:
do not : change "me"))
i)))
- ,(' j k)
+ ,('j k)
l
@@ -1195,6 +1243,6 @@ newline
[fn:5] To run the tests in the wisp testsuite with a separately built GNU Guile, you can use any given guile interpreter by adjusting the following command: =PATH=~/guile-2.0.11/meta:${PATH} ./runtests.sh=
-[fn:6] Typed Racket uses calls of the form =(: x Number)= to declare types. These forms can still be used in parenthesized form, but not in wisp-form (as the colon has to be replaced with =\:=).
+[fn:6] Typed Racket uses calls of the form =(: x Number)= to declare types. These forms can still be used directly in parenthesized form, but in wisp-form the colon has to be replaced with =\:=.
[fn:7] In most cases type-declarations are not needed in typed racket, since the type can be inferred. See [[http://docs.racket-lang.org/ts-guide/more.html?q=typed#%28part._when-annotations~3f%29][When do you need type annotations?]]
diff --git a/wisp-reader.w b/wisp-reader.w
--- a/wisp-reader.w
+++ b/wisp-reader.w
@@ -10,7 +10,7 @@
; adapted from spec.scm: https://gitorious.org/nacre/guile-sweet/source/ae306867e371cb4b56e00bb60a50d9a0b8353109:sweet/spec.scm
define-module : language wisp spec
- . #:use-module : wisp
+; . #:use-module : wisp
. #:use-module : wisp-scheme
. #:use-module : system base compile
. #:use-module : system base language
@@ -32,36 +32,48 @@ define : decompile-scheme x e opts
define wisp-pending-port : make-object-property
; Code thanks to Mark Weaver
+; define : read-one-wisp-sexp port env
+; define : read-wisp-chunk
+; if : eof-object? : peek-char port
+; read-char port ; return eof: we’re done
+; let : : s : wisp2lisp : wisp-chunkreader port
+; set! : wisp-pending-port port
+; open-input-string s
+; try-pending
+; define : try-pending
+; let : : pending-port : wisp-pending-port port
+; if pending-port
+; let : : x : read pending-port
+; if : eof-object? x
+; read-wisp-chunk
+; . x
+; read-wisp-chunk
+; try-pending
+
+
+define wisp-pending-sexps '()
+
define : read-one-wisp-sexp port env
- define : read-wisp-chunk
- if : eof-object? : peek-char port
- read-char port ; return eof: we’re done
- let : : s : wisp2lisp : wisp-chunkreader port
- set! : wisp-pending-port port
- open-input-string s
- try-pending
+ define : wisp-scheme-read-chunk-env
+ if : eof-object? : peek-char port
+ read-char port ; return eof: we’re done
+ begin
+ set! wisp-pending-sexps
+ append wisp-pending-sexps : wisp-scheme-read-chunk port
+ try-pending
define : try-pending
- let : : pending-port : wisp-pending-port port
- if pending-port
- let : : x : read pending-port
- if : eof-object? x
- read-wisp-chunk
- . x
- read-wisp-chunk
+ if : null? wisp-pending-sexps
+ wisp-scheme-read-chunk-env
+ let : : sexp : car wisp-pending-sexps
+ set! wisp-pending-sexps : cdr wisp-pending-sexps
+ . sexp
try-pending
-define : wisp-scheme-read-chunk-env port env
- if : eof-object? : peek-char port
- read-char port ; return eof: we’re done
- cons 'begin : wisp-scheme-read-chunk port
-
define-language wisp
. #:title "Wisp Scheme Syntax THIS IS EXPERIMENTAL, USE AT YOUR OWN RISK"
; . #:reader read-one-wisp-sexp
- . #:reader wisp-scheme-read-chunk-env
- . #:compilers `((scheme . ,compile-scheme)) ; this is scheme, not
- ; wisp, because I do not
- ; touch quasiquotes yet.
+ . #:reader : lambda (port env) : let ((x (read-one-wisp-sexp port env))) x
+ . #:compilers `((scheme . ,compile-scheme))
. #:decompilers `((scheme . ,decompile-scheme))
. #:evaluator : lambda (x module) : primitive-eval x
. #:printer write
diff --git a/wisp-scheme.w b/wisp-scheme.w
--- a/wisp-scheme.w
+++ b/wisp-scheme.w
@@ -1,4 +1,5 @@
-#!/home/arne/wisp/wisp-multiline.sh
+#!/bin/bash
+exec guile -L . --language=wisp -s "$0" "$@"
; !#
;; Scheme-only implementation of a wisp-preprocessor which output a
@@ -14,11 +15,7 @@
;; directly create a list of codelines with indentation. For this we
;; then simply reuse the appropriate function from the generic wisp
;; preprocessor.
-;;
-;; TODO: use match:
-;; (use-modules (ice-9 match))
-;; (define dot (call-with-input-string "." read))
-;; (match (list 'u 'v dot 'w) ((a ... b '#{.}# c) (append a (cons b c))) )
+
define-module : wisp-scheme
. #:export (wisp-scheme-read-chunk wisp-scheme-read-all
@@ -523,7 +520,7 @@ define : wisp-replace-empty-eof code
. "replace ((#<eof>)) by ()"
; FIXME: Actually this is a hack which fixes a bug when the
; parser hits files with only hashbang and comments.
- if : and (pair? (car code)) (eof-object? (car (car code))) (null? (cdr code)) (null? (cdr (car code)))
+ if : and (not (null? code)) (pair? (car code)) (eof-object? (car (car code))) (null? (cdr code)) (null? (cdr (car code)))
list
. code
@@ -578,25 +575,27 @@ Match is awesome!"
map wisp-make-improper a
a
. a
- define : syntax-error li
- throw 'wisp-syntax-error (format #f "incorrect dot-syntax #{.}# in code: not a proper pair: ~A" li)
- let check
+ define : syntax-error li msg
+ throw 'wisp-syntax-error (format #f "incorrect dot-syntax #{.}# in code: ~A: ~A" msg li)
+ if #t
+ . improper
+ let check
: tocheck improper
match tocheck
; lists with only one member
: 'REPR-DOT-e749c73d-c826-47e2-a798-c16c13cb89dd
- syntax-error tocheck
+ syntax-error tocheck "list with the period as only member"
; list with remaining dot.
: a ...
- if : member repr-dot a
- syntax-error tocheck
+ if : and (member repr-dot a)
+ syntax-error tocheck "leftover period in list"
map check a
- ; simple pair
+ ; simple pair - this and the next do not work when parsed from wisp-scheme itself. Why?
: 'REPR-DOT-e749c73d-c826-47e2-a798-c16c13cb89dd . c
- syntax-error tocheck
+ syntax-error tocheck "dot as first element in already improper pair"
; simple pair, other way round
: a . 'REPR-DOT-e749c73d-c826-47e2-a798-c16c13cb89dd
- syntax-error tocheck
+ syntax-error tocheck "dot as last element in already improper pair"
; more complex pairs
: ? pair? a
let
@@ -604,11 +603,11 @@ Match is awesome!"
tail : last-pair a
cond
: equal? repr-dot : car tail
- syntax-error tocheck
+ syntax-error tocheck "equal? repr-dot : car tail"
: equal? repr-dot : cdr tail
- syntax-error tocheck
+ syntax-error tocheck "equal? repr-dot : cdr tail"
: member repr-dot head
- syntax-error tocheck
+ syntax-error tocheck "member repr-dot head"
else
. a
a
@@ -733,3 +732,4 @@ define : wisp-scheme-read-string-chunk s
; map primitive-eval : wisp-scheme-read-file "wisp-guile.w" ; actually runs wisp-guile.w with the arguments supplied to this script.
; uncomment the previous line, then run the next line in the shell. If 1 and 2 are equal, this parser works!
; guile wisp.scm wisp-scheme.w > wisp-scheme.scm; guile wisp-scheme.scm wisp-guile.w > 1; guile wisp.scm wisp-guile.w > 2; diff 1 2
+