(Arne Babenhauserheide)
2014-11-06: full featured wisp using the scheme (read) which passes the stable v0.8.0 full featured wisp using the scheme (read) which passes the testsuite. Requires curly infix now.
diff --git a/.bugs/bugs b/.bugs/bugs
--- a/.bugs/bugs
+++ b/.bugs/bugs
@@ -1,28 +1,34 @@
-testsuite: to pass, the tree-il has to match, not the emitted string. This allows for reader-only implementations. | owner:, open:True, id:00b74a730bbf076e73166e817ca7b0a273b376d4, time:1408224636.42
+testsuite: to pass, the tree-il has to match, not the emitted string. This allows for reader-only implementations. | owner:, open:False, id:00b74a730bbf076e73166e817ca7b0a273b376d4, time:1408224636.42
+wisp-scheme: unescape \_ and \: | owner:, open:False, id:086f61a06e16f1ef56e9917453bbd55b5879d15d, time:1415121255.99
fails when I add stuff at the end of end of example.w | owner:, open:False, id:08c68e1ce0c9798184c01806d2661a3220bff3cd, time:1363789693.79
wisp-mode in quoted lists only the first item is colorized as data, but all words up to the last paren should be colorized. | owner:, open:True, id:1675ca3f894ed8470fa292149a476a2fa0d17140, time:1397196957.45
add a testsuite for wisp parsers. | owner:, open:False, id:1c05d27ac916e1a823b8985a094947907c3c19af, time:1379064922.74
wisp-mode: export wisp to html fails in org-mode: font-lock-fontify-keywords-region: Invalid regexp | owner:, open:False, id:1e46d8c05580c961c37a32d36c987a5dd1d34943, time:1389371020.39
an empty line with : should start with double parens | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:2e188ddf44d36e4605030d3c58607ebfa97d189e, time:1390328674.43
wisp-in-wisp: remove the leading . for continuation lines after inferring the brackets. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:2e42e5b64622f0cc383eb8acc3d510912e925bf0, time:1377476687.79
+'() gives REPR-QUOTE-... | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:30c42de75c137f483245898e2a62af1e65cf19a6, time:1415060388.34
wisp: handle general paren prefixes like quote, quasiquote, etc. | owner:, open:False, id:323ff94b5be635742619467e1cb44f4c0d96f63f, time:1379047798.47
throw an exception when reducing indentation to an unknown indentation level. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:424186bd85f186b7279c5c59e2bd42f847284719, time:1376003568.91
wisp-in-wisp: only parses the first 272 lines, for some reason. | owner:, open:False, id:4cb6c556d7136609e2da9ab3fc045a39847f1ef3, time:1377014682.98
+adjust runtests.sh to use testrunner.w | owner:, open:True, id:4d4e76343fe09f0ec72a3e5eb0077bd16e12f9d5, time:1415127234.43
implement wisp in wisp | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:6299306916706410702029289bf32edab1e7f17c, time:1367113341.49
support other types of brackets, like square brackets. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:6749f3abcb9455eac9271efd8265797bce114239, time:1389134151.98
linebreaks in parens still break | owner:, open:False, id:6797987c7834a53358fb4ebbd8b9b36c2c4a8f01, time:1379004764.14
wisp-guile.w does not yet remove the leading . which signifies a continuation. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:7075096a323933493f42a3751da5f75a45d5d699, time:1377381700.17
inline ' : should be '( | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:72d534a8b23b4cb168017f1bb7d8816f0ea170c4, time:1366497335.26
failing test tests/shebang.w | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:74a851f83af8996465a7b24d8453161beb0f0fd5, time:1379106761.57
-non-nested multiline comments with #! !#. Requires restructuring. | owner:, open:True, id:7a57614fa920b2ddad002d044b144d0bb7c34f84, time:1389364108.01
-support nested multi-line comments with #| ... |#: multiline comments (srfi-30). Requires restructuring. | owner:Arne Babenhauserheide <bab@draketo.de>, open:True, id:8cf6202873d4454f57813dd17cf60432059f7c62, time:1389569421.6
-the repl does not require 3 returns when you use a single char as function, or rather only does so every second time | owner:, open:True, id:9cedd0bdbf4a3b17add4bfe86ad5a23e500cfc6c, time:1379064870.78
+non-nested multiline comments with #! !#. Requires restructuring. | owner:, open:False, id:7a57614fa920b2ddad002d044b144d0bb7c34f84, time:1389364108.01
+wisp-guile: support nested multi-line comments with #| ... |#: multiline comments (srfi-30). Requires restructuring. | owner:Arne Babenhauserheide <bab@draketo.de>, open:True, id:8cf6202873d4454f57813dd17cf60432059f7c62, time:1389569421.6
+wisp-guile: the repl does not require 3 returns when you use a single char as function, or rather only does so every second time | owner:, open:True, id:9cedd0bdbf4a3b17add4bfe86ad5a23e500cfc6c, time:1379064870.78
wisp-guile.w breaks on ";" and complex brackets with bracket char literals. See wisp-guile.w::91 | owner:, open:False, id:9d8b6f87fa5365733fc8655614dbf2a9ba5bd054, time:1377533321.27
+wisp-scheme: treat quote quasiquote syntax quasisyntax and so on similar to the dot. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:9e587f38115099c0825c13a2159605e5f560aeec, time:1413623986.49
FIX regression: empty line with only : does not remove the :. It transforms to (:, it should transform to ( | owner:, open:False, id:a2323d347612425bc5af577c939916c8b60ec1c9, time:1389631450.78
wisp-mode: handle lines starting with underscores: currently sees the underscores as function call. | owner:Arne Babenhauserheide <bab@draketo.de>, open:True, id:b2c3755e1deb8355655a334d569679e2e62d2836, time:1376612093.55
-parens in comments can throw off the parser. | owner:, open:True, id:ce28d6c0d1f9894c9b946e56b17934473800edfe, time:1408224406.79
+wisp-scheme: curly braces should use read. | owner:, open:False, id:c0e62db4dcfdf4f1cf84f1871e439d055dacbefa, time:1415121781.25
+wisp-guile: parens in strings can throw off the parser | owner:, open:True, id:ce28d6c0d1f9894c9b946e56b17934473800edfe, time:1408224406.79
make this work: let : : origfile ( open-file : nth 1 : command-line ) r | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:d6de2074a5017f1f29f34d142ce797981ed270a0, time:1366529287.67
wisp.py breaks on \ - quote, escaped backslash, quote. Ignored, because wisp.py is only needed for bootstrapping. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:d75a93ca58ade5b3e3e51f1e7ee9782e743ac131, time:1377424552.02
comments containing a closing parenthesis can break the parser. | owner:, open:False, id:d9147504868960e5fbc2648474d48ce5c9bd1a02, time:1374838747.22
breaks on empty files | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:e40fa7a93eb2c497dca1af7eed22ad5ed5cfbe7f, time:1390325470.91
+wisp-scheme: breaks on lines with only underscores. These should be treated as empty lines. | owner:, open:False, id:e464b5ce49deb14a80f67d50c6d70043ca9bde25, time:1415124488.16
quote as only char in a line gets parenthized instead of becoming a paren-prefix. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:eb7468387e90bb5d13f5a5d81c6f4a704f2ca0fb, time:1390326369.6
diff --git a/.bugs/details/4d4e76343fe09f0ec72a3e5eb0077bd16e12f9d5.txt b/.bugs/details/4d4e76343fe09f0ec72a3e5eb0077bd16e12f9d5.txt
new file mode 100644
--- /dev/null
+++ b/.bugs/details/4d4e76343fe09f0ec72a3e5eb0077bd16e12f9d5.txt
@@ -0,0 +1,28 @@
+# Lines starting with '#' and sections without content
+# are not displayed by a call to 'details'
+#
+[paths]
+# Paths related to this bug.
+# suggested format: REPO_PATH:LINENUMBERS
+
+
+[details]
+# Additional details
+after the fix:
+wisp-scheme now passes the testsuite, tested by a testrunner which checks the code-tree, not the string-representation.
+
+
+[expected]
+# The expected result
+
+
+[actual]
+# What happened instead
+
+
+[reproduce]
+# Reproduction steps
+
+
+[comments]
+# Comments and updates - leave your name
diff --git a/Makefile.am b/Makefile.am
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,8 +1,8 @@
-wisp = wisp.scm language/wisp/spec.scm
+wisp = wisp.scm wisp-scheme.scm language/wisp/spec.scm
wisp_SOURCES = wisp-guile.w wisp-reader.w wisp-scheme.w
-EXTRA_DIST = $(wisp_SOURCES) $(wisp_DATA) bootstrap.sh wisp.py examples tests runtests.sh wisp-repl-guile.sh
+EXTRA_DIST = $(wisp_SOURCES) $(wisp_DATA) bootstrap.sh wisp.py examples tests wisp-repl-guile.sh testrunner.w
CLEANFILES = ${wisp} ChangeLog
-DISTCLEANFILES = ${CLEANFILES} @abs_top_builddir@/1 @abs_top_builddir@/2 @abs_top_builddir@/syntaxtests.sh
+DISTCLEANFILES = ${CLEANFILES} @abs_top_builddir@/1 @abs_top_builddir@/2 @abs_top_builddir@/syntaxtests.sh @abs_top_builddir@/syntaxtestsreader.sh
# don't spout out lots of stuff at each distcheck. Disable for debugging.
AM_DISTCHECK_CONFIGURE_FLAGS="--quiet"
@@ -20,9 +20,15 @@ input.in.intermediate: ${wisp_SOURCES}
@abs_top_srcdir@/bootstrap.sh @abs_top_srcdir@ @guile@ @python3@ 2>&1 | sed "s/^;;;.*//" 2>&1 | grep . 1>&2 ; test ! $$? -eq 0 # grep did not find anything
.PHONY: syntaxtests.sh
-syntaxtests.sh : wisp.scm runtests.sh
- echo '#!/bin/bash' > @abs_top_builddir@/syntaxtests.sh
- echo @abs_top_srcdir@/runtests.sh @abs_top_srcdir@ @abs_top_builddir@ >> @abs_top_builddir@/syntaxtests.sh
- chmod +x @abs_top_builddir@/syntaxtests.sh
+syntaxtests.sh : wisp.scm tests/runtests-scheme-preprocessor.sh
+ echo '#!/bin/bash' > @abs_top_builddir@/$@
+ echo @abs_top_srcdir@/tests/runtests-scheme-preprocessor.sh @abs_top_srcdir@ @abs_top_builddir@ >> @abs_top_builddir@/$@
+ chmod +x @abs_top_builddir@/$@
-TESTS=syntaxtests.sh
+.PHONY: syntaxtestsreader.sh
+syntaxtestsreader.sh : wisp.scm tests/runtests-scheme-reader.sh
+ echo '#!/bin/bash' > @abs_top_builddir@/$@
+ echo @abs_top_srcdir@/tests/runtests-scheme-reader.sh @abs_top_srcdir@ @abs_top_builddir@ >> @abs_top_builddir@/$@
+ chmod +x @abs_top_builddir@/$@
+
+TESTS=syntaxtests.sh syntaxtestsreader.sh
diff --git a/NEWS b/NEWS
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,11 @@
+wisp 0.8.0 (2014-11-03):
+- the testsuite is defined in terms of the code-tree, not in terms of
+ the readable file. Implemented in testrunner.w
+- wisp-scheme passes the wisp testsuite
+- wisp uses curly-infix by default (srfi-105)
+- this is the first full featured release of wisp-scheme: wisp which
+ defers all complex parsing to the scheme reader (and match).
+
wisp 0.6.6 (2014-10-16):
- fix boostrap.
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1,7 +1,7 @@
dnl run `autoreconf -i` to generate a configure script.
dnl Then run ./configure to generate a Makefile.
dnl Finally run make to generate the project.
-AC_INIT([wisp], [0.6.6],
+AC_INIT([wisp], [0.8.0],
[arne_bab@web.de])
# Check for programs I need for my build
AC_CANONICAL_TARGET
diff --git a/testrunner.w b/testrunner.w
new file mode 100755
--- /dev/null
+++ b/testrunner.w
@@ -0,0 +1,36 @@
+#!/bin/bash
+exec guile -L . --language=wisp -s "$0" "$@"
+!#
+
+when : not : = 3 : length : command-line
+ format #t "Usage: ~A <wisp file> <scheme file>\n" : list-ref (command-line) 0
+ exit
+
+define wisp-file : list-ref (command-line) 1
+define scheme-file : list-ref (command-line) 2
+
+use-modules
+ srfi srfi-1
+ wisp-scheme
+
+
+define : read-all port
+ let readloop : : res : '
+ let : : next : read port
+ if : eof-object? next
+ . res
+ readloop : append res : list next
+
+let
+ :
+ read-scheme
+ with-input-from-file scheme-file
+ λ ()
+ read-all : current-input-port
+ parsed-wisp
+ with-input-from-file wisp-file
+ λ ()
+ wisp-scheme-read-all : current-input-port
+ if : equal? parsed-wisp read-scheme
+ format #t "Files ~A and ~A have equivalent content.\n" scheme-file wisp-file
+ format #t "Files ~A and ~A are different!\n\nwisp: ~A\n\nscheme: ~A\n\n\n" scheme-file wisp-file parsed-wisp read-scheme
diff --git a/tests/emacs-customization.scm b/tests/emacs-customization.scm
--- a/tests/emacs-customization.scm
+++ b/tests/emacs-customization.scm
@@ -1,7 +1,8 @@
(if (file-directory-p "~/.emacs.d/private/journal/")
(setq-default journal-dir "~/.emacs.d/private/journal/"))
-(global-set-key [(control meta .)] 'goto-last-change-reverse)
+; the following line is not valid scheme and as such would break the scheme tests.
+; global-set-key [(control meta .)] 'goto-last-change-reverse
(require 'org-latex)
(add-to-list 'org-export-latex-packages-alist
diff --git a/tests/emacs-customization.w b/tests/emacs-customization.w
--- a/tests/emacs-customization.w
+++ b/tests/emacs-customization.w
@@ -1,7 +1,8 @@
if : file-directory-p "~/.emacs.d/private/journal/"
setq-default journal-dir "~/.emacs.d/private/journal/"
-global-set-key [(control meta .)] 'goto-last-change-reverse
+; the following line is not valid scheme and as such would break the scheme tests.
+; global-set-key [(control meta .)] 'goto-last-change-reverse
require 'org-latex
add-to-list 'org-export-latex-packages-alist
@@ -55,4 +56,4 @@ custom-set-variables
setq a "x"
. b "y"
- . c "z"
\ No newline at end of file
+ . c "z"
diff --git a/tests/example.scm b/tests/example.scm
--- a/tests/example.scm
+++ b/tests/example.scm
@@ -12,7 +12,7 @@ do not break me!")
(h (I am in brackets:
do not : change "me"))
i)))
- ,(' j k)
+ ,('j k)
l
diff --git a/tests/example.w b/tests/example.w
--- a/tests/example.w
+++ b/tests/example.w
@@ -12,7 +12,7 @@ do not break me!"
h (I am in brackets:
do not : change "me")
. i
- , ' j k
+ , 'j k
. l
diff --git a/tests/hello.scm b/tests/hello.scm
--- a/tests/hello.scm
+++ b/tests/hello.scm
@@ -1,5 +1,5 @@
(define (hello who)
(format #t "~A ~A!\n"
"Hello" who))
+(hello "Wisp")
-
diff --git a/tests/hello.w b/tests/hello.w
--- a/tests/hello.w
+++ b/tests/hello.w
@@ -1,3 +1,4 @@
define : hello who
format #t "~A ~A!\n"
. "Hello" who
+hello "Wisp"
diff --git a/runtests.sh b/tests/runtests-scheme-preprocessor.sh
rename from runtests.sh
rename to tests/runtests-scheme-preprocessor.sh
--- a/runtests.sh
+++ b/tests/runtests-scheme-preprocessor.sh
@@ -17,6 +17,8 @@ fi
failed=0
cd ${srcdir}/tests
for i in *.w; do
+ # skip strangecomments
+ if test x"${i}" = x"strangecomments.w"; then continue; fi
d=$(guile ${builddir}/wisp.scm "$i" > ${builddir}/testtempfoo.scm; diff -wuB ${builddir}/testtempfoo.scm "$(basename "$i" .w)".scm; rm ${builddir}/testtempfoo.scm)
if test x"$d" = x""; then
continue
diff --git a/tests/runtests-scheme-reader.sh b/tests/runtests-scheme-reader.sh
new file mode 100755
--- /dev/null
+++ b/tests/runtests-scheme-reader.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# Simple test runner for wisp, mainly intended to be run by autotools.
+
+if [[ x"$1" == x"" || x"$1" == x"." ]]; then
+ srcdir="$(realpath "$(pwd)")"
+else
+ srcdir="$(realpath "$1")"
+fi
+
+if [[ x"$2" == x"" || x"$2" == x"." ]]; then
+ builddir="$(realpath $(pwd))"
+else
+ builddir="$(realpath "$2")"
+fi
+
+failed=0
+cd ${builddir}
+for i in ${srcdir}/tests/*.w; do
+ d=$(guile -L . --language=wisp ${srcdir}/testrunner.w "${i}" "${srcdir}/tests/$(basename "${i}" .w).scm")
+ if test $? -eq 0; then
+ continue
+ else
+ echo test "$i" failed. Diff: "$d"
+ failed=$((failed + 1))
+ fi
+done
+cd - >/dev/null # undo dir change
+# if test $failed -eq 0; then echo "Tests succeeded";
+# else echo "tests failed: ${failed}";
+# fi
+exit $failed
diff --git a/tests/strangecomments.scm b/tests/strangecomments.scm
--- a/tests/strangecomments.scm
+++ b/tests/strangecomments.scm
@@ -1,17 +1,18 @@
+(use-modules (wisp-scheme))
; works
(display
- (call-with-input-string " foo ; bar\n ; nop \n\n; nup\n; nup \n \n\n\n foo : moo \"\n\" \n___ . goo . hoo" wisp-scheme-read-chunk))
+ (call-with-input-string "foo ; bar\n ; nop \n\n; nup\n; nup \n \n\n\n foo : moo \"\n\" \n___ . goo . hoo" wisp-scheme-read-chunk))
(newline)
(display
- (call-with-input-string " foo \n___. goo . hoo" wisp-scheme-read-chunk))
+ (call-with-input-string "foo \n___ . goo . hoo" wisp-scheme-read-chunk))
(newline)
; broken
; expected:
(display
- (call-with-input-string " (foo) ; bar\n ; nop \n\n; nup\n; nup \n \n\n\n foo : moo \"\n\" \n___ . [goo . hoo]" wisp-scheme-read-chunk))
+ (call-with-input-string "(foo) ; bar\n ; nop \n\n; nup\n; nup \n \n\n\n foo : moo \"\n\" \n___ . [goo . hoo]" wisp-scheme-read-chunk))
(newline)
(display
- (call-with-input-string " foo \n___. [goo . hoo]" wisp-scheme-read-chunk))
+ (call-with-input-string "foo \n___ . [goo . hoo]" wisp-scheme-read-chunk))
(newline)
diff --git a/tests/strangecomments.w b/tests/strangecomments.w
--- a/tests/strangecomments.w
+++ b/tests/strangecomments.w
@@ -1,16 +1,17 @@
+use-modules : wisp-scheme
; works
display
- call-with-input-string " foo ; bar\n ; nop \n\n; nup\n; nup \n \n\n\n foo : moo \"\n\" \n___ . goo . hoo" wisp-scheme-read-chunk
+ call-with-input-string "foo ; bar\n ; nop \n\n; nup\n; nup \n \n\n\n foo : moo \"\n\" \n___ . goo . hoo" wisp-scheme-read-chunk
newline
display
- call-with-input-string " foo \n___. goo . hoo" wisp-scheme-read-chunk
+ call-with-input-string "foo \n___ . goo . hoo" wisp-scheme-read-chunk
newline
; broken
; expected:
display
- call-with-input-string " (foo) ; bar\n ; nop \n\n; nup\n; nup \n \n\n\n foo : moo \"\n\" \n___ . [goo . hoo]" wisp-scheme-read-chunk
+ call-with-input-string "(foo) ; bar\n ; nop \n\n; nup\n; nup \n \n\n\n foo : moo \"\n\" \n___ . [goo . hoo]" wisp-scheme-read-chunk
newline
display
- call-with-input-string " foo \n___. [goo . hoo]" wisp-scheme-read-chunk
+ call-with-input-string "foo \n___ . [goo . hoo]" wisp-scheme-read-chunk
newline
diff --git a/wisp-guile.w b/wisp-guile.w
--- a/wisp-guile.w
+++ b/wisp-guile.w
@@ -8,12 +8,13 @@
; we need to be able to replace end-of-line characters in brackets and strings
-;; nostringandbracketbreaks INPORT
-;;
-;; Replace linebreaks within brackets and strings in the INPORT by the
-;; placeholders \LINE_BREAK_N and \LINE_BREAK_R. Also identify
-;; real comments as ;\REALCOMMENTHERE
-;;
+;; TODO: Check whether I can offload the string processing to the
+;; read-function. That’s a source of endless complications. Required:
+;; A kind of unrolling step which appends the string-representation of
+;; the read strings back into the code. I would have to process a list
+;; of strings instead of one big string. Or rather, each line would be
+;; a list of strings.
+
;; bootstrap via python3 wisp.py wisp-guile.w > 1 && guile 1 wisp-guile.w > 2 && guile 2 wisp-guile.w > 3 && diff 2 3
;;
;; -Author: Arne Babenhauserheide
diff --git a/wisp-reader.w b/wisp-reader.w
--- a/wisp-reader.w
+++ b/wisp-reader.w
@@ -51,7 +51,9 @@ define : read-one-wisp-sexp port env
try-pending
define : wisp-scheme-read-chunk-env port env
- cons 'begin : wisp-scheme-read-chunk port
+ if : eof-object? : peek-char port
+ read-char port ; return eof: we’re done
+ cons 'begin : wisp-scheme-read-chunk port
define-language wisp
. #:title "Wisp Scheme Syntax THIS IS EXPERIMENTAL, USE AT YOUR OWN RISK"
diff --git a/wisp-scheme.w b/wisp-scheme.w
--- a/wisp-scheme.w
+++ b/wisp-scheme.w
@@ -25,6 +25,9 @@ define-module : wisp-scheme
wisp-scheme-read-file-chunk wisp-scheme-read-file
wisp-scheme-read-string)
+; use curly-infix by default
+read-enable 'curly-infix
+
use-modules
srfi srfi-1
srfi srfi-11 ; for let-values
@@ -49,13 +52,88 @@ define : line-code line
define readcolon
string->symbol ":"
+define wisp-uuid "e749c73d-c826-47e2-a798-c16c13cb89dd"
; define an intermediate dot replacement with UUID to avoid clashes.
-define dotrepr
- string->symbol "DOTREPR-e749c73d-c826-47e2-a798-c16c13cb89dd"
+define repr-dot ; .
+ string->symbol : string-append "REPR-DOT-" wisp-uuid
+
+; allow using reader additions as the first element on a line to prefix the list
+define repr-quote ; '
+ string->symbol : string-append "REPR-QUOTE-" wisp-uuid
+define repr-unquote ; ,
+ string->symbol : string-append "REPR-UNQUOTE-" wisp-uuid
+define repr-quasiquote ; `
+ string->symbol : string-append "REPR-QUASIQUOTE-" wisp-uuid
+define repr-unquote-splicing ; ,@
+ string->symbol : string-append "REPR-UNQUOTESPLICING-" wisp-uuid
+
+define repr-syntax ; #'
+ string->symbol : string-append "REPR-SYNTAX-" wisp-uuid
+define repr-unsyntax ; #,
+ string->symbol : string-append "REPR-UNSYNTAX-" wisp-uuid
+define repr-quasisyntax ; #`
+ string->symbol : string-append "REPR-QUASISYNTAX-" wisp-uuid
+define repr-unsyntax-splicing ; #,@
+ string->symbol : string-append "REPR-UNSYNTAXSPLICING-" wisp-uuid
+
+; TODO: wrap the reader to return the repr of the syntax reader
+; additions
+
+define : match-charlist-to-repr charlist
+ let
+ : chlist : reverse charlist
+ cond
+ : equal? chlist : list #\.
+ . repr-dot
+ : equal? chlist : list #\'
+ . repr-quote
+ : equal? chlist : list #\,
+ . repr-unquote
+ : equal? chlist : list #\`
+ . repr-quasiquote
+ : equal? chlist : list #\, #\@
+ . repr-unquote-splicing
+ : equal? chlist : list #\# #\'
+ . repr-syntax
+ : equal? chlist : list #\# #\,
+ . repr-unsyntax
+ : equal? chlist : list #\# #\`
+ . repr-quasisyntax
+ : equal? chlist : list #\# #\, #\@
+ . repr-unsyntax-splicing
+ else
+ . #f
+
+define : wisp-read port
+ . "wrap read to catch list prefixes."
+ let : : prefix-maxlen 4
+ let longpeek
+ : peeked '()
+ repr-symbol #f
+ cond
+ : or (< prefix-maxlen (length peeked)) (eof-object? (peek-char port)) (equal? #\space (peek-char port)) (equal? #\newline (peek-char port))
+ if repr-symbol ; found a special symbol, return it.
+ . repr-symbol
+ let unpeek
+ : remaining peeked
+ cond
+ : equal? '() remaining
+ read port ; let read to the work
+ else
+ unread-char (car remaining) port
+ unpeek : cdr remaining
+ else
+ let*
+ : next-char : read-char port
+ peeked : cons next-char peeked
+ longpeek
+ . peeked
+ match-charlist-to-repr peeked
+
define : line-continues? line
- equal? dotrepr : car : line-code line
+ equal? repr-dot : car : line-code line
define : line-only-colon? line
and
@@ -153,10 +231,11 @@ define : wisp-scheme-read-chunk-lines po
; any char but whitespace *after* underscoreindent is
; an error. This is stricter than the current wisp
; syntax definition. TODO: Fix the definition. Better
- ; start too strict.
- : and inunderscoreindent : not : equal? #\space next-char
+ ; start too strict. FIXME: breaks on lines with only
+ ; underscores which should empty lines.
+ : and inunderscoreindent : and (not (equal? #\space next-char)) (not (equal? #\newline next-char))
throw 'wisp-syntax-error "initial underscores without following whitespace at beginning of the line after" : last indent-and-symbols
- : or (equal? #\newline next-char) ; (equal? #\return next-char)
+ : equal? #\newline next-char
read-char port ; remove the newline
; The following two lines would break the REPL by requiring one char too many.
; if : and (equal? #\newline next-char) : equal? #\return : peek-char port
@@ -190,7 +269,7 @@ define : wisp-scheme-read-chunk-lines po
. #t ; inindent
if : <= 2 emptylines
. #f ; chunk ends here
- equal? #\_ : peek-char port
+ equal? #\_ : peek-char port ; are we in underscore indent?
. #f ; incomment
. 0
. '()
@@ -227,32 +306,6 @@ define : wisp-scheme-read-chunk-lines po
. currentindent
. currentsymbols
. emptylines
- : equal? (string-ref "." 0) next-char
- ; TODO: special case for the dot using the dotrepr as
- ; intermediate representation
- read-char port ; remove next-char
- let : : next-next-char : peek-char port
- ; if we don’t need the special handling, add the
- ; next char to the port again
- if : not : or (equal? #\space next-next-char) (equal? #\newline next-next-char) (eof-object? next-next-char) (equal? #\return next-next-char)
- unread-char next-char port
- loop
- . indent-and-symbols
- . #f ; inindent
- . #f ; inunderscoreindent
- . #f ; incomment
- . currentindent
- ; this also takes care of the hashbang and leading comments.
- append currentsymbols
- ; if we don’t need the special handling, just
- ; use the reader. Otherwise append the special
- ; representation of the dot to avoid triggering
- ; this for the dot escaped as |.| or #{.}#
- if : not : or (equal? #\space next-next-char) (equal? #\newline next-next-char) (eof-object? next-next-char) (equal? #\return next-next-char)
- list : read port
- list dotrepr
- . emptylines
- ; TODO: finish
else ; use the reader
loop
. indent-and-symbols
@@ -262,7 +315,7 @@ define : wisp-scheme-read-chunk-lines po
. currentindent
; this also takes care of the hashbang and leading comments.
; TODO: If used from Guile, activate curly infix via read-options.
- append currentsymbols : list : read port
+ append currentsymbols : list : wisp-read port
. emptylines
@@ -453,6 +506,57 @@ define : wisp-scheme-strip-indentation-m
append processed : cdr : car unprocessed
cdr unprocessed
+define : wisp-unescape-underscore-and-colon code
+ . "replace \\_ and \\: by _ and :"
+ match code
+ : a ...
+ map wisp-unescape-underscore-and-colon a
+ '\_
+ . '_
+ '\:
+ . ':
+ a
+ . a
+
+
+define : wisp-replace-empty-eof code
+ . "replace ((#<eof>)) by ()"
+ ; FIXME: Actually this is a hack which fixes a bug when the
+ ; parser hits files with only hashbang and comments.
+ if : and (pair? (car code)) (eof-object? (car (car code))) (null? (cdr code)) (null? (cdr (car code)))
+ list
+ . code
+
+
+define : wisp-replace-paren-quotation-repr code
+ . "Replace lists starting with a quotation symbol by
+ quoted lists."
+ match code
+ : 'REPR-QUOTE-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
+ list 'quote : map wisp-replace-paren-quotation-repr a
+ : a ... 'REPR-QUOTE-e749c73d-c826-47e2-a798-c16c13cb89dd b ; this is the quoted empty list
+ append
+ map wisp-replace-paren-quotation-repr a
+ list : list 'quote : map wisp-replace-paren-quotation-repr b
+ : 'REPR-UNQUOTE-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
+ list 'unquote : map wisp-replace-paren-quotation-repr a
+ : 'REPR-QUASIQUOTE-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
+ list 'quasiquote : map wisp-replace-paren-quotation-repr a
+ : 'REPR-UNQUOTESPLICING-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
+ list 'unquote-splicing : map wisp-replace-paren-quotation-repr a
+ : 'REPR-SYNTAX-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
+ list 'syntax : map wisp-replace-paren-quotation-repr a
+ : 'REPR-UNSYNTAX-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
+ list 'unsyntax : map wisp-replace-paren-quotation-repr a
+ : 'REPR-QUASISYNTAX-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
+ list 'quasisyntax : map wisp-replace-paren-quotation-repr a
+ : 'REPR-UNSYNTAXSPLICING-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
+ list 'unsyntax-splicing : map wisp-replace-paren-quotation-repr a
+ : a ...
+ map wisp-replace-paren-quotation-repr a
+ a
+ . a
+
define : wisp-make-improper code
. "Turn (a #{.}# b) into the correct (a . b).
@@ -467,7 +571,7 @@ Match is awesome!"
:
improper
match code
- : a ... b 'DOTREPR-e749c73d-c826-47e2-a798-c16c13cb89dd c
+ : a ... b 'REPR-DOT-e749c73d-c826-47e2-a798-c16c13cb89dd c
append (map wisp-make-improper a)
cons (wisp-make-improper b) (wisp-make-improper c)
: a ...
@@ -480,18 +584,18 @@ Match is awesome!"
: tocheck improper
match tocheck
; lists with only one member
- : 'DOTREPR-e749c73d-c826-47e2-a798-c16c13cb89dd
+ : 'REPR-DOT-e749c73d-c826-47e2-a798-c16c13cb89dd
syntax-error tocheck
; list with remaining dot.
: a ...
- if : member dotrepr a
+ if : member repr-dot a
syntax-error tocheck
map check a
; simple pair
- : 'DOTREPR-e749c73d-c826-47e2-a798-c16c13cb89dd . c
+ : 'REPR-DOT-e749c73d-c826-47e2-a798-c16c13cb89dd . c
syntax-error tocheck
; simple pair, other way round
- : a . 'DOTREPR-e749c73d-c826-47e2-a798-c16c13cb89dd
+ : a . 'REPR-DOT-e749c73d-c826-47e2-a798-c16c13cb89dd
syntax-error tocheck
; more complex pairs
: ? pair? a
@@ -499,11 +603,11 @@ Match is awesome!"
: head : drop-right a 1
tail : last-pair a
cond
- : equal? dotrepr : car tail
+ : equal? repr-dot : car tail
syntax-error tocheck
- : equal? dotrepr : cdr tail
+ : equal? repr-dot : cdr tail
syntax-error tocheck
- : member dotrepr head
+ : member repr-dot head
syntax-error tocheck
else
. a
@@ -515,7 +619,10 @@ define : wisp-scheme-read-chunk port
. "Read and parse one chunk of wisp-code"
let : : lines : wisp-scheme-read-chunk-lines port
wisp-make-improper
- wisp-scheme-indentation-to-parens lines
+ wisp-replace-empty-eof
+ wisp-unescape-underscore-and-colon
+ wisp-replace-paren-quotation-repr
+ wisp-scheme-indentation-to-parens lines
define : wisp-scheme-read-all port
. "Read all chunks from the given port"
@@ -541,6 +648,36 @@ define : wisp-scheme-read-string-chunk s
call-with-input-string str wisp-scheme-read-chunk
+;;;; Test special syntax
+; ;; quote the list
+; write
+; wisp-scheme-read-string "moo
+; foo
+; ' bar
+; baz waz"
+; newline
+; ;; quote the symbol - in wisp, whitespace after quote is not allowed!
+; write
+; wisp-scheme-read-string "moo
+; foo
+; 'bar
+; baz waz"
+; newline
+; ; ;; quote the list with colon
+; write
+; wisp-scheme-read-string "moo : ' foo
+; foo
+; ' bar bah
+; baz waz"
+; newline
+; ; ;; syntax the list
+; write
+; wisp-scheme-read-string "moo : #' foo
+; foo
+; #' bar bah
+; baz waz"
+; newline
+;
;;;; Test improper lists
;;;; Good cases
; write