wisp
 
(Arne Babenhauserheide)
2015-04-11: merge source-properties and d20world experiment. stable v0.8.3

merge source-properties and d20world experiment.

diff --git a/.bugs/bugs b/.bugs/bugs
--- a/.bugs/bugs
+++ b/.bugs/bugs
@@ -1,4 +1,5 @@
 testsuite: to pass, the tree-il has to match, not the emitted string. This allows for reader-only implementations. | owner:, open:False, id:00b74a730bbf076e73166e817ca7b0a273b376d4, time:1408224636.42
+wisp-scheme: backtraces should show the wisp source.         | owner:Arne Babenhauserheide <bab@draketo.de>, open:True, id:0475df81a594a52d171a1b811752ca64e5a71df5, time:1426792099.58
 wisp-scheme: unescape \_ and \:                              | owner:, open:False, id:086f61a06e16f1ef56e9917453bbd55b5879d15d, time:1415121255.99
 fails when I add stuff at the end of end of example.w        | owner:, open:False, id:08c68e1ce0c9798184c01806d2661a3220bff3cd, time:1363789693.79
 wisp-mode in quoted lists only the first item is colorized as data, but all words up to the last paren should be colorized. | owner:, open:True, id:1675ca3f894ed8470fa292149a476a2fa0d17140, time:1397196957.45
@@ -6,10 +7,12 @@ add a testsuite for wisp parsers.       
 wisp-mode: export wisp to html fails in org-mode: font-lock-fontify-keywords-region: Invalid regexp | owner:, open:False, id:1e46d8c05580c961c37a32d36c987a5dd1d34943, time:1389371020.39
 an empty line with : should start with double parens         | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:2e188ddf44d36e4605030d3c58607ebfa97d189e, time:1390328674.43
 wisp-in-wisp: remove the leading . for continuation lines after inferring the brackets. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:2e42e5b64622f0cc383eb8acc3d510912e925bf0, time:1377476687.79
+interpret ` , : correctly.                                   | owner:, open:True, id:2feb5f048b55274c1bc7c8168c8cb358c0c8dd1d, time:1426777900.6
 '() gives REPR-QUOTE-...                                     | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:30c42de75c137f483245898e2a62af1e65cf19a6, time:1415060388.34
 multiple escaped initial underscores must be unescaped.      | owner:, open:True, id:314e45488da4c7c8298c4c64ece03359918d057b, time:1415959749.14
 wisp: handle general paren prefixes like quote, quasiquote, etc. | owner:, open:False, id:323ff94b5be635742619467e1cb44f4c0d96f63f, time:1379047798.47
 throw an exception when reducing indentation to an unknown indentation level. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:424186bd85f186b7279c5c59e2bd42f847284719, time:1376003568.91
+LANG=C breaks bootstrap: python encoding error: it uses ascii. | owner:, open:True, id:43c7461bfb6f35a90ff3f4497c8232e2457ce1c7, time:1427819877.7
 wisp-in-wisp: only parses the first 272 lines, for some reason. | owner:, open:False, id:4cb6c556d7136609e2da9ab3fc045a39847f1ef3, time:1377014682.98
 adjust runtests.sh to use testrunner.w                       | owner:, open:False, id:4d4e76343fe09f0ec72a3e5eb0077bd16e12f9d5, time:1415127234.43
 wisp-scheme: REPL: sometimes the output of a command is only shown after typing the next non-empty line. | owner:, open:False, id:56d2f81e9c89accb0b0bc668ddc8feed3b60e9b2, time:1416584789.23
@@ -20,7 +23,7 @@ wisp-guile.w does not yet remove the lea
 inline ' : should be '(                                      | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:72d534a8b23b4cb168017f1bb7d8816f0ea170c4, time:1366497335.26
 failing test tests/shebang.w                                 | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:74a851f83af8996465a7b24d8453161beb0f0fd5, time:1379106761.57
 non-nested multiline comments with #! !#. Requires restructuring. | owner:, open:False, id:7a57614fa920b2ddad002d044b144d0bb7c34f84, time:1389364108.01
-wisp-scheme: interpret , : as ,() similar to : ,             | owner:Arne Babenhauserheide <bab@draketo.de>, open:True, id:85e150dcb10c49d8f51db525e07d24e83bdba0f1, time:1416432201.21
+wisp-scheme: interpret , : as ,() similar to : ,             | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:85e150dcb10c49d8f51db525e07d24e83bdba0f1, time:1416432201.21
 wisp-guile: support nested multi-line comments with #| ... |#: multiline comments (srfi-30). Requires restructuring. | owner:Arne Babenhauserheide <bab@draketo.de>, open:True, id:8cf6202873d4454f57813dd17cf60432059f7c62, time:1389569421.6
 wisp-scheme: Does not recognize the . #!curly-infix request for curly-infix or other reader syntax. | owner:Arne Babenhauserheide <bab@draketo.de>, open:True, id:91f27adb7d4e378e034b3408b6e4616f707f9587, time:1418162368.88
 wisp-guile: the repl does not require 3 returns when you use a single char as function, or rather only does so every second time | owner:, open:True, id:9cedd0bdbf4a3b17add4bfe86ad5a23e500cfc6c, time:1379064870.78
@@ -36,4 +39,5 @@ comments containing a closing parenthesi
 breaks on empty files                                        | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:e40fa7a93eb2c497dca1af7eed22ad5ed5cfbe7f, time:1390325470.91
 wisp-scheme: breaks on lines with only underscores. These should be treated as empty lines. | owner:, open:False, id:e464b5ce49deb14a80f67d50c6d70043ca9bde25, time:1415124488.16
 quote as only char in a line gets parenthized instead of becoming a paren-prefix. | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:eb7468387e90bb5d13f5a5d81c6f4a704f2ca0fb, time:1390326369.6
-in the REPL output can be delayed by one line: Appears only when submitting the next command. | owner:, open:True, id:f1e42bbd4c17a2dec886c26d9c14e770bcff66d2, time:1415972414.48
+in the REPL output can be delayed by one line: Appears only when submitting the next command. | owner:, open:False, id:f1e42bbd4c17a2dec886c26d9c14e770bcff66d2, time:1415972414.48
+wisp-scheme: parser problem with dotted pair: use-modules : (ice-9 popen) #:select ((open-input-pipe . oip)) | owner:Arne Babenhauserheide <bab@draketo.de>, open:False, id:ff078cba853c1a2fdbd41cf0228ad7920a642c0d, time:1428358435.75
diff --git a/Changelog b/Changelog
deleted file mode 100644
--- a/Changelog
+++ /dev/null
@@ -1,32 +0,0 @@
-0.3.1
-	* wisp.py: parentheses in comments no longer break the parser
-	* wisp.py: inline " ' : " is turned into " '("
-	* multithreaded-magic.w: New example: Easy multithreading.
-	* hello-world-server.w: Show local time instead of UTC and be a
-	bit more friendly.
-
-0.3
-	* wisp-multiline.sh: started with emacs support. Not yet nice.
-	* wisp.w: renamed to wisp-guile.w to show that it uses guile
-	scheme.
-	* wisp.w: started wisp in wisp. Does not work, yet.
-	* hello-world-server.w: First actually running example code.
-	* wisp-multiline.sh: directly execute the typed script in guile
-	scheme. Robust shell-script commandline parsing.
-	* Readme.txt: Note the license and add references and footnotes.
-	* Readme.txt: Fix the examples and add a stdin-example with guile.
-
-0.2
-	* wisp.py: got more resilient.
-	* wisp.py: condense multiple inline : into multiple brackets
-	without whitespace.
-	* wisp.py: refactored into multiple distinct phases for easier
-	maintainability.
-	* wisp.py: allow escaping : and _ with \.
-	* wisp.py: added websafe indent with _
-	* wisp.py: a colon at the end of the line is interpreted as ()
-	* wisp.py: don’t interpret wisp code in brackets or strings.
-
-0.1
-	* wisp.py: first version.
-	* Readme.txt: Added a readme.
diff --git a/NEWS b/NEWS
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,7 @@
+wisp 0.8.3 (2015-04-10):
+- add partial source-properties support: show line numbers in backtraces.
+- d20world: start of experiement to plot the results.
+
 wisp 0.8.2 (2015-03-18):
 - resolve REPL delayed output bug.
 - forbid dot at the end of a line and as only symbol in a line to reserve them for future use.
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1,7 +1,7 @@
 dnl run `autoreconf -i` to generate a configure script. 
 dnl Then run ./configure to generate a Makefile.
 dnl Finally run make to generate the project.
-AC_INIT([wisp], [0.8.2],
+AC_INIT([wisp], [0.8.3],
         [arne_bab@web.de])
 # Check for programs I need for my build
 AC_CANONICAL_TARGET
diff --git a/examples/d20world.w b/examples/d20world.w
--- a/examples/d20world.w
+++ b/examples/d20world.w
@@ -17,6 +17,9 @@ define-module : examples d20world
               . #:export : world neighbors d20-as-text d20-diffuse
 
 use-modules : ice-9 format
+use-modules 
+  : ice-9 popen
+        . #:select : open-output-pipe close-pipe
 
 define world : make-vector 20 0
 define neighbors : make-vector 20
@@ -211,6 +214,7 @@ define : latlon2cellidx lat lon
 
 display : d20-as-text world
 newline
+
 format #t "Diffuse ~A\n" 0.01
 d20-diffuse world neighbors 0.01
 display : d20-as-text world
@@ -282,3 +286,52 @@ let loop : : steps 1000
         loop : 1- steps
 display : d20-as-text world
 newline
+
+; now plot the result
+let : : port : open-output-pipe "python"
+  format port "from mpl_toolkits.mplot3d import Axes3D, art3d
+import numpy as np
+import scipy as sp
+from matplotlib import cm
+import matplotlib.pyplot as plt
+from scipy.spatial import Delaunay
+
+def Icosahedron():
+    h = 0.5*(1+np.sqrt(5))
+    p1 = np.array([[0,1,h],[0,1,-h],[0,-1,h],[0,-1,-h]])
+    p2 = p1[:,[1,2,0]]
+    p3 = p1[:,[2,0,1]]
+    return np.vstack((p1,p2,p3))
+
+Ico = Icosahedron()
+tri = Delaunay(Ico)
+CH = tri.convex_hull
+points = tri.points
+
+fig = plt.figure(figsize=(4.0,4.0))
+ax = fig.add_subplot(111, projection='3d')
+
+print points
+for i in range(points.shape[0]):
+    neighbors = tri.neighbors[i,:]
+    for n in range(points.shape[0]):
+        pts = []
+        for u in range(points.shape[0]):
+            pt = np.zeros((3,3))
+            pt[0,:] = points[(i),:]
+            pt[1,:] = points[(n),:]
+            pt[2,:] = points[(u),:]
+            # print pt
+            pt *= 0.5
+            pt += 0.5
+            pts.append(pt)
+        tr = art3d.Poly3DCollection(pts)
+        tr.set_color([(0.9*i)/points.shape[0]] + [(0.9*n)/points.shape[0]]*3)
+        ax.add_collection3d(tr)
+# ax.plot_surface(x, y, z, color='g')
+
+plt.show()
+
+exit()\n"
+  close-pipe port
+
diff --git a/examples/d6.w b/examples/d6.w
--- a/examples/d6.w
+++ b/examples/d6.w
@@ -2,7 +2,7 @@
 ; !#
 
 define-module : examples d6
-   . #:export : roll check 
+   . #:export : roll check
 
 ; basic d6 rules, implemented in guile
 
@@ -23,3 +23,5 @@ define : check skill target effect-thres
 display : check 12 9 3
 newline
 display : roll
+
+
diff --git a/examples/multithreaded-magic.w b/examples/multithreaded-magic.w
--- a/examples/multithreaded-magic.w
+++ b/examples/multithreaded-magic.w
@@ -1,6 +1,9 @@
-#!./wisp-multiline.sh
+#!/usr/bin/env sh
+exec guile -L $(dirname $(dirname $(realpath "$0"))) --language=wisp -e '(@@ (examples multithreaded-magic) main)' -s "$0" "$@"
 ; !#
 
+define-module : examples multithreaded-magic
+
 ; Mathematical magic: Always get one.
 ; 
 ; Via http://www.liv.ac.uk/HPC/HTMLF90Course/HTMLF90CourseQuestionsnode18.html
@@ -8,6 +11,8 @@
 ; 
 ; this is the wisp scheme version which I want to compare with the fortran version.
 
+; Call as PATH=~/guile/meta:$PATH ./examples/multithreaded-magic.w
+
 use-modules 
   ice-9 format
   ice-9 futures
@@ -15,12 +20,17 @@ use-modules
 
 define : magic-threaded mutex futures integer
   ; this can cause unordered output. It’s fun anyway : 
-  set! futures : append futures : list : future : with-mutex mutex : format #t "~30r\n" integer
-  if : not : = integer 1
-    if : even? integer
-      magic-threaded mutex futures : / integer 2
-      magic-threaded mutex futures : truncate : + 1 : / integer 3 
-    for-each touch futures
+  let
+    :
+      futures
+        cons : future : with-mutex mutex : format #t "~30r\n" integer
+          . futures
+    if : not : = integer 1
+      if : even? integer
+        magic-threaded mutex futures {integer / 2}
+        magic-threaded mutex futures
+              truncate : + 1 {integer / 3}
+      for-each touch futures
 
 define : magic integer
   magic-threaded
@@ -35,7 +45,8 @@ define : magic-simple integer
       magic-simple : / integer 2
       magic-simple : truncate : + 1 : / integer 3 
 
-display ";;; multithreaded magic ;;;\n"
-magic 456189456156456196152615
-display ";;; simple magic ;;;\n"
-magic-simple 456189456156456196152615
+define : main args
+         display ";;; multithreaded magic ;;;\n"
+         magic 456189456156456196152615
+         display ";;; simple magic ;;;\n"
+         magic-simple 456189456156456196152615
diff --git a/tests/dotted-pair.scm b/tests/dotted-pair.scm
new file mode 100644
--- /dev/null
+++ b/tests/dotted-pair.scm
@@ -0,0 +1,3 @@
+(use-modules ((ice-9 popen) #:select ((open-input-pipe . oip))))
+
+
diff --git a/tests/dotted-pair.w b/tests/dotted-pair.w
new file mode 100644
--- /dev/null
+++ b/tests/dotted-pair.w
@@ -0,0 +1,1 @@
+use-modules : (ice-9 popen) #:select ((open-input-pipe . oip))
diff --git a/tests/quotecolon.scm b/tests/quotecolon.scm
--- a/tests/quotecolon.scm
+++ b/tests/quotecolon.scm
@@ -9,4 +9,4 @@
   (a b)
   (c))
 
-
+(define a (quasiquote ,(+ 2 2)))
diff --git a/tests/quotecolon.w b/tests/quotecolon.w
--- a/tests/quotecolon.w
+++ b/tests/quotecolon.w
@@ -8,3 +8,5 @@ define a ' : 1 2 3
 define
   a b
   c
+
+define a : quasiquote , : + 2 2
diff --git a/wisp-reader.w b/wisp-reader.w
--- a/wisp-reader.w
+++ b/wisp-reader.w
@@ -70,11 +70,13 @@ define : read-one-wisp-sexp port env
               pending wisp-pending-sexps
             set! wisp-pending-sexps : list ; : cdr wisp-pending-sexps
             ; write : cons 'begin pending
-            cons 'begin pending
+            if : = 1 : length pending
+                 car pending
+                 cons 'begin pending
   try-pending
 
 define-language wisp
-  . #:title "Wisp Scheme Syntax THIS IS EXPERIMENTAL, USE AT YOUR OWN RISK"
+  . #:title "Wisp Scheme Syntax. See SRFI-119 for details. THIS IS EXPERIMENTAL, USE AT YOUR OWN RISK"
   ; . #:reader read-one-wisp-sexp
   . #:reader : lambda (port env) : let ((x (read-one-wisp-sexp port env))) x
   . #:compilers `((scheme . ,compile-scheme))
diff --git a/wisp-scheme.w b/wisp-scheme.w
--- a/wisp-scheme.w
+++ b/wisp-scheme.w
@@ -43,7 +43,11 @@ define : line-real-indent line
                . indent
 
 define : line-code line
-         cdr line
+         let : : code : cdr line
+             ; propagate source properties
+             when : not : null? code
+                    set-source-properties! code : source-properties line
+             . code
 
 ; literal values I need
 define readcolon 
@@ -110,6 +114,16 @@ define : wisp-read port
            cond
              : or (< prefix-maxlen (length peeked)) (eof-object? (peek-char port)) (equal? #\space (peek-char port)) (equal? #\newline (peek-char port)) 
                if repr-symbol ; found a special symbol, return it.
+                  ; TODO: Somehow store source-properties. The commented-out code below does not work.
+                  ; catch #t
+                  ;     lambda ()
+                  ;         write : source-properties symbol-or-symbols
+                  ;         set-source-property! symbol-or-symbols 'filename : port-filename port
+                  ;         set-source-property! symbol-or-symbols 'line : 1+ : port-line port
+                  ;         set-source-property! symbol-or-symbols 'column : port-column port
+                  ;         write : source-properties symbol-or-symbols
+                  ;     lambda : key . arguments
+                  ;         . #f
                   . repr-symbol
                   let unpeek
                     : remaining peeked
@@ -258,6 +272,10 @@ define : wisp-scheme-read-chunk-lines po
                          if : not : line-empty? parsedline
                             . 0 
                             1+ emptylines
+                     when : not : = 0 : length parsedline
+                         ; set the source properties to parsedline so we can try to add them later.
+                         set-source-property! parsedline 'filename : port-filename port
+                         set-source-property! parsedline 'line : port-line port
                      ; TODO: If the line is empty. Either do it here and do not add it, just
                      ; increment the empty line counter, or strip it later. Replace indent
                      ; -1 by indent 0 afterwards.
@@ -353,12 +371,53 @@ define : line-strip-lone-colon line
            . line
 
 define : line-finalize line
-         . "Process all wisp-specific information in a line and strip it"
-         line-code-replace-inline-colons 
-           line-strip-indentation-marker
-             line-strip-lone-colon
-               line-strip-continuation line
+       . "Process all wisp-specific information in a line and strip it"
+       let
+         :
+           l
+             line-code-replace-inline-colons 
+               line-strip-indentation-marker
+                 line-strip-lone-colon
+                   line-strip-continuation line
+         when : not : null? : source-properties line
+                catch #t
+                  lambda ()
+                    set-source-properties! l : source-properties line
+                  lambda : key . arguments
+                    . #f
+         . l
 
+define : wisp-add-source-properties-from source target
+       . "Copy the source properties from source into the target and return the target."
+       catch #t
+           lambda ()
+               set-source-properties! target : source-properties source
+           lambda : key . arguments
+               . #f
+       . target
+
+define : wisp-propagate-source-properties code
+       . "Propagate the source properties from the sourrounding list into every part of the code."
+       let loop
+         : processed '()
+           unprocessed code
+         cond
+           : and (null? processed) (not (pair? unprocessed)) (not (list? unprocessed))
+             . unprocessed
+           : and (pair? unprocessed) (not (list? unprocessed))
+             cons
+               wisp-propagate-source-properties (car unprocessed)
+               wisp-propagate-source-properties (cdr unprocessed)
+           : null? unprocessed
+             . processed
+           else
+             let : : line : car unprocessed
+               if : null? : source-properties unprocessed
+                   wisp-add-source-properties-from line unprocessed
+                   wisp-add-source-properties-from unprocessed line
+               loop
+                 append processed : list : wisp-propagate-source-properties line
+                 cdr unprocessed
 
 define : wisp-scheme-indentation-to-parens lines
          . "Add parentheses to lines and remove the indentation markers"
@@ -447,7 +506,7 @@ define : wisp-scheme-indentation-to-pare
                                append processed 
                                  if : line-continues? current-line
                                       . line
-                                      list line
+                                      wisp-add-source-properties-from line : list line
                                cdr unprocessed ; recursion here
                                . indentation-levels
                            : < current-line-indentation next-line-indentation
@@ -537,8 +596,16 @@ define : wisp-replace-paren-quotation-re
                         list : list 'quote : map wisp-replace-paren-quotation-repr b
              : 'REPR-UNQUOTE-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
                 list 'unquote : map wisp-replace-paren-quotation-repr a
+             : a ... 'REPR-UNQUOTE-e749c73d-c826-47e2-a798-c16c13cb89dd b 
+                append
+                        map wisp-replace-paren-quotation-repr a
+                        list : list 'unquote : map wisp-replace-paren-quotation-repr b
              : 'REPR-QUASIQUOTE-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
                 list 'quasiquote : map wisp-replace-paren-quotation-repr a
+             : a ... 'REPR-QUASIQUOTE-e749c73d-c826-47e2-a798-c16c13cb89dd b ; this is the quoted empty list 
+                append
+                        map wisp-replace-paren-quotation-repr a
+                        list : list 'quasiquote : map wisp-replace-paren-quotation-repr b
              : 'REPR-UNQUOTESPLICING-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
                 list 'unquote-splicing : map wisp-replace-paren-quotation-repr a
              : 'REPR-SYNTAX-e749c73d-c826-47e2-a798-c16c13cb89dd a ...
@@ -613,7 +680,6 @@ Match is awesome!"
                a
                  . a
 
-
 define : wisp-scheme-read-chunk port
          . "Read and parse one chunk of wisp-code"
          let : :  lines : wisp-scheme-read-chunk-lines port
@@ -621,7 +687,8 @@ define : wisp-scheme-read-chunk port
                 wisp-replace-empty-eof
                   wisp-unescape-underscore-and-colon
                     wisp-replace-paren-quotation-repr
-                      wisp-scheme-indentation-to-parens lines
+                      wisp-propagate-source-properties
+                        wisp-scheme-indentation-to-parens lines
 
 define : wisp-scheme-read-all port
          . "Read all chunks from the given port"