Unread-char behavior deviates from spec?
On the general Lisp HyperSpec page for unread-char
- see
here - it says all of the following things:
-
"The unread char is intended to be an efficient mechanism for allowing Lisp readers and other parsers to execute single-character representations on an input stream."
-
"An error occurred concurrently calling unread-char on the same stream without an intermediate call to read-char (or some other input operation that implicitly reads characters) on this stream."
I am looking into how to add multi-character support for CL streams for a parser that I plan to write, and just to confirm the above, I ran the following code:
(defun unread-char-test (data)
(with-input-from-string (stream data)
(let ((stack nil))
(loop
for c = (read-char stream nil)
while c
do (push c stack))
(loop
for c = (pop stack)
while c
do (unread-char c stream)))
(coerce
(loop
for c = (read-char stream nil)
while c
collect c)
'string)))
(unread-char-test "hello")
==> "hello"
It doesn't throw an error (on SBCL or CCL, I haven't tested it on other implementations yet), but I don't see how there could be any read operations (implicit or explicit) happening in the thread between successive calls before unread-char
.
This behavior is good news for multi-valued lookups if it's consistent, but why isn't it a bug?
source to share
In response to user jkiiski's comment, I did some more digging. I defined a function similar to the one above, but which takes a stream as an argument (for easier reuse):
(defun unread-char-test (stream)
(let ((stack nil))
(loop
for c = (read-char stream nil)
while c
do (push c stack))
(loop
for c = (pop stack)
while c
do (unread-char c stream)))
(coerce
(loop
for c = (read-char stream nil)
while c
collect c)
'string))
Then I ran the following in the second REPL:
(defun create-server (port)
(usocket:with-socket-listener (listener "127.0.0.1" port)
(usocket:with-server-socket (connection (usocket:socket-accept listener))
(let ((stream (usocket:socket-stream connection)))
(print "hello" stream)))))
(create-server 4000)
And in the first REPL the following:
(defun create-client (port)
(usocket:with-client-socket (connection stream "127.0.0.1" port)
(unread-char-test stream)))
(create-client 4000)
And he made the mistake I was expecting:
Two UNREAD-CHARs without intervening READ-CHAR on #<BASIC-TCP-STREAM ISO-8859-1 (SOCKET/4) #x302001813E2D>
[Condition of type SIMPLE-ERROR]
This suggests that jkiiski's assumption is correct. The original behavior was also observed when reading input from a text file:
(with-open-file (stream "test.txt" :direction :output)
(princ "hello" stream))
(with-open-file (stream "test.txt")
(unread-char-test stream)))
==> "hello"
My guess is that when dealing with local file I / O, the implementation reads large chunks of the file into memory and then read-char
reads from the buffer. If correct, this also confirms the assumption that the error described in the specification is not generated by typical implementations when unreading from a stream whose content is in memory.
source to share