Common Lisp reading from file and saving as a list

So, do some general lisp exercises and everything went well until I ran into this strange behavior. I have read text from file (brown.txt) in corpus variable and should be saved as a list. However, I suspect it is not, although sometimes it works as one, but other times it fails.

Here is a basic read from file -> append for list -> list of stores in corpus (split / tokenized in space):

(defun tokenize (string)
  (loop
     for start = 0 then (+ space 1)
     for space = (position #\space string :start start)
     for token = (subseq string start space)
     unless (string= token "") collect token
     until (not space)))

(defparameter *corpus*
   (with-open-file (stream "./brown.txt" :direction :input)
     (loop for line = (read-line stream nil)
           while line
           append (tokenize line))))

      

And below are two expressions that should work, but only the last one (body one). The first returns NIL.

(loop for token in *corpus* do
     (print token))
*corpus*

      

I suspect it has something to do with reading from a file as a stream object and that (append ...) does not create a list from that stream, but instead lazy waits until I want to later or sum it up, at which time it just decided not to work anymore? (makes little sense to me).

+3


source to share


1 answer


This expression:

(loop for token in *corpus* do
     (print token))

      



returns NIL

because it does not contain a clause RETURN

or accumulation condition (for example, COLLECT

or APPEND

). It just calls PRINT

multiple times, but discards the return value.

+8


source







All Articles