Why is Java DirectoryStream so slow?

I've done some testing with Streams, specifically DirectoryStreams from the nio package. I am just trying to get a list of all files in a directory, sorted by last modified dates and sizes.

The JavaDoc of the old File.listFiles () pointed out a note on the method in file s :

Note that the Files class defines a newDirectoryStream method to open a directory and iterate over the filenames in the directory. This can use fewer resources when working with very large directories.

I run the code below many times (first three times below):

First perspective:

Run time of Arrays.sort: 1516
Run time of Stream.sorted as Array: 2912
Run time of Stream.sorted as List: 2875

      

        
        
        
      

    

Second perspective:

Run time of Arrays.sort: 1557
Run time of Stream.sorted as Array: 2978
Run time of Stream.sorted as List: 2937

      

        
        
        
      

    

Third perspective:

Run time of Arrays.sort: 1563
Run time of Stream.sorted as Array: 2919
Run time of Stream.sorted as List: 2896

      

        
        
        
      

    

My question is: Why are streams performing so badly?

import java.io.File;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.FileTime;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class FileSorter {

  // This sorts from old to young and from big to small
  Comparator<Path> timeSizeComparator = (Path o1, Path o2) -> {
    int sorter = 0;
    try {
      FileTime lm1 = Files.getLastModifiedTime(o1);
      FileTime lm2 = Files.getLastModifiedTime(o2);
      if (lm2.compareTo(lm1) == 0) {
        Long s1 = Files.size(o1);
        Long s2 = Files.size(o2);
        sorter = s2.compareTo(s1);
      } else {
        sorter = lm1.compareTo(lm2);
      }
    } catch (IOException ex) {
      throw new UncheckedIOException(ex);
    }
    return sorter;
  };

  public String[] getSortedFileListAsArray(Path dir) throws IOException {
    Stream<Path> stream = Files.list(dir);
    return stream.sorted(timeSizeComparator).
            map(Path::getFileName).map(Path::toString).toArray(String[]::new);
  }

  public List<String> getSortedFileListAsList(Path dir) throws IOException {
    Stream<Path> stream = Files.list(dir);
    return stream.sorted(timeSizeComparator).
            map(Path::getFileName).map(Path::toString).collect(Collectors.
            toList());
  }

  public String[] sortByDateAndSize(File[] fileList) {
    Arrays.sort(fileList, (File o1, File o2) -> {
      int r = Long.compare(o1.lastModified(), o2.lastModified());
      if (r != 0) {
        return r;
      }
      return Long.compare(o1.length(), o2.length());
    });
    String[] fileNames = new String[fileList.length];
    for (int i = 0; i < fileNames.length; i++) {
      fileNames[i] = fileList[i].getName();
    }
    return fileNames;
  }

  public static void main(String[] args) throws IOException {
    // File (io package)
    File f = new File("C:\\Windows\\system32");
    // Path (nio package)
    Path dir = Paths.get("C:\\Windows\\system32");

    FileSorter fs = new FileSorter();

    long before = System.currentTimeMillis();
    String[] names = fs.sortByDateAndSize(f.listFiles());
    long after = System.currentTimeMillis();
    System.out.println("Run time of Arrays.sort: " + ((after - before)));

    long before2 = System.currentTimeMillis();
    String[] names2 = fs.getSortedFileListAsArray(dir);
    long after2 = System.currentTimeMillis();
    System.out.
            println("Run time of Stream.sorted as Array: " + ((after2 - before2)));

    long before3 = System.currentTimeMillis();
    List<String> names3 = fs.getSortedFileListAsList(dir);
    long after3 = System.currentTimeMillis();
    System.out.
            println("Run time of Stream.sorted as List: " + ((after3 - before3)));
  }
}

      

        
        
        
      

    

Update

After applying Peter's code, I get the following results:

Run time of Arrays.sort: 1615
Run time of Stream.sorted as Array: 3116
Run time of Stream.sorted as List: 3059
Run time of Stream.sorted as List with caching: 378

      

        
        
        
      

    

Update 2

After some research on Peter's solution, I can tell that reading file attributes for ex. Files.getLastModified should be crunchy heavy. Changing only this part in the comparator:

  Comparator<Path> timeSizeComparator = (Path o1, Path o2) -> {
    File f1 = o1.toFile();
    File f2 = o2.toFile();
    long lm1 = f1.lastModified();
    long lm2 = f2.lastModified();
    int cmp = Long.compare(lm2, lm1);
    if (cmp == 0) {
      cmp = Long.compare(f2.length(), f1.length());
    }
    return cmp;
  };

      

        
        
        
      

    

Gets an even better result on my computer:

Run time of Arrays.sort: 1968
Run time of Stream.sorted as Array: 1999
Run time of Stream.sorted as List: 1975
Run time of Stream.sorted as List with caching: 488

      

        
        
        
      

    

But as you can see, object caching is the best way. And as mentioned by jtahlborn, it is kind of a stable species.

Update 3 (the best solution I've found)

After a bit more research, I've seen that the Files.lastModified and Files.size methods do a great job of doing the same thing: Attributes. So I checked three versions of the PathInfo class:

• Peters version described below
• Old version of the version file where I do Path.toFile () once in the constructor and get all values ​​from that file with f.lastModified and f.length
• Peters version of solution, but now I read Attribute object with Files.readAttributes (path, BasicFileAttributes.class) and did everything on that.

Putting it all in a loop to execute 100 times each, I came up with these results:

After doing all hundred times
Mean performance of Peters solution: 432.26
Mean performance of old File solution: 343.11
Mean performance of read attribute object once solution: 255.66

      

        
        
        
      

    

The code in the PathInfo constructor for the solution is better :

public PathInfo(Path path) {
  try {
    // read the whole attributes once
    BasicFileAttributes bfa = Files.readAttributes(path, BasicFileAttributes.class);
    fileName = path.getFileName().toString();
    modified = bfa.lastModifiedTime().toMillis();
    size = bfa.size();
  } catch (IOException ex) {
    throw new UncheckedIOException(ex);
  }
}

      

        
        
        
      

    

My result: Never read attributes twice and object caching has discontinuous performance.