List first N files in SFTP using JSch
I need to get a recursive list of filenames in an SFTP folder. Now I am using the following code:
private void getFilesRecursive(List<LsEntry> fileList, List<FileDto> response, String dirParentName,
SftpManagerWithPool manager) throws SftpException {
for (LsEntry file : fileList) {
if (!file.getAttrs().isDir()) {
response.add(new FileDto(file.getFilename(), StorageType.FOLDER.getName(),
new Attributes(file.getAttrs().getATime(), file.getAttrs().getMTime(),
file.getAttrs().getAtimeString(), file.getAttrs().getMtimeString(),
file.getAttrs().getPermissionsString(), getPathWitoutRootDirectoryt(dirParentName),
file.getAttrs().getSize(), file.getAttrs().isDir())));
} else if (file.getAttrs().isDir() && !".".equals(file.getFilename())) {
List<LsEntry> files = manager.listFiles(context.getBasePath().concat("/")
.concat(dirParentName.concat("/").concat(file.getFilename())));
getFilesRecursive(files, response, dirParentName.concat("/").concat(file.getFilename()), manager);
}
}
}
public List<FileDto> getFiles() throws ServiceException {
Session session = null;
SftpManagerWithPool manager = null;
try {
session = getSession();
manager = new SftpManagerWithPool(session);
manager.connect();
List<FileDto> response = new ArrayList<>();
List<LsEntry> files = manager
.listFiles(context.getBasePath().concat("/").concat(context.getPathToProcess()));
getFilesRecursive(files, response, context.getPathToProcess(), manager);
return response;
} catch (Exception e) {
throw new ServiceException(HttpStatus.INTERNAL_SERVER_ERROR.value(), e.getMessage(),
e.getLocalizedMessage());
} finally {
if (manager != null) {
manager.disconnect();
try {
returnSession(session);
} catch (Exception e) {
throw new ServiceException(HttpStatus.INTERNAL_SERVER_ERROR.value(), e.getMessage(),
e.getLocalizedMessage());
}
}
}
}
Below is the method in SftpManagerWithPool:
@SuppressWarnings("unchecked")
public List<LsEntry> listFiles(String pathFrom) throws SftpException {
if (c == null || session == null || !session.isConnected() || !c.isConnected()) {
throw new SftpException(ErrorCode.LIST_FILES.ordinal(), "Connection to server is closed. Open it first.");
}
String filePath = pathFrom + "/";
return c.ls(filePath);
}
Everything works well with files no larger than 5k, but when we have more files it takes a long time and this leads to timeouts.
My question is, how can I use the method c.ls(filePath);
to list only the first N files in a folder? I'm looking for something similar to the Linux shell commandls -U | head -4
---- EDIT -----
I changed my method listFiles
as follows, but I still don't get the best performance:
public List<LsEntry> listFiles(String pathFrom, int top) throws SftpException {
if (c == null || session == null || !session.isConnected() || !c.isConnected()) {
throw new SftpException(ErrorCode.LIST_FILES.ordinal(), "Connection to server is closed. Open it first.");
}
String filePath = pathFrom + "/";
List<LsEntry> response = new ArrayList<>();
c.ls(filePath, new LsEntrySelector() {
@Override
public int select(LsEntry record) {
if (response.size() <= top) {
response.add(record);
return LsEntrySelector.CONTINUE;
} else {
return LsEntrySelector.BREAK;
}
}
});
return response;
}
Thank you very much:)
source to share
Use this method overload ChannelSftp.ls
that accepts an LsEntrySelector
interface :
public void ls(String path, LsEntrySelector selector)
Submit LsEntrySelector.select
to collect file records. When you have enough of them, return it BREAK
.
source to share