From 6a2d06e36fe73e592e02a16ff5025ec622eb015e Mon Sep 17 00:00:00 2001 From: Alexander Neumann Date: Sun, 22 Dec 2013 20:45:40 +0100 Subject: Implement asynchronous readdir Accessing directories with many (several thousand) files over sshfs is slow, because most SFTP server implementations only send a fixed amount of entries per READDIR command (e.g. OpenSSH SFTP: 100 entries). This patch implements sending several READDIR commands in parallel, in order to speed up directory listing in these cases. An option (sync_readdir) is also added so that users can easily switch on the old behaviour. The performance improvement is astonishing. Accessing a directory with 30k files in from a remote server that has a RTT of 15ms via OpenSSH SFTP: Synchronous readdir: $ ./sshfs -o sync_readdir host:/tmp /mnt/temp $ time "ls" -1 /mnt/temp/test | wc -l 30000 "ls" -1 /mnt/temp/test 0.07s user 0.01s system 1% cpu 6.928 total Asynchronous readdir: $ ./sshfs host:/tmp /mnt/temp $ time "ls" -1 /mnt/temp/test | wc -l 30000 "ls" -1 /mnt/temp/test 0.07s user 0.01s system 12% cpu 0.605 total Accessing a directory with 100k files shows even more dramatic improvement: Synchronous readdir: $ ./sshfs -o sync_readdir host:/tmp /mnt/temp $ time "ls" -1 /mnt/temp/test2 | wc -l 100000 "ls" -1 /mnt/temp/test2 0.67s user 1.22s system 0% cpu 3:31.56 total Asynchronous readdir: $ ./sshfs host:/tmp /mnt/temp $ time "ls" -1 /mnt/temp/test2 | wc -l 100000 "ls" -1 /mnt/temp/test2 0.20s user 0.03s system 14% cpu 1.631 total This can easily be reproduced by creating a directory on a server and touching a lot of files in it: $ mkdir /tmp/test $ cd /tmp/test $ for i in $(seq 1 30000); do touch $i; done Signed-off-by: Alexander Neumann --- sshfs.1 | 3 +++ 1 file changed, 3 insertions(+) (limited to 'sshfs.1') diff --git a/sshfs.1 b/sshfs.1 index 4c18876..0bad826 100644 --- a/sshfs.1 +++ b/sshfs.1 @@ -58,6 +58,9 @@ synchronous writes \fB\-o\fR no_readahead synchronous reads (no speculative readahead) .TP +\fB\-o\fR sync_readdir +synchronous readdir +.TP \fB\-o\fR sshfs_debug print some debugging information .TP -- cgit v1.2.3