Implement asynchronous readdir

Accessing directories with many (several thousand) files over sshfs is
slow, because most SFTP server implementations only send a fixed amount
of entries per READDIR command (e.g. OpenSSH SFTP: 100 entries). This
patch implements sending several READDIR commands in parallel, in order
to speed up directory listing in these cases.

An option (sync_readdir) is also added so that users can easily switch
on the old behaviour.

The performance improvement is astonishing. Accessing a directory with
30k files in from a remote server that has a RTT of 15ms via OpenSSH
SFTP:

Synchronous readdir:
 $ ./sshfs -o sync_readdir host:/tmp /mnt/temp
 $ time "ls" -1 /mnt/temp/test | wc -l
 30000
 "ls" -1 /mnt/temp/test  0.07s user 0.01s system 1% cpu 6.928 total

Asynchronous readdir:
 $ ./sshfs host:/tmp /mnt/temp
 $ time "ls" -1 /mnt/temp/test | wc -l
 30000
 "ls" -1 /mnt/temp/test  0.07s user 0.01s system 12% cpu 0.605 total

Accessing a directory with 100k files shows even more dramatic
improvement:

Synchronous readdir:
 $ ./sshfs -o sync_readdir host:/tmp /mnt/temp
 $ time "ls" -1 /mnt/temp/test2 | wc -l
 100000
 "ls" -1 /mnt/temp/test2  0.67s user 1.22s system 0% cpu 3:31.56 total

Asynchronous readdir:
 $ ./sshfs host:/tmp /mnt/temp
 $ time "ls" -1 /mnt/temp/test2 | wc -l
 100000
 "ls" -1 /mnt/temp/test2  0.20s user 0.03s system 14% cpu 1.631 total

This can easily be reproduced by creating a directory on a server and
touching a lot of files in it:
 $ mkdir /tmp/test
 $ cd /tmp/test
 $ for i in $(seq 1 30000); do touch $i; done

Signed-off-by: Alexander Neumann <alexander@bumpern.de>

0 files changed, 0 insertions, 0 deletions