wicket FileChannelPool has scalability issues

I benchmarked the FileChannelPool, which is used in the pagemanager's DiskStore, against a simple unpooled 'new RandomAccessFile(name, "rw").getChannel()' implementation. The filechannelpool caches 50 handles by default to 50 session diskstore files, and keeps these handles open as long as possible or necessary.

The benchmark results are performed from 1 to 8 threads and with active file set size varied. I tested 10, 50, 60 and 100 active files. To understand the results, FCP stands for the time spent on wicket's FileChannelPool. FC stands for time spent on unpooled FileChannel.

The times are the time it took to perform 100000 per thread of the following:

• open a randomly chosen file from set of active files
• read a randomly positioned 10k chunk of a simulated session file that is 100000 bytes long (seek to 0 .. 90000 and then read 10000 bytes)
• close file

Results for 10 files:

• FCP: 79 161 318 481 763 1239 1355 1423 ms
• FC: 531 660 848 859 1141 1285 1372 1509 ms

Results for 50 files:

• FCP: 82 165 323 516 756 1248 1560 1587 ms
• FC: 529 666 800 851 1150 1268 1389 1490 ms

Results for 60 files:

• FCP: 219 857 2535 3669 4584 5641 6443 7235 ms
• FC: 520 639 901 1034 1196 1314 1501 1559 ms

Results for 100 files:

• FCP: 471 3588 5989 7814 10048 11790 13580 15130 ms
• FC: 547 667 903 1093 1195 1388 1446 1603 ms

Bonus result: near worst case behavior of filechannelpool of just 10 cached channels and 100 files active set:

• FCP: 744 4669 8182 11295 14934 17799 21959 24964 ms
• FC: 549 681 905 1112 1338 1383 1440 1564 ms

Discussion of the results: it appears that there is little difference in performance on the unpooled file channel for any active set size. This is not unexpected: there are no synchronization points and the kernel doesn't much care either way, because all these files together only took 100 * 100k of memory, or about 10 MB – easily cached entirely in my 4 GB. However, the FileChannelPool shows significant performance degradation as soon as the active set exceeds the pool's size. It seems to perform order of magnitude worse with 100 files as opposed to 50, indicating that wicket websites may suffer rapid performance degradation under load. In case the number of active sessions is much larger than the pool, the very worst behavior is simulated as the last result.

FileChannelPool's key problem is that it optimizes performance for the lightly loaded case, and actually harms performance when load gets higher. The harm is not much – in the worst case, only something like 29 microseconds per request – but the best case win isn't great either, about 5.6 microseconds per request (10 files, 1 thread). In the interests of simplifying pageStore, I propose deleting FileChannelPool entirely.