In case many consecutive files are skipped in a directory, the
recursive next() implementation recursed for every file, i.e.
calling itself repeatedly.
Instead, loop over all directory items until either a file or directory
is found which should be indexed, or there are no more items left.
Move the "shouldIndexHidden" variable out of the loop, as it is
invariant.