When comparing a very large number of files, the -1 flag to
fdupes makes it much to manage the output (IMO). Then each
line printed is a list of duplicate files.
But it depends on your own needs.
After seeing what you're trying to do, I'd say fdupes is a
perfect fit.
paulm