If the system gets under serious memory pressure it'll happily discard
your text pages too (and later reload them from disk). The same
for any file data you might need to access.
swapoff will only affect anonymous memory, but not all the other
memory you'll need as well.
There's no way around mlock/mlockall() to really prevent this.
Still even with that you could still lose dentries/inodes etc which
can also cause stalls. The only way to keep them locked
is to keep the files always open.
-Andi
-