RE: [malware-list] TALPA - a threat model? well sorta.

Previous thread: invalidate caches before going into suspend by Mark Langsdorf on Wednesday, August 13, 2008 - 9:41 am. (22 messages)

Next thread: [2.6.27-rc3] section mismatches and compilation warnings by Rufus & Azrael on Wednesday, August 13, 2008 - 9:53 am. (1 message)
From: Eric Paris
Date: Wednesday, August 13, 2008 - 9:36 am

So after about a week of trying to squeeze information out of
anti-malware companies I'm starting to feel like I can better speak for
their needs (although they probably don't like what I have to say).  I
would like to point out that many enterprises are going to run this
stuff on their machines.  Period.  End of story.  Personally I'd rather
support a clean interface than have to try to support support problems
my customers have when their hacked fragile systems have trouble.

So, what is it that anti-malware companies do?  They scan files.  That's
it.  Many of the arguments they make and marketing speak we hear or
think we hear claim more grandious things.  But from what I've seen they
aren't the real deal.  Most of the security model descriptions that
people on list want actually are framed under the belief that these
products need to or attempt to completely block some class of attacks.
They don't.  If you think they do you need to fix your frame of
reference.  The only class of attacks this interface is supposed to
address is those that can be found by scanning files.  This is NOT an
LSM.  This is NOT an IDS.  This is a file scanner.  There may be all
sorts of marketing material or general beliefs that they provide
security against all sorts of wide and varied threats (and they do),
but in all reality the only threats they provide any help against are
those that can be found by scanning files.  Simple as that.  Some may
argue this isn't "good" security and I'm not going to make a strong
argument to the contrary, I can stand here for days and show cases where
this is highly useful but no one can provide a threat model more than to
say, "we attempt to locate files which may be harmful somewhere in the
digital ecosystem and try to deny access to that data."

It can be easily shown (and I can come up with LOTS of example) that
this is of value.  Lets consider a simple Linux worm:

http://www.ca.com/us/securityadvisor/virusinfo/virus.aspx?id=47869

This worm needs 4 ...
From: Alan Cox
Date: Wednesday, August 13, 2008 - 9:24 am

Good so lets instead have a discussion about making the file event
notification more scalable. That is the same thing I want for content
indexing. It is the same thing you want for certain kinds of smart
archiving, for on-line asynchronous backup and other stuff.

It ought to be a simple clean syscall interface.

Alan
--

From: Eric Paris
Date: Wednesday, August 13, 2008 - 9:47 am

Are you willing to make it blocking?  I'm not sure how to make what we
have capable of assuring that the object you got a notification about is
actually the object you are acting on.  Thoughts on how to accomplish
that?  I'm here to code and I'm willing to throw all my work in the
garbage if someone can show me how to actually do it better.

-Eric

--

From: Alan Cox
Date: Wednesday, August 13, 2008 - 9:37 am

On Wed, 13 Aug 2008 12:47:45 -0400

I don't think you need to be blocking if you passed up a file handle ?

	fd = fileeventmumble(somestuff);
	do_stuff
	close(fd);

[taking care not to end up recursing as a result]
--

From: Eric Paris
Date: Wednesday, August 13, 2008 - 10:00 am

Without blocking and waiting how do you deny access?  Maybe I needed
another thing they do.  "They do file scanning and deny access to bad
files."

async scanning on close/write is great.  but you need blocking/access

[you pointed out the whole point of process exclusions in the original
work]

--

From: Alan Cox
Date: Wednesday, August 13, 2008 - 12:59 pm

Denying access is easy enough - chmod it or set an SELinux label on it.

There are a limited number of useful arrangements I can see here

'opened for write deny [some set of] access until verified'

Thats an selinux state transition on open for write, and one from the
user space end on the other

'run around after things'

Which is simply a notifier and a relabel as 'contaminated' or similar (or
a chmod or mv ...) by the daemon end of things.

Now I can see why you might want a 'has been open for a write and dirty
for a while' notifier - again for indexing as well as virus scanners and
the like.

What other semantic do you have in mind given that you have to deal with
open by me, open by writer, content modified.. after I have it open
anyway ? It seems the rest is a discussion about time intervals ?
--

From: Press, Jonathan
Date: Wednesday, August 13, 2008 - 2:24 pm

it.

I may be missing something about your suggestion, but I don't see how
this would work.  Who does the chmod?

Here's a sequence:

- Application opens file
- AV scanner notified in some way without blocking
- Application reads file into memory
- AV scanner determines file is infected.
- AV scanner chmod's file -- oops, too late.
- Application sends file over the wire to another machine with a more
vulnerable OS

How would this be prevented?


Jon Press
--

From: Alan Cox
Date: Wednesday, August 13, 2008 - 2:13 pm

I don't think you can. In your case how does your AV scanner deal with
the case where the application opens the file while another user has it
open and the other user (or even other task with the same handle) changes
the content possibly via mmap.

Content may also directly be shared between users of a file using the
mmap interfaces so your scan on read model is rather dysfunctional.

--

From: Rik van Riel
Date: Wednesday, August 13, 2008 - 2:35 pm

On Wed, 13 Aug 2008 17:24:28 -0400

Chmod is also not a solution to the hierarchical storage (or incremental
restore from backup) problem.

I believe we really do need the block-on-open.

-- 
All Rights Reversed
--

From: Alan Cox
Date: Wednesday, August 13, 2008 - 2:23 pm

On Wed, 13 Aug 2008 17:35:29 -0400

The block on open is orthogonal really. Useful for HSM, useful for
certain very primitive scanning but not much else that I can see.

And its a minor mod to the security hooks to allow it as far as I can see
--

From: Eric Paris
Date: Thursday, August 14, 2008 - 8:25 pm

So here's where I run into trouble.  Lets assume I want to be helpful
and engineer in a vacuum for this unknown HSM user as well.  Clearly
file scanners need the file to be there which means the inode ("on disk"
kind) needs to be there and stuff like that.  I assume that the HSM user
is going to need to hook long before these things even exist.  Where
would they need to hook?

Should I just design for my own needs and include stacking and a
priority number and hopefully the HSM people can use it later?  I don't
know the details of what might someday by needed for a project I know
nothing about *smile*

-Eric

--

From: Jan Harkes
Date: Friday, August 15, 2008 - 1:16 pm

Or use either the Fuse or Coda kernel modules and handle such requests
in userspace. With FUSE you should even be able to block on a per-page
granularity, Coda only has session semantics so it will only notify
userspace of the open and close events, while read and write and mmap
are passed directly to the underlying file.

Jan

--

From: Arjan van de Ven
Date: Friday, August 15, 2008 - 3:05 pm

On Fri, 15 Aug 2008 16:16:22 -0400

I believe more that we need block-on-read ;-)
(on open we can start an async scan to cut latency)

read() (or mmap etc) is where the actual use/transfer of
contaminated data happens, not in the open.


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--

From: Eric Paris
Date: Sunday, August 17, 2008 - 4:19 pm

I could probably buy that, but I don't know how an HSM would work.
Would we have everything we need at open for them to fire off?

/me is HSM clueless and trying to include their needs is proving a
challenge.

-Eric

--

From: Arjan van de Ven
Date: Sunday, August 17, 2008 - 4:26 pm

On Sun, 17 Aug 2008 19:19:37 -0400

I bet it's the same; until you read/mmap... no need to get the data.

you wouldn't want an "ls -l" or "find" to restore your entire system
from tape after all :)

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--

From: David Collier-Brown
Date: Sunday, August 17, 2008 - 2:11 pm

The HSMs I'm familiar with keep the inode and the first block of the file on-disk,
so that stat and file commands work. For Windows, you also need to keep the pointer
to the icon to use valid, or create a fake one.

--dave
-- 
David Collier-Brown            | Always do right. This will gratify
Sun Microsystems, Toronto      | some people and astonish the rest
davecb@sun.com                 |                      -- Mark Twain
cell: (647) 833-9377, bridge: (877) 385-4099 code: 506 9191#
--

From: Alan Cox
Date: Monday, August 18, 2008 - 8:33 am

So don't bother. Its a theoretical use for the most part so we can mangle
the interface later. 

Alan
--

From: Rik van Riel
Date: Monday, August 18, 2008 - 9:43 am

On Mon, 18 Aug 2008 16:33:13 +0100

Think of a consumer HSM application: backup to rsync.net
or Amazon S3.

Instead of waiting for the whole backup to be restored,
you can start using the filesystem immediately.  The 
block-on-open hook can be used by the restore program
to fetch files from the remote backup site on an
as-needed basis, with a full restore going on in the
background.

If the block-on-open hook can be used for that (even
with additional magic, like creating empty HSM inodes
with a special attr to notify "the data lives elsewhere"),
HSM should be good.

The "data lives elsewhere" bit/xattr/whatever could also
be used on directories, so not even the whole directory
tree would have to be restored right on restore :)

-- 
All rights reversed.
--

From: Christoph Hellwig
Date: Wednesday, August 13, 2008 - 10:07 am

Also for hierachial storage management, which also shares they other
requirement with the AV crowd that it want to be able to block the
calling process until the notification is ACKed (for recalling data

I was wondering whether to piggy-back on the audit code was the best
idea here..
--

From: Arnd Bergmann
Date: Thursday, August 14, 2008 - 6:00 am

In this case, it seems to be the same kind of requirement that lead to
dnotify and inotify, right?

So in order to get the third notification syscall interface right, the
obvious questions are:

1. How do you make it cleaner than inotify?
2. How do you make it more scalable than inotify?
3. Do you need it to do anything that inotify doesn't do, beyond 1. and 2.?

	Arnd <><
--

From: Greg KH
Date: Wednesday, August 13, 2008 - 9:57 am

Just repost your patches, with a description of what is happening with
them, taking into account the previously mentioned comments and we can
review it again :)

No problem at all, just a normal patch lifecycle...

thanks,

greg k-h
--

From: Arjan van de Ven
Date: Wednesday, August 13, 2008 - 10:39 am

On Wed, 13 Aug 2008 12:36:15 -0400
Eric Paris <eparis@redhat.com> wrote:





I agree about the "read" case.
"write" is tricky... because there's many vectors to write, from
write() system call to mmap() system call to truncate() system call.
(yes you can write a stream of bytes that passes virus scan, and then
truncate to be suddenly a live virus)

I would propose to use the word "dirty" instead, we all know what we
mean by that (it's all of them) and doesn't tie our thinking to the
write() system call.

I would like to introduce a concept in your discussion you did not
mention yet in this email: the difference between synchronous scanning
and asynchronous scanning.

It's clear from the protection model that you described that on 'read'
you want to wait until the scan is done before you give the data to the
process asking for it... and that's totally reasonable: "Do not give
out bad data" is a very clear line in terms of security.

for the "dirty" case it gets muddy. You clearly want to scan "some
time" after the write, from the principle of getting rid of malware
that's on the disk, but it's unclear if this HAS to be synchronous.
(obviously, synchronous behavior hurts performance bigtime so lets do
as little as we can of that without hurting the protection).
One advantage of doing the dirty case async (and a little time delayed)
is that repeated writes will get lumped up into one scan in practice,
saving a ton of performance.
(scan-on-close is just another way of implementing "delay the dirty
scan").
Based on Alans comments, to me this sounds like we should have an
efficient mechanism to notify userspace of "dirty events"; this is not
virus scan specific in any way or form. And this mechanism likely will
need to allow multiple subscribers.

for the open() case, I would argue that you don't need synchronous
behavior as long as the read() case is synchronous. I can imagine that
open() kicks off an async scan, and if it's done by the time the first
read() happens, no ...
From: Theodore Tso
Date: Wednesday, August 13, 2008 - 11:15 am

Something else to think about is what happens if the file is naturally
written in pieces.  For example, I've been playing with bittorrent
recently, and it appears that trackerd will do something... not very
intelligent in that it will repeatedly try to index a file which is
being written in pieces, and in some cases, it will do things like
call pdftext that aren't exactly cheap.  A timeout *can* help (i.e.,
don't try to scan/index this file until 15 minutes after the last
write), but it won't help if the torrent is very large, or the
download bitrate is very slow.  One very simple workaround is to
disable trackerd altogether while you are downloading the file, but
that's not very pleasant solution; it's horribly manual.

Most of this may end up being outside of the kernel (i.e.,some kind of
interface where a bittorrent client can say, "look this file is still
being downloaded, so it's don't bother scanning it unless some process
*other* than the bittorrent client tries to access the file".  And
maybe there should be some other more complex policies, such as the
bittorrent client explicitly telling the indexer/scanner that the file
is has been completely downloaded, so it's safe to index it now.

But what this points out is that if you want a good solution, (a) it
probably shouldn't all be in the kernel, since trying to get all of
this complexity into the kernel will be painful, and (b) the policy
about whether or not a bittorrent client should be allowed to say,
"it's OK not to check the file until it's completely downloaded, even
if I am handing out pieces to other people over the network --- after
all the entire file has its own SHA checksum for data integrity
verification --- is very much a policy question where different system
administrators will come down on different sides about what should and
shouldn't be allowed --- and therefore this kind of policy decision

We have an i_version support for NFSv4, so we have that already as far
as the version of the file.  We can ...
From: Arjan van de Ven
Date: Wednesday, August 13, 2008 - 11:21 am

On Wed, 13 Aug 2008 14:15:49 -0400

exactly. Even more, since this is async work, the scheduling of the
order of work also is a policy.. and userland is again the right place

well I was assuming we only store this in memory (say in the inode) and
just rescan the file if we destroy the in memory inode.
I don't see the need for this to be persistent data; in fact I assume
(Eric, please confirm) that this data is not *supposed* to be

yes that's a hard question; what if you have 2 virus scanners active.

(they could register a version of the database with the kernel, and the
in kernel version-cookie could be a hash of all registered versions I
suppose.. if anything changes ever we just rehash and scan as if we
have to do that)

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--

From: tvrtko.ursulin
Date: Thursday, August 14, 2008 - 2:18 am

It is in fact really simple with the proposed inode versioning approach. 
Any client which wants to invalidate the cache just needs to bump the 
global version number which will force a rescan on next access.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

From: Eric Paris
Date: Wednesday, August 13, 2008 - 12:02 pm

I never suggested putting a scanner in kernel.  Sound like you want the
"allow don't cache" response from your userspace scanner while this is
going on.  The kernel doesn't need to be making decisions about when to
send events, nor should userspace tell the kernel not to send events.
Its up to whatever the scanner is to agree not to actually do any

No.  How in the heck can some out of kernel database store information
about what inodes have been scanned in any even slightly sane way?  And
people think the race between open and read is too large and you suggest
moving clean/dirty marking to a userspace database?  I MUCH prefer my
(and it sounds like arjan agrees) clean/dirty versioned flag in inode.

-Eris

--

From: Theodore Tso
Date: Wednesday, August 13, 2008 - 12:29 pm

And if the system isn't running a virus checker, but just a file
indexer (ala tracker), it shouldn't go to userspace at all.  In that
case all that is necessary is an asynchronous notification.

Also something else that is needed is support for multiple clients.
(i.e., what happens if the user runs two virus checkers, or a virus
checker plus a hierarchical storage manager driving a tape robot, or
all of the above plus trackerd --- where some clients need to block
open(2) access, and some do not need block open(2) --- and in the case
of HSM, ordering becomes important; you want to retrieve the file from

Don't ask me; I think most AV checkers for linux are security theater
and not very much use (other than making money for the AV company's
shareholders) anyway.  I thought you were the one who wanted to record
information about which version of the virus db a particular file had
been scanned against.  The place where I can see this being useful is
what happens you get a new virus DB, and so you need to start scanning
all of the files in your 5TB enterprise file server --- and then the
system crashes or it needs to be taken down for scheduled maintenance.

You want to have *some* off-line database for storing this
information, since it would be silly to want to have the first thing
that happens after a new virus DB gets downloaded is to interate over
the entire filesystem, clearing a persistent the "clean" bit --- that
would take *forever* on a 5TB filerserver; and what happens if you
crash in the middle of clearing the "clean" bit..  And if the system
gets shutdown in the middle of the scan, you need some way of
remembering which inodes have been scanned using the "new" db, and
which ones haven't yet been scanned via the new virus db.  All of this
should be kept in userpsace, and is strictly speaking Not Our Problem.

I'm just arguing that there should be absolutely *no* support in the
kernel for solving this particular problem, since the question of
whether a file has been ...
From: Press, Jonathan
Date: Wednesday, August 13, 2008 - 2:15 pm

The issue of multiple clients does need to be accounted for.  However, I
will mention that it is unusual (at least in my experience) to actually
run two AV products at the same time in "realtime" mode.  We strongly
recommend that anyone who installs our product should remove any other
AV product on the system -- for technical reasons, not financial --
since they've already paid for ours by the time they get to this point.

Caching of previous results can be done in either user space or kernel
space.  We have seen it both ways.  Wherever it is done, I would say
that rather than record AV signature file version numbers, there should
be a mechanism for the application to invalidate or flush the cache
whenever a signature update is done.  There are other circumstances
where that would also be useful -- such as if the user changes a
scanning option in a way that increases the strictness of the scan.  In
other words, a file that was marked as clean based on one level of
strictness may not be clean under a stricter scan.  You wouldn't want
the cache to prevent it from being scanned under such circumstances.


Jon Press
--

From: tvrtko.ursulin
Date: Thursday, August 14, 2008 - 2:30 am

Hm, maybe by implementing a facility with which a client can register it's 
interface usage intent? Something like:

register(I_HAVE_NO_INTEREST_IN_CONTENT);
register(I_WANT_TO_EXAMINE_CONTENT);

All former ones would run first because they only want to have the 
opportunity to block and do something unrelated to file content (like 
HSMs), and later group would be ran last since they want to examine the 
content.

Ordering inside those two groups is not important because I don't see how 
a model other than restrictive can make sense with content security 

The thing is the idea never was for clean-dirty "database" to be 
persistent but to have the same lifetime as the inode (in memory one). And 
once the cache/database gets invalidated re-scanning happens on-demand so 
the 5TB problem does not exist. Concerns about multiple clients where 
every has a different versioning scheme are also not relevant with the 
proposed versioning scheme (see my reply to Arjan).

So this problem can be solved in kernel in a much more performant and 
secure way than it would be possible in userspace and all that with just a 
handfull of lines of code.

--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos

"Views and opinions expressed in this email are strictly those of the 
author.
 The contents has not been reviewed or approved by Sophos."
 

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

From: Press, Jonathan
Date: Thursday, August 14, 2008 - 5:03 am

I'm not sure I understand why "interest in content" means not blocking,
and vice versa.  However, I think this is a good idea if made more


Jon Press

--

From: tvrtko.ursulin
Date: Thursday, August 14, 2008 - 5:27 am

Small misunderstanding because both would block. If you go back to Ted's 
original post I was replying to, he was worried about how would 
anti-malware scanning interact with HSM since both may end up using the 
same interface. HSM, as far as I understand it, needs to block on open and 
"plant" the right file in place, while anti-malware also needs to block 
and examine the right content. That is why ordering matters, anti-malware 
needs to run after the content is put in place. And that is what my idea 
solves (slight overstatement since I spent only seconds on it) by 
separating them in two groups of clients. First which has no interest in 
content and second which does.

--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos

"Views and opinions expressed in this email are strictly those of the 
author.
 The contents has not been reviewed or approved by Sophos."
 

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

From: Pavel Machek
Date: Friday, August 15, 2008 - 7:31 am

I don't see why HSMs are mixed into this discussion -- they appear
very different. For one, you probably have just one filesystem with
tape storage, so it makes sense to implement HSM at filesystem level;
probably with something like FUSE+unionfs.
							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--

From: Theodore Tso
Date: Thursday, August 14, 2008 - 6:24 am

So in essence, what I hear you saying is that all AV products want to
work in a mode where the moment the inode falls out of the inode
cache, and we lose the "clean" bit, when the inode is brought back
into the cache, it will be scanned again.  That is, the "clean" bit is
never persistent, and never needs to be stored in memory.

That seems fair; if it turns out there is an AV product that wants to
optimize this a bit further, as long as we provide a persistent inode
version/generation number, they can always do their own persistent
database in userspace.

							- Ted
--

From: Eric Paris
Date: Thursday, August 14, 2008 - 6:48 am

There needs to be a way to say that an inode in cache needs to be
rescanned.  3 states this flag can be.  Clean, Dirty, Infected.  The
current talpa solution involves a global monotomically increasing
counter every time you change virus defs or make some "interesting"
change.  If global == inode flag we are clean.  If global == negative
inode flag we are infected.  if global > inode flag we are dirty and

exporting i_version might be useful for better userspace caching,
although I've yet to see any reasonable description of how a userspace
database can map between data on disk and what they have in userspace.
How can a userspace process, given 2 file descriptors know they are
actually the same thing on disk?

--

From: Theodore Tso
Date: Thursday, August 14, 2008 - 8:50 am

"Infected" just means to instantly return an error when the file is
opened or if an already opened file descriptor is read or mmap'ed,
right?  If file is already mmaped(), what's the plan?  Send a kill -9
to the process, even if it ends up kill off an emacs or openoffice

If a userspace database knows that inode X, i_version Y was checked a
day ago, and inode X still has i_version Y, even if that inode has
been evicted from memory, the contents will be the same absent root
messing about with direct access to the block device.  If there was an
intervening boot, the someone could remove the disk, edit the disk
block directly -- but that person could also add a backdoor to the
kernel while they were at it.

If your threat model is, "we do file scanning; that's it", then having
an external database which uses the inode number and i_version as a
tuple makes a lot of sense --- for filesystems where i_version is
getting bumped on every disk write, which is needed to support NFSv4
cache support, anyway.  

							- Ted

--

From: Eric Paris
Date: Thursday, August 14, 2008 - 10:29 am

We don't have a revocation mechanism in linux and this isn't about
adding one.  Your trying to turn this into something it isn't.  If you
have it opened and mmap'd you've got that file for as long as you want.
I've already said that given Arjan's belief that we can move it
read/write instead of open/close we are moving the open->read race to a
mmap->fault race.  It isn't perfect at stopping bad data flow, but its

is i_version an on disk think?  didn't realize that and just assumed it
was in in core thing.  I wouldn't have an issue sending i_version to the
userspace scanner for them to use as they like.

-Eric

--

From: Theodore Tso
Date: Thursday, August 14, 2008 - 12:17 pm

It's on-disk for some filesystems, in order to support NFsv4 advanced
caching semantics (which means i_version has to survive a reboot,
which means it has to be on disk).  It is *not* on disk for ext3,
although it is for ext4.

					- Ted
--

From: Eric Paris
Date: Thursday, August 14, 2008 - 12:20 pm

Is there some pragmatic way in kernel to know if the inode I have
corresponds to a persistent or in core only inode?  I'll poke around
myself and look.

-Eris

--

From: Christoph Hellwig
Date: Thursday, August 14, 2008 - 12:34 pm

It's not used at all on regular files except for ext4 with a non-default,
undocumented mount option.  XFS will grow it soon in a similar way as ext4,
except that it will be documented or I might have even figured out by
then how to just enabled it from nfsd.
--

From: Theodore Tso
Date: Thursday, August 14, 2008 - 12:41 pm

We do need a standardized way of enabling it (since it does cost you
something in performance, so not everyone will want it on), and a
standardized way of reading i_version out to userspace.  Maybe a mount
option is the right way to do it, maybe not.

We may want to take this to the linux-fs list and try to get
agreements on these points; the main reason why it's not enabled by
default in ext4 is because the NFSv4 advanced caching code is in
common use (is it even in mainline)?

						- Ted
--

From: Christoph Hellwig
Date: Thursday, August 14, 2008 - 1:20 pm

I have not yet seen code actually using it at all, neither in mainline
nor on one of the many nfs development lists.

--

From: J. Bruce Fields
Date: Thursday, August 14, 2008 - 2:21 pm

Oops, I'd love to, and it should be very easy.  How do I find out if
i_version is supported on a given superblock?

There's nothing particularly "advanced" about this, by the way--this is
a very minor variation on the caching model that nfs has always had, and
our nfsv4 server is currently pretty broken without it.

Actually, it's pretty broken even on nfsv2/v3 for filesystems with poor
time resolution.

--b.
--

From: Theodore Tso
Date: Thursday, August 14, 2008 - 4:34 pm

We don't have a way of exporting this fact at the moment.  I assume

Well, if you're willing to try it out, as I've mentioned on my
blog[1][2], ext4 is working pretty well on my laptop --- I'm running
it as my primary filesystem.  There are a few problems with ext3
filesystems converted to use ext4, as opposed to starting afresh via
"mke2fs -t ext4dev /dev/hdXX" that we've just found in the past week
(and fixed within a day or two, although they haven't been pushed to
Linus yet), but overall, it's been pretty stable.

So this would be a good time for someone who is familiar wiht NFSv4 to
try it out and let us know if the i_version support is as you would
like in ext4 --- we're in the bugfixing/stablization phase right now,
so this would be an ideal time to get that feedback.  For more
information, on how to get started, please see:

http://ext4.wiki.kernel.org/index.php/Ext4_Howto

for instructions, and mount the filesystem with the "-o i_version"

And that's fixed in ext4 as well....

						- Ted

[1] http://thunk.org/tytso/blog/2008/06/30/ext4-is-now-the-primary-filesystem-on-my-laptop/
[2] http://thunk.org/tytso/blog/2008/08/08/fast-ext4-fsck-times/

--

From: J. Bruce Fields
Date: Tuesday, August 19, 2008 - 2:43 pm

Neato, thanks.  I set up a toy ext4 filesystem (just a 512 meg sparse
file loopback mounted) on one of my test machines--so I just need to
add the superblock flag and a bit of nfsd code and then I should be able

That's an improvement, yes.  Of course the time is still updated only
every jiffy, so there's still a race:

				file updated
	client checks ctime
				file updated again within a jiffy
	client checks ctime again, concludes file hasn't changed.

--b.
--

From: david
Date: Thursday, August 14, 2008 - 6:44 pm

could you do something like defining a namespace inside posix attributes 
and then setting up a mechanism in the kernel to alert if the attributes 
change (with the entire namespace getting cleared if the file gets 
dirtied)?

this would have the advantage of storing the clean/dirty state in a known 
location on the filesystem (you would need to enable posix attributes, but 
that should be an acceptable limit), and would allow multiple scanners 
(virii, indexing, etc) to do their own thing at their own schedule when 
things get dirtied.

the userspace library calls that open the files would be able to take care 
of the issue of deciding which of the configured scanners on a system 
should be called for each attribute that's not set on a file.

This would seem to require minimal kernel support

1. reserve a attribute namespace

2. clear that namespace if the file is dirtied

3. provide a notification mechanism for programs to subscribe to that 
lists all files where the namespace was not already empty when the file 
was dirtied.

everything else can be userspace.

David Lang
--

From: Theodore Tso
Date: Thursday, August 14, 2008 - 7:04 pm

According to Eric Paris the clean/dirty state is only stored in
memory.  We could use the extended attribute interface as a way of not
defining a new system call, or some other interface, but I'm not sure
it's such a great match given that the extended attributes interface
are designed for persistent data.

I agree that doesn't actually work very well for the tracker use case,
where you the clean/dirty bit to be persistent (in case the tracker is
disabled due to the fact you are running on battery, for example, and
then you reboot).

						- Ted
--

From: Arjan van de Ven
Date: Thursday, August 14, 2008 - 8:41 pm

On Thu, 14 Aug 2008 22:04:00 -0400

but we need a "give me all dirty files" solution, not a "is this file
dirty" solution.

I do not want a virus scanner to constantly have to poll the whole fs
for dirty files ;-)

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--

From: david
Date: Thursday, August 14, 2008 - 10:05 pm

I'm not sure.

there are two situations (with the transition between them)

1. unscanned system, we want to do everything. (this happens immediatly 
after a new signature file is deployed)

here you do just want to filter out the files that have been scanned from 
the list of everything, and you probably want to check at the time of 
scanning the file in case it was opened (and scanned) in the meantime.

2. mostly scanned system, we only want to scan files that have been 
dirtied.

here you don't need to scan everything, you only need to scan in two cases

2a. the file was dirtied (you learn about it and add it to the queue of 
files to scan when you get around to it)

2b. an unscanned file is opened (the library detects that the file was not 
marked approved by all the current scanners, so it invokes the scanners on 
this file before completing the open, or copy for mmap, or whatever)



In the first case the attributes work "don't bother scanning me".

In the second case they also work (becouse you aren't trying to scan 
everything)


the only time there is a headache is in the transition between them when 
you have scanned a lot of the system, but not all of it, and the machine 
was rebooted so you lost track of what you had scanned.

it shouldn't be too hard to deal with this. even if you never resume the 
scan you are still safe (becouse of the scan-on-open), but it's also 
possible to either do a find f(or equivalent) or files without the 
attribute and store the results (similar to updatedb) and then updating 
the file to mark the files as being scanned (update in place, change the 
first character or something to be fairly crash safe). after the full list 
of files has been scanned shift to the second mode.

the sweep scan should be a background task, possibly disabled when on 
battery power.

why would this not satisfy the requirements?

David Lang
--

From: Johannes Weiner
Date: Thursday, August 14, 2008 - 10:12 pm

Hi,


2b could also be used as a general lazy scanning mechanism, no?  If
there is a new signature or the file is dirty or unknown, scan it.
Always.

So this should all just be one case, no?

	Hannes
--

From: david
Date: Thursday, August 14, 2008 - 10:28 pm

this depends on the policy in userspace.

you could:

1. plan on doing all scanning as files are opened (this is going to be 
noticable to users)

2. run the background scanner continuously as a low priority (as 
soon as you finish one scan, do the next)

3. do one background scan when you get a new signature, and after you 
complete that only scan files that change.

and probably others

#3 has the advantage that the only time a user will hit a delay when 
opening a file is if the file was just recently modified and the scanner 
daemon(s) haven't had a chance to scan it yet.

David Lang
--

From: david
Date: Thursday, August 14, 2008 - 10:36 pm

one way this approach would be a pain is that as signature files got 
updated the attributes would accumulate.

a couple ways of dealing with this.

1. scanners clean up after themselves (when they go to add a new one the 
remove the old one)

2. background sweep through the system removing all tags that aren't in 
the current 'blessed' set

David Lang
--

From: david
Date: Thursday, August 14, 2008 - 9:48 pm

why would you not want the results of the viris scan to be persistant?

assuming that you are not dual-booting( or otherwise taking explicit 
action to go around the OS), the fact that you scanned the file with 
signature set 123 is just as valid now as it was yesterday.

If you are not trying to work around the actions of root, why are you 
trying to work around the actions of the person who has physical control 
of the system? (who could easily become root if they wanted to)

David Lang
--

From: Alan Cox
Date: Friday, August 15, 2008 - 1:51 am

This is another application layer matter. At the end of the day why does
the kernel care where this data is kept. For all we know someone might
want to centralise it or even distribute it between nodes on a clustered
file system.

Alan
--

From: Pavel Machek
Date: Friday, August 15, 2008 - 7:37 am

Me thinks about naming my machine
host-inline-real-virus-or-at-least-identifiable-part-here.cz, then
having fun with people not able to access their apache logs.

Returning an error when you think you see a virus is a great
invitation to DoS attack, really.
							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--

From: Eric Paris
Date: Wednesday, August 13, 2008 - 11:57 am

I was thinking about LD_PRELOAD.  you're right for most things it could

/me doesn't want to ignore things that help his argument, even if they

I already have that concept!  my open time scanning is sync and can
return an EACCESS.  my close time scanning is async and only updates the
in inode clean/dirty mark.  It makes no access decisions.  That way on
the next open we are good to go!  And yes yes, mmap write after close

I'm certainly willing to go down the inotify'ish path for async
notification of 'dirty' inodes instead of implement my own async

An interesting addition.  Trying to keep these queues of events gets
much more complex, but if people really think the open to read race is


This means what exactly?  Some new syscall?  You want some program to
say "notify userspace about this inode" and then every userspace thing
that wants the notification gets it?  Not sure how much brains I want to
build into generic userspace apps.  I certainly wouldn't want to make
any userspace app easily be able to avoid AV scanning even if I make no
promises about malicious apps.  Just because I don't make gurantees
doesn't mean I should make it easy.  Just because I can't be 100% that a
fence won't keep the kids from falling in the pool doesn't mean I
shouldn't build it.  Differentiating between don't AV scan and don't


scan on mmap read?  How do I implement this?  Do we update atime on mmap

My first suggestion is we don't, and worry about it as soon as we have

We don't.  Let the LSM do its job?

--

From: Arjan van de Ven
Date: Wednesday, August 13, 2008 - 2:39 pm

On Wed, 13 Aug 2008 14:57:44 -0400

it's not "just" about open-to-read race.
it's about open being non-blocking, and if read is not immediate, never
hitting the latency at all.

The real point is that "read" is the actual point  you want to
intercept, not "open" (you even wrote that in your description).. so
why not just do that ?

on calling mmap(); not at fault time.


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--

From: Eric Paris
Date: Thursday, August 14, 2008 - 7:12 am

I've been thinking about this more over night.  I really like the idea
for perf reasons but I'm scared of programs not expecting and thus
poorly handling -EACCESS from read.  Every program ever is going to
expect that back from open, but once you have the fd open its not
common.  

The idea of multiple concurrent outstanding async notifications is going
to be much more difficult to code, but hey, who am I to complain.  We
could have outstanding async scanning requests for any number of writes,
any number of closes, and any number of opens all at the same time.  The
current interface believes that every request out requires a response
but this type of interface basically requires some sort of coalescing.

Would you be opposed to a first patch round that did blocking
enforcement on open like we have today and do sync scanning (blocking or
return EWOULDBLOCK) on read if needed due to concurrent writes?

--

From: Arjan van de Ven
Date: Thursday, August 14, 2008 - 8:57 am

On Thu, 14 Aug 2008 10:12:13 -0400


well... do you really need a response?


I think we need to sort the interface issue on this for sure, and


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--

From: Helge Hafting
Date: Friday, August 15, 2008 - 3:07 am

It seems to me that this "scan on file open" business is the
wrong way to do things - because it reduces performance.

If you scan on file open, then your security sw is too late and
getting in the way.

It is better to scan in advance. Most machines has lots of idle time.
Use that time to scan in advance, and mark the files as "clean".

A "clean" file can be opened without further checking anytime - giving
normal high performance.  A file that gets written to becomes "dirty"
until checked again. Some mechanism for making a clean copy of a
clean file might help avoid excessive "dirtying".

"Scan on open" might still be useful for cases when the system
haven't kept up with writing, but please don't aim to have
this be the _primary_ mode of scanning. A file server
where most of the stuff is pre-scanned will likely perform much better
than  one scanning everything on open.

Helge Hafting
--

From: Peter Zijlstra
Date: Friday, August 15, 2008 - 3:37 am

This is the source of all that scan on write/close rambling I've seen go
past the last couple of days..

So I'd not worry about this particular aspect

--

From: Press, Jonathan
Date: Friday, August 15, 2008 - 6:10 am

The problem is that you have to account for the cases where the malware
made it onto the system even if you were trying to catch it ahead of
time.  For example:

- Administrator turns off or reduces AV protection for some reason for
some period of time.  It happens all the time.

- New infection makes it onto the machine before the signatures have
caught up with it.  This also happens.  There is an ongoing PR race
among AV vendors about who was faster on the draw to get out signatures
to detect some new malware.  The fact that this race exists reflects
that reality that there is some window during which new malware will

It isn't the primary mode.  It's the mode that catches things as they
arrive, and as they are about to be used.  Most sites will also employ
the applications' features for running regular scans of the whole system
on a scheduled basis, to catch anything that may have slipped through.


Jon Press

--

From: douglas.leeder
Date: Friday, August 15, 2008 - 6:18 am

Not to mention removable media - it might be old hat, but infected/malware
files can come in on floppies, CDs or USB flash discs careless left on the
pavement outside an office.

-- 
Douglas Leeder

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

From: Theodore Tso
Date: Friday, August 15, 2008 - 10:04 am

Let's go back to the threat model.  The Threat Model which Eric Paris
has suggested is that we are only trying to solve the Scanning
Problem.  Just Scanning.

That implies if the malware has been written to the disk, we will
catch it once AV catching is turned on and the user attempts to run or
otherwise access the file with the bad content.  However, if the
malware starts running, then regardless of whether the malware is
running with user privileges, or manages to get root privileges via
some buffer overflow that wasn't caught via
LSM/SELinux/AppAmor/whatever, this is out of scope of Eric's proposal.

Are we agreed on that?  There may be other components of the solution
such as LSM, SELinux, etc., that will very likely be useful in
protecting the system once the malware starts running.  But I thought
Eric's proposal proposed excluding that from the Threat Model for the
purposes of the interface we are trying to solve.  If that's not true,

That's not a problem given the scanning model proposed by Eric; when
you insert removable media, it will get scanned when it is first
accessed.

	  	    	     	     	     - Ted
--

From: Press, Jonathan
Date: Friday, August 15, 2008 - 11:09 am

That is exactly the idea.  However, the context of this particular
thread was the following statement by Helge Hafting:

	It seems to me that this "scan on file open" business is the
wrong 
	way to do things - because it reduces performance.

	If you scan on file open, then your security sw is too late and 
	getting in the way.

 
We were just pointing out that this is not a good argument in practical
terms AGAINST scanning on open.  In fact, your reply completely
reinforces that point.


Jon Press

--

From: Helge Hafting
Date: Monday, August 18, 2008 - 3:09 am

Scanning on open should be a last resort. Scan in advance when you can.
Of course, removable media cannot be scanned until it is inserted and 
mounted,
that is obvious. The scanning can start as soon as the filesystem is 
mounted though,
there is no reason to wait until users try to access something.

A CD inserted into a CD-server may not necessarily be needed immediately, so
scanning in advance will help here too. The user inserting a CD in a home
computer may start to use stuff right away, or perhaps he spends
some time reading the docs before a complicated install. Sill room for some
scanning in advance, which also may end up with the nice effect of 
caching the CD.

Helge Hafting



--

From: Peter Zijlstra
Date: Monday, August 18, 2008 - 3:14 am

Hmm, then there is the issue that an active scanner will avoid me from
removing the media again. Most annoysome when the machine won't promptly
give back the disk.

Imagine me browsing through a stack of unmarked dvd-rw media trying to
find the one with the right file on it, but having to wait every time
for the whole media to get scanned..

Not a nice picture.

--

From: tvrtko.ursulin
Date: Monday, August 18, 2008 - 3:24 am

I agree. And not only that but also the fact that most of the time I would 
not want mount to trigger a background scan of everything because not all 
might be accessed later. Therefore it would be just wasting resources and 
ruining user experience. This argument is flawed in a way that is 
unsolvable in the same way normal preload to page cache is unsolvable 
because it is impossible to predict the usage pattern.

--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos

"Views and opinions expressed in this email are strictly those of the 
author.
 The contents has not been reviewed or approved by Sophos."
 

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

From: douglas.leeder
Date: Monday, August 18, 2008 - 3:25 am

It might be useful to scan in advance, on mount, or scanner start. But 
that
can be simulated using a user-space program, and kernel-based on-open 
blocking scans:
Just have a background program open the files, and they will be scanned 
and 
cached.
The background program could also do clever things like check if running 
on battery, check
load level, keep hot-lists of files to scan/scan first.

All of which can be implemented on top of the Talpa interface already 
specified.

In practice scan on-open with blocking is fast enough for general usage, 
even using our existing 
hacky kernel module approach.

-- 
Douglas Leeder

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

From: david
Date: Friday, August 15, 2008 - 9:25 am

according to the threat model actions of the administrator do not matter.

David Lang
--

From: Press, Jonathan
Date: Friday, August 15, 2008 - 9:30 am

matter.

Sorry, I don't know what you mean.

Jon

--

From: david
Date: Friday, August 15, 2008 - 10:33 am

the threat model that was posted two days ago in the initial message of 
this thread specificly stated that actions of root are not something that 
this is trying to defend against.

David Lang
--

From: Press, Jonathan
Date: Friday, August 15, 2008 - 10:40 am

I think you may have missed the point of any such statement.

Just to clarify...

The model does not exclude root-owned processes from the notification
and scanning sequence.  If root attempts to execute a file, that file
would be scanned before the execution is allowed.  If a root-owned
process attempts to open a file, that access would be blocked until the
file is scanned.  If a root-owned process closes a file that has been
written to, that file would be scanned.

In addition, to generalize from the incorrect idea that the actions of
root are not being defended against to the idea that the possible
impacts of an administrator's actions in configuring an application
should not be accounted for at all in our thinking doesn't make sense to
me anyway.


Jon Press



--

From: david
Date: Friday, August 15, 2008 - 10:47 am

questions had been raised about how this model could defend against all 
the tricky things that root can do, the answer was that they are not 
trying to defend against root doing tricky things.

turning off the scanner, letting things get infected, and turning it back 
on would fall in the same catagory as marking a file that the scanner 
marked as bad as sucessfully scanned.

in any case the vunerability is limited as the next time the signatures 
are updated the files would get scanned again, so I don't think it's a big 
problem in practice.

David Lang
--

From: Valdis.Kletnieks
Date: Friday, August 15, 2008 - 11:06 am

This problem is actually identical to "new file scanned, but you don't have
the signature available yet so malware isn't detected".

Those of us who have seen large mail servers pile up queues in the 10s of
millions in the 45 minutes between when the worm went critical-mass and when
we got a signature might disagree on it not being a big problem in practice.

Of course, if that's considered "outside" the threat model, somebody better
start writing down exactly what small corner of threat model this is actually
helping against...

From: david
Date: Friday, August 15, 2008 - 1:05 pm

go back to the beginning of this thread. that is defining the (very 
limited) thread model that they are trying to defend against.

the rest of us are not trying to defend against this threat model. we are 
trying to identify the appropriate infrastructure that could be used by 
the TALPA folks for their work, that can also be reasonable for inclusion 
in the kernel (which includes being suitable for other similar purposes, 
such as filesystem indexing)

we agree that the threat they are trying to defend against is a very small 
portion of the overall threat, but there are other components (including 
SELinux) that are available to deal with other portions of the overall 
threat.

if you want to say that the solution is too limited to be worth while, 
then you need to write a new threat model that you think is what should be 
defended against and then we can start discussing how to defend against 
it.

David Lang
--

From: Theodore Tso
Date: Friday, August 15, 2008 - 1:17 pm

For a mail server, I really think something specialized like ClamAV is
a much better solution than something in userspace, which will
probably decide it has to rescan every single file that gets written,
including your mail server logs.   :-)

A specialized solution for a mail server is *always* going to be able
to a more efficient, more practical, and be able to do
application-specialized things (such as refusing the e-mail while the
connection is still open, so you don't have to worry about being RFC
compliant about sending bounce mails when the SMTP return-path is most
likely bogus).

						- Ted
--

From: Press, Jonathan
Date: Friday, August 15, 2008 - 11:17 am

Well, I agree that there are things you can't prevent, that's for sure.
But the point is to build the "threat model" and application
functionality around the idea that IF they happen, you want to be able
to plug the resulting holes as well as you can.  You can't simply close
your eyes to the possibility.


Jon Press
--

From: david
Date: Friday, August 15, 2008 - 1:08 pm

correct, and the threat model that was documented as what we are trying to 
defend against explicity says that actions by root (or other running 
programs, including running malware) are not part of the threat model they 
are trying to address (it doesn't mean that they aren't important, just 
that they are not trying to deal with them). All they are trying to do is 
to have a way to scan files. Since there is use in having support for this 
(for security theater, a little real security, and for other purposes like 
indexing) we are figuing out how to provide tools that would allow this.

David Lang
--

From: Helge Hafting
Date: Monday, August 18, 2008 - 3:02 am

And when new signatures arrive, everything is dirty again. Well, you can 
optimize
and say that everything only need to be checked with the new signatures,
since they passed the old ones already. Maybe that helps performance a 
little.

The "clean" concept works fine, just make sure dirtying happens in all these
extra cases.


Helge Hafting
--

From: tvrtko.ursulin
Date: Friday, August 15, 2008 - 3:44 am

It can never be free - what ever you do it has to happen some time and 
that can and will clash with something else. So your reason why it is 

The first thing you have to make clear is whether in your vision inode 
clean-dirty-unknown status is persistent or not? But in any case I think 
you are making a problem where there isn't one, think about it a bit.

But the idea about the ability to make a clean copy is interesting. So we 
would need a copy done without userspace intervention and preserving the 
inode cache status alongside it. Maybe splice or tee could easily do it? 
Although I am not sure how often copying would happen and whether this 
would be such a gain.

--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos

"Views and opinions expressed in this email are strictly those of the 
author.
 The contents has not been reviewed or approved by Sophos."
 

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

From: tvrtko.ursulin
Date: Thursday, August 14, 2008 - 2:46 am

Do I understand correctly that everyone agrees scanning whenever an inode 
gets dirty would be a terrible thing for performance? 

Another thing we have here is that malware could not be neccessariliy 
identified until the very last write (one example where it will always be 
the case are PDF files (I think)).

So the whole question is at which point should be performing an async 
scan. Close seems like a natural point which should be ideal for majority 
of applications, I don't see how any time-based lumping/delaying scheme 

This really sounds pretty interesting. Not necessariliy so much as a 
performance optimisation, because I am not sure there are so many programs 
where first read comes long after the first open, but as closing the 
open-read race. 

Could the implementation be not so complicated after all? If we generated 
the same (roughly) event on reads and pass it for scanning if cache has 
been invalidated in the mean time? The only thing is this could be a big 
performance hit so some benchmarking might be in order depending on which 
the read hook could be made run-time optional.
 
--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos

"Views and opinions expressed in this email are strictly those of the 
author.
 The contents has not been reviewed or approved by Sophos."
 

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

From: Arjan van de Ven
Date: Thursday, August 14, 2008 - 6:46 am

On Thu, 14 Aug 2008 10:46:55 +0100

close isn't the answer just because you can write content to the file
after that (and that's not just theoretical, glibc stdio supports mmap
etc); "dirty" *has* to be the event anyway. It's not impossible to
solve; even say a 1 second rearming delay would avoid 99.9% of the
useless rescans while still making sure everything gets scanned at some
point. Anyway this kind of policy can be done in userspace (and you can

there's quite a few programs that open() but never read.
open+fstat is not uncommon as programming pattern for example;
with async-in-open (and then wait or sync in read) we wouldn't have the
big hit caused by the latency for the sync scan.
(I realize this pattern is much more likely to happen on posixy systems
than it is on windows.. one of those differences ;-)


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--

From: david
Date: Thursday, August 14, 2008 - 6:37 pm

all you need is the ability to mark a file as 'dirty', and some way for 
programs that are interested in dirty files learning about it later and 
decideing to do a scan. if the file gets dirtied again after they do they 
scan they will need to do another one (this is a classic trade-off between 
the 'security' of looking for things quickly or 'efficiancy' of only 
looking when you don't think they will change again. in other words, 
policy -> userspace)

David Lang
--

From: david
Date: Thursday, August 14, 2008 - 6:31 pm

the kernel should not kick off a scan, instead it should check to see 
an open/read should not kick off a scan, instead it should check to see if 
the scan generation tag(s) are current should be enough (remember, you may 

definantly not the kernel. the intent of this is to keep linux from being 
a storage repository for malware used by other systems. there is no need 
to penalize linux-only apps by making them wait for a scan to take place. 
If it lives in glibc there should be a way for linux apps that know that 
they will not be exporting files to other systems to tell the library not 
to do a scan.

for example, why should a log analysis program looking at apache logs be 
forced to wait while multiple 'virii scanners' go through several gigs of 
logs before it can start looking at them.

you are going to need some way to bypass the checks anyway so that you can 
avoid the recursive case of the scanners triggering scans on files that 
they open.

by keeping the scans all in userspace it also simplifies things greatly. 
All the kernel should do is to maintain the tags with the file (posix 

not a problem, in fact multiple agents scanning in parallel is a good 
thing, it lets them all see the data with one pass through the disk.

they will all need to set different tags anyway (the fact that agent1 

this is not part of the threat model.

David Lang
--

From: Pavel Machek
Date: Friday, August 15, 2008 - 9:06 am

How does it work? Memory can still change after mmap; scanning at the
mmap time is _NOT_ enough.

You could do 'when app attempts to dirty memory, synchronously unmap
it from all apps that have it mapped' and then do sync scan on
pagefault time; but that sounds impractical.
							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--

From: david
Date: Monday, August 18, 2008 - 5:21 am

what is the threat you are trying to defend against?

for some threats you are right, for others the scan at mmap time is 
enough.

David Lang
--

From: Pavel Machek
Date: Monday, August 18, 2008 - 6:30 am

I don't see any threats when check at mmap time is okay.

As soon as file servers use mmap, this race can bite you even in very
simple 'make sure Linux fileserver does not pass on windows malwar'
threat model.
							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--

From: david
Date: Monday, August 18, 2008 - 5:03 pm

if you check the file at the time you mmap it and don't expect other 
programs to change the file under you while you are reading it, there's 
nothing to worry about.

shared mmap with one program being able to write to it is a expecially 
hard case of the general problem that if someone writes to a file as you 
are serving it, you don't know what goes out, it may be part of the old 
file and part of the new file. it's just easier to see this problem with 
mmap and harder to catch it.

David Lang
--

From: Andi Kleen
Date: Wednesday, August 13, 2008 - 11:17 am

On Wed, Aug 13, 2008 at 12:36:15PM -0400, Eric Paris wrote:

I miss a clear answer to the question: is this 
supposed to protect against malware injected as root or not?


But you need some LSM like protections to be able to protect the file

This means you need significant LSM components simply to protect
the integrity of the file scanner against root. It's even 
unclear it's possible in the general case (e.g. X server doing

Ok so you're implying it's ok to not protect against root?

In the later case that means that you don't have to scan anything
that only root can touch and you can trust file permissions,
which makes a lot of things easier.

I would suggest again to clarify this important point first. It has
significant impact on the whole design.

Personally I would think not protecting against root would be quite
limiting (e.g. it would mean that e.g. if some worm trojans rpms
people download then they wouldn't be caught because rpms are
installed as root), but on the other hand if you protect against
root you need most of selinux/aa/other lsm functionality simply
to guarantee the integrity of the scanner.  Also it has impact
on some apps, e.g. X server running as root would be usually out due to
the arbitary DMA issue. Also protect block devices could theoretically
have significant impact on some sysadmin tasks.

-Andi
--

From: Arjan van de Ven
Date: Wednesday, August 13, 2008 - 11:24 am

On Wed, 13 Aug 2008 20:17:14 +0200


on argument could be that root apps like that could/should do explicit
scanning regardless.
(if we have an explicit interface to scan a file that's not too hard)


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--

From: H. Peter Anvin
Date: Wednesday, August 13, 2008 - 11:21 am

Without Treacherous Computing, it isn't possible, even in kernel space.

In Treacherous Computing you can put it in the hypervisor, which of 
course just means the hypervisor is now much bigger and likely to 
contain security holes.

	-hpa
--

From: Eric Paris
Date: Wednesday, August 13, 2008 - 11:40 am

I thought I had summed that up.  We are not interested in providing
protections against maliciously attacking programs be they root or not.
We are interested in scanning files read and written by root.  We are
especially interested in programs run by root.

yum (as root) downloads trojan.rpm from youareanidiot.repo.  We aren't
worried about yum maliciously attacking the system.  What's going to
happen is that the scanner is going to scan the trojan.rpm when yum
calls rpm and rpm is going to be denied access to that file.  Doesn't
matter how you download it, its going to get scanned when you try to
exec it.

Stop thinking this is an LSM or as a new security model.  It's a file
scanner and "it ain't perfect security."  But its very useful and
practical.  If some malicious root application wants to turn it off it
can and I make no claims otherwise.  This isn't supposed to help once
root has been subverted, you've already lost as you admit, its supposed
to help keep root form getting subverted.


--

From: Mihai
Date: Wednesday, August 13, 2008 - 5:18 pm

I honestly don't think we should worry about root. Sure, if the AV scanner 
happens to catch something (as a consequence of it's implementation), then 
very well. But designing an antimalware solution which assumes the root is 
compromised will throw us into security talks for years and I don't think 
we'll live to hear the end of them.

We should focus on the regular users and fix (if needed) the current userland 
apps (ie. the ones that need root access to do their job). For anymore than 

If GPG signatures don't work, then please fix the rpm design and if the user 
willingly installs a .rpm which is not signed (not from a known trusted host) 
and somehow doges the basic antimalware scanner, then too bad. We've done all 

I think we need to define the 'desktop user' and provide a decent protection 
mechanism for his common activities (edit documents, listen music, navigate 
the web, see movies, run scripts which change the IM status etc). For the 
rest, there are two possibilities:
    1. education (_extremely_ important);
    2. SELinux (or similar);

I don't think there will ever be an AV product using the marketing line: "it 
allows you to run your favorite rootkit and enjoy the pretty text it shows, 
with no worries".

In conclusion: everything AV related should stop at the user root. Popular 
distro-s already provide a way to do your daily office tasks without super 
user rights, which _is_ the correct thing to do.

-- 
Mihai Donțu
--

From: Press, Jonathan
Date: Thursday, August 14, 2008 - 4:58 am

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBtYWx3YXJlLWxpc3QtYm91bmNl
c0BkbWVzZy5wcmludGsubmV0IFttYWlsdG86bWFsd2FyZS1saXN0LQ0KPiBib3VuY2VzQGRtZXNn
LnByaW50ay5uZXRdIE9uIEJlaGFsZiBPZiBNaWhhaSBEb24/dQ0KPiBTZW50OiBXZWRuZXNkYXks
IEF1Z3VzdCAxMywgMjAwOCA4OjE4IFBNDQo+IFRvOiBBbmRpIEtsZWVuDQo+IENjOiBwZXRlcnpA
aW5mcmFkZWFkLm9yZzsgbGludXgta2VybmVsQHZnZXIua2VybmVsLm9yZzsgbWFsd2FyZS0NCj4g
bGlzdEBsaXN0cy5wcmludGsubmV0OyBoY2hAaW5mcmFkZWFkLm9yZzsgdmlyb0B6ZW5pdi5saW51
eC5vcmcudWs7DQo+IGFsYW5AbHhvcmd1ay51a3V1Lm9yZy51azsgYXJqYW5AaW5mcmFkZWFkLm9y
Zw0KPiBTdWJqZWN0OiBSZTogW21hbHdhcmUtbGlzdF0gVEFMUEEgLSBhIHRocmVhdCBtb2RlbD8g
d2VsbCBzb3J0YS4NCj4gDQo+IE9uIFdlZG5lc2RheSAxMyBBdWd1c3QgMjAwOCwgQW5kaSBLbGVl
biB3cm90ZToNCj4gPiBPbiBXZWQsIEF1ZyAxMywgMjAwOCBhdCAxMjozNjoxNVBNIC0wNDAwLCBF
cmljIFBhcmlzIHdyb3RlOg0KPiA+DQo+ID4gSSBtaXNzIGEgY2xlYXIgYW5zd2VyIHRvIHRoZSBx
dWVzdGlvbjogaXMgdGhpcw0KPiA+IHN1cHBvc2VkIHRvIHByb3RlY3QgYWdhaW5zdCBtYWx3YXJl
IGluamVjdGVkIGFzIHJvb3Qgb3Igbm90Pw0KPiANCj4gSSBob25lc3RseSBkb24ndCB0aGluayB3
ZSBzaG91bGQgd29ycnkgYWJvdXQgcm9vdC4gU3VyZSwgaWYgdGhlIEFWIHNjYW5uZXINCj4gaGFw
cGVucyB0byBjYXRjaCBzb21ldGhpbmcgKGFzIGEgY29uc2VxdWVuY2Ugb2YgaXQncyBpbXBsZW1l
bnRhdGlvbiksIHRoZW4NCj4gdmVyeSB3ZWxsLiBCdXQgZGVzaWduaW5nIGFuIGFudGltYWx3YXJl
IHNvbHV0aW9uIHdoaWNoIGFzc3VtZXMgdGhlIHJvb3QgaXMNCj4gY29tcHJvbWlzZWQgd2lsbCB0
aHJvdyB1cyBpbnRvIHNlY3VyaXR5IHRhbGtzIGZvciB5ZWFycyBhbmQgSSBkb24ndCB0aGluaw0K
PiB3ZSdsbCBsaXZlIHRvIGhlYXIgdGhlIGVuZCBvZiB0aGVtLg0KPiANCj4gV2Ugc2hvdWxkIGZv
Y3VzIG9uIHRoZSByZWd1bGFyIHVzZXJzIGFuZCBmaXggKGlmIG5lZWRlZCkgdGhlIGN1cnJlbnQg
dXNlcmxhbmQNCj4gYXBwcyAoaWUuIHRoZSBvbmVzIHRoYXQgbmVlZCByb290IGFjY2VzcyB0byBk
byB0aGVpciBqb2IpLiBGb3IgYW55bW9yZSB0aGFuDQo+IHRoYXQgd2UnbGwgbmVlZCBhIHN1cGVy
IHVzZXIgdGhhdCBzdXBlcnZpc2VzIHJvb3QuIEFuZCB0aGVuIGFub3RoZXIgb25lLg0KDQpJIHRo
aW5rIHRoYXQgc29tZSBwZW9wbGUgYXJlIG1pc3NpbmcgdGhlIGltcG9ydGFudCBwb2ludCBvZiBF
cmljJ3MgcmVjZW50IG9yaWdpbmFsIHN0YXRlbWVudCBvZiB0aGUgInRocmVhdCBtb2RlbCIuICBX
aGV0aGVyIHdlIG1vdmUgZnVydGhlciBpbiB0aGUgZGlyZWN0aW9uIG9mIG90aGVyIHNlY3VyaXR5
IHByb3RlY3Rpb25z ...
From: Mihai
Date: Thursday, August 14, 2008 - 5:34 am

I see. Well, as long as everyone sticks to _just_ the file scan. To be honest, 
the only immediate use of the patch that is/was in question, is a "natural" 
scanner for file servers (Samba, NFS etc). 7v5w7go9ub0o, however, might have 
some more ideas. :)

I admit and I apologize, I got pretty worked up when people started asking 
questions like: "how do we protect the file scanner", when the answer should 
have been obvious: the way we protect any other daemon (service) today, by 

-- 
Mihai Donțu
Again, this mail == my own opinion
--

From: 7v5w7go9ub0o
Date: Wednesday, August 13, 2008 - 5:14 pm

(this was posted in linux.kernel, before I realized there was a 
linux.kernel.malware.    Hope it helps your discussion)

(FYI. Dazuko may have trailblazed some of the issues now under
discussion re: libmalware.so. It has worked well for me. It used to be
an LKM, it is now a source patch. It is used in a number of commercial
products)


<http://dazuko.dnsalias.org/wiki/index.php/Main_Page>

"A Virtual Device Driver to Allow Online File Access Control

A common interface is needed, which allows userland applications to
perform online file access control. Dazuko aims to provide that interface."

FWIW, I'm not associated with Dazuko or Antivir; I've been happily using
Dazuko with AntiVir for a year or so.

1. AntiVir includes numerous Linux signatures as well as Windows. So I
scan both 'ix downloads, as well as the process of compiling new software.

2. Other AntiMalwares are using Dazuko, though many are scanning for
Windows malware only.

3. The AntiVir/Dazuko combination with full heuristics has blocked
access to clearly dangerous JS scripts in my browser cache.

4. IMHO, what is needed is a Dazuko or libmalware/Integrity database
link. If an md5 of an executable or script is new or has changed, access
is blocked 'til a response to a popup is given. Access can be blocked;
one-time allowed; or permanently allowed, in which case the md5 is updated.

Hope This Helps.

<next msg>


Andi Kleen wrote:
 > 7v5w7go9ub0o <7v5w7go9ub0o@gmail.com> writes:
 >
 >> (FYI. Dazuko may have trailblazed some of the issues now under
 >> discussion re: libmalware.so. It has worked well for me.
 >
 > Against what exactly did it protect you? Please give a concrete example.
 >
 > -Andi
 >

1. This came in a few minutes ago:

Aug 13 14:56:31 tux antivir[6381]: AntiVir ALERT: [EML/FakeLink.F]
/jail/tbird/root/.thunderbird/0r2957kg.default/Mail/L
ocal Folders/Junk.XXX <<< Contains detection pattern of EML/FakeLink.F
in EML form

2. I have not retained the logs of "suspicious ...
From: 7v5w7go9ub0o
Date: Wednesday, August 13, 2008 - 7:25 pm

I just noticed a separate discussion about integrity-checking LKMs and LSMs.

Obviously, a libmalware.so or Dazuko based integrity-checker would block 
a kernel from loading in a Trojaned LKM - noting that the MD5 had 
changed, and asking you to block, temporarily allow, or permanently 
allow the changed module.

Another security benefit of your pursuit.

HTH







--

Previous thread: invalidate caches before going into suspend by Mark Langsdorf on Wednesday, August 13, 2008 - 9:41 am. (22 messages)

Next thread: [2.6.27-rc3] section mismatches and compilation warnings by Rufus & Azrael on Wednesday, August 13, 2008 - 9:53 am. (1 message)