Date: Sun, 20 Apr 2008 05:14:26 +0100
From: "Oleg Verych"
To: "sed users"
Subject: gsed man pages; custom sed news; `sed` in the wild.

== GNU sed functionality documentation ==

When i've reported some bugs[0] with GNU sed recently, i've screen some
old ones.Very little work required to close all those man-page related stuff.
Many users without `info` journeys can know much more features implemented
there.

[0] http://bugs.debian.org/475464 475474
   Bug#475464: sed: lost cycle in address range with last-line-address operation
   Bug#475474: sed: performance anomaly of /^$/

== custom sed news ==

Making short RE match was outlined in custom sed[1]
,--
|with Perl-style minimal pattern matching:
|           *?   minimal match of *
|           +?   minimal match of +
|           ??   minimal match of ?
`--
I don't know if this was implemented somewhere. Anyway, this approach seem
to be very bad. BRE or ERE syntax is already too twisted, i haven't even see a
reliable highlighting for it. E.g. colored expand [2] didn't do \( \) matched,
GNU Emacs 21, i use, hasn't any so called fontlocking for `sed` or BRE.

What i'm proposing[3] is S/// command, which will have shortest RE match.
I don't know if glibc regexp based implementation of GNU sed can have it, but
ordinary `sed` must have shortest match in context (line) addresses with RE.
To scan more there, is quite waste of time and energy. Thus same short
functionality flag can be used in new S/// command.

[1] http://www.ptug.org/sed/custom_sed.html
[2] http://sed.sourceforge.net/local/scripts/expand.sed.html
[3] http://kernelnewbies.org/olecom/sed-and-sh++#sed

== `sed` in the wild ==

I also don't know if posts like this one are OK for mostly Q&A forums, but let
me share some of my experience and scripts. While i didn't get any feedback
about text processing WRT coding style, this time i want to show some really
simple yet very useful for C programmers stuff.

Real security guys have hard time maintaining and porting big patches to ever
growing number of changes and versions of 2.6 Linux. I've started to look at
PaX patch[4] to see, what i can do for more automated and simple porting of it.

[4] http://kerneltrap.org/node/15996

Any comments are welcome. TIA.

== last, but not least ==
[2] http://sed.sourceforge.net/local/scripts/expand.sed.html

i think, i've made more optimized version of this one[5]. Don't claim,
i'm a guru,
just a small funny experience. Evaluation is welcome.

[5] ftp://flower.upol.cz/crazy/expand.sed.sh
-- 
sed 'sed && sh + olecom = love' << ''
-o--=O`C
 #oo'L O
<___=E M

To: sed users
Date: Wed, Apr 23, 2008 at 12:59 AM

Tim Chase @ Wed, Apr 23, 2008 at 12:39 AM:

> > I am trying to get SED to keep the first three characters in a line,
>  > and discard all others.
>
>  You should be able to use
>
>   sed 's/^\(...\).*/\1/'
>
olecom@flower:$ echo 123uio | sed 's-^\(...\).*-\1-'
123

more funny examples (:

olecom@flower:$ echo 123uio | sed -n 's-...-&\n-;P'
123
olecom@flower:$ echo 123uio | sed 's-...-&\n-;P;d'
123

Which is faster?

,-- man perl:
|The Perl motto is "There's more than one way to do it."  Divining how
|many more is left as an exercise to the reader.
`--

ho-ho-ho! ha-ha-ha!

Yea, i can have many ways to laugh at it. Anyway.

And answer is!

== input ==

root@flower:/tmp# ls -lh /tmp/func.def-0#7-
-rw-r----- 1 olecom root 12M 2008-03-19 02:53 /tmp/func.def-0#7-

== output ==

root@flower:/tmp# time nice -n -19 sed 's-...-&\n-;P;d' /tmp/func.def-0#7- >/dev/null

real 0m0.189s
user 0m0.176s
sys  0m0.008s
root@flower:/tmp# time nice -n -19 sed 's-...-&\n-;P;d' /tmp/func.def-0#7-  >/dev/null

real 0m0.189s
user 0m0.184s
sys  0m0.004s
root@flower:/tmp# time nice -n -19 sed 's-^\(...\).*-\1-' /tmp/func.def-0#7- >/dev/null

real 0m3.086s
user 0m3.072s
sys  0m0.008s
root@flower:/tmp# time nice -n -19 sed 's-^\(...\).*-\1-' /tmp/func.def-0#7- >/dev/null

real 0m3.120s
user 0m3.112s
sys  0m0.004s
root@flower:/tmp#

kid+bit+sed http://mid.gmane.org/20080519104809.GJ21428@flower.upol.cz

and <20080519114731.GK21428@flower.upol.cz>:

Forgot one thing.

> == Let strange quark decay ==
>
> $time sed -n '/:Version: /s`.*/\(.*\):Version: \([^:]*:\)*`\1_`p' <sed.input >here2
>
> real    0m0.284s
> user    0m0.268s
> sys     0m0.020s
> $diff -u here here2 | head
> $
>
> So, nearly one order of magnitude?

$time sed -n '/:Version: /s`.*/\(.*\):Version: \([^:]*:\)*`\1_`p' <sed.input >here2

real    0m0.283s
user    0m0.268s
sys     0m0.012s
$
$time sed -n '/:Version: /s`.*/``p' <sed.input | sed 's`\(.*\):Version: \([^:]*:\)*`\1_`' >here3

real    0m0.142s
user    0m0.188s
sys     0m0.012s
$
$time sed -n '/:Version: /s`.*/``p' <sed.input | sed 's`\(.*\):Version: \([^:]*:\)*`\1_`' >here3

real    0m0.142s
user    0m0.184s
sys     0m0.008s
$
$diff -u here3 here2
$

Linuxthreads in past TODO or yet TO DO NPTL for faster ERE in glibc?

I think, making processes in kernel light, new scheduler, splice(), etc,
etc, is better way, rather than inventing userspace-multi-tasking.

I wonder, who needs it anyway, except big closed enterprise...
____