Kris Kennaway wrote:

quoted text
>

> This is coming along very nicely indeed!

>

> One suggestion I have is that as more metrics are added it becomes

> important for an "at a glance" overview of changes so we can monitor for

> performance improvements and regressions among many workloads.

 >

quoted text
> One way to do this would be a matrix of each metric with its change

> compared to recent samples.  e.g. you could do a student's T comparison

> of today's numbers with those from yesterday, or from a week ago, and

> colour-code those that show a significant deviation from "no change".

> This might be a bit noisy on short timescales, so you could aggregrate

> data into larger bins and compare e.g. moving 1-week aggregates.

> Fluctuations on short timescales won't stand out, but if there is a real

> change then it will show up less than a week later.

I agree that there's a need for an overview and some sort of

notification. I've been collecting historical data to get a baseline for

the statistics and I'll try to see what I can do over the next weeks.
> These significant events could also be graphed themselves and/or a

quoted text
> history log maintained (or automatically annotated on the individual

> graphs) so historical changes can also be pinpointed.

>

> At some point the ability to annotate the data will become important

> (e.g. "We understand the cause of this, it was r1.123 of foo.c, which

> was corrected in r1.124.  The developer responsible has been shot.")

There's a field in the database for this sort of thing. I just think it

needs some sort of authentication. That'll have to wait a bit.
> P.S. If I understand correctly, the float test shows a regression?  The

quoted text
> metric is calculations/second, so higher = better?

The documentation on Unixbench is scarce, but I would think so.
BTW if anyone's interested my SVN repo is online at:
svn://littlebit.dk/website/trunk    (Pylons project)

svn://littlebit.dk/tracker/trunk    (sh/Python scripts for runnning the

server and slaves)
Be careful with your eyes - this is my first attempt at both shell

scripting and Python :-)
Erik

_______________________________________________

freebsd-performance@freebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-performance

To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"