Re: [PATCH] x86: TSC resync

Previous thread: perfmon3 interface overview by stephane eranian on Thursday, September 25, 2008 - 2:48 pm. (2 messages)

Next thread: [PATCH] IPoIB: Fix crash when path record fails after path flush by Roland Dreier on Thursday, September 25, 2008 - 3:28 pm. (1 message)
From: Michael Davidson
Date: Thursday, September 25, 2008 - 3:02 pm

This patch is a slightly cleaned up version of one which has
been in use at for some time now at Google.

It uses an HPET based time source to resynchronize the TSC on
systems where it would otherwise be unsynchronized - eg early
AMD Opteron based systems where the TSC rate drifts when going
in and out of the C1E halt state.

While the approach is quite crude it has been effective for systems
where user space code relies on the TSC advancing at a constant rate
and being reasonably well synchronized between CPUs. The skew between
TSC's on different processors as seen from user space is typically
less than +/- 1000 clock cycles which has proved to be sufficient for
the applications that we care about.

I don't expect this patch to be of much general interest, but if you
happen to be unlucky enough to have a system where the TSC is not
synchronized across CPUs and user space code which relies on the
assumption that it is, then this patch may be useful.

Signed-off-by: Michael Davidson <md@google.com>

---

Index: linux-2.6.26.5/arch/x86/kernel/tsc_resync.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6.26.5/arch/x86/kernel/tsc_resync.c	2008-09-25 12:19:40.826853000 -0700
@@ -0,0 +1,419 @@
+/*
+ *	TSC resynchronization
+ *
+ *	Copyright 2006-2008 Google Inc.
+ *	All Rights Reserved
+ *
+ *	Author: md@google.com
+ */
+#include <linux/init.h>
+#include <linux/smp.h>
+#include <linux/notifier.h>
+
+#include <asm/uaccess.h>
+#include <asm/hpet.h>
+#include <asm/idle.h>
+#include <asm/proto.h>
+
+static int	tsc_resync_enabled = -1;
+static int	tsc_lazy_resync = 0;
+
+/*
+ * TSC is synchronized to a reference clock which is provide by the HPET.
+ * Since the HPET counter is only 32 bits we have to synthesize a 64 bit
+ * value and do periodic updates to deal with 32 bit overflow.
+ */
+static DEFINE_SEQLOCK(ref_clock_lock);
+
+static uint64_t		ref_clock_base;
+static ...
From: Ingo Molnar
Date: Friday, September 26, 2008 - 12:02 am

hm, this patch syncs the TSCs every 20 seconds. That is enough to sync 
up AMD CPUs where the TSC slows down _slightly_ (at 10 ppm per second or 
so) when it's in HLT.

How does it behave in face of 'TSC stops' systems - systems with C2/C3 
sleeps? Basically all modern CPUs that save power are affected by that: 
the TSC get brutally cut when idle - almost all modern power saving 
laptop, desktop and server CPUs.

Also, what does it do in face of cpufreq-affected TSCs? That too is a 
large category of systems. (but most currently shipping CPUs fortunately 

This actually looks pretty interesting IMO, and the code looks simple, 
clean and straightforward enough - but it might not be enough to be a 
generic solution.

	Ingo
--

Previous thread: perfmon3 interface overview by stephane eranian on Thursday, September 25, 2008 - 2:48 pm. (2 messages)

Next thread: [PATCH] IPoIB: Fix crash when path record fails after path flush by Roland Dreier on Thursday, September 25, 2008 - 3:28 pm. (1 message)