Now and again there has been talk on the netdev list of adding PTP support into Linux. One part of the picture is already in place, the SO_TIMESTAMPING API for hardware time stamping. This patch set offers the missing second part needed for complete IEEE 1588 support. * Why all the CCs? 1. IMHO, the patches should go through netdev. 2. A reviewer on netdev said, this should appear on lkml. 3. One driver is for PowerPC, and adds device tree stuff. 4. One driver is for the ARM Xscale IXP465. * Open Issues: ** DP83640 In order to make this work, one line must be added into the MAC driver. If you have the DP83640 and want to try the driver, you need to add this one line to your MAC driver: In the .ndo_start_xmit function, add skb_tx_timestamp(skb). ** IXP465 I do not know how to correctly choose the timestamp "channel" based on the port identifier: +#define PORT2CHANNEL(p) 1 +/* + * PHYSICAL_ID(p->id) ? + * TODO - Figure out correct mapping. + */ Krzysztof, can you help? * PTP Patch ChangeLog ** v5 *** general - Added a hook into the PPS subsystem - Corrected max_adj in all drivers - Removed unnecessary sysfs stuff - Replaced spinlock with mutex in class driver *** gianfar - Added PPS support - Changed underscore to minus in device tree bindings - Use of_iomap instead of ioremap *** ixp465 - Added an external trigger event - Corrected in_progress logic *** phyter - Added an external trigger event - Added support for phy status frames ** v4 *** general - Added a clock driver for the National Semiconductor PHYTER. - Added a clock driver for the Intel IXP465. - Made more stylish according to checkstyle.pl. *** gianfar - Replace device_type and model with compatible string ("fsl,etsec-ptp") - Register only one interrupt, since others are superfluous. - Combine ptp clock instance with private variable structure. - ISR now returns NONE or HANDLED properly. - Print ...
This patch adds an infrastructure for hardware clocks that implement IEEE 1588, the Precision Time Protocol (PTP). A class driver offers a registration method to particular hardware clock drivers. Each clock is exposed to user space as a character device with ioctls that allow tuning of the PTP clock. Signed-off-by: Richard Cochran <richard.cochran@omicron.at> --- Documentation/ptp/ptp.txt | 95 +++++++ Documentation/ptp/testptp.c | 306 ++++++++++++++++++++++ Documentation/ptp/testptp.mk | 33 +++ drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/ptp/Kconfig | 27 ++ drivers/ptp/Makefile | 5 + drivers/ptp/ptp_clock.c | 514 ++++++++++++++++++++++++++++++++++++++ include/linux/Kbuild | 1 + include/linux/ptp_clock.h | 79 ++++++ include/linux/ptp_clock_kernel.h | 137 ++++++++++ 11 files changed, 1200 insertions(+), 0 deletions(-) create mode 100644 Documentation/ptp/ptp.txt create mode 100644 Documentation/ptp/testptp.c create mode 100644 Documentation/ptp/testptp.mk create mode 100644 drivers/ptp/Kconfig create mode 100644 drivers/ptp/Makefile create mode 100644 drivers/ptp/ptp_clock.c create mode 100644 include/linux/ptp_clock.h create mode 100644 include/linux/ptp_clock_kernel.h diff --git a/Documentation/ptp/ptp.txt b/Documentation/ptp/ptp.txt new file mode 100644 index 0000000..46858b3 --- /dev/null +++ b/Documentation/ptp/ptp.txt @@ -0,0 +1,95 @@ + +* PTP infrastructure for Linux + + This patch set introduces support for IEEE 1588 PTP clocks in + Linux. Together with the SO_TIMESTAMPING socket options, this + presents a standardized method for developing PTP user space + programs, synchronizing Linux with external clocks, and using the + ancillary features of PTP hardware clocks. + + A new class driver exports a kernel interface for specific clock + drivers and a user space interface. The infrastructure supports ...
Have you considered integrating the subsystem into the Posix clock/timer framework? I can't really tell from reading the source if this is possible or not, but my feeling is that if it can be done, that would be a much nicer interface. We already have clock_gettime/clock_settime/ timer_settime/... system calls, and while you'd need to add another clockid and some syscalls, my feeling is that it will be more .ioctl is gone, you have to use .unlocked_ioctl and make sure that your ptp_ioctl function can handle being called concurrently. You should also add a .compat_ioctl function, ideally one that points to ptp_ioctl so you don't have to list every command as being compatible. Right now, some commands are incompatible, This data structure is incompatible between 32 and 64 bit user space, so you would need a compat_ioctl() function to convert between the two formats. Better define the structure in a compatible way, avoiding Same problem here, timespec is either 64 or 128 bits long and has different Try avoiding versioned ABIs. It's better to just add new ioctls or syscalls when you need some extra functionality and leave the These are also incompatible. Arnd --
You are not the first person to ask about this. See this link for longer explanation of why I did not go that way: http://marc.info/?l=linux-netdev&m=127669810232201&w=2 You *could* offer the PTP clock as a Linux clock source/event device, and I agree that it would be nicer. However, the problem is, what do you do with the PHY based clocks? Just one 16 bit read from a PHY clock can take 40 usec, and you need four such read operations just to get the current time value. Also, I really did not want to add or change any syscalls. I could not see a practical way to extend the existing syscalls to accommodate PTP clocks. Richard --
Why does it matter how long it takes to read the clock? I wasn't thinking of replacing the system clock with this, just exposing the additional clock as a new clockid_t value that can be accessed using the existing Why did you not want to add syscalls? Adding ioctls instead of syscalls does not make the interface better, just less visible. Out of the ioctl commands you define, we already seem to have half or more: PTP_CLOCK_APIVERS -> not needed PTP_CLOCK_ADJFREQ -> new clock_adjfreq PTP_CLOCK_ADJTIME -> new clock_adjtime PTP_CLOCK_GETTIME -> clock_gettime PTP_CLOCK_SETTIME -> clock_settime PTP_CLOCK_GETCAPS -> new clock_getcaps PTP_CLOCK_GETTIMER -> timer_gettime PTP_CLOCK_SETTIMER -> timer_create/timer_settime PTP_FEATURE_REQUEST -> possibly clock_feature Arnd --
Okay, now I see. You are suggesting this:
clock_gettime(CLOCK_PTP, &ts);
clock_settime(CLOCK_PTP, &ts);
I like this. If there is agreement about it, I am happy to implement
I bet that, had I posted patch set with new syscalls, someone would
have said, "You are adding new syscalls. Can't you just use a char
device instead!"
If you add syscalls and introduce CLOCK_PTP, then you add it to
everyone's kernel, even those people who never heard of PTP. A char
device has the advantage that can it be simply ignored. Also, a
syscall has got to have the right form from the very beginning. If the
next generation of PTP hardware looks very different, then it is not
that much of a crime to change an ioctl interface, provided it has
versioning.
Richard
--
Very possible, but after considering both options, I think we would It's a matter of perspective whether you consider this an advantage or disadvantage. I would expect that since you are trying to get support for PTP into the kernel, you'd be happy for people to know about it and No, that's just a myth. The rules for ABI stability are pretty much the same. We try hard to avoid ever changing an existing ABI, for both syscalls and ioctl. In either case, if you get it wrong, you have to support the old interface and create a new syscall or ioctl command. As mentioned, versioning does not solve this, it just adds another indirection (which we try to avoid). One difference is that more people take a look at your code when you suggest a new syscall, so the chances of getting it right in the first upstream version are higher. Another difference is that we generally use ioctl for devices that can be enumerated, while syscalls are for system services that are not tied to a specific device. This argument works both ways for PTP IMHO: On the one hand you want to have a reliable clock that you can use without knowing where it comes from, on the other you might have multiple PTP sources that you need to differentiate. Arnd --
Yes, I agree. In normal use, there will be only one PTP clock in a system. However, for research purposes, it would be nice to have more than one. I've been looking at offering the PTP clock as a posix clock, and it is not as hard as I first thought. The PTP clock or clocks just have to be registered as one of the posix_clocks[MAX_CLOCKS] in posix-timers.c. My suggestion would be to reserve three clock ids in time.h, CLOCK_PTP0, CLOCK_PTP1, and CLOCK_PTP2. The first one would be the same as CLOCK_REALTIME, for SW timestamping, and the other two would allow two different PTP clocks at the same time, for the research use case. Using the clock id will bring another advantage, since it will then be possible for user space to specify the desired timestamp source for SO_TIMESTAMPING. Richard --
I don't think there is a point in making exactly two independent sources available. The clockid_t space is not really limited, so we could define an arbitrary range of ids for ptp sources that could be used simultaneously, as long as we have space more more ids with a fixed number. Would it be reasonable to assume that on a machine with a large number of NICs, you'd want a separate ptp source for each of them for timestamping? Or would you preferably define just one source in such a setup? I think both could be done with the use of class device attributes in sysfs for configuration. Maybe you can have one CLOCK_PTP value for one global PTP source and use sysfs to configure which device that is. If you also need simultaneous access to the specific clocks, you could have run-time configured clockid numbers in a sysfs attribute for each God point. Arnd --
Oh no. I'm starting to waffle here. :) So while I'm not a fan of the duplication of the posix clocks interface that the proposed chardev interface introduced, I'm not sure if absorbing the PTP clocks as a clockid is much better. Mainly due to the fact that userland apps now have to chose between two clockids that supposedly represent the same thing (the current wall Why would you have CLOCK_PTP0 == CLOCK_REALTIME? Whats the point of So how exactly would one map CLOCK_PTPx to an eth device? So this is a little bit further out there, but assuming PTPd can sync the PTP clock correctly, why could the kernel itself not sync the PTP clock to system time? So instead of the sync path looking like: 1) packet from master arrives on NIC, is timestamped by PTP clock 2) PTPd calculates offset between PTP clock and master time 3) PTPd corrects PTP clock freq/offset 4) PTPd corrects system time freq/offset It would look like: 1) packet from master arrives on NIC, is timestamped by PTP clock 2) PTPd calculates offset between PTP clock and master time 3) PTPd corrects system time freq/offset 4) kernel corrects PTP clock freq/offset I'm guessing the indirect PTP clock sync would have greater error, but it avoids the fractured sense of time. thanks -john --
I've been working turning the PTP stuff into syscalls, but here is a little issue I ran into. The PTP clock layer wants to call the PPS code via pps_register_source(), but the PPS can be compiled as a module. Since the PTP layer is now offering syscalls, it must always be present in the kernel. So I need to make sure that the PPS is either absent entirely or staticly linked in. How can I do this? Thanks, Richard --
You might want to use callbacks for these system calls that you can register to at module load time, like it is done for the existing syscalls. The simpler way (e.g. for testing) is using Kconfig dependencies, like config PTP bool "IEEE 1588 Precision Time Protocol" config PPS tristate "Pulse per Second" depends on PTP || !PTP The depends statement is a way of expressing that when PTP is enabled, PPS cannot be a module, while it may be a module if PTP is disabled. Arnd --
THis did not work for me. What I got was, in effect, two independent booleans. Thanks, Richard --
The struct k_clock contains callback functions to clock source specific implementations of the syscalls and other functions. Adding a new clock source that is (potentially) implemented as a module means you have to define a new k_clock in that module that you register using register_posix_clock. If you need additional syscalls to be implemented by the same module, you can put more callback functions into struct k_clock Hmm, right. I wonder what I was thinking of then. You can certainly do config PTP bool "IEEE 1588 Precision Time Protocol" depends on PPS != m but that will hide the PTP option if PPS is set to 'm', which is normally not what you want. Arnd --
Arnd, Perhaps you were thinking of the vhost example (taken from drivers/vhost/Kconfig): config VHOST_NET tristate "Host kernel accelerator for virtio net (EXPERIMENTAL)" depends on NET && EVENTFD && (TUN || !TUN) && (MACVTAP || !MACVTAP) && EXPERIMENTAL They have a similar construct with both TUN and MACVTAP there. Perhaps the parens are a necessary part of the "X || !X" syntax? Just a random guess. Ira --
Yes, that's the one I was thinking of. My mistake was that the effect is slightly different here. VHOST and TUN are both tristate. What we guarantee here is that if TUN is "m", VHOST cannot be "y", because its dependency cannot be fulfilled for y. Arnd --
On Mon, Aug 16, 2010 at 4:17 AM, Richard Cochran Hey Richard! Its very cool to see this work on lkml! I'm excited to see more work done on ptp. We had a short private thread discussion earlier (I got busy and never replied to your last message, my apologies!), but I wanted to bring up the concerns I have here as well. As I mentioned earlier, I'm not a huge fan of the char device interface for abstracted PTP clocks. If it was just the direct hardware access, similar to RTC, which user apps then use as a timesource, I'd not have much of a problem. But as I mentioned in an earlier private mail, the abstraction level concerns me. 1) The driver-like model exposes a char dev for each clock, which allows for poorly-written userland applications to hit portability issues (ie: /dev/hpet vs /dev/rtc). Granted this isn't a huge flaw, but good APIs should be hard to get wrong. 2) As Arnd already mentioned, the chardev interface seems to duplicate the clock_gettime/settime() and adjtimex() interfaces. 3) I'm not sure I see the benefit of being able to have multiple frequency corrected time domains. In other words, what benefit would you get from adjusting a PTP clock's frequency instead of just adjusting the system's time freq? Having the PTP time as a reference to correct the system time seems reasonable, but I'm not sure I see why userland would want to adjust the PTP clock's freq. thanks -john --
And maybe just to clarify, as I saw your response to Arnd, I'm not suggesting using PTP clocks as clocksources for the internal timekeeping core. Instead I'm trying to understand why PTP clocks need the equivalent of the existing posix clocks/timer interface. Why would only having a read-time interface not suffice? thanks -john --
For PTP enabled hardware, the timestamp on the network packet comes from from the PTP clock, not from the system time. Of course, you can always just leave the PTP clock alone, figure the needed correction, and apply it whenever needed. But this has some disadvantages. First of all, the (one and only) open source PTPd does not do it that way. Also, only one program (the PTPd or equivalent) will know the correct time. Other programs will not be able to ask the operating system for time services. Instead, they would need to use IPC to the PTPd. The PTP protocol (and some PTP hardware) offers a "one step" method, where the timestamps are inserted by the hardware on the fly. Here you really do need the PTP clock to be correctly adjusted. All of the PTP hardware that I am familiar with provides a clock adjustment method, so it simpler and cleaner just to use this facility to tune the PTP clock. Richard --
Why would system time not be adjusted to the PTP time? This is my main concern, that we're presenting a fractured API to userland. Suddenly there isn't just system time, but ptp time as well, and possibly multiple different ptp times. Hmm. So trying to recap here to see if I'm understanding properly: The PTP clock is a bit of hardware (usually on the NIC) that can put timestamps on packets (both incoming or outgoing?). The need to offset/freq correct the PTP clock is important so that the timestamps (incoming and outgoing) are correct, which makes future offset calculations simpler. Hmm. So I'm actually starting to come around to the chardev interface. In a way, it seems it has some similarities to how the RTC device is used in NTPd. Granted, NTPd doesn't correct the RTC (the kernel does when we're synced, but its not a perfect parallel), but it can be used as an input the steer the clock. So while to me, it think it would be more ideal (or maybe just less different) to have a read-only interface (like the RTC), leaving PTPd to manage offset calculations and use that to steer the system time. I can acknowledge the need to have some way to correct the freq so the packet timestamps are corrected. I still feel a little concerned over the timer/alarm related interfaces. Could you explain why the alarm interface is necessary? So really I think my initial negative gut reaction to this was mostly out of the fact that you introduced a char dev that provides almost 100% coverage of the posix-time interface. That is duplication we definitely don't want. Also I think the documentation I've read about PTP (likely just due to the engineering focus) has an odd inverted sense of priority, focusing on keeping obscure hardware clocks on NIC cards in sync, rather then the the more tangible feature of keeping the system time in sync. This could be comically interpreted as trying to create a shadow-time on the system that is the "real time" and "yea, maybe we'll let the ...
John, it is a good thing to make thoughts about the big picture with PTP clocks and the system time, like you are doing. However, the Not only on the NIC. There are bunch of new products doing the timestamping in the PHY or in a switch fabric attached to the host like a PHY. The synchronization that one can achieve with PHY The PTPd need not change the system time at all for PTP clock to be The timer/alarm stuff is "ancillary" and is not at all necessary. It is just a "nice to have." I will happily remove it, if it is too The reason why I modelled the char device on the posix interface was to make the API more familiar to application programmers. After the recent discussion (and having reviewed the posix clock implementation in Linux), I now think it would be even better to simply offer a new posic clock ID for PTP. You are right. As John Eidson's excellent book points out, modern computers and operating systems provide surprisingly little support for programming based on absolute time. It is not PTP's fault. PTP is actually a step in the right direction, but it doesn't yet really fit in to the present computing world. Okay, here is the Big Picture. 1. Use Case: SW timestamping PTP with software timestamping (ie without special hardware) can acheive synchronization within a few dozen microseconds, after about twenty minutes. This is sufficient for very many people. The new API (whether char device or syscall) fully and simply supports this use case. When the PTPd adjusts the PTP clock, it is actually adjusting the system time, just like NTPd. 2. Use Case: HW timestamping for industrial control PTP with hardware timestamping can acheive synchronization within 100 nanoseconds after one minute. If you want to do something with your wonderfully synchronization PTP clock, it must have some kind of special hardware, like timestamping external signals or generating one-shot or periodic outputs. The new API (whether char ...
Right, obviously an ok-solution is often more useful then no-solution. But that doesn't mean we shouldn't shoot for a good or even If there's a compelling argument for it, I'm interested to hear. But again, it seems like just yet-another-way-to-get-alarm/timer-functionality, so before we add an extra API (or widen an existing API) I'd like to understand the need. But maybe it might simplify the discussion to pull it for now, but I'm definitely interested to see what you come up with here. I'm still hesitant with adding a PTP clock_id, but extending the posix-clocks interface in this way isn't unprecedented (see: CLOCK_SGI_CYCLE) I just would like to make sure we don't end up with a clock_id namespace littered with oddball clocks that were not well abstracted (see: CLOCK_SGI_CYCLE :). For instance: imagine if instead of keeping the clocksource abstraction internal to the timekeeping core, we exposed each clocksource to userland via a clock_id. Every arch would have different ids, and each arch might have multiple ids. Programming against that would be a huge pain. So in thinking about this, try to focus on what the new clock_id provides that the other existing clockids do not? Are they at comparable levels of abstraction? 15 years from now, are folks likely to still be You'll have to forgive me, as I haven't had the time to check out that book. What exactly do you mean by operating systems provide little Again this illustrates the inversion of focus: system time is merely one of many possible PTP clocks in the larger PTP framework. The way I tend to see it: PTP is just one of the many ways to sync These specialized applications are part of what concerns me the most. For example, I can see some parallels between things like audio processing, where you have a buffer consumed by the card at a certain rate. Now, the card has its own crystal it uses to time its consumption, so it has its own time domain, and could drift from system time. Thus you want to ...
We don't really need it, IMHO.
But if we offer clockid_t CLOCK_PTP, then we get timer_settime()
Arnd convinced me that clockid_t=CLOCK_PTP is a good fit. My plan
would be to introduce just one additional syscall:
SYSCALL_DEFINE3(clock_adjtime, const clockid_t, clkid,
int, ppb, struct timespec __user *, ts)
ppb - desired frequency adjustment in parts per billion
ts - desired time step (or jump) in <sec,nsec> to correct
a measured offset
Arguably, this syscall might be useful for other clocks, too.
I think the ancillary features from PTP hardware clocks should be made
available throught the sysfs. A syscall for these would end up very
ugly, looking like an ioctl. Also, it is hard to see how these
features relate to the more general idea of the clockid.
In contrast, sysfs attributes will fit the need nicely:
1. enable or disable pps
2. enable or disable external timestamps
3. read out external timestamp
PTP was not invented to just to get a computer's system time in the
ball park. For that, NTP is good enough. Rather, some people want to
use their computers for tasks that require close synchronization, like
industrial control, audio/video streaming, and many others.
This is a good example of the poverty (in regards to time
synchronization) of our current systems.
Lets say I want to build a surround sound audio system, using a set of
distributed computers, each host connected to one speaker. How I can
be sure that the samples in one channel (ie one host) pass through the
The clock and its adjustment have nothing to do with a network
socket. The current PTP hacks floating around all add private ioctls
Okay, then will you support an elegant solution for case 3, that also
Thanks for your comments!
Richard
--
My point was that a syscall is better than an ioctl based interface here, which I definitely still think. Given that John knows much more about clocks than I do, we still need to get agreement on the question that he raised, which is whether we actually need to expose this clock to the user or not. If we can find a way to sync system time accurate enough with PTP and This is a mix of adjtime and adjtimex with the addition of the clkid parameter, right? Have you considered passing a struct timex instead of ppb and ts? Is using ppb instead of the timex ppm required to get the accuracy you want? Arnd --
At the very least, one user application (the PTPd) needs to see the Sort of, but not really. ADJTIME(3) takes an offset and slowly corrects the clock using a servo in the kernel, over hours. For this function, the offset passed in the 'ts' parameter will be immediately corrected, by jumping to the new time. This reflects the way that PTP works. After the first few samples, the PTPd has an estimate of the offset to the master and the rate difference. The PTPd can proceed in one of two ways. 1. If resetting the clock is not desired, then the clock is set to the maximum adjustment (in the right direction) until the clock time is close to the master's time. 2. The estimated offset is added to the current time, resulting in a jump in time. That is one very good reason. Another is this: can you explain what the 20+ fields mean? Consider the field, freq. The comment says "frequency offset (scaled ppm)." To what is it scaled? The only way I know of to find out is to read the NTP code (which is fairly complex) and see what the unit really is meant to be. Ditto for the other fields. The timex structure reveals, AFAICT, the inner workings of the kernel clock servo. For PTP, we don't need or want the kernel servo. The PTPd has its own clock servo in user space. Richard --
Could you expand on this? Could we not add a adjustment mode ADJ_SETOFFSET or something that would You're right that the timex is a little crufty. But its legacy that we will support indefinitely. So following the established interface helps maintainability. So if the clock_adjtime interface is needed, it would seem best for it to be generic enough to support not only PTP, but also the NTP kernel PLL. In effect, this would make clock_adjtime(or clock_adjtimex) identical to adjtimex, but only applicable to different CLOCK_ids. Additionally, it would simplify the code for the CLOCK_REALTIME case as you could just call directly into do_adjtimex(). Of course, extending adjtimex for ADJ_SETOFFSET would be needed first, but that seems like a reasonable expansion of the interface that could be used by more then just PTP. thanks -john --
We need to able to specify that the call is for a PTP clock. We could add that to the modes flag, like this: /*timex.h*/ #define ADJ_PTP_0 0x10000 #define ADJ_PTP_1 0x20000 #define ADJ_PTP_2 0x30000 #define ADJ_PTP_3 0x40000 Yes, but we would also need to add a struct timespec to the struct timex, in order to get nanosecond resolution. I think it would be We can use it for PTP, with the modifications suggested above. Or we For the proposed clock_adjime, what else is needed to support clock adjustment in general? I don't mind making the interface generic enough to support any (realistic) conceivable clock adjustment scheme, but beyond the present PTP hardware clocks, I don't know what else might be needed. Richard --
My suggestion was actually to have a new syscall with the existing structure, and pass a clockid_t value to it, similar to your Yes, that's exactly what the padding is for. Instead of timespec, you can probably have a extra values for replacing the existing ppm values with ppb values. Arnd --
Right, although the ppm/ppb issue shouldn't be a problem as the timex allows for much finer then ppb resolution changes. The only adjustment to the adjtimex/timex interface that may be needed is the ability to set the time by an offset (ie: ADJ_SETOFFSET), rather then slewing the offset in (ADJ_OFFSET, or ADJ_OFFSET_SINGLESHOT). This avoids the calc offset, gettime(&now), settime(now+offset) method where any latency between the gettime and settime adds to the error. thanks -john thanks -john --
Multiple PLLs, at least with containers and certain classes of system you want different containers in different timespaces, especially when doing high precision stuff where you need your system tracking say a local master clock for syncing musical instruments and sound events while Put the clock type in the new fields. It becomes u16 clock_type; [clock type specific data] saves having to guess. Alan --
I wasn't suggesting adding the clock multiplexing to the timex, just using the timex to specify the adjustments in the clock_adjtime call. So I was asking why a timex was not suitable instead of using just the The existing struct timeval in the timex can be also used as a timespec. Again, I think you misunderstood my suggestion. I was suggesting I think using the timex struct covers most of the existing knowledge of what is needed. thanks -john --
Sorry for the slow response here, I got busy with other work at the end of last week. Sure. There are some clear parallels and the API seems to match, but what I'm asking is: does it make sense from an overall API view that Sure, but are they conceptually neutral? There are other clock synchronization algorithms out there. Will they need their own similar-but-different clock_ids? Look at the other clock ids and what the represent: CLOCK_REALTIME : Wall time (possibly freq/offset corrected) CLOCK_MONOTONIC: Monotonic time (possibly freq corrected). CLOCK_PROCESS_CPUTIME_ID: Process cpu time. CLOCK_THREAD_CPUTIME_ID: Thread cpu time. CLOCK_MONOTONIC_RAW: Non freq corrected monotonic time. CLOCK_REALTIME_COARSE: Tick granular wall time (filesystem timestamp) CLOCK_MONOTONIC_COARSE: Tick granular monotonic time. CLOCK_PTP that you're proposing doesn't seem to be at the same level of abstraction. I'm not saying that this isn't the right place for it, but can we take a step back from PTP and consider what your exposing in more generic terms. In other words, could someone use the same packet-timestamping hardware to implement a different non-PTP time synchronization algorithm? Further, if we're using PTP to synchoronize the system time, then there shouldn't be any measurable difference between CLOCK_PTP and So yea, obviously the syscall should not be CLOCK_PTP specific, so we would want it to be usable against CLOCK_REALTIME. That said, the clock_adjtime your proposing does not seem to be sufficient for usage by NTPd. So this suggests that it is not generic This may be a good approach, but be aware that adding stuff to sysfs Things to consider here: Do having these options really make sense? Why would we want pps disabled? And if that does make sense, would it be better to do so via the existing pps interface instead of adding a new ptp specific one? Same for the timestamps and periodic output (ie: and how do they differ Of course not! Just because I'm ...
Just my 2 cents on this issue. PTP is very often used in embedded systems, where the PTP timestamps will go into some dedicated hardware, for instance an FPGA. I'm currently working on a project where it is not necessary to adjust the Linux system time via PTP (although it would be a nice to have), but we only need the timestamps from the PHY to put them into our FPGA device. So we need some kind of API to access the PTP timestamp registers. Kind regards, Stephan
Wait.. I thought we weren't using PTP to steer the clock? But now we're using the pps signal from it to do so? Do I misunderstand you? Or did I assume here you mean PTPd is steering the PTP clock according to the system time (which is NTP/GPS/whatever sourced)? And then the PTP clock So first of all, thanks for the extra explanation and context here! I really appreciate it, as I'm not familiar with all the hardware details and possible use cases, but I'm trying to learn. So in the two cases you mention, the time "flow" is something like: #1) [Master Clock on Network1] => [PTP Clock] => [PTPd] => [PTP Clock] => [PTP Clients on Network2] #2) [GPS] => [NTPd] => [System Time] => [PTPd] => [PTP clock] => [PTP clients on Network] And the original case: #3) [Master Clock on Network] => [PTP clock] => [PTPd] => [PTP clock] With a secondary control flow: [PPS signal from PTP clock] => [NTPd] => [System Time] Right? So, just brainstorming here, I guess the question I'm trying to figure out here, is can the "System Time" and "PTP clock" be merged/globbed into a single "Time" interface from the userspace point of view? In other words, if internal to the kernel, the PTP clock was always synced to the system time, couldn't the flow look something like: #3') [Master clock on network] => [PTP clock] => [PTPd] => [System Time] => [in-kernel sync thread] => [PTP clock] So PTPd sees the offset adjustment from the PTP clock, and then feeds that offset correction right into (a possibly enhanced) adjtimex. The kernel would then immediately steer the PTP clock by the same amount to keep it in sync with system time (along with a periodic offset/freq correction step to deal with crystal drift). Similarly: #2') [GPS] => [NTPd] => [System Time] => [in-kernel sync thread] => [PTP clock] => [PTP clients on Network] and #1') [Master Clock on Network1] => [PTP Clock] => [PTPd] => [System Time] => [in-kernel sync thread] => [PTP Clock] => [PTP Clients ...
The master node in a PTP network probably takes its time from a
precise external time source, like GPS. The GPS provides a 1 PPS
directly to the PTP clock hardware, which latches the PTP hardware
clock time on the PPS edge. This provides one sample as input to a
clock servo (in the PTPd) that, in turn, regulates the PTP clock
Yes, but in this case, "system time" has nothing to do with the Linux
system time. For a PTP master clock, it really doesn't matter whether
the Linux time is correct, or not. It doesn't hurt either (but see
I would only draw the PTP clock once, perhaps like this:
+------------------------------+
| ^
V |
[Master Clock on NW 1]--->[HW timestamp]--->[PTPd]--adj-->[PTP Clock]
Nope. More like this:
+------------------------------+
| ^
V |
[GPS]----------PPS--------->[Latch timestamp]--->[PTPd]--adj-->[PTP Clock]
More like:
This is the core issue and source of misunderstanding, in my view. The
fact of the matter is, the current generation of computers has
multiple clocks, and these are usually unsynchronized. I think we
should not try too hard to cover up or work around this. It is a fact
of life.
It would be nice if there were only one clock, and that clock could do
everything that we need. Indeed, the next generation of SoC computers may
all have PTP build in to the main CPU. Well, one can always wish.
If we can make it appear that multiple clocks are just one clock, then
I agree that we should do it. But I would not want to sacrifice
synchronization accuracy or application features just to keep that
I don't like it. The experience with PTP boundary clocks already shows
that the errors in servo loops cascade. It ...In this case I don't think you can. Their divergence is rather difficult to handle unless you have a GPS to hand. But all this talk of "PTP this" and "PTP that" is not helpful. Any interface for additional time sources should be generic with PTP being one use case. Alan --
But TAI and UTC progress at the same rate, and UTC differs from TAI by a constant offset. In fact, the needed conversion is provided by the protocol, so it is not hard to take a 1 PPS from GPS and set the PTP To tell the truth, my original motivation for the patch set was to support PTP clocks and applications. I don't think that is such a bad idea. After all, the adjtimex interface was added just to support NTP. At the same time, I can understand the desire to have a generic hardware clock adjustment API. Let me see if I can understand and summarize what people are asking for: clock_adjtime(clockid_t id, struct timex *t); and struct timex gets some new fields at the end. Using the call, NTPd can call clock_adjtime(CLOCK_REALTIME) and PTPd can call clock_realtime(CLOCK_PTP) and everyone is happy, no? Thanks, Richard --
For a new syscall you could equally make it If you only have one clock that you are calling 'the PTP clock' - but is that a good assumption ? I agree with your fundamental arguments as I understand them - That it's another clock or clocks possibly not synchronized with the system clock - That there should be a sensible API for doing slews and steps on other clocks but the systen clock. I'm concerned about the assumption that there is a single magic PTP clock, and calling it a PTP clock for two reasons - There can be more than one - PTP is just a protocol, in five years time it might be TICTOC or something newer and more wonderous, in some environments it'll be a synchronous distributed clock generation not PTP etc. Wiring PTP or IEE1588v2 into the clock name doesn't make sense. I'd be happier with a model which says we have some arbitary number of synchronization sources which may or may not have a connection to system time, and may be using all sorts of synchronization methods. Clock in fact seems almost a misnomer. Alan --
In message: <20100827140205.GA3293@riccoc20.at.omicron.at> Except for leap seconds, this is true. However, Unix time isn't UTC either. Unix time is UTC that pretends leap seconds just don't exist. POSIX enshrined this long ago, and nobody is going to change that any time soon. I don't believe IEEEv2 propagates leap seconds, does it? Warner --
Not necessarily. AFAIK, the time distributed by IEEE1588v2 can either be based on the "PTP epoch" (timePropertiesDS.ptpTimescale=TRUE) and thus represent TAI or be based on an implementation specific arbitrary epoch (timePropertiesDS.ptpTimescale=FALSE) and represent time on some arbitrary time scale. Christian --
The amount of time a thread has been granted by the kernel is really not connected to the real passage of time, at least not in a direct This one comes from commit 2d42244ae71d6c7b0884b5664cf2eda30fb2ae68 and is surely a special case, unrelated to the other clock ids. The commit message mentions that this was added to help the btime.sf.net project. That project does not seem to have had any activity since 2007. If we can justify adding a clock id in this case, surely we can These were added in commit da15cfdae03351c689736f8d142618592e3cebc3 Yes, of course. There is nothing at all in the patch set about the PTP protocol itself. It just lets you access the hardware. In short: 1. SO_TIMESTAMPING delivers timestamped packets 2. the PTP API lets you tune the clock. When using software timestamping, then the clocks are one in the same. When using PTP, with the PPS hook to synchoronize the Linux system time, the clocks will be a close as the servo algorithm provides. I have not measured this yet, but it cannot be much different than using I don't think we need to support ntpd. It already has adjtimex, and it Yes, it will be properly documented and maintained. I have already implemented the ancillary stuff in two ways, via sysfs and with a character device. The next patch set will include them both, and you Yes, since they represent the PTP clock's hardware features. As I explained previously, if you don't have any hardware interfaces, then having your clocks synchoronized to under 100 nanoseconds does not If you are a master clock, then you want to take your PPS from an external time source, like GPS. If you leave the PTP PPS events on, then they will occur close in time to the GPS PPS events and may add The posix timer calls won't work: I have a PTP hardware clocks with multiple external timestamp channels. Using timer_gettime, how can I specify (or decode) the My point was this: The application requires that the soundcard DA clocks (*not* the ...
But PTP isn't really a clock - its a time sync protocol. You can (and may need to) have multiple clocks of this form on the same host because it's PTP is not a clock, it's many clocks so a clock id doesn't really work. You could assume a single time domain and add a CLOCK_TAI plus then use PTP to track it I guess ? The question then is who would consume it and how ? Generic applications want POSIX time, which is managed by NTP but could in userspace also be slewed via the existing API to track a PTP source if someone wanted and if there is a GPS source around they can compute UTC from it. Specialist applications will presumably need to know which time source or sources they are tracking and synchronizing too out of multiple potential PTP sources Kernel stuff is more of a problem. I'm not sure shoehorning a source of many clocks and time sync bases into a jump up and down and make it fit single time assumption is wise. Making system time bimble track a source makes sense just as with NTP but making it a new clock seems the wrong model extending a non-too-bright API when you can just put the time sources in a file tree. Alan --
Okay, I really meant "for PTP hardware clocks". In general the discussion is about supporting a kind of hardware and not about the PTP network protocol. In fact, the hardware clocks and clock servo loops are not at all part of the IEEE 1588 standard. Sorry for causing confusion, but please understand "a hardware clock with timestamping capabilities than can be used for PTP support" whenever I wrote "PTP" or "PTP clock." Well, what I just said is not entirely true. In fact, most of the current crop of PTP hardware clocks operate by recognizing PTP network frames and timestamping them. One clock I know of can timestamp every frame, but that seems to be the exception rather than the rule. So, in theory they are just clocks, but actually Yes, and even without a GPS, the PTP protocol (this time I really do Don't get your meaning here, what did you mean by "file tree?" Thanks, Richard --
Which seems fine to me too - its an implementation detail of that time Not only that but consumers of different time synchronizations will need to be able to describe which time source they want to talk about from a Something like /sys/class/timesource/<name>/... at which point we don't have to enumerate them all, add special system calls and then fret about the fact you can't access them from things like shell scripts. The fact SYS5.4 Unix and SuS got obsessed with numbering things rather than using names unlike Unix doesn't mean it's the right model to number them - usually the reverse is true, a heirarchy of file names is rather more future proof. Alan --
Hi! I'd like to add my two cents about the discussion. Just to shortly introduce myself: I'm working with PTP since version 2002 (now 2008 or PTPv2) and I'm developing matching network cards, drivers, and also sometimes a bit of the stack. I always had the problem of different HW implementations (even my own) and how to access the clocks there. So reading this thread, I strongly support the idea to provide a driver class, which allows the userspace to run certain standard operations (settime, gettime, adjtime, ...) as proposed and leave the detailed implementation to a specific PTP clock driver. I always made my own char device, but I don't want to open this discussion again as for me it doesn't matter. Concerning the long discussion about PTP clock, system time, etc. I'd like to say, that from a point of view of PTP every node/host has only one clock. That means, that if you have a NIC with integrated clock (required for HW timestamping) it is counted as a node. As soon as you want to use multiple NICs this is actually outside the PTP protocol definition, unless you have only one clock available to both network interfaces (which is hard to achieve). So the solution to treat a PTP enabled NIC as a time source for the system time is in general sufficient, unless for applications that require precise timestamps in applications. I know of use cases where code gets instrumented to measure time delays, processing time, and sequence in SW, e.g. distributed databases for trading systems at the stock exchange. Summarizing I think it would be a huge step forward, if all the different HW implementations could be controlled via a standardised interface through the kernel. I need timestamps from my network card with ps resolution and to steer the onboard clock from user space. Then I would be happy already. :-) Thanks, Patrick --
I'm not opposed to adding a clock id, I just want it to be generic enough to make sense. So while MONOTONIC_RAW it was spurred into being from btime, it isn't a CLOCK_BTIME interface. I've also been pushing the RADClock folks to use it since it avoids the ugly raw hardware access they want to get access to a constant freq counter. In addition, I've used it to monitor freq adjustments done by Right, but what they represent and how its different from CLOCK_REALTIME Right. So CLOCK_SO_TIMESTAMP is maybe a better description? And isn't it So this is the duplication I don't like. If the API represents CLOCK_PTP or whatever as something different from CLOCK_REALTIME, there shouldn't be a mode where they're the same. That would just cause confusion. Instead CLOCK_PTP calls just return -EINVAL and the PTPd application I strongly disagree. If the new interface isn't generic enough that NTPd couldn't use it to adjust CLOCK_REALTIME, then its too specific to only your needs. There will be other clock sync algorithms down the road, so if we have to expose this sort of timestamping hardware to userland, we need to allow those other sync methods to use the API you're providing, so if you're API isn't a superset of the existing adjustment needs, then its already missing something. I'm not saying we have to implement the kernel PLL adjustment and every other adjustment mode for every CLOCK_ID right now, but we should be Doesn't the pps subsystem have its own way to control the pps signal interrupt? I'm not totally sure here, but given your point above that having multiple pps events it seems like they should be selectable. It seems something that we'd want to control via the global pps interface, rather then having a pps-enable flag on every random bit of hardware I guess I'm not following you here. Again, I'm not super familiar with the hardware involved. Could you clarify a bit more? What do you mean by So yea.. I think this gets to the main point of ...
No, the packet timestamp occurs in the PHY, MAC, or on the MII bus and is an essential feature to support the PTP. An external timestamp is just a wire going into the clock and is an optional feature to make the clock more useful. The clock can latch the current time value when an edge is dectected on the wire. Using external timestamps, you correlate real world events with the absolute time in the clock. Typically, a clock offers two or more such input channels (wires), but timer_gettime does not offer a way to differentiate between them, and Well, the clock interface needs to offer basic services: 1. Set time 2. Get time 3. Jump offset 4. Adjust frequency This is similar to what the posix clock and ntp API offer. Using a chardev, should I make the ioctls really different, just for the purpose of being different? To me, it makes more sense to offer a familiar interface. I was perfectly happy with the chardev idea. In fact, that is the way I first implemented it. Now, I have also gone ahead and implemented I would like to repeat the sentiment in this last paragraph! I already implemented and would be content with either form for the new clock control API: 1. Character device 2. POSIX clock with dynamic ids Please, just take your pick ;^) Thanks, Richard --
Just to reopen this nearly dead but very interesting thread: I'm happy with every solution that has no impact on the accuracy of the PTP clock. If I can choose, I would prefer the POSIX clock with dynamic ids solution. Kind regards and thanks for your effort, Stephan Gatzka
Well how about something much more straightforward: #define CLOCK_FD 0x80000000 fd = open("/dev/clocks/some_clock_name", O_RDONLY); clock_gettime(CLOCK_FD | fd, &ts); This would provide all of the standard character-device semantics that everyone is accustomed to, and furthermore you could allow non-root users to manage specific clocks using just DAC permission bits. You'd still probably need to add a clock_adjtimex() syscall, but it could easily fail with EINVAL for software or per-thread clocks, but that seems like it would nicely abstract the hardware without forcing a numeric address space. Cheers, Kyle Moffett --
I am about to post another patch set for discussion, so please comment on it when it appears. Thanks, Richard --
This PTP clock simply uses the NTP time adjustment system calls. The driver can be used in systems that lack a special hardware PTP clock. Signed-off-by: Richard Cochran <richard.cochran@omicron.at> --- drivers/ptp/Kconfig | 12 ++++ drivers/ptp/Makefile | 1 + drivers/ptp/ptp_linux.c | 136 +++++++++++++++++++++++++++++++++++++++++++++++ kernel/time/ntp.c | 2 + 4 files changed, 151 insertions(+), 0 deletions(-) create mode 100644 drivers/ptp/ptp_linux.c diff --git a/drivers/ptp/Kconfig b/drivers/ptp/Kconfig index cd7becb..3aa517a 100644 --- a/drivers/ptp/Kconfig +++ b/drivers/ptp/Kconfig @@ -24,4 +24,16 @@ config PTP_1588_CLOCK To compile this driver as a module, choose M here: the module will be called ptp_clock. +config PTP_1588_CLOCK_LINUX + tristate "Linux system timer as PTP clock" + depends on PTP_1588_CLOCK + help + This driver adds support for using the standard Linux time + source as a PTP clock. This clock is only useful if your PTP + programs are using software time stamps for the PTP Ethernet + packets. + + To compile this driver as a module, choose M here: the module + will be called ptp_linux. + endmenu diff --git a/drivers/ptp/Makefile b/drivers/ptp/Makefile index b86695c..1651d52 100644 --- a/drivers/ptp/Makefile +++ b/drivers/ptp/Makefile @@ -3,3 +3,4 @@ # obj-$(CONFIG_PTP_1588_CLOCK) += ptp_clock.o +obj-$(CONFIG_PTP_1588_CLOCK_LINUX) += ptp_linux.o diff --git a/drivers/ptp/ptp_linux.c b/drivers/ptp/ptp_linux.c new file mode 100644 index 0000000..f93ae0c --- /dev/null +++ b/drivers/ptp/ptp_linux.c @@ -0,0 +1,136 @@ +/* + * PTP 1588 clock using the Linux system clock + * + * Copyright (C) 2010 OMICRON electronics GmbH + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + ...
The eTSEC includes a PTP clock with quite a few features. This patch adds support for the basic clock adjustment functions, plus two external time stamps and one alarm. Signed-off-by: Richard Cochran <richard.cochran@omicron.at> --- Documentation/powerpc/dts-bindings/fsl/tsec.txt | 57 +++ arch/powerpc/boot/dts/mpc8313erdb.dts | 14 + arch/powerpc/boot/dts/mpc8572ds.dts | 14 + arch/powerpc/boot/dts/p2020ds.dts | 14 + arch/powerpc/boot/dts/p2020rdb.dts | 14 + drivers/net/Makefile | 1 + drivers/net/gianfar_ptp.c | 527 +++++++++++++++++++++++ drivers/net/gianfar_ptp_reg.h | 113 +++++ drivers/ptp/Kconfig | 13 + 9 files changed, 767 insertions(+), 0 deletions(-) create mode 100644 drivers/net/gianfar_ptp.c create mode 100644 drivers/net/gianfar_ptp_reg.h diff --git a/Documentation/powerpc/dts-bindings/fsl/tsec.txt b/Documentation/powerpc/dts-bindings/fsl/tsec.txt index edb7ae1..f6edbb8 100644 --- a/Documentation/powerpc/dts-bindings/fsl/tsec.txt +++ b/Documentation/powerpc/dts-bindings/fsl/tsec.txt @@ -74,3 +74,60 @@ Example: interrupt-parent = <&mpic>; phy-handle = <&phy0> }; + +* Gianfar PTP clock nodes + +General Properties: + + - compatible Should be "fsl,etsec-ptp" + - reg Offset and length of the register set for the device + - interrupts There should be at least two interrupts. Some devices + have as many as four PTP related interrupts. + +Clock Properties: + + - tclk-period Timer reference clock period in nanoseconds. + - tmr-prsc Prescaler, divides the output clock. + - tmr-add Frequency compensation value. + - cksel 0= external clock, 1= eTSEC system clock, 3= RTC clock input. + Currently the driver only supports choice "1". + - tmr-fiper1 Fixed interval period pulse generator. + - tmr-fiper2 Fixed interval ...
This patch adds a driver for the hardware time stamping unit found on the
IXP465. The basic clock operations and an external trigger are implemented.
Signed-off-by: Richard Cochran <richard.cochran@omicron.at>
---
arch/arm/mach-ixp4xx/include/mach/ixp46x_ts.h | 78 ++++++
drivers/net/arm/ixp4xx_eth.c | 191 +++++++++++++
drivers/ptp/Kconfig | 13 +
drivers/ptp/Makefile | 1 +
drivers/ptp/ptp_ixp46x.c | 359 +++++++++++++++++++++++++
5 files changed, 642 insertions(+), 0 deletions(-)
create mode 100644 arch/arm/mach-ixp4xx/include/mach/ixp46x_ts.h
create mode 100644 drivers/ptp/ptp_ixp46x.c
diff --git a/arch/arm/mach-ixp4xx/include/mach/ixp46x_ts.h b/arch/arm/mach-ixp4xx/include/mach/ixp46x_ts.h
new file mode 100644
index 0000000..729a6b2
--- /dev/null
+++ b/arch/arm/mach-ixp4xx/include/mach/ixp46x_ts.h
@@ -0,0 +1,78 @@
+/*
+ * PTP 1588 clock using the IXP46X
+ *
+ * Copyright (C) 2010 OMICRON electronics GmbH
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef _IXP46X_TS_H_
+#define _IXP46X_TS_H_
+
+#define DEFAULT_ADDEND 0xF0000029
+#define TICKS_NS_SHIFT 4
+
+struct ixp46x_channel_ctl {
+ u32 Ch_Control; /* 0x40 Time Synchronization Channel Control */
+ u32 Ch_Event; /* 0x44 Time ...This patch adds support for the PTP clock found on the DP83640. The basic clock operations and one external time stamp have been implemented. Signed-off-by: Richard Cochran <richard.cochran@omicron.at> --- drivers/net/phy/Kconfig | 29 ++ drivers/net/phy/Makefile | 1 + drivers/net/phy/dp83640.c | 904 +++++++++++++++++++++++++++++++++++++++++ drivers/net/phy/dp83640_reg.h | 261 ++++++++++++ 4 files changed, 1195 insertions(+), 0 deletions(-) create mode 100644 drivers/net/phy/dp83640.c create mode 100644 drivers/net/phy/dp83640_reg.h diff --git a/drivers/net/phy/Kconfig b/drivers/net/phy/Kconfig index a527e37..a2d0753 100644 --- a/drivers/net/phy/Kconfig +++ b/drivers/net/phy/Kconfig @@ -77,6 +77,35 @@ config NATIONAL_PHY ---help--- Currently supports the DP83865 PHY. +config DP83640_PHY + tristate "Driver for the National Semiconductor DP83640 PHYTER" + depends on PTP_1588_CLOCK + depends on NETWORK_PHY_TIMESTAMPING + ---help--- + Supports the DP83640 PHYTER with IEEE 1588 features. + + This driver adds support for using the DP83640 as a PTP + clock. This clock is only useful if your PTP programs are + getting hardware time stamps on the PTP Ethernet packets + using the SO_TIMESTAMPING API. + + In order for this to work, your MAC driver must also + implement the skb_tx_timetamp() function. + +config DP83640_PHY_STATUS_FRAMES + bool "DP83640 Status Frames" + default y + depends on DP83640_PHY + ---help--- + This option allows the DP83640 PHYTER driver to obtain time + stamps from the PHY via special status frames, rather than + reading over the MDIO bus. Using status frames is therefore + more efficient. However, if enabled, this option will cause + the driver to add a mutlicast address to the MAC. + + Say Y here, unless your MAC does not support multicast + destination addresses. + config STE10XP depends on PHYLIB tristate "Driver for STMicroelectronics STe10Xp PHYs" diff --git ...
