How does one go about setting up Precision Time Protocol (PTP, high precision NTP)) IEEE 5888

shodanx · October 22, 2024, 7:40am

Hi,

I see many router NICs claim IEEE 5888, also wifi should support wifi-ptp.

But how do you set it up, what hardware can you use as the time source (if you need accurate time and not just accurate time synchronization between devices)

As for packages I see kmod-ptp and linuxptp but not sure how to set them up and how to test them once their work.

Some more reading

Two uses that come to mind for this are phase coherent audio playback and RTK GPS. But just having really accurate time is a good thing on its own.

evs · October 22, 2024, 7:42am

I can answer the time source part. Never done PTP but i'm interested too.

GPS. Or GPS disciplined local oscillator. And then upgrade your oscillator to whatever precision/accuracy you want. (i.e. thermally stabilised, how many PPM you need, get an atomic clock etc)

edit: Just to clarify for readability so this works well with @brada4 's comment: an oscillator is a clock source

brada4 · October 22, 2024, 7:46am

Easiest client is chrony. There should be network card, sometimes even PCI hub with autonomous clock. Or at least kvm ptp driver.
Probably you need to add new kmod packages with all needed clock support.

EDIT: this is to test concepts and the software chain involved, the network time distibution takes that microsecond callibration.

You can disable conntrack for NTP and grow from centisecs to sub-millisec precision.

moeller0 · October 23, 2024, 7:34am

I will bite? How that, if I might ask respectfully? As far as I can tell NTP with internet time servers is at best good for millisecond precision (based on jitter/latency variance to host on the internet), for sub-miilisecond a local NTP server seems required, but with a local server, how does conntrack matter?

shodanx · October 23, 2024, 4:43pm

I don't know the specific and that would have to be experimentally demonstrated, but if the packets have an extra round trip into the memory of a non-deterministic preemptive kernel then it makes sense that could add extra error to the timing signal. Especiallly if the system is under heavy load.

It probably matter less, the stronger your cpu is. I'm curious to know if that really works and makes a difference.

But on the other hand, people who need a good time reference would probably use an externally calibrated clock or a GPS signal (that's probably the easiest, but I have no idea how hard it is to get accurate GPS timing through an rs232 port)

Personnally, I am more curious about having a PTP relative time reference for my LAN (to synchronize sound on LAN) than to have an absolute time reference from the outside world.

brada4 · October 23, 2024, 6:59pm

The sync process is unidirectional, so you just need every network switch to patch up packets by their own forwarding latency... Given variety of hardware platforms that number is not easy to estimate.

evs · October 23, 2024, 7:09pm

Okay so I did some research as I got up way too early this morning =P.

Meinberg looks to have the best blog post / application note?
Quite recent too, at least compared to what else I could find.
The next best I could find was skyworks an1208?

Anyway the TL;DR is basically a TCXO will probably be fine if you don't lose GPS sync?

In reading some other documents, looks like there's some peculiarities regarding clock selection with PTP and selecting your clock sources (edit: out of localyl available, if you have multiple sources?) I don't know enough....

aleks-mariusz · May 20, 2025, 3:50pm

Just chiming in if anyone's interested, after i had just recently replaced my wrt32x with a mt6000 for my main home routing, i realized that the wrt32x can be re-used to achieve something along these lines w/ the high precision time goal discussed above:

openwrt on linksys wrt32x, connected to a affordable GPS module (connected to the router's usb port) which has a built in usb-serial (CH341 chipset).
gpsd then is set up to accept/process the NMEA0183 data stream over the usb-serial channel
for the best in precision/accuracy, it also involved minor hardware modification of the wrt32x (more info below in the challenges section) to accept the PPS signal (1hz pulse) from the GPS module (to have within-nanosecond-precision marking the beginning of each second tick), fed in via a GPIO channel (made available by disabling the ~~anyway-useless~~ never-used WPS button).
chrony set up to read gpsd shared-memory, and the PPS input (with kmod-pps-gpio configured via device-tree modification of the platform) to discipline the system clock - which then achieves a stratum-1 ntp server on my network:

root@wrt32x:~# chronyc sourcestats
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
GPS                        20  11   306     +2.960     26.350   +166ms  2994us
PPS                        22  13   337     +0.000      0.001     +1ns   167ns <-- not bad eh?
ntp1.dmz.terryburton.co.>  11   7  172m     -0.044      0.079   -182us   161us
ntp01.ix1.inferno.net.uk    8   4   77m     -0.036      0.193   -384us   131us
ntpsvr2.npl.co.uk          10   7  111m     -0.078      0.172   -409us   179us
x.ns.gin.ntt.net            6   3   43m     -0.031      0.135   -512us    40us
root@wrt32x:~# chronyc tracking   
Reference ID    : 50505300 (PPS)
Stratum         : 1 <-- woohoo
Ref time (UTC)  : Tue May 20 15:34:08 2025
System time     : 0.000000000 seconds slow of NTP time
Last offset     : +0.000000113 seconds
RMS offset      : 0.000000072 seconds
Frequency       : 336.817 ppm fast
Residual freq   : +0.000 ppm
Skew            : 0.001 ppm
Root delay      : 0.000000001 seconds
Root dispersion : 0.000024747 seconds
Update interval : 16.0 seconds
Leap status     : Normal

but we can do even better with PTP:

addition of linuxptp package to be a PTP grandmaster for my network (this package lacks any uci-config / init scripts, so i wrote my own).. the wrt32x's lan ports support hardware time-stamping which is why i decided to use (repurpose really) this router

essentially, gps+pps disciplines the wrt32x system clock thanks to gpsd/chrony - providing time-services using NTP (if you're interested in the protocol, see this video), then we use phc2sys (from linuxptp) to discipline the NICs' hardware timestamping clock, ptp4l (from linuxptp) "advertises" its time-services using PTP (if you're interested in how the protocol itself works, have a look at this video)

I then set up my linux-running NAS-host to be a PTP slave.. i have a supermicro mobo with 4x 10gbps ports (2 copper, two SFP+), one of which has hardware timestamping support.. however i forego phc2sys on the slave since chrony has support to discipline the system clock directly from the PHC (PTP Hardware Clock):

$ chronyc sourcestats
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
PHC0                        6   3     1     +0.097      0.633    +13ns    77ns <-- PTP provided time from /dev/ptp0
wrt32x.my-domain>           8   5   30m     +0.434      0.032    +25us  8859ns <-- additional NTP service
161.76.155.90.in-addr.ar>   6   3   86m     +0.636      0.975   -540us   433us
ntp01.pingless.com          6   3   86m     +0.627      0.914   +539us   441us
ntp.uk.eria.one            19   9  258m     +0.701      0.056   +800us   320us
ntp2.as200552.net          19  11  258m     +0.613      0.089   -137us   510us
$ chronyc tracking
Reference ID    : 50484330 (PHC0)
Stratum         : 2
Ref time (UTC)  : Tue May 20 15:48:05 2025
System time     : 0.000000102 seconds fast of NTP time
Last offset     : -0.000000033 seconds
RMS offset      : 0.000000228 seconds
Frequency       : 27.068 ppm slow
Residual freq   : -0.010 ppm
Skew            : 0.221 ppm
Root delay      : 0.000000000 seconds
Root dispersion : 0.000000738 seconds
Update interval : 0.3 seconds
Leap status     : Normal

You can verify your own hardware's support for hardware timstamping using the command:

root@wrt32x:~# ethtool -T lan1
Time stamping parameters for lan1:
Capabilities:
        hardware-transmit     (SOF_TIMESTAMPING_TX_HARDWARE)
        hardware-receive      (SOF_TIMESTAMPING_RX_HARDWARE)
        hardware-raw-clock    (SOF_TIMESTAMPING_RAW_HARDWARE)
PTP Hardware Clock: 0
Hardware Transmit Timestamp Modes:
        off                   (HWTSTAMP_TX_OFF)
        on                    (HWTSTAMP_TX_ON)
Hardware Receive Filter Modes:
        none                  (HWTSTAMP_FILTER_NONE)
        ptpv2-l4-event        (HWTSTAMP_FILTER_PTP_V2_L4_EVENT)
        ptpv2-l4-sync         (HWTSTAMP_FILTER_PTP_V2_L4_SYNC)
        ptpv2-l4-delay-req    (HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ)
        ptpv2-l2-event        (HWTSTAMP_FILTER_PTP_V2_L2_EVENT)
        ptpv2-l2-sync         (HWTSTAMP_FILTER_PTP_V2_L2_SYNC)
        ptpv2-l2-delay-req    (HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ)
        ptpv2-event           (HWTSTAMP_FILTER_PTP_V2_EVENT)
        ptpv2-sync            (HWTSTAMP_FILTER_PTP_V2_SYNC)
        ptpv2-delay-req       (HWTSTAMP_FILTER_PTP_V2_DELAY_REQ)

and on the linux-nas host:

$ sudo ethtool -T eno3
Time stamping parameters for eno3:
Capabilities:
        hardware-transmit
        software-transmit
        hardware-receive
        software-receive
        software-system-clock
        hardware-raw-clock
PTP Hardware Clock: 0
Hardware Transmit Timestamp Modes:
        off
        on
Hardware Receive Filter Modes:
        none
        all

You'll see in both instances that PTP Hardware Clock: has a number, this number 0 for me corresponds to the /dev/ptp0 device that is created, this is what phc2sys (and chrony) act upon.

notes for those interested in the hardware: make sure to get a GPS receiver with a PPS output pin - also i chose the one linked above because it has multiple frequencies (both L1 and L5) which helps get more stable fixes despite varying atmospheric interference - no idea how much that's really needed, but from my research i learned atmospheric conditions are always changing and can affect gps satellite signals traveling through the atmosphere, causing signal delays. but, the receivers with multiple frequencies can compute the position more accurately with access to signals from two or more different frequency bands, utilizing algorithms in the chipset to correct these atmospheric delays.

A few challenges I encountered on the way:

the GPS receiver has a 6 or so pin header (4 of which are UART, i didn't need it since the module also has a built in CH341 usb-serial chipset), one of these headers tho is important as it has PPS-output, but i needed a way to connect this to one of the GPIO channels on the wrt32x - so in order to do this i had to disassemble the wrt32x, and solder a single-pin header to one of the legs of the WPS button (it has 4 legs, 3 of them to ground, the 4th is what the gpio channel connects to, since i don't ever plan on using WPS (a bit insecure really), i needed to disable the WPS button in the device-tree for this device, and instead tell the kernel to direct signals from this GPIO channel to the kmod-pps-gpio module (which creates a /dev/pps0 device which chrony can then access). i also had to make a small hole in the bottom of the case to allow access to the pin. The appearance of the router is hardly different tho since the hole is only small enough for a single jumper wire and it's located on the bottom of the router anyway
the 4x lan ports all support hardware timestamping, but i could not get the NAS to see the PTP packets with transport-type UDPv4 or L2 - possibly because the lan port is by default part of the br-lan bridge (and I also have vlan-tagging in the mix). So I got around this by dedicating one of the ports (lan 4) and removing it from br-lan, and then hooking up to my switch (a zyxel xgs1210 running the managed firmware hack), as a non-trunked / non-tagged (so a plain access-port on the vlan w/ the NAS). This allowed my NAS (which also does vlan-tagging) to see it, but i needed to pass to ptp4l the "de-tagged" interface (eno3.2222), this worked fine for the L2 transport mode, but still not UDPv4. I haven't looked into getting this to work w/ UDPv4 (or UDPv6 as i run full dual-stack at home), other than possibly eventually also dedicating an additional ethernet port from the NAS also, but this presents other challenges since my eno3 port is also part of br0 which my libvirt-managed VM rely on).

Anyway if you're interested, i can provide more detailed guide. I am considering creating a PR to the linuxptp to make the /etc/config/linuxptp file i created along with the related init-scripts upstreamed.

efahl · May 20, 2025, 5:10pm

Good work, nice write up!

I haven't got PTP working yet, but NTP is running just fine.

I've been wanting to do move it to one of my x86 routers, but the challenge there is getting the PPS signal in, as the boxes have no GPIO, grrr. I'm currently running my ntp server on an old RPi: it's super simple, install gpsd, chrony and plug in six wires, literally 10 minutes of work to get it running. I'm using a $10 Neo-7M with an equally cheap active antenna.

I love the xgps tool (part of the gpsd package on most distros), as it comes up on my screen aligned with the house, and I can see at a glance which satellites are visible and deduce what's blocking the antenna. 29 is over the peak of the roof and 19 is behind the chimney...

spence · May 20, 2025, 6:02pm

It was common to use a pin on DB9 or DB25 serial ports or even parallel ports and possibly others for pps input so if that might be an option for any of your systems of interest do a web search of "pps on serial port" or similar searches. It looks like there is/was kernel level support.

efahl · May 20, 2025, 6:37pm

Ah, good point! My N5105 minipc has serial pins on the mobo, and my plan was to use those for the GPS data stream, but if I get a module that has usb-serial, like @aleks-mariusz uses, then I could put that stream over USB and repurpose the serial input as PPS. Hmm...

moeller0 · May 21, 2025, 8:48am

So I use an rPi4b with a GPS receiver board for NTP and software PTP clock. The raspberry Pi5 actually has a hardware PTP clock... if I would actually need PTP instead of just playing around with it, I probably would upgrade the Pi...