LatencyMon Benchmarks

[w-u-2-o]

I recently built a new computer in order to achieve higher performance, glitch free, VAC audio. So far it has worked very well and has achieved its goals. I thought it might be valuable to post some benchmarks from the LatencyMon tool, which is a very good mechanism for looking at computer performance for real time audio.

PC configuration: Win10Pro, motherboard EVGA Z270FTW, CPU i7-7700K overclocked to 5GHz, RAM 16GB Corsair Vengeance LPX DDR4 3200MHz, SSD 250MB Samsung 960EVO, Video EVGA 1070ti Hybrid.

LatencyMon needs to run with all of the other programs of interest running at the same time. In my case that was: PowerSDR mRX 3.4.1, Pro Tools, DDUtil, Voicemeeter Banana and Chrome with a bunch of tabs open. I also tested with all programs shutdown except for PowerSDR. There really wasn't much difference.



Most interesting is that there is essentially no pagefault resolution time.

73,

Scott

[WA0VY]

Scott, as we discussed, the most significant benefit of my recent re-build is the reduction of latency to near zero. I ran LatencyMon with HPSDR and all related audio software for over an hour last night with beautiful results. I repeated that result today but eventually I would see a brief isolated kick-up in some process or driver, usually an hour or so in. See attached screenshot which was taken 30 minutes in.

Screen Shot 205.jpg (542.78 KiB) Viewed 50309 times

I also saw zero page faults and attribute that to the Samsung 960 Pro on an M.2 connection which is direct to the CPU, as is yours. No more SATA bus. Real world, I have been listening all day and have not heard a single drop. To compare and contrast, on my old build, LatencyMon would be in the mid-red on interrupt-to-process latency within seconds. Page faults and time to resolution were also in mid to high red within minutes. With my old GTX 980ti, the NVIDIA drivers were also in the red, almost instantly, and the result was horrid on HPSDR. The AMD Firepro W5100 improved the driver latency issue, and drops and pops, but at the cost of lousy and slow video. However, it did not resolve the interrupt-to-process latency or the page faults. The new build included the ASUS GTX 1080ti. The current NVIDIA drivers are flawless (as is performance). More on that in the related thread on the new build.

[w-u-2-o]

That's incredible performance, Brent! You mentioned the video card and SSD, what CPU are you using?

[WA0VY]

The new build includes a Xeon E5 2696 V4 CPU which I was able to get as a "new pull." I purchased some G.Skill 3200 RAM also but it is no faster with that processor than my old Corsair 2133. The E5 2696 only supports RAM speed to 2400 and if you try to OC the RAM, it won't post. So I have reinstalled my old RAM and will send the 3200 back. The E5 2696 can be substantially OC'd and is a very impressive, cool running chip. With HPSDR and related audio processes running, CPU usage is 3-4% and temps are 29-33C. The MB is an ASUS ROG Edition 10 which replaces my older ASUS X99-E WS/USB 3.1. There are pluses and minuses with the new MB. On the plus side, it is built like a tank, has a PS2 connection, lots of PCIe slots, modern diagnostics, smooth running dual Intel gigabit NIC's, M.2 and U.2 slots. On the minus side it does not have a COM header and only one 2x USB 2 header (I need 4 total). I did work-arounds on those issues and installed a dual physical COM PCIe card and routed some of the onboard USB devices to the available 3.0 slots on the back. I have attached partial Passmark results.

Screen Shot 206.jpg (107.96 KiB) Viewed 50292 times

One issue remains and that is the 1080ti does not perform well on the 2D benchmark test. It is coming in at around 690. It should be near 1,500. The 3D score is close to where it should be and sometimes beats the baseline. I have tried tweaking, OCing, etc., but nothing moves the results. I did see a tiny tear on one pin of the PCIe connector but the majority of the pin is present. I will consult with ASUS support but its hard to imagine how the card could benchmark at or above the 3D and GPU compute baseline numbers if the connector was defective.

While I can't generate a world class complete Passmark benchmark with 2133 RAM and a low 2D score, this machine runs fast, cool, smooth and with no driver or latency issues. That's a real world result that is exactly what I wanted.

[w-u-2-o]

That's a hell of a CPU! What is the full Passmark you are getting now?

[WA0VY]

I don't want to publish that until I see if I can fix my 2D problem. Needless to say, its nothing to brag about. Also, I'm not sure that a 2D benchmark is relevant anymore. Other modern benchmark software ignores it. Scott (and others), check this one out:

https://benchmark.unigine.com/superposition.

It fun to use and really gives your GPU a workout. My 1080ti got a 20k result which is not world class but also not embarrassing.

[WB8LBZ]

Wow, what a speed daemon. I did a quick look on the internet and those chips won't be in the low cost option. More power to you. I'm interested in seeing your final numbers.

73,

[W1AEX]

Scott and Brent,

Your machines are in a league of their own in the speed and latency departments! I'm using one of my retired gaming machines for use on the radio bench to handle all my SDR tasks. When I built this tired old beast back in 2011 it was a fire breather, but at this time... not so much:



At any rate, I thought I'd take a look at how it does with Latency Mon and it didn't exactly knock it out of the park but it seems to be fairly decent, although I notice that it managed to incur a hard page fault in this session.



Latency Mon reports that apparently I forgot to disable CPU throttling which is definitely something to avoid with real-time streaming. I must have missed that the last time I updated the bios on this motherboard.

Nice job putting together those fire breathing machines!

73,

Rob W1AEX

[w-u-2-o]

Your's looks speedier than mine, latency-wise, Rob. I wish I knew how to bring my latency down further. Common sense says it should be better.

[WA0VY]

Rob, that looks pretty darn good to me. I assume you have no issues with pops and drops. Now, let me show you a bit of history which might make you feel very good about your setup and help explain why I decided to do my latest build. This is a screenshot of LatencyMon sometime after my 2016 build using an i7 5930k and a GTX 980ti.

Untitled 34.jpg (93.2 KiB) Viewed 50151 times

This is a screenshot after I replaced the 980ti with the AMD FirePro W5100.

Screen Shot 49.jpg (100.61 KiB) Viewed 50151 times

Notice the utterly amazing improvement? No?

Actually, as awful as this looks, there was a noticeable improvement in pops and drops once I rid myself of the poor NVIDIA drivers associated with the 980ti. But I still had significant pops and drops in HPSDR, as you might expect looking at this mess. Anyway, I set out to cure that and have now apparently succeeded. I have been operating with the new build for about 4 days now and not had a single page fault, no significant driver or interrupt-to-process latency and, most important, not a single pop or drop on HPSDR. See above for screenshot of typical current LatencyMon results. Obviously, I'm very happy with this result but I think any build that achieves system compatibility will show similar LatencyMon results and give you great operational results in HPSDR even if you don't have a world class gaming machine.

[W1AEX]

Hi Brent and Scott,

Before I build a system I scour the video capture/video rendering forums to see what motherboard chipsets are not problematic for those guys. They really start screaming about DPC and other high latency issues so they are a great "test bed" for anyone who is doing real-time streaming like we do. The Z68 Intel chipset had their approval so that's what I went with for the chipset with the ASUS motherboard in my SDR system. I have heard quite a few others mentioning the NVIDIA drivers as being problematic but I have never found them to cause issues with any of my systems. For me, it's usually ethernet drivers, soundcard drivers, or some of my USB devices that stir up trouble.

I would like to say that this machine always behaves nicely but there is certain software (the Malware Bytes file system protection and website blocking modules and my screen capture software come to mind) that will push my machine over the edge to the point where DPC events will begin to show up. I don't experience audio dropouts at all, especially since Warren switched the communication protocol from TCP to UDP, unless you count the VAC ring buffer under-runs that Scott hears sometimes when I'm transmitting as we chat on the air.

At any rate, it would be great if these issues never happened at all but I have to admit there is a certain nerdy satisfaction when you can get it all squared away! Your systems definitely are very squared away so congrats!

73,

Rob W1AEX

[Tony EI7BMB]

Hi Rob, I'm thinking of replacing my ASRock B75 pro3 board (intel ivy bridge) . Are there any particular boards you would recommend right now ?

[W1AEX]

Hi Tony,

Unfortunately, the last gaming system that I built was in March of 2014 and it uses an ASUS Maximus VI board with the Intel Z87 chipset. At this time it looks like that Intel chipset has been left in the dust by the Intel Z97, Z99, Series 100, and Series 200 chipsets. As is usually the case, a few months after anyone builds a system everything moves forward and nearly everything learned in the building process becomes obsolete. I'm afraid that I can't be much help as I haven't done any research to keep up.

However, I'll bet Scott and Brent could make some very good suggestions since their builds are very recent!

73,

Rob W1AEX

[Tony EI7BMB]

Thanks Rob, some good pointers. Scotts 270 board looks like a good direction to go with the small issue of the Killer NIC not working just yet .

[w-u-2-o]

I'm pretty sure that the Killer NIC, which is really just a Qualcomm Atheros NIC, will run with the Qualcomm drivers. I just haven't had a chance to fool with it yet.

The MSI, EVGA and ASUS motherboards seem to be the top of the heap. If you are planning to overclock, go with an MSI or EVGA I think. And the MSI BIOS seems to be very feature rich. I went with the EVGA because it was less expensive than the equivalent MSI.

73!

Scott

[Tony EI7BMB]

Thanks Scott, hope you get the killer NIC working.

[WA0VY]

Update: I did some tweaking to the CPU power management settings in the BIOS and found settings yielding a significant improvement in latency with no real downside in terms of CPU temperatures. The settings all involved disabling anything that would reduce the power state of the CPU, even at idle. Looking at the in-depth LatencyMon results, the improvement is very dramatic in "High_Level IRQL" test setting, which is challenging. Here is my latest LatencyMon result.

Screen Shot 228.jpg (480.95 KiB) Viewed 49863 times

However, I have seen anomalies. I can't always get this far out without ntoskrnl going into a late-green-early-red-ish state. Sometimes I will note a "snap" sound in the back ground on RX and simultaneously see a spike in DPC and/or I-to-P latency. Still, I have not seen a single instance of a TX pop with this new build. If I do hear a "snap" I can usually cure it by restarting HPSDR. At that point, I am usually good to go for quite some time before there are signs of instability. Without HPSDR running, latency is essentially zero.

In sum, I have about a 95% improvement but there are still some issues out there. There is tracing software I plan to employ to see what call is involved when ntoskrnl goes all squirrelly. Stay tuned!

[W4WMT]

Hi Scott,

Just curious, what drove your decision to go for an i7 rather than one of the Xeons?

73, Bryan W4WMT

[w-u-2-o]

Cost. I had a budget. I only blew it by $100, too

I really wanted a CPU that cost more than my entire build!

[W4WMT]

Ok, I understand. I was hoping you were going to say the i7s are better than the Xeons

I really don't understand why Intel runs those two product lines in parallel. Are the Xeons really supposed to be superior to the i7s? If so, then why? I don't get it.....

73, Bryan W4WMT

[w-u-2-o]

It's mostly about the number of cores, lots of them on Xeons. There are a few other differences, too. Just google "i7 vs Xeon".

[WA0VY]

Basically, an i7 would be better for gaming while a Xeon would be better for server applications where you have to move a lot of data at once. There is a lot you can do with an i7 that you can't do with a Xeon. For instance, overclocking is limited (and difficult) with a Xeon and it can't use high speed RAM. Been there, tried it. I went with the Xeon because it was well priced (sort-of) and I am betting that it will perform better with Protocol 2. We'll see if that turns out to be true.

Scott, an update: having upgraded from Win 7 to 10 on two different drives and migrated the build to the 960 Pro, I was having glitches in many programs. So, I did a clean install of Win 10 (twice - first one went squirrelly). Now, all program glitches are gone but the NIVIDIA drivers are causing a high latency issue again. So far, it is not causing drops and pops but I have done everything I can think of to resolve it without success. I'm trying to find some professional help, meaning hire-a-geek type help, not a psychiatrist.

[w-u-2-o]

It sounds like you are getting close, Brent, don't give up!

73!

Scott

[WA0VY]

BREAKING NEWS!!!

I did an update to the chipset drivers and achieved a somewhat interesting result. This is a 30 minute-ish LM result with HPSDR, ASI software and Firefox running, all of which usually present a challenging environment.

IMG_1175.JPG (730.51 KiB) Viewed 49751 times

Not bad. I took this shot with my IPhone. I already knew that if I did a screen cap with Debut, the NIVIDIA driver would go red. As predicted, it did.

Untitled 1.jpg (669.95 KiB) Viewed 49751 times

However, there were no pops or drops at any time. I'm wondering if the Xeon, with 22 cores, has the effect of isolating the NIVIDIA driver latency in a core or two and does not effect the other normal software functions through other cores. With my six core i7, I always heard the issues when LM looked like this. But now I don't.

[K9RX]

Gentleman,

Forgive me but it seems this post has diverged here and there.... there are a number of posts showing what you all are getting individually - but the title here is "Benchmarks" .... so I ask : what are the benchmarks!?

I had an Flex5000 at one time and their manual had values NOT TO EXCEED ... and I believe also optimal ones... that would be nice to have. I'm currently at about 800us for the "Highest reported ISR routine" value ... and pretty low numbers for all others ... it says my system is fine for high speed audio processing and I've never had what would appear to be an issue ... but I'd like to know if these are borderline. This was taken with
-the latest release mRX running, my logging program, CommCAT, WSJT-X, IE, and a few other programs for the amp etc... [i5-4460K with intel video]

thanks

Gary
K9RX

[w-u-2-o]

There are no benchmarks the way you want them, Gary. What people have posted here are "informal" benchmarks. That is to say, people have posted what they are getting and how it works for them.

73,

Scott

[K9RX]

and sure enough - go back and let LatencyMon run and within a few minutes it starts to rise ... it now says it is not suitable for audio and the values are above 3mS for the Highest Reported DPC and ISR.

I guess I'll need to try to hunt down what is causing it - assuming it is one thing. It says the 'culprit' is NDIS.sys (network driver), suggesting I turn off the wifi - its not on ... and wdf01000.sys or Kernel Framework Driver Routine for the DPC (says to check cpu settings).

I'll do a google search on these - if anyone knows up front (wouldn't that be fortunately) what to look for I'd (and others I'm sure) would appreciate it.

Gary

still can't say that I've ever heard any anomalies re the audio though.

[WA0VY]

Gary, I'm not aware that there any published "benchmarks" showing what others are seeing in LatencyMon (LM). There are published benchmarks associated with Passmark, but that is measuring overall and system sub-component speed performance, which is not the same as latency. As far as where you are good to go in terms of latency, obviously, the lower, the better. But, if LM is indicating that you are capable of running real-time audio, and you are not seeing audio drops and pops, you're there. I am not presently seeing any drops or pops even though LM indicates occasionally spikes associated with the NVIDIA driver. I am currently using Windows Performance Analyzer (WPA) and related Recorder to try to track that down. Its a very complex program but it is giving some useful, and surprising, data. More on that later.

[WA0VY]

I did not realize that this thread had gone into page 2 when I posted my last response so I did not see Scott’s or your last response before I posted. Sorry to hear that you have the dreaded latency issue. However, if it’s not causing drops or pops, resolution is more a matter of seeking perfection than necessity. But I have latency OCD so I’m not one to suggest that any one shouldn’t chase this thing down and put a stake through it. I think you will be disappointed with the info you will get from Google. Other than updating your drivers, most of the information is so generic and unspecific to your build and software environment as to be basically worthless.

As mentioned, I am currently using Windows Performance Analyzer (WPA) to trace this. It’s a time-consuming and complex process. You can’t run the recorder too long as the files become huge and unmanageable. So, I had to keep running the recorder while all potential problem software was running as well as latencyMon (LM) and dump the data if I didn’t see an interesting event within one minute. After about 10 minutes in on 7/19, DPC latency jumped to early red and I captured it in about 31 seconds of data. I processed it and will show you in some screen shots what I found. Again, every different build and software environment will probably show something different, but if you download and run WPA, you can get a pretty good idea what your specific issues are. The first screenshot is of the entire 31 second recording. I arranged the data to show GPU and CPU usage in detail. I could see from the other graphs that nothing interesting was happening in storage, power or memory. The event occurred at about 18.5 seconds in and is pretty obvious in the graph.

ScrnShot 2.jpg (445.2 KiB) Viewed 46720 times

The second shot shows the mouse hovering in the center of the event generating a horizontal line through both graphs which indicates that (a) Firefox was the process involved and (b) it did not cause a spike in CPU usage, only in GPU activity under rendering. That wasn’t a huge surprise since I am using a Xeon E5 2699 v4 with 22 cores, so it takes a serious anomaly to get its attention.

ScrnShot 3.jpg (448.93 KiB) Viewed 46720 times

The next shot shows (in the blue spikes) an “unknown” process is generating some GPU activity/latency due to “memory transfers.” I have no idea exactly what that means but I note that there is no unusual system RAM data so I assume that is in the graphics GDDR.

ScrnShot 4.jpg (442.63 KiB) Viewed 46720 times

The next shot shows the fairly large amount of time that was involved in rendering in Firefox at the inception of the event. It is about 60% of all the rendering time used by Firefox in the entire 31 seconds of the recording, so A LOT! Hence, the DPC spike.

The last shot shows the comparative highest amount of time involved in the mystery “memory transfer” issue. It suggests that is not the problem, whatever process is causing it.

So, what do I know now that I didn’t before? Microsoft Edge will become the only browser up when I’m running HPSDR.

[WA0VY]

For whatever reason, this blog would not allow me to post the last two shots in the previous post, so here they are:

ScrnShot 5.jpg (496.98 KiB) Viewed 46720 times

ScrnShot 6.jpg (495.24 KiB) Viewed 46720 times