The Chip Collective Testing Guidelines (IN PROGRESS)

Please throw some money my way either directly VIA PayPal or with my Ko-Fi maybe don’t do my Patreon until I fully set it up.

We at The Collective are developing a testing methodology in order to bring in more data from outside sources. These are just guidelines for different tests and not a whole suite of tests that someone interested in contributing should run. You can go out and run just a single test from this list and that work would be greatly appreciated.

Global Method

• Note your hardware specifications
• Note your OS version
• Note UEFI version
• Note AGESA version
• Note if any software or hardware security patches are applied and enabled
• Scores collected with HWINFO64 or GPU tuning software running should be discarded
• Scores collected with nothing running in the background are acceptable only

CPU TESTING

There are multiple test categories that The Collective is interested in.

• Stock CPU
• Overclocked CPU
• Motherboard CPU Combo Differences
• How RAM Alters CPU Benchmarks
• Cache Speed & FCLK
• Power Efficiency Testing (Intel)
  • Use UEFI Settings or XTU/ThrottleStop to Limit the CPU at a fixed package power with C-States enabled to find out where the boost goes under different loads
  • Reset the UEFI and enable your highest XMP profile
  • Limit the power to spec TDP multiplied by 1.25 and then work down by 5w for each test iteration
  • Do all cores enabled and then step down to the next lowest until you reach ⁴⁄₈
  • Test CB R15 under load and take the score from the 2nd multi-core run
  • Remember to note steady state temps as well as voltage and full power at full stock for your specific motherboard

CPU Testing Software

• AIDA64
• GeekBench
• Asus Realbench
• Cinema4D R15
• Cinema4D R20
• HWINFO64
• Passmark PerformanceTest
• Y-Cruncher

CPU Testing Method

• Benchmarking the CPU itself
  • If you are testing the CPU itself you should verify that using default or auto settings in the UEFI actually conform to some sort of specification
• “IPC” testing
  • For “IPC” testing NavJack27 already has historical data collected at 3.2GHz so he’d like to stick to that fixed clock speed
  • You should note that this isn’t true IPC testing and more so just Cinema4D single-threaded clock-for-clock testing
• Maximum overclock testing should be the highest stable under Asus Realbench (10 loops of the benchmark, not the stress test)

GPU Testing

Benchmarking cards with synthetic workloads is a very useful metric for determining if the card is working correctly.

GPU Testing Software

• 3DMark
• Passmark PerformanceTest
• AIDA64 GPGPU Compute
• MSI Afterburner
• EVGA Precision
• HWINFO64

GPU Testing Method

• Card should be tested at default limits initially
• Card should later be tested with max power targets and voltage
• Overclocking should be tested and reported with a HWINFO64 graph during testing load

Gaming Benchmarks

These are holistic tests of a whole system and harder to fully pin down. Everything about the test system and game in question must be known. What the version of the game is will be important.

Game Testing Software

• The Game in Question
• HWINFO64

Game Testing Method

blah

How should I organize and collect my data?

blah

How do I submit my results?

Just get in touch with NavJack27 on Twitter or Discord until we’ve developed a better way to facilitate this.

To-Do

Danilo “18pF flip-flop” Cominotti Suggestions

• SPECworkstation is really good! Yes, it takes a while to run, but it tests like ~20 real softwares. I’m a big fan of PCMark10 and how easy it is to export its data (including CPU clocks monitoring, etc.) and compare stuff.
• I suggest including AIDA’s full report, because it points to OS settings, services, active processes, etc. IMHO, some FW/OS settings should be explicitly stated: 1) Memory timings, 2) Windows Defender (disabled via GPO), 3) Fast Startup (disabled), 4) Background apps (disabled).
• Solid scientific methodology would suggest running tests at least in triplicate. It’s very important and I’ve seen many runs that were negative outliers.
• Super-Pi

Ben Hull Suggestions

• cpu-z benchmark?

SkyJuice Suggestions