Author Topic: GATT Benchmarking Development forum  (Read 39622 times)

Offline W1nTry

  • Administrator
  • Akatsuki
  • *****
  • Posts: 11329
  • Country: tt
  • Chakra 109
  • Referrals: 3
    • View Profile
  • CPU: Intel Core i7 3770
  • GPU: Gigabyte GTX 1070
  • RAM: 2x8GB HyperX DDR3 2166MHz
  • Broadband: FLOW
  • Steam: W1nTry
  • XBL: W1nTry
GATT Benchmarking Development forum
« on: April 16, 2009, 08:57:05 AM »
This thread was formally 3DMark Vantage, however i've taken this opportunity to turn it into a development discussion lookinf forward for a weighted benchmarking scheme to determine overall ranking of rigs as opposed to just 3DMark.

Verification Apps
CPU-Z (1.50) (CPU, cache, memory, mainboard, spd) must be shown
GPU-Z (0.3.3)

Gaming/Overall Benchmark Apps
3DMark06 - Run @ 1280x1024 Default settings
3DMark Vantage Run @ 1280x1024, Default settings
PCMark Vantage Run @ 1024x768, Default settings

Memory Benchmark
SiSoftware Sandra - Within this suite are MANY benchmarking, stress testing and information tools. We would be interested primarily with the Memory Bandwidth Benchmark

HD
HDTach - Run with Long Bench (32 mb zones) N.B. Must be run as Administrator in Vista and in Windows XP SP2 compatibility mode.

Stability Apps (not part of the results system, just to verify validity of results as REPEATABLE)
OCCT - Linpack Medium, 16-20 minute run
ATI Tool - Used to detect Artifacts
Linkpack/Linx
Prime95 32-bit - Stress Test with as many threads as there are physical cores
Prime95 64-bit - Stress Test with as many threads as there are physical cores

Then we establish a weighted system to calculate a final score.
« Last Edit: April 17, 2009, 07:35:19 PM by W1nTry »

Carigamers

GATT Benchmarking Development forum
« on: April 16, 2009, 08:57:05 AM »

Offline Spazosaurus

  • Dr. Herp Derpington
  • Administrator
  • Akatsuki
  • *****
  • Posts: 7685
  • Country: tt
  • Chakra 52
  • Referrals: 3
    • View Profile
    • The Awesome Company
  • CPU: i5 3470
  • GPU: GTX 780
  • RAM: 8GB Corsair
  • Broadband: Blink 2Mb + Flow 20Mb
Re: 3DMark Vantage Rankings
« Reply #1 on: April 16, 2009, 12:13:12 PM »
I dont agree with the system must be stable bit. If a man wanna do a max oc to post his ultimate score, he should be able to do so.

Offline W1nTry

  • Administrator
  • Akatsuki
  • *****
  • Posts: 11329
  • Country: tt
  • Chakra 109
  • Referrals: 3
    • View Profile
  • CPU: Intel Core i7 3770
  • GPU: Gigabyte GTX 1070
  • RAM: 2x8GB HyperX DDR3 2166MHz
  • Broadband: FLOW
  • Steam: W1nTry
  • XBL: W1nTry
Re: 3DMark Vantage Rankings
« Reply #2 on: April 16, 2009, 12:36:44 PM »
OK we'll discuss for now but as soon as it's finalized i'll remove all these 'conversational posts'. So why shouldn't it be stable? Doesn't that already apply to the 3DMark06 rankings that determines the top rig in gatt? I mean if I had the time i'd develop the weighted system... in fact let's discuss a weighted system now. I will expect feedback from you, arc, n00b and all others on gatt that benchmark as we will develop a suite of apps that are easy to acquire, tried and tested and the process for such. So first we decide the apps to use for verification:

CPU-Z
GPU-Z

Wanna add aything to that?

Offline Spazosaurus

  • Dr. Herp Derpington
  • Administrator
  • Akatsuki
  • *****
  • Posts: 7685
  • Country: tt
  • Chakra 52
  • Referrals: 3
    • View Profile
    • The Awesome Company
  • CPU: i5 3470
  • GPU: GTX 780
  • RAM: 8GB Corsair
  • Broadband: Blink 2Mb + Flow 20Mb
Re: GATT Benchmarking Development forum
« Reply #3 on: April 16, 2009, 12:56:12 PM »
CoreTemp or Everest preferably coretemp as its free. im not sure if there's a non paid version of Everest? (Arc can clear up)

Also, multiple tabs of CPU-Z. E.g one instance showing the cpu, one the memory and even another one showing the mainboard.

(More suggestions for verification programs are welcome)

As far as system stability is concerned, I can speak for myself that my 4ghz oc is not entirely stable. The rule of thumb around the forums is to run 10 LinX tests for 'guaranteed' stability. My thread will show that my 4ghz overclock does indeed pass that test, but when put under extreme test to setermine MAXIMUM stability (read 100tests plus), it fails. So can THAT be considered "Stable"? What is stable? 100%? 10%?

Most of us are aware that some of us (you know, those of us who are not 'p*$$ies) overclock systems to the MAX so that it is just stable enough to successfully run benchies. For a lot of us, benchie speed is NOWHERE near what we would run our systems for everyday use,If we WERE to go down the stability road, i'd say a minimum of 100 LinX tests utilizing at least HALF of your installed memory should be the standard, but I wont recomment going down that road.

Offline Nephilim

  • Sannin
  • *****
  • Posts: 2698
  • Country: 00
  • Chakra 56
  • Referrals: 0
    • View Profile
  • CPU: Q94
  • GPU: 4870
  • RAM: 8GB
  • Broadband: Flow
Re: GATT Benchmarking Development forum
« Reply #4 on: April 16, 2009, 12:59:01 PM »
i think stable is a relative term here.

what might fail a 20 pass at linpak might pass a 06/vantage. my system runs 15 passes and fails. but everything else runs without fail, and im hard pressed to decide...should i tweak more to get more passes stable? or should i be content that the system has been running fine, gaming, multitasking etc for the last few days?

So perhaps before we say 'it should be stable' we should define what 'stable' is for our purposes


^what he said

Carigamers

Re: GATT Benchmarking Development forum
« Reply #4 on: April 16, 2009, 12:59:01 PM »

Offline W1nTry

  • Administrator
  • Akatsuki
  • *****
  • Posts: 11329
  • Country: tt
  • Chakra 109
  • Referrals: 3
    • View Profile
  • CPU: Intel Core i7 3770
  • GPU: Gigabyte GTX 1070
  • RAM: 2x8GB HyperX DDR3 2166MHz
  • Broadband: FLOW
  • Steam: W1nTry
  • XBL: W1nTry
Re: GATT Benchmarking Development forum
« Reply #5 on: April 16, 2009, 01:39:31 PM »
Ok, now we don't want to have TOO much variety in the verification programs since standardization I think is a must. What about Prime95 for stress testing stability? I for one also know that my system fails prime95 when OCed, hence just as with Capn and Linpack 10 vs 100 there has to be tolerences for what 'stability' means. We need a vote on if marginally stable system score are acceptable or if some semblence of stability should be shown. My OC fails prime95, but there s a built in stress test in AOD that I can runn 100% utilization for over an hour and it's ROCK solid. So we will have to devise what we consider 'stable'

E.g. run Prime95 or Linkpack or AOD for x minutes without reboot/failures. Run a PI calculation program (whilst not an app to 'stress' a CPU, it DOES reveal instability when the calculations fail or reveal errors). We'll address 'STABILITY' shortly. Let's continue to develop the verification app list first. I will add to the FIRST post the results of our discussions.

Offline Netizen1

  • Global Moderator
  • Kage
  • *
  • Posts: 1764
  • Country: tt
  • Chakra 10
  • save de Earth, iz de only planet with chocolate!
    • :ps3: :psp:
  • Referrals: 0
    • View Profile
    • Trinidad Christian Center
  • Broadband: :flow:
  • PSN: Netizen1
  • Steam: Netizen1
Re: GATT Benchmarking Development forum
« Reply #6 on: April 16, 2009, 02:42:31 PM »
How about we post the latest version along with the URL to acquire the said version?

e.g.

CPU-Z (1.50) http://www.cpuid.com/download/cpuz_150.zip
GPU-Z (0.3.3) http://www.techpowerup.com/downloads/1363/TechPowerUp_GPU-Z_v0.3.3.html

Offline TriniXaeno

  • Administrator
  • Akatsuki
  • *****
  • Posts: 18836
  • Country: tt
  • Chakra 14
    • :ps3::wii::xbox360:
  • Referrals: 35
    • View Profile
    • http://www.carigamers.com
  • CPU: Intel Core i7-2600K
  • GPU: Geforce GTX 680 2GB
  • RAM: 16GB
  • Broadband: :flow:
  • MBL: Nexus 5x
  • PSN: TriniXaeno
  • XBL: TriniXaeno
Re: GATT Benchmarking Development forum
« Reply #7 on: April 16, 2009, 03:29:36 PM »
The KISS approach to development might apply here.

Offline Kraeoss

  • Jonin
  • ***
  • Posts: 894
  • Country: 00
  • Chakra 8
  • Referrals: 0
    • View Profile
  • CPU: Intel Core i5 2500k @ 3.3Ghz
  • GPU: HD 4870 (512mb)
  • RAM: 8GB DDR3 Corsair vengeance
  • Broadband: Blink
Re: GATT Benchmarking Development forum
« Reply #8 on: April 16, 2009, 03:37:23 PM »
I dont agree with the system must be stable bit. If a man wanna do a max oc to post his ultimate score, he should be able to do so.
world records are made @ suicide lol so if it passes the bench then it's good even if it bsod's right after pics are taken lol
"Upgrades..." Neo, The Matrix Reloaded.

Offline Spazosaurus

  • Dr. Herp Derpington
  • Administrator
  • Akatsuki
  • *****
  • Posts: 7685
  • Country: tt
  • Chakra 52
  • Referrals: 3
    • View Profile
    • The Awesome Company
  • CPU: i5 3470
  • GPU: GTX 780
  • RAM: 8GB Corsair
  • Broadband: Blink 2Mb + Flow 20Mb
Re: GATT Benchmarking Development forum
« Reply #9 on: April 16, 2009, 04:43:28 PM »
Ive been out of the Prime game for at least a couple years now but I do remember enough to know that there is no specific test you can run that stops after a predetermined period of time. Linpack of LinX on the other hand is capable of that. Now im not pushing it down yall throats eh.

That said, I think 10 tests in Linx should be enough to prove at least low to medium stability.

That coupled with the gpuz and cpuz shots should be sufficient.

Fullstop.

Offline W1nTry

  • Administrator
  • Akatsuki
  • *****
  • Posts: 11329
  • Country: tt
  • Chakra 109
  • Referrals: 3
    • View Profile
  • CPU: Intel Core i7 3770
  • GPU: Gigabyte GTX 1070
  • RAM: 2x8GB HyperX DDR3 2166MHz
  • Broadband: FLOW
  • Steam: W1nTry
  • XBL: W1nTry
Re: GATT Benchmarking Development forum
« Reply #10 on: April 16, 2009, 04:52:57 PM »
You've raised another issue... GPU stability... you can complete 3DMark Benchmarks and suffer artifacting! that should not be counted... now here's where honesty comes into play as you'll have to either fess up or lower the clocks. I don't think insane conditions should be considered, meaning if you rig is border line stable (aka passes benchmarks and crashes or artifacts) those results should NOT be acceptable. If we do decide to go that route of accepting borderline stable, then I will insist that a run of no less than 5 passes should be made (aka a batch run with 5 repeats for EACH test). That would increase both the time the benchmark takes as well as stress on the system to at least show the results are SOMEWHAT REPEATABLE!

As was Originally intended we should also have performance based benchmarks, I like SiSoftware Sandra for Memory benchmarks (we'll ignore latency and cache) and perhaps HDtach for HDs. Audio is DIFFICULT to benchmark especially with the issues with Soundblasters and Vista so we'll ignore sound unless someone can recommend a 'turn-key' application for testing audio.

Last I would like 1 game benchmark (this will be more controversial) e.g. Farcry 2 has a built in benchmark app.

Offline Kraeoss

  • Jonin
  • ***
  • Posts: 894
  • Country: 00
  • Chakra 8
  • Referrals: 0
    • View Profile
  • CPU: Intel Core i5 2500k @ 3.3Ghz
  • GPU: HD 4870 (512mb)
  • RAM: 8GB DDR3 Corsair vengeance
  • Broadband: Blink
Re: GATT Benchmarking Development forum
« Reply #11 on: April 16, 2009, 04:54:00 PM »
occt 3.0 you have linx pack from IBT and a P95 derivative type test you can also stress your gfx card and psu lol
"Upgrades..." Neo, The Matrix Reloaded.

Offline Kraeoss

  • Jonin
  • ***
  • Posts: 894
  • Country: 00
  • Chakra 8
  • Referrals: 0
    • View Profile
  • CPU: Intel Core i5 2500k @ 3.3Ghz
  • GPU: HD 4870 (512mb)
  • RAM: 8GB DDR3 Corsair vengeance
  • Broadband: Blink
Re: GATT Benchmarking Development forum
« Reply #12 on: April 16, 2009, 04:56:26 PM »
crysis too but wrt the hdd benching not everyone has a raptor or a super fast un-fragged drive so... we could leave that out for now ?
"Upgrades..." Neo, The Matrix Reloaded.

Offline Arcmanov

  • Administrator
  • Akatsuki
  • *****
  • Posts: 10642
  • Country: tt
  • Chakra 126
  • Gamer/Enthusiast
    • :n64: :gcn: :dreamcast: :xbox360: :computer:
  • Referrals: 1
    • View Profile
    • Arcmanov's rig
  • CPU: Intel Core i7 5820K @ 4.3 GHz
  • GPU: 2 x [Gigabyte] GeForce GTX 1070
  • RAM: 4 x 8GB G.Skill.TridentZ RGB DDR4-2400
  • Broadband: :flow:
  • MBL: LG V20
  • Origin ID: Arcmanov
  • PSN: Arcmanov
  • Steam: Arcmanov
  • XBL: Arcmanov
Re: GATT Benchmarking Development forum
« Reply #13 on: April 16, 2009, 06:37:00 PM »
Everest is not free.  You will have to acquire that one.  There are some very nice memory read/write benchies
built into it.  Current version is 5.0.

As for the standard benching apps, I think its all been covered.

As for game benchmarks, we have to standardise the version of the game being used.
I think Far Cry 2 already has about 2 patches out.
Systems United Navy - Accipiens ad Astra


Offline Spazosaurus

  • Dr. Herp Derpington
  • Administrator
  • Akatsuki
  • *****
  • Posts: 7685
  • Country: tt
  • Chakra 52
  • Referrals: 3
    • View Profile
    • The Awesome Company
  • CPU: i5 3470
  • GPU: GTX 780
  • RAM: 8GB Corsair
  • Broadband: Blink 2Mb + Flow 20Mb
Re: GATT Benchmarking Development forum
« Reply #14 on: April 16, 2009, 06:38:36 PM »
I think we going a wee bit too far. W1n, I understand you point on artifacts, but there's no easy way for someone to prove that they dont have artifacts on a level of overclocked that they achieved. We just have to have faith our members that they wont post scores in which artifacts occur. We cant police every single last detail.

As for cpu stability, seeing as it can work with both AMD and Intel cpu's now, can everyone agree that 10 passes of LinX is sufficient in proving at least moderate stability and go with that for 'stability proof'?

Offline Nephilim

  • Sannin
  • *****
  • Posts: 2698
  • Country: 00
  • Chakra 56
  • Referrals: 0
    • View Profile
  • CPU: Q94
  • GPU: 4870
  • RAM: 8GB
  • Broadband: Flow
Re: GATT Benchmarking Development forum
« Reply #15 on: April 16, 2009, 06:52:35 PM »
yeah we going lil too far.

the futurmark tests
cpuz, gpuz with validation, and 10 pass linX should be good.

a game bench and prob Everest after and we shall call him..'george'

Offline W1nTry

  • Administrator
  • Akatsuki
  • *****
  • Posts: 11329
  • Country: tt
  • Chakra 109
  • Referrals: 3
    • View Profile
  • CPU: Intel Core i7 3770
  • GPU: Gigabyte GTX 1070
  • RAM: 2x8GB HyperX DDR3 2166MHz
  • Broadband: FLOW
  • Steam: W1nTry
  • XBL: W1nTry
Re: GATT Benchmarking Development forum
« Reply #16 on: April 16, 2009, 10:46:33 PM »
I don't have a raptor myself nor do I intend to get on or a SSD drive for that matter but as was originally the benchmark idea it was supposed to be the entire rig as the 'gaming' experience is best when one combines not just the best CPU and GPU but also sound and supporting hardware i.e. HD, Sound Card, speakers, keyboard & Mouse, Screen and even layout. I have scaled back tremendously from that rather rigorous undertaking and HDTack s a 'turn-key' benchmark for HD so I don't see a reason to leave it out.

Since Everest is NOT free, I suggest Sandra for the memory benchmark and stability that LinX that capn talking bout, once I can run it on the various AMD rigs that I have sure I don't see a reason to not use it as the stability verification engine.

Lastly PCMark Vantage test the PC overall to see how it performs in apps, gaming, everything. It's free to download and I don't see a particular reason why it can't be included. As Arc said, we'll have to decide on what game though, but I think synthetic benchmarks whilst great cannot replace actual IN GAME experience. So I vote FC2.

Offline TriniXaeno

  • Administrator
  • Akatsuki
  • *****
  • Posts: 18836
  • Country: tt
  • Chakra 14
    • :ps3::wii::xbox360:
  • Referrals: 35
    • View Profile
    • http://www.carigamers.com
  • CPU: Intel Core i7-2600K
  • GPU: Geforce GTX 680 2GB
  • RAM: 16GB
  • Broadband: :flow:
  • MBL: Nexus 5x
  • PSN: TriniXaeno
  • XBL: TriniXaeno
Re: GATT Benchmarking Development forum
« Reply #17 on: April 17, 2009, 12:15:39 AM »
FC2 as in the freely available FC2 DEMO that allows benchmarking???

I think these apps should be free and readily available.

What about having your cake and eating it too?

Why not have both a Amateur Benchmark Top Rig and a Hard Core Benchmark Top Rig list.

The Amateurs can get in the door by simply running 3dmark 2006. Simple and sweet. If the machine can survive the benchmark and screenshot taken with accompanying cpuz info, then voila, score accepted.

The Hard Core benchmark will be a more elite list, with rigorous testing as is being decided here.

We could look to doing something as an incentive for those who jump through all the hoops to earn a certified spot on the GATT Hard Core Benchmark Top 20 list. (An OC'er T-Shirt, Medocino Core amulet, Brazilian chick, etc...)

Offline W1nTry

  • Administrator
  • Akatsuki
  • *****
  • Posts: 11329
  • Country: tt
  • Chakra 109
  • Referrals: 3
    • View Profile
  • CPU: Intel Core i7 3770
  • GPU: Gigabyte GTX 1070
  • RAM: 2x8GB HyperX DDR3 2166MHz
  • Broadband: FLOW
  • Steam: W1nTry
  • XBL: W1nTry
Re: GATT Benchmarking Development forum
« Reply #18 on: April 17, 2009, 12:48:54 AM »
I was always an advocate for incentive for jumping through hoops, just never found the time to create a long, detailed guide to the benchmark. Even the suggestion for various catergories instead of just best of the best, at least giving credit to the older rigs, etc. We can still bring all this up here and then create appropriate threads for each.

Offline Nephilim

  • Sannin
  • *****
  • Posts: 2698
  • Country: 00
  • Chakra 56
  • Referrals: 0
    • View Profile
  • CPU: Q94
  • GPU: 4870
  • RAM: 8GB
  • Broadband: Flow
Re: GATT Benchmarking Development forum
« Reply #19 on: April 17, 2009, 07:21:00 AM »
all the vantages are run once and then pay for.

so you gonna have to use your acquire-no-jutsus

Carigamers

Re: GATT Benchmarking Development forum
« Reply #19 on: April 17, 2009, 07:21:00 AM »

 


* ShoutBox

Refresh History
  • Crimson609: yea everything cool how are you?
    August 10, 2022, 07:26:15 AM
  • Pain_Killer: Good day, what's going on with you guys? Is everything Ok?
    February 21, 2021, 05:30:10 PM
  • Crimson609: BOOM covid-19
    August 15, 2020, 01:07:30 PM
  • Shinsoo: bwda 2020 shoutboxing. omg we are in the future and in the past at the same time!
    March 03, 2020, 06:42:47 AM
  • TriniXjin: Watch Black Clover Everyone!
    February 01, 2020, 06:30:00 PM
  • Crimson609: lol
    February 01, 2020, 05:05:53 PM
  • Skitz: So fellas how we go include listing for all dem parts for pc on we profile but doh have any place for motherboard?
    January 24, 2020, 09:11:33 PM
  • Crimson609: :ph34r:
    January 20, 2019, 09:23:28 PM
  • Crimson609: Big up ya whole slef
    January 20, 2019, 09:23:17 PM
  • protomanex: Gyul like Link
    January 20, 2019, 09:23:14 PM
  • protomanex: Man like Kitana
    January 20, 2019, 09:22:39 PM
  • protomanex: Man like Chappy
    January 20, 2019, 09:21:53 PM
  • protomanex: Gyul Like Minato
    January 20, 2019, 09:21:48 PM
  • protomanex: Gyul like XJin
    January 20, 2019, 09:19:53 PM
  • protomanex: Shout out to man like Crimson
    January 20, 2019, 09:19:44 PM
  • Crimson609: shout out to gyal like Corbie Gonta
    January 20, 2019, 09:19:06 PM
  • cold_187: Why allur don't make a discord or something?
    December 03, 2018, 06:17:38 PM
  • Red Paradox: https://www.twitch.tv/flippay1985 everyday from 6:00pm
    May 29, 2018, 09:40:09 AM
  • Red Paradox: anyone play EA Sports UFC 3.. Looking for a challenge. PSN: Flippay1985 :)
    May 09, 2018, 11:00:52 PM
  • cold_187: @TriniXjin not really, I may have something they need (ssd/ram/mb etc.), hence why I also said "trade" ;)
    February 05, 2018, 10:22:14 AM

SimplePortal 2.3.3 © 2008-2010, SimplePortal