Author Topic: GATT Benchmarking Development forum (Read 40119 times)

Kraeoss · « **Reply #20 on:** April 17, 2009, 08:21:56 AM »

i need a new monitor....

Carigamers · « **Reply #20 on:** April 17, 2009, 08:21:56 AM »

W1nTry · « **Reply #21 on:** April 17, 2009, 09:23:34 AM »

Quote from: Nephilim on April 17, 2009, 07:21:00 AM

all the vantages are run once and then pay for.

so you gonna have to use your acquire-no-jutsus

ohho...

TriniXaeno · « **Reply #22 on:** April 17, 2009, 10:16:22 AM »

That won't be a problem for the Hard Core guys, I'm sure.

W1nTry · « **Reply #23 on:** April 17, 2009, 10:20:50 AM »

Well thing is, the 'acquisition' is simply a key. As you have the FULL application from the free download. It's the key that you pay for... I am sure almost everyone can 'acquire' the key and if not, they can ask someone who can. That aside (not that I am condoning piracy or anything like that XD). I have been using OCCT on recommendation in this very thread and I am finding it to be BOSS. I endorse this app and it's free. But not only that it has multiple stress tests in it 2xCPU, 1xGPU, 2xMemory and it also shows temperatures, CPU utilization AND it produces GRAPHS showing system readings throughout the run (temps, voltages) OCCT gets W1nTry Stamp of approval as the GATT defacto stability app!!! the only question is the level of the stress test... there's 3 settings in the Linpack run low, medium and High. I have been using Medium (50% memory) for 16 min runs. What say ye?

Nephilim · « **Reply #24 on:** April 17, 2009, 10:35:37 AM »

the new OCCT using linpak?

i used to run OCCT foir about 12 hours, if system pass i say stable.

the entire move to the linpaks was to reduce testing time. 12 hours vs about 15mins was a huge +.

if OCCT doing all that and then some i say +1 to it.

Carigamers · « **Reply #24 on:** April 17, 2009, 10:35:37 AM »

Kraeoss · « **Reply #25 on:** April 17, 2009, 10:37:35 AM »

finally a man using de truth hehe... yea 10 min lin pack run is enough for stability confirmation i did a 72 hr and it fared just as well

W1nTry · « **Reply #26 on:** April 17, 2009, 05:44:56 PM »

I've spent my share of time and effort to put this forward as an example of the results we could use:
http://www.carigamers.com/cms/forums/index.php/topic,5808.msg209816.html#msg209816

Arcmanov · « **Reply #27 on:** April 17, 2009, 06:32:59 PM »

I keep seeing the word 'validation' being thrown about.

Please elaborate on this.

W1nTry · « **Reply #28 on:** April 17, 2009, 06:38:47 PM »

Validation refers primarly to showing screenshots that show you have the hardware you claim to have. Appended to that and still subjective is validating that the settings you are running are 'stable' and can be REPRODUCED.

W1nTry · « **Reply #29 on:** June 16, 2009, 03:11:59 PM »

It's come to my attention that every now and again that 3DMark scores can be close, how close? well within 100 pts. The question I pose is how much of a difference in score can be attributed to random skew? many review sites throw deviations in score out the window and as such considering we too would suffer the sme skew what should we consider reasonable skew? I suggest 100pts.

Thus should a user have a score within 100 (inclusive) pts of another it is considered a tie. And with such scenarios there would be no place about it i.e. tie for 12th means no 13th, next placing would be 14th.

What do yall think?

Also to reduce skew I suggest we introduce repeat runs at least 3.

Spazosaurus · « **Reply #30 on:** June 16, 2009, 09:12:08 PM »

I move that for any two (or more) members whose score difference amounts to 99 points or less, a tie be declared between (among) said members bla bla just like w1n said.

Repeat runs at least 3? Please clarify. If you mean that members run 3 passes of 3dmark...sure, I guess, but that would not solve the problem as it still remains very possible that even after 3 runs the scores are still within 100pts of each other and can just as easily land within less than 100pts of the higher/lower ranked member.

W1nTry · « **Reply #31 on:** June 16, 2009, 09:44:35 PM »

Yes I meant 3 passes. Whilst it will not eliminate the 100 issue it should reduce 1 off freak scores which DO happen from time to time. It adds the idea of REPEATABLE scores. I waiting on Arc input and men like n00b and others.

Nephilim · « **Reply #32 on:** June 16, 2009, 09:55:12 PM »

3Dmark has a setting to do this automatically, so its not really an inconvenience. run a few times then take the avg.

This is under the settings option in 3Dmark06 (2nd from left, button to the bottom right)

I'm for it, if only for the purpose of statistical/scientific accuracy.

Arcmanov · « **Reply #33 on:** June 16, 2009, 10:15:54 PM »

* Concurs *

If the scores are STILL within 100 pts after 3 loops, then tie it is.

...but we running the risk of making this thing too complex now.

TriniXaeno · « **Reply #34 on:** June 17, 2009, 02:11:13 AM »

what arc said.

I think the complexity is best left to the hard core benchmark list. (Did that ever get off the ground btw?)

The current list is simple and sweet. If one person's rig remains higher (even by one point) after the triple run, why not give them the higher spot? Margin of error or not.

For example.

#1: Arcman Fitness Rig 2010: 45,001 3D Marks
#2: NoobgoneWild GTFO Ultra Rig: 45,000 3D Marks
#3: Redfish WSHB PC: 40,000 3D Marks

So yeah, my rig is pretty much a tie to arcs but #1 and #2 still looks a lot easier on the eyes than this:

#1: Tie!!!! Arcman Fitness Rig 2010: 45,001 3D Marks & NoobgoneWild GTFO Ultra Rig: 45,000 3D Marks
#3: Redfish WSHB PC: 40,000 3D Marks

Spazosaurus · « **Reply #35 on:** June 17, 2009, 07:08:06 AM »

Just out of curiosity, what does WSHB mean?

As you put it like that Noob, you make a complelling point, but its highly unlikely that we'll have a situation like that. Even if two people have IDENTICAL hardware, it still doesnt guarantee that they would both get the same score.

W1nTry · « **Reply #36 on:** June 17, 2009, 07:22:56 AM »

I agree with capn. If you look at the scores now there is USUALLY a decent margin of difference between each score. So when similar (within 100) pt scores occur I think it fair to have a tie.

Kraeoss · « **Reply #37 on:** June 17, 2009, 10:25:47 AM »

so in other words i train hard and long on the OC track tighten belt an thing buy meh l33t parts and then if i get a score then somebody else who is clearly behind after all the heats by say 1 pt then we tie ? that makes no sense, i say is better yuh make a top 50 instead cause to me all is fair to who passes the mark at it's highest point and if all that separates us is a 1-2-99 marks then well then be a good sport and take yuh proper place. no ties... if my system and arc system score 45000 3Dmks after 3-6 runs then that's a tie

Netizen1 · « **Reply #38 on:** June 17, 2009, 10:36:12 AM »

I don't think we need more than a top 20 as yet. There's not enough movement on and off the list to warrant it.

However, someone did suggest we keep a history of past performers somewhere, so we can look back a year from now and say, wow that was all my old rig was capable of?

Unfortunately, not everyone started their own Benchmark thread to keep such a history (per user)

W1nTry · « **Reply #39 on:** June 17, 2009, 10:36:44 AM »

You've missed the key point here kraeoss, ALL review site bar NONE allow for skew. You can run 3Dmark today for 10 passes then tomorrow not be able to reporduce that score. It may vary up, down or sideways. To begin with score SHOULD BE REPEATABLE and as such you EXPECT that the score will VARY with a certain skew (I suggested 100pts). You system SHOULD be able to repeat results within a 100pt Skew of even in a months time.

The take home here is if a man beat you by 1 pt today i'd bet you a doubles that tomorrow he could get a lower score and you'll beat him. Which makes NO SENSE. Thus this is to better validate the scores and give ppl a fair playing ground.

So far the odds are
Capn, W1n, Arc FOR
n00b, kraeoss Against.

The next question would be to put a poll to the general gatt populace once the staff can agree or disagree on this issue.

Carigamers · « **Reply #39 on:** June 17, 2009, 10:36:44 AM »

Author Topic: GATT Benchmarking Development forum (Read 40119 times)

Carigamers

Carigamers

Carigamers

ShoutBox