How To Analyze Data Using the Average

@Craig: Thank you!

Hi! You really did a great job on this. I really liked it. I just want to know how did you get the return of investments ? thanks

[…] the basic characteristics. Every footprint has a depth, width, and height. Every data set has a mean, median, standard deviation, and so on. These universal, generic descriptions give a rough […]

how to take average of probability values

Just wanted to add my voice to those thanking you for this article,. A marvelous explanation.

What is the function of the carat (^) in these examples?

If Portfolio A is 0.5% loss per year, wouldn’t Portfolio B be 4.25% rather than 4.6% loss per year? 17%/4
Portfolio A:
Return: 1.1 * .9 * 1.1 * .9 = .98 (2% loss)
Year-over-year average: (.98)^(1/4) = 0.5% loss per year (this happens to be about 2%/4 because the numbers are small).
Portfolio B:
1.3 * .7 * 1.3 * .7 = .83 (17% loss)
Year-over-year average: (.83)^(1/4) = 4.6% loss per year.

@Rick: The carat represents an exponent, so 2^3 = 8.

When measuring the “average return per year over 4 years”, we can’t just do a division(17/4) because the effects compound. 17/4 = 4.25, but (1 - .0425)^4 = .84 (i.e., 4.25% applied 4 times results in an 16% loss, not the 17% we were expecting).

@Luis: Glad you liked it!

Hi, the whole thing is going sound here and ofcourse every
one is sharing facts, that’s really good, keep up writing.

Having read this I thought it was very enlightening. I appreciate you taking the time and energy to put this informative article together.
I once again find myself spending a significant amount of time both reading
and commenting. But so what, it was still worthwhile!

Hi there just wanted to give you a quick heads up.
The text in your post seem to be running off the screen in
Chrome. I’m not sure if this is a format issue or something to do with internet browser compatibility but I thought I’d
post to let you know. The layout look great though!
Hope you get the issue fixed soon. Cheers

Hey there! Do you use Twitter? I’d like to follow you if that would be ok. I’m definitely enjoying your
blog and look forward to new updates.

pros and cons of range?

Loved the interchange between Kalid and G in posts 35 to 41!
It’s nice to see how much good can come from disagreement when the discussion is focused on the issue at hand, not how ‘wrong’ the other person is.
I just finished a post in the statistics section and found this quite interesting. My first thought was: 'You’re both giving different answers, and you’re both right, because you were answering different questions.'
To me the differences between 2 machines in series and 2 machines in parallel was obvious because I’m accustomed to looking for it. In the verbiage, however, it seems that other details were in the forefront and made the two scenarios seem the same. The distinction between pipelining and series data transfer, or simultaneous production vs. ‘one to completion’ then begin the second, became apparent after a bit of back and forth. It’s more evidence that even the correct application of mathematics is useless without good understanding of just what mathematics applies to our situation and why.

From my post in the Statistics section I’ll only repeat my Rule #1 for stats:
Your results don’t mean anything until and unless they mean something.

@Steve

I’ve never seen that particular algorithm. To the best of my knowledge it is not a standard method used in mathematics for any measure of central tendency.

It may, however, have some use in some scenario; without further details of where you saw it I couldn’t say. I can tell you what it will give you if you use it and why it appears ‘moody’ as you call it.

First just looking at it for what it is, it provides something like an average for a rolling data set. That means if you are watching something that is changing, growing, progressing through time, or in any other way continuing to spit out numbers at you, and want to know the average, you can use this method to retain one number (and no other information about what has come before) to get a sense of the average to compare to your new piece of data. This ‘Steve average’ just places great emphasis on the one new piece of data and tends to downplay all older data equally, even the one previous piece of data.
Here’s a quick breakdown of the Steve Average (SA):
For 2 elements: each contributes half of its value toward the SA
For 3 elements: final element contributes 1/2, each previous contributes 1/4, all the previous elements together contribute 1/2
For 4 elements: final element 1/2, each previous 1/8, all previous, 3/8
For 5 elements: final element 1/2, each previous 1/16, all previous, 4/16

For 20 elements (# of students in a class?):
final element 1/2 contribution to SA
each previous contributes 0.0000001 of its value towards SA
all the first 19 together contribute 0.000019 of their combined value towards SA

Conclusion:
As you watch the Steve Average while new numbers pour in, the SA is nothing more than just half of whatever value was the most recent with almost no regard to any other data,even the one most recent data point. This conclusion gets stronger as the number of elements grows but with as few as 7 elements the final element makes up over %90 of the entire contribution to the SA. The SA has the benefit that it is easy to calculate and only requires keeping memory of one number for all previous data.

If that is the behavior you’re looking for then it is useful. If you agree with me that this behavior doesn’t help provide any meaningful analysis of the real world then it is not useful.
Please don’t ever be afraid to ask a stupid question, for the only stupid question is the one you don’t ask. And do not discount a theory or method of analysis until you have looked at it for what it really is, and what benefit or behavior it provides. Only then can you know if it is useful.

By the way, there does exist another method for keeping track of a rolling average that is mathematically equal to the arithmetic mean. It is almost as easy to calculate as the SA and it only requires memory of 2 numbers for all previous data.

  • keep track of running average and n= number of elements
    -when new number comes in do the following
    -multiply old ave by old n
    -add new data
    -divide by new n

Excelsior,
Eric V

Great post. Clearer than most. :slight_smile:

Wondering if you can help me clarify something that I don’t seem to be able to classify:

The question came up recently about an average that I found use for a long time ago, but it’s been so long I can’t remember why. Boring details aside, it worked like this:

  1. Take sample
  2. Is this the first sample?
    a. Yes? Set it as the average. Get next sample.
    b. No? Add it to the old average and divide by 2. Get next sample.

What would this be classified as? It provides a much moodier output than the standard mean algo.

@Eric V

Ok, cool thanks. Smoothing rings a bell, basically a filter. I’m no mathematician hehe but when intrigued, I tend to hunt things like this down. Thanks :wink:

The only way I can make sense of the harmonic mean is resistors in parallel. My way of explaining it is adding up areas (reciprocal of resistance) of water pipes.

I used to ask, how to find the geometric mean of a continuous function. It’s called product calculus, where the product derivative, y* = exp(y’/y), can be seen as an instantaneous interest rate. The geometric mean of f(x) between [a,b] is $exp(\int_a^b \ln f(x) d x/(b-a))$. I derived the formula by myself but it has been discovered by Volterra a century ago, even though its not in calculus textbooks.

Googling leads me to finding the root mean squared and harmonic mean of a continuous function.


https://www.google.ca/url?sa=t&source=web&rct=j&ei=Z0WyU8iCKpH4oASzy4DQCA&url=http://www.gauge-institute.org/calculus/PowerMeansCalculus.pdf&cd=1&ved=0CBoQFjAA&usg=AFQjCNGejNgqZjaLjf_Z4RiE43-tp3Lk_w&sig2=3q4lx8h7FcZe87AdoqwfHA

Comment from a reader (Michael):

Hello Kalid,

I love your site and approach to maths. However, how do you get the numbers (rate of return) 1.1 * .9 * 1.1 * .9 = .98 (2% loss)) in regard to Portfolio A?

And
(1.3 * .7 * 1.3 * .7 = .83 (17% loss)) in re to Portfolio B?

I don’t understand how you derived those returns? Thank You, Michael DeYoung

*also, with +10 -10 +10 -10 shouln’t that = 0?


Hi Michael, great question. I should probably clarify the post.

A 10% return is the same as turning $100 into $110, and we can model this by multiplying by 110/100 or 1.1

A -10% return is the same as turning $100 int $90, and we can model this by multiplying by 90/100 or .9.

If you had a portfolio that gained 10%, then lost 10%, it would be

Start: $100
Year 1 (gain 10%): $100 * 1.1 = $110
Year 2 (lose 10%): $110 * .9 = $99

Something interesting happened – when we lost 10%, we went from $110 down to $99, not back to $100! This is because losing 10% after you’ve grown is worse than a 10% on your original amount. So we end up lower than before.

If we continue that pattern:

Year 3 (gain 10%): $99 * 1.1 = $108.90
Year 4 (lose 10%): $108.90 * .9 = $98.01

The same thing happened: we gained 10% on $99, then we lost 10% on $108.90, and that loss took out a larger chunk since it happened on the larger amount.

So, the strange effect is that gaining and losing the same percentage will eventually bankrupt you :). It’s because your losses are happening on the larger amount.

We might think “Ok, what happens if we have the loss first?”

Start: $100
Year 1 (lose 10%): $100 * .9 = $90
Year 2 (gain 10%): $90 * 1.1 = $99

Uh oh. Even having the loss first doesn’t help. It means we lose a large amount (losing 10% on $100) but then only make back a smaller amount (gaining 10% on $90 is only $9, not $10, and doesn’t cancel out the loss.)

This seems unfair, right? Shouldn’t zig-zagging keep you even?

Thinking about it more, I realize there’s an unfair element here:

When the gains grow you, the losses get bigger. The gains are actually “helping” the future losses become even bigger.

Now, the silver lining is that losing 10%, then 10%, then 10% isn’t as bad as losing 30% all at once:

Start: $100
Year 1 (lose 10%): $100 * .9 = $90
Year 2 (lose 10%): $90 * .9 = $81
Year 3 (lose 10%): $81 * .9 = $72.90

We “only” lost 28%, instead of the expected 30%. That’s because the losses are shrinking the amount we can lose, so they work against themselves that way too.

Phew. Hope that helps clarify – some of this can be really unintuitive, and we have to go through the numbers vs. trying to estimate internally.

hi: u xplained the stuff in a very better way.can you please tell me that from the given set of data how can we analyse which type of average among arithmetic,geometric or harmonic mean is better to be used?