Zumwalt Poems Online

Posts tagged ‘Data Analysis’

Thoughtful Thursday: What to observe?

“The more we look, the more to see: altering this reality”  — Zumwalt

In order to interpret, evaluate and ultimately understand, one has to observe.

Concept3

However, there is so much out there to observe.  It’s overwhelming.  So we run into an apparent paradox: one must know, to some degree, what to observe, yet one cannot know without first observing, interpreting, recognizing, filtering, connecting, comparing, evaluating and have some level of realization.

And so one cannot ever start from scratch. There will always be some initial observation from earlier or from others that one relies on in order to determine what next to observe.

Observing then becomes a process of narrowing down and expanding what one observes.  To be efficient this is not a random process, but one driven by the formulation of an objective.

For example, “I need to eat” is an objective.  With that objective one then observes — or collects data — in areas that will be more likely to provide the necessary input to increase one’s success at an action designed to obtain food for eating.

If we are a retail chain,  like Kohl’s, Macy’s or Sears, we must determine our objective.  Is it to increase revenue, lower costs, or increase profit margin.? Or is it a less basic objective like increase in-store traffic or increase brand awareness which may be considered an objective in order to achieve the more specific objective of increased revenue through building customer loyalty?

Having a hypothesis can help narrow down what to observe, but a hypothesis should not be formulated too early in the process or opportunities to achieve an objective can be missed.   Instead it is best to look for patterns in the data, and then determine what is a likely hypothesis based on that data as opposed to starting with an hypothesis and thus missing the opportunity of identifying more likely hypotheses.

For example, if my objective is to cross a river and I start with the hypothesis that one can cross the river on a structure that floats on water, I may miss out on noticing that 200 yards downstream someone has built a bridge, or the river becomes shallow enough to cross on foot, or that the river becomes an underground stream.

So important to keep the objective in mind, identify what data can be collected (what can be observed), and not prematurely limit the data that one will analyze.

With companies like JC Penney and Macy’s currently fighting against declining same- store sales, it seems like they missed out on observing several years ago how Amazon was increasing book sales and ultimately positioning itself to sell other items online — items that were also sold in shopping mall department stores.  What were they observing? What was their hypothesis?

Five year stock chart for JC Penney
jcp

Five year stock chart for Macy’sm5y

At this point, have these companies learned the hard way what data they need to look at?  (If so, now may be a good time to buy their stocks.  If not, expect further losses for JCPenney and further same store sale declines for Macy’s.)

There is a lot out there to observe.  The first step is to clearly understand one’s objective and then focus on observing those things that will help achieve that objective.  For music, if one wants to dance, observe (listen to) the beat  — this means focus on the drums and the bass, If one wants wants to play along, focus on what the chord progressions are for the verse and chorus.  If one wants to ignore the music, focus on something besides the music.

Don’t worry about formulating an hypothesis until we has examined enough relevant data for that intended objective to see what are recurring patterns.  Fortunately in the data analysis world there is software (Artificial Intelligence, Machine Learning) that can identify patterns in the data.  Without software, one can take notes, reflect and follow the steps in my diagram below: observe a significant amount of data, recognize what are the patterns that appear relevant to your objective, filter out the irrelevant data to focus on that data that has the patterns of interest, evaluate that data, understand its nature and then form a plan of action to gather or observe more of that data as necessary.

Concept3

Eventually one can form a hypothesis and scrutinize that hypothesis each time one goes through the “comparison”, “evaluate” and “realize” steps to know if that hypothesis is correct or not.  If that hypothesis is not appropriate, then one needs to formulate a better hypothesis that is aligned with previously collected data and will predict future observations.  There is no such thing, though, as a correct hypothesis — there is only a workable hypothesis — one that provides the necessary guidance to act successfully. Where people get in trouble is when they stubbornly or subconsciously hold on to a premise or hypothesis that is not successfully predicting outcomes and then continue to act on that premise or hypothesis as if it was reliable.

No belief is important enough to hold on to when it doesn’t align with verified observations. Such beliefs end up doing everyone more harm than good.

Advertisement

Thoughtful Thursday: Preconceptions

most_miscon

“You must challenge your preconceptions or they will challenge you” — Penny A.Proctor (You may hear this first quote in the upcoming new Star Trek series on CBS taken from a short story by Penny A. Proctor)

“Mistakenly thinking you know usually results in not knowing mistaken thinking later on.” — Zumwalt

IQ tests sometimes contain questions that, though not particularly hard, will not be answered correctly if one falls back on preconceptions rather than solving the problem on its own terms: doing the work rather than taking a short cut by falling back on what one accepts as given truths

A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball. How much does the ball cost?

If one starts with preconceptions, one goes for the easy answer.  If one just works through this without jumping at the misleading “short cut” then one gets the correct answer.

If one even checks their preconceived wrong answer, then they will soon have the right answer.  The wrong answer of 10 cents for the ball doesn’t stand up as 10 cents for the ball, plus one dollar extra than the price of the ball for the bat at $1.10 sums up to $1.20.

One way to solve the problem is rely on simple algebra:

x (price of ball); y (price of bat)

x + 1.00 = y  –> y-x = 1.00

x + y = 1.10 –> y+x = 1.10

Add these 2 equations up and we get 2y – 2.10  or y = 1.05 so the bat is $1.05 and the ball is .05.

Even if we don’t use algebra but try a few guesses, checking our guesses will eventually get us to the right number.

But the way that life works for most of us, most of the time, is that we make assumptions and never ever check them, making decisions on top of bad assumptions.

Bill is stalking Mary. Mary is stalking John. Bill is married, but John is not. Is a married person stalking an unmarried person?

  1. Yes
  2. No
  3. Cannot be determined from the given information

Once again, if we fall into the trap of lazy thinking, we may never work through this problem finding the right answer and end up agreeing with 80% of people that choose answer #3. It really doesn’t matter that we don’t know Mary’s marital status.  If she is single, it is then Bill, the married person, that is looking at an unmarried person, and if she is married, then Mary is the married person looking at an unmarried person.

We rarely object to an instance of a person saying “that’s my spouse” (“that’s my wife”, “that’s my husband”) or “that’s my child.”  And so, it’s no issue to say “that’s my dog” or “that’s my cat.”  But we would object to saying “I am the owner of this husband” or “I am the owner of this child.”  Should we also object to one saying “I am this dog’s owner.” Does one own the dog or is it a partnership? If the dog runs away, the “owner” can get the pet back.  The “owner” can decide to trade the dog for money or put the dog to sleep. In ancient Greece and in 19th century America, people owned other people, and could trade them and even put them to death.  The people in the past re-examined ownership of people and determined this was unacceptable in any way, shape or form.  Should we examine our relationships with our pets, or just accept our preconceptions and continue on.

Anchoring is a sales and marketing ploy that takes advantage of our tendency to have preconceptions.  This is very common in the cruise industry where cruise lines offer “limited” 2-for-1 permanent cruise sales or have a brochure starting price that’s double (or more) than the actually offered price:cruise3

Preconceptions are the most troublesome when we don’t know about them. In doing data analysis it’s the bad data that we don’t know about the messes up the results.  Any belief that one doesn’t examine for its accuracy (maybe not examined as one isn’t aware of the belief or thinks it is correct) will influence one’s final decisions and analyses, often making actions based on such decisions and analysis worse than random actions. Anchor prices are used to take advantage of our tendency to lock on to the earliest data we have about something and to subconsciously put more weight or more credence in the earlier or earliest obtained data than later data, even though analytically we often chose to prefer more recent data over older data.

Take the time to examine what data is used to make an important decision and challenge the validity of that data.  By identifying the bad data or erroneous preconceptions and not using them to base a decision on,  one can significantly increase their likelihood of acting appropriately.

 

 

 

 

 

 

 

%d bloggers like this: