Big Data, Flawed Data, Business Intelligence, Where’s The Future and What Has Been Our Past…A World With ”Algo Duping” of Society and Consumers

You see the items mentioned in the title in the news every day and granted there’s a lot of good stuff out there for sure that makes us smarter, but as you see in the news every day we are getting down to the core of how some of the business world, especially in the financial world has operated and profited from “flawed” data.  I called flawed data the information used by algorithms that were created for “desired” results versus “accurate” results. The two should be the same but they are not always. 

What has been big news of late is the Libor scandal with the many flaws listed in how the rates were set. How many consumers, businesses were affected by this?  We really don’t know but in the pursuit of the truth we are certainly finding out how some benefited with millions or billions in profit. It is so “flawed” that banks don’t even loan to each other or trust each other and who knows better than those creating the formulas.  There are some very intelligent individuals in the US and those are the ones with the money and this took a lot of time to build as one scheme was never challenged, so others jump in with their own and again as we read today, nobody gets held accountable most of the time.

Big data can really be a good thing but again we come back to the old saying of “garbage in and garbage out”.  You can take credible data and really get some great information and knowledge if queried properly with information that has not been spun for marketing.  Does that data exist today, sure it does but the problem of figuring out what is “accurate” versus “desired” is a chore and a bit of work.  Programming is just like anything else in life on which path anyone chooses, you can work to create solutions and use credible data or you can choose the alternative route for those “desired” results and probably pocket billions of dollars.  The average consumer has no clue on how this has evolved over the last few years but algorithms move and make money.  You can’t see the algorithms, talk to them, etc. and some go rouge here and there. 

We have government agencies that have been spoofed as well with some of the flawed data that results from the “intelligence” that emerges from business intelligence that is working for the “desired” results that may not be totally accurate.  Again it’s hard to tell the difference at times and when you spin some of the information multiple times and use the old methodology of repetition, we start believing that some of the flawed data is the truth until someone proves otherwise and thus we continue with the world of insanity to where it starts getting out of control as the claims are way beyond what is commonly known to be true.  Getting to the bottom of what really is the truth is becoming a big challenge. 

It really gets interesting when a source of “flawed” data is challenged and we want answers.  What happens?  When the data and media spin it to the point of being out of belief, those folks go back and create more formulas and reports to substantiate the past and some of it does get down the the truth and some of it is just a further “spinning” of information to substantiate the original questions and of course this is the hot topic when formulas and algorithms have either denied consumers or made big profits on conclusions of data queried that are not accurate.  I think we all know too that journalists have to meet certain ratings to keep their jobs so again not to fault some of the reporting out there as they need their jobs too but does this impact what we get out there, sure it does and if I had to make sure I kept my job I would probably be doing the same thing. 

We are seeing more and more of the second methodology being used as people and companies of course want save face and substantiate their actions. If you look at the entire financial mortgage business, none of it would have been possible without the use of technology and the pursuit of “desired’  results, which in that case made billions for banks and other businesses.  Predictive queries are of course all the rage and some of it is pretty good but some of it is garbage too,  You can read the post below on how data mined on the web that is perhaps not verified as credible from social networks and other areas hurts people as items get connected to create more “flawed data” and people are judged by this and don’t get jobs for one example and those very same folks have to spend hours, days, etc. cleaning up “flawed data” that they did not create. 

Flawed Data–Mined by Corporations Online Provides Background Checks Riddled With Errors–Attack of the Killer Algorithms Part 7

We all want transparency but do we want “flawed” transparency?  It certainly makes for big news headlines all the time when money is involved and yet it’s just one more case of “desired” results battling against “accurate” results out there. 

Here’s one of the most notable examples of “flawed data” created to make a buck as well and even most consumers understood this process as not being in the pursuit of “accurate” results but rather “desired” to make money.   CoreLogic has formed a partnership with FICO who is already in the process of selling algorithms with mismatched data which connects public information about you and combines it with your credit score to tell if you as a medical patient will take your prescriptions.  Somebody needs to start calling some of these folks on “mis matched data” the discriminates as now we have “Discrimination by the Algorithm” and it’s showing in more places all the time.  As far as I am concerned the FICO Medication Adherence Scoring is nothing but mismatched data analytics created to sell software and is the work of some “underground” think tanks on how to generate more money and profits.  It makes one wonder how gullible the pharma and insurance business if if they buy into this and pay money for such analytics?

”Hey dude let’s crunch some numbers and see if we can come up with some analytics to sell”

Reports and analytics for profit versus accuracy is something that is out there digging around in more ways than one.  Sure sometimes you can be crunching numbers and run across something has does have some real outstanding value when looking at groups of people but the weird part of some of this now is that folks are taking crunched numbers that offer some intelligence and bringing it down to the individual level to “score” and rate people and this is the problem as there’s data in there used for predictive forecasting that is not credible. 

I became fascinated with all the “flawed data” out there a couple years back when I found my former doctor who had been dead for 8 years still listed on many of the doctor/hospital “compare” sites…yes how does a dead doctor stack up compared to peers…is this valuable?  I don't think so unless I would feel that my doctor could still function as a physician from the grave.  The AMA even chatted with me about this area of “flawed data”. 

HealthGrades And Other MD Rating and Referral Sites List “Dead Doctors” on Their MD Information Pages And Even Include the Insurance Plans the “Dead Doctors” Honor

We see all the reports on ranking hospitals too.  Long and short of all of this is that it creates jobs for people to do this and a lot of it is still human generated and I see it as opinions as you are comparing apples to oranges.  While it is certainly acceptable for any hospital to blow their own horn on procedures, research etc. that they are doing, and they really should, but do we need the statisticians that create these opinions?  In my opinion it makes for clutter confusion for the consumer on the web as it all depends on who’s opinion you want to buy in to.  Combining data for comparison is certainly ok but can leave out the rankings?  I had to laugh yesterday at all the articles out there about Johns Hopkins slipping to number 2 with all the additional opinion pieces this created.  A couple weeks ago I said the rating game is getting to be old news and somewhat useless as more “flawed data” enters the picture and it depends again on who’s opinion you want to buy in to.

HealthGrades Puts Out Top Hospital List–Time to Drop All the Hospital Ratings Sites As Nobody Cares, Many Don’t Have Time To Read and Too Much “Flawed Data”

Did this report do anything..not really..just created a lot of chatter and took the focus off of the good and great things that hospitals really do.  What would happen if all the “rating games” went away?  Probably not much other than the fact that those who create the numbers would have to look for something else to do:)  When it comes to the business side of what Johns Hopkins does, stories like this link below are certainly contain a bit more of what I liked to read as it shows business and the healthcare world working together. 

The Johns Hopkins Center for Population Health IT Is Created to Improve and Expand Use of Medical Records

Now with big data making big moves, one has to wonder are there any levels of control here so we can keep some of the flaws out and get some real intelligence without the algorithms for “desired” results conflicting with the ones for “accurate” information?  It’s making people crazy and we have heard the stories about brokers committing suicide, etc. when they are questioned as most of them don’t really understand the processes they used but they become millionaires doing it.  We had the man who did the drama queen case and took a poison pill on national TV and then yesterday I ran across this USB broker who used sling shots to deal with his stress.  What are we doing out there and what are we creating?

Stress On the Job, USB Broker Takes It Out With Sling Shots Breaking Windows in Beverly Hills and Encino Gets Arrested–No We Are Not All Mentally Ill, Just Those Who Have the Wrong Financial Algorithms Crawling Up Their Back Side

I think the truth of the matter relative to what we are reading is making us crazy as via repetition folks are in some areas trying to sell us on what we know is not true as they created statistics, reports, etc. in order to sell some of this for money.  Remember when Colbert and Stewart had their big show about bringing back sanity?  Guess what that’s what their show was all about in the roots of what they were trying to say.  Shoot they did this show back in 2010.

Rally To Restore Sanity–A Big Success With Some Real Straight Talk About The Disruption and Distraction We Live With Today

Read this book for some hints on what stats and numbers are all about and ask yourself where you might have been sucked in, happens to all of us and it substantiates a lot of what I have said here as it does exist.  The Dark Arts of Mathematical Deception are alive and well so try and figure out if or where you have been duped with formulas for profit that cross the line of ethics. 

“Proofiness–The Dark Side of Mathematical Deception”–Created by Those Algorithms–New Book Coming Out Soon

Three years ago myself I said this, again seeing some of the insanity that has been created to make and take money.  I wrote this up during the Madoff discovery when the SEC has not appeared to appreciate math and computer formulas yet and the teeth that they have. 

“Department of Algorithms – Do We Need One of These to Regulate Upcoming Laws?

So in summary do I like technology?  Yes I love it and the good things it creates but I’m really tired of the “Algo Duping” that is taking place and that is somehow how I got on the case of writing my posts on the “Attack of the Killer Algorithms” to hopefully bring an awareness around.  Do all the rich folks and politicians understand this themselves?  Probably not but they have enough money to hire some “Algo Dupers” to keep their riches in place.  What happens when formulas go rogue and get out of control, look at the Facebook IPO for one example and items as such are not going away any time soon.

---

Attack of the Killer Algorithms–Digest & Links for 35 Chapters–on How Math and Crafty Formulas Today Running on Servers 24/7 Make Life Impacting Decisions About You–Updated 6-24-2012

 

Facebook IPO – The Ultimate “Attack of the Killer Algorithms” Chapter 32 - Nobody is Immune In the World of Complicated Computer Code and Formulas Today

---

I thought I was the only one talking about this but recently was contacted by the National Institute of Statistical Sciences that told me to keep blowing the horn and that someone will eventually listen.  It’s all about the math, algorithms, data, and “flawed data” that creates markets sometimes where they don’t belong, so this is what I am doing. 
It’s really looking like “the world does belong to the geeks” as nobody else wants it in the words of Robert Scoble, the king of social networks.  BD

As one more side note here’s one more post you might find of interest on how criminals are also using technology that will also make you think and the same thing is said here, those who control the code control the world.  So again give this some thought as you live your life today and be aware of the “Algo Dupers” that live among us and they sometimes blend in too well with those who are trying to create good solutions for all of us.  When big profits and billions of dollars are up for grabs with creating “desired” versus “accurate” results, well you kind of see what we have out there today.  BD 

A Vision of Crime in the Future–TED Video By Marc Goodman–”If You Control the Code You Control the World” And A Deep Look at DNA Use by Criminals