29 posts categorized "analytics & machine learning"

07 April 2016

Better evidence for patients, and geeking out on baseball.

Health tech wearables

1. SPOTLIGHT: Redefining how patients get health evidence.

How can people truly understand evidence and the tradeoffs associated with health treatments? How can the medical community lead them through decision-making that's shared - but also evidence-based?

Hoping for cures, patients and their families anxiously Google medical research. Meanwhile, the quantified selves are gathering data at breakneck speed. These won't solve the problem. However, this month's entire Health Affairs issue (April 2016) focuses on consumer uses of evidence and highlights promising ideas.

  • Translating medical evidence. Lots of synthesis and many guidelines are targeted at healthcare professionals, not civilians. Knowledge translation has become an essential piece, although it doesn't always involve patients at early stages. The Boot Camp Translation process is changing that. The method enables leaders to engage patients and develop healthcare language that is accessible and understandable. Topics include colon cancer, asthma, and blood pressure management.
  • Truly patient-centered medicine. Patient engagement is a buzzword, but capturing patient-reported outcomes in the clinical environment is a real thing that might make a big difference. Danielle Lavallee led an investigation into how patients and providers can find more common ground for communicating.
  • Meaningful insight from wearables. These are early days, so it's probably not fair to take shots at the gizmos out there. It will be a beautiful thing when sensors and other devices can deliver more than alerts and reports - and make valuable recommendations in a consumable way. And of course these wearables can play a role in routine collection of patient-reported outcomes.


Statcast

2. Roll your own analytics for fantasy baseball.
For some of us, it's that special time of year when we come to the realization that our favorite baseball team is likely going home early again this season. There's always fantasy baseball, and it's getting easier to geek out with analytics to improve your results.

3. AI engine emerges after 30 years.
No one ever said machine learning was easy. Cyc is an AI engine that reflects 30 years of building a knowledge base. Now its creator, Doug Lenat, says it's ready for prime time. Lucid is commercializing the technology. Personal assistants and healthcare applications are in the works.

Photo credit: fitbit one by Tatsuo Yamashita on Flickr.

30 March 2016

$15 minimum wage, evidence-based HR, and manmade earthquakes.

Fightfor15.org

Photo by Fightfor15.org

1. SPOTLIGHT: Will $15 wages destroy California jobs?
California is moving toward a $15/hour minimum wage (slowly, stepping up through 2023). Will employers be forced to eliminate jobs under the added financial pressure? As with all things economic, it depends who you ask. Lots of numbers have been thrown around during the recent push for higher pay. Fightfor15.org says 6.5 million workers are getting raises in California, and that 2/3 of New Yorkers support a similar increase. But small businesses, restaurants in particular, are concerned they'll have to trim menus and staff - they can charge only so much for a sandwich.

Moody's Analytics economist Adam Ozimek says it's not just about food service or home healthcare. Writing on The Dismal Scientist Blog, "[I]n past work I showed that California has 600,000 manufacturing workers who currently make $15 an hour or less. The massive job losses in manufacturing over the last few decades has shown that it is an intensely globally competitive industry where uncompetitive wages are not sustainable." 

It's not all so grim. Ozimek shows that early reports of steep job losses after Seattle's minimum-wage hike have been revised strongly upward. However, finding "the right comparison group is getting complicated."


Yellow Map Chance of Earthquake

2. Manmade events sharply increase earthquake risk.
Holy smokes. New USGS maps show north-central Oklahoma at high earthquake risk. The United States Geological Survey now includes potential ground-shaking hazards from both 'human-induced' and natural earthquakes, substantially changing their risk assessment for several areas. Oklahoma recorded 907 earthquakes last year at magnitude 3 or higher. Disposal of industrial wastewater has emerged as a substantial factor.

3. Evidence-based HR redefines leadership roles.
Applying evidence-based principles to talent management can boost strategic impact, but requires a different approach to leadership. The book Transformative HR: How Great Companies Use Evidence-Based Change for Sustainable Advantage (Jossey-Bass) describes practical uses of evidence to improve people management. John Boudreau and Ravin Jesuthasan suggest principles for evidence-based change, including logic-driven analytics. For instance, establishing appropriate metrics for each sphere of your business, rather than blanket adoption of measures like employee engagement and turnover.

4. Why we're not better at investing.
Gary Belsky does a great job of explaining why we think we're better investors than we are. By now our decision biases have been well-documented by behavioral economists. Plus we really hate to lose - yet we're overconfident, somehow thinking we can compete with Warren Buffet.

16 March 2016

Equity crowdfunding algorithms, decision-making competitions, and statistical wild geese.

Circleup

1. CircleUp uses algorithm to evaluate consumer startups.
Recently we wrote about #fintech startups who are challenging traditional consumer lending models. CircleUp is doing something similar to connect investors with non-tech consumer startups (food, cosmetics, recreation). It's not yet a robo adviser for automated investing, but they do use machine learning to remove drudgery from the analysis of private companies. @CircleUp's classifier selects emerging startups based on revenue, margins, distribution channels, etc., then makes their findings available to investors. They've also launched a secondary market where shareholders can sell their stakes twice annually. The company has successfully raised Series C funding.

2. Student decision-making competition.
In the 2016 @SABR case competition, college and university students analyzed and presented a baseball operations decision — the type of decision a team’s GM and staff face over the course of a season. Contestants were required to construct and defend a 2016 bullpen from scratch for any National League team, focusing on that team's quality of starting pitching, defense, home ballpark, division opponents, and other factors. The Carnegie Mellon team from the Tepper School of Business won the graduate division.

3. For many, writing is an essential data science skill.
Matt Asay (@mjasay) reminds us data science breaks down into two categories, depending on whether it's intended for human or machine consumption. The human-oriented activity often requires straightforward steps rather than complex digital models; business communication skills are essential. Besides manipulating data, successful professionals must excel at writing paragraphs of explanation or making business recommendations.

Writing for data science

4. Chasing statistical wild geese.
The American Statistical Association has released a statement on p-values: context, process, and purpose. There's been a flurry of discussion. If you find this tl;dr, the bottom line = "P-values don't draw bad conclusions, people do". The ASA's supplemental info section presents alternative points of view - mostly exploring ways to improve research by supplementing p-values, using Bayesian methods, or simply applying them properly. Christie Aschwanden wrote on @FiveThirtyEight that "A common misconception among nonstatisticians is that p-values can tell you the probability that a result occurred by chance. This interpretation is dead wrong, but you see it again and again and again and again. The p-value only tells you something about the probability of seeing your results given a particular hypothetical explanation...." Hence ASA Principle No. 2: “P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.” Nor can a p-value tell you the size of an effect, the strength of the evidence, or the importance of a result. The problem is the way p-values are used, explains Deborah Mayo (@learnfromerror): “failing to adjust them for cherry picking, multiple testing, post-data subgroups and other biasing selection effects”.

Photo credit: Around the campfire by Jason Pratt.

10 March 2016

Analytics disillusionment, evidence-based presentation style, and network analysis.

Polinode New_Layout_Algorithm

1. Visualizing networks.
@Polinode builds innovative tools for network analysis. One nifty feature allows creation of column charts using a set of nodes. A recent post explains how to use calculated network metrics such as centrality or betweenness.

2. Analytics are disconnected from strategic decisions.
An extensive study suggests analytics sponsors are in the trough of disillusionment. The new MIT Sloan-SAS report, Beyond the hype: The hard work behind analytics success finds that competitive advantage from analytics is declining. How can data do more to improve outcomes?

Analytics insights MIT-SAS report

The @mitsmr article notes several difficulties, including failure to drive strategic decisions with analytics. "Over the years, access to useful data has continued to increase, but the ability to apply analytical insights to strategy has declined." Dissemination of insights to executives and other decision makers is also a problem. The full report is available from SAS (@SASBestPractice).

3. Evidence shows graphics better than bullets.
There's new empirical evidence on communicating business strategy. 76 managers saw a presentation by the financial services branch of an auto manufacturer. Three types of visual support were displayed: bulleted list, visual metaphor, and temporal diagram. Each subject saw only one of the three formats. Those who saw a graphical representation paid significantly more attention to, agreed more with, and better recalled the strategy than did subjects who saw a (textually identical) bulleted list version. However, no significant difference was found regarding the *understanding* of the strategy. Also, presenters using graphical representations were more positively perceived those who presented bulleted lists.

4. Linking customer experience with value.
McKinsey's Joel Maynes and Alex Rawson offer concrete advice on how to explicitly link customer experience initiatives to value. "Develop a hypothesis about customer outcomes that matter. Start by identifying the specific customer behavior and outcomes that underpin value in your industry. The next step is to link what customers say in satisfaction surveys with their behavior over time."

5. Never mind on that reproducibility study.
Slate explains how Psychologists Call Out the Study That Called Out the Field of Psychology. In a comment published by Science, reviewers conclude that "A paper from the Open Science Collaboration... attempting to replicate 100 published studies suggests that the reproducibility of psychological science is surprisingly low. We show that this article contains three statistical errors and provides no support for such a conclusion. Indeed, the data are consistent with the opposite conclusion, namely, that the reproducibility of psychological science is quite high." Evidently, OSC frequently used study populations that differed substantially from the original ones - and each replication attempt was done only once.

02 March 2016

NBA heat maps, FICO vs Facebook, and peer review.

Curry-heatmap

Curry-heatmap2016

1. Resistance is futile. You must watch Steph Curry.
The Golden State Warriors grow more irresistible every year, in large part because of Curry’s shooting. With sports data analytics from Basketball-Reference.com, these heat maps illustrate his shift to 3-pointers (and leave no doubt why Curry was called the Babyfaced Assassin; now of course he’s simply MVP).

2. Facebook vs FICO.
Fintech startups are exploring new business models, such as peer-to-peer lending (Lending Club). Another big idea is replacing traditional credit scores with rankings derived from social media profiles and other data: Just 3 months ago, Affirm and others were touted in Fortune’s Why Facebook Profiles are Replacing Credit Scores. But now the Wall Street Journal says those decisions are falling out of favor, in Facebook Isn’t So Good at Judging Your Credit After All. Turns out, regulations and data-sharing policies are interfering. Besides, executives with startups like ZestFinance find social-media lending “creepy”.

3. How to fix science journals.
Harvard Med School’s Jeffrey Flier wrote an excellent op-ed for the Wall Street Journal, How to Keep Bad Science from Getting into Print [paywall]. Key issues: anonymous peer reviewers, and lack of transparent post-publishing dialogue with authors (@PubPeer being a notable exception). Flier says we need a science about how to publish science. Amen to that.

4. Longing for civil, evidence-based discourse?
ProCon.org publishes balanced coverage of controversial issues, presenting side-by-side pros and cons supported by evidence. The nonprofit’s site is ideal for schoolteachers, or anyone wanting a quick glance at important findings.

04 February 2016

How Warby Parker created a data-driven culture.

 

4 pic Creating a Data Driven Organization 04feb16

 

1. SPOTLIGHT: Warby Parker data scientist on creating data-driven organizations. What does it take to become a data-driven organization? "Far more than having big data or a crack team of unicorn data scientists, it requires establishing an effective, deeply ingrained data culture," says Carl Anderson. In his recent O'Reilly book Creating a Data-Driven Organization, he explains how to build the analytics value chain required for valuable, predictive business models: From data collection and analysis to insights and leadership that drive concrete actions. Follow him @LeapingLlamas.

Practical advice, in a conversational style, is combined with references and examples from the management literature. The book is an excellent resource for real-world examples and highlights of current management research. The chapter on creating the right culture is a good reminder that leadership and transparency are must-haves.

UglyResearch_Action_Outcome

Although the scope is quite ambitious, Anderson offers thoughtful organization, hitting the highlights without an overwhelmingly lengthy literature survey. Ugly Research is delighted to be mentioned in the decision-making chapter (page 196 in the hard copy, page 212 in the pdf download). As shown in the diagram, with PepperSlice we provide a way to present evidence to decision makers in the context of a specific 'action-outcome' prediction or particular decision step.

Devil's advocate point of view. Becoming 'data-driven' is context sensitive, no doubt. The author is Director of Data Science at Warby Parker, so unsurprisingly the emphasis is technologies that enable data-gathering for consumer marketing. While it does address several management and leadership issues, such as selling a data-driven idea internally, the book primarily addresses the perspective of someone two or three degrees of freedom from the data; a senior executive working with an old-style C-Suite would likely need to take additional steps to fill the gaps. The book isn't so much about how to make decisions, as about how to create an environment where decision makers are open to new ideas, and to testing those ideas with data-driven insights. Because without ideas and evidence, what's the point of a good decision process?

2. People management needs prescriptive analytics. There are three types of analytics: descriptive (showing what already happened), predictive (predicting what will happen), and prescriptive (delivering recommended actions to produce optimal results). For HR, this might mean answering "What is our staff retention? What retention is expected for 2016? And more importantly, what concrete steps will improve staff retention for this year?" While smart analytics power many of our interactions as consumers, it is still unusual to get specific business recommendations from enterprise applications. That is changing. Thanks @ISpeakAnalytics.

3. Algorithms need managers, too. Leave it to the machines, and they'll optimize on click-through rates 'til kingdom come - even if customer satisfaction takes a nose dive. That's why people must actively manage marketing algorithms, explain analytics experts in the latest Harvard Business Review.

4. Nonreligious children are more generous? Evidence shows religion doesn't make kids more generous or altruistic. The LA Times reports a series of experiments suggests that children who grow up in nonreligious homes are more generous and altruistic than those from observant families. Thanks @VivooshkaC.

5. Housing-based welfare strategies do not work, and will not work. So says evidence from LSE research, discussing failures of asset-based welfare.  

28 January 2016

Everyone's decision process, C-Suite judgment, and the Golden Gut.

Househunters_decision_checklist

1. SPOTLIGHT: MCDA, a decision process for everyone. 'Multiple criteria decision analysis' is a crummy name for a great concept (aren't all big decisions analyzed using multiple criteria?). MCDA means assessing alternatives while simultaneously considering several objectives. It's a useful way to look at difficult choices in healthcare, oil production, or real estate. But oftentimes, results of these analyses aren't communicated clearly, limiting their usefulness.

Fundamentally, MCDA means listing options, defining decision criteria, weighting those criteria, and then scoring each option. Some experts build complex economic models, but anyone can apply MCDA in effective, less rigorous ways.

You know those checklists at the end of every HouseHunters episode where people weigh location and size against budget? That's essentially it: Making important decisions, applying judgment, and juggling multiple goals (raise the kids in the city or the burbs?) - and even though they start out by ranking priorities, once buyers see their actual options, deciding on a house becomes substantially more complex.

MCDA guidance from ISPOR

As shown in the diagram (source: ISPOR), the analysis hinges on assigning relative weights to individual decision criteria. While this brings rationality and transparency to complex decisions, it also invites passionate discussions. Some might expect these techniques to remove human judgment from the process, but MCDA leaves it front and center.

Pros and cons. Let’s not kid ourselves: You have to optimize on something. MCDA is both beautiful and terrifying because it forces us to identify tradeoffs: Quality, short-term benefits, long-term results? Uncertain outcomes only complicate things futher. 

This method is a good way to bring interdisciplinary groups into a conversation. One of the downsides is that, upon seeing elaborate projections and models, people can become over-confident in the numbers. Uncertainty is never fully recognized or quantified. (Recall the Rumsfeldian unknown unknown.) Sensitivity analysis is essential, to illustrate which predicted outcomes are strongly influenced by small adjustments.

MCDA is gaining traction in healthcare. The International Society For Pharmacoeconomics and Outcomes Research has developed new MCDA guidance, available in the latest issue of Value for Health (paywall). To put it mildly, it’s difficult to balance saving lives with saving money.  To be sure, healthcare decision makers have always weighed medical, social, and economic factors: MCDA helps stakeholders bring concrete choices and transparency to the process of evaluating outcomes research - where controversy is always a possibility.

Resources to learn more. If you want to try MCDA, pick up one of the classic texts, such as Smart Choices: A Practical Guide to Making Better Decisions. Additionally, ISPOR's members offer useful insights into the pluses and minuses of this methodology - see, for example, Does the Future Belong to MCDA? The level of discourse over this guidance illustrates how challenging healthcare decisions have become.  

2. C-Suite judgment must blend with analytics. Paul Blase of PriceWaterhouseCoopers hits the nail on the head, describing how a single analytics department can't be expected to capture the whole story of an enterprise. He explains better ways to involve both the C-Suite and the quants in crucial decision-making.

3. The Man with the Golden Gut. Netflix CEO Reed Hastings explains how and when intuition is more valuable than big data. Algorithms can make only some of the decisions.

4. Embedding analytics culture. How do you compare to the Red Sox? Since Moneyball, clubs have changed dramatically. Is it possible baseball organizations have embedded analytics processes more successfully than other business enterprises?

12 January 2016

Game theory for Jeopardy!, evidence for gun control, and causality.

1. Deep knowledge → Wagering strategy → Jeopardy! win Some Jeopardy! contestants struggle with the strategic elements of the show. Rescuing us is Keith Williams (@TheFinalWager), with the definitive primer on Jeopardy! strategy, applying game theory to every episode and introducing "the fascinating world of determining the optimal approach to almost anything".

2. Gun controls → Less violence? → Less tragedy? Does the evidence support new US gun control proposals? In the Pacific Standard, Francie Diep cites several supporting scientific studies.

3. New data sources → Transparent methods → Health evidence Is 'real-world' health evidence closer to the truth than data from more traditional categories? FDA staff explain in What We Mean When We Talk About Data. Thanks to @MandiBPro.

4. Data model → Cause → Effect In Why: A Guide to Finding and Using Causes, Samantha Kleinberg aims to explain why causality is often misunderstood and misused: What is it, why is it so hard to find, and how can we do better at interpreting it? The book excerpt explains that "Understanding when our inferences are likely to be wrong is particularly important for data science, where we’re often confronted with observational data that is large and messy (rather than well-curated for research)."

5. Empirical results → Verification → Scientific understanding Independent verification is essential to scientific progress. But in academia, verifying empirical results is difficult and not rewarded. This is the reason for Curate Science, a tool making it easier for researchers to independently verify each other’s evidence and award credit for doing so. Follow @CurateScience.

Join me at the HEOR writing workshop March 17 in Philadelphia. I'm speaking about communicating data, and leading an interactive session on data visualization. Save $300 before Jan 15.

24 November 2015

Masters of self-deception, rapid systematic reviews, and Gauss v. Legendre.

1. Human fallibility → Debiasing techniques → Better science Don't miss Regina Nuzzo's fantastic analysis in Nature: How scientists trick themselves, and how they can stop. @ReginaNuzzo explains why people are masters of self-deception, and how cognitive biases interfere with rigorous findings. Making things worse are a flawed science publishing process and "performance enhancing" statistical tools. Nuzzo describes promising ways to overcome these challenges, including blind data analysis.

2. Slow systematic reviews → New evidence methods → Controversy Systematic reviews are important for evidence-based medicine, but some say they're unreliable and slow. Two groups attempting to improve this - not without controversy - are Trip (@TripDatabase) and Rapid Reviews.

3. Campus competitions → Real-world analytics → Attracting talent Tech firms are finding ways to attract students to the analytics field. David Weldon writes in Information Management about the Adobe Analytics Challenge, where thousands of US university students compete using data from companies such as Condé Nast and Comcast to solve real-world business problems.

4. Discover regression → Solve important problem → Rock the world Great read on how Gauss discovered statistical regression, but thinking his solution was trivial, didn't share. Legendre published the method later, sparking one of the bigger disputes in the history of science. The Discovery of Statistical Regression - Gauss v. Legendre on Priceonomics.

5. Technical insights → Presentation skill → Advance your ideas Explaining insights to your audience is as crucial as getting the technical details right. Present! is a new book with speaking tips for technology types unfamiliar with the spotlight. By Poornima Vijayashanker (@poornima) and Karen Catlin.

17 November 2015

ROI from evidence-based government, milking data for cows, and flu shot benefits diminishing.

1. Evidence standards → Knowing what works → Pay for success Susan Urahn says we've reached a Tipping Point on Evidence-Based Policymaking. She explains in @Governing that 24 US governments have directed $152M to programs with an estimated $521M ROI: "an innovative and rigorous approach to policymaking: Create an inventory of currently funded programs; review which ones work based on research; use a customized benefit-cost model to compare programs based on their return on investment; and use the results to inform budget and policy decisions."

2. Sensors → Analytics → Farming profits Precision dairy farming uses RFID tags, sensors, and analytics to track the health of cows. Brian T. Horowitz (@bthorowitz) writes on TechCrunch about how farmers are milking big data for insight. Literally. Thanks to @ShellySwanback.

3. Public acceptance → Annual flu shots → Weaker response? Yikes. Now that flu shot programs are gaining acceptance, there's preliminary evidence suggesting that repeated annual shots can gradually reduce their effectiveness under some circumstances. Scientists at the Marshfield Clinic Research Foundation recently reported that "children who had been vaccinated annually over a number of years were more likely to contract the flu than kids who were only vaccinated in the season in which they were studied." Helen Branswell explains on STAT.

4. PCSK9 → Cholesterol control → Premium increases Ezekiel J. Emanuel says in a New York Times Op-Ed I Am Paying for Your Expensive Medicine. PCSK9 inihibitors newly approved by US FDA can effectively lower bad cholesterol, though data aren't definitive whether this actually reduces heart attacks, strokes, and deaths from heart disease. This new drug category comes at a high cost. Based on projected usage levels, soem analysts predict insurance premiums could rise >$100 for everyone in that plan.

5. Opportunistic experiments → Efficient evidence → Informed family policy New guidance details how researchers and program administrators can recognize opportunities for experiments and carry them out. This allows people to discover effects of planned initiatives, as opposed to analyzing interventions being developed specifically for research studies. Advancing Evidence-Based Decision Making: A Toolkit on Recognizing and Conducting Opportunistic Experiments in the Family Self-Sufficiency and Stability Policy Area.

Subscribe by email